Text Analytics: Enterprise-level semantic technologies
How semantic applications help organizations achieve productivity increases.
By Denise Bedford
With the proliferation of unstructured data, semantic applications have moved out of the laboratory and into enterprise contexts. 21st century knowledge organizations are learning that semantic technologies can be designed, configured and implemented to support a wide range of business processes. Semantic technologies go beyond simple data retrieval. Today’s semantic technologies, also known as text analytics, are smart, applying rigorous natural language processing (NLP) methods to model human decision-making processes, and – perhaps most importantly – they can integrate existing organizational knowledge.
Semantic technologies, in this context, involve the processing of recorded human knowledge in a meaningful way – in other words, in a way that represents how people think about things. These technologies use several different methods, including concept extraction, rule-based classification, dynamic classification, guided summarization and cross-language translation.
The use of semantic analysis methods can improve the productivity of analytical decision-making and information and knowledge processing, and increase the effectiveness of knowledge discovery. Knowledge organizations use these semantic technologies to increase their productivity by processing higher volumes of information more so than possible by manual effort alone. Semantic technologies have shown that the unit time to process an institutional document can be reduced from 30 to 40 minutes to less than two minutes, more than a 600-percent improvement. At the same time, the level of quality of processing can significantly improve since we are explicitly codifying the mental processing steps. As a result, the opportunity costs of using subject matter expertise for standard information management tasks can be significantly reduced without risking access to organizational knowledge.
Advances in productivity are realized when organizations design and configure these technologies in sustainable enterprise applications rather than as one-time, one-off projects. Semantic technologies are no longer the new IT toy systems. Neither are they “silver bullets” for tough knowledge challenges. They are most effective when they’re built to do what people do, only they do it faster and more consistently with less effort. This frees a knowledge organization’s intellectual capital to do even more intelligent things.
Where can an organization achieve the greatest productivity gains through semantic technologies? Information and knowledge processing of unstructured data or text offer the greatest opportunities. For example, organizations have used:
- rule-based concept extraction methods to capture key numerical indicators such as project numbers, contract numbers, unique IDs, digital object identifiers, ISBN (international standard book numbers) and other key financial numbers, with high levels of reliability and quality, and minimum or no human effort;
- grammatical concept-extraction methods to characterize market reports or new stories with high precision sentiment analysis;
- grammatical concept extraction to construct descriptive maps of knowledge domains and dynamic clustering methods to illustrate the relationship of concepts within a domain; and
- rule-based categorization methods to retrospectively organized large collections of critical business documents to support faceted search with minimal human investment, or to automatically and reliably classify current content to country focus.
Realizing Productivity Improvements
These successful implementations are the result of deep modeling of business processes, with integration of organizational knowledge sources, using well-planned project development life cycles and patient learning and fine-tuning of results.
Sustainable use of semantic technologies means integrating them into everyday business processes. This means modeling and exploring existing business processes for appropriate “semantic opportunities.” It means having an idea of the type of productivity gains you’re aiming for before you begin. Business architects have important roles to play in implementing semantic technologies. This is not just the role of engineers or programmers.
Too often technology decisions are based on shallow understanding of how the technologies work and what they are designed to do. Managers often fall into the “I have a hammer, so everything looks like a nail” syndrome. It is not necessary to understand the statistical or parsing methods at a research level in order to decide whether a semantic technology has the functional components to support your business processes.
Sustainable semantic technologies are not out-of-the-box products. Rather, they are founded in well-designed architectures, including exposable and configurable knowledge bases, open matching rules, definable algorithms and interoperable semantic products. Semantic solutions, such as SAS Text Analytics, have functional components just as other applications do. Making sure that the semantic functions are fully supported by the technology you choose is critical.
Not all so-called semantic technologies are designed for enterprise use. Not all semantic technologies can process unstructured information in meaningful ways. Many technologies bundle the NLP analysis so tightly with statistical processing that they are impossible to implement given that they are “canned” applications. As a result some are suited to only a small set of structured data processing problems. They may not be able to process unstructured data other than as statistically occurring data bits.
Design and Integration of Organizational Knowledge
A 21st century semantic technology will have the capability to consume existing organizational knowledge. However, a smart technology will not require that this knowledge be encoded as deeply embedded rules. Organizational knowledge is always evolving. Smart technologies will consume but not control organizational knowledge sources. Just as we no longer embed our data into code (we learned that lesson back in the 1990s), we should never embed our organizational knowledge into a semantic technology.
Semantic technologies need to be able to use different kinds of knowledge – represented as different kinds of structures. One size does not fit all types of knowledge. In fact, knowledge design and representation is a critical success (or failure) factor in implementing semantic technologies. Knowledge architects and knowledge engineers play critical roles in effective implementations.
Some semantic technologies require intense programming level support, making it difficult for subject matter experts or knowledge professionals to work with them and fine tune over time. Subject matter experts should be the primary user of any design model for a semantic solution.
Smart Lessons for Smart Technologies
Knowledge organizations in the 21st century will find semantic analysis technologies to be among their core, strategic applications. They will be extensible, flexible systems that are configured for their purposes and designed to scale with organization needs. No semantic analysis technology is a silver bullet, and no semantic analysis technology is a solution to all business challenges. In fact, the most successful implementation of semantic technologies will be those that are seamlessly and invisibly implemented into business processes.
Successful experiences result from a deep understanding of the semantic elements of your business processes, taking the time investigating the products, a willingness to invest in new and different knowledge roles, and a willingness to commit to on-going support and expansion of the technologies. Productivity gains are substantial when organizations make a good commitment and investment.
Denise Bedford (email@example.com) is the Goodyear Professor of Knowledge Management at Kent State University. She teaches courses in economics of information, intellectual capital management, semantic analysis methods, communities of practice and other knowledge management topics. A version of this article was published in SAS’ 2010 Government Insight.