Share with your friends


Analytics Magazine

Text Analytics: Enterprise-level semantic technologies

March/April 2011


How semantic applications help organizations achieve productivity increases.

Denise BedfordBy Denise Bedford

With the proliferation of unstructured data, semantic applications have moved out of the laboratory and into enterprise contexts. 21st century knowledge organizations are learning that semantic technologies can be designed, configured and implemented to support a wide range of business processes. Semantic technologies go beyond simple data retrieval. Today’s semantic technologies, also known as text analytics, are smart, applying rigorous natural language processing (NLP) methods to model human decision-making processes, and – perhaps most importantly – they can integrate existing organizational knowledge.

Semantic technologies, in this context, involve the processing of recorded human knowledge in a meaningful way – in other words, in a way that represents how people think about things. These technologies use several different methods, including concept extraction, rule-based classification, dynamic classification, guided summarization and cross-language translation.

Productivity Improvements

The use of semantic analysis methods can improve the productivity of analytical decision-making and information and knowledge processing, and increase the effectiveness of knowledge discovery. Knowledge organizations use these semantic technologies to increase their productivity by processing higher volumes of information more so than possible by manual effort alone. Semantic technologies have shown that the unit time to process an institutional document can be reduced from 30 to 40 minutes to less than two minutes, more than a 600-percent improvement. At the same time, the level of quality of processing can significantly improve since we are explicitly codifying the mental processing steps. As a result, the opportunity costs of using subject matter expertise for standard information management tasks can be significantly reduced without risking access to organizational knowledge.

Advances in productivity are realized when organizations design and configure these technologies in sustainable enterprise applications rather than as one-time, one-off projects. Semantic technologies are no longer the new IT toy systems. Neither are they “silver bullets” for tough knowledge challenges. They are most effective when they’re built to do what people do, only they do it faster and more consistently with less effort. This frees a knowledge organization’s intellectual capital to do even more intelligent things.

Where can an organization achieve the greatest productivity gains through semantic technologies? Information and knowledge processing of unstructured data or text offer the greatest opportunities. For example, organizations have used:

  • rule-based concept extraction methods to capture key numerical indicators such as project numbers, contract numbers, unique IDs, digital object identifiers, ISBN (international standard book numbers) and other key financial numbers, with high levels of reliability and quality, and minimum or no human effort;
  • grammatical concept-extraction methods to characterize market reports or new stories with high precision sentiment analysis;
  • grammatical concept extraction to construct descriptive maps of knowledge domains and dynamic clustering methods to illustrate the relationship of concepts within a domain; and
  • rule-based categorization methods to retrospectively organized large collections of critical business documents to support faceted search with minimal human investment, or to automatically and reliably classify current content to country focus.

Realizing Productivity Improvements

These successful implementations are the result of deep modeling of business processes, with integration of organizational knowledge sources, using well-planned project development life cycles and patient learning and fine-tuning of results.

Sustainable use of semantic technologies means integrating them into everyday business processes. This means modeling and exploring existing business processes for appropriate “semantic opportunities.” It means having an idea of the type of productivity gains you’re aiming for before you begin. Business architects have important roles to play in implementing semantic technologies. This is not just the role of engineers or programmers.

Too often technology decisions are based on shallow understanding of how the technologies work and what they are designed to do. Managers often fall into the “I have a hammer, so everything looks like a nail” syndrome. It is not necessary to understand the statistical or parsing methods at a research level in order to decide whether a semantic technology has the functional components to support your business processes.

Sustainable semantic technologies are not out-of-the-box products. Rather, they are founded in well-designed architectures, including exposable and configurable knowledge bases, open matching rules, definable algorithms and interoperable semantic products. Semantic solutions, such as SAS Text Analytics, have functional components just as other applications do. Making sure that the semantic functions are fully supported by the technology you choose is critical.

Not all so-called semantic technologies are designed for enterprise use. Not all semantic technologies can process unstructured information in meaningful ways. Many technologies bundle the NLP analysis so tightly with statistical processing that they are impossible to implement given that they are “canned” applications. As a result some are suited to only a small set of structured data processing problems. They may not be able to process unstructured data other than as statistically occurring data bits.

Design and Integration of Organizational Knowledge

A 21st century semantic technology will have the capability to consume existing organizational knowledge. However, a smart technology will not require that this knowledge be encoded as deeply embedded rules. Organizational knowledge is always evolving. Smart technologies will consume but not control organizational knowledge sources. Just as we no longer embed our data into code (we learned that lesson back in the 1990s), we should never embed our organizational knowledge into a semantic technology.

Semantic technologies need to be able to use different kinds of knowledge – represented as different kinds of structures. One size does not fit all types of knowledge. In fact, knowledge design and representation is a critical success (or failure) factor in implementing semantic technologies. Knowledge architects and knowledge engineers play critical roles in effective implementations.

Some semantic technologies require intense programming level support, making it difficult for subject matter experts or knowledge professionals to work with them and fine tune over time. Subject matter experts should be the primary user of any design model for a semantic solution.

Smart Lessons for Smart Technologies

Knowledge organizations in the 21st century will find semantic analysis technologies to be among their core, strategic applications. They will be extensible, flexible systems that are configured for their purposes and designed to scale with organization needs. No semantic analysis technology is a silver bullet, and no semantic analysis technology is a solution to all business challenges. In fact, the most successful implementation of semantic technologies will be those that are seamlessly and invisibly implemented into business processes.

Successful experiences result from a deep understanding of the semantic elements of your business processes, taking the time investigating the products, a willingness to invest in new and different knowledge roles, and a willingness to commit to on-going support and expansion of the technologies. Productivity gains are substantial when organizations make a good commitment and investment.

Denise Bedford ( is the Goodyear Professor of Knowledge Management at Kent State University. She teaches courses in economics of information, intellectual capital management, semantic analysis methods, communities of practice and other knowledge management topics. A version of this article was published in SAS’ 2010 Government Insight.



Fighting terrorists online: Identifying extremists before they post content

New research has found a way to identify extremists, such as those associated with the terrorist group ISIS, by monitoring their social media accounts, and can identify them even before they post threatening content. The research, “Finding Extremists in Online Social Networks,” which was recently published in the INFORMS journal Operations Research, was conducted by Tauhid Zaman of the MIT, Lt. Col. Christopher E. Marks of the U.S. Army and Jytte Klausen of Brandeis University. Read more →

Syrian conflict yields model for attrition dynamics in multilateral war

Based on their study of the Syrian Civil War that’s been raging since 2011, three researchers created a predictive model for multilateral war called the Lanchester multiduel. Unless there is a player so strong it can guarantee a win regardless of what others do, the likely outcome of multilateral war is a gradual stalemate that culminates in the mutual annihilation of all players, according to the model. Read more →

SAS, Samford University team up to generate sports analytics talent

Sports teams try to squeeze out every last bit of talent to gain a competitive advantage on the field. That’s also true in college athletic departments and professional team offices, where entire departments devoted to analyzing data hunt for sports analytics experts that can give them an edge in a game, in the stands and beyond. To create this talent, analytics company SAS will collaborate with the Samford University Center for Sports Analytics to support teaching, learning and research in all areas where analytics affects sports, including fan engagement, sponsorship, player tracking, sports medicine, sports media and operations. Read more →



INFORMS Annual Meeting
Nov. 4-7, 2018, Phoenix

Winter Simulation Conference
Dec. 9-12, 2018, Gothenburg, Sweden


Applied AI & Machine Learning | Comprehensive
Starts Oct. 29, 2018 (live online)

The Analytics Clinic
Citizen Data Scientists | Why Not DIY AI?
Nov. 8, 2018, 11 a.m. – 12:30 p.m.

Advancing the Analytics-Driven Organization
Jan. 28–31, 2019, 1 p.m.– 5 p.m. (live online)


CAP® Exam computer-based testing sites are available in 700 locations worldwide. Take the exam close to home and on your schedule:

For more information, go to