Share with your friends










Submit

Analytics Magazine

Dark analytics: Shedding light on a new business asset

By Nitin Mittal

Deep within the astonishing volumes of raw information generated by business transactions, social media, search engines, IoT and countless other sources, valuable intelligence about customers, markets and organizations, lies waiting to be discovered.

Leveraging advanced technologies to explore this expansive universe of unstructured and “dark” data reveals hidden insights to inform decision-making and chart new paths to the future. Machine learning, robotics process automation, visualization, natural language processing, and image and video analysis allow questions to be answered and opportunities brought to light that were unimaginable only a few years ago.

We’re only beginning to explore the digital universe, yet significant business value lies within. In fact, International Digital Corporation predicts that organizations that can analyze all relevant data and deliver actionable information could achieve an extra $430 billion in productivity gains over their peers by 2020 [1].

The influence of dark data is inescapable, increasingly driving innovation and value to customers and stakeholders. From financial services to healthcare, we’re starting to see the powerful impact of evolving technologies, new types of data and its potential to drive competitive advantage and shape entire industries. In the automotive industry, connected cars could help make usage-based insurance possible and accurately predict when vehicles need maintenance. Some automakers are beginning to experiment with an augmented reality dashboard.

Defining Dark Data

With a digital universe expected to reach 44 zetabytes by 2020, and 90 percent of it being unstructured data from the IoT and non-traditional sources [2], how is dark data defined? Today’s dark analytics efforts usually focus on three dimensions:

  1. Untapped data already in your possession. Most organizations have large collections of structured and unstructured data sitting idle. On the structured side, it’s often due to the difficulty of making connections between disparate data sets. Insights lie waiting to be discovered: One company mapped the addresses of employees against workplace satisfaction ratings and retention data, finding that one of the biggest factors fueling voluntary turnover was commute time.

Unstructured data is often text-based, and until recently, tools and techniques needed to leverage them efficiently did not exist. Today, scanned patient records could hold the key to better understanding disease and prescribing effective treatments; executive emails and communications could unearth wisdom needed to pass along to a younger generation of workers.

  1. Nontraditional unstructured data. Another dark analytics dimension focuses on data such as audio and video files and still images that could not be explored until now. Now, computer vision, advanced pattern recognition, and video and sound analytics allow companies to mine this data to better understand customers, employees, operations and markets.

This insight can be illuminating; retailers have a window into customer sentiment by analyzing in-store posture and facial expressions or online browsing patterns. Oil and gas companies can use acoustic sensors to monitor pipelines and algorithms to provide visibility into flow rates and fluid composition.

  1. Data in the deep web. The deep web may offer the largest of body of untapped information – data curated by academics, government agencies, communities and other third-party domains – that is often hidden behind firewalls. Organizations may soon be able to curate competitive intelligence using search tools designed to help target specific data types. Analysis of this “deep web” data is especially promising in its potential to allow better prevention, detection and response to cyber threats.

Light Up the Dark Side

Developing a strategy for discovering value in unstructured data can help your organization generate insights today and prepare for even greater opportunities in the years ahead. How can your organization get the most value out of the mountains of data that it has created, owns or has access to? To help optimize the value of this business asset, consider these practical steps:

Ask the right questions. Work with business teams to identify specific questions that need to be answered. Then identify the sources of data that make the most sense for your analytics efforts. To boost sales of sports equipment, analytics teams can focus on sales transactions, inventory and product pricing in a specific geographic area. Supplementing the data with unstructured data, such as instore analysis of foot traffic or social media trends, generates more nuanced insights.

Look outside of your organization. Augmenting your data with publicly available demographic, location and statistical information helps put insights in context. For example, when a physician makes recommendations to an asthma patient about how to manage symptoms, he can also provide short-term solutions to help her deal with flare-ups during pollen season.

Expand analytics talent for impact. Optimizing dark data analytics depends on assembling the right teams to meet specific needs and identify emerging opportunities. Build a cohesive team that encompasses organizational, business and technical knowledge.

View analytics as a business-driven effort. Plan your project with your organization’s business goals in mind to determine the value that must be delivered, define the questions to ask and decide how to harness data to generate the right answers. Data analytics then becomes an insight-driven advantage that wins support throughout the organization and fuels future endeavors.

Think broadly. As new strategies and capabilities for using dark analytics are developed, consider how they can be extended across the organization, as well as to customers, vendors and business partners.

By tapping into dark data, organizations have an opportunity to turn previously hidden or unknown patterns and correlations into powerful insights. Uncovering this business value leads to new opportunities, reduced risk and increased analytics ROI.

The potential is exciting, but staying grounded with specific business questions that have a defined scope and measurable value is critical. Focus your dark analytics efforts on areas that matter to your business – and avoid getting lost in the increasingly vast unknown.

Nitin Mittal is a principal in Deloitte Consulting LLP and leads Deloitte Consulting’s Analytics & Information Management practice. His team works with companies from many industries to fully leverage the potential of analytics and emerging technologies.

References

  1. “IDC FutureScape: Worldwide big data, business analytics, and cognitive software predictions,” International Data Corporation, 2016.
  2. EMC Digital Universe with research and analysis by IDC, “The digital universe of opportunities: Rich data and the increasing value of the Internet of Things,” April 2014, www.emc.com/leadership/digital-universe/2014iview/executive-summary.html

Related Posts

  • 82
    Deep within the astonishing volumes of raw information generated by business transactions, social media, search engines, IoT and countless other sources, valuable intelligence about customers, markets and organizations, lies waiting to be discovered. Leveraging advanced technologies to explore this expansive universe of unstructured and “dark” data reveals hidden insights to…
    Tags: data, analytics, business, dark
  • 70
    Content/Interactive Marketing Opportunities Analytics-Magazine.org can help you build a successful content marketing program or interactive lead generation program. Enhance your position as an industry thought leader and expert in the analytics profession by promoting the following content formats on Analytics-Magazine.org. Product Videos Software Demonstrations White Papers Case Studiesa Research Reports…
    Tags: analytics, analysis, data
  • 69
    The CUNY School of Professional Studies is offering a new online master of science degree in data analytics. The program prepares its graduates for high-demand and fast-growing careers as data analysts, data specialists, business intelligence analysts, information analysts and data engineers in such fields as business, operations, marketing, social media,…
    Tags: data, analytics, business
  • 68
    Benjamin Franklin offered this sage advice in the 18th century, but he left one key question unanswered: How? How do you successfully drive a business? More specifically, how do you develop the business strategy drivers that incite a business to grow and thrive? The 21st-century solution has proven to be…
    Tags: data, analytics, business
  • 67
    “Drive thy business or it will drive thee.” Benjamin Franklin offered this sage advice in the 18th century, but he left one key question unanswered: How? How do you successfully drive a business? More specifically, how do you develop the business strategy drivers that incite a business to grow and…
    Tags: data, business, analytics

Analytics Blog

Electoral College put to the math test


With the campaign two months behind us and the inauguration of Donald Trump two days away, isn’t it time to put the 2016 U.S. presidential election to bed and focus on issues that have yet to be decided? Of course not.


Save



Headlines

Study: Salaries for early career data scientists decrease for first time

Salaries for early career data scientists decreased year over year for the first time in four years as did the percentage of early career data scientists with a Ph.D. while demand for data scientists continued to increase, according to a recently released Burtch Works’ 2017 salary study of data scientists. Salaries for more experienced data scientists generally held steady or increased slightly depending on an individual’s focus area, responsibility and geographic base, according to the report. Read more →

Generous health insurance plans encourage overtreatment, but may not improve health

Offering comprehensive health insurance plans with low deductibles and co-pay in exchange for higher annual premiums seems like a good value for the risk averse, and a profitable product for insurance companies. But according to a forthcoming study in a leading scholarly marketing journal, the INFORMS journal Marketing Science, such plans can encourage individuals with chronic conditions to turn to needlessly expensive treatments that have little impact on their health outcomes. This in turn raises costs for the insurer and future prices for the insured. Read more →

UPCOMING ANALYTICS EVENTS

INFORMS-SPONSORED EVENTS

CONFERENCES

2017 INFORMS Healthcare Conference
July 26-28, 2017, Rotterdam, the Netherlands

CAP® EXAM SCHEDULE

CAP® Exam computer-based testing sites are available in 700 locations worldwide. Take the exam close to home and on your schedule:


 
For more information, go to 
https://www.certifiedanalytics.org.