Democratization of analytics: New frontier of data economy
By (l-r) Srujana H.M., Sanjay S. Sharma and Amitava Dey
“Each business is a victim of Digital Darwinism, the evolution of consumer behavior when society and technology evolve faster than the ability to exploit it. Digital Darwinism does not discriminate. Every business is threatened.”
– Brian Solis
Analytics influence every aspect of our existence, from the way we think and behave to how we work and do business. Every second, millions of records are generated by organizations, on social media and in the consumer space, leading to a humungous explosion of data of at least 2.5 quintillion bytes  produced every day. By the year 2020 we are expected to see a 4,300 percent increase in the annual data generation . Google alone processes 3.5 billion searches per day and houses 10 exabytes of data.
Data is generated by multiple nodes and exchanged seamlessly between channels, ranging from digital to voice to mobile. Mobile data is experiencing exponential growth, especially in the developing world, where it already boasts 5 billion users.
Individuals generate 70 percent of the overall data, but enterprises store 80 percent of this data . MIT Technology Review indicates that even though digital data created by consumers is doubling every two years, almost all of it remains unused or unanalyzed. Research indicates that 99 percent of new data is never used, analyzed or transformed. Of what use is the data if it is trapped in silos and not analyzed effectively?
Many organizations are sitting on a gold mine of data but do not have effective resources to analyze the data in order to frame effective policies. A study by IBM found that key executives globally spend 70 percent of their time finding data and only 30 percent analyzing it. This challenge becomes more complicated because data is not just obtained in a structured format. The vast majority of data exists in semi-structured or unstructured forms as social media turns all of us into the data-generating agents.
Most of the data in cyberspace remains untapped because there aren’t enough data scientists to analyze the data and derive meaningful information from it. To perform impactful analysis, a data scientist should have a deep understanding of mathematical concepts, proficiency in computational programming and sound domain knowledge. Unfortunately, relatively few data scientists possess all three skill sets, creating a scarcity of data science talent across the globe. A study by McKinsey & Company corroborates this, noting a grave shortage of analytics with the needed skill sets and that the United States alone faces a shortage of 190,000 data scientists. Consequently, businesses are constantly exploring alternatives to making data available to businesses and academia to unleash the value and derive meaningful insights from this data. In this light, “democratization of data and analytics” is the next promising frontier for business success.
Democratization of data and analytics is the phenomenon of making data available to people who need it and have the skill sets to deriving meaningful insights from it. By freeing themselves from data silos and the traditional practice of data collection, storage and access, agile businesses can not only improve their dynamic decision-making, but they can also expedite enterprise data integration and decentralization. While a plethora of analytics and data visualization tools have opened up new possibilities for sharing data across a business, they have also introduced a new set of challenges for business owners and analytics teams.
Democratization of Business Data and Analytics
In a business context, data democratization primarily entails making data and analytics available to all the layers of the organization while transcending departmental boundaries. For example, inventory data and forecasts may be of importance to finance teams in planning the budget allocation for purchases. Historically in most of the companies, useful business data is just confined to IT folks and a handful of senior executives who need to make decisions based on that data. Such a data handling and analysis process, compounded with different and complex data analysis tools, poses severe limitations for management to get a unified and single version of the true story across all the domains of the enterprise. This further impedes the effective decision-making process.
Democratization of data and analytics is emerging as a provident solution to this problem. Democratization of analytics has necessitated the use of open source platforms such as Hadoop and programming languages like R. Artificial intelligence and machine learning further add to the accessibility and applicability the democratization process.
Effective democratization of data analytics across the enterprise demands a three-pronged strategy:
1. Enterprise cross-functional resource architects. A marketing team can build a campaign for a product by working alongside the pricing team, which can take input from the product development team. The product development team can then take input from the market research team about the demand for such a product, the competition, etc., thus building a cross-functional business intelligence resource with enterprise-wide level access. Along with creating greater access, the organization needs to breed a culture of grooming “resource connecters,” people who are involved in connecting employees at all levels to BI (business intelligence) resources and tools. These connectors play a pivotal role in mobilizing democratization of data and analytics in an organization.
2. Effective governance of seamlessly integrated data. Data governance is a crucial aspect that determines the success of democratization of data and analytics. Data governance entails two aspects: 1) upstream data process consisting of data sourcing, transformation, storage, etc.; and 2) downstream data processes consisting of usage and consumption of data. Traditionally data governance was viewed as a means to control data access, but now there is a paradigm shift toward performing data governance to drive business agility.
The major data governance challenge confronted by businesses today is striking a balance between borderless data and maintaining data security. There is constant concern of business sensitive data falling into the wrong hands and prevention of undue confidential data usage by competition that may hamper business growth. Most of businesses do not want to take the risk of providing wider access to its business-critical data. This results in a “do nothing” attitude that hampers innovation, discovery and business growth.
Masking data could provide the way out of this problem. Data masking would involve changing the actual figures in the data while maintaining its original characteristics, thereby ensuring the compliance with data privacy norms and simultaneously promoting democratization of data. Data could be masked dynamically at the application or permanently at the source depending on the nature of sensitivity. A good data governance strategy focuses on building a futuristic business state through:
- assessment of leading practice and high performance model fit,
- identification of future state objectives and operating models,
- conducting a gap analysis, and
- creation of an implementation plan and roadmap.
3. Striking a balance between data quality, scalability and agility. As scale, complexity and availability increase, so do the challenges associated with maintaining quality. Data profiling carried out to determine anomalies, inconsistencies and redundancies in content, structure and relationships help fix challenges and help maintain relevant versions at the source, thereby facilitating meaningful insight generation. Defining data quality capability through appropriate business rules and key performance indicators is the first step toward enabling the clear vision of data through all phases of data management. Such a strategy will prove to be the next frontier of competitive advantage facilitating democratization of data and analytics.
Democratization of Non-Business Data and Analytics
Despite the skeptics of democratization of data and analytics at the corporate sector, these practices are already receiving great acceptances at the research, academic and public sectors (primarily governmental organizations). Governments, through censuses and surveys, collect lots of data on an annual basis, but this data is buried in governmental websites in the form of zip files and is not used as often as it should be. Dynamic data analysis to frame relevant policies remains a farfetched dream in this context. This data, however, can be used by researchers and academicians to perform analytics and derive meaningful inferences, which can be fed back to governments in the form of reports, proceedings of symposiums and conferences, thereby enabling effective and relevant policy formulation and smart governance.
Websites such as “FindTheData” are facilitating such democratization of analytics by organizing the publically available government data into more structured compilations and adding relevant filters, pivots and charts so that researchers can slice and dice the data for meaningful analytics. “Socrata” is another such public data discovery platform that facilitates massive amounts of data to create different views and analyze hidden patterns in the governmental data that were previously not looked at to decipher policy impacts. Socrata gives consumers access to public data and also equips users with the tools they need to draw insights from the data taking democratization of data analytics to the next level.
Data has become the new raw material for business and fundamental basis for social existence in this century. Many studies in recent years (EMC, etc.) have indicated that, on a global scale, data and information are inexorably doubling in less than two years, yet an estimated 99 percent of the overall data in the world is not used or analyzed.
Unleashing this data to derive meaningful information would require people with deep analytics skills. Unfortunately, a huge gap in the demand and supply of analytics professionals is impeding the data analysis potential of corporate and governmental organizations. Democratization of data and analytics seems to be a promising solution to this problem. As the next frontier of competitive differentiation, the democratization of data and analytics will help business become more agile, productive, adaptable and profitable, while helping governmental organizations make real-time policies that can better cater to the relevant needs of all stakeholders.
Srujana H.M. is a marketing analytics manager at Accenture Digital, an executive member of the Analytical Society of India and an internationally certified Project Management Professional.
Sanjay S. Sharma is managing director & India lead of Advanced Analytics in Accenture Digital. He leads a multi-functional, multi-industry team of about 500 analytics professionals. He is a member of INFORMS.
Amitava Dey is a data science principal director for marketing analytics India at Accenture Digital. He has 12 years of experience working across different functional domains and industries with a keen interest in learning new techniques and applying them to real-life analytical problems.
- Sources: https://www.linkedin.com/pulse/4300-increase-annual-data-generation-2020-calls-change-yaron-haviv
- http://www.csc.com/big_data/flxwd/83638-big_data_just_beginning_to_explode_ interactive_ infographic
• “The Democratization of Big Data,” Journal of National Security, Law & Policy, Vol. 7, p. 325
• OECD report of data driven innovation for growth and wellbeing, 2014
• “Big Data, Big Analytics: Emerging Business Intelligence and Analytic Trends,” Wiley Publishers, Michael, Michele & Ambiga.
• Accenture white paper on “Data Governance and Data Management,” 2011, 2012.