Share with your friends


Analytics Magazine

Executive Edge: Democratization and crowdsourcing

July/August 2015

Creating value in an era of pervasive analytics and cheap computing.

Paddy PadmanabhanBy Paddy Padmanabhan

We are entering an era of democratization and crowdsourcing of analytics, driven by the acceleration of technological innovation and the open source movement, coupled with a global pool of highly qualified data scientists and big data technologists. All this is happening rapidly while the costs of computing and data storage continue to fall.

Today, we’re seeing a convergence of demand and supply-side forces that is unlocking the potential of advanced analytics. Some of these are:

  • increasing importance of analytics for enterprise growth and profitability,
  • falling prices for data storage and computing capacity,
  • acceleration of technology innovation, and
  • a huge demand for data scientist talent.

Democratization of Analytics

In recent years, expensive and complex analytical tools and technology have become accessible and affordable to people and business. The computational horsepower needed to conduct analysis on vast amounts of data has also dropped significantly in recent years in line with Moore’s Law. With storage and computing capacity becoming very affordable and practically available on tap, analytics is no longer constrained by scalability and affordability.

IBM Watson Healthcare and Microsoft Azure machine learning as a service are examples of how complex technologies and analytical models are being made available freely for teams to build solutions on. This puts the power of advanced analytics in the hands of increasing numbers of non-technical executives and individuals, creating a democratization effect [1].

Crowdsourcing Comes to Data Sciences

Much is being written about the huge shortage of data scientist talent and the pursuit of “unicorns” – analytics superheroes who can single-handedly wrangle the biggest and baddest data sets into yielding pearls of insights that translate into “ka-ching” sounds at the cash register. There’s a catch, though. Actually, two. One, these people don’t exist. Two, if they do, they cost $250,000 a year and are barely out of college.

Large enterprises are nevertheless falling over each other to hire as many of these individuals as they can. Talk about tulipmania in the 21st century.

There’s no doubt a giant bottleneck in the supply side for data scientist talent, as enterprises look to gain competitive edge through advanced analytic models and machine-learning algorithms that can provide that extra insight into profit-making opportunities. As with all such situations, creative solutions are bound to emerge and restore balance through a “reversion to the mean” in data-scientist speak. We’re seeing that happen through a variety of approaches, including a team approach (hiring two or three individuals with a combination of the required skills), and offshore sourcing, especially from India. An emerging trend is crowdsourcing.

Companies are building business models that range from creating online marketplaces where you can “shop” for algorithms, and throwing out a specific problem for the community at large to find a solution to. Generally speaking, I am personally skeptical about “plug and play” algorithms. Crowdsourcing seems more likely to provide a unique solution to meet your needs. There are obviously cost-benefit tradeoffs.

Automation, Consumerization and Open Everything

Trends such as democratization and crowdsourcing of analytics are marketplace responses to steady and growing demand on the one hand and a serious shortage of talent on the other. The good news is that accessibility to data, software, hardware, education and even research results are available for free or low cost. Today you can learn a programming language for free from a variety of online services and use free or inexpensive hardware and software for your analysis.

Open source software, such as R and Python, are rapidly becoming the de facto standard tools for analysis by many, particularly in the big data analytics disciplines. This will democratize analytics even further while driving down costs.

Finally, the wearables revolution – Fitbit, Jawbone, Apple watch and the like – will consumerize analytics in all walks of life with their always-on tracking of our movements, fitness and health data. They will also further miniaturize the devices through which we can gain access to the awesome power of big data and cognitive computing platforms such as IBM Watson. What Google Glass has failed to do, maybe Apple Watch will, at least in terms of functionality.

What’s not to like about all this? Not much, except that one shouldn’t get carried away too much about the possibilities. Advanced analytics is still in nascent stages, and the reliability and quality of analytic insights and algorithms has to be carefully examined before being put to work in high-stake decisions. Wearables have a long way to go in delivering on-demand advanced analytics. The Apple Watch, for instance, is a work in progress with many incomplete features, such as the health app.

A nagging question that keeps coming up in boardrooms across the nation is: What is the ROI on analytics investments? The answers are not clear today. If the costs of analytical experimentation are low and the opportunities to experiment are many, this will encourage a higher analytics adoption rate and lead to newer use cases. This will give us the breakthroughs we need to bring about sustained value creation in every sector.

Democratization and crowdsourcing may be our best bets to achieve that in the near future.

Paddy Padmanabhan is CEO of Damo Consulting Inc., a management consulting firm that provides advanced analytics solutions to healthcare enterprises. A business leader and entrepreneur with more than 15 years of experience in the healthcare sector, Padmanabhan has held global leadership positions in top-tier consulting firms and led large global operations teams in India, the Philippines and South America. He has successfully built a $150 million healthcare business, led analytics start-ups and developed cloud-based platforms. For more, see



  1. For a detailed discussion on democratization in analytics, download a white paper by Larry Boyer and Paddy Padmanabhan here.


business analytics news and articles


Related Posts

  • 51
    Organizations of all sizes and types are awash in data possibilities, yet most of them cannot capitalize on the potential for a variety of reasons. The good news, however, is that with the right decisions and focus, these possibilities can turn quickly into realized opportunities.
    Tags: data, technology, analytics, big
  • 47
    International Data Corporation (IDC) recently released a worldwide Big Data technology and services forecast showing the market is expected to grow from $3.2 billion in 2010 to $16.9 billion in 2015. This represents a compound annual growth rate (CAGR) of 40 percent or about seven times that of the overall…
    Tags: data, big, technology, will, growth
  • 46
    November/December 2014 Big data needs advanced analytics, but analytics does not need big data. By Eric A. King Thanks big data! Now we’re even more data-rich … yet remain information-poor. After staggering investments motivated by an overabundance of buzz and hype, big data has yet to produce cases that reveal…
    Tags: data, big, analytics, will, advanced
  • 43
    The 2012 U.S. presidential election is over, and from a statistical viewpoint, the winner was a small group of people armed with analytics who out-predicted many so-called political experts (who relied mostly on gut instinct and experience). The election demonstrated that analytics fueled by big data and advancement in computing…
    Tags: data, analytics, big, computing, technology
  • 42
    Features Why optimization models fail By Patricia Randall Supply chain and manufacturing: How to avoid chaos in the field by combining simulation and real-time optimization. AI: Path to an intelligent enterprise By Joseph Byrum Imagine a future guided by artificial intelligence: Augmenting human decision-making at the enterprise level to generate…
    Tags: analytics, data, enterprise, scientists, growth, advanced


Former INFORMS President Cook named to U.S. Census committee

Tom Cook, a former president of INFORMS, a founding partner of Decision Analytics International and a member of the National Academy of Engineering, was recently named one of five new members of the U.S. Census Bureau’s Census Scientific Advisory Committee (CSAC). The committee meets twice a year to address policy, research and technical issues relating to a full range of Census Bureau programs and activities, including census tests, policies and operations. The CSAC will meet for its fall 2018 meeting at Census Bureau headquarters in Suitland, Md., Sept. 13-14. Read more →

Gartner identifies six barriers to becoming a digital business

As organizations continue to embrace digital transformation, they are finding that digital business is not as simple as buying the latest technology – it requires significant changes to culture and systems. A recent Gartner, Inc. survey found that only a small number of organizations have been able to successfully scale their digital initiatives beyond the experimentation and piloting stages. “The reality is that digital business demands different skills, working practices, organizational models and even cultures,” says Marcus Blosch, research vice president at Gartner. Read more →

Innovation and speculation drive stock market bubble activity

A group of data scientists conducted an in-depth analysis of major innovations and stock market bubbles from 1825 through 2000 and came away with novel takeaways of their own as they found some very distinctive patterns in the occurrence of bubbles over 175 years. The study authors detected bubbles in approximately 73 percent of the innovations they studied, revealing the close relationship between innovation and stock market bubbles. Read more →



INFORMS Annual Meeting
Nov. 4-7, 2018, Phoenix

Winter Simulation Conference
Dec. 9-12, 2018, Gothenburg, Sweden


Applied AI & Machine Learning | Comprehensive
Sept. 10-13, 17-20 and 24-25

Advancing the Analytics-Driven Organization
Sept. 17-20, 12-5 p.m. LIVE Online

The Analytics Clinic: Ensemble Models: Worth the Gains?
Sept. 20, 11 a.m.-12:30 p.m.

Predictive Analytics: Failure to Launch Webinar
Oct. 3, 11 a.m.

Advancing the Analytics-Driven Organization
Oct. 1-4, 12 p.m.-5 p.m.

Applied AI & Machine Learning | Comprehensive
Oct. 15-19, Washington, D.C.

Making Data Science Pay
Oct. 29 -30, 12 p.m.-5 p.m.


CAP® Exam computer-based testing sites are available in 700 locations worldwide. Take the exam close to home and on your schedule:

For more information, go to