Share with your friends










Submit

Analytics Magazine

Data professionals spend almost as much time prepping data as analyzing it

Nearly 40 percent of data professionals spend more than 20 hours per week accessing, blending and preparing data rather than performing actual analysis, according to a survey conducted by TMMData and the Digital Analytics Association. More than 800 DAA community members participated in the survey held earlier this year. The survey revealed that data access, quality and integration present persistent, interrelated roadblocks to efficient and confident analysis across industries.

Having analyzed the results, TMMData and the DAA reported the following key findings:

Disjointed, inaccessible data is a major productivity inhibitor for analysts, diverting skilled resources from contributing to valuable business intelligence.
Nearly two in five (38.7 percent) data professionals are spending more than half of their work week on tasks unrelated to actual analysis: 43.8 percent of managers reported that 51 percent or more of their team’s work week is spent collecting, integrating and preparing data rather than analyzing it, while 31.3 percent of analysts said they spend 21 or more hours a week on data housekeeping.

Many data professionals struggle with data access. Forty-three of respondents named access as one of their top two analytics challenges. Nearly three in five respondents (56.9 percent) said it takes days or weeks to access all the data they need, and nearly 10 percent (9.8 percent) say they can rarely or never access a complete range of data sources. Only a third of data professionals (33.4 percent) are immediately able to access all their data or can get it in less than a day.

As a result, a majority of analysts find it necessary to learn programming languages specifically to help them access or prepare data for analysis. Outside of mandates from their employers, a full 70 percent of analysts reported taking it upon themselves to learn to code for this reason, and more than a quarter of those analysts have spent 80 or more hours learning to program.

Data professionals lack confidence in data accuracy, which poses an existential threat to the industry.
The industry is split in terms of confidence in data accuracy, with a little more than half saying they are always or reasonably confident in the data their teams work with regularly (51.7 percent), and slightly less than half reporting that they question the accuracy of that data (48.3 percent). In fact, second only to data access, 26.7 percent of respondents identified inaccurate data as one of their top two analytics challenges. Data professionals tasked with analyzing organizational information meaningfully and actionably can’t adequately perform their core job function without accurate data, incenting the industry to invest in data quality.

A lack of formalized data governance programs could be to blame for some of the uncertainty surrounding data quality; less than a quarter of respondents said their organization has a governance program in place (22.7 percent). Nearly one in five (18.2 percent) reported that each department is responsible for its own data governance, rather than having a comprehensive organizational governance program ensuring consistency across business units.

The industry recognizes its areas of inefficiency and is poised to make important investments that will free data professionals up to perform next-generation analysis.
The survey findings indicate a consensus among industry professionals on the importance of solving data access, integration and governance problems. In addition to the statistics already discussed, data integration was the most frequently cited investment priority for the upcoming year (as reported by more than 40 percent of management and 37.8 percent of staff). Nearly a third (32 percent) of respondents’ organizations are planning or researching a formalized data governance program, and nearly 20 percent (19.4 percent) are in the early stages of rolling out their governance programs, primarily with the goal of ensuring that everyone is working with consistent data.

 

Related Posts

  • 87
    With the rise of big data – and the processes and tools related to utilizing and managing large data sets – organizations are recognizing the value of data as a critical business asset to identify trends, patterns and preferences to drive improved customer experiences and competitive advantage. The problem is,…
    Tags: data
  • 85
    The Internet of Things (IoT) is considered to be the next revolution that touches every part of our daily life, from restocking ice cream to warning of pollutants. Analytics professionals understand the importance of data, especially in a complicated field such as healthcare. This article offers a framework on integrating…
    Tags: data
  • 81
    Big data investments continue to rise but are showing signs of contracting, according to a recent survey by Gartner, Inc. The survey revealed that 48 percent of companies have invested in big data in 2016, up 3 percent from 2015. However, those who plan to invest in big data within…
    Tags: data, percent
  • 81
    Businesses are greatly expanding the autonomous capabilities of their products, services and manufacturing processes to better optimize their reliability and efficiency. The processing of big data is playing an integral role in developing these prescriptive analytics. As a result, data scientists and engineers should pay attention to the following aspects…
    Tags: data
  • 79
    Frontline Systems releases Analytic Solver V2018 for Excel Frontline Systems, developer of the Solver in Microsoft Excel, recently released Analytic Solver V2018, its full product line of predictive and prescriptive analytics tools that work in Microsoft Excel. The new release includes a visual editor for multi-stage “data science workflows” (also…
    Tags: data


Headlines

Challenges facing supply chain execs: leadership, labor, legacy technology

While most companies recognize the value of a digitally enabled supply chain – empowered by new technologies like artificial intelligence, blockchain, big data and analytics – many chief supply chain officers (CSCOs) are not leveraging their C-suite counterparts to help reinvent the supply chain function and transform it into an engine of new growth models and customer experiences, according to new research from Accenture. Read more →

Data Science Bowl: Using AI to accelerate life-saving medical research

Imagine unleashing the power of artificial intelligence to automate a critical component of biomedical research, expediting life-saving research in the treatment of almost every disease from rare disorders to the common cold. This could soon be a reality, thanks to the fourth Data Science Bowl, a 90-day competition in which, for the first time, participants trained deep learning models to examine images of cells and identify nuclei, regardless of the experimental setup – and without human intervention. Read more →

UPCOMING ANALYTICS EVENTS

INFORMS-SPONSORED EVENTS

INFORMS International Conference
June 17-20, 2018, Taipei, Taiwan

INFORMS Annual Meeting
Nov. 4-7, 2018, Phoenix

OTHER EVENTS

Advancing the Analytics-Driven Organization
July 16-19, noon-5 p.m.


Making Data Science Pay
July 30-31, 12:30 p.m.-5 p.m.


Predictive Analytics: Failure to Launch Webinar
Aug. 18, 11 a.m.


Applied AI & Machine Learning | Comprehensive
Sept. 10-13, 17-20 and 24-25

CAP® EXAM SCHEDULE

CAP® Exam computer-based testing sites are available in 700 locations worldwide. Take the exam close to home and on your schedule:


 
For more information, go to 
https://www.certifiedanalytics.org.