Share with your friends










Submit

Analytics Magazine

Executive Edge: It pays to modernize your data architecture

September/October 2015

business analytics news and articles

Arvind PurushothamanBy Arvind Purushothaman

In today’s world where data is collected at every interaction, be it over the phone, mobile, PC, sensors, with or without us knowing, it becomes important to have a strategy around data. Traditionally, data has been seen as something to “run the business,” but, in today’s context, it can actually “be the business” if monetized well. An example of Internet of Things (IoT) data in a customer context is the wristband one wears at amusement parks that provides real-time data about customer interaction at all times, and this data can be processed in near real time to push out relevant offers and alerts to enhance the customer experience. The question is: How do organizations prepare themselves to take advantage of data?

The key lies in building a modern data architecture that is open, flexible and scalable, something that can accommodate your existing data assets as well as potential new ones. Before we talk about specific steps to modernize data architecture, let’s look at typical challenges:

  1. Many applications within the organization have been around for 20 or more years. While the usage for some of them is known, it is still not clear who is leveraging the data in each application and for what purpose. How do we find out?
  2. To meet their reporting needs, organizations have built multiple data assets including data warehouses and data marts. Additionally, they have power users collating data from multiple sources and creating reports using MS Excel. Numbers are inconsistent and vary based on who is preparing them and the intended purpose.
  3. Organizations have multiple applications and data assets starting with mainframe-based ones, client-server, Web applications and some newer cloud-based applications, all co-existing. They struggle to find the right people to support the applications, especially the older ones.
  4. Organizations are aware of the new developments in the big data space including NoSQL databases and the Hadoop ecosystem, and have typically embarked on some initiatives to get started on this. The main challenge is around integrating this with the traditional data warehouse technologies.
  5. People, and by extension, their skills, are the biggest assets of any organization. CIOs are concerned about having to find an army of programmers for populating Hadoop-based data repositories. The other big concern is how to leverage existing SQL skills, which people have acquired over the years.

These are valid concerns, and some are more applicable than others based on the context. Nonetheless, given the inevitable need to be able to better monetize data and modernize technology platforms, it is important to have a strategy. I recommend the
following approach:

  1. Data asset inventory: Create a complete list of data assets – legacy, data warehouses, data marts, data islands. Identify the data flows between these assets and the usage patterns. It might be particularly hard for some legacy systems, but this serves as the starting point for any consolidation and modernization.
  2. Data asset rationalization: Based on the list of data assets and the usage, it is important to rationalize them. What this means is to identify if the same data is coming from multiple applications, and if so, which is the authoritative source, and which ones can be retired. This is a very important exercise and can help consolidate the number of data assets to a manageable few. In this context, master data management is critical to ensure you have good quality data.
  3. Data lineage: Undertaking a data lineage exercise to identify data flows – creating detailed documentation especially for the legacy applications – is a must. This greatly reduces the risk of dependency on key personnel and also makes it easier to migrate to a future state architecture.
  4. Data infrastructure: Have a big data and cloud strategy in place to bring in newer technologies in a pilot mode. Start with a non-legacy application to understand the technology, and move applications over in conjunction with data asset rationalization. The “data on cloud” is going to be an important component of modern architecture especially when dealing with IoT data.
  5. Data technology: It pays to understand the different options available in a very crowded and rapidly evolving marketplace, and to select the right technologies that fit into your architecture from a technology standpoint as well as a people standpoint. For example, using a data integration tool with big data connectors will eliminate the need for people who can write MapReduce code.

Creating a holistic data strategy in light of changes in the business, and taking a structured approach, will definitely help lay a solid foundation that will be the basis for monetizing data.


Arvind Purushothaman is the practice head and senior director of Information Management & Analytics at Virtusa’s Chennai ATC of Virtusa, an information technology services provider with global reach. He has 19 years of industry experience, with a focus on planning and executing data management and analytics initiatives. Prior to taking on this role, he was involved in architecting and designing centers of excellence, as well as service-delivery functions focused on information management encompassing traditional data warehousing, master data management and analytical reporting. He holds an MBA from Georgia State University.

Related Posts

  • 63
    Imagine a sensor inside an offshore drilling rig. The sensor checks for damage to a critical valve. To do so, the sensor regulates pressure in the oil well 7,000 feet below the ocean’s surface. This sensor generates data that might have gone unnoticed half a decade ago. Back then, the…
    Tags: data, big, things, internet
  • 56
    In today’s world where data is collected at every interaction, be it over the phone, mobile, PC, sensors, with or without us knowing, it becomes important to have a strategy around data. Traditionally, data has been seen as something to “run the business,” but, in today’s context, it can actually…
    Tags: data, architecture, internet, things
  • 53
    FEATURES Fulfilling the promise of analytics By Chris Mazzei Strategy, leadership and consumption: The keys to getting the most from big data and analytics focus on the human element. How to get the most out of data lakes By Sean Martin A handful of requisite business skills that facilitate self-service…
    Tags: data, big, things, internet
  • 52
    IoT (Internet of Things) devices have become increasingly popular in recent years. They are all around us – from fitness trackers on our wrists to smart thermostats in our homes – and adoption will only continue to grow in the coming years. In fact, Gartner, Inc. reported that 5.5 million…
    Tags: data, things, internet
  • 50
    Through advances such as big data and the Internet of Things (IoT), the field of analytics has been growing by leaps and bounds. However, much of the focus, particularly in the United States, has been in the consumer (i.e., business-to-consumer, or B2C) market. Indeed, much of the innovation coming out…
    Tags: data, things, internet


Headlines

Former INFORMS President Cook named to U.S. Census committee

Tom Cook, a former president of INFORMS, a founding partner of Decision Analytics International and a member of the National Academy of Engineering, was recently named one of five new members of the U.S. Census Bureau’s Census Scientific Advisory Committee (CSAC). The committee meets twice a year to address policy, research and technical issues relating to a full range of Census Bureau programs and activities, including census tests, policies and operations. The CSAC will meet for its fall 2018 meeting at Census Bureau headquarters in Suitland, Md., Sept. 13-14. Read more →

Gartner identifies six barriers to becoming a digital business

As organizations continue to embrace digital transformation, they are finding that digital business is not as simple as buying the latest technology – it requires significant changes to culture and systems. A recent Gartner, Inc. survey found that only a small number of organizations have been able to successfully scale their digital initiatives beyond the experimentation and piloting stages. “The reality is that digital business demands different skills, working practices, organizational models and even cultures,” says Marcus Blosch, research vice president at Gartner. Read more →

Innovation and speculation drive stock market bubble activity

A group of data scientists conducted an in-depth analysis of major innovations and stock market bubbles from 1825 through 2000 and came away with novel takeaways of their own as they found some very distinctive patterns in the occurrence of bubbles over 175 years. The study authors detected bubbles in approximately 73 percent of the innovations they studied, revealing the close relationship between innovation and stock market bubbles. Read more →

UPCOMING ANALYTICS EVENTS

INFORMS-SPONSORED EVENTS

INFORMS Annual Meeting
Nov. 4-7, 2018, Phoenix

Winter Simulation Conference
Dec. 9-12, 2018, Gothenburg, Sweden

OTHER EVENTS

Applied AI & Machine Learning | Comprehensive
Sept. 10-13, 17-20 and 24-25


Advancing the Analytics-Driven Organization
Sept. 17-20, 12-5 p.m. LIVE Online


The Analytics Clinic: Ensemble Models: Worth the Gains?
Sept. 20, 11 a.m.-12:30 p.m.


Predictive Analytics: Failure to Launch Webinar
Oct. 3, 11 a.m.


Advancing the Analytics-Driven Organization
Oct. 1-4, 12 p.m.-5 p.m.


Applied AI & Machine Learning | Comprehensive
Oct. 15-19, Washington, D.C.


Making Data Science Pay
Oct. 29 -30, 12 p.m.-5 p.m.


CAP® EXAM SCHEDULE

CAP® Exam computer-based testing sites are available in 700 locations worldwide. Take the exam close to home and on your schedule:


 
For more information, go to 
https://www.certifiedanalytics.org.