First, there was data—valuable information, but stored in separate areas or “silos” across a company. These siloed data stores made it possible to use data in different ways, but the disparate databases were isolated from each other. Enterprise data warehouses could collect data from various sources into more integrated locations, while data lakes emerged to support analytic use cases.
QUICK TAKES
Data-driven organizations are 23 times more likely to acquire customers, six times as likely to retain customers, and 19 times more profitable, according to a McKinsey survey.
Digitally mature firms are 26% more profitable than their peers, shows MIT research.
85% of data and analytics projects fail, research by Gartner found.
90% of data in an organization is never successfully used for any strategic purpose, according to a joint report from IBM and Carnegie Mellon.
Then came data mesh, “a decentralized data architecture that organizes data by a specific business domain—for example, marketing, sales, customer service, and more—providing more ownership to the producers of a given dataset,” according to IBM.
“Typically, organizations’ primary considerations in enterprise data architecture evolution were storing and managing data effectively,” write Deloitte authors in a Wall Street CIO Journal article on data mesh. “Today, however, that focus is shifting toward using the data, with the dual aims of effectiveness and efficiency. Enter the data mesh, a federated data architecture that allows organizations to address the growing demand for access to enterprise data throughout the organization by breaking down monolithic architectures into modular domains, decomposing the pipeline, and strategically embedding specialized data talent.”
But as AI and ML thought leaders Arun Marar and Prashanth Southekal argue in a recent Dataversity article, data fabric — a powerful architecture that standardizes data management practices while providing consistent capabilities across hybrid multi-cloud environments — may be the ultimate solution. As they point out, data-driven organizations are 19 times more profitable than their peers, but 85% of data and analytics projects fail.
“With this backdrop, we introduce the ‘data analytics fabric’ (DAF) concept,” write Marar and Southekal, “to answer this fundamental question: ‘What is required to effectively build a decision-enabling system from data science algorithms to measure and improve business performance?’”
The authors present data analytics fabric as five key manifestations:
1. Measurement Focused: This includes descriptive analytics (which asks the question, “What happened?”; predictive analytics (“What will happen?”); and prescriptive analytics (“How can we make it happen?”). These are the three main types of analytics to measure and improve business performance.
2. Variable Focused: “Data can also be analyzed based on the number of variables available,” write Marar and Southekal. Data analytics techniques can be univariate, based on the pattern present in a single variable; bivariate, a correlation technique based on two variables; or multivariate, used for analyzing more than two variables.
3. Supervision Focused: This pertains to “training” the data, and such data analytics fabric can either be causality (labeled data) and non-causality (not requiring labeled data).
4. Data Type Focused: “This dimension or manifestation of the data analytics fabric focuses on the three different types of data variables related to both the independent and dependent variables that are used in the data analytics techniques for deriving insights,” explain the authors. It can involve nominal data, such as gender, product description, and customer address; ordinal or ranked data (i.e., market capitalization, vendor payment terms, customer satisfaction scores, and delivery priority); or numeric data.
5. Results Focused: Business value can be driven by analytics either through products or projects. A data analytics product is a reusable data asset to serve the long-term needs of the business while a data analytics project is designed to address a particular or unique business need and has a defined or narrow user base or purpose.
“The world’s economy will dramatically transform in the coming years as organizations will increasingly use data and analytics to derive insights and make decisions to measure and improve business performance,” conclude Marar and Southekal. “The DAF and its five manifestations discussed here can enable analytics to be deployed effectively based on business needs, available capabilities, and resources.”