People, Technology and Evolving Data Management

By Patrick Diaz in Analytics, General, System Integrations, Data Management | November 18, 2022

Data Fabric. Data Mesh. The latest buzzwords in the ever-growing data analytics community. Utility companies have been working on solutions to address their operational data and how to keep it organized and available. Each solution presents its strengths and weaknesses and the implementation of each is dependent on the technology and/or organizational change management a given company is willing to acquire or implement. At Xtensible, we have worked with many companies deliver solutions for operational data management and analytics. This article outlines the similarities and differences between a Data Fabric and Data mesh and how Xtenisble’s Affirma product supports both approaches.

Defining a Data Fabric and Data Mesh

Per Gartner, a data fabric is defined as a data management design for achieving flexible, reusable and improved data management which is achieved through metadata. Paralleled to previous methods, “active metadata and semantic inference are critical new aspects of data fabric.” As Gartner states, it is not a “rip and replacement of existing data management infrastructure” but rather an architecture that is meant to evolve over time¹. The idea being that existing systems share their metadata at first and then as the data fabric architecture matures, the participating systems adjust based on recommendations generated by the fabric which is made possible using technology such as automation and/or artificial intelligence. A couple of key points about a data fabric are that systems are integrated and there is reusable data.

A data fabric is not a product but rather a design concept centered around metadata which can live in different domains including but not limited to a company’s data center or in the cloud.

Another paradigm to support enterprises in their effort for data organization and data management is the concept of a data mesh. The data mesh was first documented by Zhamak Dehghani of Thoughtworks. The true definition of a data mesh may change from vendor to vendor or organization to organization, but the concept as described by Dehghani of Thoughtworks is as follows: “distributed data products” oriented around domains and owned by “independent cross-functional teams” who have embedded data engines and product owners, using common “data infrastructure” as a platform to host, prep and serve their data assets.

Unlike traditional approaches, the data mesh would treat data lakes, warehouses and other technologies as nodes versus the center of the overall architecture.

The data mesh concept is focused on four characteristics:

Domain-oriented decentralized data ownership
Data as a product
Instant data access via a self-service platform
Dispersed data governance.²

The Stack-Up between Data Fabric vs. Data Mesh

One can see that the two architectures described above have many similarities and share the same goal: getting users access to data with technology and people. But these concepts do have their differences that should be considered as well. On the surface, one of the immediate differences is that data fabric focuses on the metadata and the utilization of technology. The data mesh architecture focuses more on organization change utilizing data subject matter experts and data governance to drive the solution. A data fabric requires active metadata management as a result of the continuous learning and expansion of metadata. The data mesh utilizes domain data owners and independent cross-functional teams using data as a product and following a centralized governance model. The approach outlined for a data mesh could be considered a bottom-up approach and begs the question do you have the attention of the SMEs? On the other hand, data fabrics could be seen as a top-down approach connecting data across multiple sources and woven with an integration layer.

The Affirma Data Journey supports Data Fabric and Data Mesh

Whether an organization decides to utilize a data fabric, a data mesh or a combination of the two, the basis for implementing either one is defining a structure to access data across different technologies and/or platforms. A good roadmap to do this is to start down the path of a common information model. This semantic modeling method is the core of Xtensible’s enterprise semantic and metadata management approach. For years, Xtensible has been working with utility companies to identify ways to move away from data silos and into a more centralized data approach. With the repeatable process we have developed over the years, Xtensible has developed Affirma, which is a single solution for enterprise semantic modeling and metadata management. With Affirma teams can setup a centralized managed enterprise semantic model that create a consistent data definition and semantics across all enterprise interfaces such as APIs, data pipelines and data stores. Additionally, Affirma increases the ability to adapt and change without major impact to other systems and supports the building of a microservice and a data mesh architecture. Finally, utilities can gain access to metadata facilitating the ability to integrate data required for decision making, analysis, planning, risk management and reporting.

Interested in learning more about Affirma and how and how to navigate change? Speak to a member of the Xtensible team.

Sources:

Back To Blog