Secrets of Data Analytics Leaders

The data mesh paradigm is in a nascent stage with data personas and organizations craving clarity and quick answers. Best practices are yet to be crystallized. Mesh done incorrectly runs the risk of degenerating into silos. Published at: https://www.eckerson.com/articles/caution-data-leaders-plan-carefully-before-rushing-to-data-mesh

I got energized walking the show floor at the Gartner Data & Analytics event last month and learned a few things about the future of our industry. Published at: https://www.eckerson.com/articles/quick-recap-of-gartner-conference-2023

The modern data stack is a loose collection of technologies, often cloud-based, that collaboratively process and store data to support modern analytics. It must be automated, low code/no code, AI-assisted, graph-enabled, multimodal, streaming, distributed, meshy, converged, polyglot, open, and governed. Published at: https://www.eckerson.com/articles/twelve-must-have-characteristics-of-a-modern-data-stack

AutoML and the emerging approach of declarative ML help simplify the process of creating and refining ML models. Published at: https://www.eckerson.com/articles/automl-and-declarative-machine-learning-comparing-use-cases

One version of the truth is the holy grail of data and analytics. However, the promise of one version of the truth still evades us because even with consistent data, the truth is, as the film My Cousin Vinny demonstrates, a matter of perspective and context. Published at: https://www.eckerson.com/articles/one-version-of-the-truth-according-to-my-cousin-vinny

Data leaders know the importance of change management, but few understand the dynamics involved in driving adoption. A new book by Damon Centola shows how social networks spread and inhibit change. Published at: https://www.eckerson.com/articles/how-change-happens-driving-technology-adoption

Over the past 20 years or more, data architecture practices have focused almost exclusively on managing data for analytics. Operational data is much more than source data for analytics. We must give attention to operational data architecture or pay the price in data disparity, data friction, and technical debt. Published at: https://www.eckerson.com/articles/operational-data-architecture

Designed and implemented well, automated workflows can make the modern business just a little less chaotic and complex. This blog explores the opportunity for automated workflows to help cross-functional teams collaborate and standardize organizational master data. Published at: https://www.eckerson.com/articles/master-data-management-and-operational-workflows-two-modern-use-cases

Data fabric is one of those buzzwords that’s used so much and in so many ways that it often elicits an eyeroll—undeservedly so. The phrase is shorthand for a complex and important set of issues that we’re all working to manage. In this article we’ll review what data fabric is and why it’s important. Published at: https://www.eckerson.com/articles/data-fabric-s-use-of-abstraction-and-metadata

The data pipeline market comprises four segments: data ingestion, data transformation, DataOps, and orchestration. This blog defines three principles for successful pipelines: (1) watch the innovative startups; (2) use suites where you can; and (3) use point tools where you must. Published at: https://www.eckerson.com/articles/modern-data-pipelines-three-principles-for-success

The data mesh framework doesn’t specify a key component that completes the last mile of the architecture: a data provisioning environment. New technology that underpins modern data marketplaces complement data mesh, providing a frictionless way for data providers and data consumers to exchange data. Published at: https://www.eckerson.com/articles/data-mesh-s-missing-ingredient-a-data-marketplace

Traditional techniques for managing data quality break at scale. Machine learning algorithms can automate aspects of the data quality workload, ensuring that the data the business users consume is reliable. This article profiles three tools and approaches that use ML to automate data quality. Published at: https://www.eckerson.com/articles/three-data-quality-automation-tools-you-should-consider

We must treat metadata like a fully-vested member of the enterprise data landscape. A unifying taxonomy is a good place to start making metadata a focus of data management rather than just a tool. This article explores how to start wrangling diverse and distributed metadata. Published at: https://www.eckerson.com/articles/wrangling-metadata-making-it-the-object-of-data-management

We enter 2023 in a haze of uncertainty. Enterprises must rationalize analytics projects, shift to lower-risk use cases, and control cloud costs. They also must measure the ROI of analytics projects and use data governance to reduce business risk. Published at: https://www.eckerson.com/articles/analyzing-a-downturn-five-principles-for-data-analytics-in-2023

This blog defines governed data integration and describes how it enabled two manufacturers to synchronize data flows from the factory floor to the customer. Published at: https://www.eckerson.com/articles/governed-data-integration-for-manufacturers

A rising number of financial services firms are adopting the discipline of governed data integration to build 360-degree customer views. Published at: https://www.eckerson.com/articles/governed-data-integration-for-financial-services

Synthetic data and artificial intelligence (AI) complement each other but are both subject to the risk of AI bias. Consequently, companies need to implement architectural and governance controls to reduce the bias that synthetic data can inject into AI models. Published at: https://www.eckerson.com/articles/mitigating-the-risk-of-bias-in-synthetic-data-for-ai

As enterprises grow more dependent on the cloud and as the economy convulses, FinOps will soon become mandatory. Published at: https://www.eckerson.com/articles/the-rise-of-finops-cost-governance-for-cloud-based-analytics

Business domains have a range of data & analytics capabilities that enterprise data teams must support. The key is to ensure domain activity aligns with enterprise standards and best practices to ensure data consistency and avoid silos. Published at: https://www.eckerson.com/articles/an-operating-model-for-data-analytics-part-iv-red-team-composition

Active metadata is not a type of metadata, it’s a way of using metadata to power systems. Active metadata is a critical feature of modern data architectures such as data fabric and data mesh. It makes things work such as data access management, data classification, and data quality management. Published at: https://www.eckerson.com/articles/active-metadata-the-critical-factor-for-mastering-modern-data-management

This blog explores four use cases for data orchestration and examples of the supporting architectural elements. Published at: https://www.eckerson.com/articles/architecting-data-orchestration-four-use-cases

There are many models for bridging business and technical teams. These models can be more centralized or decentralized in nature, depending on the culture of the organization and nature of the business domain. Each requires a strong enterprise data teams comprised of multiple departments and roles. Published at: https://www.eckerson.com/articles/an-operating-model-for-data-analytics-part-iii-team-composition-and-dynamics

Data observability provides intelligence about data quality and data pipeline performance, contributing to the disciplines of DataOps and FinOps. Vendors such as DataKitchen, DataOps.live, Informatica, and Unravel offer solutions to help enterprises address these overlapping disciplines. Published at: https://www.eckerson.com/articles/the-blending-disciplines-of-data-observability-dataops-and-finops

Hybrid development teams are critical to the success of a data & analytics program. Data leaders must invest time, energy, and thought to the creation of these teams and how best to support them. It’s critical that they allocate staff time to nurture knowledge flows between component groups. Published at: https://www.eckerson.com/articles/an-operating-model-for-data-analytics-part-ii-knowledge-flows

“Metadata is data about data” is a bad definition. It’s vague and recursive. It’s like saying climate is the weather of weather. But if metadata is not data about data, then what is it? Follow my thought process working toward a better understanding of metadata and its role in today’s data landscape. Published at: https://www.eckerson.com/articles/fixing-metadata-s-bad-definition

Data Mesh gets a lot of discussion. Some see it as revolutionary—the first new data architecture thinking in years. Others view it as a dangerous backward slide to the chaos of data silos. The reality lies somewhere between—a big shift in architecture thinking with some inherent risk. A few good architecture practices help to realize the benefits while managing the risks. Published at: https://www.eckerson.com/articles/a-thoughtful-approach-to-data-mesh

Many machine learning projects fail because data scientists don’t have the right data. Techniques such as synthetic data is a novel algorithmic approach to address algorithmic risks. Published at: https://www.eckerson.com/articles/synthetic-data-for-ai-definition-risks-and-strategies

Many machine learning projects fail because data scientists don’t have the right data. Techniques such as synthetic data is a novel algorithmic approach to address algorithmic risks. Published at: https://www.eckerson.com/articles/synthetic-data-for-ai-definition-risks-and-strategies

Data orchestration uses caching, APIs, and centralized metadata to help compute engines access data in hybrid or multi-cloud environments. Data platform engineers can use data orchestration to gain simple, flexible, and high-speed access to distributed data for modern analytics and AI projects. Published at: https://www.eckerson.com/articles/data-orchestration-simplifying-data-access-for-analytics

Data access management (DAM) is the process of defining and enforcing policies that control access to application data throughout the enterprise. New approaches to DAM provide greater access and better protection through centrally managed policies that are universally enforced. Published at: https://www.eckerson.com/articles/data-access-management-zero-sum-game-over

This article, the third in a series, dives into the technologies that underpin modern approaches to location intelligence. It explores databases for industrial-scale geospatial applications, advanced business intelligence (BI) tools for exploratory analysis, and simple use-case specific platforms. Published at: https://www.eckerson.com/articles/location-intelligence-part-iii-enabling-technologies

The data mesh makes business domain experts the owners of their data, which they deliver as a “data product” to analytics teams using a self-service data platform and a federated governance framework. Published at: https://www.eckerson.com/articles/why-enterprises-should-implement-the-data-mesh-with-dataops

An operating model for data & analytics is critical for aligning resources across the enterprise and balancing the needs for agility and governance. An effective operating model is critical to data & analytics success and its creation and upkeeping should be the primary focus of a chief data officer. Published at: https://www.eckerson.com/articles/an-operating-model-for-data-analytics

In this article, Lawson Abinati lays out core principles for market positioning that apply across all industries, as well as to Business Intelligence (BI) professionals. Although many business intelligence (BI) managers see themselves as technologists first, unless they understand the soft skills of sales, marketing, and communication, they won't succeed professionally or make good on their organization's investments in BI. Published at: https://www.eckerson.com/articles/what-is-positioning-and-why-is-it-important

This article summarizes the major characteristics of a modern data architecture and serves as a guide for organizations that are in the midst of developing a new data strategy for the modern age. Published at: https://www.eckerson.com/articles/ten-characteristics-of-a-modern-data-architecture

An analytics center of excellence is the cornerstone of every data strategy, yet few data leaders know how to design one that works effectively. The key is to embrace federated techniques that balance standards and speed, agility and governance. This article explains the core components of an analytics center of excellence. Published at: https://www.eckerson.com/articles/how-to-design-an-analytics-center-of-excellence

Design patterns have proven valuable in many endeavors. Can data pipeline design patterns help to break the data engineering logjam? Published at: https://www.eckerson.com/articles/data-pipeline-design-patterns

The need for adaptable data management architecture has never been more pressing. Yet getting there seems to be more confusing than ever. The field is rampant with buzzwords: data lake, data lakehouse, data fabric, data mesh, data hub, data as a network. Making sense of the confusion begins with sorting out the buzzwords. Published at: https://www.eckerson.com/articles/data-architecture-complex-vs-complicated

Data mesh is an evolutionary concept that’s gained a lot of traction in the software engineering world. It’s both exciting and unnerving—exciting because it represents a sea change in engineering attitudes toward data; unnerving because of its potential to create organizational confusion. Published at: https://www.eckerson.com/articles/the-exciting-unnerving-vision-of-data-mesh

This article, the second in a series, shares cutting-edge examples of location intelligence applications from the real world to help readers understand what’s possible with geospatial data. Published at: https://www.eckerson.com/articles/location-intelligence-part-ii-real-world-examples-from-businesses-that-use-geospatial-data

The data mesh is an attempt by software engineers to remake the data industry in their image. There is a lot of goodness in the data mesh but will it work? The track record for other reform efforts is not good, although new technologies are putting wind into the methodology. Published at: https://www.eckerson.com/articles/data-mesh-translated-software-engineers-try-to-reform-data

Data Stewardship Experience strategies (personal growth, community, societal contribution, and disruptive innovation) can meet several cognitive, social, and psychological needs, and motivate professionals to become productive data stewards. It also removes the stigma of data governance as a rigid and bureaucratic gatekeeping discipline. Published at: https://www.eckerson.com/articles/improving-the-data-stewardship-experience-dsx-productive-motivational-strategies-for-data-governance

Decision AI is an emerging discipline that enables faster, smarter operational decisions by combining decision intelligence and contextual intelligence. Published at: https://www.eckerson.com/articles/decision-ai-and-the-opportunity-for-smarter-faster-actions

This audio blog, the third and final in a series, recommends five guiding principles for success. Published at: https://www.eckerson.com/articles/the-customer-360-data-program-and-cloud-connectors-guiding-principles-for-success

Consider key trends and challenges as you design an effective organizational architecture for data governance while generating value with pervasive analytics. Published at: https://www.eckerson.com/articles/organizational-architecture-can-make-or-break-your-data-governance-program

At Summit 2022, Snowflake paints a picture of a utopian data future where all data is treated equally regardless of where it comes from or what language is used to process it, where processing resources are limitless, and all data is secured and governed. Published at: https://www.eckerson.com/articles/snowflake-summit-2022-growth-vision-and-strategies

Providing analysts with tools that increase their efficiency is critical to helping them do more work with less effort. Published at: https://www.eckerson.com/articles/a-data-analyst-s-guide-to-the-data-catalog

This blog recommends criteria for enterprises to evaluate cloud connectors and ensure the benefits outweigh the costs. Published at: https://www.eckerson.com/articles/evaluating-cloud-connectors-for-the-customer-360-data-program

Machine learning models help respond to the time-based value and risks of business events. To achieve this on an ongoing basis, enterprises should build a streaming ML program based on sound business objectives, a cross-functional team, open platforms, and phased execution. Published at: https://www.eckerson.com/articles/machine-learning-and-streaming-data-pipelines-part-iii-guiding-principles

Enterprise data teams embrace DataOps to achieve new levels of efficiency and effectiveness in delivering data-driven solutions. Roche shows what’s possible when you combine a state-of-the-art cloud data platform with a data mesh architecture and DataOps solution. Published at: https://www.eckerson.com/articles/what-can-dataops-do-for-you-ask-roche