7 Steps for Building a Data Culture for DataOps

DataOps represents the future of data management. An evolution of the practices of DevOps applied to enterprise data challenges, DataOps can solve many of the problems companies now face when trying to unlock the power of data by expanding its use to every corner of an organization.

As DataOps becomes more broadly adopted, it is clear that the practice must be tailored to the unique needs of an organization. It must also be able to evolve to address issues such as data governance, security, and access control in an automated, scalable fashion. In addition, the underlying infrastructure supporting DataOps must be built to evolve to achieve operational agility.

But a big part of this evolution will be cultural. We see the following cultural dimensions as crucial to successful adoption of DataOps: collaboration; automation and a metadata mindset; data as a shared asset; end-to-end design thinking; enlightened and guided empowerment; silo paranoia; and push-down decision-making.

Although many of these areas overlap and support each other, the end result for DataOps – as it was for DevOps – will be an organization that is more unified and works together at a much faster pace.

1. Collaboration

Collaboration in the world of DevOps is simpler than it will be in DataOps. DevOps unifies the worlds of IT operations and software development. These two powerful and complex functions were relatively focused and highly technology oriented. In DataOps, the landscape is much bigger. You’re not just talking about two engineering-oriented disciplines. DataOps encompasses everyone from the beginning of the data supply chain where data originates to all the people who model and blend data all the way to those who put it to use in applications and analytics. Collaboration in DataOps thus has many more dimensions than it does in DevOps.

2. Automation and Metadata Mindset

Too often today, people solve their data challenges in one-off ways using isolated spreadsheets or data marts that they maintain themselves. Just getting the job done with spreadsheets can creates problems, like relying on stale data or introducing errors in formulas or links. The world of DataOps embraces automation, with metadata as its foundation to facilitate broad use of the latest data, eliminating the frustration and delays that led people to use spreadsheets. With DataOps, users want the same type of operational simplicity and rapid iteration as with DevOps, but to achieve it, all of the functions have to be automated.

3. Understanding Data as a Shared Asset

Everyone talks about data, but its economic value is less frequently explored. Unlike other assets, data does not depreciate, becoming less valuable the more it is used. Instead, data gains value as it is used. In the past, organizational boundaries created ownership around certain types of data. In order to access that data, you would have to go through the organization that owned that data. The instinct has been to control data and this won’t go away easily, as data is a valuable resource and therefore a source of power. But we need to have the governance, access control and security to enable prudent sharing, because unlimited sharing doesn’t work in most venues. If companies allow hoarding of data, it can lead to new silos.

4. End-to-End Design Thinking

One way to understand both DevOps and DataOps is as an application of “design thinking.” Many of the principles of design thinking are in play in DataOps. DevOps expanded the scope of the problem, seeing it not as a Dev problem or an Ops problem, but a DevOps problem, which included the impact on the users. DataOps does the same thing with organizations thinking through the flow of data from its creation to its use. But with DataOps, far more groups are impacted, as the entire organization relies on data. The key transformation is not that specialities won’t exist anymore, but by putting everything in the service of a larger automated system people are allowed to create and maintain data supply chains that operate efficiently.

5. Enlightened and Guided Empowerment

What we’re looking to achieve with DataOps is guided and enlightened empowerment. We want to provide a simpler way for end users to get the data they need. We want data pipelines to be easier to manage and security and governance less cumbersome to administer. It means making complex technology used for infrastructure easier to manage for operational agility. It means expanding automation at all levels. Therefore, self-service must not be about just giving someone a simplified system, but one that offers suggestions and guidance. Also, self-service shouldn’t be a barrier to collaboration with experts. We want data experts to provide help when it is needed.

6. Silo Paranoia

DataOps and DevOps sought to break down silos that emerged naturally between parts of the organization. The impulses that created these silos will not disappear overnight, whether it’s attempting to control assets or gain power. Part of DataOps culture is staying vigilant and on the lookout for any time silos begin to reemerge. In a DataOps culture, it should be possible to complain when unreasonable restrictions on using data are stopping productive work and have that complaint heard.

7. Push Down Decision-Making

Perhaps the biggest cultural change from DataOps relates to decision-making. If data is flowing everywhere and informing more people, it is only logical that they make decisions based on that data. Push down decision-making, where those using the data make the decisions, is a major cultural change. Too often, people at the edge of the business are not accustomed to making decisions confidently because they have never done so before and often weren’t hired to do so. In addition, autonomous systems will play a role in pushing decision-making down and toward the edge. This type of change, where decision-making moves to the edge with more empowered people using data-based insights, alters the approach to management and oversight and builds decision-making “muscle” throughout the enterprise.

Culture is a broad and tricky subject, and this discussion is certainly not the last word on the topic. However, as it evolves, I invite you to read through the related content and join the discussion. It’s a fascinating time to be in data.

This story is an abridged version of the Hitachi Vantara white paper: Impact of DataOps: Collaboration, Automation and the War on Silos.

Related Stories

• Hitachi Vantara Launches Lumada Industrial DataOps for IIoT Scalability – Steve Garbrecht
• DataOps for the Data-Driven – Manish Jain
• Read the Press Release: Hitachi Vantara Unlocks Data-Driven Innovation