Madhup Mishra
Director, DataOps Product Marketing, Hitachi Vantara
March 23, 2022
It’s one thing to talk about orchestrating and automating your organization’s data operations. It is quite another to gain the confidence that comes with having a unified view of your data. This just-in-time view of the truth simultaneously reduces data privacy risk and enables your business to pursue data-driven goals.
This article will look at the significant strides taken by organizations that are using the data integration and data cataloging capabilities of Hitachi Vantara ‘s Lumada Data Catalog to optimize the entirety of their data fabric across their data center, edge, and cloud.
A unified view is one that contextualizes all data across all repositories. This makes it possible for an organization to develop a complete, searchable, holistic view of all its data. Such a comprehensive perspective is fundamental to decision-making because it enables business owners to apply consistent terms and business meaning to data. This glossary creates a contextual landscape that makes insights coherent so that they are then accessible to decision-makers. Such uniformity also eliminates data silos, making it possible to arrive at consensus truth. Automation can then be added to update and maintain that truth, relying continually on the consistent language.
A regional commercial bank based in the northeast USA had large amounts of customer and reference data that constantly changed, making it quite difficult and costly to manage. The lack of consistency made it impossible to search, explore, and publish consolidated data (that single truth) to the company’s Snowflake data cloud. The result was a significant drag on the organization’s ability to deliver a functionally seamless and convenient experience across their customer-facing applications.
Using Hitachi Vantara Lumada Data Catalog, the bank automated its discovery process. The organization applied machine learning to handle data classification and feed a ubiquitous data catalog that unified all of the bank’s relevant data from across their hybrid cloud environment. Not only was this fast and more accurate than any previous method the organization had employed, but Webster Bank also lowered operating costs.
The bank also automated its data quality assessment process, using LDC to derive a readily understandable score for its various data. In addition, a comprehensive business glossary dramatically increased the accuracy, consistency, and completeness of the organization’s data. Together, these additions make it possible for business users to readily consume data, secure in the knowledge that the data has been assessed for reliability. The results are applications that provide better customer experiences and increase the bank’s competitive advantage.
A 2020 study conducted by Ventana Research found that “Through 2023, three-quarters of Chief Data Officers’ primary concerns will be governing the privacy and security of their organization’s data.” No wonder. Companies have abundant data but often do not know what private information is hidden within those resources. It turns out that data quality is about more than accuracy. Today, data quality is closely aligned with an organization’s ability to ingest, curate, and publish data consistently and reliably.
That can be difficult to achieve. Given an ever-increasing variety of data types, the speed with which data moves through companies, complicated data architectures, as well as an ever-changing regulatory landscape, it is no wonder that company’s struggle with privacy. Many companies lack the essential tools to know what to secure or even how to tag their data. They too often rely on manual solutions that are slow, delay access to data, and cannot scale. Companies in this scenario can easily fall out of compliance with regulations such as GDPR. Moreover, they are at substantially increased risk for reputational damage if a security breach occurs or their sloppy data practices are exposed.
Data privacy should not slow down a business. That is why a data catalog is invaluable. A data catalog ensures that a company knows where data is, how data should be protected, and who should/should not have access to it. A data catalog effectively empowers privacy and compliance staffers, giving them the tools they need to meet operational compliance requirements faster and with greater certainty and completeness. When combined, effective data integration, automated data discovery, and cataloging are a significant part of the answer to data privacy and compliance challenges.
Lumada Data Catalog uses an AI-driven data catalog to identify sensitive information automatically that needs protection. The platform can tag all the sensitive information so that problems can be fixed immediately. Relying on spreadsheets simply won’t work any longer for meeting modern data security requirements. Companies need full governance policies, with governance rules that can be enforced by AI modeling.
A leader in energy services and solutions in Europe wanted a more efficient and effective way to enforce privacy and regulatory compliance across its hundreds of data sources and formats. Accounting and billing accuracy was a particularly critical requirement from a regulatory reporting standpoint.. The problem was exacerbated because, in addition to the company’s current customer base, the company also had to retain data about millions of prior customers.
Using Lumada Data Catalog, the organization was able to automate the data discovery process. Applying machine learning to data classification significantly reduced the organization’s reliance on time-consuming and error-prone manual profiling. In addition, this automation supports data classification at a far greater scale, providing a more complete and reliable solution for enforcing privacy rules across all their data sources. Moreover, the energy provider can now maintain updates continuously without significantly increasing costs. As a result, the application of data controls satisfied the regulatory and privacy requirements for 10+ million energy customers overnight.
Pulling together a unified view of your data resources and then tuning data operations is the foundation that enables business users to find meaning from data. By taking these first steps today, your organization is positioning itself to quickly and effectively meet the requirements of a data-driven business initiative, no matter how diverse your data sources become.
Get started with the Lumada Data Catalog today.
Related News
Be sure to check out Insights for perspectives on the data-driven world.
Madhup drives product marketing for Hitachi Vantara's DataOps portfolio business. He has +20 years' enterprise software experience, and covers a range of topics including data operations, big data analytics, data governance and Internet of Things (IoT).
