Closing the Gap Between Software Engineering and Ops

The rapid software transformations that erupted in the 1980s and accelerated in the early 2000s with advances in software and multi-core processors, has reached yet another inflection point as big manufacturers tap into the power of cloud computing to digitally connect their business operations. 

The trend is a harbinger of myriad operational improvements to come, not the least of which will be the further integration of IT and OT. But before IT and OT can be drawn closer together, we need to drive more effective coordination between software engineering and operations.

Think back to how the application development world operated 10 to 15 years ago. It followed a defined process involving many different functions starting with requirements, architecture, and design all the way through to development, testing, deployment and support. Development organizations also set up separate functions to manage these different stages.

Today, the software world is a lot more agile. Instead of being marked as separate, discrete steps that predominated in the early 2000s – requirements, architecture, design, development, testing and deployment – have come together under the umbrella of DevOps. (The only outlier is support, which remains its own separate organization.) The genius of DevOps was to break down the barriers that had long existed between development and operations. No longer hemmed in by organizational silos, operations for development have been automated and integrated so that when a developer clicks a button and checks on their code, the result instantly appears in production. 

But before reaching the gates of the digital promised land, DevOps will need to reach beyond development into operations itself. In practice, that means approaching operations with an engineering lens - not as an afterthought or as a function that’s only expected to maintain code thrown over the wall to production.  

Changing the engineering mindset won’t be easy. Too often, engineering teams tend to focus on product development to the exclusion of all else. If we’re going to make progress, we must push code writers to also consider how their code is going to impact production. 

It also means treating operations as an integral part of engineering, applying site reliability engineering (SRE) principles so as to design reliability, cost and security into the code when it’s still in development.

Lastly, production support should become part of the engineering life cycle, where we don’t need separate, specialized support organizations created with their own processes, tools, and methodologies to maintain code. 

The Future Evolution of IT/OT Convergence

This tighter coordination between engineering and Ops can pay huge dividends in areas like transportation, energy, finance, or manufacturing. Take the example of an organization that operates a fleet of vehicles. It relies on constant inputs from IoT sensors to relay real-time information about the state of the battery or engine performance. The sensor data flows into a large database where applications constantly analyze it to spot anomalies or predict mechanical failures. But what happens when a microservice experiences an anomaly at the same time? In the current state, the resolution of these problems tie back to two separate processes and two separate factions both experienced on troubleshooting IT issues, but not OT.

In the future, solutions like our Hitachi Application Reliability Centers (HARC) can help navigate challenges like these. With HARC, you can bring together cross-function IT and OT expertise and use the algorithms that we have created to operate on either data in similar fashion to ensure reliability, regardless of whether its an IT or OT issue.

The upshot is that blending software engineering into operations and eventually IT and OT will provide organizations with better, timelier, and actionable information that will only grow more vital as they look to make better use of data.

Premkumar Balasubramanian Head of Technology and Solutions, Hitachi Vantara

Related

• Cloud Management for the Modern Workload
• Management Services for a Touchless Cloud
• Video: Insights by Hitachi Vantara: Explore Cloud