There’s some great dialog in the final standoff between Batman and the Joker in the movie The Dark Knight. It’s no-rules anarchy versus incorruptibility – “this is what happens when an unstoppable force meets an immovable object”- as the Joker maniacally puts it.
In some ways it’s analogous to the friction existing between development and IT service management (ITSM) – especially how each group views DevOps. If you ask each team what DevOps means to them you’ll probably get two different answers. On the one hand, developers may stress freedom of action and faster releases, while on the other, ITSM practitioners might say DevOps changes nothing. After all, processes and controls painstakingly developed over many years is the ‘tough love’ needed to ensure regulatory compliance and address many other governance related issues.
Unstoppable force meets immovable object
Some ‘modernists’ will of course argue that old-style ITSM can be excluded from DevOps initiatives. After all, it’s a set of practices designed for a style of business computing where risk tolerance was low. So armed with new terms like lean, agile and fail-fast, it’s a case of get with the program or get out of the way. Well good luck with that, because without recalibration, those traditional incident, problem, change and release management contact points between development and ITSM will become even more abrasive. So enrolling the support of existing ITSM roles and practices is critical; turning naysayers and opponents into advocates. But this isn’t going to be easy and requires some deft organizational footwork. If everything remains equal nothing will change In order to remove friction, DevOps leaders should start by clearly communicating why it’s necessary to change. Care should be taken to avoid over hyping DevOps; preferring instead concise explanations as to why the change is occurring in the context of business impact and outcomes. During this early stage it’s also important to set a collaborative foundation; giving strong consideration to temporarily seconding key ITSM influencers to the DevOps program so as to build trust.
In many industries, computing controls, especially in areas such as change and release management, exist to ensure compliance with regulatory mandates. To development these appear cumbersome, but have been specifically designed to mitigate risk – even if that means slowing down processing. Furthermore, these controls deliver auditable proof that compliance procedures are being followed. The problem is that many of these controls might be too rigid to support development projects where risk tolerance is higher, so it’s critical for teams to optimize or right-size sets of controls for specific use cases. Here, care should be taken not to abrogate risk responsibilities by simply passing control ownership (for example, enabling development managers to approve changes but still carry all auditing responsibilities), since that might lead to increased friction and resistance to change – where you least want it – within the development group itself.
In terms of optimizing existing (but necessary) controls, this could involve enacting faster and more reliable ways to meet compliance requirements. For example, employing automated test suites during the actual development process – versus having auditing ‘gatekeepers’ come in at the end of the process and discover the system isn’t compliant.
In God we Trust – everyone else brings data!
Organizations have usually made a significant investment in IT service management tools. These tools, especially the knowledge bases supporting processes like incident, problem and change management can provide teams rich sets of information to drive DevOps improvements. Change records correlated with performance-related incidents and problems could help teams focus on non-functional aspects of development and testing requiring attention. Additionally, emergency change procedures could be reviewed to determine their applicability in supporting business-critical or urgent software updates. In all cases, however, teams should ensure flexibility doesn’t increase business risk – for example – by teams choosing the path of least resistance to avoid governance scrutiny. There are many other ITSM contact points teams can review to reduce friction. In incident management developers often complain that it takes too long for them to be notified of problems related to their code – only after lengthy level 1 and level 2 operations review. This causes friction because developers might be taken off projects to fire-fight problems that due to time delays have become more complex to diagnose and remediate.
To address this, teams should carefully review notification procedures; perhaps even changing the first point of escalation to be the development group responsible for the application or service – even after hours. Expect push-back where you least expect it. Developers may resist mandated on-call support. Therefore it’s important to impress how their early involvement in incident response is critical to drive improvements. It’s also a good idea to equip them with analytic tools and proactive methods that help them resolve complex and emerging issues. Finally, an important, but often understated bi-product of this ‘skin in the game’ approach is developers working to improve the ongoing supportability of applications. For example, it could result in improving documentation and fault logging so they only need to be called in when absolutely necessary.
Ignoring the points of friction between DevOps and older (but still important) ITSM processes will cause initiatives to stall or fail. The only way to ensure success is when teams put all governance and risk-versus-speed and agility concerns on the collective table and enact improvements in the context of required business outcomes. Always consider that without constant engagement, staff on both sides will revert to sub-optimal practices – the ones that stifle innovation or carry huge risk.
This article was contributed by Peter Waterhouse, Senior Strategist, CA Technologies