Following a sparkly pressie from the guys at TOPdesk, we got to thinking here at Enterprise Opinions towers about what should go in our emergency kit for dealing with Major Incidents.
To be fair, my Starbucks habit is slightly worrying but staying caffeinated helps me stay on the ball. When dealing with a crisis, sometimes you just need a second to figure out the next step. Taking a sip of your drink, be it coffee, green tea or water, takes you out of the situation momentarily and gives you a chance to clear your head and come up with a plan. That said, this is effectively me on a bad day:
Key Phone Numbers
Picture the scene, my second day at a Problem Management gig and a contracter accidentally hits the EPO button in the data centre. For the uninitiated, an EPO or Emergency Power Off button is something that instantly takes out the power to a room and is there as a safety measure in the event of a fire or someone suffering an electrical shock. They’re usually bright red and labeled EPO. Unfortunately in this case, it’s proximity to the door meant the chap in question mistook it for the door release button and pressed it taking out all services to the building and 8 major customers. As this wasn’t a DR test, there was no way to fail over cue an epic Major Incident and the Service Desk sat in shock because they had no working phones and no corporate address book to look up phone numbers. Not our finest hour. No one was saying anything so I did the only thing I could think of at the time; told everyone to use their mobiles to call everyone they had numbers for, starting from the top down until we were able to restore power, promising that I would personally pay their mobile bills if the finance department rejected the resulting phone bill. It was the only option we had at the time and luckily we were back up with the basics in about 30 minutes but my overriding memory of that day was feeling really out of control. Let’s face it, that level of faffery in a Major Incident is never good.
I love my iPhone. It’s pink (obvs) has every app I can think of and goes everywhere with me. It has unfortunately naff all battery life so I carry a charger with me at all times.
Feedback that I’ve had time and time again when I’ve done Incident Management type roles is how calm I seem when things are kicking off. I have no idea why people think I’m calm, I can promise you that it’s all a huge act. Inside my head, I’m having kittens or reciting every swear word I can think of or wishing I could hide under my desk but when I have a Service Desk full of analysts relying on me, I’m not going to let everyone down by panicking and then making mistakes. I guess you could say it’s a bit like parenting, as a mum of three I can tell you that kids can sense uncertainty, fear and in the case of my little darlings, chocolate buttons at twenty paces so the trick is to have a total air of “I’ve got this”. Fake it til you make it; act as though everything’s grand, you’ll calm down which will in turn calm down everyone around you and you can focus on getting everything fixed.
What would you have in your “break glass in case of emergency kit”? Let us know in the comments!
Following on from last week’s article about the advantages of Knowledge Management and how to get started, let’s look at the process in more detail. When I’m running ITIL foundation courses I generally hit Knowledge Management as part of the Service Transition stage of the lifecycle towards the end of day 2. Put yourselves in the shoes of the poor delegate for a second and think after 2 solid days learning about 20 odd processes and 4 functions even the brightest person in the room is starting to get a bit tired of all the terminology. To try and fix that; here’s my handy guide to Data Information Knowledge and Wisdom aka the Dick Whittington model for Knowledge Management.
First up we have Data. No, not the character from Star Trek TNG (although – spoiler alert – I’m still heartbroken by the ending of Nemesis) but the facts and figures which relay something specific. ITIL describes data as a discrete series of facts about events. When we talk about data; it’s raw in format, not organised in any way and providing no further information regarding patterns, structure or context. Data represents singular facts or numbers but by themselves, data items have little meaning.
The key Knowledge Management activities include:
Capturing accurate data
Reviewing data and adding context so that it can be transformed into information
Ensuring only relevant data that adds value is being captured as lets face it, anything else is just noise.
Data becomes Information when it can be viewed in a specific context. According to ITIL, for data to become information it must be contextualised, categorised, calculated and condensed. If data is a series of facts, information is generally stored in some sort of structure for example, e-mails, documents or spreadsheets.
The key Knowledge Management process around information is managing the content in a way that adds value. In other words, ensuing information is easy to capture, query, find, reuse and re learn from experiences so we don’t keep making the same mistakes and duplication is reduced.
For information to become knowledge it must be processed organised or structured in some way, or else as being applied or put into action. Knowledge combines information with experience and can be used as a basis for decision-making or taking an action. Knowledge is made up of the experiences, ideas, insights, values and judgements of your people. When we introducing formal Knowledge Management; creating the right culture is absolutely critical so that people feel comfortable adding to Knowledge Bases and articles ensuring the right knowledge is captured. Done well, Knowledge Management will engage and up skill your people so it really is worth focusing on.
Wisdom is the trickiest stage to explain. ITIL defines wisdom as being the ultimate discernment of the material and having the application and contextual awareness to provide a strong, common sense judgement. I’ve been in IT long enough to realise that you can’t teach common sense but by having the right training and support in place goes a long way to avoid a herding cats situation.
My favourite way of explaining Wisdom to ITIL foundation delegates is this example from Irish legend Paul Howard (author of the Ross O’Carroll Kelly books)
Knowledge is knowing that a tomato is a fruit. Wisdom is knowing that if you put it in the Nutribullet with vodka, it's a hangover cure.
As most of our regular readers will know I’ll jump on any opportunity to get my ITIL geek on so when I heard about the new ITIL Practitioner qualification, I was all ears. For those of you new to the ITIL qualification scheme, or if you’re simply old school like me (I was a V2 Manager / red badge) here are the basics.
ITIL Qualification Scheme Explained
There are five levels:
The foundation exam is your basic guide to ITIL and Service Management. It’s a three day course with an exam at the end which will give you the basics in ITIL. The foundation is a prerequisite for any other ITIL qualification and is worth 2 credits.
The practitioner qualification is new and aims to bridge the gap between the foundation and intermediate qualifications. As a former consultant and trainer, one of the things that stood out when we moved to version 3 in 2007 was the jump from foundation to intermediate levels. When you’re only got 3 days to explain 20+ processes and 4 functions at foundation level; jumping straight into the intermediate levels came as a shock to the system for some delegates. ITIL practitioner aims to build on the knowledge gained at foundation level and add in practical guidance on CSI, organisational change management, communication and measurement & metrics; giving delegates a more rounded experience as they continue through the qualification scheme. The practitioner qualification is worth 3 credits.
The intermediate levels are the next stage in the certification scheme and can either be life cycle based (Strategy, Design, Transition, Operation and CSI) or capability based (Operational Support & Analysis, Planning, Protection & Optimisation, Release, Control & Validation and Service Offerings & Agreement). The lifecycle intermediate exams are worth 3 credits and the capability exams are worth 4 credits. The Managing Across Lifecycle (MALC) module sits between the Intermediate and Expert levels and is the final required qualification gaining Expert status. It is intended to help you apply and integrate your knowledge of ITIL in real-world settings and in your own workplace. MALC is worth 5 credits.
The expert level aka the purple badge, is awarded automatically when you reach 22 credits made up of the foundation and MALC qualifications and then any combination of the intermediate and practitioner qualifications.
The master level is the final stage in the ITIL certification scheme and validates the candidate’s ability to apply the principles, methods and techniques from ITIL in the workplace.
Focusing On: ITIL Practitioner
So back to the practitioner qualification – why do I think it’s so important? As someone that’s worked in ITSM forever, one of the main reservations I had about the ITIL v3 qualification scheme was that it was too focused on the perfect exam answer (and a multiple choice exam answer at that) rather than what works in real life. One of the fundamental principles of ITIL is that it’s a framework. It’s not prescriptive and you can flex the approach to suit your organisation and your people. When the version 3 certification was launched, one of my concerns was that the intermediate courses simply didn’t give delegates enough real life practical experience (even with MALC) when you compared them to the version 2 manager course. To me, the inclusion of the new practitioner level will change that, as it will give delegates an extra level of understanding of IT Service Management as well as building on their foundation knowledge with the nine guiding principles:
Focus on value
Design for experience
Start where you area
Keep it simple
Let’s face it, anything that empowers delegates and gives them more practical, real life experience can only be a good thing! I’m so excited about the new course, I’m going back to the classroom at my old stomping ground Pink Elephant to do the course in May. I used to work for Pink so I’m really excited to see their sparkly new take on the practitioner qualification. Here’s their official summary of the practitioner qualification:
‘The qualification aims to demonstrate that IT Service Management (ITSM) professionals are equipped with the skills to apply ITIL concepts in their organisation, ensuring maximum business value by delivering fit-for-purpose and fit-for-use services. At the same time, it’s designed to give confidence to managers that the members of their team are ready to initiate and successfully carry out required improvement initiatives.’
I’ll be reviewing the course and exam experience during their upcoming course in May. If you’d like to find out more or register, you can do so here.
What do you think? Will you be doing the practitioner course? Let us know in the comments!
There’s been a lot of buzz recently about IT4IT; posts on the Back2ITSM forum on Facebook, questions on our discussion boards and it was one of the main topics of conversation at the speakers dinner ahead of last month’s Service Manager Day event in Holland. Let’s get our collective ITSM geek on and take a look at all things IT4IT!
According to the Open Group, the IT4IT Reference Architecture prescribes holistic management of the business of IT with continuous insight and control, enabling Boundaryless Information Flow™ across the entire IT Value Chain.
The Open Group IT4IT Reference Architecture standard comprises a reference architecture and a value chain-based operating model for managing the business of IT. It provides prescriptive guidance on how to design, procure and implement the functionality needed to run IT. The end-to-end, ‘how to’ emphasis of the IT Value Chain and IT4IT Reference Architecture also enables the state of services that IT delivers to be systematically tracked across the service lifecycle.
In other words, IT4IT is an operating model for IT to deliver value to the rest of the business. So far so reasonable, right? It is directly comparable to eTOM and BIAN models. The biggest question around IT4IT is “but doesn’t ITIL already do this?”. The answer is it depends. ITIL is a framework – it’s best practice and non prescriptive. IT4IT is an open standard, managed by The Open Group. It’s also free to use – what’s not to love right?
Why do I need it?
Lets face it; in IT we’re great at being reactive, fixing things, resolving Incidents within SLA, your basic superheroes saving the world. What we’re not so great is the long term stuff. Strategy, forward thinking and effective planning. Enter IT4IT which is aimed at helping companies’ IT departments address the strategic challenges brought about by the changing IT landscape. The aim of IT4IT is to support management and execution across the IT value chain and will allow enterprise CIOs to deliver services faster and with reduced cost and risk.
If that doesn’t get your attention then how about this? A recent Gartner study found that for an IT department’s budget of $1bn per annum, the initiative could save between 5% and 20% of the total.
The IT Value Chain
The IT Value Chain has four value streams supported by a reference architecture to drive efficiency and agility. The four value streams are:
Strategy to Portfolio
Requirement to Deploy
Request to Fulfill
Detect to Correct
Strategy to Portfolio ensures that you have the right strategy in place to balance and promote your Service Portfolio. It ensures that there is a consistent view across the Project Management Office (PMO), Enterprise Architecture and IT Service Management and that the right data is available at the right time to support decision making. Strategy to Portfolio is the part of the architecture that ensures that the appropriate KPIs are in place to improve business communication. This ties in to Agile – what epics do you want to work to? Where ITIL can be very analytical about future services, IT4IT uses the Lean startup model – let’s test a hypothesis and prove it.
Requirement to Deploy is the part of the framework that defines, builds, tests and deploys the right service at the right time for the right cost. Requirement to Deploy supports both traditional and Agile deployment methods and ensures that both continuous integration and continuous delivery control points are in place.
The Request to Fulfill value stream manages the transition to a service broker model using an offer catalogue to manage subscriptions and route fulfillment. Request to Fulfill helps your IT organisation:
Transition to a service broker model
Present a single catalogue with items from multiple supplier catalogues
Efficiently manage subscriptions and manage total cost of service
Manage and measure fulfilments across multiple suppliers
It contains very robust requirements for service catalogue creation and is scalable all the way up to the largest cloud providers.
Detect to Correct is the part of the value stream that brings together IT operations and effectively detects and corrects issues before business users are adversely impacted. It uses a shared configuration model to improve service visibility and reduce the mean time to repair.
Benefits of using IT4IT
If you’re struggling with managing your IT strategy then IT4IT could help you with the following:
Optimised cost of services
Improved service availability
Reduced Incident resolution times
More efficient usage of technology
Reduced transactional friction
More effective and Agile results
Increased customer satisfaction
Where can I go to find out more?
There’s loads of useful information out there; here are our top picks for learning more about IT4IT
One of the ITIL processes that tends to be glossed over is Knowledge Management which is a shame because it’s the process that can empower your people the most. Used effectively, Knowledge Management can empower your people, reduce Incident resolution times and increase customer satisfaction.
So what is Knowledge Management?
Knowledge Management is the process responsible for sharing perspectives, ideas, experience and information, and for ensuring that these are available in the right place and at the right time. The Knowledge Management process enables informed decisions, and improves efficiency by reducing the need to rediscover knowledge.
In other words, Knowledge Management is the process that takes all the information rattling around in our heads and puts it into a database / management system where it can be captured, shared and backed up.
What are the benefits? Too many to count!
How about increased engagement and staff retention? Take it from someone who knows staff attrition can be a nightmare especially in a Service Desk environment. Anything that can be done to improve morale and self esteem will increase engagement and help with staff retention. This can be in the form of training, mentoring or shift left.
Improved first time fix rates and improved Incident resolution times. If your people have the right skills, they will be able to resolve Incidents more quickly reducing call waiting times, improving up time and increasing ability to meet agreed service levels.
Less failed Changes. If service information is captured correctly the Change can be impact assessed more accurately and your team will be less likely to miss things. You know those wash up meetings where a Change has broken something because of a really daft reason? The meetings where the Incident and Change Managers are trying to write to the business explaining said daft reason? It happens all the time. Off the top of my head I can remember a critical trade floor application being out of service for 8 hours because a time change wasn’t done correctly and the time the transactional website of a large retail back was down for over 2 hours because we forgot to restart a database as part of a planned Release. Both really daft things that caught us out because when we went back to look at our processes, they weren’t documented properly.
Easier to find the right information at the right time – no more faffing about trying to find that key how to guide – it will be saved and linked to in one central location. This is particularly important in big organisations where things can get lost or misplaced within massive intranets.
No more reinventing the wheel. Having a Knowledge Management process means that reusing ideas, processes and experience is so much easier – making our processes repeatable and accurate using models and templates.
Getting the message out – again it’s about getting it right first time – the right information to the right audience.The Knowledge Base can be a great way to communicate with our customers; think about it – if it’s the go to place for everyone to check in use it to communicate new services or maintenance windows.
Promoting accurate, repeatable processes procedures and work instructions – a standard way of working that stands up to audit and external review. If everything is templated in a central location with an agreed review process, your processes and procedures will be accurate, useful and have a consistent look and feel.
Making niche or specialised knowledge more widely available. When I worked in second line support for a tech company 15 years ago, I was fascinated with Lotus Notes. It used to really bug me that every time we had a user call in with a Notes issue we had to sanity check and then bounce the call to third line support, so one day I went to see the e-mail team and asked them if there were some basic things they could teach me. There may have been some beer related bribery involved but I got some solid experience supporting the application and was able to share with the rest of the team. Also – when the Notes guys were looking for guinea pigs to try out a new instant messaging product – guess who was first in line? Everyone was a winner!
Empowering your customers and taking their experience to the next level – Self service and self help. We live in the world of Google, Amazon and Facebook; no one wants to spend 10 – 15 minutes on the phone to the Service Desk if it’s something they can take care of themselves so let’s start building this into our Service Desk functions!
Struggling to know what level of empowerment to aim for?This is how I’d love every business customer to feel when using IT services.
Speed of delivery– you can react quicker if you’re lean, streamlined and organised. If you have effective Knowledge Management in place you can be quicker to market – no more faffing about for a key document. RFP response template or spreadsheet – they’re all stored in one place.
Continual Service Improvement or CSI – improve improve improve. Knowledge Management drives CSI – gathering Data and processing Wisdom (more about this soon) enables us to focus on the most business critical areas to improve. Keep getting better. As the saying goes- knowledge is power so use it to ensure quality is inherent in everything you do.
What’s not to love?
How do I get started?
Let’s start with the basics. I know ITIL suggests the SKMS but being realistic – not everyone can afford a tool which has been hyped up to be effectively Google. Also – there is no one size fits all; the Knowledge Management requirements of a global investment bank regulated to the hilt will be completely different to those of a tech startup made up of thirty people so flex your approach accordingly.
If you don’t have a tool can you start capturing the basics on a network share or simple SharePoint form?
Have a chat with the Service Desk. Chances are they’re already doing some sort of Knowledge Management even if it’s pretty informal. Look at the shift left principle; empowering those the next tech level down from you to drive efficiency. If you work on the Service Desk, invite the second line support guys to your team meetings once a month to give you trouble shooting tips. The first line support guys get to add to their skillset and second line support are freed up to concentrate on the more complex issues.
The main thing? Do something. Seriously – it’s that simple.
Anything you do will be better than not having anything in place. Start with the system that you know you’re on dodgy ground with support wise. Think about it – there’s always some quirky legacy system that depends on the expertise of one or two people. Having support rockstars is all well and good but what if they get sick or win the lotto and decide to relocate to Disneyworld? Ask them what their top ten support tips are and stick them in your Knowledge Base – even if it’s just a spreadsheet or word document. You’ve made a start in capturing key information about a difficult to support system so I’m calling that a win. At least it’s a start right? And that’s the thing – once you’ve made a start with Knowledge Management you can build on it over time until you’ve got a process that supports and empowers your people.
Up next I’ll be talking about the Knowledge Management process and the DIKW model so stay tuned. What are your thoughts on getting started with Knowledge Management? Let us know in the comments!
Following on from our previous articles on Configuration Management, this week we’ll be taking a look at Status Accounting and Verification.
Status Accounting is the part of the process responsible for recording and reporting the lifecycle of each configuration item; essentially making sure that each CI has a valid lifecycle status, accurately recorded in the CMS. The thing to remember about status accounting is that you can have all the statuses (statusi?) in the world but if you have too many, it will be a nightmare to maintain your data. Again, my advise would be to start small and then build up over time as your process matures. Some example Configuration Management statuses could include:
In Pre production
Status Accounting ensures that all CIs that make up the service baseline or snapshot have been captured and that all Changes have been captured by Change Management and reflected in the CMS. I would also make the case that if a Change has failed or a planned Release CI has defects, then a Known Error (with a workaround if appropriate) should be raised and linked within the CMS.
Automated discovery tools make it easier to manage your estate; they can do in hours what it would take people days or even weeks to complete. If you don’t have a dedicated asset discovery tool could you use an existing tool such as SMS or Alteris (commonly used for deploying software releases)?
Verification & Audit
Our final part of the Config process is to look at the activities responsible for ensuring that information in the CMS is accurate and that all configuration items have been identified and recorded.
Verification includes routine checks that are part of other processes – for example, verifying the serial number of a desktop PC when a user logs an Incident or checking that the version of software updated in a planned Change has been added to the CMS.
Audit is a periodic, formal check. Configuration audits generally include the following activities:
Defining audit schedule and procedures
Identifying who will perform the audits
Performing audits on the established baselines
Generating audit reports
When defining an audit schedule look to the rest of the business for guidance. Do you have any regulatory requirements such as SOX, BASEL 3, IL3 or NGN224 or any standard s such as ISO 20000 that need to be adhered to? If so, they will probably come with a defined audit cycle. When preparing for external audits, the best thing you can do is run an internal audit first so that you can correct any potential issues or at least come up with a plan to improve in the case of any major findings. Ideally, get someone from outside your department to carry out the audit as they will have a fresh perspective and there will be no room for bias (however unintentional).
Configuration audits will include a mixture of automated and physical checks on established service baselines. Once complete, the audit report will be generated summarising the results of the audit and highlighting any differences between the CMS and production CIs. In the event of discrepancies, a get well plan should be planned and acted on as soon as possible, perhaps with the support of your Problem Management team to understand the root cause and suggest actions to prevent further discrepancies. I would also suggest keeping a lessons learned log for Configuration Management and ensuring that it is updated and acted upon following all audit activity.
That’s our take on Status Accounting and Verification & Audits – what do you think? Let us know in the comments!
Overview Of Service Asset & Configuration Management
Service Asset & Configuration Management (SACM) is the process responsible for ensuring that the assets required to deliver services are properly controlled, and that accurate and reliable information about those assets is available when and where it is needed. This information includes details of how the assets have been configured and the relationships between assets.
Done correctly, Configuration Management is a key enabler for resolving Incidents, identifying the root cause of Problems, impact assessing Changes and building the technical layer of a Service Catalogue. It’s also the process that tends to make people panic because of all the databases, information systems and scary terminology; so here is our panic free guide to Configuration Management. Configuration Management is made up of the following steps and we will look at each one in turn:
Let’s start with the basics. Configuration Management is one of those processes where you really, really need to have a plan. I’m not talking any old plan either; I’m talking Hannibal from the A Team level planning. Why is planning so important? One of the biggest reasons Configuration Management fails is because people try to do too much so setting the process scope is a key activity.
Let’s start with the inventory layer. Inventory Management takes care of consumables; keyboard, mice, USB sticks and SecureID cards. Stuff that needs to be tracked for monetary value and to make sure it’s in stock but that’s about it. The next layer is Asset Management. Examples of assets include PC’s, Laptops and printers. Stuff that we need to keep track of for monetary reasons, to make sure they’re in stock when needed, locational info and how they are supported. The final layer is Configuration Management. Items under the control of Configuration Management (CIs) are the big chunky items that make up your critical services. Servers, network devices and software applications should all be under the control of Configuration Management to make sure they are managed, supported and subject to the appropriate Change control.
The plan should also explain any naming conventions or nomenclature. Confused? Every CI or item that is under the control of Configuration Management should have a unique name or identifier. When I’m tasked with implementing a Configuration Management process from scratch I try to use a naming model that will make it easy for the CI to be identified so I tend to use the following:
Type of Service – Location – Level of Complexity
So a WinTel server based in Dublin requiring third line support would be WinTel1234 – DUB – L3 and a business spreadsheet located in London requiring first line support would be Exl-LON-L1 and so on. In terms of naming conventions; it doesn’t matter what framework you use, as long as everything has a unique identifier. When I worked for an investment bank in London, the naming convention was food based so all WinTel servers were named after Italian food, all UNIX servers were named after Chinese food. Another example is from a firm that I worked for in Reading; all WinTel servers were named after characters from Terry Pratchett books and all Network devices were named after monsters from ancient Greek mythology. Both worked really well and as an Incident Manager it was pretty hard to get stressed during a Major Incident when someone shouted Rincewind’s fallen over again!
The plan should include a reference section. When you’re building a CMDB you will be talking to support teams, service architects and project managers. Ensure that you capture the source of the information whether it be from a Service Catalogue, support documentation or even from SLAs so that it can be verified before it’s placed in your CMDB.
Does your organisation have a Configuration plan? Let us know in the comments!
Enterprise Service Management Webinar Module 3: CONFIGURATION MANAGEMENT
Following on from our previous posts; here is our final article in the series of how to do Release Management in real life.
Distribution & Installation
Unfortunately actually deploying a Release is not as easy as this:
Releases should be deployed to the live environment in accordance with the existing Release Management policy. For software releases it’s a good idea to use automation where possible as it will lessen the impact on support teams, decrease the time of the release and reduce the likelihood of human error.
If the Release has to be deployed without automation, then the release implementation plan should contain detailed instructions for deployment, resources, timings, escalations and contact details. An example release plan could include the following details:
Example Release Timeline: Commercial Website Deployment
Pre-requisites that must be carried out the day before the release:
(1)Development must provide the release notes to the Release Manager and all resources on the timeline by 17:00 the day before the release.
(2)Development must provide the Release Manager with business sign off by 17:00 the day before the release.
(3)Release Manager must raise a corresponding change record for the release and ensure that it has been approved at the CAB.
Reference Number 1234 – Commercial Website Release – Date
Divert all website traffic to the DR web servers.
**Communication checkpoint to Windows Team ***
Joe Bloggs – Network Team
Commercial Web amendments for Production web servers
Jane Doe -Windows Team
Testing & Validation
**Communication checkpoint to Network resource**
A. N. Other -Web Team
Divert all traffic to the Production web servers & Stop DR web server traffic.
**Communication checkpoint to Windows Team.
Joe Bloggs – Network Team
Commercial Web amendments for DR web servers
Jane Doe -Windows Team
Testing & Validation
**Communication checkpoint to Network resource**
A. N. Other -Web Team
Normalise network traffic to all web servers
Joe Bloggs – Network Team
**Communication to Release & Change Management about the release status **
A. N. Other -Web Team
Escalation Point: IT Services Manager, IT Services Senior Manager
Early Life Support
It’s really important to make sure all Releases have the appropriate support in place immediately after implementation. In the words of Rob England at #PINK16 “we need to avoid dead cat syndrome; aka as the Dev guys chucking something over the fence into production and expecting the Ops guys to make it work seamlessly”.
It may be useful to introduce an “intensive care period” whereby extra support resources are available for a period after the Release to ensure that resources are in place to troubleshoot any issues immediately. Floor walkers made up of Service Desk and Support Staff could be used to support users on the morning of the Release. This intensive care period could be tracked via a short daily meeting or conference call and attendees should include:
Service Desk Management
Again, this isn’t about red tape, it’s about making sure everything is as it should be and that any issues are caught early and zapped rather than be allowed to spiral out of control.
A Warranty period could be built into the Release whereby the new functionality is supported by the development team until 2 weeks after the Release has been deployed, providing there are no outstanding Major Incidents or Problems associated with the Release. The Release Management policy should include provision for warranty periods and guidelines for transition into BAU activities.
Review & Close
A post implementation review should be held after each release to track any outstanding actions and to document any lessons learned. Inputs to a post implementation review could include:
Incidents associated with the release
Problems and Known Errors associated with the release
Issues log (if using PRINCE2 as a project methodology to manage the release)
Change and Release Management feedback
Customer Satisfaction Ratings
Customer complaints and compliments
Outputs from the post implementation review will include:
Lessons learned log
Known Errors and workarounds
Confirmation that the CMDB / CMS / ancient spreadsheet that everyone acknowledges as the single source of the truth has been updated appropriately
Service improvement suggestions
Breaches to SLAs / OLAs / Underpinning Contracts
Required amendments to SLAs / OLAs / Underpinning Contracts
Updated work instructions
Keeping a lessons learned log to build on previous Releases and to keep a documented audit trail of all learnings, good and bad. The lessons learned log should be reviewed regularly; at least on a quarterly basis and before the implementation of major Releases to ensure past mistakes do not recur (because let’s face it, if we don’t learn from our mistakes thats just embarrassing). A sample lessons learned log could look like the following:
Example of a Lessons Learned Log
RFC – 1234
Windows Patching – Office X
Critical servers unavailable in Office X due to patching failure.
Though testing of all Windows server patches prior to deployment into the production environment.
Revise the process as any issues arise or build any more significant improvements into a Service Improvement Plan (SIP).
That’s our take on Release Management; what do you think? Let us know in the comments!
Following on from Part 1 of our article on how to do Release Management – here are some tips on Release acceptance, rollout planning and communication & training.
Any development code that results in a change to the live environment should be under the control of Change Management and be managed by the Release Management process. If you are finding that a high number of Release RFC’s are being rejected at CAB meetings then you need to investigate the reason for this. All rejected Release RFC’s should be tracked and reported on by Change Management.
The Release should be tested in a controlled test environment with known hardware and software configurations. The current list of release acceptance criteria should be reviewed and updated if appropriate. Release acceptance criteria will differ widely for different organisations as different companies will have different requirements. Your list of release acceptance criteria could include the following:
Has the release performed as expected across all development and test environments?
Has the back out plan been tested successfully
Are the release deployment instructions correct?
Has all appropriate support documentation been updated?
Has a training schedule been created / have all relevant teams received adequate training?
Has the release undergone extensive User Acceptance Testing (UAT) with involvement from all impacted customer / user groups?
Is the release performing as expected in test environments and free from defects?
If testing has identified any defects with the release is it possible to deploy with a workaround or should the release be postponed?
Have all impacted support teams received training in any new support functionality resulting from the release?
I like to borrow the Quality Gate concept from Six Sigma to make Release acceptance as effective as possible.Put simply, quality gates are a way of enabling your Release to move through the process quickly and safely making sure that all the quality criteria have been met. Quality Gates are not, I repeat NOT, road blocks or red tape; if anything they speed up the process (some checks can be automated) and cycle time is reduced because we’re getting it right the first time. Quality Gates are a set of predefined quality criteria that a software development project must meet in order to proceed from one stage of its lifecycle to the next and ensure. Quality Gates ensure that formal checklists are used throughout the life of a project so nothing can be lost, missed or ignored and that formal sign-off and acceptance occurs at each gate ie no nasty surprises.
Some examples of quality gates include:
Code review: The senior developer will look at acceptable coding techniques and adherence to IT standards and best practices. The software design will be verified against the coding to identify any coding errors. Upon successful completion of the code review, the reviewing party will highlight required changes and corrections to the Release Manager so the appropriate action can be taken..
Environment review; the test manager will look at the environments used for testing the Release to ensure they are fit for purpose, managed effectively and are being refreshed at pre agreed intervals.
Ops review: The project manager will work with the support teams to review all support tasks needed to support the development environment and ensure all appropriate work instructions are in place..
Sponsor review: The project manager will review the overall project performance with the project / release sponsor. This review will determine the status of project costs and schedule.
Test review: The test lead and representatives from the Quality team will attend the test review to see if all builds were tested correctly and make sure the appropriate test scripts and procedures were followed.
Deployment review: The Release Manager sits down with the relevant dev and support teams to run through the implementation plan and to ensure everyone is comfortable.
Defect review; The Release Manager meets with business representatives, the Service Desk and the Project Sponsor to make a decision on if in the event of any defects; the Release is delayed or installed with Known Errors and workarounds.
There are lots of ways to deploy a Release safely into your environment. There is no one size fits all, it depends on the size and complexity of your organisation as well as appetite for risk. In the immortal words of Optimus Prime: “Autobots roll out!”
Phased / Pilot approach
Review the Release plan to include the exact details of the Release and how it will be executed. The release approach should also be considered to ensure it is appropriate of the type of release; different approaches such as “Big Bang”, phased / pilot, or parallel approaches can all be useful for different types of releases. A big bang release is whereby the release is deployed to all recipients at the same time. Advantages and disadvantages of the big bang approach are summarised in the following table:
Advantages and disadvantages of the Big Bang deployment method:
Release is deployed to all users at the same time
More risky than other deployment methods because if the Release fails or causes a Major Incident the Release must be backed out for all users
Training is only required for the new system and not for running both the old and new systems in parallel
Training must be scheduled for all stakeholders prior to the release adding additional pressure to key release personnel
There is one deployment date which has been communicated to all stakeholders preventing any confusion around release scheduling
As there is only one deployment date, any delay to the release may cause adverse impact to certain departments
Phased or pilot releases can be used to introduce new functionality to the end user base in a scheduled approach ensuring that the release has been successful at each stage before moving on to the next. Advantages and disadvantages of the phased / pilot approach are summarised in the following:
Advantages and disadvantages of the Phased / Pilot deployment method:
Less risky than “big bang” deployment as the release is deployed to a set group of users at a time, thereby if the release fails it is backed out from one or a small number of user groups rather than the whole organisation.
Release implementation will take longer as it will be deployed in a staged manner rather all at once.
May enhance the relationship between IT and the selected pilot groups as the two will work very close together if a pilot approach is used.
Support for the release could be more expensive due to longer implementation windows eg needing contractors / consultants for longer
A parallel approach can be used to reduce risk for business critical releases. A parallel release works by having both the old and new system run simultaneously for some period of time after which, if the criteria for the new system are met, the old system is disabled. Advantages and disadvantages of the parallel approach are summarised in the following table:
Advantages and disadvantages of the Paralleldeployment method:
Less risky than “big bang” deployment as the release as the original configuration is still available to users
Expensive as it involves running two versions of a system in parallel for a length of time requiring appropriate support personnel, licensing costs and system capacity.
Less risky than phased / pilot releases as you have an instant roll back in that your original service is still available
Risk of confusion to the user base as both systems are available at the same time
Additional training may be required for running both the old and new systems in parallel
Push deployments are used where the service component is deployed from the centre and pushed out to target locations.
Advantages and disadvantages of the Push deployment method:
IT are in control of when the Release is deployed and to which user groups
End users could be inconvenienced if the update is pushed during an important task
Can be automated or built into a Microsoft group policy
The network could experience performance issues if too big an update is pushed out
Ideal for critical security patches or antivirus updates
A Pull deployment approach is used for software releases where the software is made available in a central location but users are free to pull the software down to their own location at a time of their choosing. As some users will never pull a release it may be appropriate to allow a pull within a specified time limit and if this is exceeded a push will be forced, e.g. for an antivirus update.
Advantages and disadvantages of the Pulldeployment method:
Users can schedule updates at a time that best suits them
Some users will never “get round” to installing the software so a combined Push / Pull approach should be considered eg users can pull at their convenience but If this hasn’t been done in x number of days the software is push out to the CI
IT doesn’t become a bottleneck; clients contact the server independently of each other, so the system as a whole is more scalable than a ‘push’ system
Scalability can become a bottleneck; unless you deploy several master servers and keep them in sync, that one master will start getting swamped as you add more and more clients and thus will become your bottleneck
Follows the self service model – end users feel empowered
Communication & Training
It’s important to ensure that an appropriate level of governance is in place to support your Release Management is the introduction of governance. Setting up governance around a Release Management process will ensure the releases are implemented to a higher quality; it’s not just a case of channeling your inner Mr Burns (although that would be pretty cool).
A Release Board should be set up to control the formulation and implementation of Release strategy and in this way ensure the fusion of business and IT. The Release Board will also be responsible for managing risk, testing and formal senior management sign off in addition to the CAB approval discussed in the previous section. The Release Board will often make the final Go / No Go decision on whether or not the release can go ahead if a last minute defect is found.
The Release Board should meet frequently and should be made up of some of the following:
Project Office / Manager
IT Senior Management
Business Senior Management
Governance paths should be set up to underpin the Release Management process and establish guidelines, an example of which could be no first of type releases can be deployed by a big bang deployment and all impacted department should have additional floor walker support.
Regular release check point meetings should be held so that all stakeholders of the release are aware of the implementation details. Examples of meetings include:
Pre release implementation checkpoint meeting
Post implementation meeting
Monthly process review meetings with stakeholders in the Release process to review the Release schedule, any issues, SIP, etc.
A release / service readiness review should be carried out to ensure the production environment is ready for the release. Such a review could be carried out in conjunction with Capacity Management to ensure that all capacity issues are tracked and addressed in the Capacity Plan.
The release implementation plan, back out plan and any associated technical documentation should be distributed to all stakeholders well in advance of the go live date to prevent any confusion. A communication to end users / customers should be issued via the Service Desk if a Release will have a noticeable effect on end users; it is useful to build up a bank of template Release e-mails so Releases are communicated in a standardised way.
Regular meetings should be held with Problem Management so that any Known Errors can be distributed to impacted stakeholders prior to the Release going live. There should also be a documented procedure for providing the Service Desk with a list of Known Errors before go live so that they are able to log incidents appropriately and relate them to the correct Known Error. Before any release is implemented, the Service Desk should receive training on any key changes to existing functionality and should be provided with “quick fixes” from the support teams to ensure that simple issues can be fixed by at the first point of contact where possible. New codes and templates for the release should also be set up in the Service Desk tool.
Join us for our second live ITSM webinar on Release Management on Thursday 25th February at 2:00PM GMT. You can watch live, or on demand by registering here.
That’s all for now, come back soon for our final article in the series where we’ll look at go live, early life support and review & close.
One of the questions I used to get asked all the time as a consultant was how to get started with Release Management. Most organisations start with Change Management and then as they mature; look to add additional governance and control with Release Management.
Here are some areas to focus on when looking at ways to formalise your Release Management process:
Design & Build
Communication and training
Distribution & installation
Early life support
Review & close
Release Management Policy
A solid policy is one of the most aspects of a good Release Management process. Put simply, your policy is a list of “thou shalts” and “thou shalt nots” regarding the Release process. No matter who the customer is; whenever I create a Release Management policy, I ensure the following three things are addressed:
Definition of a Release
Let’s start with the basics. ITIL terms a Release as “One or more changes to an IT service that are built, tested and deployed together. A single release may include changes to hardware, software, documentation, processes and other components.” In practice, every organisation will have a slightly different criteria for selecting the Release route. Some organisations have very defined release criteria, conversely, I’ve worked with organisations where anything touching the code of a transactional website was classed as a Release, everything else was a Change. Whatever your setup, I’d recommend a simple matrix that guides people as to which cycle to follow.
Scheduling needs to be addressed as part of the policy. How many releases do you need to have? Some organisations go for monthly or quarterly Release cycles; at the other end of the scale you have Amazon who deploy a new software release every 11.6 seconds. Make sure timescales are set in your policy and that it ties in with the related Change Management policy.
Appropriate levels of governance must be in place to support the Release Management process. The policy should set out what Releases can simply be approved at CAB and what Releases need a higher level of approval eg from a Project or a Release Board.
Make sure that the content and scheduling of each release is agreed early on; so regular meetings with both development teams and business representatives is a must. Make sure the Release schedule (documented in your policy) is combined with the Change Schedule. The easiest way to do this is to raise a Change for each Release and then link the information; that way it shows up on the schedule, CAB are aware of the timings and because the Change record contains a link to the Release documentation there’s no duplication of effort.
Effective Release planning means that downtime and inconvenience to the business is minimised as multiple Changes are packaged into one Release. This approach also saves money as avoiding multiple downtime windows means less overtime, external support call outs and paying out for service credits.
Design & Build
Carry out a review of your supporting players. Work with Configuration Management to ensure that the software in your Definitive Media Library (DML – or DSL; Definitive Software Library if you’re old school) and Hardware Store (HS) are consistent with the Configuration Management System (CMS). Wow – just reading back that sentence; that’s a lot of ITIL terminology – here’s a quick beginners guide if you’ve just read this and wanted to panic and / or cry:
DML: Definitive Media Library – one or more locations in which the definitive, authorised and licenced versions of all software Configuration Items (CIs) are stored. In practice; The DML is your application library or server; it’s there to make sure only authorised and safe software is installed across your company.
HS: Hardware Store – secure storage of definitive hardware spare components and assemblies. In practice this is your store of spare PCs and laptops for spares and “hot swaps”.
CMDB / CMS: A database used to store configuration records throughout their lifecycle. The configuration management system maintains one or more configuration management databases, and each database stores attributes of configuration items, and relationships with other configuration items. If that’s still making your head hurt, here’s a quick diagram to help explain:
Diagram 1: Scary Terminology Explained
Now that we’ve got the technology squared away; by checking that software from DML and hardware from DHS are consistent with CMS you may find unused, “spare” software licences and you may find hardware that can be used in production.
Look at the environments (if any) you have for testing Release content. If more are needed but money is tight could the cost be shared with other departments initially? A training environment could also double as a pre production environment. Tight management can reduce the need for multiple environments; someone (usually the Release or Test Manager) looks after who is using the environments using a “booking out” process and ensures that the environment is refreshed on a pre agreed regular basis.
Come back soon for Part 2 of this article; where I’ll give further tips on building a Release Management process.