Practical Ways to Eliminate Alert Fatigue

Tips to avoid Alert Fatigue

In March 2014, US Retailer, Target revealed that its security software had detected its now infamous data breach five

months earlier, and that at least eight IT employees had seen the threat alert but decided not to act on it.  Some commentators jumped on the firm for its apparent incompetence, but security experts say its reaction was pretty normal.

So how and why do data breaches, equipment failures and disasters go undetected by humans when the monitoring systems are doing their jobs? The constant stream of alerts can cause engineers to check out, a syndrome that some refer to as ‘alert fatigue’.

Reacting to this influx of alerts uses your engineers’ time and resources, costs money, and can prevent your IT department from playing a more strategic role at your company.  This article will explore four actions that you can perform now to address alert fatigue.

Here are the four recommended actions.

Action One: Plan

You could think of a notification model in four levels of maturity, listed here from least to most mature:

  • Level 1 – reactive
  • Level 2 – tactical
  • Level 3 – integrated
  • Level 4 – strategic

IT is complicated enough.  Your IT tech people and engineers receive a stream of notifications that range from innocuous (someone has accessed an asset or logged into a system) to important but only to certain people (a project has achieved a milestone) to urgent (a server is down or security has been breached). Responding – or even reacting – every time a notification comes up can be time-consuming and irritating.

Do the work on the front end: Plan for alerts, escalations and automated processes for different scenarios to make sure your intelligent communications work well.  The system must have every stakeholder’s contact information, device preference, schedule and commitment to be available. You must build this in advance of an emergency.


Action Two: Automate

Suppose your business experiences a power outage.  A full-scale emergency will require a series of manual instructions and emails to the IT team, engineering and everyone whose business and safety may be affected.  However, you can still automate some important features, alerting first responders, letting purchasing know you need new servers, and even cutting off power to the server room.

What about more limited incidents, such as an employee laptop failure? Once the incident is recorded, the engineering tech replacing laptops receives an alert, a step that can be automated, and the employee can receive an automated notification that a fix is in progress. What if the employee reports the issue after hours?  Do you alert the tech on a mobile device, or can it wait until morning? If you plan your processes well, you can automate every step based on the urgency of the incident.

Time is critical, especially if you are servicing employees in global offices, as some employees are losing valuable work time. That could mean sales opportunities missed or incomplete timesheets.  Based on urgency, location, time, each person’s preferred device and work schedule, you can automate whether to alert engineers right away or wait until the morning. Depending on the rules in place, the message can be sent two ways: automatically triggered by the event, or at the push of a button, usually by the IT lead.


Action Three: Be Proactive

Another important function for efficiency is the enablement of easy status updates. IT techs frequently experience disruptions from answering queries on the status of an open ticket. Whilst it’s understandable that customers want to know the status of outstanding events, IT techs would rather be resolving issues than answering enquiries. Status updates send automatic messages to clients with expected time to resolution.

Proactive communications don’t have to be just for incidents. They can let employees know of impending software updates, let customers know of enhancements, or let an employee know a new laptop has been ordered and is on its way. The proactive alerts can ease the minds of the recipients, whilst freeing IT leaders from such enquiries.


Action Four: Target

A good way to enable your engineers to avoid alert fatigue is through targeted alerts, as alerts go to the subset of employees who need to know either to take action or to simply be in the know.  You should also target alerts by preferred device, so IT techs receive notifications where they’ll see them and respond. A good way of doing this is with subscriptions, enabling stakeholders to subscribe to relevant alerts and unsubscribe from others. When you combine automation, targeted alerts and subscriptions, you create more efficient alerting processes to help support IT Service Managers and IT departments.


With these recommended actions you should be able to drastically reduce the number of alerts received and help to restore some energy into your alerting.


Image Credit

How to manage third-party service, support and security

U.S. retailer Target saw one of the largest thefts of credit card data in recent history
U.S. retailer Target saw one of the largest thefts of credit card data in recent history

Productivity expert David Allen once stated that his approach, “Getting Things Done,” was based on the simple premise that you can’t do everything. In IT, we face this problem every day. Whether it is due to lack of domain specific expertise or simply not enough resources to handle all of our IT services, there are many reasons why we might look to third parties to help support our requirements.

Third party access can come in various guises – from full IT support and service operations, to specialist knowledge that is required on an irregular basis. The majority of this support is delivered remotely over the internet, making third-party outsourcers an even more cost-effective solution.

A research report by Ovum last year highlighted how many third parties have access to company IT networks. While 12% of organisations ran everything themselves, the majority of companies (56.3%) surveyed across Western Europe had granted access to between one and four suppliers, while 28.3% had between five and 29 suppliers. One company admitted that it had more than one hundred organisations with permission to access their networks.

Why does this matter? 

One word: Security.

Third party access is only going to grow, as more devices become internet-enabled and more specialist knowledge is required to keep them running. However, third party access is also one of the areas where control and management is often overlooked. There are plenty of options out there for remote access to networks, but the security and management of those tools is not as mature. Too often, access is binary and broad. The third-party either has access to the entire network, or it doesn’t.

This is a significant security risk, as witnessed by the attack on U.S. retailer Target last year, one of the largest thefts of credit card data in recent history. Poor third party access management opened the door for hackers to access the entire Target network via the vendor responsible for managing the firm’s air conditioning services. Once in, the attackers were able to use a variety of tricks to navigate from that section of the network and to the credit card database servers.

The current press attention around remote access security should drive better industry practices, but there are further proactive steps that service desks can take now to protect themselves.

Steps to take

For companies running their own service desks, security around third party access should be part of the overall request management process. When internal customers ask for new services or need help that a third party will provide, consider the management of the session as part of the request process.

This includes being able to control access. Why should a third party have access to everything on the network, when they are being asked to fix a specific problem? Locking down access – either to a specific section of the network, or only allowing the third party access to access certain devices or applications – is one option that service desks can look at in more detail. Service desks should also capture a full audit trail of every action a third-party technician takes while on their network, and set up alerts for any suspicious activity, such as a vendor logging in in the middle of the night.

For third-party service providers, keeping their customers’ networks secure should be top of mind. Just as the Doctor’s Hippocratic oath states, “Do No Harm”, so too should third-party providers reduce security risks to their customers around remote access. Implementing secure remote access tools and best practices will help service providers set themselves apart from competitors and improve customer loyalty.

Ultimately, third party access has to be secure, auditable and controlled. At the same time, the requirement for more flexibility in how services are delivered will make remote access by third parties even more common than it is today. Within the overall service delivery strategy, keeping this third party access under control is a key management task to consider.

Image Credit