open
  1 (866) 447-2526 Resources Events Blog

Overwhelmed with False Alarms (Part 5)

Blog

Overwhelmed with False Alarms (Part 5)


 

This article is part of a 5 part series covering some of ways to deal with the top challenges for IT Operations and how machine learning techniques can be applied to address them.

 

  • Having Trouble Finding The Root Cause
  • Stuck In Reactive Firefighting Mode?
  • Not Sure What is Important
  • Can not See the Forest for the Trees
  • Overwhelmed with false alarms?

Part 5 of a 5 part series

-  -  -  -

What can be done about all the false alarms? In many cases, lots of alarms are simply noise. They are repeated alerts, -alerts that are not interesting. They are alerts that are known not be interesting where nothing ever breaks and nobody cares about them. Ideally the system should take into account the operator’s feedback in the same way for how an email inbox deals with spam - once it is marked as spam the system learns and stops showing those emails. 

Reinforcement Learning

The reinforcement learning approach can be applied to address this issue. Reinforcement learning is built on the premise that there is an agent (the learning system) that introduces an action into an environment (the operator behind the computer screen). This action means to show or dismiss an alert.

When the system decides to show an alert it will look for the feedback from this environment. The operator can decide whether this was actually a good alert or that it was just complete noise. 

The learning system takes this feedback information and looks for two items:

  • The reward which could be positive or negative. This is basically the feedback given by the operator suppressing or escalating the alert
  • The state associated with this reward.

For instance, if the alert is noise, then the operator behind the computer indicates this as noise and it will be reported to the learning system as a negative reward (or penalty). In the state that shows a repeated alert, the system adapt its internal knowledge to realize that the user does not like to see the repeated alerts and decrease the probability of showing these alerts in the future. Operators train the system and actually indicate what is important to them and what they would like to be alerted on.

See Evolven in action!
Unlock the power of actual changes. Register now for a live demo.

About the Author
Bostjan Kaluza, PhD

Boštjan Kaluža is the Chief Data Scientist at Evolven. He's also a hardcore researcher who's done a lot of research into artificial intelligence and intelligent systems, machine learning, predictive analytics and anomaly detection. Prior to Evolven, Boštjan served as a senior researcher in the Department of Intelligent Systems at the Jozef Stefan Institute, the leading Slovenian scientific research institution and led research projects involving pattern and anomaly detection, machine learning and predictive analytics.

 

Focusing on the detection of suspicious behavior and data analysis, Boštjan has published numerous articles in professional journals and delivered conference papers. In 2013, Boštjan published his first book on data science, Instant Weka How-to, exploring how to leverage machine learning using Weka. Boštjan is now working on his second book Practical Machine Learning in Java, scheduled to be published later this year. Boštjan is also the author and contributor to a number of patents in the areas of anomaly detection and pattern recognition.

 

Boštjan earned his PhD at Jožef Stefan International Postgraduate School in Ljubljana, Slovenia, rigorously defending a doctoral dissertation entitled Detection of Anomalous and Suspicious Behavior Patterns.