A building undergoing increasing structural damage. A bridge in different climates experiences qualitatively different vibrations. An aircraft begins on the runway, accelerates, cruises in the sky while experiencing turbulence, then begins descending. These are all examples of structures that undergo avariety of stresses in distinct contextual ‘states’. It would be advantageous for operators to be able to detect these contextual shifts in both controlled and non-controlled settings.

Now, let us suppose we can record data by attaching an array of sensors (accelerometers, pressuresensors, thermometers) to the structure to try and divine when such transitions occur. How should we approach such a problem? Preliminary visual inspection of accelerometer data on aircrafts and other structures gives no clues as to how this might be done (fig:aircraft phases, fig:building experiment blog). Even worse, as you can see, sensor data is naturally very noisy, and so any method we think up of needs to be able to contend with this type of unstructured, messy, time-correlated data, and anomalous transitions may not be able to be captured by a simple set of features.

In this blog post, we introduce our proposed methodology for detecting anomalies in context; specifically, we will use our methodology to detect transitions between different damage levels in an experimental building data set (the ‘building’ or ‘bookshelf’ data set). The data set we will focus on is accelerometer data coming from a sensor array attached to a miniature three-storeyed building structure. To introduce anomalies into the data, damage was introduced to certain sections of the structure (bolts loosening, removing brackets, etc.). The goal is to create an algorithm that can automatically identify when a part of the underlying structure undergoes an increase in damage.

In our background investigation of this problem, we trained and reported the performance of a number of baseline models - off-the-shelf machine learning algorithms - that could potentially solve the damage detection problem, with models such as feature-extraction-SVMs and 1D convolution networks. However, we ran into a number of issues which commonly plague all anomaly detection algorithms. A lack of training data on the most important anomalies means that any supervised algorithm could learn normal behaviour quite well, but struggles to generalise beyond the baseline. The complexity (and noisiness) of building accelerometer data means manual feature engineering can hardly capture the necessary differentiators between different damage levels, leading to poor performance. If we opt for automatic feature extraction (as in 1D convolution networks) then we run afoul of model explainability:in high-risk contexts such as building/aircraft/bridge monitoring, understanding the how and why in anomaly detection is almost as important as the power of the model itself.

Is it a banana or a toaster? Knowing how a model categorises something as anomalous is essential in high-impact environments, where a false negative could imply disastrous consequences..

In order to create more intelligent, well-performing and explainable models that do not rely on the existence of prior training data, we need to introduce a number of key players that we used in our project to efficiently solve the contextual anomaly detection problem.

Figure 1: Two different distinct cruise phases for an aircraft, with data for Cruise Phase 2 in both healthy and damaged aircrafts.

Figure 2: Time series data of a building structure being shaken as the structure is increasingly damaged (state: healthy, mild damage, large damage and extreme damage, in that order). While healthy data is easy to tell apart from the rest, it is not easy to visually categorise the three different levels of damage!

In order to create more intelligent, well-performing models with the above characteristics, we need to introduce a number of key players that we used in our project to efficiently solve the contextual anomaly detection problem.

Follow-up: If you want to run the code locally and check the results yourself, please clone a local version of our repository from here. Thank you for reading!