What about Methodologies?

28 January 2022

Our project work is contained in a number of Jupyter notebooks embedded in the page below. You can also run the code yourself by cloning our GitHub repository here. See in particular the final output images and CAC animations in the offline and online notebook sections respectively (scroll down!).

Offline Notebook

See below for a statically embedded Jupyter notebook showcasing our offline methodology. The basic idea (described fully in a previous blog post) is that we transform the data, concatenate all of it together, compute the matrix profile and Corrected Arc Curves (CACs), combine predictions using the Wasserstein barycenter and use a peak-finder to figure out where the anomalies are. For a quick summary, look out for the output images of the data at each step of the processing pipeline and the final result: a CAC curve that clearly demarcates the true anomalous transitions without any false positives!

Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
view raw end2end.ipynb hosted with ❤ by GitHub

Online Notebook

See below for a statically embedded Jupyter notebook showcasing our online methodology. The basic idea remains the same as the offline algorithm, except we now transform the data in batches (mimicking "arrival in real-time"), and update the matrix profiles and CACs in a time-efficient way. Every 100 points added (corresponding to approximately 0.06 seconds of real time), we record the updated CAC in an array. Look out for the gifs of the CAC changing in time as anomalies are recorded! We see a demonstrable clear peak for each anomalous transition that occurs in our data. Compare also the effects of using only a single sensors worth of data to compute the CAC (the first three gifs in the notebook) versus using all 24 sensors (the final gif of the notebook): there is an enormous improvement in smoothness and clarity in the anomaly signal, which bodes well for live operational use.

Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.