A little more fun with Always On blips - Downward spikes in hourly Always On are definitely a good indicator of Sense monitor issues, either from a data transfer or a data read perspective. I very carefully analyzed my Power Meter waveforms from Jan to mid-Aug 18 for two types of monitor issues, data dropout and negative Total Usage. I then mapped those issues vs. downward spikes in Always On. The X’es below represent downward spikes from the 12 hour moving average that is greater than 80W. The dots represent visible errors in Sense data.
I then tried to use a couple of different algorithms to find similar issues solely via my Always On waveform from late-Aug to mid-Nov 18, using my Jan to mid-Aug data for training. I avoided using a time domain (RNN/LSTM) algorithm for a couple of technical reasons, but instead used the two adjacent hourly Always On readings on each side of the sample point, plus the sample point reading as the “features”, to give me a time element. Here’s the resulting waveform and list with predictions based on the Optimal Weighted Nearest Neighbor Classifier (“ownn” model in caret) …
No surprises, but it looks like the neural network did an OK job figuring this out. All 18 “Gaps” it found looked similar to this from a Power Meter perspective:
Both of the Negative Total Usage points look like this:
But the “ownn” predictor only found the 2 starting hours of a 24hr period of Negative Total Usage that in total looked like this:
But if you notice, Always On was zeroed out that whole time, so it “knew” something was up.
Now, you may be asking why my smart little neural network didn’t flag the hourly Always On downward dip to zero in the 1AM hour on Dec 4th that I have circled (I asked that question immediately). Amazingly, the neural network is smarter than me ! Here’s the Power Meter waveform - perfectly normal, so not all downward dips are issues !
The power of data science and prediction !