Trialing a Time Series Edge Detection Technique
So after a couple of wasted evenings on different edge detection approaches using the delta between the current 15min utility reading and the previous one, I determined three things:
-
It’s impossible, at least in my house, to set separate detection thresholds that find all the positive and negative car charging edges, based on just the the 15 min delta values alone. There are just too many other heating and cooling events that can masquerade as car charges based on the limited time resolution data I have.
-
There are no simple time series approaches to filtering out heating and cooling events - they are just not regular enough. I can almost filter out the cooling events using data from my Ecobees, but I haven’t cracked the access to the data hidden in my NuHeat flooring heaters.
-
So my best approach has been to set the delta thresholds low enough to catch all the car charging event edges, as well as many extras, then use other parameters to filter out all the unlikely pairs of positive and negative ramps. The most promising technique seemed to be to look at every positive spike that passed the threshold, match it up with the nearest and next nearest negative spike, then look at the time distance (NextNeg) and the mean energy consumed between the positive and negative going time intervals (PeakMean).
Doing #3 for my 2019 Sense data merged with my 15min utility data gave me an interesting chart. This chart shows the two parameters I mentioned earlier for every positive spike / delta that was the 1 hour equivalent of 6kWh (remember I’m looking at 15 min increments, but all the power/energy numbers I have extrapolated to 1 hour equivalents to simplify my thinking). NextNeg tells how many 15min periods it is to the first 6kWh downward spike / delta. And PeakMean tells me what the average energy used in the whole house was during that period.
Next I’ll filter in two ways. My cars don’t charge longer that 7 hours of so NextNeg for any charging must be less than 28 15 min periods. And I can apply a simple formula for the minimum mean energy that must be expended to charge the Model 3 to exclude other points, leaving only the “good” charging candidates in blue-green below.
It turns out after a little sniffing at a few of the boundary points, the “charge” in magenta is a false (not EV charging) positive spike that piggybacks on a real charging cycle an hour or so later. It also looks like the curves break out into two separate pattern groups with different average energy uses that seem to align with the two different EVs. Gotta investigate that later.
If I roll these 190 “Good” charge cycles / points back into hours (remember that many charging cycles cross multiple hours), I get 386 hours of charging that I can use to annotate my earlier charging detection curve of training features. I get a result that looks somewhat reassuring, but not completely.
Most of the hours that my new algorithm identified as charging hours fit my old parametric curve nicely. But a small number are buried inside of the “non charging” zone and a bunch of “non-charging” hours per my new algorithm live in the charging region of my old parametric curve. A quick confusion table shows me how the two approaches have done with respect to each other:
They both agree on 339 hours as hours when charging is taking place and agree on 5707 hours when both are certain charing is NOT taking place. But my GLM model thinks charging is taking place during 67 hours that the new time series approach does not. And the time series approach see 41 hours of charging that the GLM model does not agree with. Such is the nature of machine learning and prediction. But which one is right and are they always right when both concur ? Time to look at some of the points of disagreement.