Retroactive device detection/labeling/training

Many people on here have mentioned training sense to assist with device detection. Often the response is that it won’t work because the timing would not be precise enough between when people turn a device on and when they notifying sense that they turned the device on.

But, sense already tracks the precise time changes in energy usage occur (e.g., a notification bubble that says +1943 watts). It would be great if we could assign a “bubble” to a specific device to assist in the training (e.g., click on the +1943 watts bubble, select device AC to inform sense that the increase was when the AC turned on). By assigning the bubble, there wouldn’t be any issues with the timing of the label and power consumption not being aligned. By tracking precisely when the device turned on and off (using bubbles that were assigned to devices), Sense should be able to get a more exact reading of the power signature of that device.

2 Likes

That’s a good suggestion, with three caveats:

  • given machine learning, you wouldn’t be adding a bubble per se, but rather a training cue.
  • even if you picked the exact half second sample of the biggest upward or downward spike, you still might be missing up to a half second of the beginning of the spike, which means a half second of the on or off signature, assuming it is a major fast spike.
  • that also assumes that everyone can accurately pick the right spike every time. Even a small error rate contributed by tagging similar waveforms could have a devastating impact on accuracy, which would be hard to undo. I don’t know if you have ever mislabeled a face in photo program, but the same thing can happen on the initial recognition (vs categorization for faces) training.
2 Likes

I don’t think I was totally clear in my first post. By “notification bubble”, I meant things like the “+757x” and “-510w” in the attached image. I would like to be able to click on that and identify occurred at that event (in this case, my dehumidifier being turned on then off). By using these events, identified with precision by sense, I don’t think we have to worry about missing the on or off signature of a fast spike.

Accuracy of tagging is an issue. But there are many potential ways to mitigate this. Considering the #1 complaint people have with sense is the device detection, I think anything that helps improve it would be worthwhile.

It is an interesting thought but I think what you might run up against is this:

  • The relatively big deltas are already pretty-much nailed by Sense if they are consistent … these are the easiest things to track and spot.

  • The relatively small changes which, if somewhat inconsistent, are harder for Sense to nail but they are also more likely candidates for human mislabeling … coming up against what @kevin1 points out.

So you would question whether it’s really going to accelerate anything … and perhaps make detection worse.

1 Like

OK - now I understand what you mean about “notification bubble”. Two things you should know about that notification bubble:

  • There have been several questions on the forum about what the “notification bubble” actually measures. You are assuming it is an accurate marker, but all Sense feedback on this forum indicates that we shouldn’t try to interpret those notifications as meaningful. I’ll try to find one of those statements.
  • Even if the notification bubbles are indicative of something meaningful within that 1/2 second snapshot, they can be offset by up to a half second from the real start of the event, since the Power Meter has half second granularity.
1 Like

Uh, sorry, but no. I have several “high impact” devices which sense has failed to identify over the ~18 months that I’ve owned it. I have read all the explanations for why we can’t train it (not going to argue it), but don’t tell me that it’s reliable in identifying any particular class of load.

Examples: I have a pool pump that cycles at least once per day for months out of each summer, never identified. Mini-Split AC (it’s a more subtle signal, with gentle ramps, but large loads) never identified. Several appliances which all could come under the “heat” umbrella, which are never identified individually, but often get stirred in with different appliances of similar nature. The toaster always draws the same power, but sense seems to think that it covers a wide range of values (thus missing others like the fridge defroster, dishwasher dry cycle, etc.) Circulators for my DHW and radiant heat have never been identified.

Though last week it proudly told me of Motor7! Which had zero cycles, and zero power consumption, over the entire lifetime of my data. I deleted that one right away.

I also have seen a lot more regression in the identification, with items that had been reliable now identified as running, when they are not.

1 Like

On regression look here:

https://blog.sense.com/behind-scenes-lost-detection

Identification requires a combination of distinguishability and repeatition (consistency). In practice, bigger relative power peaks will distinguish themselves. In theory though any consistent peaks (or troughs for that matter) in the signal could aid detection.

By way of a seemingly trite example:

My toaster is quite consistently detected by Sense natively. Since it’s initial detection I have yet to figure out what will throw it. A second toaster would no-doubt make life difficult, for Sense and me! That said I haven’t deliberately tried to throw it off and, I suspect most importantly, my home has a relatively simple electrical profile … especially in the morning perhaps before everything wakes up. I have no doubt that Sense could get even that simple device wrong.

My thought experiment example falls back on a house with 2 lights. Both the same; identical bulbs; perfectly wired with identical switching. Nothing else is running. Sense (or a human for that matter) will never be able to distinguish which bulb switches on and off by only looking at the overall electrical signature from the house. Now imagine both bulbs being switched on and off with normal human regularity. The overall Sense graph will give you an accurate picture of the house power consumption (0 bulbs on; 1 bulb on; 2 bulbs on) but no clue as to which rooms/bulbs were lit. On that front it will be perfectly imprecise and there is nothing, given my perfect house, that will clue Sense in to which bulb is on, or off from the electrical signal alone.

Now imagine that the bulbs are switched on and off faster than once every half second. A lot faster. As @kevin1 points out, this is the resolution of the Power Meter view. What is your own expectation of being able to track how many bulbs are on from the Power Meter view in that case?

My point is that comparing Sense’s ability to disaggregate devices to a human’s ability is not (mostly) valid. On a macro level it often looks easy but on a micro level is where it really counts, and where it gets hard. It may seem simple from looking at the Power Meter but more often than not it may well be near-impossible. It’s hard to divorce yourself from additional knowledge: I know my toaster is on because I’m making toast!

1 Like