Taxonomy and Insights Into How Sense does Detection

I’m just a user figuring these things out. I posted a few replies over the past couple days to community questions, then realized together, they tell a fuller story.

Device Detection Taxonomy

There are four different general technical types of device identifications in Sense-land. And there could be more in the future.

  • Native Detections - for device that have “clean” on and off transitions in the timeframe Sense is looking for (1/2 second transitions). Sense learns these using machine learning - more on this later.
  • Special Native Detections - for devices that use lots of power, but don’t have on and off transitions that fit Sense’s traditional detection time windows, mostly DC-based HVAC and EV charging. Sense has developed special models for these, but they are very dependent on picking up the complex power ramps and waveforms that specific vehicle / AC /Heat pump models kick out. My Tesla detections have been found, then lost numerous times because Tesla software often changes charge ramps.
  • Smart Devices that assist with detection - Sense uses readings from a smart device to assist with learning. The actual device still needs to have clear on and off transitions, but Sense gets clues from data coming from the device. Two flavors of this that I am aware of:
    • NDI (network device identification) - Sense monitor uses network traffic to spot ons and off of some very specific smart TV models.
    • Ecobee Historic - Uses Ecobee 5 minute updates to better refine HVAC detection. The data from the Ecobee comes too infrequently to be monitored on Sense directly (bubble would be off even though AC has kicked in), but is very useful in training Sense and improving Sense HVAC models.
  • Smart Devices that measure “detection” - where the smart device regularly updates Sense with the actual usage based on a measurement or calculation from the device. This data can be used as “ground truth” for training Sense, but isn’t directly used to “learn” the specific waveforms of the connected devices. Measurement comes in a few flavors today:
    • Smart plugs like HS300 and KP115 - report back to Sense monitor every 2 seconds or so
    • Hue lights - the Hue hub sends calculated usage info to the Sense monitor.
    • DCM (dedicated circuit monitoring) - additional Sense CTs that report power usage on 1-2 specific 120V/240V circuits in your house.
  • Sense is also working on an improvement called Progressive Device Detection - more on it here:
    [Video] 2021 Data Science Device Detection Updates

Because they are doing actual measurements, the second to last category (with smart plugs and DCM) include the Always On for a device, the rest likely don’t. Example - I have Ecobee Historic and Sense now does a darn good job of identifying my AC units, but it still misses the 6W or so that are always flowing to the electronic control board and thermostat.

What Type of Transitions Qualify as “Clean” for Native Detection ?

Sense is looking for specific on and off transitions for native detection - ones that are sharp, and last less than 1/2 a second or so. Here’s one way to “see” what Sense is seeing, at least as far as traditional detection is concerned - let your Power Meter go for a while in iOS/Android app (specifically NOT the web app). The app Power Meter “tags” what I believe to the be the “transitions of interest”, the ones that meet Sense’s criteria for traditional detection, with either transition power numbers or with the name of the already-detected device. Here’s a view of some of the tagged transitions, both undetected and detected, in my house.

If you have your AC or sump pump turn on and Sense doesn’t tag the on transition, or breaks it up into several tags, then the transition doesn’t fit the model for traditional detection. Also watch the off, for a similar negative tag, because Sense can’t detect an on without a matching off, and vice versa.

If your AC or sump has both on and off transitions being tagged, then Sense either doesn’t have enough repeats, or the on/off data doesn’t pass the uniqueness criteria - that would mean that it’s on and off signatures smear together with what seem to be other devices, or are so broad in variation that it might not be one device.

How Does Sense Combine Those “Transitions of Interest” and Learn from Them ?

Here’s a partial view of what happens when those “transitions of interest” reach Sense’s “brain” back at the Sense mothership. Each transition has around 20 measured parameters or features associated with it, and Sense drops each transition point into a corresponding 20 dimensional space - here we’re seeing only 2 of those dimensions, some measure of power on the y-axis, and some kind of phase measurement on the x-axis. (you can see that there are at least 17 dimensions or “features” from the screenshot labeling).

Over time, the accumulation of transition points create clusters. If the cluster is confined enough and dense enough, and if it has a matching “off cluster”, it graduates into a detection. I think colors here reflect different clusters, though you have to remember that we’re only seeing 2 dimensions of a 20 dimensional cluster, so the clusters would be much more apparent in 20-D space.

Any questions so far ? This is what learning looks like !

Picture excerpted from video, here:

2 Likes

This is great @kevin1 - I’m going to take a longer read through this today and add it to the New User Guide if everything looks alright.

1 Like

AOK with me ! Feel free to correct or refine as needed. And feel free to rename things using Sense’s internal vernacular as you see fit - would be great to have users talking the same language as Sense developers. And of course, position correctly, in the light of ongoing development efforts :wink: