I’m just a user figuring these things out. I posted a few replies over the past couple days to community questions, then realized together, they tell a fuller story.
Device Detection Taxonomy
There are four different general technical types of device identifications in Sense-land. And there could be more in the future.
- Native Detections - for devices that have “clean” on and off transitions in the timeframe Sense is looking for (1/2 second transitions). Sense learns these on/off pairs as detections using machine learning - more on this later.
- Special Native Detections - for devices that use lots of power, but don’t have on and off transitions that fit Sense’s 1/2 second detection windows, mostly HVAC and EV charging. Sense has developed special models for these, but they are very dependent on picking up the complex power ramps and waveforms that specific electric vehicle / AC / Heat pump models kick out. My Tesla detections have been found, then lost numerous times because Tesla software often changes charge ramps.
- Smart Devices that assist with detection - Sense uses readings from a smart device to assist with learning. The actual device still needs to have clear on and off transitions, but Sense gets clues from data coming from the device. Two flavors of this that I am aware of:
- NDI (network device identification) - Sense monitor uses network traffic to spot ons and off of some very specific smart TV models.
- Ecobee Historic - Uses Ecobee 5 minute updates to better refine HVAC detection. The data from the Ecobee comes too infrequently to be monitored on Sense directly (bubble would be off even though AC has kicked in), but is very useful in training Sense and improving Sense HVAC models.
- Smart Devices that measure “detection” - where the smart device regularly updates Sense with the actual usage based on a measurement or calculation from the device. This data can be used as “ground truth” for training Sense, but isn’t directly used to “learn” the specific waveforms of the connected devices. Measurement comes in a few flavors today:
- Sense is also working on an improvement called Progressive Device Detection - more on it here:
[Video] 2021 Data Science Device Detection Updates
Because they are doing actual measurements, the second to last category (with smart plugs and DCM) include the Always On for a device, the rest likely don’t. Example - I have Ecobee Historic and Sense now does a darn good job of identifying my AC units, but it still misses the 6W or so that are always flowing to the electronic control board and thermostat.
What Type of Transitions Qualify as “Clean” for Native Detection ?
Sense is looking for specific on and off transitions for native detection - ones that are sharp, and last less than 1/2 a second or so. Here’s one way to “see” roughly what Sense is seeing, at least as far as traditional detection is concerned - let your Power Meter go for a while in iOS/Android app (specifically NOT the web app). The app Power Meter will tag “transitions of interest” using an heuristic algorithm with the power change of that transition - these are the ones that are most likely to meet Sense’s criteria for traditional detection. Here’s a view of some of the tagged transitions for my house.
If you see your device turn on and Sense doesn’t tag the on transition, or breaks it up into several tags, in the Power Meter, then the transition doesn’t fit the model for traditional detection.
But and even if Sense “sees” a transition in the Power Meter, that doesn’t mean that
it is ready to detect it as a device - the transitions must pass three other tests:
-
There must be enough transitions that “match” so that Sense can actually determine that there is a cluster of similar transitions, under a range of house conditions. More on what clustering looks like later.
-
There has to be a similar off-cluster because Sense can’t do a detection with our a matching off, and vice versa.
-
The transition has to be unique. If your device has both on and off transitions being tagged in the Power Meter, then Sense either doesn’t have enough repeats, or the on/off data doesn’t pass the uniqueness criteria - that would mean that it’s on and off signatures smear together with what seem to be other devices, or are so broad in variation that it might not be one device.
-
I also believe that the “cluster” also has to match a Sense model. I get the feeling that some electronic devices either look vastly different for different ons and off, or that the cluster of feature are so far outside the normal physics-based clusters that Sense has to treat each cluster one as a special case.
One of Detection’s Biggest Enemies - Noise
Sense uses transitions for native detection, so you would think that lots transitions would be a good thing. The reality is that transitions are only good to a point - too many transitions, especially good-sized ones create “noise” that can swamp out less frequent and smaller transitions happening in your house. Some typical noise sources:
- Devices with high variability, like washing machines
- Baseboard and other heaters that do frequent cycling
- Multiple heat pumps
- Lots of electronics with high variability (printers, computers)
More on noise and how to reduce it here:
How Does Sense Save “Transitions of Interest” to Learn and Search to Remember and Detect ?
Here’s a partial view of what happens when those “transitions of interest” reach Sense’s “brain” back at the Sense mothership. Each transition has around 20 measured parameters or features associated with it, and Sense drops each transition point into a corresponding 20 dimensional space - here we’re seeing only 2 of those dimensions, some measure of power on the y-axis, and some kind of phase measurement on the x-axis. (you can see that there are at least 17 dimensions or “features” from the screenshot labeling).
Over time, the accumulation of transition points create clusters. If the cluster is confined enough and dense enough, and if it has a matching “off cluster”, it graduates into a detection. I think colors here reflect different clusters, though you have to remember that we’re only seeing 2 dimensions of a 20 dimensional cluster, so the clusters would be much more apparent in 20-D space.
Google does something similar in this experiment- they categorize thousands of sounds by many features, then distill down to a 3D representation. You can roam around the neighborhood of a particular sound and hear similar sounds.
The Problem With Clusters
Clustering works well when clusters are truly distinct and algorithms can cleanly separate them. But unfortunately the real world isn’t usually that tidy. Clusters typically overlap in the dimension space and algorithms have sort out which overlapping cluster a “detection” belongs to. The example on the right shows an example of 3 slightly overlapping clusters. How should an algorithm slice up the boundary regions ? No matter how it is done, there are going to be some conflated detections.
How do Sense “Native Detection” Models Improve Over Time ?
As shown a little earlier, identifying devices is all about finding and divvying up the clusters of transitions in the many dimensional chart above. For purposes of this discussion, let’s call this the transition “dimension space”. So improving detection requires doing a “better” job of picking out clusters within the dimension space without upsetting existing working clusters, as much as possible. I can’t speak to the algorithms used, but can give a little insight into when the various clustering algorithms are invoked - they probably use several:
- Sense adds a new model or refines an existing model - Sense probably searches every house for likely clusters within the dimension space “neighborhood” where those model types live, and then sets about trying to cleanly cluster any unidentified groups of similar points. Sense might also use the Home Inventory to help, or at least test new model before more broadly releasing.
- Sense support helps you with a problematic model in your house - in this case, Sense probably runs a localized re-clustering in the neighborhood where the problem detection is occurring. I don’t know what kind of controls they have, but there might be things like “split this cluster into two” or “combine these two clusters”.
- A user deletes a device - this user action probably invokes re-clustering around the neighborhood where the deleted device lived. I’m not sure how far Sense looks back in time, but the neighborhood is likely to look a little different from when the models were first created, so the re-clustering will likely end up with a different, hopefully better, result.
- Regular model tuning - Sense recently shared that they do readapt the models for your house every 60 days of so to handle changes due to time of year, etc. This probably results in a “tuning” of the clusters based on the past 60 days of data. Tuning would mean readjusting the boundaries without destroying existing cluster. I suspect that this is also share Sense would detect a new cluster from a device that a user had recently commissioned.
Any questions so far ? This is what learning looks like !
Picture excerpted from video, here:
