2019 Data Science Updates

In case you don’t follow our blog: 2019 Data Science Update.

After a stellar 2018, 2019 is proving to be another year of major data science accomplishments for Sense. Work, thus far, is taking place within two broad buckets. Let’s take a look at them in turn.

The quest for “ground truth”

Here at Sense, we often talk about getting more “ground truth” data. For us, it’s the holy grail that helps us drastically boost device detection accuracy and reliability. If you’re not familiar with the term, in machine learning circles, ground truth refers to an accuracy check that pits machine learning algorithms against the real world. For us, it refers to the reality that we’re attempting to predict via our device detection models. So, if we’re working to detect your refrigerator, ground truth would refer to the actual waveform of your refrigerator that Sense is attempting to disaggregate from the cacophonous cluster of devices in your home. If we know exactly what to look for, it becomes much easier to build predictive models.

Now, you might be thinking: Why don’t you at Sense just buy a bunch of appliances and devices, run them at your offices, and build models from that small set? It’s a common misconception that Sense actually works this way — i.e., once Sense is installed in your home, it compares what it sees to a database of pristine, unchanging device waveforms. If only it were so simple! The same model refrigerator can in fact look quite different in two homes due to variances in its manufacture, other devices that are running concurrently, the physical environment in which it’s running, and even the noisy perturbations that exist on your local utility lines and in your home. While we can’t build a predictive database from just one home, we can seed one from many homes, and for that ground truth data is a huge help.

Ground truth, for us, refers to the reality in your home, and 2019 has seen a big step forward in our ground truth data gathering. 2018 saw the release of the smart plug integration, which has given us great data on the devices users are connecting them to. This began our quest for ground truth. In 2019, we’ve taken it even further. As we’ve continued to digest the data from your smart plugs (keep it coming!), we’ve also released beta integrations with certain smart thermostats. Unfortunately, due to reasons outside of our control, we’re unable to release these publicly at this time, but they have given us fantastic data about HVAC devices across our Beta team that is helping us to build out more advanced models for HVAC detection. We have some other ideas to pull ground truth data from smart thermostats, so keep watching for updates.

We’re also pulling circuit-level data from a small set of internal pilot homes, which is helping to provide granular, ground truth data for circuits that feature just one major appliance — think water heaters, pool pumps, HVAC, electric vehicles, and more. We ultimately don’t believe circuit-level monitoring is the solution to energy disaggregation (the fancy term for device detection), with its complex installation, lack of device-specific insights, and potential electrical code concerns. However, these pilot homes can provide valuable data for us to improve model building.

Ground truth doesn’t always need to be so granular. Our recent Device Inventory feature, part of the Home Details + Compare release, gives us better insight into what’s actually in your home, and will eventually allow our models to key in on your devices and not waste time looking for devices that aren’t in your home. We’ve gotten great data so far, but the benefits will take time to see. Be sure to give us a hand and fill out your device inventory.

Improved historical insights

2019 hasn’t just been about ground truth data gathering. More generally, we are working to better model large energy consumers in homes, with specific focus on HVAC and electric vehicles. For EVs, this includes analyzing our expanding data set to both refine existing models and tackle additional vehicles. We are having great success here and recently released brand new Tesla models that will significantly improve our support for Model 3 detection.

Some of these advances will start to show up in better historical reporting of energy use. Some devices aren’t easy to display as real-time bubbles in the Sense app, but can still be tracked historically. For example, one challenge with electric vehicles is that they have complicated “wind down” patterns as the battery is getting full. It’s hard to show a correctly sized bubble as the slow wind-down is happening in real time, but we can model and present the historical data to you, so you can see precisely how much energy your vehicle used over the past week, month, or year. We’re still deep in our work on this front, but hope to share our findings soon.

Beyond these two major buckets, there is a constant stream of work to make incremental updates to device models — like Mahesh’s recent work on ACs— and hopefully you’re seeing positive results from those efforts.

The future for Sense certainly looks bright. In addition to these major device detection updates, we’re excited to continue work on our partnerships with both Schneider Electric and Landis + Gyr to further integrate Sense into the common energy infrastructure. Be sure to follow us here on the blog and via our newsletter for all the latest Sense news.

7 Likes

Great blog - thanks for posting here. This blog answers a lot of questions that Sense users seem to get really fixated on. The other new one is also quite good, because the first thing a guy like me asks after hearing that I can’t train Sense directly, is “well then, how can I help get better results ?”

Can I Train Sense ?

4 Likes

Feel free to use my home. I live about 60 miles south of your HQ. :+1: Most of my 220V devices on dedicated circuits are where Sense struggles the most.

Keeping that just with Sense and partner employees for the time being, but I’ll certainly keep you in mind if we roll out for a public beta.

I was going to chime in along those lines @scorp508 but I imagine it’s a more controlled pilot and probably with Schneider loadcenters/breakers or somesuch.

Hey @RyanAtSense, if thsoe pilots actually involve a fat stack of Sense monitors, one for each circuit, it would make a nice pic that somebody like @samwooly1 would enjoy :wink:

2 Likes

Hah! I can’t divulge specifics there, but it’s not a fat stack of monitors. Which reminds me, I think I owe you some pics of our big testing rack and of course I’m working remote today so can’t grab them.

2 Likes

Great blog post, thank you!

Along the lines of the WiFi Smart Plugs, there is a lot more out there that could be captured but are not such as Zigbee and Zwave which has been around for a while. On some modultes I can pull them up on my SmartThings app and see the current energy usage including total time since the last manual “clear” of data, as a KW total.

Lots of folks are tied into SmartThing along with integration with Google Home and Alexa. Allowing Sense to be able to monitor these items including what type device they are might also help. I’m not an EE but I am in Engineering. SmartThings API allows device and Smart programs/tasks to be integrated into it.

Just throwing this out there to consider another source of direct information other than WiFi Smart Plugs (which I’ve recently purchased just for Sense integration).

1 Like

I’ve seen two issues with your idea out on the Smartthings forums:

  • Only some devices supply power info and at a rate nowhere near that of WiFi smart plugs
  • Zigbee/Zwave isn’t designed for the dat update rates supported by WiFi smart plugs.
    It could be a good source of ground truth, but not as valuable or consistent as the currently supported smart plugs.
3 Likes

I am planning on getting a couple of HS110s eventually when I can catch them for less than $22 each. I have a Zooz Zen25, which has energy monitoring (but I don’t use it for anything). I just looked at the settings in my hub (Hubitat Elevation) and the fastest I can have it refresh is every 30 seconds. I’m just curious how often the supported wifi ones update; I can’t seem to find that information.

Sense polls/receives data every 2 seconds from WiFi smart plugs. I have had a couple of other energy monitoring smart plugs, but they were rate limited by the underlying physical layer and protocols. One used Bluetooth and the other used Zigbee.

2 Likes

@kevin1 thanks for the information as I had not considered the data stream requirements! I’ve since purchased several TP-Link Kasa devices to incorporate into my set-up as I see I can add them into SmartThings as well.

1 Like

Just make sue the TP-Link Kasa devices you want to use are the ones supported by Sense, the HS110 and the HS300. Those are the only two Kasa smartplugs that also provide power/energy data.

1 Like

@kevin1 thanks, wish I had known that earlier as I purchased (3) 105’s. Back they go.

Sorry I didn’t catch you earlier… Right now I’m seeing HS110s for about 22$ on both Amazon and Walmart online.

A few useful links:

1 Like

@kevin1 no worries, mate. Amazon has free returns.

Have you had any experience with the Kasa HS300 multi strip? I understand each is controllable, but wonder if each report individual power. I’m guessing it only reports combined?

Thanks for the links. I went there after I received your first message.

Update: Disregard my query on the HS300. I see each outlet is like an individual smart plug. Now it makes more sense why it’s pretty pricey.

1 Like

Just one word of advice - the HS300s are very handy for clusters of devices that Sense is unable to currently detect, like AV systems or networking / server racks. But use your smartplugs judiciously - there’s a current montior limit of 20, where each monitored outlet counts as one. Very easy to get to 20 if you fully use three HS300s. A number of devices I have plugged into my HS300s aren’t very informative.

Looking back, I probably should have surveilled a few days of power usage on all potential smartplug devices with a Traveller HS110, to see if they were worth a dedicated smartplug.

3 Likes

Strongly agree with @kevin1 on this one.

Here’s my somewhat ordered list of smartplug adoption and method where:

  • D = device plugged in to Dedicated smartplug, HS110 or Wemo Insight

  • PD = multiple devices plugged in to Passive Power strip that is plugged in to a Dedicated smartplug, HS110 or Wemo Insight

  • A = device plugged in to an outlet on an Active (smartplugs) power strip, HS300

  • X = deliberatly NOT on a smartplug

  • SM = monitored using Sense Main CT(s).

  • SS = monitored using Sense Solar CT(s).

  1. Fridge/freezer [D] – because I’m very interested in how fridges/freezers are treated by Always On so I want to see the real data now. Sense also struggles with inverted-based frdiges like mine. Ground truth fridge data is one of the better (best!?) candidates for Sense-based failure alerts. Feed the beast! Note: Switching control is deactivated in Sense to avoid inadvertent switch-off. HS110 is a better option than Wemo Insight because the HS110 will restart to ON in the advent of a power failure.

  2. Hot Water [SM] – I have this on a second Sense monitored using the Mains CTs. Note: cheapest way in my mind to reliably monitor a device >1500W. Having the second Sense also gives me options to start splitting up the smartplug processor load and going beyond the recommended 20 in the house.

  3. Air Conditioner, 120V, 800W max [SS] + [D] – If you’ve read this far you know what I’m targeting here.

  4. OLED TV [A] – Very hard for Sense-native disaggregation.

  5. Desktop computer [A] – if it had over 750W power supply (e.g more of a “workstation”) I’d probably do [D] into a UPS.

  6. Laser printer & shredder [A] – they have very interesting signatures but can use a lot of power so shouldn’t be ganged up with high-power devices on a strip.

  7. Fan, room-to-room [D] – this is a case of wanting smartplug control options but also trying to monitor the interplay between heating/cooling and thermodynamics in my space.

  8. Kitchen exhaust [D] – this can crank up to 800W and has electronic multi-speed control so same reason as #7. Using IFTTT for "IF hood switches on THEN switch off room-to-room fan etc.

  9. Multi-speed air purifier [A], also about #7

  10. Various lights [X] – absolutely do not want any latency or smartplug UI fiddle or over complication (lol). Also, I like my old toggle-switched lights.

  11. Various lights [HUE] – Including for IFTTT: when fridge [D] goes off, HUE goes red. Despite #10 I also want certain night lights – bathroom glows red at night and has motion-detection activation.

  12. Network gear [A] --> [PD] – I’ve been running modem/router/switch/NAS gear on on HS300 but will transition that gear to a passive strip on a smartplug eventually. Once you have a few days of data you see these things are relatively stable in usage. I am interested in leaving the PoE switch on a dedicated smartplug [D] or [A] though because it gives me historical fine-grained indications of having plugged/unplugged devices from the PoE switch itself. I can also use the PoE switch’s own energy monitoring to “calibrate” smartplugs (& Sense) at the lower end of the wattage spectrum. If I had a bigger NAS or server I would do the same as #5.

  13. Heating [SS] --> [SM] – underfloor electric resistive heating (3kW) that I want to carefully monitor as I transition to adding a heat pump. Debating getting a third Sense!

  14. smartplug! [D] + [A] – HS110 plugged in to an HS300 for #5. The comparative waveforms are pretty interesting.

The HS300 reports each individually.
It’s like having six HS110’s

1 Like