High energy items not being identified

Identifying Devices in Your Home
22

One issue with some big ticket items, like EV chargers, is that they ramp up and down slowly in a way that can be very different depending on model, charger type, charge limit, battery size, and even EV software version. Sense was originally designed to detect on and off of devices immediately (bubble pops up right at the start), but EV ramps are better detected after a few second to minutes. And just the size / magnitude of a ramp doesn’t tell what specific device it is - Sense has to sort through literally thousands of possibilities. Specific recognition is not as obvious as most people think.

More on coming enhancements in “Progressive Device Detection”, here:

1 Like
23

First and foremost, I am still very bullish on the POTENTIAL abilities of Sense, from the device, to the Data Science team, but I think these points need to get back to product management QUICKLY:

  1. Decide who your market is, and please, POST IT HERE, so we KNOW what you believe to be your market. If an existing customer is not aligned with that market, then we know we should accept that and walk away from these discussions.

  2. Decide if “really cool Data Science” is your goal, or if PRODUCT detection is. Of course, it would be very nice if both could be true, but your current product simply does NOT have the “crowd sourced” feedback loop for device identification in any usable state. To me this implies the FORMER is your goal and not the latter. I really think this is KEY. It will be a company-wide strategy that needs to be decided and executed. Personally, IMHO, the latter is what will attract new customers, and with more customers, you’ll eventually be able to do more with the former.

  3. I believe you have a very potentially helpful community and you’re not using them to the fullest. I’m sure I’m far from alone on saying I really want to see “Always On” eliminated. I want every item that makes up “Always On” to be listed! And, I want to method to divide “Other”. Do I care if that happens “automatically” NO! Would I find it very cool if it did, Of course!

  4. I’ll plainly state that I purchased Sense to determine the source of “Always On”. I will repeat that: I purchased Sense to determine which devices make up my 850w of “Always On”. I have owned sense for over two months and this collection of devices are still just that, a collection of unidentified devices.

  5. I did watch the 2021 state of Data Science video on YouTube and while I understand you have a VERY, VERY difficult task to identify appliances, I continue to feel you are missing your audience/market. Let me provide two examples where I strongly believe you need human intervention:

A. Manual device identification. Yes, it’s possible. And, today I ended up doing just that in my home. I shut off everything and then, one by one turned thigs back on. In some cases, I’ll admit the circuit would have powered a few devices, but I turned off well over 98% of the home for me to FIND THE ALWAYS ON devices!!!

B. In “Manual device identification”, Sense should start with a complete list of appliances and devices that are entered by the user. That gives the program a very, very small subset of devices from which to choose. No need to figure out WHICH EV I have, I AM TELLING YOU! Fridge, Over, Range, Yep. All there. Did you use any of that data? Nope.

C. START WITH THE BIG ITEMS–they mean the most to the home owner with respect to $$. Look at EVERY spike that happens with more than 1500w and you’ll know (in the US) that it’s a 220/240v device. How many did you have to choose from? Four in my home. Probably fewer than eight in 99.44% of the homes. So GUESS!!! You’ll be right 12.5% of the time in the worst case–then let the homeowner CORRECT you. In less than an hour, you should have ALL the 240v devices identified. As with your video, if ANY of these devices doesn’t match your database, then ASK THE HUMAN for more info. You might not get it, but it’s OK to ask! Then use the observed data for a week to really get to know these devices and understand how to isolate them as all the other devices turn on and off.

D. Next come the 10-15A (120v) spikes. Just think how few devices can cause this kind of load. Big motors and resistance heating. Let’s consult with our nearby human again! Whenever you get a semi “clean” spike you have seen a few times, ASK!!! Does the human reply quickly? Perhaps not, but ask the human if you can set a notification whenever that spike happens again. I know I would answer affirmatively! Since I already would have registered ALL my energy consuming devices, go ahead and take a guess when that 1400w spike happens next. Computer - Don’t be serious. Light bulb. C’mon. space heater, Possibly… Coffee machine? Microwave, toaster? OK, now we’re getting there! If the item isn’t on the list, I’ll ADD IT!!! Not in your database? No worries, ask for more info as time goes on. You get the idea.

  1. The bottom line is I really feel you need FAR more data, and better ML algorithms to auto detect devices that are the big consumers. And until you have that, you are going to frustrate users who could be your best friend for gathering that data. You also will generate poor reviews and ultimately kill your chances for developing those great ML algorithms when product sales decline as competitors enter the market with a SMART UI that uses the homeowner for identification. That kind of competitor need not concern itself with data science or smart engineers if 95% of all devices capable of consuming 1A or more are identified. Go ahead, run a poll if you don’t believe it, and ask your community if human-aided identification or “Cool Data Science to auto identify devices (that isn’t working)” is more important to spend R&D money on. I believe you know the answer. You don’t have time on your side.
2 Likes
24

I completely agree with ATechguy. While it’s cool to have ML algorithms figure things out for you, it’s just not feasible to compel your core audience to stay on board should something better come along that does incorporate the help of the community to advance the learning.

I commend Sense for wanting to stay to its core value of trying to get ML to do the heavy lifting, but I think we’ve all learned it’s just not as advanced to keep up with the demand of the community. Sense has an opportunity to cater to the core audience that help with ideas and learning opportunities to better develop what they’ve designed. It’ll come down to WHEN, not IF another company pulls resources from their community to help learning so end users can get the big picture idea of their energy usage. Because in the end, that’s what we want.

25

@ATechGuy, I can’t speak to the target market customer criteria for Sense, but as a four year user, I can absolutely agree that I want Sense to continue to improve in the priority areas you have laid out. I also agree that in it’s early incarnations, Sense’s marketing message ran well ahead of what was delivered, and back in 2017 it really was only a product for early adopters. But I do think things have changed over the the years and that Sense has indeed adapted beyond a “machine learning only” focus. That’s why you see additional detection technologies like smart plug integrations, DCM, and specialized EV and HVAC detection capabilities (including the Ecobee integration). Sense has absolutely branched out into a hybrid detection approach that includes both machine learning and smart device data acquisition.

On the technology side, I would be interested in specifically how you would want Sense to “learn”, if not via machine learning. You seem to be under the impression that Sense could maintain some kind of individual database of devices and transitions for each home, and use that “dictionary” database for training and lookup. I got news for you - that’s been tried and has failed many time over the years for energy disaggregation. Appreciate your thoughts on priorities on which devices should be detected, but have to disagree when it comes to your thoughts on how it ought to work or what should work (because so far, your suggested approach has been a failure for this use).

I do think Sense is finding ways to nibble at the edges of the Alway On problem that you seem focused on. I have been using a traveling KP115 smart plug on all my pluggable devices to determine their Always Ons - leave them plugged into a device for a couple days and Sense will deliver an Always On number in line with their definition. Roll that into a spreadsheet and you most certainly can reduce your unknown Always On. And I’m betting that Sense is finding even better ways to leverage that kind of data. Would love to hear your thoughts on this technique (OK - you would need to use DCM for 240V and 120V wired devices).

26

One aspect of all this is that ML requires events to learn from and “always on” doesn’t constitute an event. The basic presumption of Sense is that relatively sharp transitions will allow detection, and neither “always on” nor many of our modern high usage devices have such transitions. So, the basic presumption is flawed and Sense doesn’t/can’t meet its product (e.g. the marketing) objectives.

About half of my large consumers either have very very gradual ramp up/down or no ramp at all. I can tell Sense about what I have till I’m blue in the face, but it will not be able to track power consumption for those devices.

27

@andy,
I thinks that’s an unfair statement. Sense is certainly working on long ramp transitions, otherwise it wouldn’t detect my 2 EVs or other folks’ mini-splits. You have indeed identified 240V devices with slower transitions that elude Sense detection, but your generalization to all slow ramp devices is baseless. I do think we will see more improvements in different techniques to track wired / 240V slow ramp loads.

28

This morning I was thinking again about the “Always On pie” and thought that some kind of layered “bar graph” would be very illustrating, but to your point, @andy, the EV charging has a slow ramp and I claim that with “VERY HELPFUL” human assistance, Sense has multiple ways of identifying this “pie”. My suggestion may be a bit more than some would be willing to go the distance but hear me out (Sense, are you listening??? @kevin1 may be a big community contributor but doesn’t speak for the company)

You start with EVERYTHING OFF. Then, incrementally, you start with the 240V breakers. These typically are for a SINGLE appliance, but not always. I would suggest starting with the ones that ARE for single appliances. After each one is enabled, I would enjoy a healthy dialog with Sense on the MODEL of the device and any feedback it may know from all the previous collections from the community on similar devices. When sense was ready for the next breaker, I would enable that one. I already know the VERY LARGE consumers. I just don’t know how often they turn on when I’m sleeping. But if Sense cannot capture those devices, even if they are enabled only one at a time, then my whole Human-assisted learning can go out the window. From my perspective of not being a data scientist, nor someone knowledgeable with other product in this field, I can only say that trying to find the “needle in the haystack” is/must be more difficult than being told. It’s the RANGE that just turned on. It’s a Kenmore AS32211KP Range. White. Six burner. Need more info?

Then comes the Mystery set of the “Always on”. Not too hard to isolate if you’re willing to take an hour and switch on-off breakers while having a beer. How? By doing the REVERSE.

Start with full-house on. OK, I see 1105w “Always on”. OK, so let’s pop that breaker. Ah, nothing changed. Super. Next? whoa. The 1/2 bath & Hallway just dropped 120w. OK Sense. Got that? Great. It’s interested. What’s connected? Humm, let’s go find out. Ah, TV, WiFi point, , UPS. That’s it. Turn it back on, you say? Certainly. (up goes 140w) Got that? great. Now off? Certainly. (down goes 140w). Need another round or two? OK. And remember, there are THREE devices there. If need be, let’s call them “Hallway Items (a group)” but DO NOT FORGET, there are THREE devices in that group and they are Sony TV, model BE12345, APC UPS model 54321 and a WAP, Ubiquiti HD67890.

And the process continues. In each case, we continue shutting off breakers until it’s all off (meaning the Sense break is the one you save for last). In my case, the Sense breaker is for a UPS that powers my server rack so I can keep my rack powered, but pull any energy draw so the house can almost hit 0w, with the only device powered being the Sense. In each case, we whittle those pesky 100-150w devices attached to various breakers into groups, with usually fewer than four items, ALL OF WHICH I CAN TELL SENSE PRECIOUSLY WHAT THEY ARE, and if that’s not enough, then I haven’t the foggiest idea how Sense would EVER decide what the device is from ML alone.

Does this lead to a full-identified home? No, not initially, but it breaks it down so the ML can be a LOT smarter from which to do the real cranking and analysis of each groups “signatures”. Statistically, it’s unlikely that each device in a group would remain constant, and I’m sure that overtime, small changes can be isolated, but I am not a data scientist.

As simplistic this method might sound, I FULLY appreciated that once each of these breakers get turned back on, Sense is now looking at say, 10 groups, all piled on top of each other. And so goes our helpful-human assistance. Perhaps. But I still believe that having the human identify the device, one by one, or the group contents, it must make the identification of the device from all the other inputs easier than trying to do it in the blind. That’s my premise. And, I guess until I am told otherwise, that’s where my input to ask for human-aided identification is rooted.

29

It seems the smart plug may be Sense’s method of human-aided detection. But there isn’t much “detection” going on there. And, to be frank, what’s not to have Kasa-compliant plugs (or whichever brands Sense works with) simply add an “energy” display to their app? Now we have Kasa plugs advertising to say, "Get a real-time view of your energy use by adopting our plugs for all your “energy-sucking” devices. Buy 10 and get 10% off, 25 and get 25% off. Gee, that was easy. All they need is to get China to crank some 240v 15-30, 15-40 plugs for under $10 and in my case, for about the cost of a sense, I would have my home not only identified for less than the cost of a Sense, but fully controllable from my cell phone.

Sense–time is running out. Please rethink your goals and let us know what you decide.

30

I (clearly) disagree. Since Sense won’t be solving the slow ramp up and down 240v device problem any time soon, they either need to:

  1. Clearly explain that such devices aren’t going to be handled, or, better for all of us who got drawn in with over-ambitious marketing,

  2. Provide a simple device that gets wired into 240v feed lin4es and communicates with Sense. That’s hardly complicated, think a wired in 240 version of a Kasa/tp-link. Sense could either offer such a device themselves (it’s FAR simpler than Sense itself) or partner with someone else who could provide it.

31

I totally agree, but should mention that many (most) 240 v devices do not plug in, they are wired in. So what’s needed is a Kasa like device gets wired in-line, either in the breaker box if there is room or in a standard electrical junction box where the line already connects to the device.

That would solve 100% of the hard/impossible 240 devices that draw most of our household power.

1 Like
32

In the Sense veracular, that’s their (optional) DCM add-on.

From what I’ve read, you can’t just plug that into a “240 circuit”, get Sense to identify, and then move it. It’s there for the duration. I believe you can two of them. Next thing you know you’ll want 32 of those suckers all over your breaker box (might as well get the 120V circuits while you have the cover off!) and at that point, you will have created what all the other monitoring devices currently do.

33

Good discussion on reducing Always On… I think you’re missing a couple things about Always On, though.

  • A single reading via a breaker toggle isn’t guaranteed to really give you the Always On for a device.
  • That’s why I recommend putting plug-in devices temporarily on a traveling KP115 and given them a couple days to pump out a real Always On measurement. You can add that to a spreadsheet of Always Ons. I’m guessing that Sense will figure out how to take this into account at some point in time.
  • DCM can be used in a similar way for wired devices/circuits. Put the CTs on 2 of your wired circuits and set up. Run for a couple days to get general usage data and Always On values. That’s how I know that my 240V Floor Heater Panel has a 6W Always On component (5 thermostats). Once you are done, move the CTs to figure out Always Ons for the next 2 devices/circuits. You can delete DCM devices and create new ones - it’s not there for the duration if you are looking to just quantify Always On values.

I have found this to be an easier and less disruptive way to isolate the real Always Ons, than turning off the whole house and flipping on each one individually.

34

Good to know you can move them.

As for moving thing around, I’m not sure how many would agree with one taking the break panel cover off regularly and moving things around in there. I’d love to know how many Sense owners did the install themselves vs hiring an electrician. I’m sure the Sense lawyers spent a LOT OF TIME properly wording their install instructions for CYA security. Popping the odd breaker is kid’s stuff compared with moving a pair of CTs around in a live box. Oh, did I hear you say it shouldn’t be “live”, well popping the WHOLE house in order to move the CTs around seems a bit like taking a sledge hammer to pound in a finishing nail…

1 Like
35

Good point, and what Sense had claimed to deliver without needing all those additional pieces. Deep sigh.

36

You’re right. I had an electrician do the original installs, but have moved the DCM CTs around a bit on my own thereafter. I’m an EE, but not an electrician so I have used an abundance of caution. I realize that that’s not for everybody, but did want to share what worked for me since you were tossing around different ideas and seemed really dedicated to slicing up the Always On component.

37

Agree on the the need for the low cost wired and 240V monitoring device and integration with Sense for devices that don’t fit Sense’s range of detection techniques. But I do give them credit for having some flavors of slow-ramp detection because I have 2 slow ramp devices that Sense is detecting. This past week Sense natively nailed all my home EV charge sessions for both cars, though it is still underestimating my Model S charge power. Car charger data in orange, Sense native in green. The two “aberrations” are a charge at our getaway place (red circle) and stop at a Supercharger (blue circle)

Screen Shot 2021-04-18 at 1.05.01 PM
Screen Shot 2021-04-18 at 1.05.01 PM3034×2242 527 KB

So Sense has techniques that work for some slow ramp devices…

38

Hi @ATechGuy. Appreciate you putting your thoughts together. A few things I’d like to mention.

While @kevin1 isn’t technically a Sense employee, he is very aware of the underlying methodology Sense uses for detection and has engaged with similar threads during his 4 years as a Sense user. He’s also a Beta tester for Sense, so he’s aware of some things coming down the pipeline that may address some of your feedback.

As @kevin1 mentioned, Always On isn’t guaranteed to change by turning a breaker on and off (learn more about Always On here).
That being said, we recognized there was more we could do to enable users to find their Always On devices and are releasing something soon (within the next week) that allows users to enter in estimates for Always On devices. You’ll see more about this in the upcoming release notes, and we hope that this release empowers users to learn more about some of their “unknown” Always On devices that account for a significant percentage of their bill.

I can definitely understand the logic here, and brought this up as a question when I first started with Sense. There are quite a few reasons this isn’t a great solution, one of which being a lot of devices don’t cycle on when you flip the breaker. If you were to turn each breaker on and off one by one, you’ve created an unrealistic view of what your home looks like - you’re never going to have just “one” device running. Since Sense is measuring at the mains, it needs to be able to see that device with all the normal day-to-day noise that your home generates with multiple devices on. A better analogy might be the “loud, crowded room” - Sense is essentially in a “loud, crowded room” of devices “talking”. If you turn off all the devices except for one, then yes, we can “hear” it better, but that doesn’t make it easier for us to “hear” when all the normal devices in your home are on. It just allowed us to “hear it” in one specific instance.

Since talking to more folks here that have decades of experience in the machine learning and AI fields, I’ve gained a lot of respect for the massive issue they’re trying to solve. It’s not an easy question, and our team has spent years looking at ways to approach this problem.

I’d be curious as to what goals you’re referencing.
In @kevin1 original reply, he shared this video interview with our Data Science team about what was coming in 2021, specifically mentioning Progressive Device Detection. While this doesn’t mimic your suggested approach of turning breakers on and off, it does address a lot of feedback regarding providing more clarity into what’s using energy and will significantly impact the type of data users are able to see early on. On top of that, we’re going to continue with additional integrations and continue developing features like Dedicated Circuit Monitoring and Smart plugs to expand the types of devices Sense can connect to.

We’re going to continue to add to Sense, and I’d like to believe that a lot of users that might have criticisms about Sense today also agree that we’re constantly adding to Sense to make it better for everyone. While I see the next few years as pivotal for Sense, I’m unsure of what goals outside of “making it better for everyone” are more pressing.

1 Like
39

Thanks @JustinAtSense. The “goals” are referenced earlier in the thread–are they to do “Cool ML/Data Science” or to allow your users to view their electrical devices and their energy use. While the two are not mutually exclusive, it is my belief that the latter is more important to your users, while the former might have intrigued them enough into buying Sense, knowing that the alternatives ( other than a lot of smart plugs!) are certainly less “elegant”. I’m all for the “Cool” part, but am hoping Sense focuses on ALL MEANS available to them to aid in detection, including human assistance when it can be helpful.

I get the breaker off analogy. And I’m sure I’ve read in these forums a bit about the crowded room analogy as well, but that Loud Guy in the far corner that we all hear whether the room is crowded or not is still quite LOUD. I fully understand that his voice, when mixed with the rest of the party will could look different, but if you overlay his “signature” voice on top of a party, his signature is still there. it’s just a LOT harder to identify.

I also get that turning on a circuit and subsequently one or more devices is likely to yield a situation that might not be common during several days of normal use.

So let’s try another tack. If humans were able to click on a “+158w” spike that ALWAYS happens (+/- 2-5 w) whenever you turn on a circulator pump for the forced hot water, Would that help Sense if we could label/identify that spike as CIRCULATOR!!! From my naive view, that would tell Sense to STOP looking for ANY OTHER APPLIANCE SIGNATURE whenever it sees this device turn on/off. Can it see the device turn off? The human can, and just as when it turns on and labels it, the human should have the ability to tag the drop as the SAME device now turning OFF!

I’m curious if the Sense engineers have a good “sense” (pun intended) of what most Dryers, Ranges, Hot tubs, etc “look” like. I would be blown away if this weren’t the case. And if it weren’t the case, and it’s not proprietary info to discuss it, then just what information does Sense keep in order to identify devices? Again, my naivety but I would guess that resistance heaters are similar traits, just as motors or EV chargers or light bulbs, or power supplies do. How else does Sense factor in the % of likelihood when it detects a new device? So, it’s only logical to believe that identifying power changes when a KNOWN device turns on/off, to tag it from the UI in hopes of informing Sense that it no longer needs to work on deciding WHAT KIND OF APPLIANCE that change is associated with. It doesn’t mean it will immediately know how to track that device, but I have to believe it cannot hurt the effort.

@kevin1 in the Beta program? Makes sense to me. He’s committed to the success of Sense, just as I am. And it’s very clear he’s been educated thoroughly on the difficulties of detecting devices. So how does one get into the Beta program? And what % of your user base is part of that program?

2 Likes
40

I’m gonna share a few thoughts, based on my experiences with digital signal processing and machine learning, even though I’m not a Sense employee.

  • Detection of on and off is all about finding a “signal” in a bunch of noise (other devices turning on, off, and ramping up and down)
  • In the case of Sense, the “signal” can have a very broad spread in the time domain. Some on-off transitions take place within a few 60Hz AC cycles, some take minutes (car chargers). The short ones have waveforms that generally conform to basic physics, but the longer ones are dictated by programming and electronic control systems. To stretch the “party discussions” analogy, we have people talking many octaves apart and in different languages. So a listener that can only hear the high octaves (frequency) might not even hear the proverbial loud person you are referring to. And even if they hear them, they don’t understand the language.
  • Works the same way for electronic signal detection. One has to have a sampling window or base frequency for signal detection. The original Sense window was that less than 1 second transition where the physics dominates. There, on/off transitions are detected and analyzed for maybe 20 different features that could distinguish them. But with a short detection window like that, Sense never “heard” the low frequency ramp up or down of EV charging or variable motor control, even if it was the loudest overall. So Sense has evolved new detectors that are more attuned to these long ramp devices, but the “language” (or features) that distinguish these longer ramp devices is very different.
  • Hearing a device once, even in isolation (no other noise in the house), isn’t enough to learn and classify it. It takes many “hearings” under a wide variety of conditions.
  • Just to make matters more challenging, there are many types of devices that don’t have simple on/off responses. TVs, computers, and washing machines with DC motors, all jump around in their power usage when being on - the power usage depends more on what they are doing vs. on/off state. All good candidates for smart plugs, though the washer might have enough pattern regularity to eventually be detected.
  • And finally the notion that Sense can somehow check stuff off the “device list” or “cancel out” devices that it has already detected to simplify the task, is erroneous. No way technically to do that. Even though something like subtracting out discovered waveforms sounds attractive, a time domain subtraction, assuming one could get the discovered waveform exactly right (an impossibility), would alter the frequency domain characteristics of all the other waveforms embedded in the mix (there are multiple dimensions of data embedded in the Sense data sampled at millions of samples/sec)
2 Likes
41

I’ve no background in machine learning (do have generic AI experience), so I find your explanations both interesting and valuable. Thanks.

Sense engineering has clearly tackled a huge/complex problem. I give them lots of credit for what they have been able to do, apparently it does work for some environments and some devices. Looking at my home’s signature on my (vintage) oscilloscope, it looks like a stadium full of people shouting/singing/playing instruments in hundreds of languages. I have looked at individual devices with my own current clamp, and things like the Kenmore refrigerator alone are very complex, some spikes, some ramps, some always-on. I’m not surprised that Sense couldn’t reliably detect that, thank goodness for my HS110’s.

My only faulting Sense is that their corporate execs/marketing seriously over-sold what they could deliver and they continue to mislead the public with their claims…buyer beware. In my case, Sense only detected about a dozen of my (approximately) 177 devices, and those not reliably. That’s not a small difference between expectation and reality.

1 Like