High energy items not being identified

Device Detection
#1

I’ve been using Sense about a month now and have been reading the community complaints about device discovery. I fully understand that device discovery takes time and that as the article stated you can’t “isolate” devices to learn them because the signatures vary when paired with other devices on a circuit. Also, a devices advertised wattage is a ballpark as many factors can make that fluctuate like age of the device, quality of components used/available at the time of manufacturing, etc.

Where I’m struggling to understand the device discovery though is with a heating system. Since it’s been cold enough my furnace (aux heat) has been running on our heat pump system. Despite regular occurrences like starting at 5am, and consistent electrical draw, Sense has still not identified the device. Seems to me that something like this that may not be directly identified, but causing regular usage should go into a sep category as opposed to “Other” so it could be identified by the user. Before you reference me to it, yes I’ve read about why you can’t train Sense and I understand that. I’m not talking about training, I’m talking about Sense learning from the community based on rhythmic cycling of a device - mostly of high power usage - that give a stable signature but not fully recognized. Maybe this falls into the training category, but I think it would be beneficial to have a pop out bubble for high power devices like furnaces, heat pumps, AC, and even full size office printers so these aren’t lumped with other. This would give a greater range of learning capabilities without going into the “training” aspect and still allowing Sense to learn from the ML algorithms.

This would also help the community of users see that Sense is discovering and recognizing a signature developing, just not clear enough source data to identify it. When lumping into a category of “Other” it’s not clear of Sense is picking up the device and doesn’t recognize it, or the signature isn’t stable enough to identify the item.

That being said, I love being able to watch the system build tho, it’s a great tool to help identify where you can reduce power consumption in the home. Thanks to everyone at Sense for a great product and their support. Had a question with signal checking after a reset, and got a timely response! Keep up the wonderful work and I know it will only get better!

#2

You bring up some good points here ! There should be a way to detect and categorize device waveforms that don’t fit the Sense “instant detect” transition model, but have some kind of distinct regularity / periodicity.

At the same time, you seem to assume that some kind of algorithm should be able to find these patterns, but quite honestly, research in machine learning is just beginning to evolve models that can do this - I have a son that is a researcher in time series models. Most of the existing time series models, like ARIMA, aren’t well suited for categorization, plus they aren’t particularly good for data that has multiple periodicities and patterns embedded. Recurrent Neural Network approaches, like LSTM, could work but they are limited in terms of the length of data history they can reasonably accommodate.

There are new time series models coming into their own based on an underlying Transformer model. The Transformer model is much less processor and memory intensive than a comparable RNN, using an “attention” mechanism to identify patterns and decode longer patterns over time. But these kinds of algorithms are just now making it out of research and into initial use.

ps: just realized that this might seem like tech mumbo jumbo, but when I wrote this, I had just gotten a lecture from my son on the next gen of time series modeling.

#3

Perhaps I’m way off base and not understanding something, but what if there’s an option in the Sense app that when a high draw item like dryer, office printer, furnace, etc are detected a notification is given to prompt the end user to input the information of the device if they know it or are able to enter it. We already have the ability to indicate what devices we have up front before Sense gets to working. Why not carry it into triggering events that create a flag.

When Sense has that detection and the user is prompted for the details of said device, should it be known, the user can enter it in. Then, the wonderful crew at Sense can disseminate the logged event information looking at the loading values and any other information of the device and add that to the repository of known devices once the pattern has been seen after “X” amount of cycles from "Y"amount of users. If timing isn’t right, the user can mark the device for details at a later point. It would be a great way to make the app work for the community beyond device detection and entering details after the fact of whether it’s right or needs modification.

Just a thought and my 2¢ to try and help out :grin:

2 Likes
#4

Excellent idea. Would be great to see!

#5

@goalieman86, you are thinking the right way, it’s all about event detection and classification, both on and off. A few things to think about:

  • There are events you see in the Power Meter, then there are the events Sense “sees”. Except for special cases, like EV charging ramps and some heat pumps/mini splits, Sense only sees very short (like 1 second) on and off transitions. I believe that if you look at the Power Meter in phone/tablet app, you will see Sense tag every events/transitions it “sees” with a power number. Look at the transitions tagged in the Power Meter as the only thing Sense “sees” with its primary eyes.

image
image2532×1170 217 KB

  • Seeing / tagging one event is not enough - Recognition is needed. It is not useful to tag a single event that Sense “sees”. Machine learning has be able to be able to “recognize” a bunch of similar on or off events as distinct and unique, vs other transitions, before it can begin recognizing something. I often use the photo program face recognition analogy. It doesn’t help with face recognition, to tag a face, until the program has pulled together a bunch of faces. There is a school of thought (supervised learning) that says if you could label a whole bunch of these things that Sense sees as what they are, that could also work, but supervised learning tends to take hundreds to tens of thousands of labeled examples, something most users can’t reliably deliver on. A good discussion in this video about how recognition is done and why Sense chose to go the “unsupervised learning” route.
  • There’s also the challenge of pairing up on and off transitions.
2 Likes
#6

I’m all for machine learning, but I’ll wager a quarter that the current owners of Sense are VERY, VERY interested in understanding ALL of their devices and are WILLING to HELP SENSE as much as they can. I get the feeling that the developers wanted to accomplish something VERY COOL, but eventually found it to be a LOT HARDER than they original expected. And from that original design platform, my guess is there was no thought on how the machine learning (ML) could have a human “tutor” it in the lesson plan!

As mentioned earlier, there is already something in the app where we can enter data on devices. Frankly, this could be thought of “cheating” (if we continue the “learning” references) but others might calling it “tutoring”. Here’s how it could work:

  1. User enters ALL the devices they can think of for their house, entering as much data as possible for each device, especially the heavy hitters. This includes model numbers, etc. During this entry process, the SENSE app could look into its database of devices and see if there is a match or something close. If there is a match, then the app would fill it in and note it as a match. If no match, then free-form entry would occur. (but it still would be categorized!)
  2. Whenever the user notes that any of the entered devices “just turned on”, they could tap the meter display and find some pull-down that would allow them to identify that change in wattage with a labeled device. Is that enough, most likely NO. But they can do the same for when it goes “Off”. And everytime this happens, if the human makes this identification, it would seem to me that SENSE should use this information (MY toaster just went on, just went off, just turned on, just turned off. ) that the tuturing should inform it that, hey. LIMIT the SENSE ML during each of the identified actions to TOASTERS!!!
  3. When SENSE decides it has detected a device, and it brings up the display of the choices, it should start with indicating it most likely is a Fill in the category and allow the user to find a drop-down menu of THAT category for which the human filled in the product(s) .
  4. If none of the devices appear correct, there should an option to “Do more homework”.

For my experience so far, perhaps I’m typical, maybe not, but my first detected device was the “Stove”. Two days before, I had entered the EXACT model stove I have, and since I had filled in that in devices, SENSE should have known the type of Stove I have, and if it had any signatures of this stove/range in its database, it should have compared them, in this case, rejecting the possibility of the detected device being a “stove”. In my case, the device it detected was my EV (M3/AWD using TMC). And not only was it wrong, but soon I found that even though it would report the EV as “turning on”, it was only showing the draw to be around 2KWh when in fact, it was clearly more like 7.5K and the text alerts happened precisely when charging started and stopped. In my case, perhaps out of naivety, I decided to delete the device and have SENSE go back to the books.

From reading many others experiences, It would wonderful to hear the SENSE product management and engineering teams fill us all in on how we can IMPROVE detection, rather than waiting for ML to magically do it’s stuff. It might be super-cool if it always worked, and worked in a month or less, but having the ability for the humans to “tutor” SENSE, even if it requires several “lessons”, I think we would all become much happier with the product.

1 Like
#7

@ATechGuy, I’m betting the percentage of users that would love to “tutor” Sense on their specific home devices is even larger than 25%. And you are right, there should be tighter integration between the home electrics inventory one enters and the device info that gets filled in when a device is initially “detected”. FYI - the history is that the home device inventory came much later in time than the detected device form, and is there for a a slightly different reason (understanding the device population of the customer base - my end-user understanding) so there’s still some integration to do.

That said, I think your thoughts and intentions are right on, but you make the same tricky assumption many users make about how Sense does its job and what is technically possible to infer based on the just the power change during a transition. Watch the video I posted a couple times - that’s the Sense CEO explaining why they took the approach they did, especially unsupervised learning, vs supervised (using tagged examples as you suggest). And Sense do offer some pointers on how you can help.

#8

Thanks @Kevin1. Despite trying to find more forum posts on this subject, I am finding some now from you and @RyanAtSense. I know that I am not just asking Sense to pick out a person in a room found in a home in a party with hundreds of other party-goers. I am looking for the ELEPHANT in the room. Everyone at the party knows who I am talking about when S/He talks–you can hear her/him from any other room. You KNOW when they start talking and you KNOW when they stop. (It’s not my analogy, it was Ryan’s.) And, even if they start slowly, the amplitude of their “voice” is significant and as a result, all the other conversations are drowned out. That’s the guy I want to start with for identification.

My apologies if I’m simply repeating what the last 100 new owners have requested, but perhaps Sense needs to hear this 101 times? If finding the elephant requires turning down the sampling frequency or the sensitivity–so be it. For nnyone who has read enough of the Sense articles, we GET that the leading and trailing edge of the power spike is not enough to identify a device, but toss a human in and perhaps we can get some synergy. Find me the top five SPIKES that last more than 30 seconds and we have something to talk about. My home? 1) EV, 2) Hot Tub, 3) Living Room space heater, 4) maybe the stove, and 5) The 1.1Kw consistently on “other devices”.

Ultimately, I’m hoping ML will dissect #5. But I can be patient. The first four? C’mon. Play dumb and out of the list of the top 20 domestic power suckers, GUESS! It’s OK to be wrong and the human can “suggest” the correct device (type) from that top-20 list. And most likely, it can even tell Sense the Make and Model #. Then it’s time for sense to “subtract” #1 from the party and move on to #2. and so on.

In my case, I consume just under 60 KWh per day and from the looks of things from Sense, my “floor” is about 27KWh/day for the “always on” stuff as I have yet to see my home under 1.1Kw. That leaves the remaining 55% for the “top-four”. It’s not trivial in $$, and when you’re dealing with 240v devices and decent-sized circuits (#3 is actually 110v), your power costs can get up there. Finding those devices with or without a human should be job #1 and those devices are part of my daily life so correctly identifying them and separating them should be accomplished in less than a week, if not a couple of days. Then, I can chill out and watch in awe as the ML slowly pulls the other devices out of the “other”. To get a “sense” of my frustration, SENSE seems to keep detecting what it believes to be #4. Problem is we can go days without turning on #4 (the stove). And coincidentally, it seems to detect this device whenever I am using my Tesla Mobile Charger. I guess I have to keep telling Sense “not on”. Wrong Elephant.

Sigh.

#9

Hi @ATechGuy - the resources you read referenced @RyanAtSense, who has since moved to our product team, so I’ll be your contact in the Community moving forward.

The first thing worth watching through would be the most recent video we did with Data Science. It mentions quite a few things that shed more light on device detection and what we hope to do moving forward. This includes parts on your comment regarding stove detection and the role we hope home inventory will play in the future.

We released Dedicated Circuit Monitoring last year to enable instant detection for up to 2 120V or 240V circuits. While that doesn’t take all 5 of these off your list, it can allow you to focus on the two highest consumption devices. I know this doesn’t completely solve your issue, but between a couple smart plugs and dedicated circuit monitoring you could probably take a serious chunk out of your Other bubble. I reduced my Other bubble with 1 Kasa HS-300 smart strip by 250W immediately by putting it on my office/desk set-up.

No apology necessary, but maybe add a couple extra digits here :slight_smile:

We talk about this in the video above, towards the end regarding Progressive Device Detection. There are other releases planned for this year focused on helping users understand their energy usage.

In summary, we’re definitely aware of some of the obstacles for users re: device detection as it is today. The one thing that I think every Community member would agree on is that we’re constantly working to improve Sense. Behind-the-scenes, we have quite a few thing that I’m excited to share with you all once they’re set in stone :slight_smile: .

1 Like
#10

Your “device monitoring” isn’t helpful for those of us using that circuit for solar (as originally intended). And it’s pretty clear that some of these other devices aren’t going to work with the Sense architecture.

What you need to do is to partner with someone to make an in-line (rather than plug-in) 240v monitor that works with Sense like the HS110, HS300, et al do. I’d gladly buy a few and get them wired into things like dryer, hot water heater, stove, heat pump, etc. That would allow Sense to accurately monitor all the “big users” in my home. Since Sense already has most of the software in place (all of it if the new device is HS- compatible), this should be a great solution.

1 Like
#11

We’re aware of this gap for Solar users right now. Dedicated Circuit Monitoring was a step towards figuring out how we could approach this with our current hardware. We hope to do more with this, and our long-term vision is aligned with the understanding that future homes will more frequently have things like an EV Charger, Generator back-up and/or Solar packaged together in one home.

Our Product team is always looking for more third-party integrations though, so if a product comes to market that fits our integration requirements and can report energy from 240V loads, we’d definitely explore it.

1 Like
#12

@ATechGuy, appreciate your discussion and really do feel your pain. I have 3 EVs (2 Tesla’s) plus some Sense-resistant (or inconsistent) 240V heating loads in my house, and for long while the detection of my 2 AC compressors was fairly dodgey. I have gotten a much better handle on most of these as Sense has improved AC detection (I give the Ecobee historic integration some credit), plus my addition of second Sense solely for DCM (I have Sense solar, so I had to commit an entire separate Sense unit to just 2 circuits of DCM) to monitor my floor heating subpanel plus the Model 3 on HPWC.

But I have also spent a fair amount of time poking my nose into trying to build my own EV detector, because it seems like it truly is the big elephant in the room. But when you get inside even that simple problem it is harder than it looks for a few reasons, especially when dealing with non-machine-learning algorithms.

  • the viewing window - searching for transitions requires a defined analysis window and some notion of a trigger delta. EV charging ramps don’t register as anything within Sense’s 1 second or so standard viewing window. I used a 15 min window for my (rough) detector, because I have that data window available to me. Our brains are “wired” to be more flexible so we can “see” 1 sec and 15 min elephants in the same go, but they have very different visibility to a machine, depending on window.

  • overlap - as your detection window widens, detection becomes more sensitive to overlapping transitions. Multiple on transitions and off transitions can get blended together. I had cases where two heaters turning on in the same 15 min period looked like the Model 3 charging.

  • differentiation between on and off patterns - I got my detector partially working, but then had to differentiate between the 3 and the S, because eventually detection has to match up ons with corresponding offs. Even with only two patterns, it was more difficult than it seems with a 15 min window, for several reasons. How do you tell a short charge (less than 15 min) apart between the two, or both ? What happens when you miss an on or off ? And the permutations go up exponentially.

  • EV charging profiles change over time - Sure, if I was Sense I could have build a more “sensitive” detector that used multiple narrower time windows and unique aspects of the Model 3 and S charging curves, but all of that gets thrown out the window when the charging profile changes, which seems to be every major software release for Tesla.

Based on my experiences, I sympathetic to Sense’s challenges, even for the detecting the elephants. If you want more details, here you go:

#13

And here’s a test to highlight challenges, even for human detection of elephants.

#14

If I ever get past the “needing approval” point, I would have edited my previous post on the SENSE not having the jack. I read further on the dedicated circuit monitoring page found it here in in step 8 I found my answer. So, no need to reply to that part, but Justin, please do still comment on MOVING the dedicated sensor around once the devices are “discovered”. Or, I think I get this, perhaps you don’t discover ANYTHING on those dedicated sensor, but simply report them as “Ded-1” and “Ded-2”. If that’s all you do, then I can easily expect others to chime in on “Why didn’t I just buy a dedicated circuit monitor like eGuage…?”

#15

Hi Justin,

I super appreciate your comments. My SENSE is brand new. It reports version 35.0-f8afe9db, build 1531. I don’t recall my unit having the “center” wire/socket. Did I get shipped an older version? I like your Dedicated Circuit idea, especially if I could use that for 1-2 circuits, and then possibly move it to another circuit.

For me, I didn’t have a free 240 breaker, but I did have one that wasn’t using a lot of power and chose that one to tap onto. The next time I go into the breaker box, I can verify the SENSE, but presumably, if I HAD the option for the “middle wire”, then not only would there have been a jack, but also a wire included with my SENSE. Or, perhaps the socket is there, but it requires the “flex add-on sensors?” Certainly, if that’s all I need, please talk to us about moving around those add-on sensors once detection is made. One never likes opening the breaker panel often in any case, and for me, I have five 240 circuits (in use). (EV, Stove, Sub panel for my office area, Hot Tub, and breaker for an X-10 bridge, from which one half powers an Eaton UPS/power filtering circuit for servers and the other half is not used (other than for the bridge). My 240v dryer circuit is unused as I have 110v dyer (with gas).

1 Like
#16

Great reply. I know I’m trying to trivialize this and i need to stop that. Of course, like any new user, I’m only trying to help improve the neural network.

Sticking with the elephants I understand that informing SENSE that there is an M3/AWD hung off a TMC is only useful if it can detect there might be an EV lurking on some circuit. That’s the part that is really hard for us noobs to comprehend. Instead, given the info I’ve provided I want SENSE to go elephant hunting! And not just blindly but it should pull out the M3/AWD/TMC signature info and adjust the scope appropriately. That’s all I’m asking. I know it still doesn’t make it easy but I would hope it could make it less difficult to find.

1 Like
#18

That is correct, the HS-110 and its relatives don’t “discover” anything, and what they report to Sense is what you label the power they monitor. They do not teach Sense, for a number of reasons that are discussed throughout this thread. If you disconnect and move those smart outlets, the name you assigned goes with them unless you go thru a fairly difficult un-assign and re-assign to the new device process.

That said, they are VERY useful (if costly) for all those devices Sense can’t detect. I just wish they had a similar device for our large 240v consumers. That would be just like the HS- series but would need to be installed in the lines because so many large 240v devices do not plug in.

2 Likes
#19

Can someone suggest the best method to respond to SENSE when it incorrectly identifies a device? Mine keeps identifying the “Stove” when I haven’t been near the stove in days.

#20

Hi @ATechGuy, I wrote up the below response to this question in a related thread.

1 Like
#21

Newbie question probably but I have had Sense for a couple of months now. I understand it takes time to discover devices but I have a couple of big ticket items it still hasn’t discovered. I charge my Tesla each evening at the same time and I also have a steam room. When I look at the meter it jumps up 12-13,000 watts when either are in use. Why would it take so long to identify? These are the only 2 items in my home with that kind of power requirement. Would love to be able to tell Sense what these items are. Overall, however, I am very happy with the product and it does give me great insight into my home.