How firm is Sense’s stance on no device training?
I really don’t think you’re going to get where you want to be without it and it should go a long way in hastening individual household device identification and building your database.
As data scientists will attest, at some point more data of the same patterns doesn’t help. At some point new inputs are required in order to move things along.
Based upon my research on the input here and feedback from Customer Service I reset my Sense after 4 months. As of the reset I had 35% identified. Not good enough. I love the idea but the execution is poor. How many $300 devices do you buy with the goal of having 35% functionality after 4 months and no clear expectation of it better better (working on that!). Kinda cool idea but I’m sure other devices are in the development stage that will do this AND offer control via an inexpensive in line device. Frankly I’d pay $600 for that.
@mstraka606,
A few questions, since I’ve been thinking about doing a reset.
I’ve looked at other devices, but I see deficiencies in them all, at least for US installs. For instance, the Smappee seems to be set up for all kinds of single and multi-phase systems worldwide, but in my mind it has a fatal flaw - it uses a wall plug to power the monitor and to sample voltage. In the US, that means the Smapee would only be sampling one of the two legs, and have to make assumptions about both the voltage and phase angle of the other leg, potentially leading to significant accuracy issues.
Right now, though I’m not yet enthralled by Sense’s ability to disaggregate and detect (especially when compared to the “ad” video ), I am impressed by the accuracy of it’s solar and aggregate usage measurement vs. my revenue grade PG&E meter. I have also discovered that my solar inverter is not “revenue grade” and runs about 5% more optimistic than what my house and power company actually see.
Ive sent various emails to Sense about the early adoptors and we would advance the learning of their product faster. Im into 5 months and many items are not detected. I also have a list of things it has found and it is a guessing game as to what they are. I do jot have time nor the interest to play hide and seek at this time of my life.
The devise is also nearly 100% usless for trouble shooting high bills. It would need to be able to find most of a house in a week or less.
If given the ability to mark items we could have most of the house identified in 1-2 hours. I’ve spent more that that on hide and seek and still have many items not found or sense found them but i cannot figure out what it found. ):
Knowing just enough about machine learning to be dangerous, I would suggest that “marking items” is not all that helpful for learning detection, at least until Sense has the fundamental models for the dozens of different types of load devices in a noisy environment pegged. Only after you have effective models, can you use prediction and back propagation from actual results to adjust weights in the network.
&kevin1 I just reset the device today so I’m sorry I can’t give any feedback on the results…yet. But I do note I’ll have to wait another week for the Sense to calibrate…oh, goodie!
The 35% I mentioned was the found percentage; meaning 65% was unknown. I have an i3 and the charger wasn’t found. The clothes washer and dryer were not found…but the auxiliary exhaust fan was! Total randomness. I can’t agree more with those who recommend some form of device input. I’ve recommended a lookup table of know devices (why isn’t it helpful to know what the make/model/serial # is? Anyway I’ll update this post after I have news to report. Hopefully it’s not going to take another 4 month.
Achpinc, your comment doesn’t make sense. You have time to train devices but not “play hide and seek”? Training devices would take substantially longer. Your post seems to be more in favor of the current model when listening to what you are really saying (I’m a busy guy) vs. what you are asking for.
That said, I am all for the ability to train devices. I’m busy, but it would be fun and I’m eager to do “something” to feel like I’m not just at the pure mercy of machine learning.
Even if at the very beginning I could list my items Sense would know I have x, y, z, 1, 2, & 3, as opposed to trying to first try to guess what parts there are in the equation. I’m not even talking about EVERYTHING but at least the big stuff, washer, dryer, TV, fridge, etc.
As it stands now, I’ve got my AC, Dryer, Dishwasher, Microwave, Garage Door, Master Bedroom Closet light (super random), and Fridge. It’s still missing my washer, electronics like TV, desktop computer, entertainment center hardware, and all my lights (minus my master bedroom closet). I can clearly see when these things go on in the “other” bubble, sense just hasn’t pulled them out of the hat yet.
What’s really frustrating is that I have a clean house electrically. I typically can get my “always on” down to less than 100 watts and generally doesn’t go much over 3 or 4k at any one time. I’d expect it to be able to figure it out a bit more easily than it does.
There’s a lot of value in sense for my case. If I can keep my monthly electricity usage below 1,000 kwh for the month, my electricity is $.0002 or .02 cents (practically nothing). Over 1,000 kwh it goes up to 9 cents. For the last three years I only go over three months each year - summer with the AC. Two are close, in the 1,100 kwh range, and if I can get one or two of them down the device pays for itself. However, I really want know what my washer is using. Cutting a load here or them is doable, but before I do it I want to know it’s going to have an impact in the larger picture.
Just to add my 2 cents about stalled device detection - I’ve had my Sense installed since February and I plateaued at 10 devices discovered. Only about 3 of those worked reliably. No new devices for weeks. I went ahead and reset my unit a week ago and since then it has detected 8 devices including 5 which it had not found before. It has even found a 27 watt motion activated security light in my backyard which I love being able to track on times. I’m wondering if after a while, the found device signatures are masking and slowing down discovery of new devices? It’s a big step resetting your device. You lose all of your historical total usage data. I’m wondering if the same thing couldn’t be accomplished by deleting all of your devices individually and starting from scratch that way. Or if somehow your historical total usage data could be preserved through a reset?
Day 3 post “reset” still calibrating…
@dek8ce, as I said in another thread:
We definitely want users to be involved in the device detection process. If we felt that manual training would a good use of your time, we would work on implementing it (see: How Sense Learns About Your Devices - Sense Blog for more information).
That being said, we are looking for more ways to have users involved in the device detection process. For example, we recently added the ability to let us know when a ‘Device is not on’. If you see a device on when it shouldn’t be, you can mark it as such (in the device details screen under ‘Report a problem’). This information is relayed to our data science team so that they have more information to improve our detection algorithms. Renaming devices is another way to be involved in the device detection process.
We are working on more features along these lines so that users can help our data science team and our machine learning algorithms improve device detection.
Thanks Ben. I think we’re a bit more aligned than you think - there are various levels of what “training” entails. I’m not talking about training in the sense of “turn on a device, click learn, rinse, and repeat.” I’m more talking about identifying the equipment I have in my specific home, or a more interactive Q&A as Sense detects changes.
Unlike the cat videos your blog post, electrical engineering very much follows a consistent pattern - two exact devices in my home exhibits the same combined electrical signal in my home as it does in yours (in isolation). Obviously there will be minuscule changes, but they’re not significant.
That said, if I was to list all my major electrical devices it makes the algorithm that much more quick and accurate. It’s easier to solve an equation with 12 unknowns than it is to have to first whittle down thousands of unknowns and hopefully get it right. I’m seeing quite a few threads in which people are being asked to reset their Sense. I’d imagine this is because the algorithm made a mistake early in dropping off or adding an unknown that wasn’t real. Mistakes early in the equation means everything else is questionable.
I’m not sure of your background, but I’d really like to get a data scientist’s input on how knowing that I have one refrigerator, one garbage disposal, one A/C unit, one garage door, one microwave, a GAS heater, HW system & stove, etc. wouldn’t help. I can tell from other posts that Sense is at times having trouble identifying all of these systems.
I’m really struggling with how telling Sense some things to look out for is a bad thing and would not be helpful. Obviously you feel that this is NOT helpful… I’m wanting to know why. From the looks of it, others are too.
Very true. It seems like, if I don’t have an EV, Sense shouldn’t waste time looking for it, and shouldn’t guess I have one if I don’t.
I just don’t know how to practically implement such a thing on either side. Have a page during set up where you check boxes? How do the engineers effectively limit what to look for in your house? Is that too much of an invasion of privacy? “Grow lamps x 20.”
Ah I see! Sorry for the misunderstanding on my part.
Letting us know what devices are in your home, so that we’re better aware of what to look for, is not something we’re opposed to at all! It is a feature request that we have gotten before and is something we will be looking into. We all have the same goal: improving device detection. The team is working hard to achieve that goal through improving models and adding features to get more user input. Your suggestion is something that we want to look at and I’ll be sure to share it with the rest of the team.
Better yet, I think Sense would be a lot more effective if it was prompted for what to look for. I could list what accounts for 90%+ of my usage in less than three minutes. I could see this being a very quick survey at the beginning to include the number of major appliances and installed systems and their types (gas, electric, solar, whatever), with an option at the bottom to “tell me more.” Then you can put in data like how many rooms, their names, and other details like what kind of lights, is there a TV, anything else in the room, etc.
This would suit those that want to invest minimal time - one or two minutes to list major appliances - and those like me who would like to invest 15 minutes if it allows me to get Sense really dialed in. This could easily be included in the initial set up wizard we followed when installing the device… I mean when the licensed electrician installed the device.
I always say “installed by a professional,” because, well, I’m a professional, you’re a professional. Ergo, “professionally installed.”
I could list every electrical thing in my house in 15min for sure. 15 categories or less.
To add to that, as the end user we can specify brand, model, etc. I would think that such information can only help the detection team.
Completely agree that this kind of feature would helpful and it is something we’re going to look into adding. Really appreciate the input!
Y’all have about three weeks of my usage information. Perhaps you can pick a few “involved” people from your boards and see if they’d complete a worksheet and then your data miners can see what happens if they would have had that information from the start.
I have been putting the model number and mfg in the description block for each device as I edit the Device entry. Wonder if this info is available to the Sense team?