I had wondered why I was getting (custom) notification alerts for my aquarium heater.
Discovered that Sense is now getting my pool pump 3/4 motor (single speed) confused with a 300w aquarium heater a resistive heating element. And not at all close to the actual wattage.
The motor has not been natively detected, so previously it was an Other.
The motor clearly has a huge startup spike.
The heater was one of the first things detected by Sense and had been quite accurate for years. It was one of about 4 natively detected devices that was somewhat accurate.
Guess Iāll need to add a 41st Kasa plug as Iāve completely given up hope Native device detection will ever progress to being even a remotely useful feature.
4 Likes
One of the interesting things that Iām beginning to see in the new machine learning / large language model world is that as the machines get āsmarterā, they can also lose part of their smarts due to the learning process. Why ? Because machine learning is essentially a process of adjusting millions of thresholds that determine recognition of things, and is driven by minimizing overall āerrorā across recognition of everything. So add some recognition here and you might lose a bit there.
The video in this article shows how fluid the learning process (vision recognition using transformers) is and how the thresholds change as the recognition sees more examples and gets smarter overall.
And even the most brilliant people doing some of the most incredible stuff with large language models are having challenges with backsliding and drift.
Certainly not satisfying to see in real life, but a reality of the technology.
Kevin, thanks for the links. Understanding machine learning is above my pay grade, but to boil it way down, your point is that as machines learn, they change. This is logical, of course, but may be missing the point when applied to Sense detection of devices. Here is why.
The complaint was about devices that have already been identified being lost. Machine learning is about finding new devices in the noise. You might expect that once the artificial intelligence in the cloud has tagged a device and provided its parameters to the local Sense box, the learning part would be finished.
However, the way Sense is currently written, the AI in the cloud still occasionally intervenes and passes new parameters to the local Sense box. In principle, this is to fine-tune the device definitions and improve performance. The complaint is that in practice, sometimes those tweaks make things worse.
I have seen this myself. My water heater is on a Kasa plug and also has a native detection. The link below provides a graph of the relative performance of those two. Blue in that graph is where the two agree, orange is one kind of difference, and yellow another difference. Something changed around week 20 that decreased orange yet increased yellow, leaving blue mostly unchanged. Another change around week 44 decreased yellow but sent orange through the roof and blue towards nothing.
It would seem logical that before the AI in the cloud passes a new definition to the local Sense unit, it should first compare the performance of the tweaked and original versions. The new version should only be loaded into the local unit if it is actually better. Such a comparison would be its own layer, thus not subject to drift in machine learning. This comparison step is apparently not happening, which is what I take to be the point of the opening post.
For curiosity, I brainstormed one implementation of such a comparison two years ago in this post:
2 Likes
I had to edit my original post as I stated the heater uses about 500w when I meant only 300w.
Machine learning is also way out of my league as well.
Getting smarter is a good thing the problem is that I havenāt seen any examples of that. At least not in the past couple years in relation native device detection. Progressive Device Detection was supposed to immensely help but itās implementation has been delayed for over 2 years & counting. No other improvements to detection have been announced nor a significant number of users reporting any improvements. Many have reported the opposite.
Having more data and processing more data doesnāt equate to more intelligence.
Equating it to humans some of the most āeducatedā people on the planet have the least amount of common sense and are completely perplexed when it comes to basic thing. While the āleast educatedā often make the biggest discoveries, most useful inventions and make things happen.
Much like those āeducatedā humans lacking the ability to reason using common sense AI can only do what it is trained to and nothing else.
In my above case the AI (as trained) doesnāt have the spatial ability to comprehend that a huge spike followed by 600w or so canāt possibly be a 300w resistive heater with no spike.
1 Like
Not quite. Nothing discovered has been ālostā. Sense still thinks my heater is my heater, but often also thinks the (undiscovered) motor is also the heater.
As if Sense is trying so hard to not miss a heater detection it forgets to look at the bigger picture.
My pump motor has huge spike and variable wattage, not variable as in variable speed as itās a single stage motor, but very slightly based on discharge, suction pressures, filter blockages, etcā¦ Whereas a heater is simply on/off no spike. The pump also uses about twice the wattage of the heater. So about half the motor wattage shows in the heater bubble and the other half not as anything. When both are on the heater bubble stays the same. So thatās some pretty big convolutions. *both are 120v
1 Like
That is a little weird. The signature for an AC motor typically has a significant inductive component to it, hence the spike. As you suggest, the aquarium heater should be mainly resistive though anything that includes coils of wire will pick up a slight inductive component. I do wonder how those got conflated.
Thatās what I donāt understand.
The aquarium heater was one of if not the first natively detected device and pretty accurate since June 2021 when I last reset Sense.
I had the pool pump on a DCM until this year when I repurposed it to my new HVAC system (that was a month long issue itself, but the DCM is now working fine).
The pump motor had not been natively detected. For most of the year (summer) the pump fell under other, but recently some of itās wattage has been counted as my aquarium heater, the rest of the wattage not being attributed as anything.
We finally got around to closing my pool so I wont have to worry about how it is counted until next year.
Maybe Kasa will come out with an outdoor energy monitoring plug by then.
Iāve already had to put Kasa plugs on most of my other native detected devices due to issues with inaccurate detection. Fridges, washer, dryer, dehumidifier, laser printer, etcā¦ Things with multiple components is a bit a bit more understandable.
In the meantime Iām going to have to put a smart plug on the heater because I no longer trust the accuracy of Sense detecting the most basic of things. Now that Sense is starting to conflate things at completely opposite ends of the detection spectrum Iām continuing to lose the little confidence I had left.