Venting: Theres a serious lack of data integrity when SENSE gets something wrong

Prefacing this by saying I love SENSE, and have enough enthusiasm about it to contribute as a Beta tester, but I need to get this off my chest.

There are plenty of users who have more knowledge and experience than I do. I would be grateful for any input you might have.

TLDR: Historical data leaves a lot to be desired from the moment that SENSE gets anything wrong, or an add-on integration needs to be reset. And it will get things wrong on occasion.

I’m not the first to complain about this, and now I understand exactly why its so frustrating.

One oversite by SENSE in its product and platform design is the lack of contingency, and reversibility. In my 8-months of using SENSE, this has lead to 2 irreversible problems, neither of which were the fault of me the user. Both have resulted in a loss of data by SENSE that was effectively forced onto me.

Why forced? Because I had no say in what happened, and in both cases SENSE said either ‘Its not possible to merge these identical duplicate devices that had to be re-created’, or ‘its not possible to reverse that auto-merge of devices by our algorithm, even if its completely false’. Both situations that should arguably already have re-mediation scenarios in place

Scenario 1: Philips Hue integration

The Philips Hue integration was one of the first things I enabled when I installed SENSE. For the most part, it has worked flawlessly. The few times I had problems, they were usually the result of mDNS(multicase/broadcast packets) being erroneously dropped on my local network. Usually solved by power cycling the Gateway or wifi AP.

That is until a few months ago, when Philips made a major change to its codebase for Hue Bridge API control. This killed the integration with SENSE at that time.
Once SENSE was able to update their code to address these changes, the only remediation presented was to REMOVE/DELETE the integration, and re-add it.

This seemed reasonable in the moment, until I was left with every one of my Hue ‘rooms’ listed in duplicate in SENSE. One as the active one, one as ‘not available’.

SENSE support advised me that merging the old devices (and their associated history) and the newly ‘created’ identical devices would be impossible.

What I’m left with is keeping these devices in duplicate if I want to retain the history in SENSE, or deleting them and loosing that historical usage.

But if this is the approach to integrations, why should I trust the longevity of any of the SENSE integrations, and the continuity of their historical use? Situations like this are predictable and should arguably have an easy path to resolution that doesn’t cause the customer to have to choose between a messy UI with duplicate new/old devices, or loosing that data.

Scenario 2: Auto-merged baseboard heaters into ‘Stove 1’. No way to reverse.

Around the same time that I installed SENSE, I started using MYSA electric baseboard thermostats in a few rooms of my home.

After the first month or two, SENSE had managed to detect most of the electric baseboards, and all three of the electric baseboards controlled by the MYSA thermostats because of their unique power-use profiles VS. the other baseboards.

In the case of the MYSA controlled baseboards, I labelled them and got to experience fairly accurate detection of each one for most of the last 6 months.

I specified in the device configuration settings: Unique identifiable names, defined them as ‘Electric Baseboard’, but left the ‘This is a guess’ toggle as ON, because device detection wasn’t perfect for any of them.

Fast forward to June 6th at 9:08am. SENSE notifies me that ‘Mysa Controlled Baseboard 2’ and ‘Mysa Controlled Baseboard 1’ have been combined into ‘Stove’.

As a result of this forced auto-merge:

  1. I loose the historical record of each of these two devices as they were recorded by SENSE
  2. I’m left with a ‘Stove’ device that is a 100% false detection
  3. And, according to support, there is no way to reverse this

Lets set aside speculation about the heuristics used by SENSE to make the decision to merge these devices (time of year, changes in power profile of my home with seasonal temperature changes, assumed probability of stove usage at certain times of the year, that these are heating elements, that I left ‘this is a guess’ toggle as ON, etc)…

The fact that a forced merger like this is irreversible, and that SENSE has not already designed in a way for users to specifically flag this type of scenario, is a pretty major problem to me as a user.

Particularly so because I read other posts in late 2019 from frustrated users with this complaint, which to some degree set my expectation low early on. This isn’t a new problem, its a nearly guaranteed constant that will be variable for ever home.

What I’m taking from this experience is that this problem occuring again is not an IF but a WHEN, and based on other peoples anecdotal past experience, it seems like I should expect it to get progressively worse as time passes.

It seems possible that this is due to increased ‘noise’ in the power profile of a given home. Noise that compounds its negative effect over time, and possibly diminishing the efficacy of the algorthmic detections.

But ‘noise’ and what causes it is a variable that will change constantly, if not only because of seasonal appliances. IE: AC units, multiple types of fixed and variable speed DC fans, and any other power blocks or devices that generally leak ‘noise’ into the electrical signature of a home.

I cant help but wonder if me starting to use DC fixed and variable speed fans nearly 24/7 for the last few months doesn’t have a role to play in this.

2 Likes

Four months later now. Its my 1 year anniversary with SENSE and guess what? 99% of my ‘detection’ is other.

What is this 99%? Its the electric baseboards controlled by MYSA thermostats that were conflated as a ‘stove’ and irreversibly auto merged 5-6 months ago through no action of my own.

I deleted the ‘stove’ devices several months ago in the hopes that the purportedly imoroved algorithm would do a better job. Several months into using my baseboard heaters and nothing is detected.

And the Bedroom Lights n/a duplicate object? Ya, thats still one of tue undeletable ghosts of the Philips Hue problems from March.

@sense can you help me understand how exactly this is progress and not an approach that is grossly lacking in data integrity? You dont even give us the option to choose.

@JustinAtSense

This product is really is love/hate relationship for me. I still have faith in where you are taking it but the approach, lack of communication, and forced imposition of worse device detection and badly merged devices is a major turn off.

Hey,
I can only touch on the MYSA, since I recently got one…
My baseboard heaters were previously detected (somewhat) before I installed mysa.
Before mysa they were just on cheap standard wall thermostats.
I have 3x baseboards, one in each bathroom. (the rest of the house has a central heat pump)
Sense doesn’t know I have 3 (although I did put 3 in the home inventory section, granted not make and model though) they just appear as one device. If any heater is on, the “baseboard heat” is on in sense.
I have my baseboard heat to show up in the timeline.
—getting to the point: When I first installed mysa and turned on the heater, I noticed it was cycling quickly. This is apparently a feature of the mysa, some form of modulation.
This caused my timeline to become spammed with heat on and off a few times a minute.
On the live meter, this really does look visually similar to a “stove signature” of a burner that is up to temperature maintaining heat.
—Suggestion: Maybe change the operating mode from “baseboard” to “radiant” heat mode.
This will make mysa function like a conventional thermostat (on 100% until temperature reached, then off, repeat when temperature drops below 1 degree of the setpoint)
I set mine like this for 2 reasons; 1, I want baseboard on my timeline, but don’t want my timeline spammed. 2, I don’t want luke-warm baseboards when temp is close to setpoint, I want them hot or cold.

Side note: I just installed a 2000 watt in wall fan heater in the garage, put a mysa on it, and have mysa set to “fan forced” mode for that one. Still waiting on detection, but haven’t really used it much yet either…

1 Like

I am so glad that you posted this. I also have a love hate relationship with my sense now and I’ve been posting about my issues over the past few weeks now. I haven’t been on the forum much over the past year because of personal issues but about a month or so ago I started to notice a few funky things going on with my Sense and I couldn’t figure it out so I’ve been back on here and playing around with it to try and fix my issues. I had no idea about the Philips Hue issues until reading this post and that explains a few of my issues. I tried to turn Philips Hue integration a couple times over the past two days with negative results but I was able to get my TP-Link and Wemo integration working again and seems to be running fine. I had to do a Data Reset to get everything working but before that I tried for several weeks to see if Sense could fix it. It seemed like things were moving slow on their end so my frustration got to me and I went ahead with the data reset and give my system two days and it seems to be working again minus the fact that I have lost almost two years of data.

Hi @surias.0 - first off, thank you for sharing your specific scenario. 99% of usage in the “Other” category is certainly not normal, and not the experience we’d like for Sense users. If it’s alright with you, I’d like to pass along your information to Data Science so they can take a deeper look at your monitor to see if we can notice anything going on behind-the-scenes. Would that be ok?

Definitely appreciate the faith you have here, despite the less-than-ideal experience with detection. We’re about to begin work on our yearly Data Science update where we look at what’s changed since last year and what we’re focusing our immediate effort on. I hope to post that blog sometime in November and share some updates pertinent to your original post.

1 Like

@JustinAtSense

Happy to have my details passed along. I’ll help in whatever way I can.

The 99% was hyperbole, but the percentage is over 75% now that winter has hit and the heat is on. It feels like big steps backwards to have had the baseboards that were so quickly and accurately identified a year ago now falling into the ‘Other’ category. Particularly so since they were falsely conflated and merged into a Stove device.

@Harrison

Thanks for this suggestion, but this approach for me would be working backwards. Part of the power saving potential of using these MYSA thermostats is this modulation feature. While still achieving a nearly identical heating potential, there is marginally less power used(and lost).

There are some easy to understand reasons why SENSE engineers are not looking at ‘time of day’ probability when identifying an appliance, but one would think that some automated flags would be triggered on their backend when SENSE algorithms think they’re seeing a stove turn on and off hundreds of time overnight and then not be used during the day.

Maybe there are some data scientists or ML programmers on here that could speak to that.