Can you tell approximately when the issue started ? I saw things flare up around Dec 19th/20th (below). The number dropouts dipped in early Jan when I turned off continuous monitoring of Sense realtime on an iPad in my kitchen. Correlation or causation, I don’t know ?
Do you keep a realtime display on for a lot of the time ? Either bubble or Power Meter ?
It’s tough to separate the actual internet outages ( I sometimes get one or two a week for 15 minutes or so) from the sense issues…but it appears to have gotten much worse for me starting December 6th.
I usually have a browser tab open at home/work with the bubble view or device view.
I did notice that there were almost zero data drops for a week period in December when we were off skiing (computers off) and a 5 day period last week while we were in Mexico (also computers off). So…maybe monitoring impacts it. But if so…what’s the point of the whole device? If I can’t monitor usage…without impacting my ability to monitor usage…then the device is useless? Right?
I agree with your sentiments completely… Gotta fix it, ASAP. Otherwise you have a Heisenberg Uncertainty device - every observation of the measurement renders the measurement less accurate. I’m just trying to crowdsource the root cause, with the hope that it helps Sense solve more quickly.
BTW - I have very reliable internet and house network, except when I’m trying to upgrade it for more reliability. My attempt to upgrade, partially driven by this issue, took our house offline for 1 1/2 days
So this is interesting. I have 2 data losses today, very close together. You’ll notice in the collage 2 pictures from the web app. One take shortly after “the event” which I posted earlier. I looked again, and as you can see, data actually got removed from the web log. THAT is very interesting. I did uninstall/reinstall sense on my phone to see if it would pull new data from the cloud, but no luck. I did loose my purple/dark mode and that was disturbing to look at
Out of curiosity, for those that have experienced data drop outs, are you finding that your monthly reports are not tying as well to your actual usage from power bill? @kevin1, I’m looking at you as I know you track these things better than anyone else. I’m just wondering if the data is actually missing everywhere, or just not getting pulled correctly back to the app and web correctly in graph view. I know when they have had outages before there is a difference between Sense Monitor connections and Sense Client connections.
Kevin, I know from the data analysis side of things you have seen the data drops in your exports, but if you just use the app’s “last month” under bill, does kWh line up? Or are the data dropouts too small to affect the overall number.
Yup, it looks like the first data gap expanded in the “earlier” direction when you viewed the data the second time. My speculation is that Sense has a whole storage hierarchy for data views:
Raw data generated at the probe/monitor (1 microsecond resolution for some of the measurements)
Slightly processed data that is sent back to the mothership that is stored and used for identification and training (who knows how they compress and extract data features - resolution somewhere between 1 microsecond and 1/2 second). But this includes separate data for each house leg, plus phase information.
Realtime data that is sent to our apps from the mothership when we see the moving waveforms and bubbles - 1/2 sec resolution, some computed realtime (Always On, Other bubbles, some identification bubbles), though some people are also reporting data delays in receiving this data.
Historic display data that is sent to our apps from the mothership when we either:
are displaying realtime data, but need some additional history to fill out time window
go back in time and/or change resolution.
Identification and summary data - rolled up from the historic display data AFAIK. Used for device displays, trends, usage and export.
What we see on our screen in the Power Meter is typically a hybrid of Realtime data and Historic display data. Seems like there are situations where the Realtime data in the display doesn’t get committed to the Historic display (I think one Sense guy called with the “history timeline”), even though it is there on your screen. Have also heard that one of the challenges in solving this one at Sense is that the slightly processed rawish data from our Senses is just fine at the mothership, but the “historic timeline” has gaps (this is all hints and speculation until Sense formally tells us something).
Hey, I too roll my eyes with the standard fare “unplug it and plug it back in” routine, but alas, the reason it’s so common is…because sometimes it works.
I’m left wondering if this is a network quality issue perhaps between you and Sense’s servers. Have you performed any tests recently to see if you’re experiencing excessive packet loss or jitter? Often these sorts of issues don’t effect run of the mill day to day internet use, but could screw with time sensitive things - this is why latency screws with gamers, and jitter screws with things like VoIP.
I’ll admit I’m not overly familiar with how Sense’s back end handles the stream of data being returned from devices in the wild, but if something like excessive jitter was causing severe congestion and packet loss, it could certainly explain data going missing.
A crappy internet connection is going to effect a lot of things, some more than others. Huge latency and jitter isn’t going to effect someone surfing Facebook, watching the occasional cat video, and sending emails (hence why many casual internet users may not even realize they have a terrible quality connection), but it can downright destroy usability of many other services.
Lots of people suffer from low quality internet connections that can often make third party devices and services very problematic or sometimes completely unusable. I know a friend for example who has a terrible DSL connection (and no other option due to their rural location) and simply couldn’t get a VoIP service working successfully because of such. That doesn’t mean the VoIP service provider doesn’t "have a product intended for the masses", it means that their internet connection sucks and the problem lies there, not with the innocent bystander VoIP company trying to utilize said lousy internet connection. I use the exact same VoIP provider and have excellent quality service from it, but my internet connection is also very high quality as well.
To use another analogy, If your electricity provider suffers from constant brown outs and outages leaving you in the dark, is it the fault of the manufacturer of the lightbulbs that went out…or is it actually the fault of you electricity provider?
I think that home internet IoT products are going to have to get to a place where everything is plug and play, as you suggest, but that’s not today. Today users have to get into the inner workings of their home internet and the reliability/performance of their supplier, when they have issues. I own a ton of consumer IoT products (SmartGrill anyone ?) today, and all of them have slick apps for setup, plus reasonable initial install diagnostics, but virtually none of them respond well when there are bumps in the operation of the local network or performance issues with my internet provider. Users have to have a modicum of networking knowledge, or a knowledgeable installer to handle issues when they come up. Hell, I know a CTO of an IoT company that has Carlos do his home network setup for him… I think there are real opportunities out there for simple home networking and diagnostics, but it really requires a whole new gen of networking gear that is designed to diagnose issues beyond just connecting devices. And given my recent trial, that isn’t the new Orbis or Google WiFi’s.
That said, I have spent a lot of time looking at my network and path to Sense, and I’m fairly certain my network is NOT the proximate cause of these dropouts.
I think I would disagree, comparing senses networking issues to VOIP or power companies is a bad comparison.
VOIP require real time data to work. There is no way around it. With Sense, while the real time data is cool, it does not absolutley rely on real time data. It should be able to handle periods of downtime, like store and forward. It seems it already does this to an extent it just isn’t doing it well enough. In this regard VOIP is not equal to Sense.
With your power company analogy, the light bulbs continue to work after the power comes back on. Regardless of whose fault the power outage was, I fully expect the light bulbs to work again to the fullest ability after the power outage. In this case, sense should backfill any issues from internet outages (and does for the most part, but seems to struggle under certain instances).
I appreciate your replies. I do. But it’s a poor analogy. And I’ve also never said that this was the fault of Sense. To correct your analogy, in this case the electricity you mentioned would seem to work fine for all other appliances and lightbulbs in the house, but this particular lightbulb brand still had issues…because it needed a more pure and perfect form of electricity than all other appliances in the house. Can it still be the electricity providers fault? Sure, I guess. But once the lightbulb help desk asks their end users to do something akin to “checking their network for excessive packet loss or jitter”…then they’ve already lost the fight. If the same electricity that runs everything else in the house doesn’t seem to work for this particular light bulb, and troubleshooting involves technical knowledge beyond the average purchaser, then that lighbulb isn’t a product for the masses anymore.
To be fair to Sense…they haven’t asked me to do this yet. I’m just responding to your comments in this thread.
Ultimately, I’m not saying that this is the smoking gun (and Kevin’s diagnostics are suggestive of the fact it’s not), but just throwing it out as an option.
Ultimately none of us know how Sense’s servers receive and parse the constant stream of data being sent from all of us. I’d like to agree that something like packet loss or jitter (which can cause packets to arrive out of order) wouldn’t be the root cause of these issues, but again, we don’t know.
Jitter especially can be a fickle beast to deal with when transmitting time sensitive data. I know because I went through a personal hell long ago dealing with it on a terrible ISP I was with for a period of time.