Internet was out for ~7h - is the data for that period lost forever?


#1

So my internet was out for about 7 hours yesterday. But power wasn’t out. I got an email that my Sense monitor was offline. I thought that Sense saved data internally and uploaded it later, but my data for yesterday is blank for the time that my house lost internet connectivity.

Why didn’t the date stored in the Sense get uploaded when connectivity was restored?


#2

I’ve noticed some variability too - sometimes it backfills, sometimes not. I haven’t been through enough occurrences to figure out factors involved yet.

Most recent was last week, I don’t even think our internet was out. Strange behavior, the solar cts kicked in hours before the mains came back online:


#3

I’ve seen this behavior also and have raised it with Tech Support. In my case, the LAN connection is close to the reasonable limit, not a Sense issue, and I need to add another WAP, which may be complicating matters, as my wireless LAN may be impeding its ability to catch up .

I recommend you send a note to Tech Support as they can see if there are any LAN as opposed to WAN outages and also see if Sense is rebooting in an attempt to reconnect, which I suspect is what is dumping the buffered data. This may identify something you can do, but more likely will be one more report logged to help prioritize this issue for action in a future software release - I suspect that there are things that could be done to preserve the data while trying to recover / await restoration of the connection.


#4

Hmm. @aaiyar, has that data since appeared? Sense does store up to 8 hours of data but backfill sometimes takes a little while.


#5

@BenAtSense

No, the data hasn’t reappeared yet.


#6

Mine either.

I think there are problems when there’s an internet traffic issue. If it’s a router-based problem (loss of wifi or a power disconnect), data can backfill, but if there’s other things on the network cramming things through then the data is going to be lost.


#7

Hi all, just wanted to check in again and see whether that data appeared for you or not. As @dave_n_s mentioned, opening up a ticket with support would make sense so they can take a deeper look for you. Thanks for raising the issue so we can keep an eye on it!


#8

@BradAtSense -

This has happened three times since January 12th. The first two times it happened was because of issues with my internet provider. The third time (last night/this morning) was because I just installed a new mesh network and Sense dropped off the network. The missing data hasn’t repopulated on all three occasions.

So, I finally got around to sending an email to Support through the app. Will update this thread as needed.


#9

Looping in @RayshawnAtSense who might have more insight into this.


#10

Thanks @BradAtSense. I did contact tech support and someone is looking into it. I am wondering if my monitor has a issue …

For anyone else interested, here are the gaps in Sense data that never repopulated.


#11

Although we don’t know for sure how much data Sense is sending to the back end systems, I doubt it’s going to be sufficient to tax even a slow urban connection unless that connection is also being used to push lots of data into the cloud at the same time. It would probably struggle with a rural DSL at the limit of copper connectivity (say 1 mb/s upstream)

We do know that Sense reboots after a longish period as part of its strategy to restore the connection, perfectly reasonable, but appearing to dump buffered data in the process.

I’ve seen several outages on my data, due to my WAP location, now fixed, that also aren’t explained by a reboot strategy, so I do think there is some work to do on handling the buffer.

I’d suggest that the engineering team look at “warm” restart strategies if not already in work. These can preserve data and reinitialize external connectivity, so that the router reassigns a new IP address for instance, of a conflict was causing the loss of connectivity.


#12

Hi All, just wanted to update this thread because we have some more specifics about the data buffer for you based on some of our recent testing. Sense can consistently store up to 6 hours of data on its own (not 8 hours as we had mentioned previously) to cover for periodic connectivity issues and the like. It will backfill that data as soon as a solid connection is re-established, however if it has been power cycled during this time (due to a power disruption, or manually flipping the breaker), it won’t be able to backfill that data.

Very sorry for the confusion!


#13

Hi @BradAtSense,

I’ve had data gaps and seen others report them that are probably not explained by the above.

I obviously don’t have specifics about other people’s power outages / glitches, however I’d expect Sense to motor through typical glitches of tens of milliseconds, ideally more given the function - is there a spec for that? Those of us who have back-up power systems because we live where outages are a fact of life expect change-over from primary to back-up and back to happen perhaps 10 times a year. Those changeovers are going to take a second or more to go from primary to generator, if that is the back-up. A small market nationally, but I suspect a much larger percentage of Sense users. Is there a rechargeable battery power supply strategy to keep Sense operating through those outages, or at least not doing a cold restart?

That leaves the question of Sense being remotely rebooted as part of your strategy to regain the connection. I’ve been assured that happens. If it’s a cold restart, logically it should only happen at the end of the 6 hrs data gathering, but arguably not even then as a restart isn’t going to fix a real internet service outage.

What can you say about this?


#14

@BradAtSense

Just happened again last night. I lost internet for about 17 minutes (8:49 - 9:06 pm) (see screenshots below). The Sense monitor was NOT rebooted, but the lost data has not repopulated yet.

I am in contact with tech support, but wanted to share this update with you and everyone else following this thread.


#15

Ah, I see. Thanks for the clarification here @dave_n_s,

So this sounds like a case where the changeovers or power glitches are knocking Sense offline, and it isn’t able to reconnect without a cold restart? It’s possible that a remote soft reboot could solve the issue without losing the past 6 hours of data, however, if the Sense monitor hasn’t successfully reconnected to the wireless network, a remote reboot command could go unanswered.

We have heard other requests for a large cap or battery backup that could help Sense across these types of gaps so that it doesn’t dump those 6 hours of recorded data. Hope being that as soon as the genie comes online and the Wi-Fi & LAN are back up and running, Sense reconnects. We don’t have a good solution for this at the time being, but it’s a great suggestion.

Does this capture what you’re thinking of? I want to make sure we’re understanding your use case correctly. Thanks for your help with this!


#16

I lost mine for about 15min last night as well - 450-505ish mountain time. Happens maybe a few times each month, unrelated to anything specific. Had good internet connection through other devices on the same router/network at the time.


#17

Did the data ever backfill @NJHaley?


#18

Did this data ever backfill for you?


#19

@BradAtSense

Not yet.

Edit - Support just emailed me that the network engineer was out sick today, but they will run it by him when he returns.


#20

Nope. Funny thing was that I was watching it in real time, checking my internet connection with other devices just to see if I was still online. Nothing else going on at the time, no computer use, etc.