Lost data that never backfills

wifi

#1

I’ve had a recurrent issue with missing data for varying periods of time. I’m aware that there is meant to be an onboard buffer that infills missing data provided the unit isn’t rebooted. In the report I lodged with Sense on this, I’ve been told that this is a WiFi problem. So far I’ve purchased a larger antenna for the Sense unit, added an additional access point (AP) to improve signal strength. However even with these, the Sense unit continually moves from the strongest AP to the weakest and not the reverse as expected.

If the onboard data buffer worked consistently, wouldn’t the transient WiFi signal be immaterial?


Multiple Acces Points / Wifi Devices
#2

The onboard data buffer holds about 8 hours of data, but sometimes due to repeated dropouts, it can fill up over that time. We are actively looking into issues reported with multi-AP roaming. AP station roaming is handled by the WiFi drivers provided by Texas Instruments and we are working with them to track down what is going on. We have recently found (and will be pushing a fix shortly to) an issue where repeated roaming can cause a long dropout, often requiring a power cycle, which we believe will fix some of the worst drop-outs. This is priority #1 on the engineering teams queue and we are working hard to track down and fix these problems. We will be posting more updates on this shortly.


#3

That will be good as I had a lock up last night for about 30 minutes. I was actively watching my phone when data stopped streaming in. I even switched to the access point by Sense and it had a strong WiFi connection even at the other end of the house where I was. I logged out and rebooted the app with no luck. I went down to the panel and tried to have it find a new WiFi signal as in the settings Wi-Fi section it was blank. When I did that it could not (would not) find any WiFi at all. I resorted to power cycling the unit and about 30 seconds later it was back on the normal WiFi. So, something locked up in the unit.


#4

Thanks Rhoda, I reported this problem (#11286) on September 22nd 2016 - three months ago - so I’m glad to know this data loss problem is the top priority of the engineering team. Can you please give us advanced warning of exactly when the partial fix will be deployed? I’d like to be able to record before and after behavior.


#5

Hi Brian, It is currently in testing. If all goes well, we’d like to deploy it later this afternoon. This won’t fix all WiFI issues - this is an initial fix for the power cycle issue - we are working with TI to resolve some of the other issues related to Sense roaming. I’m heading out for a few days of vacation! Enjoy your holiday and be back soon.


#6

Can you clarify whether you believe this is a fix for the lost data problems I’ve been having since I installed Sense in September 2016? If so, I will power down the new wireless access point and remove the larger antenna I purchased at your urging to test whether the problem is fixed in my environment.


#7

Rhoda, when you refer to repeated drop outs causing the data buffer to fill over time, are you talking about a case where there is zero WiFi connectivity for over 8 hours, or a case where there is intermittent WiFi connectivity and the buffered data still isn’t transferred to your servers? The first case is expected, but the second is a cause for concern. What conditions need to occur before a newly reconnected Sense unit will transmit it’s buffered data, confirm its been received and purge it from the buffer?


#8

Hi, just filling in for Rhoda - the fix we deployed should correct the first case you mentioned in your last comment, as the fix specifically addressed situations where Sense was being knocked off WiFi and sometimes never coming back online. Are you still seeing gaps in your historical data?

In the second case, this can still happen if Wi-Fi signal is too low or is going in and out enough so that the average transfer rate falls below 2k/s sustained over the whole day, or more realistically ~20kb/s for when Wi-Fi is connected if it is going in and out. If the transfer rate is not high enough the buffer will not be drained fast enough and that can cause gaps. Given the low rates we are not seeing this problem very often, but please let me know if you are still seeing issues and we can have someone to look into it. Thanks!


#9

Thanks for the information about buffer draining and minimum speeds to achieve it. Getting data like this really helps us understand the constraints of the device and why we are seeing problems in specific circumstances.


#10

I’m on my second sense unit. the first one was a disaster and wouldn’t even show stats and info in the sense status page. And it logged hardly anything.

The 2nd one worked perfectly for 24hrs now is getting dropouts anywhere from 8-3 hours…

not sure what to think at this point. I’ve submitted screenshots but as usual 2-3 days response time from Sense seems to be the norm… :frowning:


#11

I recommend that you take a look at your access point management software to find the Sense unit’s connection. Check the connection speed with Maarja-Liis comments above in mind. To work around the lost data problems I purchased and installed an additional access point and a replacement antenna. Report back on your signal strength over time and how much missing data you have (adding screen shots from the Sense app is good).


#12

Got fantastic connectivity, I’m 8 feet away from the router so all is good there. I’m suspect one of the PCs ran a backup yesterday and ate up the bandwidth. Mysteriously a few blocks of missing data filled in overnight. I’ll watch it for a few days…


#13

My understanding is that Sense has an 8 hour data buffer so the backfill is “by design” behavior. Provided you are not starving for WiFi and internet bandwidth, the backfill should occur after an outage. Take a look at Maarja-Liis description of the required bandwidth for the buffer to drain above.


#14

Had a very brief power outage last night and was bit by the bug where the Monitor didn’t want to re-connect to wifi until I power cycled the unit…which didn’t happen until 7 hours later. It has been back online for the past 3 hours, but I am not seeing any lost data being backfilled. I have a fairly decent connection to my AP (-67 dBm) and no drop outs according to the AP logs. My Internet upstream is only utilized a fraction at this point so there shouldn’t be an issue with it emptying this buffer. Has anyone seen where it successfully backfilled data after a break in connectivity?


#15

Not if you cut power, otherwise I’ve gotten several hours back if Wi-Fi only is lost.


#16

:confused: Well the root issue needs to be fixed then since I had to cut power to restore its connectivity.


#17

Yep, no question there. There was a post a little while back that discussed the problem, essentially trying to figure out when and why Sense needed to be rebooted. I don’t know that anything was uncovered, though - maybe the order in which things came back on? Like if Sense booted before your router, and then gave up searching for a signal too soon?

Edit - I see you found it, and it likely isn’t boot order…