I’m on a mission now to “scrape” and recover data displayed in the Sense web app, back into relatively higher resolution data than we can get out of export. The good news is that I have been able to successfully recover data using just the web vector graphics data. But it’s still challenging:
- Data is displayed by Sense as a HTML vector path construct - pretty easy to extract
- The current path data for a day on the screen actually consists of about 3 days worth of data - has to be clipped to a single day window
- The data is all in terms of the on-screen coordinates - data has to be rescaled to a 24hr period and to the vertical scale maximum for that day
At this point in time, I have 4288 data points per day all properly scaled, with a few more challenges to unpack.
- All the data is in a path format so I still need to reduce to a single point per time format - simple reduction
- But before that, I have to make sense of the data which has 2 (or really 4 until I do the path to point reduction) power values for each time point. The way Sense displays the data, it does a right-ward sweep of data, then comes back with a left-going sweep via the path construct where the right going and left going sweeps, put down different power values for the same point in time.
For well behaved days, the numbers aren’t all so different and the left looks like a shadow of the right trace.
But other days, where there has been a data dropout or 3, I get less correspondence between the right and the left:
Any creative thoughts on the best way to resolve the two lines ??
Matching Sense web app view below.