Looking back at some device learning numbers in the Sense app, I noticed that the numbers that I pulled on January 1st 2017 seemed to have changed. After noticing this, I pulled all the numbers again to try and gauge the magnitude of the change over September 20th 2016 to December 31st 2017.
These are the rolling 7-day averages for learned devices from my manual extracts on January 1st and January 13th. Note that numbers appear to have been “restated” across the entire time Sense has been running.
Where I had originally seen Learned devices peak at just under 60% in Nov 2016 and decline to under 40% in Dec 2016, the restated numbers show that the peak was under 40% and dropped almost to 20% around Dec 25th before edging to above 30% at the end of December. Here are the last set of Device Learning metrics I shared at the end of the year:
And here is what they look like now that they appear to have been “restated” by Sense:
Device learning performance is measurably worse than I’d previously reported. What caused the “restatement” and the changes in these metrics? How often does Sense make these kinds of “restatements”? How can users discover that their metrics have been updated and know to re-examine them?