@kevin1 , thanks for the thorough and excellent answers. Sorry I was quiet for so long. My Sense went offline for some reason and I am away from home (which is part of the reason I bought it) and had to wait until I could get someone to go over and switch it off and on. Now I’m back online and have downloaded the data and will start to play around with it based on your suggestions. I was aware of pivot tables but I never used them yet. I’m pretty good at almost everything else in Excel so I guess I should learn them.
So far I’m only interested in the Total Usage. Sense doesn’t seem to have done a great job detecting which devices are which. I’m not familiar with Sense yet to know if I can override its decisions or help it learn correctly by turning things on and off and telling Sense what device it was. I also don’t know if I can correct any wrong decisions when I’m not at the house, but I certainly cannot turn things on and off until I get back there.
I still consider it a bug / bad programming decision that they don’t allow me to export year data until after the year is over. Same goes for months and weeks. My sense was installed July 28. Luckily even when offline, Sense seems to have kept the data and uploaded it when it got back online. If I asked for the data for this month, I get August’s data, even though we are 12 days into September. If I ask for the previous month’s data then I will get those last few days of July. If I want September’s data, I have to go by week and day to get everything up until yesterday, if I want it before the end of September. I also wish they would let me override the date range and select any range I want.
Hmmm, that’s weird. I have no problem Exporting data for September to date from the Month view or Exporting 2022 to date from the Year view.
The September data looks fine:
Same for 2022
I typically pull data for the whole year, then filter in in Excel or R for the portion I need.
That’s exactly what I would like to do. Let’s see if I can figure out how to paste screenshots…
Notice there are no green bars. And no right arrow to move to the next year. And when I click Export…
It’s showing me last year. And no way to override the dates.
So could it be something wrong with my configuration? Or firmware version?
This is the website I am accessing it with: https home dot sense dot com slash usage. I recall links are not allowed.
My firmware version: 1.37.3689-fbf0c631-master in Settings Sense Monitor
I also see Version 13 build b8e915b in Settings About
I think there is a known issue displaying the use bars in Safari (Mac), but something else is wrong if your Export jumps to a different time period. The top (Sept) Export was done from Safari when my bars were not showing. The bottom (Year) was done from Chrome, where the bars were visible.
My monitor is on the same firmware
I looked more closely at the month display. At the top of the graph it says “This Month” and there is no right arrow to go to a next month. But the number of days shown in the graph is 31. September has 30 days. When I press the left arrow it displays “Last Month” and it still has 31 days. One more press and it displays “Jul, 2022” but only shows 30 days. So definitely the title does not match the graph and the export data range does match the graph but not the title.
I’m using Google Chrome but I also tried on Microsoft Edge and on Chrome on a Chromebook as well as on the PC. Same issue.
This is definitely one of support. What is the iOS or Android app showing ?
Here is the PC screenshot. This month (September for readers in the future) showing 31 days.
On my Android phone it shows the correct range, This month shows partial September.
But there’s no way to export in the app.
Now this is weird. I assumed something was wrong with my device since all the computers I tried and all the browsers all showed the wrong date range. (I even tried Raspberry Pi’s browser.) But the app works correctly. So now what could be causing the error?
Well the first suggested topic below has the answer. Not sure anyone else gets the same suggestions I get. And I can’t put links here. The topic is " Web Page Month Not Showing Current Month". Hmm, it pasted the link too so let’s see if it works. In case not, the topic number is 15665. Anyway, I am overseas and the time zone on my PC doesn’t match the time zone on the Sense device. When I make them match, it works correctly. I shouldn’t have to do that though so I’ll report this to support. The topic was written in October 2021 so they probably already know and haven’t fixed it yet, so maybe I’m out of luck.
A little more testing… The Sense is in Arizona. If I set the time zone to Arizona, California, Hawaii, presumably anywhere to the west but short of the international date line, the date range is correct. If I set to Central or Eastern or presumably anywhere to the east, the date range is incorrect.
Ahhh, interesting - great sleuthing. The phone / tablet app grew up before web app ever came out and I’m pretty sure they use very different underlying graphical packages, and some features are still different between the two - only the web app has export.
ps: I’m splitting this topic because it has two different questions and solutions.
I submitted a ticket to support. Once they reply I’ll update this topic.
I changed the time zone so I could get all the data in one file. I downloaded both hourly and daily for the whole year, since I installed Sense until now. The data in the daily file is all sorted according to date. The data in the hourly file is not. It’s broken into three chunks. Within each chunk, the date ascends. But the first and last chunks would have to exchange places for the entire file to be monotonic. Do you get this too? I guess with pivot tables it doesn’t really matter.
Here’s a plot of the dates from Excel. The steeper sloped parts are when the monitor was offline and so it only exported Total usage and not any of the detected devices.
Very interesting chart and discovery. In most of my uses for Sense exported files, the ordering of the data is immaterial because the data either gets sorted or reshaped / aggregated (pivot table in Excel, cast in R). I generally do like to test for the list of totally missing hours of data, typically due to power outages or network issues. Your chart has given me an idea on that front.
The steep part at the very beginning of chunk 1 and the matching part at the end of chunk 2 are when the device was offline for about two weeks. They are steep because there are fewer data points per day. The little blip in the middle was when I turned around one of the current clamps to fit in the panel better. I realized something wasn’t right after a day or two and read the manual and turned around the other clamp to match.
Yes, I have noticed similar behavior. It seems to occur only on full periods. That is, if you are looking at monthly view and the month is historical, you get chunked data output. If the month is the current month and has not yet finished, the data output is consecutive.
You inspired me to take a closer look at my Export ordering, and also whether your charting - DateTime vs. Index number would enable the to see missing data.
It looks like I am served up Export .csv completely sorted, at least for my Hourly Export for 2022. The red dots mark hours that are missing any data. At this resolution, it was impossible to see any missing data.
I pushed into looking at just the “Total Usage” data since I have essentially 40 data points for every hours, figuring I might be able to see the missed hours (14 total hours missing). But not much different.
If zoom in closely on either of those, the missing hours are visible - here’s the missing data pattern around Jan 6th, 2022.
Based on my observation above, a yearly export for the current year is expected to be sorted properly. If you try the previous year, you may see chunking. I believe this also applies to weekly or billing period exports.
Sorry I reversed your observation. But even when I try 2021 Hourly, either a newly generated Export or one I Exported much earlier this year (maybe Jan), I still see don’t see any chunking in mine. I am able to see some distortions for at least a couple of the missing data patches.
1-hour data from Jan 01, 2022 to Jan 01, 2023.csv (261.8 KB)
Here is the exported file in case it sheds any light.
I first thought the data was incomplete when it didn’t start on July 28 but then realized the data is all there, just out of order.
I added this issue to my support ticket so hopefully we’ll get an answer soon.