Stacked power plot for Always-on, ID's appliances, and other?

[kevin1]

Finally got the code to run on my iPad with the R Compiler App. But it took a lot of changes.

• Due to the 128K input file limit, I had to move to a daily rather than hourly download from Sense, plus I had to seriously reduce the file size even for Jan-April by removing 70% of my devices (only kept the top 12 or so)
• Had to give up on reading the data file in directly and do as the coder of this app recommends - read the data file into the In (stdin), then read.csv (in the code) from “stdin”. Typical R offers much more graceful access to data.
• And forget about debugging on the iPhone/iPad - one error and the compiler craps out giving a cryptic message, with no indication of which line of code caused it.

The combination of code and datafile below should work on iPad/iPhone if you can get everything into the right place on your device.

SimplePlot2.R (2.2 KB)
Test2.csv (111.2 KB)

Here’s the output from my iPad…

[kevin1]

Playing with the hourly version of the stacked bar. But for hourly, an area chart seems to work better.

[chrisstarkey]

@RyanAtSense I’ve seen requests for this feature in a number of posts, but, it most clearly described here. This is an incredibly valuable visualization for tracking down energy use. I am a green building consultant, and, versions of this graph are the baseline tools we use to explain loads to clients and target energy conservation measures. It is especially useful for behavioral ECMs. The bubbles are cute, though, I’m afraid, not actually all that useful. It seems to me this is a tiny bit of coding that would have a large impact on the user experience.

[chasec]

I really want this too. I would love to be able to use this to help identify some appliances, and to see what’s actually happening in the moment. On the web interface this is possible to do without too much effort and I use the web interface a lot more than the mobile interface.

Is anyone at Sense actually looking at this feature?

[jlj.pers]

Any update on this possible feature?

[cjdickey36]

I am also extremely interested in a feature such as this. It would be so much more useful than the single-color Power Meter chart with just labels for when devices turn on and off. Even if it didn’t replace the Power Meter chart itself, I am sure I would use it as often or more often.

[kevin1]

If you want a substitute, at least to try, you can use this web app I cooked up. It takes either a monthly or yearly exported file from Sense, at “Day” resolution (not “Hour”) and converts it into these two graphs.

The link is here… Once in, just Browse to your exported / downloaded data file. LMK if you have any questions.

https://pgstats.shinyapps.io/SenseUsagePlot/

[jefflayman]

@cjdickey36, your idea for replacing the Power Meter chart with the colorful chart caught my imagination! I know that implementing everything @kevin1 has done with R, from scratch into the Sense app, is a programmer’s nightmare. That won’t get traction from Sense, but the idea is still awesome!

A scaled back version is possible. All of us have an Always On bubble, so adding a second color which would plot that item across the bottom of the power meter graph might be a small enough lift to actually happen. Everyone also has an Other bubble, so that could be a third color which Sense might actually go for.

If colors on the graph are popular with users, the feature could be expanded later. Rather than checking which are the top items as Kevin did so nicely (which would be very difficult to code outside of R), they could allow the user to assign any one device to each of a short list of colors.

And if that were popular, the next expansion could be allowing colors to represent a category to which the user would assign multiple devices. Behold, a road map towards implementation of your idea!

[JustinAtSense]

Thanks for sharing this again, @kevin1. I’ve started getting enough data that seeing this trend is pretty actionable. First step - actually turning the smart plug on my roommates PS4 off…

[jlj.pers]

@kevin1, is your web app still working? I downloaded all of 2022 at 1 day resolution, but when I try to upload I get an error message

[kevin1]

I’m traveling right now, but will check on Monday.

[kevin1]

@jlj.pers,

I just tried on my exported data for 2022 (by Day) and it worked ! I have a huge spike in Oct, but that issue is in the data - Sense firmware change caused my Wiser light switch to report 500kW usage when turned on !

Wondering if you read the file into Excel or and saved, or anything like that - that actually changes the format of the DateTime column ?

If that’s not the issue, you can message me the file so I can take a look at it for debugging. How bit is it ? Mine is about 1.5M.

[jlj.pers]

So I tried again. I was not opening the file in Excel before.

• When I try the whole year (1-Jan-2022 till today) it gives me the same error. I’ll message you the file, it is 342 kB
• However I tried the last bill, and that one does work!
  • My other is negative though, not sure why. Thoughts?

[kevin1]

Yup, I have a bug in the way I aggregate the non Top 8 energy on a weekly basis - you have one week where there is no device detected outside of the Top 8 consumers. I manually added one non-Top 8 device reading using a text editor (not Excel which screws up DateTime) and it’s working.

ps: looks like you are over-detecting usage which leads to negative Other. I have had that at various points in time when Always On is artificially large, when offs of large devices are missed or when smartplugs lead to double-counted detections. Will take a quick look to see if I notice anything.

Also looks like something weird happened with energy usage in your house throughout May 2022.

[kevin1]

Hi folks,

I have finally updated my web app that direct reads in the Yearly Sense Export files at DAILY resolution and plots device usage via stacked bars on a weekly basis. Based on what I have learned, there are a couple of additional enhancements and bug fixes. It now zeros out all negative Other, and also creates a table of monthly usage for the top 9 devices.

As yo usan see from my usage graph below, it does not fix errant data that exists in the Sense export. I have a huge spike in Oct thanks to an issue with the Wiser integration.

Give it a try using this link with your 2022 Sense Export data.
https://pgstats.shinyapps.io/UsagePlot/

[jefflayman]

Thanks, @kevin1 ! The enhancement to zero out negative Other makes this tool now useful for me.

[dannyterhaar]

$ ls -lh sense-1-hour\ data\ from\ Jan\ 1,\ 2022\ to\ Jan\ 1,\ 2023.csv
5.4M Jan 11 16:07 ‘sense-1-hour data from Jan 1, 2022 to Jan 1, 2023.csv’

last few lines of file is this:

Please note that there may be discrepancies between your electric bill and what Sense reports due to data interruptions (such as Wi-Fi signal loss or loss of power) or other issues.
DateTime,Device ID,Name,Device Type,Device Make,Device Model,Device Location,Avg Wattage,kWh
2022-12-31 16:00:00,nYxHmhPt,upright light,Smart Plug,,,,3.650,0.004
2022-12-31 16:00:00,sjxs0rid,Fridge Kitchen,Smart Plug,,,,68.926,0.069
2022-12-31 16:00:00,4iIfz7IK,oil filled radiator office,Smart Plug,,,,362.424,0.362
2022-12-31 16:00:00,IbqdQmEw,light atelier,Smart Plug,,,,12.651,0.013

[kevin1]

@dannyterhaar, the uploader can’t handle the size of hourly data for a year. Export daily instead. It should work.

[dannyterhaar]

Beautiful
Thank you very much.
Very enlightning!

[jim.gupta]

I would like to see the power meter as a stacked graph. The stacks would be the Devices + Always on + Other.

I would also like the stacked graph to have a legend and allow the user to turn on/off devices by clicking on legend color (pretty much what chart.js does)