What methods do people use to capture the data? I saw the github project that has a script to do it, at least I think that’s what it does. But another method that I used was to use the Burp Proxy tool (free!) and run all the monitoring traffic through that and see the json data in the websockets window. The data updates every 1-2 seconds, so it’s a lot.
I’m pretty new to having solar panels (about 2 weeks) so I’m looking for some cool analysis ideas!
I run a modified version of the github python code and capture the sense data every 15 seconds. I write it to InfluxDB and look at it with Grafana. All free running in Docker containers on my NAS.
The red and yellow bars are HVAC’s and the purple bar is a garage gas heater. Green line is gas consumption and yellow line is Sense electricity.
That’s really well done @ben3 (tagged you to get another badge!)
I see in the online Sense graphs that I can see my usage, and I can see my solar generation, and there are places that I can see them side by side. I think I might want to create something more fine-grained using these data points. I might also use this extra data that I’m discovering to self-identify more of my “Other” devices in the house.
I’ll take a look at Grafana, I haven’t heard of that one before, or InfluxDB. I just did a quick lookup and saw they are built for exactly this type of purpose, so that’s pretty awesome. Thanks!
@samheidie: I can’t answer that exactly because I record a BUNCH of data. I started this project over XMas break so my data starts Jan 1. Total storage in use by InfluxDB is 67M. I’m pulling weather data from several sources, HVAC states from 4 systems, etc.
@serovner86: I’m using RTL-AMR to read the gas meter’s broadcast through this: https://www.amazon.com/gp/product/B011HVUEME
Cool, thanks! I was also trying to think of how I could get better real-time gas data. The provider sends a monthly update, but would love to get something that tracks similar to what Sense does.
I made the decision not to try to store/retain my own data, partially because many of my sources, my power utility, my thermostats, Sense and my solar provider all supply hourly export (or even finer granularity). When I do want to see at tighter resolution, I do use the APIs, plus @duanetiemann’s utility for HS110’s.
And just for a little fun, the existing Sense Power Meter waveforms can be harvested from the web app for limited analysis, like this…
@patricklaverty: This is the library I used to get started: GitHub - bemasher/rtlamr: An rtl-sdr receiver for Itron ERT compatible smart meters operating in the 900MHz ISM band.
Essentially, what’s happening is that my meter broadcasts a very low power RF signal, un-encrypted, with a small data packet that contains the meter ID and the current reading. This USB dongle is actually a radio receiver. RTL-SDR is software defined radio using the RTL2832U chipset. See here: About RTL-SDR. A small app runs in the background that controls the RTL chipset and serves the data via TCP. RTLAMR is a small Go app that connects to the RTL-TCP app and receives the data and serves it up to my Python script that writes it to InfluxDB. My to-do list has a line to rewrite all of this and simplify the flow of data, but for now this is working.
Ahh, so you set up a receiver close enough to your meter, and it frequently captures data and stores that? That’s pretty cool.
Exactly. The antenna on top connects to the USB dongle on the back of the NAS.
How many feet/meters is the broadcast range of your meter?
My antenna is about 30 feet from the meter, but I pick up 5 or 6 of my neighbors too, so I don’t think range will be a limitation.
@ben3 does your area have smart water meters? do you know if the RTL-SDR picks that up?
I’ve been dying for gas & water meter data… where I am, we have all smart meters… I’m thinking I’ve got to try this…
thanks for sharing this info…
We do, but I’m not seeing the data in the default scan range of the RTL. When it warms up I’m going to take the ground cover off my water meter pit (it’s in the front yard) and get the model number and figure out how to track that as well.
ah, ok… did you use a freq counter to get the initial frequency? or use scan features on the RTL? or just rely on info you could get from the web?
The default range for RTLAMR worked for my meter, so I didn’t experiment farther. There is a parameter to RTL-TCP that controls the freq range. Best thing would be to google your meter’s model number and see what it uses.
