Do a detailed analysis
Sometimes you don’t have any choice on this if you want to get to the bottom of significant differences after using one or both of the simpler checking techniques above. Detailed analysis presumes that you can pull daily, or better yet hourly, data from your utility, something most larger utilities enable.
The Ingredients and Tools
Here’s what you’ll need to investigate:
-
Sense Export - The web app in Sense lets you pull out hourly or daily data for a week, month or year for your own analysis. Export is hidden in the upper right hand corner of the Trends page, and works on the the data period that you have selected in the Trends. You can pull data for the entire calendar year to date with an hourly interval using two steps in the Sense web app (sorry, no iOS or Android app here).
-
Utility data download - My utility, PG&E, like most utilities, has a “Green Button Data” access which allows you to download your energy usage information. Generally you will have the option to download your data for some period of time in whatever time interval your utility uses for billing via Green Button Data. My utility offers data for up to a 1 year period with a 15 minute sampling interval. And even if you don’t trust your utility, you have to treat their data as the golden reference for energy usage since they are measuring via a revenue certified device.
-
A spreadsheet, or better yet R or Python, programming languages to manipulate the data from these two sources so that you can do direct hourly or daily comparisons. I tend to prefer R since it offers an incredible amount of automation and charting, but Excel will work fine as long as you are experienced with some of the more complex functions like merge and Pivot Tables.
The Process:
To do a detailed comparison, you’ll need to do 4 steps
-
Download the data from both sources and convert it to a common interval. My utility puts out the data in 15 minute intervals, but Sense’s minimum interval is an hour so I have to aggregate/sum my data into hourly data. In Excel there are nice ways to do this in a Pivot Table.
-
Align and filter the Sense data - Sense puts out all the data for the house including all devices in the export. For comparison with utility data, we need to filter out everything except Total Usage, or in the case of solar, Usage minus Solar (net usage). You might also need to fill data at this stage if your utility or Sense has left out data for a specific hour or day. If you don’t fill in missing intervals, you might either encounter errors during the next step, or you might actually miss discrepancies where Sense or your utility did not measure data. I do a completeness check and fill with either 0 for missing hours or NA (Not Available) depending on what I am trying to do.
-
Merge the comparison data by interval - I tend to merge hourly data since it gives more detail into where any mismatches occur, but there are also some advantages from merging at the daily level. Be careful not to ignore merge error messages, since you do want to double check for days/hours where either of the two power usage sources has missing data.
-
Analyze and compare the two measurement sources on a daily or hourly basis. - Time to have some fun. You now (hopefully) have data side-by-side for a range of time intervals, so you can use various techniques to figure out where the differences between the two are coming from. Here’s where we can use lots of esoteric statistical functions and a little sleuthing to find layers of error sources.