What's new in Web App v4: Data Export

I appreciate this new feature, but it appears to have relatively limited use at the current resolution.

I am looking for much higher resolution data. I think second or sub-second is necessary for meaningful analysis.

Uses for “second-level data”:

  • Calculate short-term usage of unidentified systems (High resolution data allows me to look at the information, and pick out the systems that sense has not yet identified, and calculate their consumption).
  • Create aggregate energy consumption patterns for complete systems (i.e., combining dishwasher, well pump, and water heater to identify the total power consumption of the system, rather than just the dishwasher).
  • Track usage spikes and evaluating how to limit aggregate consumption if necessary for current limited systems.
  • Identify areas for improvement on combined or custom systems to improve efficiency and reduce consumption (i.e., the control system on a home brew rig could have timing adjusted to smooth or reduce consumption).
  • Identify timing of events to determine cause and correlate usage.
  • Evaluating multiple configurations of a single appliance (i.e., a washing machine at full load vs half load, where is the actual savings? Is it just in water usage, or are there other meaningful improvements?)
  • Troubleshooting possible appliance issues (i.e., an LED lighting system often seems dim after time, checking the energy consumption profile during the time it is on could reveal reduced power consumption over time suggesting overheating could be an issue).

Sense is a very useful tool, but without the granular details for export and analysis, it is very difficult to evaluate and / or to combine with other information.

Thank you, Chris

3 Likes

I noticed that the data shows “mains” and individual devices during the same hour. Is the data accumulative? or does mains already contain the wattage for the given devices during the hour ?
E.g. I have this and are wondering if the 2.935 kWh includes the individual ones or not…

|1/6/18 11:00|mains|Total Usage|||||2934.925|2.935|
|1/6/18 11:00|65faba51|Oven|Oven||||42.45|0.042|
|1/6/18 11:00|g0Tx2WEZ|Stove|Stove Top||||348.015|0.348|

Thanks
-Stephan

Total Usage (mains) is the total sum usage for the house, and is the sum usage of all identified devices, including Always On, plus Other (which is not included in export today).

This is just a thought. Maybe for users (like me) who want more than hourly data, the app would log realtime to a file on the remote device. Would that alleviate any of the issues currently limiting the resolution to hourly?

I agree with this. I would pay a small/nominal yearly fee in order to access this.

Or even just expose an endpoint that we can subscribe to on the device itself (pub/sub) if possible, then we could access some of the raw data.

1 Like

Definitely appreciated. The increased granularity to 15 or 5 minute would benefit us as a community, and allow for some very interesting integrations to happen. Then all of a sudden everyone wanting to use it with their smart home hubs and servers would be flocking to buy Sense.

Agreed! Nothing beats free advertising just by letting smart folks use raw data on their own time.

+1 for higher resolution data and API access. @cgray raised some good use cases. I’d like to be able to analyze heating/cooling duty cycles vs. thermostat settings and outdoor temperature.

1 Like

This is great that you can export. I would love to use this data to see if it is more cost effective for me to switch to “Time Of Use” metering. 404 ERROR

Has anyone been able to work on a formula to figure out total KwH for a certain period (ideally a year) To figure out total cost of “On Peak” vs “Off Peak”? I got it in excel but can’t seem to get a formula right.

I built a ToU calculator in R rather than Excel, because it seemed actually easier… Not sure how your ToU or billing cycles work, but with my utility, there are 6 different rates - 3 per season, but the schedule is different between weekdays and weekends/vacation days.

The easiest way to do it in Excel or R is to do things in a few stages:

  • Start with an hourly usage download from Sense for whatever time period you want to use
  • Write an equation that converts the DateTime (day plus hour) into the name for the rate schedule for that hour. For me that was complicated because:
    • I had to mark HOWE (holidays and weekend) days “TRUE”
    • I had to also mark days as WINSUM (winter or summer) depending on my utilities calendar
  • Here are the two equations that did the TOU logic for each hour, once I knew HOWE and WINSUM for each hour of the download. The end result is a text name in TOUPeriod column.
# Mark each hour with TOU tags
DownloadEnergy$PERIOD <- ifelse (DownloadEnergy$HOLWE, 
                                 ifelse ((hour(DownloadEnergy$DateTime) > 14 & hour(DownloadEnergy$DateTime) < 19), "On", "Off"), 
                                        ifelse ((hour(DownloadEnergy$DateTime) > 13 & hour(DownloadEnergy$DateTime) < 21), "On", 
                                               ifelse (((hour(DownloadEnergy$DateTime) > 6 & hour(DownloadEnergy$DateTime) < 14) | (hour(DownloadEnergy$DateTime) > 20 & hour(DownloadEnergy$DateTime) < 23)) ,
                                               "Partial", "Off")))
DownloadEnergy$TOUPeriod <- paste (DownloadEnergy$WINSUM, DownloadEnergy$PERIOD, sep='')

  • Once you have tagged every hour with the rate schedule for that hour, you can aggregate that column by doing a pivot table in Excel for that block of data, doing a count of entries in the TOUPeriod column.

Thanks, I’ll have to give this a shot.

Plus 1 for allowing download at finer time resolutions.

I would think 15 min, 5 min, 1 min and 1s options would allow different people to do lots of different things. If it’s easier for the sense folks, please start by allowing e.g. 15 min and 5 min soon, and the others could come later.

1 Like

You know how you can receive notifications when a device turns on and off? Is it possible to download a list of those event times, i.e. the on/off thresholds and relative wattage amounts? If not, can I request the ability to do so?

I’m trying to compare the list of dates and times when Sense has correctly detected my garage door opening or closing, vs. the couple of times that it has given a false positive (by examining the history data from the Chamberlain MyQ app.)

Once I have this info, I’ll be contacting Sense support to ask how to flag specific instances as false positives. Thanks!

You can probably do some of what you want by having Sense IFFFT events write entries into a Google sheet. But you can’t get a previous history of that from Sense, only forward going once you enable IFFFT correctly.

You can get a .csv export directly out of the Sense web app for your entire history, but it will either be the hourly or daily consumption for every detected device.

Kevin is right. No native way to do that at the moment, but IFTTT could get it for the future or you could do some pruning of your data export.

the only place I can see a export icon is in the Trends page of the Web App - this downloads the trend as daily or hourly device consumption.

I’m interested in the aggregated results of my consumption (and one day generation) on a hourly basis - is there any way to get at that data in an efficient manner?
And how long a time-window can I still see an hourly resolution?

Thanks,

Export will give you hourly Total Consumption, Solar Generation plus ALL individual hourly device data for up to an entire calendar year in a single download. From there, you can do just about anything you want in Excel, R or Python.

OK - thanks a bunch - all the data mixed in together confused me - but now I see that the mains are labelled and track throughout.

Can’t wait to get some solar and generation on here too.

BTW - does Sense have a storage monitoring solution? Could it be integrated with a Telsa battery wall or equivalent?

Cheers,

No dedicated Sense monitor for the battery backfeed/charging paths. But a number of Sense users figuring out the different approaches: