What's new in Web App v4: Data Export


#21

I appreciate this new feature, but it appears to have relatively limited use at the current resolution.

I am looking for much higher resolution data. I think second or sub-second is necessary for meaningful analysis.

Uses for “second-level data”:

  • Calculate short-term usage of unidentified systems (High resolution data allows me to look at the information, and pick out the systems that sense has not yet identified, and calculate their consumption).
  • Create aggregate energy consumption patterns for complete systems (i.e., combining dishwasher, well pump, and water heater to identify the total power consumption of the system, rather than just the dishwasher).
  • Track usage spikes and evaluating how to limit aggregate consumption if necessary for current limited systems.
  • Identify areas for improvement on combined or custom systems to improve efficiency and reduce consumption (i.e., the control system on a home brew rig could have timing adjusted to smooth or reduce consumption).
  • Identify timing of events to determine cause and correlate usage.
  • Evaluating multiple configurations of a single appliance (i.e., a washing machine at full load vs half load, where is the actual savings? Is it just in water usage, or are there other meaningful improvements?)
  • Troubleshooting possible appliance issues (i.e., an LED lighting system often seems dim after time, checking the energy consumption profile during the time it is on could reveal reduced power consumption over time suggesting overheating could be an issue).

Sense is a very useful tool, but without the granular details for export and analysis, it is very difficult to evaluate and / or to combine with other information.

Thank you, Chris


Time of use pricing
#22

I noticed that the data shows “mains” and individual devices during the same hour. Is the data accumulative? or does mains already contain the wattage for the given devices during the hour ?
E.g. I have this and are wondering if the 2.935 kWh includes the individual ones or not…

|1/6/18 11:00|mains|Total Usage|||||2934.925|2.935|
|1/6/18 11:00|65faba51|Oven|Oven||||42.45|0.042|
|1/6/18 11:00|g0Tx2WEZ|Stove|Stove Top||||348.015|0.348|

Thanks
-Stephan


#23

Total Usage (mains) is the total sum usage for the house, and is the sum usage of all identified devices, including Always On, plus Other (which is not included in export today).


#24

This is just a thought. Maybe for users (like me) who want more than hourly data, the app would log realtime to a file on the remote device. Would that alleviate any of the issues currently limiting the resolution to hourly?


#25

I agree with this. I would pay a small/nominal yearly fee in order to access this.


#26

Or even just expose an endpoint that we can subscribe to on the device itself (pub/sub) if possible, then we could access some of the raw data.


#27

Definitely appreciated. The increased granularity to 15 or 5 minute would benefit us as a community, and allow for some very interesting integrations to happen. Then all of a sudden everyone wanting to use it with their smart home hubs and servers would be flocking to buy Sense.


#28

Agreed! Nothing beats free advertising just by letting smart folks use raw data on their own time.


#29

+1 for higher resolution data and API access. @cgray raised some good use cases. I’d like to be able to analyze heating/cooling duty cycles vs. thermostat settings and outdoor temperature.


#30

This is great that you can export. I would love to use this data to see if it is more cost effective for me to switch to “Time Of Use” metering. https://www.coned.com/en/save-money/energy-saving-programs/time-of-use

Has anyone been able to work on a formula to figure out total KwH for a certain period (ideally a year) To figure out total cost of “On Peak” vs “Off Peak”? I got it in excel but can’t seem to get a formula right.


#31

I built a ToU calculator in R rather than Excel, because it seemed actually easier… Not sure how your ToU or billing cycles work, but with my utility, there are 6 different rates - 3 per season, but the schedule is different between weekdays and weekends/vacation days.

The easiest way to do it in Excel or R is to do things in a few stages:

  • Start with an hourly usage download from Sense for whatever time period you want to use
  • Write an equation that converts the DateTime (day plus hour) into the name for the rate schedule for that hour. For me that was complicated because:
    • I had to mark HOWE (holidays and weekend) days “TRUE”
    • I had to also mark days as WINSUM (winter or summer) depending on my utilities calendar
  • Here are the two equations that did the TOU logic for each hour, once I knew HOWE and WINSUM for each hour of the download. The end result is a text name in TOUPeriod column.
# Mark each hour with TOU tags
DownloadEnergy$PERIOD <- ifelse (DownloadEnergy$HOLWE, 
                                 ifelse ((hour(DownloadEnergy$DateTime) > 14 & hour(DownloadEnergy$DateTime) < 19), "On", "Off"), 
                                        ifelse ((hour(DownloadEnergy$DateTime) > 13 & hour(DownloadEnergy$DateTime) < 21), "On", 
                                               ifelse (((hour(DownloadEnergy$DateTime) > 6 & hour(DownloadEnergy$DateTime) < 14) | (hour(DownloadEnergy$DateTime) > 20 & hour(DownloadEnergy$DateTime) < 23)) ,
                                               "Partial", "Off")))
DownloadEnergy$TOUPeriod <- paste (DownloadEnergy$WINSUM, DownloadEnergy$PERIOD, sep='')

  • Once you have tagged every hour with the rate schedule for that hour, you can aggregate that column by doing a pivot table in Excel for that block of data, doing a count of entries in the TOUPeriod column.

#32

Thanks, I’ll have to give this a shot.