RyanAtSense,
I know this topic is old and past, but for myself… All of this is new.
The hourly data currently provided is good for budget analysis and not much more… But I would be anxious to learn of other uses.
“What would I use more refined data for?” My entire tasking since installation is to correlate detected devices and match them to the devices I have in operation. My “Device 1” is so random… Without overlaying it with other things, I stand no chance in catching it for identification. Meanwhile, I am running all over the house with a KP115 but having no luck. I even installed the dedicated add-on sensors, moving about circuit breakers, but without seeing it live or in greater detail I stand no chance. A suggestion, in recent activity, 15 Minute segments of collected data in one minute increment? I could see that as helpful. Not meaning to appear disrespectful… We all know that we are streaming extremely detailed data out, I don’t think it is too much to ask for a way of getting some back As that is the only way I know of since we can not overlay on the “Meter”. By the time something triggers: “Device One Just turned on” it is over, “Device One Just turned off”.
I am completely open to listen to the wisdom of the data junkies that came before me… If someone knows, please let me know. It has to be better than sitting in a chair with a tablet starring at a device that is on randomly for 30 seconds to verify. Advice… Anyone?
Two thoughts:
- If you are handy with Python and don’t mind storing your own data, you can use the informal API. @MarkDodrill posted a sample here a few days ago.
- If you want a data collection platform that you can used to overlay graphs, consider using Home Assistant. It has a built-in integration using the Sense informal API, plus allows you to use InFluxDB and Grafana add-ins to graph the collected data.
Sense - Home Assistant
I tried the links but get
Is there an application I need to install first? Open communication in PF Sense, my Firewall?
I’m betting you hit the install button for the Sense integration. That won’t work until you have an instance of Home Assistant up and running on either a server in your house or on dedicated hardware. I run mine on an old Intel NUC using the Home Assistant Operating System option.
Warning - there’s a moderate amount of configuration required to get Home Assistant, InfluxDB, and Grafana up and running and connected. There’s no real programming required and all the setup is well documented, but it’s not like just running an application - you are building and configuring a server platform with a bunch of tested and ready modules.
Would this be a Virtual Machine in any particular OS? Or a dedicated Machine/OS? Any link to detailed operation/install would be extremely appreciated. I have two servers currently in operation; Dell T110 with 15TB storage mainly for Surveillance cameras, and a Dell R710 with approximately 40TB as Files Server, Media Server, Web Server to name a few. I definitely have the storage space for data and adding more is easily accomplished. It is just the quest/tasking of how to extract data for analysis. The hourly data is excellent once you have everything identified. Good for budget analysis, etc. My desire… If I could only capture: Current waveform of Input 110 Legs consumption (Divides my search by 2!). Device On-Off on a time line/Spreadsheet. Of my 15 reported devices on Sense, I have only 3 that I know for sure what they are; Well Pump, Microwave, and Coffee Maker. Everything else is just noise in my background as I can not see or predict the items. For example, another “Pump” is identified. I eliminated the Boilers Condensate Pump (Which is virtually silent) using the KP115. But seriously, How to you verify the Ice Maker Pump, and the other possibilities without overlaying them somehow? Starring at an item waiting for the “On” to be notified is simply driving me crazy… Not to say I have not been there for a long time already. 
If you already have a server and are using some kind of container for VMs, then your can use the Home Assistant Container approach: Standalone container-based installation of Home Assistant Core (e.g. Docker).
Look through the Home Assistant Getting Started docs plus info on the UI/Lovelace and the community
The built in Home Assistant dashboard UI can pretty quickly be configured to see power usage overlaid between different entities (the Dashboards are set up nicely to chart like units coming from Sense - W, kWh)
There are are other techniques for identifying new detections depending on how often they occurs and how predictable they are:
- Breaker by breaker - make it disappear
- The alert and sound method - that’s how I found my sink hot water heater.
@007DJ ,
Here’s what overlaid graphs from Sense look like in native Home Assistant (Lovelace UI) Dashboard. It’s easy to overlay like units. In this case I have asked Home Assistant to chart 48 hours of my two voltage leg plus my biggest energy user and total usage, that all come from the Sense Integration.
The resulting graph charts like units, Voltage on top, Watts on the bottom.
@kevin1 ,
I was finally able to install Home Assistant as a VM in ESXi. I will say it was a real challenge… Until I spotted haos_ova-7.2.ova on the net. I have no idea how I found it (Had too many windows open to trace my steps. TG I saved a copy, because I am going to have to nuke it and start over.in my attempts to do what you did (Create a graph with overlays) Somehow I managed to duplicate everything, then I broke it by eliminating everything. Although, I must agree with you that HA is very helpful.
If you could be so kind, Could you please guide me how to edit “Overview” that is the one view I had that is now corrupt.I have searched the web for help, but the examples there assume an older version that has other functions like editor. I see how to make other overview pages; however everything there is again single icons.History graphs are probably the closest I have so far but I noticed that new sense detections are nowhere to be found. Correct me, if i am wrong, did you earlier say that you could export the data from HA to a file that could be manipulated using external graphing tools?
Hi @007DJ,
My path to to learning and using HA was the step-by-step path below that took a couple of weeks. The key was familiarizing myself with what happens at every step.
Basics
- Load image onto target platform / container and boot HA environment - I’m on 7.2 as well, but using dedicated NUC running HA OS.
- Log into HA server via browser and set up password and profile
- Add a couple of integrations (Sense and Ecobee at the start). If you are coming from a clean start, this step should regenerate the initial Overview
- Add a couple of add-ons (File Editor and Terminal & SSH). File editor is crucial for future configuration.
- Learn about all the items in the Configuration panel - poke into them and read docs.
- Take a look at the Devices and Entities created by those integrations. There will be a mix of binary_sensors and sensor entities. You will mostly want to graph sensor entities in the next step. This a good place to learn the naming conventions used for Devices and Entities and to amend the Device/Entity tables with Location and other information.
- Create new dashboards beyond the Overview. There’s one more “native” dashboard called Energy that has its own set up in Configuration section. That’s interesting, but I would save that for later. Try creating your own History type dashboards (and other types as well).
Now you should be able to create the overlay HA History charts highlighted earlier.
Advanced Topics
- Next I started playing with HACs and more customized integrations. I want to use the Tesla EV HACs integration to pull charging data into my graphs. HACs comes with a bunch of additional less-vetted integrations, that are still fairly well tested and ready for action.
- Once I was able to bring in Tesla data, I discovered the limits of the native HA Dashboard / Lovelace environment. It’s great for charting like-against-like (Watts vs Watts) from different entities, but cannot directly do dual unit charts (W vs charging rate). At this point, you have to choose a course if you want more flexible charting:
-
Configure calculations into the HA Recorder. There are many snippets within the community postings on how to do this, using the Add-in File Editor or another editor within your HA container.
-
Configure the HA Recorder to create an output .CSV or whatever you want. You can also find snippets on how to do this within the HA community.
-
Add the InfluxDB and Grafana add-ins and configure them to ride your HA database. This will require new entries in your main YAML file - the documentation for InfluxDB and Grafana highlight how to do this. InfluxDB creates a separate parallel time series database that pulls in all the data that HA saves via the Recorder (unless you configure differently). Both InfluxDB and Grafana then allow you to query the data, do some calculation and put the resulting data into charts. InfluxDB is the first thing you need to bring up. Once it is up and going and you have a feel for it, Grafana will be easy to add and brings additional flexibility. Both offer CSV output for charted data. I personally use the R language to pull data from the InfluxDB for analysis and charting.
- Configure the HA Energy Dashboard - I’m adding this at the end because that dashboard can be confusing unless you first understand the data flow from Sense to HA. HA samples data from Sense on a regular basis - Active sensors: Updated every 60 seconds, Trend sensors: Data provided for daily, weekly, monthly and yearly scales. Updated every 5 minutes. Because the HA data is sampled on these intervals, you will see some differences between Sense and the HA Energy Dashboard.
Hopefully that’s enough to help you get further down the path this time.
I put my HA build on a raspberry PI. Definatly the easiest raspberry pie build I have ever done. I used a program called balena etcher thats on their site. Put my micro sd card in my laptop, opened balena etcher, clicked start … 5 mins later. Put the SD card in the raspberry PI and I was up and running.
@kevin1, I wanted to first off thank you for assisting me in my new adventure. I agree with your earlier comment about getting started in Home Assistant (HA), as it is definitely very powerful and full of options.
All I have to do now is learn the variables within YAML. I have the language and syntax down pat, it is more of finding all of the devices and the corresponding variables. If you know of any books, web links or anything that would help, please post / e-mail.
This topic was automatically closed 365 days after the last reply. New replies are no longer allowed.