Is this data export available for the voltage profiles? I can only download the max/min voltages. I would like to download the last 30+ days of 15min or hourly voltage data…
@smbye1, not today. The only exported data for Sense Labs analysis (Motor Stalls, Power Quality, and Floating Neutral) are the actual “error events”. Good suggestion though.
Sense does give you the graph for both legs for a previous 24 hours period which accomplishes something similar to what you are asking for.
Will this or can this feature be added?? Sooner the better…
Thanks.
@smbye11 Since this has been requested once or twice elsewhere, i’ve moved this to the Product Wishlist forum so other people can upvote it. We consistently reference the most upvoted/liked product wishlist threads when gauging prioritization for new features or additions to existing ones.
4 Likes
Hi Senses community! I too would like an export function that would allow for the raw detailed numbers to be saved. Not sure how it could be done, but if we could link a storage account like g-drive, Dropbox, iCloud, and or nextcloud, and have it auto export to a CSV every week. Then that would be amazing!
The power where I live is pretty low quality and has the tendency to kill electronics sooner then they should even when on high quality surge protectors. Since Sense measure from the mains, it would be nice to have that data to show the power company and tell them to update that transformer.
Thank you all for the hard work you have put I. Up to this point.
1 Like
Good suggestion, though in my mind, it would be even better for Sense to fully productize Power Quality into a first-class view, like the Power Meter, where one has a full-history, scrollable, zoomable, alertable, real-time view of both mains. That would also bump off many of the Wishlist items for the Sense Labs version of Power Quality:
https://community.sense.com/t/graphs-of-utility-voltage-for-detecting-issues/3698
ps: I you are a bit handy with computers and want this kind of power quality trace fairly badly, it can be done with HomeAssistant and the HomeAsistant Sense integration today.
2 Likes
Thank you for the Home Assistance integration link. I did setup home HA just a few days ago but haven’t had much time to mess with it yet, maybe I can do something with that.
I see and acknowledge the number of other request for more detailed power quality information and ways to view it.
My credentials are and I.T. professional with over 11 years of experience and I’m a certified AWS Cloud Practitioner. (Not flexing, only providing context for my point of view.) I can see in my network flows that my Sense unit makes regular uploads to AWS S3 buckets (Simple Storage Service) I know that has a reoccurring monthly fee based on amount of data uploaded and amount of data stored over time, and that cost varies depending on the storage tier (speed and access frequency) I personally would like to keep a year of data on my own storage for logging and submission to the power company. I however don’t think that’s sustainable for Sense to pay for considering the product while high priced then competition also has no reoccurring fees. I also, will not pay a monthly subscription for what amounts to storing and occasionally processing a CSV file. I can do that my self.
I also felt it would be beneficial for the short term if my suggestion would be not code heavy. A productized interface would provide a lot of value and would be the way for the company to go long term IMO, especially for less technical people.
I’m on mobile at the moment so I don’t have the date in front of me, but I think the original request for granular data visualization is 3+ years old. Schedule weekly export of raw data CSV to storages with existing APIs (except iCloud) would be reasonable for an experienced Dev to drop in under a month. It’s also something that could be worked into the productized interface later for more advanced users that would like more agency over their data.
I’m not always the best with how I word my response. I hope this doesn’t come off as combative or condescending. And I do thank you for sharing what you did. I know I don’t have the full picture or any insight to the inner workings. Again thank you for the link to HA. It will probably be a while before I have the chance to try, but if I can use that to make simple exports of the L1 and L2 voltages over time, then that would be what I’m looking for.
1 Like
@zechkacey , Appreciate your comments and thoughts since you’re someone with a actual view on the hard costs associated with running a big data operation like Sense’s. For me the subscription-free data history is extremely valuable and I realize the data storage value Sense is providing. Not that many users are cognizant of the costs so think nothing of asking for more “free data”. It would be interesting to hear your opinion on the actual costs. Note that Sense does compress aged Power Meter data via aggregation/reduced resolution (data point per minute) and transfer to cold storage after a few months.
On the other hand, I don’t think this particular request would be costly from a data perspective, because based on what I have seen of Sense’s internal tools in limited videos, they are already storing the voltages in the same persistent database as the Power Meter. Here’s one video that shows some of their internal tools. Use the YouTube Chapter summary to navigate to the last 3 sections.
There’s a bit more on working with Sense-provided L1 and L2 voltages in Home Assistant here.
1 Like
Oh boy did your post have me going down a bunny trail. I now have tabs opened for HA to Sense integrations, HA data retention, HA History/Recorder, and HA and InfluxDB lol!
Thanks for that thread. I have L1 and L2 showing up in History in HA. I will need to read the stuff about InfluxDB and reporting info to that for longer retention later. HA’s history isn’t catching some of the dips that happened. Last night had several dips down to 33v on both legs because of weather. It’s in the Sense labs chart, but not in the HA chart. At that same time HA reports the lowest V as 116.9v. So I will have to see if there is an option to increase the polling frequency on a per sensor bases.
As to compression. I did notice that the data gets compressed. It does look like they at least try to retain and min/max in their calculations so that it doesn’t loose to much resolution over time.
Point of clarification. I recently got my AWS cert. I’m not really practicing yet as my work hasn’t shift my team in to monitoring/managing the AWS side at this time. That is currently up to the Devs. So my experience is minimal, but studding for the cert did give me some insight into the tools that they make available, terminology, and a high level of the products AWS offers. With that disclaimer out of the way, lets continue.
I don’t know if me posting links would get flagged, so I will share how to find the info in stead. AWS has a pricing calculator that can be used by project manager (or anyone for free) to get an idea of how some of their products would cost. It can be found via the good by searching “aws s3 cost calculator” The second link, “Amazon S3 Simple Storage Service Pricing,” is a document that talks about how they do they do the pricing for the various S3 products. There are about 9 S3 products. This is where some studying for the cert can help, as the different products are segregated by how often and what level of performance is needed for the storage. Several of the storage options also get a discount the more GB per month that is stored.
Here is a snip from my firewall of items uploaded from sense to S3. Looks like it varies from 1.1 MB to 2 MB with a few outlier in size, more towards the 1.1 and 1.2 mark in this time snip, so lets call it 1.4 MB?
Times are all over the place though. Nowt sure how they determine when an upload needs to be done. 11min, 19min, I see a step of 34 min… hmmmm… Though I know around 12:30am to 2am I had some power issues. So that probably had an effect on the upload schedule. Lets make an assumption of every 15 min would be the “normal” expected upload.
– I had a picture here but it told me I couldn’t embed media –
Back to S3. There are so many products for different uses, I’m not sure what they are using. Not in small part because I don’t know what data they are uploading. Sense also talks to monitorapi . sense . com, time . apple . com, mb1 . bridge . sense . com, and time . g . aaplimg . com. However, the largest uploads are to s3 . amazonaws . com so I assume that is where the energy telemetry data is going. Given that Sense will analysis the data, run it through ML (probably also hosted in AWS as they have a number of products for that too,) us it for training and identification of devices. We will assume that the data gets used for an amount of time, then not touched for a while except for historical data, where it is probably then transformed to be reduced in size. I would GUESS they are using S3 Intelligent - Tiering. That product will keep data in S3 Standard (the fastest S3 type) until it has not been accessed for 30 days, then it will auto move it to S3 Standard Infrequent Access where there will be some cost savings so long as the data is not touched.
With that said, for this estimate I will run the calculation only of S3 Standard (fastest S3 product) for a worst case scenario. So 1.4 MB (assumption) every 15 min (assumption)
On average 1 month is 43800 minutes. Divide that by 15 (time periods between uploads), 2920 uploads in a month, multiply by 1.4 (assumed average upload) that’s 4,088 MB (big assumed wide margin of error) per month. Divided by 1024 to convert to GB (for price calculator input) that’s 3.9921875 GB (round that to 3.99 GB) per month. For the sake of the calculator, I’m also assuming that each upload is going to be read at least once by server in their VPC (that would be the GET request)
… 1 cent upfront and 12 cents per month? This is my first time actually using the AWS calculator… so… what? I really thought that was going to be more. Dang, their were not kidding that S3 is very reasonably price.
– I had a picture here but it told me I couldn’t embed media –
Now the kicker is that S3 is “Object” storage. So each thing that’s uploaded can NOT be edited. If you need to change one word in a document, for example, you have to re-upload the entire document.
I hope that answers your question. Let me know if that scratches your curiosity itch. Also let me know if there is something that maybe I didn’t explain so well.
1 Like
Side note, that would be 12 more additional cents every month that they hold on to data in S3 standard assuming the same upload schedule. So after a year that would $1.44 on the 12th month for that month if they kept all that data for the whole year without compression and moving it to a slower S3 tier. Then that times however many users.
1 Like
You should be able to add links and images now…The forum is trust-based, so your capabilities increase the more you post.
On sample rates, I think HA is capped at 2 samples per minute by the underlying AP and rate limiting. Sounds like your spikes and dips might be quite short. Sadly, that might limit what you can see and do in HA, plus shows the data challenge of catching short events using sampling. You might need to download the out-of-range Power Quality spike/dip CSV every 30 days. You could probably automate that as well with curl or Selenium with cron.
On data, my Main Sense, with solar and 32 smart plugs and a Hue, uploads about 7GB of data per month. My second Sense, with no solar and DCM, uploads 1.6GB per month. I’ll have to search out the calculator you mentioned and try my numbers our.
1 Like
Would you happen to know of any resources for information on how get that CSV? That would be a big help.
For my home I only have 8 smart plugs setup right now and whatever sense happens to identify. I’m loving those TP-Link Kasa plugs for Apple Homekit. They seem stable, and at least the ones I have say they are rated for 15 amps, and they are reasonably priced. I would appreciate an integration for the EVE plugs though, it would be nice to both add power monitoring and extend my thread network. And if they could get the EVE plugs to work, then they could probably get the EVE light switches to work. I think they also have power monitoring.
@zechkacey,
First off, the CSV spike/dip download is located in the Sense web app, in the Power Quality view (though mine won’t offer much to see):
You can simply pull it every 30 days. To automate, you would need to use a web scraping tool like Selenium to automate:
- Login to the web app at home.sense.com
- Navigate to the Power Quality view
Sense - Find the Download link and either click on it or navigate to the associated URL.
As for power-monitoring EVE Energy devices - There is no Sense integration for them, and AFAIK, there is isn’t an easy way to get power info out of them, except in the EVE app. I bought a couple a while back because I wanted HomeKit-compatible power-monitoring plugs before Kasa offered a HomeKit integration. But the v1 EVE/HomeKit doesn’t expose the power info. I see posts about hopeful users with the v3 EVE Energy, but even that doesn’t look promising.
The only way I have found to pull the energy data is via the EVE app, using the Measurements tab buried deep inside.
1 Like