What would it take to open up your integrations system as an API platform that we could contribute to? As you no doubt are aware, there are just way too many types of smart devices out there and there just isn’t any way your small team could possibly create official integrations for everything that is needed. Why not enable us to build custom integrations that conform to your predefined interface? Imagine how much more excellent data you’d get in return!
Cool idea, but I’m betting that that is much harder than it might seem for both parties. As far as I can tell, the functionality for collecting integration data is built into the monitor firmware and is custom for each devices’ (HS110, WeMo Insight and Hue) specific API. I’m guessing that only the most capable amongst this audience, not including me, would be able to write robust firmware routines for real-time polling, data acquisition and integration into the uploaded streams.
To give you some idea of what the IF looks like, for the HS110, the Sense monitor sends out an HS110s specific broadcast to all the HS110 active port every 2 sec requesting status data. They respond (or not) and the Sense probe accumulates the time-stamped data, for eventual upload.
The Hue model might be a little easier. I’m not sure if the Sense polls the Hue, or if the Hue messages the Sense monitor when there is a change in a light(s)’ specific level, along with the new estimated power/energy consumption for the new setting.
Pretty sure that Sense polls the hue bridge, don’t recall the frequency. Hue responds with an xml of all the lights and thier statuses. If I recall, sense polls group changes every 15 minutes, which is ok since you likely aren’t moving lights around frequently.
Is anyone else here using a socket connection with one of the unofficial API’s? If so, have you noticed weird issues if you’re pulling data via API and then you open the Sense mobile app and try to view the real-time meter? I used to be able to do this just fine, but lately my API agent seems to basically kill the real-time meter in the app.
I just recently started pulling from the WebSocket too. I do notice that if you over connect (reconnect to the socket too many times in a row) it will lock it up temporarily. I’d imagine it’s some sort of rate limit detection.
I’m thinking that, too. My websocket connection rarely reconnects once the connection is going and has kept itself running just fine for weeks at a time. It’s only recently I’ve started seeing an impact on the real-time meter if my agent is also running.
Really want an API here too! Would be great to use the solar production to know if it’s a cloudy day and adjust internal lighting accordingly
wow, what a neat idea. like playing minecraft with your house
The informal API details helped me head down a decent path to grab some data and hack things together. I’m by no means a developer/coder - but I do like my metrics! I pull the API and segment out details around my overall consumption as well as individual devices. I pull the web-socket for a few seconds and then kill the connection. I really just need a snapshot - wish there was a way to get a single polling of that data than a kill -9 on the script!
But it does make for some fun graphs:
Any chance you guys could look into what @brbeaird reported above?
The integration with SmartThings he developed goes a long way towards making the data generated by Sense useful and actionable beyond just looking at bubbles. I think it’s a big opportunity for Sense to gain some traction in the established ST HA market without much resource cost if you don’t intend to offer an official API to access our data. Accessing the data using his integration is limiting the functionality of both the Sense mobile app and home.sense.com - which in either case should probably be looked at since it’s something that could potentially affect support in other instances as well. Thanks in advance!
We have rate limits in place, which might be causing his issues. Otherwise, it could be related to the real-time services issues mentioned in Data Gaps in Timeline/Meter, which we’re still working on. He’s certainly welcome to write into Support and they can advise if it might be related to the RTS data gap issue.
It’s not data gaps (at least in my case) - the data, when it loads, is continuous and complete in the app. It’s just the real-time monitoring from the app that’s slow. The odd part is that it’s the Sense mobile app that’s impacted, I would think it would be the other way around.
Were rate limits changed or added recently, say in the last 1-2 months or so? That’s when the issue cropped up - it hadn’t been a problem before then.
When I’ve dealt with rate limits, those typically come up when working with excessive polling using HTTP requests. In this case, it’s a live websocket connection, so I don’t think it’s polling in the same way.
What seems most likely to me is there was a “concurrent websocket” type limit added a couple months ago that makes it impossible to listen on the websocket in more than 1 instance at a time.
Pardon the delay. Had to reach out to the Eng team. Yes, rate limits were implemented recently as websocket misuse was negatively affecting the app experience of other Sense users. There was some egregious over-polling happening. This rate limit happens in situations that you’re describing @brbeaird, where more than two concurrent clients are connected, as those are the scenarios that hit our systems the hardest.
Note that these are not officially supported means to access Sense data. Ultimately, we need to protect the core experience of the majority of our users. We’re certainly interested in providing deeper access to Sense data in the future — whether through the WS or a full, documented API — but dev resources must be located elsewhere for now.
@RyanAtSense can you share what the rate limit rules are so that those of us polling don’t exceed them?
No doubt - and no complaints from me here. If multiple websocket connections cause strain, that’s definitely understandable.
For future reference, the main things I’m looking for via API are:
- Event hook that a device turned on/off
- Relatively fresh numbers on each device’s current usage
I realize IFTTT was intended to help with the first one, but it’s limited in that you have to manually set up an applet for every single device, which I found to be too tedious. If there were an option to just send ALL events to an IFTTT trigger that I can then sort out myself later, that would work. On a related note, the current websocket connection refreshes far more frequently than necessary to satisfy both of those requirements. One idea might be to put out an additional websocket endpoint that only pushes data once a minute or so.
If you get some developers freed up sometime in the future and need some people to bounce ideas off or test things, feel free to hit me up.
Thanks for looking into it and getting back to us.
I’ve been able to successfully retrieve historical data using the API published here, but I get an error when attempting to stream active data using web sockets.
Invoking update_realtime returns this error: ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1045)
Anyone else having better luck? Is this something with the server API or is it user-error? I’m still hoping to use the active value to control lighting in my home.
Websocket is still working for me over nodeJS, although I’m modifying it to only hold the stream open for one tick of data to lessen the load.
Any tips about what I might be doing wrong would be greatly appreciated! Do I need to have an SSL certificate?
Are you trying to run this behind a firewall? I regularly get SSL verification errors at work because we have a firewall that basically sits in the middle of Internet SSL and client machines. As a workaround, I usually have to disable SSL verification on whatever app I’m using that verifies by default.