Our Sense coverage and accuracy


#1

I got a Curb in October and have been comparing the data to Sense. Curb is a simple monitor that just deploys CTs on circuits.

I now have 2 Curbs and 6 extra CTs. The standard Curb comes with 8 30 amp CTs. It can hold up to 12, though usually 2 are reserved for the mains. I can monitor 22 circuits. My switch box has 40 circuits. Some 240V circuits are balanced so only one leg needs to be monitored. But some devices sometimes treat the 240V as 2 120’s for some of their components. Dryers for example. I find that our air conditioners are fairly well balanced, but one side may be different by (on average) a constant. You can also combine circuits in one CT.

I’m still mapping out my breaker box. I know should have done that long ago when it was replaced, but… I was surprised to see a 240V double pull supporting 2 separate 120V circuits. Also surprised to find 2 wires going into the 15A same breaker – our basement lights (~700W) and some basement outlets (sump pump). I’ve haven’t been able to use my radial arm saw from a basement outlet. I think I understand why now. i.e. I usually have the lights on when using the saw. I’ll be using a different electrician in future.

One nice Curb feature is that a CT can be further away. You can extend the wire with an Ethernet cable. So I could monitor my solar feed (outside) without moving the hub. Sense would require me to move the hub outside.

I have real time data from both Curb and Sense.

Note that the real time data from Sense is fraught. We must always be careful to compare it to the more processed (cooked) data. There are often large differences. This appears to be by design. Sense has a patent on the idea that the real time data should be further cooked later. There is a cooked version of the data with 1s resolution available on the phone app (Power Meter). Currently I can only process the hourly and real time Sense data. The per device 1s data on the phone is just an image. It would sure be nice if Sense would facilitate letting us acquire that data for our own processing. Maybe just disclose the API. Failing that, we have to try to decrypt the Sense phone app session. That looks possible, but painful.

Curb data has some issues. Hourly data is downloadable, but only for the current month. The hourly data is often updated a day or so later. Thus data for the last few days of the month is sketchy.

The downloaded data identifies each circuit by the current, user provided, name. I move the CTs around a lot so I have to track those changes myself rather than relying on Curb to use the names I provided.

I dropped one CT. It broke.

Curb support is a problem. My first contact took 5 days for a response. I waited over a month for the last response and most have been ignored. Curb provides API access to their data. They also provide a usage example on github. It took me a couple weeks to get access to their real time feed. Wireshark was much more useful than their documentation. In theory they’ll provide a client id for apps like mine. Still waiting for that. There is phone support, but they hang up after a couple minutes waiting. You can leave a number, but…

I’m not happy with my Curb app. I don’t want to publish it until I get my own client id going. It uses their demo client id which appears to have authorizations specific to their demo app. They say they change that id periodically…

A regular Curb hub supports 12 CTs. The “Pro” supports 18 CTs. The Pro is not available to home owners directly. However, if you order a regular curb and request 6 more CTs, they say they will automatically ship a hub with additional 6 CT capacity. You can also send a follow up email to sales@energycurb.com saying exactly what CTs you want. I wish that had been mentioned somewhere when I was buying my equipment. I ordered a 3rd Curb so I’ll end up with 3 12 CT Curb hubs instead of 2 18s. And a bunch of 100A and 50A CTs I have no use for.

Needless to say, I much prefer the way Sense treats customers.

Everyone generally agrees on the house total consumption. Although, I noticed 1 hour where Sense disagreed with everyone else (ConEd utility, Solar, Curb, Smappee). Discrepancies are rare. I haven’t tried to check the rest of them out.

Sense knows about my smart plugs, but I haven’t told it what’s behind them. Some of them have multiple devices behind them. Most do not have a Sense identified device behind them.

I’ve aligned the smart plug data with Curb. I’m regarding Curb and smart plug data as authoritative except where problems are obvious.

I ignore Always On and Other, though I provide Other (Gray) as total less the device sum for both Curb and Sense. I don’t see much value in Always On. That seems harsh, but I don’t see it as actually reflecting the Always On devices, and the value of that, even if it were done accurately, seems low to me. And now, with Smart Plug values being subtracted from it…

My Curb equipment has been evolving, providing a more comprehensive accounting. Currently, I seem to be around 90% for Curb and Plugs and 40% for Sense. With the 3rd Curb, I should hit 100% coverage.

It’s Winter now. Summer is the more relevant season. I’m focusing on the higher consumption devices first. I’m interested in Dryers, the air conditioners, Fridge2, MyDehumidfier, and Dehumidfier(sic). I present some Dryer and AC data below. I haven’t sorted the others out yet. I’m suspicious that there is something more subtle going on with them. I’m not sure of their identification and they may have morphed over time.




Dryers is the most accessible as I can compare more directly against Curb since they’re still active in the winter. We have two clothes dryers, but Sense doesn’t distinguish them. I tried switching phases on one, but that didn’t help. Everyone said it wouldn’t help. They were right. Curb, of course, can monitor both phases on each dryer separately. We haven’t been using the 1st Floor Dryer much recently so most data is for the Basement Dryer.

Since 12/06/2018 when I started monitoring the dryers with Curb, Curb has reported about 30% more consumption than Sense. The amount varies between 25% and 50% for recent weeks. For individual hours, it varies much more.

Sense seems to miss the dryer tumbler motor. It frequently misses one phase of the 2 phase 240V consumption.



I’m less confident with individual samples of AC runs. In my spot checks Sense sometimes misses AC altogether, sometimes reports higher values than Curb, and sometimes agrees with Curb. Sense does not see the 2nd Floor Air Handler which I run 30 minutes every hour. I wonder if Sense wonders about me running the AC sporadically with temps in the 20s. Watching the AC in context, in the summer would be a better test.

As we know, Sense has trouble detecting off times. It will often report the exact same value for many intervals. The plots below show Sense missing the air handler running alone, reporting higher consumption than Curb, for a longer period of time. The cooked data (Power Meter) cut the on-time back a bit from the real time data, but not all the way back to what Curb reports. To data Curb reports have been consistent with my observations. 2nd Floor Air Handler Phase A and 2nd Floor Condenser Phase B are estimates based on the other phase, but they have been reliable so far.



#2

Cool comparisons. Glad you could share the pros and cons of each. Appreciate the comprehensiveness of the Curb approach, but given my 400A service, with 2 breaker boxes and 68 different circuits, I’m not really a candidate for Curb. Plus I like the fact that Sense maintains my historic data for me - that’s a major consideration as they continue to learn.


#3

I don’t know that Curb is the right solution. It is a lot more money. Financially, they’re 10th out of 10 in their class (whatever that is). I don’t know that they will survive. Given the way they treat customers, I don’t know that they should. Right now they’re just a data source.

If I had to recommend a system for someone with 400A service, I would have more work to do than I’ve done. A year from now, I would hope it would be a no brainer to recommend Sense.

Our home may not be typical. But the above data does seem reasonable cause for concern to me. I can imagine Sense product management has interesting avenues to explore. Maybe they know that Sense PhDs can up their game. We’ve just had some progress in device identification announced… Maybe they could incorporate Curb type CTs and add detail breaking down devices on each CT. Maybe they could emphasize serving as a hub for aggregating smart plugs. AFAIK, no one else is doing that yet. I do worry about unknown Sense identified devices residing behind smart plugs, though. That has got to be a data headache.

Additional CTs provide clarity and accuracy. The big 240V circuits are a good example. But they’re pricey and in themselves tend to bunch things we might not want bunched.

For my part, right now, as a beta tester, I want to contribute an honest assessment of where we stand with Sense. I think there is work to be done, but I wouldn’t count Sense out.


#4

I suppose for 2 breaker boxes with Sense one would get 2 Senses for $600. With Curb, maybe it would be 2 Curb Pros for $800 + $240 for extra probes = $1040.

Sense will get low coverage but can surface individual devices on a circuit. Given the above, I worry about accuracy, but it would be better if someone else confirmed that result.

Curb could get 100% coverage with good accuracy if you double up on a lot of the circuits. I expect high consumption devices will be on their own circuits, but coffee pots and toaster ovens, etc. would be indistinguishable. AFAIK, Curb doesn’t support smart plugs so, for now at least, they seem stuck at the circuit level unless you’re willing to do your own processing.

For all their faults, Curb does have a really nice graphic of your consumption. It is stacked more or less like mine, but it is interactive and you can just click a circuit in the plot and it will zero in on that circuit. Click again to get back to the full graphic. I haven’t seen that before. They use nvd3 to get it done.

I notice that smart breaker boxes are coming. I wouldn’t be surprised to see them in new houses…


#5

FWIW, it turns out (as I understand it) that one Curb can handle up to 9 breaker boxes - - with ever decreasing precision. At 9, you would only know what each box is consuming, not any individual circuits.


#6

Thanks for this thorough comparison and the kind words. There’s some great points here, so I passed it along to the team as well.

A couple little things I can comment on…

Power Meter is actually 1/2s resolution, but your point stands about the difficulty of export there.

We’d really love if you’d tell us what’s behind them! That’s what enables us to get better ground truth data and thus improve device detection down the line.

We actually do have extension cables for solar CTs. You can find them here: https://store.sense.com/collections


#7

What’s behind.
I do want to help Sense learn, but right now I’m kind in audit mode. I do have a few plugs with a single Sense device behind them, but I’m expecting that I’ll use them to see how accurate Sense is rather than joining the learning effort. In fact, I think it would help if a few (but not all) of the Beta team did the same.

Extension
That’s good to know. Support told me I’d have to move the whole unit outside.