Nice! So now if you super-impose the four and use color?
The “Down/Up” terminology threw me at first … that also implies a transition state 
1 Like
Well, cooling season is almost over and I decided to do one more monthly correlation analysis between my Ecobee’s hourly cooling runtimes and the power consumed by various Sense AC detections, AC, AC1 and AC2. FurnaceDown and FurnaceUp are hourly power used by the smarplugs monitoring each of my furnaces (mostly the furnace blower used by AC during the summer). Remember that I reset my Sense unit to ground zero in April, then put in new AC compressor units in mid-July. It’s interesting how the detections sort themselves out at the end of the cooling season.
Here’s the story for downstairs…
- No detections for AC in April
- Two AC detection in May, AC and AC2 - “AC” is most closely associated with the downstairs compressor
- New compressors in July upset the status quo. A new detection, AC3, aligns with the downstairs compressor and correlation continues to grow over time.
$`2019-04`
Down_Cool
AC NA
AC2 NA
AC3 NA
FurnaceUp NA
FurnaceDown NA
$`2019-05`
Down_Cool
FurnaceDown 0.96457804
AC 0.40228237
FurnaceUp 0.38490212
AC2 -0.02461567
AC3 NA
$`2019-06`
Down_Cool
FurnaceDown 0.9923947
FurnaceUp 0.6044697
AC 0.3183210
AC2 0.2872561
AC3 NA
$`2019-07`
Down_Cool
FurnaceDown 0.98625399
AC3 0.56490245
FurnaceUp 0.51139072
AC2 0.32950875
AC 0.06935545
$`2019-08`
Down_Cool
FurnaceDown 0.9939775
AC3 0.7953435
FurnaceUp 0.6062248
AC2 0.1969622
AC 0.1884249
$`2019-09`
Down_Cool
FurnaceDown 0.98889843
AC3 0.95784466
FurnaceUp 0.39951440
AC2 0.36528948
AC 0.01010691
Here’s the story for upstairs…
- No detections for AC in April
- Two AC detections in May, AC and AC2 - “AC” correlates very closely with the upstairs compressor runtimes. Why was AC2 even created in the first place ???
- New compressors in July upset the status quo. A new detection, AC3, aligns with the upstairs compressor initially, but eventually lonely AC2 muscles in and correlates best with upstairs.
$`2019-04`
Up_Cool
AC 0.9791817
FurnaceUp 0.9769824
FurnaceDown -0.1541840
AC2 NA
AC3 NA
$`2019-05`
Up_Cool
AC 0.99412114
FurnaceUp 0.97723727
FurnaceDown 0.14122794
AC2 -0.02226503
AC3 NA
$`2019-06`
Up_Cool
FurnaceUp 0.9892451
AC 0.6702095
FurnaceDown 0.5909811
AC2 0.1680601
AC3 NA
$`2019-07`
Up_Cool
FurnaceUp 0.9850746
AC3 0.5987115
FurnaceDown 0.5123134
AC2 0.3918143
AC 0.1948437
$`2019-08`
Up_Cool
FurnaceUp 0.9980361
FurnaceDown 0.6066122
AC3 0.5671459
AC2 0.4513146
AC 0.1011073
$`2019-09`
Up_Cool
FurnaceUp 0.9990777
AC2 0.8910393
FurnaceDown 0.4199229
AC3 0.4070204
AC 0.2234853
I keep promising myself that I’ll delete AC to help improve things, but I kind of want to see if it goes away on it’s own. If I look at the amount of energy that was ascribed to each detection on a monthly basis, I can see AC is on a downward trend, but hasn’t found a route to disappear yet.
Month AC AC2 AC3
1 2019-04 24.005 0.000 0.000
2 2019-05 84.903 0.596 0.000
3 2019-06 314.848 48.725 0.000
4 2019-07 44.516 151.708 260.085
5 2019-08 12.921 215.626 399.085
6 2019-09 16.744 125.794 61.367
3 Likes
One final readout at the end of Sept. I really can get rid of the AC device !
Downstairs correlation by month. AC3 eventually correlates well with Ecobee cooling - 0.94946833
$`2019-04`
Down_Cool
AC NA
AC2 NA
AC3 NA
FurnaceUp NA
FurnaceDown NA
$`2019-05`
Down_Cool
FurnaceDown 0.96457804
AC 0.40228237
FurnaceUp 0.38490212
AC2 -0.02461567
AC3 NA
$`2019-06`
Down_Cool
FurnaceDown 0.9923947
FurnaceUp 0.6044697
AC 0.3183210
AC2 0.2872561
AC3 NA
$`2019-07`
Down_Cool
FurnaceDown 0.98573638
AC3 0.56490245
FurnaceUp 0.51124528
AC2 0.32950875
AC 0.06935545
$`2019-08`
Down_Cool
FurnaceDown 0.9939775
AC3 0.7953435
FurnaceUp 0.6062248
AC2 0.1969622
AC 0.1884249
$`2019-09`
Down_Cool
FurnaceDown 0.96806223
AC3 0.94946833
FurnaceUp 0.56990137
AC2 0.52039402
AC -0.01887556
Upstairs correlation by month. AC2 eventually correlates fairly well with Ecobee cooling - 0.8995683
$`2019-04`
Up_Cool
AC 0.9791817
FurnaceUp 0.9769824
FurnaceDown -0.1541840
AC2 NA
AC3 NA
$`2019-05`
Up_Cool
AC 0.99412114
FurnaceUp 0.97723727
FurnaceDown 0.14122794
AC2 -0.02226503
AC3 NA
$`2019-06`
Up_Cool
FurnaceUp 0.9892451
AC 0.6702095
FurnaceDown 0.5909811
AC2 0.1680601
AC3 NA
$`2019-07`
Up_Cool
FurnaceUp 0.9850562
AC3 0.5987115
FurnaceDown 0.5131142
AC2 0.3918143
AC 0.1948437
$`2019-08`
Up_Cool
FurnaceUp 0.9980361
FurnaceDown 0.6066122
AC3 0.5671459
AC2 0.4513146
AC 0.1011073
$`2019-09`
Up_Cool
FurnaceUp 0.9810785
AC2 0.8995683
FurnaceDown 0.5922095
AC3 0.5786095
AC 0.1425038
1 Like
Here’s a more graphical view on how Sense detections eventually hone in on the actual operation of my upstairs and downstairs compressor units as measured by the Ecobee thermostat commands. The number shown is the weekly Pearson correlation between hourly Sense detection energy usage and my upstairs and downstairs Ecobee cooling runtimes. A correlation of 1 means that there is a perfect linear correlation between energy usage and runtime. 0 means no correlation and a negative number means negatively correlated.
Upstairs Compressor
The upstairs compressor is on for more of the year since my upstairs heats up more (hot air rises, cool air falls and I have a big open entryway). Initially Sense detected my upstairs unit as device AC and had good correlation for a while until the unit was running pretty much all day long. Then, for some reason, a new AC detection from early May, AC2, that hadn’t correlated to anything particularly well, suddenly started matching more with my upstairs usage (and downstairs as well). Then, around July 10th I replaced both compressors and Sense was forced to sort thing out again. Eventually AC2 matched up with my upstairs compressor and AC3 matched up with my downstairs unit.
Downstairs Compressor
My downstairs compressor turns on much later in the season because my downstairs stays cooler.
The best part, is that I actually get a view that Sense is getting smarter as time goes by, despite my efforts to confuse it (new compressors).
EcobeeVsSense3.R (5.2 KB)
1 Like
Put differently, if I look at my results from Sept. 1, and onward, where Sense correlation with my Ecobee is relatively good, I can compute each AC units’ estimated hourly power usage by dividing hourly energy from Sense for AC2 and AC3 devices by the corresponding Ecobee runtimes (in hours). If I look at the distribution of results, I see very reasonable answers for each unit.
My upstairs 3 ton compressor uses about 2.1kW
My downstairs 4 ton compressor comes in at about 3kW
1 Like
One more way to view Sense vs. Ecobee. This one can be done in Excel with a little ingenuity - Detected hourly power usage vs. Ecobee runtime scatter plot. I’ve charted all the hours where the Ecobee reported non-zero cooling runtime. Dots on the power = x axis (power = 0) represent missed detections.
Upstairs - AC ran 360 hours - 30 worst mismatches in red (some red points overlap each other)
Downstairs - AC ran 211 hours - 5 worst mismatches in red
1 Like
Good news ! As the weather gets warmer my cooling system has started to kick in again. And Sense has managed to remember the separate upstairs and downstairs compressor units. The previous winter, Sense forgot all learning it had accumulated the previous cooling season, when it was fairly accurately detecting both units. But Sense seems to not to have drifted this year, at least since I put in the new compressors in July 19. When I look at the weekly correlation, it is very close to 1.0 between each unit on the Ecobee and the most closely associated device. And the mystery device, AC has gone away entirely (OK, I deleted it after it dropped to zero detections). Numbers on the bottom are weekly runtimes in hours.
Upstairs Compressor
Downstairs Compressor
3 Likes
What’s your takeaway? That Sense has gotten better, or that your nearly-failed compressors had enough of a change YoY that Sense had trouble seeing them as the same units? Or both?
Looking back on my old (summer of 2018) experiments, I think Sense is doing a better job. In the summer of 2018, Sense detected 5 different AC devices in a pretty short span and seemed to drift around between associations during different months.
I did a full factory reset in April ‘19 and after that, Sense detected fewer AC devices (2, same as number of compressors) with one being added after I installed the new compressors. Part of the better results might be due to the more stable operation of the new compressors, but the original experiment was done a whole season before the older compressor failed, and I replaced them both.
I think Mahesh has been making some progress, perhaps aided by better “ground truth” from the Ecobee Historic integration.
2 Likes
what a fascinating thread, thanks for downloading, analyzing, and sharing all of your data, I have what is in comparison a relatively simple system comprised of three major components, Heat Pump, Furnace, and Heat Pump electric assist. I find Sense has detected all three but only detects their operation a small percentage of the time. I am hopeful that my recently added EcoBee provides Sense the “ground truth” data needed to improve and that my recently added detail of equipment part numbers and details will help as well.
1 Like
Thanks ! I like having tangible feedback on how well Sense is tracking, and this is the rare case where I can make at least an hourly comparison. I have simple 240V single stage compressors that are easier to “see” than electronically controlled heat pumps of variable speed motors. But presumably Sense can build and test new models more easily with the 5 minute interval data that comes out of the Ecobee.
1 Like
Actually my heat pump is also a simple single stage unit, scroll compressor, fan motor, and reversing valve. Based on the increased cost of variable displacement units when I upgraded the system 5 years ago there was no payback in the higher complexity higher efficiency units for my use case.
All that to say it should be easy for Sense to detect the compressor and fan motor
1 Like
I’m jealous. It is still too early in the year for us so our AC systems have not yet run. The last two years our 2 air handlers and our 2 condensers have constantly been confused with each other. Our downstairs system is ~2.4 ton and the upstairs system is ~3.0 ton.
this year will be better.
1 Like
Wishing you luck. Mine are Bryant 3 and 4 ton SEER 16 systems.
Updated on 7/24 - added 3 more weeks of graphs
I decide to revisit my AC Sense vs. Ecobee correlation results with a little more of the cooling season under my belt. I had gotten a little concerned since Sense seemed to be falling off in it’s ability to correlate with my Ecobee, especially upstairs, where my AC unit was running far more than downstairs. You can see that Sense’s detection of my upstairs compressor (AC2) dipped for a few weeks, below. The number at the bottom indicates the number of hours that the AC had been running for that weeklong period.
I decided to push a little deeper into seeing what happened during the weeks of May 18 th through Jun 22nd, so I converted Ecobee runtime data into energy data based on my upstairs unit running at about 2.5kW (my unit’s rating - in blue), then plotted against the Sense hourly data for my AC2 device (in red). Sense nailed it during the first week and last week in that list, but had some correlation issues in the middle - what was different ?
In weeks 2-5, it looks like Sense has trouble keeping track of all day runs of the upstairs AC, while it seems very accurate on shorter runs or even all day runs with breaks in the middle. I’m guessing Sense might time out, thinking it missed an off signature after 4 hours of continuous running.
1 Like
I can see where this is going …
1 Like
Yup,
-
I could use the 5 min Ecobee data to generate 15 min synthesized energy data that I could employ to cancel out AC waveforms in my PGE data stream, as well as synthesizing the air handler as well. That could help simply EV detection, though I would still have to contend with the floor heaters, which have their own thermostat API similar to the Ecobees, but I haven’t cracked yet.
-
I could also use Ecobee data to clean up my Sense AC detections, for purposes of my usage analysis. Looks like on a bad correlation day, I’m missing 10-15kWh of AC that shows up in Other instead. Maybe I should just be synthesizing the new data and replacing. Hmmm.
I have the same NuHeat thermostats I believe but haven’t bothered with the API yet … and now I’ve freed-up a Sense monitor to dedicate to my one floorheat loop. Ground-truth on the ground! Waiting for Winter.
Meanwhile, I wonder if the non-ramped resistive element in the floor heat (= predictable plateaued load) isn’t somewhat “easy” to synthesize and cull … at least statistically. I suppose you’ve pondered that. Maybe I missed that in your prior work. You’re good at histograms!
If I had a tighter sampling interval than 15 min for energy, I probably could, but based on quick analysis the population of spikes goes up too quickly for continued use with my simple matching algorithm once I get past a delta threshold of 5kWh. I did a simple analysis here - number of candidate edges double with each 1kWh decrease in thresholds. There’s kind of a sweet spot about 5-6kWh that is pretty much occupied by just EVs.
I am going to do a more complex statistical analysis of deltas, Total Usage, neighbors within different window sizes, to see if some kins of smarter matching is possible, but right now, I need more distinguishing features than 15 min delta and Total Usage, to pair up ons and off of spikes above 2kWh hours, where most edges for the heater and AC lie.
1 Like
I decided to take one more deep look at detailed correlation between my Ecobee runtimes and my Sense detection for my upstairs AC compressor (AC2) to see what things look like when Pearson correlation between the hourly usage is good (very close to 1.0) vs. bad (less than 0.7) on a weekly basis. Here’s the weekly correlation chart between hourly energy usage and Ecobee hourly runtime. The numbers on the bottom are the number of hours that the downstairs AC ran that week at about 2.15 kW / hour (3 ton unit). One of my motivations for a more detailed look is that I just replaced my evaporation coil upstairs, as well as the furnace, including the blower/air handler for my AC. I want to see if I see any differences in performance.
First, let’s look at good correlation for the week of 6/22. Here’s the week view on an aggregated hourly basis. Looks quite good except Sense seems to have missed an off-signature or 2 on June 26th.
Is there any way to get a better idea what happened there ? Maybe if we look at hourly Sense vs. Ecobee’s 5 minute runtime data below. Then again, maybe not. Given the number of actual on/off cycles, it’s hard to resolve the Sense hourly vs. the full set of transitions. Also makes me aware of how important it is to line up sampling intervals when doing visual comparisons. For the nerds out there, your simple AC system is run by pulse width modulation (PWM).
Now it’s time to look at far less good correlation the week of June 8. Very clear what the main mode of failure was here. Sense either detected an errant off-signature or decided to time out after what looks to be about 2 hours. June 10th and 11th were a fairly abnormal heat spell where the upstairs AC was pressed to run for 12 hours continuous.
Hour by hour:
Sense hourly vs. Ecobee 5 min.
A final look at one more errant week, May 25th. Looks like the same phenomenon - Sense seeing an off too early on days when the AC is running for uninterrupted for long periods, though Sense looks like it hung in there for much longer on May 25th. I’m wondering if Sense occasionally trips up on my upstairs off-signatures because it interprets an off-signature on my larger downstairs AC unit as its off ? Maybe work looking closely at one of two of these mistaken off events.
Hour by hour:
Sense hourly vs. Ecobee 5 min.
If I push down into 4 days where Sense seems to lose the AC2 (upstairs AC compressor) detection too early, when my Ecobee is trying to run the upstairs AC for many hours, I can add what my upstairs Ecobee is doing in black and my downstairs Ecobee in orange lines. What I see is that there is a lot of on and off activity on my downstairs AC when my upstairs Ecobee is calling for the the upstairs AC to remain on. Given the proximity between Sense losing the AC2 detection and my downstairs AC turning off, there is an extremely good chance that Sense misread one of the downstairs off-transitions as an upstairs off-signature and ended the detection.
1 Like