Nice! So now if you super-impose the four and use color?
The “Down/Up” terminology threw me at first … that also implies a transition state
Nice! So now if you super-impose the four and use color?
The “Down/Up” terminology threw me at first … that also implies a transition state
Well, cooling season is almost over and I decided to do one more monthly correlation analysis between my Ecobee’s hourly cooling runtimes and the power consumed by various Sense AC detections, AC, AC1 and AC2. FurnaceDown and FurnaceUp are hourly power used by the smarplugs monitoring each of my furnaces (mostly the furnace blower used by AC during the summer). Remember that I reset my Sense unit to ground zero in April, then put in new AC compressor units in mid-July. It’s interesting how the detections sort themselves out at the end of the cooling season.
Here’s the story for downstairs…
$`2019-04` Down_Cool AC NA AC2 NA AC3 NA FurnaceUp NA FurnaceDown NA $`2019-05` Down_Cool FurnaceDown 0.96457804 AC 0.40228237 FurnaceUp 0.38490212 AC2 -0.02461567 AC3 NA $`2019-06` Down_Cool FurnaceDown 0.9923947 FurnaceUp 0.6044697 AC 0.3183210 AC2 0.2872561 AC3 NA $`2019-07` Down_Cool FurnaceDown 0.98625399 AC3 0.56490245 FurnaceUp 0.51139072 AC2 0.32950875 AC 0.06935545 $`2019-08` Down_Cool FurnaceDown 0.9939775 AC3 0.7953435 FurnaceUp 0.6062248 AC2 0.1969622 AC 0.1884249 $`2019-09` Down_Cool FurnaceDown 0.98889843 AC3 0.95784466 FurnaceUp 0.39951440 AC2 0.36528948 AC 0.01010691
Here’s the story for upstairs…
$`2019-04` Up_Cool AC 0.9791817 FurnaceUp 0.9769824 FurnaceDown -0.1541840 AC2 NA AC3 NA $`2019-05` Up_Cool AC 0.99412114 FurnaceUp 0.97723727 FurnaceDown 0.14122794 AC2 -0.02226503 AC3 NA $`2019-06` Up_Cool FurnaceUp 0.9892451 AC 0.6702095 FurnaceDown 0.5909811 AC2 0.1680601 AC3 NA $`2019-07` Up_Cool FurnaceUp 0.9850746 AC3 0.5987115 FurnaceDown 0.5123134 AC2 0.3918143 AC 0.1948437 $`2019-08` Up_Cool FurnaceUp 0.9980361 FurnaceDown 0.6066122 AC3 0.5671459 AC2 0.4513146 AC 0.1011073 $`2019-09` Up_Cool FurnaceUp 0.9990777 AC2 0.8910393 FurnaceDown 0.4199229 AC3 0.4070204 AC 0.2234853
I keep promising myself that I’ll delete AC to help improve things, but I kind of want to see if it goes away on it’s own. If I look at the amount of energy that was ascribed to each detection on a monthly basis, I can see AC is on a downward trend, but hasn’t found a route to disappear yet.
Month AC AC2 AC3 1 2019-04 24.005 0.000 0.000 2 2019-05 84.903 0.596 0.000 3 2019-06 314.848 48.725 0.000 4 2019-07 44.516 151.708 260.085 5 2019-08 12.921 215.626 399.085 6 2019-09 16.744 125.794 61.367
One final readout at the end of Sept. I really can get rid of the AC device !
Downstairs correlation by month. AC3 eventually correlates well with Ecobee cooling - 0.94946833
$`2019-04` Down_Cool AC NA AC2 NA AC3 NA FurnaceUp NA FurnaceDown NA $`2019-05` Down_Cool FurnaceDown 0.96457804 AC 0.40228237 FurnaceUp 0.38490212 AC2 -0.02461567 AC3 NA $`2019-06` Down_Cool FurnaceDown 0.9923947 FurnaceUp 0.6044697 AC 0.3183210 AC2 0.2872561 AC3 NA $`2019-07` Down_Cool FurnaceDown 0.98573638 AC3 0.56490245 FurnaceUp 0.51124528 AC2 0.32950875 AC 0.06935545 $`2019-08` Down_Cool FurnaceDown 0.9939775 AC3 0.7953435 FurnaceUp 0.6062248 AC2 0.1969622 AC 0.1884249 $`2019-09` Down_Cool FurnaceDown 0.96806223 AC3 0.94946833 FurnaceUp 0.56990137 AC2 0.52039402 AC -0.01887556
Upstairs correlation by month. AC2 eventually correlates fairly well with Ecobee cooling - 0.8995683
$`2019-04` Up_Cool AC 0.9791817 FurnaceUp 0.9769824 FurnaceDown -0.1541840 AC2 NA AC3 NA $`2019-05` Up_Cool AC 0.99412114 FurnaceUp 0.97723727 FurnaceDown 0.14122794 AC2 -0.02226503 AC3 NA $`2019-06` Up_Cool FurnaceUp 0.9892451 AC 0.6702095 FurnaceDown 0.5909811 AC2 0.1680601 AC3 NA $`2019-07` Up_Cool FurnaceUp 0.9850562 AC3 0.5987115 FurnaceDown 0.5131142 AC2 0.3918143 AC 0.1948437 $`2019-08` Up_Cool FurnaceUp 0.9980361 FurnaceDown 0.6066122 AC3 0.5671459 AC2 0.4513146 AC 0.1011073 $`2019-09` Up_Cool FurnaceUp 0.9810785 AC2 0.8995683 FurnaceDown 0.5922095 AC3 0.5786095 AC 0.1425038
Here’s a more graphical view on how Sense detections eventually hone in on the actual operation of my upstairs and downstairs compressor units as measured by the Ecobee thermostat commands. The number shown is the weekly Pearson correlation between hourly Sense detection energy usage and my upstairs and downstairs Ecobee cooling runtimes. A correlation of 1 means that there is a perfect linear correlation between energy usage and runtime. 0 means no correlation and a negative number means negatively correlated.
Upstairs Compressor
The upstairs compressor is on for more of the year since my upstairs heats up more (hot air rises, cool air falls and I have a big open entryway). Initially Sense detected my upstairs unit as device AC and had good correlation for a while until the unit was running pretty much all day long. Then, for some reason, a new AC detection from early May, AC2, that hadn’t correlated to anything particularly well, suddenly started matching more with my upstairs usage (and downstairs as well). Then, around July 10th I replaced both compressors and Sense was forced to sort thing out again. Eventually AC2 matched up with my upstairs compressor and AC3 matched up with my downstairs unit.
Downstairs Compressor
My downstairs compressor turns on much later in the season because my downstairs stays cooler.
The best part, is that I actually get a view that Sense is getting smarter as time goes by, despite my efforts to confuse it (new compressors).
EcobeeVsSense3.R (5.2 KB)
Put differently, if I look at my results from Sept. 1, and onward, where Sense correlation with my Ecobee is relatively good, I can compute each AC units’ estimated hourly power usage by dividing hourly energy from Sense for AC2 and AC3 devices by the corresponding Ecobee runtimes (in hours). If I look at the distribution of results, I see very reasonable answers for each unit.
My upstairs 3 ton compressor uses about 2.1kW
My downstairs 4 ton compressor comes in at about 3kW
One more way to view Sense vs. Ecobee. This one can be done in Excel with a little ingenuity - Detected hourly power usage vs. Ecobee runtime scatter plot. I’ve charted all the hours where the Ecobee reported non-zero cooling runtime. Dots on the power = x axis (power = 0) represent missed detections.
Upstairs - AC ran 360 hours - 30 worst mismatches in red (some red points overlap each other)
Downstairs - AC ran 211 hours - 5 worst mismatches in red
Good news ! As the weather gets warmer my cooling system has started to kick in again. And Sense has managed to remember the separate upstairs and downstairs compressor units. The previous winter, Sense forgot all learning it had accumulated the previous cooling season, when it was fairly accurately detecting both units. But Sense seems to not to have drifted this year, at least since I put in the new compressors in July 19. When I look at the weekly correlation, it is very close to 1.0 between each unit on the Ecobee and the most closely associated device. And the mystery device, AC has gone away entirely (OK, I deleted it after it dropped to zero detections). Numbers on the bottom are weekly runtimes in hours.
Upstairs Compressor
What’s your takeaway? That Sense has gotten better, or that your nearly-failed compressors had enough of a change YoY that Sense had trouble seeing them as the same units? Or both?
Looking back on my old (summer of 2018) experiments, I think Sense is doing a better job. In the summer of 2018, Sense detected 5 different AC devices in a pretty short span and seemed to drift around between associations during different months.
I did a full factory reset in April ‘19 and after that, Sense detected fewer AC devices (2, same as number of compressors) with one being added after I installed the new compressors. Part of the better results might be due to the more stable operation of the new compressors, but the original experiment was done a whole season before the older compressor failed, and I replaced them both.
I think Mahesh has been making some progress, perhaps aided by better “ground truth” from the Ecobee Historic integration.
what a fascinating thread, thanks for downloading, analyzing, and sharing all of your data, I have what is in comparison a relatively simple system comprised of three major components, Heat Pump, Furnace, and Heat Pump electric assist. I find Sense has detected all three but only detects their operation a small percentage of the time. I am hopeful that my recently added EcoBee provides Sense the “ground truth” data needed to improve and that my recently added detail of equipment part numbers and details will help as well.
Thanks ! I like having tangible feedback on how well Sense is tracking, and this is the rare case where I can make at least an hourly comparison. I have simple 240V single stage compressors that are easier to “see” than electronically controlled heat pumps of variable speed motors. But presumably Sense can build and test new models more easily with the 5 minute interval data that comes out of the Ecobee.
Actually my heat pump is also a simple single stage unit, scroll compressor, fan motor, and reversing valve. Based on the increased cost of variable displacement units when I upgraded the system 5 years ago there was no payback in the higher complexity higher efficiency units for my use case.
All that to say it should be easy for Sense to detect the compressor and fan motor
I’m jealous. It is still too early in the year for us so our AC systems have not yet run. The last two years our 2 air handlers and our 2 condensers have constantly been confused with each other. Our downstairs system is ~2.4 ton and the upstairs system is ~3.0 ton.
this year will be better.
Wishing you luck. Mine are Bryant 3 and 4 ton SEER 16 systems.
Updated on 7/24 - added 3 more weeks of graphs
I decide to revisit my AC Sense vs. Ecobee correlation results with a little more of the cooling season under my belt. I had gotten a little concerned since Sense seemed to be falling off in it’s ability to correlate with my Ecobee, especially upstairs, where my AC unit was running far more than downstairs. You can see that Sense’s detection of my upstairs compressor (AC2) dipped for a few weeks, below. The number at the bottom indicates the number of hours that the AC had been running for that weeklong period.
I decided to push a little deeper into seeing what happened during the weeks of May 18 th through Jun 22nd, so I converted Ecobee runtime data into energy data based on my upstairs unit running at about 2.5kW (my unit’s rating - in blue), then plotted against the Sense hourly data for my AC2 device (in red). Sense nailed it during the first week and last week in that list, but had some correlation issues in the middle - what was different ?
In weeks 2-5, it looks like Sense has trouble keeping track of all day runs of the upstairs AC, while it seems very accurate on shorter runs or even all day runs with breaks in the middle. I’m guessing Sense might time out, thinking it missed an off signature after 4 hours of continuous running.
I can see where this is going …
Yup,
I could use the 5 min Ecobee data to generate 15 min synthesized energy data that I could employ to cancel out AC waveforms in my PGE data stream, as well as synthesizing the air handler as well. That could help simply EV detection, though I would still have to contend with the floor heaters, which have their own thermostat API similar to the Ecobees, but I haven’t cracked yet.
I could also use Ecobee data to clean up my Sense AC detections, for purposes of my usage analysis. Looks like on a bad correlation day, I’m missing 10-15kWh of AC that shows up in Other instead. Maybe I should just be synthesizing the new data and replacing. Hmmm.
I have the same NuHeat thermostats I believe but haven’t bothered with the API yet … and now I’ve freed-up a Sense monitor to dedicate to my one floorheat loop. Ground-truth on the ground! Waiting for Winter.
Meanwhile, I wonder if the non-ramped resistive element in the floor heat (= predictable plateaued load) isn’t somewhat “easy” to synthesize and cull … at least statistically. I suppose you’ve pondered that. Maybe I missed that in your prior work. You’re good at histograms!
If I had a tighter sampling interval than 15 min for energy, I probably could, but based on quick analysis the population of spikes goes up too quickly for continued use with my simple matching algorithm once I get past a delta threshold of 5kWh. I did a simple analysis here - number of candidate edges double with each 1kWh decrease in thresholds. There’s kind of a sweet spot about 5-6kWh that is pretty much occupied by just EVs.
I decided to take one more deep look at detailed correlation between my Ecobee runtimes and my Sense detection for my upstairs AC compressor (AC2) to see what things look like when Pearson correlation between the hourly usage is good (very close to 1.0) vs. bad (less than 0.7) on a weekly basis. Here’s the weekly correlation chart between hourly energy usage and Ecobee hourly runtime. The numbers on the bottom are the number of hours that the downstairs AC ran that week at about 2.15 kW / hour (3 ton unit). One of my motivations for a more detailed look is that I just replaced my evaporation coil upstairs, as well as the furnace, including the blower/air handler for my AC. I want to see if I see any differences in performance.
First, let’s look at good correlation for the week of 6/22. Here’s the week view on an aggregated hourly basis. Looks quite good except Sense seems to have missed an off-signature or 2 on June 26th.
Is there any way to get a better idea what happened there ? Maybe if we look at hourly Sense vs. Ecobee’s 5 minute runtime data below. Then again, maybe not. Given the number of actual on/off cycles, it’s hard to resolve the Sense hourly vs. the full set of transitions. Also makes me aware of how important it is to line up sampling intervals when doing visual comparisons. For the nerds out there, your simple AC system is run by pulse width modulation (PWM).
Now it’s time to look at far less good correlation the week of June 8. Very clear what the main mode of failure was here. Sense either detected an errant off-signature or decided to time out after what looks to be about 2 hours. June 10th and 11th were a fairly abnormal heat spell where the upstairs AC was pressed to run for 12 hours continuous.
Hour by hour:
Sense hourly vs. Ecobee 5 min.
A final look at one more errant week, May 25th. Looks like the same phenomenon - Sense seeing an off too early on days when the AC is running for uninterrupted for long periods, though Sense looks like it hung in there for much longer on May 25th. I’m wondering if Sense occasionally trips up on my upstairs off-signatures because it interprets an off-signature on my larger downstairs AC unit as its off ? Maybe work looking closely at one of two of these mistaken off events.
Hour by hour:
Sense hourly vs. Ecobee 5 min.
If I push down into 4 days where Sense seems to lose the AC2 (upstairs AC compressor) detection too early, when my Ecobee is trying to run the upstairs AC for many hours, I can add what my upstairs Ecobee is doing in black and my downstairs Ecobee in orange lines. What I see is that there is a lot of on and off activity on my downstairs AC when my upstairs Ecobee is calling for the the upstairs AC to remain on. Given the proximity between Sense losing the AC2 detection and my downstairs AC turning off, there is an extremely good chance that Sense misread one of the downstairs off-transitions as an upstairs off-signature and ended the detection.