Solar Production Comparison

Nice… It would be interesting to see your X-Y chart of this data to see if the main error term appears random or systematic.

Where are you getting your production numbers in Enphase? There are two sets, at the Inverters and at the Power Meter. The Power Meter is very close to Sense on my system. Using inverter numbers is always higher.

Yesterday: At the Inverters was 7.14 kWh, at the Power Meter (in the Combiner Box) was 6.58 and Sense was 6.55.

1 Like

@MikeekiM,
I couldn’t help but do the graphs…

Looks really good except for 12/21. You should go back and look at your Sense Power Meter for 12/21. I wouldn’t be surprised if you saw some kind of data dropping out on that day…

1 Like

@Ray13, I OCRed your data and plotted on scatter (X-Y) plot. Very linear, but off for two systematic reasons.

  1. You have a y-intercept of about -450W.
  2. You have a slope of about 0.9758

That means that your SenseSolar = ( 0.9758 * EnPhase ) - 450Wh, very consistently.
Adjusted R-squared: 0.9998

That -450Wh fixed difference (y intercept) really messes up your % calculated error. You’ll find your error to be about 2.42% + 450Wh/Enphase Wh

1 Like

Thanks @kevin1

Forgive me if it’s obvious, but what exactly did you plot on x and y axis on those charts?

I see the linear relationship with the outlier data point…but I am not 100% sure I know how you charted this… Sorry if I am being dense here! :slight_smile:

Thanks again!

No prob… There first plot has your SolarEdge energy output reading on the x-axis and your corresponding (same day) Sense output reading on the y-axis. Similarly, the second plot is your Utility measurement for each day on the x-axis, with the corresponding Sense reading on the y-axis.

With this kind of plot, you should see points close to the “unity line”, where x = y (slope = 1, starting at 0,0). I tend to put the “golden measurement” on the x-axis so one can calculate the Sense output as a function of the golden measurement (utility or inverter, though the inverter is usually not really a golden measurement - there are some losses before it reaches your main breaker box)

FYI - for your solar, Sense = (0.97 * SolarEdge) + 60Wh.
Adjusted R-squared: 0.9999
So Sense is typically 3% below your SolarEdge measure, very similar to my SolarEdge.

for your net usage, Sense = (1.005 * Utility) + 295Wh
Adjusted R-squared: 0.9997
Your Sense is almost dead-on with your Utility except for that 295Wh offset

To get that type of chart in Excel, just select the two columns for comparison and Insert > Chart > X Y (Scatter)

1 Like

Thanks so much for doing this @kevin1! So what does that -450W Y intercept mean? Could it be from a day where the data read was interrupted on my Sense device?

The beauty of doing the plot and the linear regression is that it lets you look past bad data points from things like interrupted reads, Sense data dropouts, etc.

  • Bad data due to random measurement errors shows up as points well off of the best fit (regression) line. In your case, there are two points (points/rows 2 and 29) in your spreadsheet that are well off the fit line, in the red circle, that you should probably investigate further for missing reads or data dropouts.

  • The 0.9758 slope and the -450Wh are systematic error sources - the error mechanism affects every measurement in the spreadsheet. One possible interpretation could be that your inverter measures total energy flowing from the panels, but then the inverter uses 450W plus 2.4% of the energy delivered to your panel, just to do the conversion from DC to AC. There might be other reasons for these systematic errors/differences, but neither of those values are due to just one point - they built into due to all the data points. ps: The -450Wh is the spot where the fit line crosses the y-axis, in magenta. It indicates what one would expect Sense to read when the inverter is on, but producing 0 power.

1 Like

Thanks for all of your input on this @kevin1! It’s been very helpful and I’m really learning a lot!

So with everyone’s help so far, I have been able to plot my data Enphase vs. Sense in an Excel XY Plotter Chart and I figured out how to show the Y and R2 of my data automatically!

Based on my chart it is showing Y=0.9676x - 0.3573 and R2 = 0.9993.

As you pointed out @kevin1 , the first data points are off the slope. I went back and looked and I did reset my Sense device on that day so there was a brief period where it did not collect any data.

With that being said, how did you come up with the -450W value?

Thanks,
Ray

Following this further, I went and created 3 XY plot graphs comparing the following:

  • Sense/Enphase (solar monitoring)
  • Enphase/CMP (utility)
  • Sense/CMP

As you can see the slope is fairly consistent comparing Sense to Enphase.

However, when comparing either Enphase or Sense to my utility, the results are horrible! What is this telling me?



Nice regression work. I have a slightly different fit line because I removed the two outlier points (2,29) before doing the fitting / linear regression. If didn’t remove those two, my result comes in much closer than that, Y=0.968181x - 0.364204, with Adjusted R-squared (R2) = 0.9993. Still a little different, but small enough that it can be attributed to slightly different regression algorithms between Excel and R.

R gives me some tools for analyzing the residual for each point (error/difference from the fit line). You can see why points 2 and 29 can be easily discarded, or at least investigated as errant data.

I think the problem is that CMP can’t really tell you solar production - a net meter will only give you “to grid” data and “from grid” data. Your utility “to grid” measurement is distinctly different from Solar Production - To get technical, “to grid” is really the integral of Solar Production - Total Usage over time, but only for the periods where that value is positive. “from grid” is the same type of integral, but for the period where that difference is negative.

You can compare the net usage between all three measurement sources because they represent the same thing.

2 Likes

Good point @kevin1!

So I went back and plotted all of the NET values.

  • CMP (utility)/Enphase NET
  • CMP/Sense NET
  • Enphase/Sense NET

What do you make of this? Do you think Enphase and Sense are closer to each other than either one of them being more accurate with CMP?

Nice work.

My 2c - Sense and Enphase are very linear with a tight distribution around the fit line. That tells me that they are measuring the same thing in a very consistent way, plus only 2% differential between them. But both comparisons against CMP have broader distributions around the fit line. That kind of suggests that there are some bad points in your CMP data. If I had the data, I would zoom in on a couple of the biggest CMP outliers and see if there were any hourly data discrepancies (missed reads or extra reads) in the CMP data. Usually I view the utility data as the gold measure, but my experience with ‘actual’ reads and ‘estimated’ reads has taught me that the utility data is only golden for an ‘actual’ read.

Here’s a detailed view on the size of the residuals, distance from the fit line for each point, corresponding to each of your 3 graphs. You can see that the largest residuals for both charts including CMP is 4x the largest residuals in the Sense/Enphase comparison. This Residual plot also shows the 3 points with the most significant residuals. You can see that the residuals for points/rows 10 and 19 are common to the Sense/CMP and Sense/Enphase plots. That tells me those are the two days that are the furthest off for CMP (with points 33 and 28 next). BTW - point / row numbers refer to count from the top/first data entry row, not the spreadsheet row #.

1 Like

@kevin1 here is my utility data from that timeframe.

CMP Usage Data 2020_12_05 to 2021_01_06.csv (51.6 KB)

@Ray13, thanks for the hourly data. Quick question - how did you get 86.805 for your first CMP daily value on 12/5 ? I either get 89.714, or 90.005 if I offset by 1 hour backwards to compensate for the way CMP reports.

1 Like

Good catch @kevin1! It should be 90.005. I went back and applied it to the two charts.

Not sure if that even helped much?

You might need to re-regress the fit lines. But with that change, the fit is much better as told by the residuals. In my case, point 33 and point 1 look like the worst. Point 33 is missing 1 hour, because I had to do the offset and only have the CMP data through 1/6/21 23:00. Picture the residuals as an exaggerated view of the points on either side of the fit line.

If I pull point 33 (1/6/21), both graphs look pretty good.

One thing that is highly noticeable, due to the way the automated labeling worked on these graphs, is the negative y-intercept between your Enphase and CMP net usage, similar to what we saw between Enphase and Sense on the pure solar side (Enphase was always seeing 450W more solar production than Sense, plus a 2.4% differential). If I look at look at the linear models for each pair, I see:

Enphase = (0.974 * CMP) - 0.741 Wh
Adjusted R-squared: 0.9999

Sense = (0.995 * CMP) + 195 Wh
Adjusted R-squared: 0.9997

Sense net usage looks to match your Utility net usage a little better than Enphase, but all three seem close. And I’m guessing from the solar analysis earlier, that Enphase always sees itself as producing a fixed amount more solar energy than is actually seen at the meter.

I think you have done a a good job showing that you can trust your utility meter net result. Now you are ready to get fancy and chart a few more months, all on one graph using CMP as the gold measurement.

The big gap between Sense and Enphase mostly stems from Enphase thinking it is producing more solar than Sense and CMP are able to see.

1 Like

Awesome! Thanks so much @kevin1!

I don’t know if I would go as far as saying I trust my utility meter. That’s the whole point of this exercise is that I don’t trust them, which is why I am using Enphase and Sense as a form of checks and balances just to keep them honest.

The other thing I’m working on is using an RTL-SDR device to try and connect to my meter directly and read the data being sent out. That’s proving to be very challenging!

1 Like

Keep on charting… With three measurement sources and a distinct set of patterns between them, you’ll be able to spot and isolate anomalies in any one them !

I used to have a Zigbee device (a Rainforest Eagle) connected to my meter but Sense kind of outclassed that device because it offered a much better UI, overlaid solar and total usage, plus had a much higher sample and display rate.