Still sorting out ways to linearize my “wing” more using the SolaR package. I’m now trying to convert from radiation striking the top of the atmosphere to radiation hitting the earth’s surface, then hitting my angled solar panels with about a 20 degree angle west of south orientation. Those factors might just be the reason for the loop pattern.
I have seen a similar kind of loop before when comparing my Sense solar generation vs. my SolarCity generation, where Sense was showing more power being produced in the morning hours vs. Solar city, and the opposite in the afternoon. I eventually tracked down to interval accounting counting being off between the two of them. But the “wings” aren’t as symmetrical around the unity line, so there’s more at play here…
I did do a quick shot at automatically locating the bad points using a simple linear model. First, I removed the 605 missing points (that number dropped from over 1100, because I’m now only looking at daytime data) that were manufactured by me to fill in the gaps. Those were simple because they had their other entries marked as NA. Then I used linear modeling to identify the 400 points with the largest positive residuals. That did a good job finding most of the “shadow wing”. I’m guessing I could now cut the power production for those points in half and stuff that data into the nearest NA or smallest neighbor. All this is under the assumption that SolarCity and the inverter are set up to work around data transmission failures.
I also decided to remove those 400 + 600 points and remodel using a new linear equation. Then I picked off the next 400 points with the highest absolute residual (plus or minus). The next bunch would come from the the bottom.
I also tried one more interesting R package called “robustbase” which offers more robust fitting algorithms that are supposed to deal with bad date. In the case of a linear model, lmrob() iteratively assigns weights to points during regression, giving smaller weights to points with the highest residuals. After a number of remodeling interactions with the weights used in regression, the weights stabilize, and you have your result. I have rounded and multiplied the weights by 10 (usually 0.0-1.0) for annotating the “wing” below. As you can see, this also helps pick off the shadow wing above the wing, as well as low lying power production during high B0 periods (everything 7 and below). But I’m sure that some of the “shadow wing” is still obscured by the good data. Plus weight 7 and 6 data below the wing is probably legit.
