Ground truth in the UI

Science!

Thanks @RyanAtSense for cross-posting that blog entry … I missed that and it is very informative.

So, speaking of ground truth data:

  • Should the UI perhaps display waveforms for Sense-native-detection vs Ground Truth (smartplug & Mains/Solar PM) differently?

  • Posting screenshots to the Community sometimes reveals the ambiguity: “Is that the device PM or the Mains PM?”

  • A different color or label in the PM would be a couple of ways of doing it. e.g. Fridge via smartplug; Fridge detected by Sense.

  • In general, if a user sees a difference in the waveform rendering (color and/or a label) it will go some way toward explaining (without too much explaining and UI clutter) the distinct difference between ground truth data and Sense-algorithm-based-detection.

I don’t see that as being a downer like “Oh, great, a gray waveform” so much as a UI move toward reflecting the reality of the detection optimism.

Taking the waveform look another step into the future one could imagine that there might be a visual confidence indicator. “It’s gray, so wait”

Can I ask a foundational question here: What utility would that have? A double tap will bring you directly to the device so you can see if it’s a smart plug device or not. How would it being obvious in the bubble UI be helpful?

I wasn’t even thinking of “Now” … but breaking it down further:

Lets say Sense-native stuff is GRAY (philosophically at least) and smartplugs are ORANGE.

  • In Now>Bubbles you would see ORANGE and GRAY and immediately see which ones are ground truth AND that are potentially switchable because they are ORANGE.

  • In a Sense-detected GRAY PM utility comes in the form of

    1. User thinks “Oh, it’s gray so it’s not precise (yet)”

    2. If user posts waveform or Support refers to the interface there is a color confirmation.

  • In a smartplug ORANGE PM you can likewise see and confirm that the device data is ground truth.

  • The default ORANGE Mains PM and Solar colors would also, by association, read as “ground truth”.

  • Let’s imagine also that at “95% tracking confidence” the once-GRAY device waveforms in the PM turn light-ORANGE or somesuch.

2 Likes

Interesting. I do like the idea, but I know that confidence can be quite tricky and might be taken as gospel by the average user. In reality, even with a 95% confidence as is, if someone suddenly starts using a different mode on that device or something else in the home interferes with it, that confidence can be diminished.

I’m curious what others think.

Confidence reduction = color trends GRAY.

That’s a more aspirational aspect of what I was thinking and GRAY is more symbolic than what an actual UI color could be. The confidence aspect is secondary (future) to the primary idea of clarifying known-to-be-ground-truth-data vs guessed-data.

I am confused by the concept of Ground Truth. If I have my Plasma plugged in thru the Kasa and I watch the waveform on the Sense UI, the Kasa is WAY behind on updating the Sense as to how much power is being used for the Plasma (which is ALWAYS changing). Doesn’t seem like it can display what is actually happening. Perhaps it can at least display a total that is accurate.

If we are looking for Ground Truth shouldn’t we also connect to the Smart Meter in the house or the Solar Controller via it’s API? Of course that might open a can of worms, my Enphase system seems to have a much greater delay updating their UI about how much power has gone out than the KASA does. I have been using the Sense as a ground truth for the Enphase system.

And as long as we are talking truth, does the Sense handle Kilowatts correctly? Or does it just calculate Volt Amps and displays them as Watts? Not that I have a ton of Inductive or Capacitive loads, just curious how it would handle being set up in an environment where they did.

1 Like

4 thoughts here.

  1. I’m not sure how your are watching the Kasa data update for your plasma, but my take is that the Kasa app updates are infrequent, compared to the HS110/HS300 updates going to the Sense monitor (I think that is once every two seconds). The smartplugs don’t update to Sense in realtime, hence we don’t see moving waveforms in the device Power Meters, but the accuracy should be pretty good as long as your network can handle the traffic and the Sense monitor doesn’t get get too loaded down (more than 20 or so smart plugs). Any even 2 second sample of usage isn’t perfect ground truth for machine learning, but is hugely better than trying to use human input about what’s going on WRT activity on each monitored device.

  2. It wouldn’t be a bad idea for Sense to have integration with Smart Meters or the solar API, but that ground truth isn’t really needed for machine learning, but just for basic calibration and data integrity checking. My measurements show is that Sense is highly accurate for both my solar data and total usage data for my house. The biggest discrepancies come when either my network/internet connection or Sense monitor get hosed up and Sense stops reporting for a bit (dropouts)
    https://community.sense.com/t/how-accurate-is-sense-vs-utility-metering/6451/13

  3. I think your are hitting the crux of the ground truth problem when you mention delay or latency in measurements. Fortunately, most APIs timestamp poll power usage or production time interval data so it’s easy to do direct comparison. That’s the case for my solar and utility providers - makes it easy to compare data directly.

  4. AFAIK the Sense numbers your see are real power, not apparent power. It kind of shows when all measurements are in Watts rather than Volt-Amperes. But Sense internally does measure the phase angle on each of the circuit legs in your house and uses changes in the phase, or phases on both legs in the middle of a transition as features input into machine learning.

1 Like

I see ground truth, as with many problems, as a matter of scale. The rough (OLED TV!) and the smooth (a single LED bulb?).

Mandelbrot (and Slartibartfast on some level) cited the impossibility of measuring the “ground truth” coastline of Norway. Beyond the method, it’s a matter of the measurement scale. Beyond the scale it’s a matter of the measurement method.

Having just read Thermodynamic Weirdness, I also think of Rudolf Clausius’s seminal papers on thermodynamics and the intertwined definitions of system, state, and boundary. From a ground-truth energy perspective, boundary quickly devolves into Heisenberg territory: uncertainty.

That said, Sense device detection (vs. absolute energy integration) ground truth is also a matter of scale:

  1. At the highest level = “is it on/off?”

  2. Mid-level = “roughly” how much energy is it using during a certain period T, where T is as small as possible.

  3. Low-level = looking at successive T periods, is there sufficient information to disaggregate devices? This feeds back to #2 and leads to the refinement of T. Zoom in/zoom out. Of course the instantaneous state (V/A/phase/frequency) matters a lot for complex real-world scenarios but theoretically, with sufficient resolution, you can always disaggregate in some cases (many) and never (perhaps) disaggregate in others … this feeds back to #1.

For #1, you don’t need to care about actually power, you are just concerned with “is it on or off”. There is also the possibility of ground truth by a process of elimination:

This is certainly not the best illustration, but as I posted elsewhere I think smartplugs are tracking usage well enough and closely enough to Sense’s ground truth input (the CTs + wires) that the combined systems lead to more information. An interesting feature of these waveforms is that the mis-calibration (wrong absolute power) doesn’t necessarily matter for disaggregation … in fact you can see that the timing is very precise and this speaks to @kevin1’s #3.