Continuing Analysis of "Always On" Calculations

I’ve convinced myself there is no easy way to assess the precision (vs. accuracy) of AO anyway. Perceptions vary. How precise should AO be?

[Apologies if this has been hashed out elsewhere]

Case in point: a fridge (on a smart plug so I trust the watts accuracy):

  • In my mind (and perhaps technically) a fridge is Always On but the standby use on mine is ~1w; the average use is 42w (as listed) ==> 376kWh/yr although the actual use (as listed) will be around 300kWh because “it switches on and off periodically”

  • Personally, I want the AO listing for my fridge (not all devices in the same way) to show average usage based on actual annual usage of 300kWh / (365 x 24) = 34w (or thereabouts).

  • From an actionable standpoint I can see why Sense goes with the AO being what it is (I think) … to motivate consumption reduction … you can’t unplug your fridge/freezer.

  • From my reaction to watching AO I can tell that by only accounting for my fridge’s standby as the AO that I am getting a false sense of what really is my Always On total.

  • The Wemo that the fridge is plugged in to shows “Average when on” of 33w and I can set the On/Standby Threshold wattage (currently 2w … the minimum) … which makes me wonder if a User-settable (and/or Auto Threshold) similar to Wemo wouldn’t work for a more meaningful (to the individual) AO?

All the while with AO I feel like I’m missing something. Ironic.

2 Likes

I did a long thought experiment on how to automatically classify smartplug device power usage in this history of postings.

I think I would rather have Sense do some kind of statistical clustering and allow us to assign names/states to the various clusters, where one user-assignable state would be Always On. The whole thresholding thing presumes just one breakpoint between two states. I would rather have Sense look for the real states.

Would appreciate your thoughts…
Plus it’s time to do another analysis no that I 30 or so devices on smartplugs in my house.

1 Like

@ixu,

My most recent profiles on smart plug devices since April - I wish I could export data with 1 minute resolution. I think the clusters of power usage would be better defined. The real question is what states are associated with which cluster, plus which cluster really corresponds to Always On.

Green triangles highlight the centers of clusters, red triangles indicate the valleys between clusters. I only marked the biggest two clusters with the triangles, but the same technique could be used to find more.

AV Receiver - standby vs showing a movie

Washing machine - standby vs washing

Video server - Mac Mini - Who knows ??

Toto Washlet - standby vs heated seat vs active

Tivo Mini - hardly used - mostly standby

Time Capsule - access point and back-up running

Apple Airport Extreme Router (main house router) - Who knows ?

Surfboard Modem - maybe it’s in standby every once in a while ??

Recirc Pump on timer - off, on for part of an hour, on for full hours (shows resolution issue)

LaserPrinter - standby and printing (for only a small portion of an hour)

Upstairs Furnace - standby and in operation

Downstairs Furnace - standby and in operation

Plug-in hybrid charging - My son isn’t so great about plugging in but he pays for the gas.

Main switch for the house - maybe it get’s to sleep a little every once in a while

AppleTV - Very power efficient when in standby or even when streaming

Cable TV amplifiers

Office - mainly laptop and monitor - 3 modes, traveling (laptop not using anything), standby-ish, and heavy computing.

Playroom - 2 gaming PCs - 3 modes - standby, daughter gaming and both kids gaming…

2 Likes

I do ha e a problem with using smart power strips (HS300) and Always On. You would think that the use of the plugs would have the AO devices plugged into it properly placed in the AO category, mine doesn’t. One of my routers, CCTV DVR and CCTV camera are plugged in but don’t show in AO, they have their own lines on the device page. These fluctuate 1 watt and are otherwise constant.
So I sure won’t be out buying more for AO Purposes until they get this working like it should be.

Thanks @kevin1 for these updates and apologies for not calling out your extensive (and very satisfying!) prior analysis in my post but I had actually gone through it and as you can detect I was hinting at a slightly different aspect of AO. Perhaps more of a meta-analysis. I’ll explain [and apologies for the rehashing]:

SCALE!: Starting at the “bottom” it’s clear from attempting analysis that you can never get enough resolution. Well, maybe, but the more the better. As you pointed out

So that said, and holding to the view of a single device, what Sense users see (short of your analysis) is the Power Meter waveform combined with a knowledge (and awareness) of the function of the actual device from which the waveform is generated.

SCALE!: Starting near the “top”, and using again the Fridge as an example, some devices (fridge/freezer/UPS(?)) for some people have what I would call an encapsulated energy profile. Ice in a fridge/freezer and the thermal mass of the stuff therein is essentially an energy store (in reverse?) so I would “average out” the energy consumption to the greater timescale of the true On to the true Off (i.e. switching on/off manually). These are special cases but I would argue that if you ask a non-Sense user to name the device in their house that is truly “Always on” they will reply: “The fridge”.

A UPS or other high-capacitance device presents the same philosophical AO quandry: e.g. A high capacity battery charges in 15min but then could dissipate through usage over an AO-scale time period. How do you classify it? I would argue: It depends what is doing the dissipation.

And if you plug a fridge into a UPS? You get my point.

SCALE!: Starting near the “top” again, and using an OLED TV as an example (crazy variable waveform while On that reflects the content being watched), when I go into the menus of the TV I have the option of adjusting the settings for “Quick Startup” [Note to watt-watchers, here is a place to save a few]. If I switch “Quick Startup” on I’m guessing the Standby power will increase and my TV tells me as much. Here, I am deliberately choosing to increase or decrease the AO power by switching the Quick Startup on/off. On another level I can also switch the TV on/off manually or with a Smart Plug. The point here is that the TV has no real encapsulated energy … or does it? If I deliberately set the TV for Quick Startup vs regular Standby Mode vs switch it Off-Off am I not also modifying the AO expectation? This would seem to be self-evident in the waveform, ready for AO analysis, but without Power Mode awareness the interpretation of different Standby currents would be the same. The real point here is that through analysis (and reference to some real Device specifications knowledge) Sense can potentially detect regular Standby vs Quick Start and Sense can alert me and say “It looks like your TV is set to Quick Start mode. You can save some energy by switching off that mode”

SCALE!: Looking near the “bottom” somewhere, low power electronics like modems/routers are in theory AO but in practice and as time goes by I wouldn’t be surprised if more aggressive Power Saving modes are invoked in those as well … Nobody at home = auto shutdown. This will complicate the AO equation. My argument would be that Sense’s device awareness would treat this kind of device’s AO like a fridge regardless of its auto power cycling. User: “Yeah, my modem and router are always on of course”.

As an amusing aside, I suppose you could point to a motion-sensing light: “Oh yeah, that light is always on”.

I’m drilling a little bit deep I know but I wanted to calibrate the analysis, at least in my own mind.

1 Like

You make a great argument - the timescale / interval we’re using to look at power usage really affects how we perceive it. And depending on what trends we’re looking for, the on-signature of a motor, vs the on-signature of an EV, vs the typical daily use of AC, vs. the typical seasonal use of AC, vs degradation of solar production from PV panels, all require different analysis timescales.

On the flip side, I see Sense’s definition of Always On to be somewhat of an absolute measurement, rather than a type of follow-on analysis. They attempt to quantify and categorize a subset of usage that will never be detectable using the Sense ML techniques. The thing that’s a bit confusing to users and makes house-level Always On a bit incongruous, is that it needs to be “measured” using different techniques than all the other real devices, and measured on a different time scale in order to filter out all the daily cyclic activity. House-level Always On is also a little discordant in that it represents components from many different devices, although you can also get Always On from a single device using a smartplug.

Really, Always On is an orthogonal measurement to Sense ML device detection, meant to find real-time detectable power/energy usage that the Sense ML techniques never could. And in fact, it finds opponents of your refrigerator and furnace usage, that’s never identified, even if Sense picks up the regular cycles of your fridge or HVAC.

So again, the current Always On represents a real fundamental but orthogonal measurement, not an scale-based analysis as you describe above. The 24-48 hour measurement window Always On uses, may make it seem like an analysis rather than a measurement, but that filter was carefully picked to remove daily cycles of things that Sense should pick up (recirc pumps, lights, etc. on timers) on the low end and anomalous events on the longer end (you don’t want to use 1 month because that could let a network outage dominate your Always On for a month).

But I’ll admit there is a gray area in the Always On measurement, for transitions at low end of the power scale for specific devices on smartplugs (devices that Sense is either incompletely or never detecting using ML). Consider the profiles of measured devices like my modem, Time Machine, or AppleTV. All of them have a relatively low power operating mode, but look like they have a still lower standby mode as well, that would likely be more clearly defined with a tighter data sample window. The current Sense Always On algorithm for that device is going to home in on the lower end of that distribution (1% bin), but given Sense wouldn’t likely detect the changes, the whole of the device usage really fits into the house level definition of Always On. The Always On for the house and the Always On for a device are calculated similarly, but have somewhat different meaning because of the scale you highlighted. That’s where your notion of a threshold makes sense. But I would rather that Sense do some mode analysis for us, to help us make that decision.

2 Likes

Going to digest this for a little and look at your charts.

On the threshold: I completely agree that a single threshold is less meaningful than letting Sense dynamically determine things but where I would say a user-input “threshold” applies is actually more related, at the highest level and beyond any absolute measurements, to whether the user (or Sense-wide auto-magic for that matter) determines that a device is AO by definition. At the next level of devices that aren’t clearly AO, you drill down (smaller scale) and do analysis on shorter use-independent time periods and balance that with the longer use-related scale.
User: “I switch my fridge off when I go on a sabbatical”
Sense: “I see you switched your fridge off, are you sure you don’t want to leave your hot water tank in vacation mode?”

The feedback from users seems to indicate ongoing confusion about how AO is determined. I can understand the engineering factors but they seem to bifurcate the explanation of what exactly AO is. Perhaps its just semantics (and I do start to feel like I’m forcing the issue) … I guess @RyanAtSense nailed it when he expressed concern about splitting the thread!

On to digestion.

The Sense definition of Always On is very precise for both house-level and smart plug measurements, though it has changed a bit over time. Sense has codified and documented.

But the explanation of what that measurement means to the the user bifurcates between house-level and smart-plug level.

I also think users have trouble with all-house Always On for four other reasons:

  • Orthogonal - It is an orthogonal measurement, both in terms of measurement method (48 hour vs. immediate based on ML) and in terms of what category represents, as a summation for all the devices in your household vs. a number for a specific device. So in the bubble diagram, it’s always the bubble where “one of these things is not like the others”. Plus since it is used as part of the Other calculation, it helps make Other less predictable.
  • Unseeable - If I read the current definition correctly, it’s the 1% bin for each of the two mains added together. That makes the raw data going into Always On invisible to the user for two reasons: 1) the current Power Meter UI doesn’t let us see the power on each main independently, just the sum, and 2) the sum of the 1% bin is not the same as the 1% bin of the sum. So we’re never really seeing exactly what the Always On calculation is seeing.
  • Statistical - Sense’s low water mark for each main is the 1% bin. That’s a statistical concept that is likely to be unfamiliar to many users, especially since Sense doesn’t present a statistical (histogram) view of power usage for each main. If Sense offered a histogram view of each main, I think people might grasp Always On better.
  • Anomolous - Since it is an extreme (1%) statistic calculation based on a 24-48 hour cycle, Always On is prone to weird results when the Sense power meter encounters anomalous data on the low-end (data dropouts and even negative going data due to dropouts). 30 min of, or even less, of zero or negative data can result in whacked out Always On numbers (I had several days of negative Always Ons - I can explain more if your are interested)

If Sense improved on those last two, I think Always On would be even more relevant and understood by users.

I’ve seen one weird thing w/r/t Always On and smart plugs.

I had a whole series of devices that were listed “under” Always On, showing their contributions. But then, suddenly, today, they all “split out” again, and none have any contributions listed.

I didn’t change anything, they just suddenly de-classified themselves, all at once.

Is this something that just…happens?

Weird,
My Smartplug Always Ons are all still arrayed under my Always On in devices.

image

Yeah, that’s why I thought it was weird that they just suddenly all ‘reset’. We’ll see what happens after a few more weeks…maybe they’ll go back again.

I reset a couple weeks ago. Nothing changed in the home and have many smart plugs that some were in always on. Not anymore, always on contains nothing under it.

Mine has ‘fixed itself’ and ‘broken’ a few times since I posted that. It was busted again on Friday, but this morning it’s back to what I’d expect, for the most part.

I honestly don’t know what’s causing it to break, nor what causes it to ‘fix itself’…