Sense Communication Crashing with Kasa Devices Connected

This thread is being created as an offshoot of another thread where we have been trying to determine issues relative to KP115’s disconnecting from Sense.

In an effort to help on that threat I purchased 5 x KP115s and 3 x HS300s. I have connected many of those devices to the network and Sense has picked them up. The issues I’ve experienced are the opposite, whereas the Sense Monitor drops connections to the Sense Servers and the Kasa devices remain active.

It should be noted that this never happened prior to connecting the Kasa devices.

Original Thread: [BUG] - All five kp115 smart plugs went offline according to Sense but are still really on
Tech Support Ticket: #246852
First Occurrence: 07/02/21 @ 3:50PM (PST)
Second Occurrence: 07/04/21 @ 4:42PM (PST)
PCAPS: Available in the tech ticket
Total Outlets Connected: 19 (3 x HS300 / 1 x KP115)
Noticed Commonalities:

  • Both times the Sense Power Meter reports that the data dropped out within 30 minutes prior to the crash, as if there was a “mini crash” that it recovered from (see images).
  • The Sense monitor remains connected to the AP/Network and responds to pings, sends ARP Requests, UDP Broadcasts and responds to unicast messages in return to the Broadcasts. No disconnect events are noted in the AP logs.
  • Pings appear to be elevated @ 100-300ms but loss rate remains 0
  • Power cycle on the Sense monitor restores normal communication
  • No power cycle performed on any other devices including the Kasa devices.

Conditions checked:

  • I tested channel changing on the Sense monitor to see if that had any ill affect on the monitor and while the monitor does go “offline” for a period of time, it recovers and fills in the gaps with what was locally stored. During this process there is a disassociation and re-association logged.
  • I checked the network logs for any changes during the time frames where the monitor went offline and nothing was noted. No IP changes, cellular failover events, channel changes, channel utilization issues or anything of significance.
  • There are no other devices experiencing difficulty at the same time.


2 Likes

Glad you have your Kasa devices for experimentation. Wanted to note that with those quantities, you are just above the quoted Sense max of 20 for monitored smart plug outlets.

Although I have 23 devices, only 19 are connected.

Additionally, per the Sense blog post “The number of smart plugs is limited primarily by Wi-Fi network bandwidth”.

I’m running enterprise grade networking equipment with a combined 3.9 Gbps aggregate frame rate, 1.2 Gbps of which is dedicated to the 2.4Ghz channel.

I would get alerted if I was exceeding bandwidth

2 Likes

FWIW, you might double check the mini crashes. I’ve seen that several times, and when I dramatically resized the scale of the display, or exited and re-entered the app, the gaps we no longer present. No wifi disconnect/reconnect/roam of the Sense device. I.E. the behavior I observed appeared to be with the back end data query rather than the Sense device itself.

If you continue seeing issues, I would suggest that it’s time to start generating pcap files for the Sense folk to examine. Not just comm with the plugs, but with the backend as well.

FWIW, I’m running 4 HS300s and 5 KP115s without issue. Though not all the plugs on the HS300 are being used, Sense polls them anyway. I really want a way to disable individual plugs in Sense.

@dennypage I’ve seen where the graph will glitch when zooming that’s not the case here.

I have generated pcaps as noted in the original post and attached them to the tech ticket

The pcaps show that Sense stops communication with the Sense servers. It doesn’t even try anymore.

2 Likes

I love ops guys. :slight_smile:

1 Like

That’s good to know. Do you have Hue devices as well? That’s kinda making me think it might not be Kasa or maybe it’s a combination of the two… Either way I’m hoping that the tech team can extract something from the logs on the monitor.

This is more a limitation on the Kasa side. When sense asks for the emeter data the Kasa device responds with everything. Even if you could disable the plugs in Kasa it would still likely respond showing the deviceid but it would probably have a key value of disabled. So the traffic would be negligibly different. There is an option to hide the plugs from the timeline which I’ve done for most of mine because personally I don’t really like the bubbles (that’s a thread on its own).

1 Like

For the HS300, Sense gets a list of plugs from the device, and then polls each plug separately ptp.

Edit: No hue devices.

Yes you are correct that sense communicates with the HS300 asking for each plugs emeter data which is represented by either 01/02/03/04/05/06 appended to the child_id. But when Sense sends a broadcast the HS300 responds as part of the get_sysinfo the number of active plugs represented as 6 under the child_num key value. That value should never change. The state of the device (0 or 1 under the object state value) would change if it’s on or off and in order for the Sense to detect if the state changes from on to off. This is why I mention that it will reply with the data regardless.

So … rethinking I guess you could tell Sense to not ask for that data for the child_id. It would just have to be an if locally disabled then don’t query. I could see pros and cons to this. From a power user prospective it would probably be fine. But I could also see a bunch of tech tickets when someone plugs something in and doesn’t know how to re-enable the device. You would have to make a “show disabled” option in the Sense app in order to revive a disabled device.

This is the most thorough bug report format I’ve seen so far @DevOpsTodd :slight_smile:
There are a few issues I’m looking into here, but I’d like to get someone’s eyes on this from our team to provide a more technical response on what’s happening on the Sense side of things.

2 Likes

Ok, so here’s the follow-up from tech support in a nut shell:

My device is seeing increased CPU usage because of all the plugs I’m adding, Hue devices etc. They have tweaked my monitor to poll less frequently which should help bring the CPU numbers down and keep things running smooth.

I will watch the monitor with the changes applied by support and report back if it doesn’t improve.

2 Likes

I just installed 2 KP125s last night and my Sense is constantly crashing. Added 4 more for a total of 6, and still the same issue.

Two of the units work in Sense, but the rest show n/a most times (I saw a few of them work for a brief moment). I’ve updated firmware, they work in Kasa and HomeKit… just not Sense.

The lockups are bad though, has done it three times since last night requiring a breaker flip each time. Ugh!

@wojo , a couple questions:

  • How long had your Sense been working without crashes / issues prior to the addition of the KP125s ?
  • What kind of networking setup do you have ?

I don’t have any KP125s right now, but I know that my single KP115 is responsible for more syslog messages with my Ubiquiti network, than any other client device by a factor of 2x or so. Syslog messages don’t imply errors, but do give some indication that the KP115 creates some kind of stress on the network.

About one week, a new Sense user but never have had it crash – only a few gaps here and there in data.

I’m running an enterprise network using HPE Aruba APs all hardwired, very reliable and excellent compatibility with my devices (old IoT to the latest WiFi 6 gear). I’ll check logs for anything interesting.

During this entire time however the HoneKit and Kasa apps are performing perfectly with the 6 outlets.

The Sense unit crashed yet again, just about a hour since the last power cycle.

As you have seen, there are a bunch of us that seem to have issues with TP-Link smartplugs dropping out (NAs). Haven’t seen a case where the Sense monitor locked up, except in the situation that @DevOpsTodd commented on, where the number of smart-plugs exceeds the Sense suggested limit (20 outlets), and the communication with the plugs chews up too much Sense monitor CPU.

It also seems that the the Sense / Kasa integration, and Kasa smart plugs in general, are more sensitive to more sophisticated switching / routing infrastructures. I seldom had problems with my prior fairly simple wired Apple Basestation based network. More issues with Ubiquiti.

There are only two flavors of messages kicking out of my UDM relating to the KP115 - Lots of these where the “satisfaction_now” varies between 71 and 79.

daemon.info mcad: mcad[2830]: wireless_agg_stats.log_sta_anomalies(): bssid=UniFi AP MAC radio=ra0 vap=ra1 sta=KP115 MAC satisfaction_now=71 anomalies=tcp_latency

and a smaller number of these. Not sure what a “fixup” event is in UniFi land.

daemon.info stahtd: stahtd[2845]: [STA-TRACKER].stahtd_dump_event(): {"message_type":"STA_ASSOC_TRACKER","mac":"KP115 MAC","vap":"ra1","event_type":"fixup","assoc_status":"0","event_id":"2","auth_ts":"0.0","dns_resp_seen":"yes"}

This topic was automatically closed 365 days after the last reply. New replies are no longer allowed.