[BUG] - All five kp115 smart plugs went offline according to Sense but are still really on

Great to have a pro on the “job”. There’s a fair amount of info on protocol with a limited amount of Wiresharking here:

That appears to be for Wemo and it was reported to be bad power levels being sent by Wemo and Sense trusting the device. I don’t think there’s much to glean from that.

Welcome to the club. :slight_smile:

I appreciate any insight you could give. It’s become a bit of an obsession getting it solved.

FYI, my leases were and still are 1440 min. When I set the Sense and the KP115’s to static IP’s the other day, I did so as a reservation on the FIOS DHCP server. The router forces you to assign a unique hostname at that point. They all previously were “KP115” (KP115.fios-router.home). They now have unique names.

The frequency of the past five disconnects doesn’t seem to correlate directly to lease time. They were 5 days, 5 days, 5days, 6 days and 5 days give or take 1-2 hours. The last “quasi-disconnect” after setting static IP and unique hostnames was 725 minutes, off for 5 min and then back on with no intervention.

I use 192.168.1.x addressing.

2 Likes

That’s crazy how they are all basically the same which would amount to 5 renewals. I’m leaning towards an issue with firmware on the plug that’s bugging out. It could be lease related or something else but if it’s right at the day mark and your lease is 1 day that’s what I would focus on. I think the time being consistent like that is very helpful. I hate to speculate and not being able to get hands on drives me up the wall! I ordered 3 different Kasa plugs from Amazon when I saw this thread so tomorrow I’ll take a look and see what I can gather.

1 Like

Try reading the whole thread. Sorry I didn’t distill it into TP-Link only segments. Plenty on TP-Link throughout, including info directly from Sense / Jonah. Glad you are going to be experimenting, but some of the stuff in there should save you a little time.

1 Like

Kevin… You never told me to scroll up (feels dumb) :crazy_face:

This confirms all of my research so far. Broadcast UDP is being used which won’t be blocked by networks because that’s also what DHCP ARP etc use. That’s great and that clears up a bunch! Additionally, IP address doesn’t matter at all because it’s a shotgun approach. Unicast is used on the return so there’s the 1 to 1 relationship of data transfer and whichever IP the device has Sense doesn’t care.

So next steps are to figure out if the plug is getting the broadcast udp and if it is, why would the unicast fail. If it’s not then the plug is at fault.

1 Like

Remember that ALL KP115’s on the network fail at the exact same time, every time. By my uninformed logic, that seems unlikely to be a plug issue unless some network message is causing all plugs to crash?

It could still be a network"ing" issue and related to the lease. As I mentioned earlier I don’t think there’s a lot to look for here. Especially now knowing directly from Sense how they communicate with the plugs. If it was a network infrastructure issue then it’s unlikely you would have loss like you’re experiencing. I’m excited to get the plugs tomorrow I want in on this action!

Oh yeah and there’s a ton of HA users with the same issue regarding these plugs. Soooo, I don’t think Sense is at fault. I have found a few posts where people are reporting that band steering is causing the plugs to lock up. Kasa products keeps disconnecting from wifi - TP-Link Community

3 Likes

Great find, this is super helpful. I’m passing this along to our team.

It’s my hypothesis that the Sense monitor might not know what to do with unicast responses from 5 altered / reallocated IP addresses (in @Offthewall case). It suddenly sees no data coming from the IP addresses it knows about and a bunch of new data from “non existing” smartplugs. It could be that the Sense monitor doesn’t pay attention to the ARP traffic that would inform it of the reallocation or the smart plugs do something a little out of the ordinary that screws up the Sense monitor’s understanding of the reallocation.

I don’t think that the smart plug’s unicast response packet contains any other source-identifying information beyond the IP address in the header, but it’s been a while since I have looked.

Kevin,

It would be bad engineering to associate a network device by their IP so I don’t think Sense did that. Devices are identified by their MAC address and sometimes in rare cases (such as devices that often get changed out, or modern mobile devices with randomized MACs) by their hostname. The ARP is just a query as to what MAC belongs to the IP so the IP must be known first (or queried). For additional discovery there’s also RARP, BOOTP, multiple form of name resolution including broadcast and multicast.

When Sense sends a broadcast message it has to be seen by all the devices on the network. That’s just how it works. The unicast message is just the 1-1 exchange that follows. The plug initiates this response back to the Sense using the port and IP designated in the broadcast. Sense should know who it is based on the MAC of the device.

Now it could very well be that the plug doesn’t reply. That’s definitely plausible. Although broadcast messaged must be seen by all devices, it doesn’t require that all devices reply. For example, an ARP request for 10.0.0.4 will be disregarded by all devices who don’t have that IP and therefore only replied by the intended target.

I hate posting theories … I just want my plugs already so I can look into the traffic :laughing:

Here’s something for anyone to digest should they be really bored:

There has to be like 1000 more examples. But these things just seem to be problematic.

1 Like

FWIW, most of these cannot apply here. No roaming, no fixed layer 2 isolation, no VLAN, no steering (the KP115 has no 5Ghz support), and 2.4Ghz support is obviously present. [No AP support for density bitrate is a new one for me, never encountered any issue with that.]

However it has become a common issue that various wifi devices attempt to intelligently interfere with broadcast. It’s fairly common to only allow broadcast for the first few minutes after connect, when it is still commonly used, and then cleverly shut it off if possible. At least one major vendor has taken the step of completely disabling broadcast by default.

Sorry, AP vendors being overly clever is one of my pet peeves… hard to get me to shut up about it sometimes. I’ll be quiet now.

Some of the settings in these devices can be confusing. For example, I think UniFi had an option to block LAN to WLAN Multicast and Broadcast. That is essentially the same as (part of) DMZ Isolation. It can be more common to block Multicast but Broadcast and Unicast cannot be blocked otherwise nothing will work unless you have local proxy arp or some vendor version thereof … we’re just going down rabbit holes now, but that’s essentially part of DMZ Isolation.

This is also considered part of “mesh” in many home user environments.

This would be on the AP Side. It’s common for APs to attempt to get 2.4 devices to move to 5.

Mostly on older IOT Devices. But even some new ones. I’ve seen issues with Vector WiFi Rat Traps where the density bitrate needs to be really low.

Don’t be quiet. We can all learn from each other and knowledge is endless. Most of my work is in security of network infrastructure and its architecture. I don’t spend my days troubleshooting IOT devices that wont talk to each other. Over the years I’ve run into a ton of weird and crazy things.

I’m definitely awaiting your smart plug delivery as well. But unless there’s something I’m missing something:

  • Each KP115 smart plug only issues a single response packet to the Sense monitor broadcast. There is no 1-1 exchange / session, only a single one-way response (more for the Kasa HS300)
  • The data field contained in that packet is softly encrypted JSON text that looks like this for each:

Broadcast from Sense Monitor

“emeter”:{“get_realtime”:{}}}

HS110s receiving that message will send a message back to the requesting IP address of the following form (with a little encryption in between)

{“emeter”:{“get_realtime”:{“current”:0.028915,“voltage”:120.798141, “power”:0.527563,“total”:18.045000,“err_code”:0}}}}

HS300s respond with the same data, with current and voltage reversed and in different units. There’s more to an HS300 response, because it has 6 outlets to report on.

{“emeter”:{“get_realtime”:{“voltage_mv”:121917,“current_ma”:177, “power_mw”:16194,“total_wh”:90051,“err_code”:0}}}}

And the KP115 response is in the same units as the HS300, but with current and voltage reversed. Note that the response below is for a situation where the KP115 is turned off via the Kasa/Sense On/Off switch. So there’s no specific data returned that tells that the smartplug is turned off.

{“emeter”:{“get_realtime”:{“current_ma”:0,“voltage_mv”:122525, “power_mw”:0,“total_wh”:107,“err_code”:0}}}

Kevin I agree with what you are saying … by 1 to 1 I’m referring to unicast, uni - single (or one). Meaning that the device is communicating directly with the Sense or one singular device on the LAN instead of broadcast which is to everything.

While what you are showing looks normal, we need to find the abnormal. So that’s what I’m hoping to find. Where does it break down :mag_right:

2 Likes

For the KP115s, sense sends a single UDP packet to 255.255.255.255:9999. Each plug responds ptp to senseip:9999.

For the HS300, Sense sends an individual ptp query for each socket.

1 Like

@Offthewall, curious about something. Do you have Network listening enabled or disabled in Sense?

That’s the way broadcast works. Completely normal. 255.255.255.255 is the reserved broadcast address and 9999 is the port that those devices use for communication.

Sounds correct. It should use a /0 /1 /2 etc for each socket

Network listening is Enabled.

1 Like