Saw this the other day in a Computex keynote and it occurred to me that it might have some relevance to Sense. DjayPro uses AI to separate out vocal, instrument and drum tracks in real-time on Windows using a next generation NPU (neural processing unit). That seems somewhat akin to detecting and separating out different mixed current / power streams in a home, especially from devices that don’t have clear, distinct on / offs. But to put in context, we’re talking about a hell of a lot of processing power dedicated to separating just 3 streams that are already have fairly distinct features. Yet, it appears that another variation of the separating mixed streams is beginning to be solved.
The first link is to a 20-second promotional video for the software mentioned above. The platform here is iPad rather than Windows, since Apple products have built-in neural hardware.
Next is a 6-minute demo of that iPad app. All the neural calculations happen on-device in real time. Cloud AI to separate music into three tracks was already available, but takes longer to access.
1 Like
That is neat. I enjoyed reading all of this. Since Endlesss ceased operations, maybe there is some new music software I should play with (if I want to pay another $5 a month).
Not exactly a 1:1 comparison, but Sense has a pretty old video that talks about some of this stuff, separating music (signals) so it knows what it is is and does, as a comparison for what is done in the home https://www.youtube.com/watch?v=P7Bf_Ynnsj0
I’m aware I’m digging it up from the archives, so forgive me if you’ve seen it already 
@jefflayman , thanks for the pointer to the iPad product. My overall view is that we’re finally getting to the place in terms of hardware, and next generation AI model sophistication where we can dissect / disaggregate complex streams of data that are mixed together. Guessing that it takes new transformer technology to encode and tokenize incoming data.Starting to see research papers like this one.
Unknown appliances detection for non-intrusive load monitoring based on vision transformer with an additional detection head
1 Like