I’m a new Sense user. Is there anyway to export or download off a server the history data of my power consumption? Being able to have raw data to use in excel for other analysis would be beneficial.
Not yet, although they’ve hinted maybe sometime this year?
Indeed - @NJHaley is correct. Our initial barrier was getting the web app out into the world, and we’re hoping to build some data export functionality into that this year!
1 Like
If you know the R language here’s a junky little program that will scrape data using the RSelenium package. You can do the same using Selenium with Python, but I like getting the data into R-Studio. R and R-Studio are free and much better for data analysis than Excel.
The code opens up a Chrome web window, logs into your account, navigates to the monthly use page and scrapes the monthly usage data for the current month plus the previous month (that’s what is available from that page). You may need to play with the timing (Sys.sleep() statements) if something is happening too early, but this code works for me most of the time, except when Sense web app is slow to update. You’ll also have to code in your email and password for your Sense account where indicated.
library(RSelenium)
# Set up Chrome session
rD <- rsDriver (port = 4445L)
remDr <- rD$client
# Login to the website
remDr$navigate ("https://home.sense.com/login")
Sys.sleep(10)
webElem <- remDr$findElement(using = "xpath", '//*[@id="application__main"]/div/div[1]/form/div[1]/input')
webElem$sendKeysToElement(list("YourEmail\n"))
Sys.sleep(1)
webElem <- remDr$findElement(using = "xpath", '//*[@id="application__main"]/div/div[1]/form/div[2]/input')
webElem$sendKeysToElement(list("YourPassword\n"))
Sys.sleep(5)
# Go to usage page for current month. Can add navigational steps to do multiple months, or solar if wanted
remDr$navigate ("https://home.sense.com/trends/usage")
Sys.sleep(5)
# Extract the date and kWh data on that page...
# Data will include current month (to date), plus previous month
webElem <- remDr$findElements(using = "class", 'bar-graph__hover-info')
date.list <- sapply(webElem, function(x){x$getElementAttribute("outerHTML")})
webElem <- remDr$findElements(using = "class", 'consumed')
energy.list <- sapply(webElem, function(x){x$getElementAttribute("outerHTML")})
# Get rid of all the extra HTML gleet (could probably do in a single line)
date.list <- gsub ('"'," ", date.list)
date.list <- gsub (""," ", date.list)
date.list <- gsub ("<.*/div>","", date.list)
energy.list <- gsub ('', "", energy.list)
energy.list <- gsub ('', "", energy.list)
# Pad energy.list to be the same length as date.list
length (energy.list) <- length(date.list)
# Convert to dataframe
energy.df <- data.frame(matrix(unlist(date.list), ncol=1))
energy.df$Energy <- energy.list
colnames(energy.df) <- c("Date", "Energy")
# Output CSV format
write.csv(energy.df, "./Energy.csv")
2 Likes
Nifty little bit of code, @kevin1. I hope that downloadable data for a month would be at least down to the hourly level; preferably even finer (down to the minute or second - perhaps depending on the interval chosen - e.g.: several days could be to the second, several weeks would be to the minute).
Good point,
My code is limited to whatever the web app displays. I hadn’t tested it earlier on anything other than the default “Month” viewing option, but it appears to work on Day (hourly resolution), Week (daily resolution) and Year (monthly resolution) if you run the script in pieces and navigate to those in lieu of the navigation directives in the script.
Looking forward to a more flexible data export as you describe. Until then, I’ll fish for whatever is available.
Relevant to this thread: What's new in Web App v4: Data Export
1 Like
This topic was automatically closed 365 days after the last reply. New replies are no longer allowed.