Claude: Understanding Rate Zones on DCM: Heat Pump and Range
This is part two of my series on testing CSV data with LLMs.
Today I’m presenting my findings on Question 1 Using Claude by Anthropic with the model 4 Sonnet.
For clarification:
Anthropic is the company making the product.
Claude is the Chatbot that helps you interact with the language model.
Sonnet 3.7 and 4 are the language models.
As luck would have it, on 21 March 2025 I did all my testing for Question 1 on Claude 3.7 Sonnet and realized I made some major errors I needed to fix. My initial response was I was not impressed at all with the chat / text results I was being provided.
Once I went to redo my testing I saw Anthropic released Claude 4 Sonnet, a likely response to Google Releasing a new Gemini Model. The results for Clauder, 4 Sonnet turned out to be the same or worse. I thought it would do better.
Question 1 Understanding Rate Zones on my DCM Devices Heat Pump and Range
“Please provide me with the total usage for my heat pump and range.
Then tell me how much my heat pump and range cost me per rate zone.
How much did both cost me together and separately?”
My default electricity rate is 33.5¢.
I have 3 rate zones:
All 3 rate zones start January 1, 2025, and end March 23, 2025.
- “Sunday All Day” – This is for all day Sunday. Cost: 25¢ per kWh
- “Daytime Weekday Peak” – Monday through Friday, 10 AM to 3 PM. Cost per kWh: 55¢
- “Nighttime Cheap-o” – Saturdays 2 AM–4 PM. Cost per kWh: 3¢
Data and Sources:
Link to CSV data I uploaded into Claude: https://drive.google.com/file/d/1ZJCy0uLBTPnaOVbTfIPyLoDXPelBAnGT/view?usp=sharing
Link to CSV Used to break apart and calculate everything (not uploaded into Claude):
Results:
Sonnet 4
Claude Chat: https://claude.ai/share/02c89833-6efe-48d6-a45a-17ec08f5231f
Screenshot of Chart:
Let me also add that even though I allow the interactive artifact to be “published” and shared with a link whenever I click on it outside of walled off browser where I am signed into the test Claude account it shows up with this page:
So the design is clearly not meant for serious usage yet if I can’t be confident my findings can actually be shared. Therefore I had to share this data in screenshots.
Claude Interactive Artifact v1: https://claude.ai/public/artifacts/6fcb1ea5-5905-4c17-b2c1-9200e58101ec
Screenshot: https://drive.google.com/file/d/17PZppwqAkABuWWr5yyt28HO67iedCP3K/view?usp=drivesdk
Claude Interactive Artifact v2: https://claude.ai/public/artifacts/a530c137-5b11-424f-bff6-8161d0c8ac04
Screenshot: https://drive.google.com/file/d/1EjEPWtND5JfcfldOA3BHGjib4IJL1UEs/view?usp=drivesdk
Claude Interactive Artifact v3::
https://claude.ai/public/artifacts/e9d25e17-7385-46f5-9933-c7a929f46bf6
Screenshot:
Original Test
Sonnet 3.7: This is for you to see how Claude actually got slightly more accurate numbers (for the heat pump and thus overall) on the initial run. https://claude.ai/share/18641b19-d5ad-4d47-9a72-16371c1db3fc
N/A for Interactive Artifact for next Table.
Claude would not understand me when I wanted to combine usage for the interactive artifact. I asked several times trying to be specific and then it told me I maxed out the chat: https://drive.google.com/file/d/1cziLbASv6RXJ7mafkVeQcXez-NfejRZ3/view?usp=drivesdk
So I am not going to get my answers. Since this would be a normal users experience I am going to quit trying to get Claude to redo my answers here.
Conclusion: Inaccurate, Did not count the full data set.
Claude did not correctly count the total number of kWh for the heat pump or range and therefore could not be depended on to then use these numbers to determine usage and cost per rate zone. Claude chat just didn’t get the simple numbers right and it didn’t get the rate zones right either, not by a longshot.
What you can see though is that the interactive artifact, a major distinguishing factor from the other LLMs, actually did quite well in getting the correct numbers. It seemed that Chat and Artifact were not talking to each other because one gave good numbers and was able to add up a spreadsheet and make an easy to view graphic with it. The other just spat out… well, crap.
The IA also had trouble understanding as well but in a different way. I wanted to modify it by asking it to combine the heat pump and range and provide kWh and cost for both. The IA ended up just spitting out the same artifact unchanged for 3 versions then halted me. Asking it to combine data was just a little much I suppose.
Stay turned for more. Let me know if you want to help test out some models or want to help build upon or provide suggestions for my methodology - Also happy to share my templates.
Thanks!
Disclaimer: uploading your data into an LLM, especially a free one, may not necessarily be private and will likely be used as training data for said firm’s current and future models.