I thought I was getting into NILM too and became dismayed when I read a paper by Jack Kelly on Disaggregation of Domestic Smart Meter Energy Data.
However, I recently realised that’s not what I’m trying to do. The NILM goals all cited are to understand in more detail how to attribute bills to appliances around the house. I’m not aiming for that.
I’m aiming to identify when a small number of simple appliances are being used. I don’t really need to disaggregate a washing machine’s power use from the overall power profile. I do need to know when a kettle was boiled or a toaster used. These are simple on/off devices and their addition of approx. 800w or 3kw respectively for about 90s or 1-2 mins respectively is evidenced on the profile by a jump up when the device switches on and down when it switches off. The duration between the two with no repeat (for the kettle, which could be mistaken for a dryer) is the giveaway. You don’t need 16kHz sample rates and powerfactor or harmonics to do it. Nor should I need to train a neural net.
I have actually already implemented rudimentary device ID on the ESP itself that reliably spots my toaster but as I plan to enhance the algorithms over time, I preferred to keep the brains away from the user devices for ease of management.
And there’s nothing inadequate about a clamp meter. It gives 1 second resolution.
I’m making progress. Having cleaned up my data feed to improve the timing, I can easily spot a jump or drop using
Yes, that’s pretty much it. It really helps to see what the cumulative load looks like. I can also annotate when (say) I put on the washing machine, dryer or microwave and move back in time on demand to look at specific time periods. It’s really very useful while I’m learning about time-series data and Influx may be flexible enough to do what I want, work out the gotchas and so on. The alerting capability is also perfect for what I’d call a “least viable product” - something I can demonstrate actually does what it’s supposed to.
So now I’m trying to contend with a power rise/fall spanning two seconds and still need to work out how to add one data value to the previous one in the sequence… Any ideas?
I’m still not convinced you will be able to discern serveral appliances turned on at the same time.
How are you coping with solar power being generated and intefering with your power plot? Take a look at how messy your graph becomes once the sun starts shining: click for graph
Furthermore you need to think about the use of Grafana. You can make your calculations in/with Grafana, in C (esp32), in your database (stored procedures) or in a separate program (python c.a.). In my opinion it’s best to keep the complexities out of Grafana. I’m making most of the calculations on a esp8266. You can take a look at the calculated values on the webpage hosted on my ct sensor here
You’ve got 3 problems to solve:
adequately detecting a relevant rise/fall of power
adequately detecting which appliance is causing this rise/fall of power
logging starttime rise/fall of power + endtime rise/fall of power + delta power + name appliance
Once the data has been saved you should not add the values but the delta values.
You were absolutely correct about the problems to solve.
I’ve been learning a little about Python and Jupyter Notebooks. Early days but I’ve managed to get my InfluxDB data into a Pandas Data Frame and I’m learning how to analyse it.
In the meantime, I’ve been exporting CSV samples of my kettle activations to a spreadsheet and playing with formulas to detect activation/deactivation and the “On-time”. I think I’ve got a minimalist working version and with that as a reference, I hope to translate it into Python code.
It’s basic, needs refinement to “clean” the noise a bit and now needs testing in a “busy” environment, with fridge/freezer/microwave/oven/washing machine etc. all going off to see where it breaks down, but baby steps, eh?
If several devices all turn on within a second or two, I don’t yet have the means to distinguish them. However, I anticipate a “teaching” session or two when my device is first installed, so I can correlate at least some appliances with the data. I’ll work on it once I have the basics.
So far so good
I’ve relegated Grafana to what it does best. The smarts are likely to be serverless Python scripts.
