Improve CM160 readings #2418

diepeterpan · 2023-03-12T05:32:32Z

Added more attributes and total kWh

Build and tested on MacOS

Added more attributes and total kWh

diepeterpan · 2023-03-12T05:42:31Z

Updated labels for raw values

zuckschwerdt · 2023-03-12T13:13:32Z

src/devices/oregon_scientific.c

@@ -142,6 +142,26 @@ static unsigned long long cm180i_total(uint8_t const *msg)
    return val;
 }

+static unsigned int swap_nibbles(unsigned char byte) {


types should be uint8_t

zuckschwerdt · 2023-03-12T13:14:26Z

src/devices/oregon_scientific.c

@@ -142,6 +142,26 @@ static unsigned long long cm180i_total(uint8_t const *msg)
    return val;
 }

+static unsigned int swap_nibbles(unsigned char byte) {
+ return (((byte&0xf)<<4) | (byte>>4));


Indent is 4 spaces please. Add spaces around operators () << 4)

Should be resolved

zuckschwerdt · 2023-03-12T13:15:21Z

src/devices/oregon_scientific.c

+ return (((byte&0xf)<<4) | (byte>>4));
+}
+
+static unsigned int cm160_power(uint8_t const *msg)


remove both cm160_ functions and put the code directly in the block below.

zuckschwerdt · 2023-03-12T13:16:43Z

src/devices/oregon_scientific.c

+
+static double cm160_total(uint8_t const *msg)
+{
+ double total = ((uint64_t) swap_nibbles(msg[10]) << 36) | ((uint64_t) swap_nibbles(msg[9]) << 28) |


No space after a cast: (uint64_t)swap_nibbles

zuckschwerdt · 2023-03-12T13:17:25Z

src/devices/oregon_scientific.c

-        unsigned short int ipower = (rawAmp /(0.27*230)*1000);
+
+        //channel add from https://github.com/magellannh/rtl-wx/blob/dbaf3924815903c47b230c65841d18b263421854/src/rtl-433fm-decode.c
+        int channel = msg[0]>>4;


This can't work. It was 0x2? and actually is 0 at this point.

Fixed, grabbed the channel now before manupilation of msg for checksum check.

zuckschwerdt · 2023-03-12T13:24:11Z

src/devices/oregon_scientific.c

+        //Alternate formula = (raw_ipower * 0.07) * 230 - https://github.com/cornetp/eagle-owl/blob/master/src/cm160.c
+
+        double raw_total_energy = cm160_total(msg);
+        double total_kWh = (raw_total_energy * 230 ) / 3600.0 / 1000.0;  //Assuming device is running in 230V country


If kWh = raw * 230 / 3600 / 1000 then J = Ws = raw * 230, so the unit is Ws or J, or actually As = raw which is strange but plausible.

Fixed, this device measures amps, current and total.

Makes sense. But then "total_amps" should be "total_ampsec" -- it is energy (A*s) not power (A).
And in data_make below it should be "current_A" (but this is not really A? There is a factor of 0.27?) and "total_As"

Added unit A to current and total amps, there should be no need to indicate seconds as "One Ampere is defined as the current that flows with electric charge of one Coulomb per second"

Re: factor .27 ; this is above my pay grade, that's how I found it ;-) ; maybe the formulae can be improved but am not sure how the original author determined it.

Ok, it's easy to confuse per second with x seconds ;) You will see kmh and know it's per second, i.e. km/h, but here I really meant As not A/s :)
An "instant flow of electric current" is A or like you found C/s. But an "accumulated amount of electric charge" is C wich is A·s :)
They are proportional with power and energy (work), i.e. multiply with voltage and you get W and `Ws (=J) respectively.
In short it's not both "amps", they are very different, the later is "accumulated by time" -- work, the thing you actually pay for.

Yes, that's a very strange thing about those meters. To count energy (or accurately show power) you will need to measure voltage. Pretty sure the device is doing that, the usual chips measure with "True RMS" -- they measure voltage very detailed, i.e. not only the effective ~230 V but the whole sine wave.

The (0.27*230)*1000 is not correctly written, why would you divide by voltage?
The (same factor) of 0.07 * 230 is better, but it's just *16.1 -- a random factor that likely was just estimated (linear regression?)
So maybe it is really just plain *16 -- which is exactly 4 bits, so maybe we are missing those bits?
Then no 230 needed and it would always work, everywhere.

Strange about the energy though, we do need the 230 and the *230/3600/1000 just looks perfect.
But then 3600/230 is also nearly exactly 16! A good guess would be that the field actually is not As but Wh already…

Seems all very much more likely than random conversions.

Can you test if current_watts = (current_amps * 16); // are we missing a nibble?
and total_kWh = total_amps / 16.0 / 1000.0; // should (msg[5]&0xf) belong to current_watts actually?
work with accuracy? They seem much more probable.

I tested, unfortunately, off by a couple of watts which is not consistent with the device's LCD value. Nor am I convinced that the external device sending the 433 signals and clamping onto the power wire is aware of the volts. During setup the volts are captured on the internal LCD device and it is not transmitted to the external clamp as the calculation of watt occur on the internal device.

Oh, it's a contactless current-clamp only. I wasn't aware. Thanks for testing.

Oh, it's a contactless current-clamp only. I wasn't aware. Thanks for testing.

I have a second one on another property I will also verify the kWh calc with, I was able to reset it recently so the number is fairly low and visible on the LCD. Then if there's a discrepancy we can chat again.

diepeterpan · 2023-03-12T15:57:01Z

After latest changes

zuckschwerdt · 2023-03-12T18:22:22Z

src/devices/oregon_scientific.c

@@ -806,16 +811,46 @@ static int oregon_scientific_v3_decode(r_device *decoder, bitbuffer_t *bitbuffer
        return 1;
    }
    else if ((msg[0] == 0x20) || (msg[0] == 0x21) || (msg[0] == 0x22) || (msg[0] == 0x23) || (msg[0] == 0x24)) { // Owl CM160 Readings
-        msg[0] = msg[0] & 0x0f;
+
+        int channel = msg[0]>>4; //let's grab the channel before we manipulate the msg for checksum validation


But we know (msg[0] == 0x20) || (msg[0] == 0x21) || (msg[0] == 0x22) || (msg[0] == 0x23) || (msg[0] == 0x24)
So msg[0]>>4 must be == 2, right?

I see, was the original code maybe swap_nibbles(msg[0]) >> 4? Then we'd get 0,1,2,3,4 which sounds like a channel, yes.
Use msg[0] & 0xf then.

Agree and changes accordingly.

zuckschwerdt · 2023-03-12T18:24:54Z

src/devices/oregon_scientific.c

@@ -142,6 +142,11 @@ static unsigned long long cm180i_total(uint8_t const *msg)
    return val;
 }

+static uint8_t swap_nibbles(unsigned char byte)


other type should also be uint8_t, we don't use unsigned char

diepeterpan · 2023-03-12T20:25:37Z

Latest output

diepeterpan · 2023-03-12T20:35:18Z

Removed channel, unless the device is channel hopping it is incorrect.

zuckschwerdt · 2023-03-12T21:24:08Z

Thanks for looking into the possible "channel". I always thought that 0x20 to 0x24 are device versions maybe, as the 0x25 is a CM180i and the 0x26 a Owl CM180.
Strange that you observe it randomly(?) changing.

diepeterpan · 2023-03-17T13:45:49Z

I have done so much testing and googling the past day. My answer on kWh appears to be a consistent 12% lower then the OWL CM160 LCD, whether I measure for 1 hour or for 10 hours. What do you think about whether I just * 1.12 in the code? I would have like to hear other users of these units on kWh reading.

gdt · 2023-03-17T13:50:56Z

Adding in a *1.12 doesn't seem right. It seems like one thing to figure out is how the voltage is being assumed. A 10% variation in line voltage from nominal is not super surprising.

diepeterpan · 2023-03-17T13:56:14Z

Adding in a *1.12 doesn't seem right. It seems like one thing to figure out is how the voltage is being assumed. A 10% variation in line voltage from nominal is not super surprising.

The OWL CT Clamp is measuring Amp, not Volt. Unfortunately, this is the best calculation I could get to not knowing what the manufacturer is doing internally to convert it to kWh. Just wish there were more consumers of the device that could test and provide input.

gdt · 2023-03-17T14:00:41Z

Do you configure a voltage in the device? Is it assuming 125V (US)? something else? Are you trying to use that configured voltage in the decoder? Maybe the extraction of amps is wrong; perhaps use another device to crosscheck? Have you compared the total to the official power meter?

diepeterpan · 2023-03-17T14:04:44Z

Do you configure a voltage in the device? Yes, only on the LCD unit receiving the 433 codes, the CT clamp sending 433 codes in Amp measurement is not aware of he Voltage. Is it assuming 125V (US)? It is not assuming, it is set to 230V. something else? Are you trying to use that configured voltage in the decoder? Yes, the code contains 230. Maybe the extraction of amps is wrong; perhaps use another device to crosscheck? I have used two different ones. Have you compared the total to the official power meter? Yes.

zuckschwerdt · 2023-03-17T15:08:01Z

I think this now describes the setup very well. The amp clamp sends "values", the receiver "corrects" these values and displays them. We can compare the radio "values" and what the receiver displays and guess what the correction must be.

For the amps power others already found a scale factor of 0.07, i.e. amps = value * 0.07 or watts = value * 0.07 * 230
It stands to reason that some scale factor could also be needed for the kWh energy, e.g. ampsecs = value * 1.12 or wattsecs = value * 1.12 * 230 resp. kWh = value * 1.12 * 230 / 3600 / 1000

And maybe the suspected Amp/Watt seconds value is really a Amp/Watt hours or Kilo Amp/Watt hours value and the divs are just part of the correction factor anyway? It looked so neat before :/

I think we are getting closer, still I don't like this correction factor guessing the slightest.

gdt · 2023-03-17T15:14:04Z

I don't like it either but it also occurs to me that the device is taking two shortcuts that are hacky:

assuming voltage rather than measuring
assuming some kind of power factor

Really RMS current * RMS voltage is not power. Power is the average of current * voltage, which involves paying attention to phase. More complicated meters do this.

So, the idea that the meter is displaying something which is A * configured_V * k, where k is some scale factor of 0.85 (very very ish) is not crazy.

But if the A matches rtl_433 to display, then the issue isn't in decoding A. My next question is how the time integral of A(t) * V_fixed compares to the device's own display.

I still don't understand if kWh displayed on the CM160 is consistent with the utility company's meter. As an example I have an Aeotec Home Energy Meter Gen5 (2 clamps and a single voltage sense) and the integrated power values are within a few percent of the power company's meter. But that's zwave with no decoding guessing.

diepeterpan · 2023-03-17T15:37:17Z

I don't like it either but it also occurs to me that the device is taking two shortcuts that are hacky:

assuming voltage rather than measuring

assuming some kind of power factor

Really RMS current * RMS voltage is not power. Power is the average of current * voltage, which involves paying attention to phase. More complicated meters do this.

So, the idea that the meter is displaying something which is A * configured_V * k, where k is some scale factor of 0.85 (very very ish) is not crazy.

But if the A matches rtl_433 to display, then the issue isn't in decoding A. My next question is how the time integral of A(t) * V_fixed compares to the device's own display.

I still don't understand if kWh displayed on the CM160 is consistent with the utility company's meter. As an example I have an Aeotec Home Energy Meter Gen5 (2 clamps and a single voltage sense) and the integrated power values are within a few percent of the power company's meter. But that's zwave with no decoding guessing.

Yeah, I don't like it either, but unfortunately don't have more time to spend on this. The CM160 documentation state it may be accurate to 10%, hence I stopped comparing it to the utility reading. I just wanted to ensure that my kWh from the 433 cumulative total is as close as possible to the reading on the LCD console over time. Originally the calculation seemed so perfect and scientific. Thank you to everyone for their input.

merbanan · 2023-03-17T15:39:17Z

If possible we want a 1 to 1 mapping with the LCD output but if we cant do that then just do as best as possible and document it in the source code.

diepeterpan · 2023-03-17T16:03:43Z

If possible we want a 1 to 1 mapping with the LCD output but if we cant do that then just do as best as possible and document it in the source code.

Let me get some of the test evidence, I have been running webcam captures for the CM160 LCD console and have been writing the decoded and converted 433 values to a DB. Let me get some data and images and post them here. Maybe someone sees the problem. Unless it is just a factor of unknown from the vendor IP.

diepeterpan · 2023-03-17T16:20:37Z

Let's pick a start and end time, let's 1 hour +-.

LCD console @ 16:00 to 17:00 (the large number is the current Watt reading, lower right is total kWh).
19.332 - 18.937 = 0.395

Reading from database
72.788 - 72.438 = 0.35 and when times 1.12 = 0.392

I can do a longer period 3 hours and it would still be 12%, something in the formula is unknown by the vendor.

diepeterpan · 2023-03-17T16:31:56Z

Longer period and including the raw 433 values.

433 values decoded, converted, and stored

72.788 - 70.680 = 2.108 kWh * 1.12 = 2.3609

LCD 12:00 = 16.948 and 17:00 as prev posted 19.332 = 2.384 kWh

Maybe someone can from the Raw values see where the mistake is.

merbanan · 2023-03-17T16:46:09Z

The LCD console most likely only have 16 or 32 bit fixed point math to present the displayed value. I am all for to just leave it with a 1.12 factor. As long as that is better then before.

diepeterpan · 2023-03-17T16:49:11Z

The LCD console most likely only have 16 or 32 bit fixed point math to present the displayed value. I am all for to just leave it with a 1.12 factor. As long as that is better then before.

Thanks, I will do a PR.

I just did another calc; and yes, it is much closer with the 1.12.

diepeterpan · 2023-03-17T16:57:02Z

Created PR for 1.12 - #2427

If and when someone finds a more correct formula when can always adjust.

diepeterpan added 2 commits March 12, 2023 07:08

Improve CM160 readings

aed7740

Added more attributes and total kWh

Update labels to raw values

b539c80

Remove trailing white space

573657b

zuckschwerdt reviewed Mar 12, 2023

View reviewed changes

Fixed numerous findings by Christian W.

f09223d

zuckschwerdt reviewed Mar 12, 2023

View reviewed changes

Minor changes to remove white space and unit naming

9b823a9

Removed channel, not correct

2d78527

Remove un-converted values

8bb4e25

zuckschwerdt merged commit 147bc5c into merbanan:master Mar 12, 2023

diepeterpan mentioned this pull request Mar 17, 2023

Added kWh to contrib/oregon_scientific.c and minor tweaks in src/rtl_433/pulse_demod.c NorthernMan54/rtl_433_ESP#64

Closed

andrewjw pushed a commit to andrewjw/rtl_433 that referenced this pull request Sep 29, 2023

Add total energy kWh to CM160 readings (merbanan#2418)

6a30ca5

zuckschwerdt mentioned this pull request Nov 27, 2023

Improve code and annotations for cpplint #2683

Merged

Improve CM160 readings #2418

Improve CM160 readings #2418

Conversation

diepeterpan commented Mar 12, 2023

diepeterpan commented Mar 12, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

diepeterpan commented Mar 12, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

diepeterpan commented Mar 12, 2023

diepeterpan commented Mar 12, 2023

zuckschwerdt commented Mar 12, 2023

diepeterpan commented Mar 17, 2023

gdt commented Mar 17, 2023

diepeterpan commented Mar 17, 2023

gdt commented Mar 17, 2023

diepeterpan commented Mar 17, 2023

zuckschwerdt commented Mar 17, 2023

gdt commented Mar 17, 2023

diepeterpan commented Mar 17, 2023

merbanan commented Mar 17, 2023

diepeterpan commented Mar 17, 2023

diepeterpan commented Mar 17, 2023

diepeterpan commented Mar 17, 2023

merbanan commented Mar 17, 2023

diepeterpan commented Mar 17, 2023

diepeterpan commented Mar 17, 2023