Invalid characters in JSON

[dbeinder]

Unfortunately, the issue #867 is still present in 2.4.0. I traced it on my Kaifa meter:

At first ID extraction works correctly:

amsreader-firmware/lib/MeterCommunicators/src/IEC6205675.cpp

Lines 729 to 732 in 20de294

	str_len = getString(AMS_OBIS_METER_ID, sizeof(AMS_OBIS_METER_ID), ((char *) (d)), str);
	if(str_len > 0) {
	meterId = String(str);
	} else {

But then it gets overwritten with the GCM system title: (probably intended as a fallback?)

amsreader-firmware/lib/MeterCommunicators/src/IEC6205675.cpp

Lines 902 to 904 in 20de294

	if(meterType != AmsTypeUnknown) {
	meterId = String((const char*)ctx.system_title);
	}

And since the 8-byte GCM system title of my Kaifa meter is not ASCII, we end up with JSON breaking characters in the frontend, MQTT, etc:

[gskjold]

But then it gets overwritten with the GCM system title: (probably intended as a fallback?)

Indeed, thanks for the thorough debug!

[etzisim]

Hy, i updated from 2.3.8 to 2.4.0 on my esp8266 an now i get the message json im Homeassistant mode with a beginning " ’b ", maybe its same to this Problem:

Nachricht 161 empfangen auf SmartMeter/flo/power um 11:56:
b'{"lv":"","id":"SAGY\x05\xe7\xdd\x1a","type":"","P":3087,"P1":0,"P2":0,"P3":0,"Q":0,"PO":0,"PO1":0,"PO2":0,"PO3":0,"QO":0,"I1":14.94,"I2":0.76,"I3":0.95,"U1":232.70,"U2":234.50,"U3":234.20,"PF":100,"PF1":0,"PF2":0,"PF3":0,"tPI1":0.000,"tPI2":0.000,"tPI3":0.000,"tPO1":0.000,"tPO2":0.000,"tPO3":0.000,"t":"2025-01-19T10:56:00Z"}'
QoS: 0 - Retain: false

[dbeinder]

Yes, that's the same issue. The b' is just how Python prints a binary string.

[gskjold]

I hope this will fix the issue:
esp32s3.zip
esp32solo.zip
esp32.zip
esp32c3.zip
esp32s2.zip

[dbeinder]

Didn't help:

UnicodeDecodeError: 'utf-8' codec can't decode byte 0x98 in position 136: invalid start byte
Failed to parse json: b'{"id":"70:04:1D:00:C1:B8","name":"ams-og","up":49,"t":1737639270,"vcc":0.000,"rssi":-81,"temp":-127.00,"data":{"lv":"","meterId":"KFM   \x98","type":"","P":13,"Q":0,"PO":0,"QO":0,"I1":0.06,"I2":0.06,"I3":0.04,"U1":226.70,"U2":228.80,"U3":230.10,"tPI":1697.468,"tPO":0.000,"tQI":21.038,"tQO":1333.503,"rtc":1737639270},"realtime":{"h":0.00,"d":0.2,"t":5,"x":0.03,"he":0.00,"de":0.0}}'

[gskjold]

How about now:

esp32s3.zip
esp32solo.zip
esp32.zip
esp8266.zip
esp32c3.zip
esp32s2.zip

[dbeinder]

The JSON is valid now, but meterId only contains KFM:

b'{"id":"70:04:1D:00:C1:B8","name":"ams-og","up":49,"t":1737713325,"vcc":0.000,"rssi":-83,"temp":-127.00,"data":{"lv":"","meterId":"KFM","type":"","P":13,"Q":0,"PO":0,"QO":0,"I1":0.06,"I2":0.06,"I3":0.04,"U1":233.10,"U2":231.50,"U3":234.10,"tPI":1697.843,"tPO":0.000,"tQI":21.038,"tQO":1333.980,"rtc":1737713325},"realtime":{"h":0.00,"d":0.1,"t":5,"x":0.03,"he":0.00,"de":0.0}}'

[ArnieO]

The JSON is valid now, but meterId only contains KFM:

Did you check (Telnet debug / https://www.gurux.fi/GuruxDLMSTranslator) what the meter sends as ID?

[dbeinder]

It works with 2.3.7 and reports the serial 1KFM0200169990. See first post, it still does initially extract the ID correctly, it just gets overwritten later. Commenting out the 3 lines of the second snippet fixes it. Here is a raw frame with key: mbus_1KFM0200169986.zip