RTL_443 : MQTT topic remove "id" + add timestamp

Hello everyone,
I saw similar discussions in the past in this forum without able to find a solution unfortunately.
It’s about RTL433 devices and how we publish the MQTT topics.
Apparently we use this scheme : “xxxxx/GatewayName/RTL_433toMQTT/DeviceName/Channel-ID/ID”
but for some devices, the “ID” is not uniq, it changes all the time causing the creation of new mqqt topics.
In my example below, the AmbientweatherF007TH reports ID=122, then 185, 69, 126, etc.
so multiple entries are created in the MQTT database despite it’s the same device.
HomeAssistant sees this as a new device and loses the previous track.
Of course we can scripts, change things on the fly but it’s not ideal and dirty.

I’d love to see this way to report the topic name (without the #ID) : xxxxx/GatewayName/RTL_433toMQTT/DeviceName/Channel-ID/
any parameters exists already ? (can’t find it)

Another subject :
we are not publishing the timestamp, rtl_433 project does it (see in orange in the screenshot).
Timestamps are important to know if values reported are still accurate or obsolete.
would it be possible to see timestamps in a near future ?

thanks for all the jobs you are doing.
I really love this project.

1 Like

Why is your ambient weather changing its ID all the time ? With most implementations the ID is static, unless the device’s battery is changed. Is your battery too old or weak?

Timestamps is something we have been discussing.

1 Like

why ?, I’m not the manufacturer
but in any case the device must remain identical in mqtt database even after changing the battery.
we can’t redo all the config every 6 months when battery is dead
I see discussions regarding timestamps but not with rtl433 (or I’m blind)
only with SYStoMQTT which has a kind of “uptime” value
timestamps remains an important topic to ensure data are still valid.

1 Like

back with more examples
here is what was collected this afternoon
a lot of 433mhz devices living around are changing this “ID” too.
Not only it creates tons of entries in MQTT
but it’s also difficult to follow what reports what as you can see
would be very nice if we can have an option to remove this ID in the topic path
otherwise I’m afraid it’s hard to manage
thanks for your understanding

1 Like

with this little modification
1st tests made this morning are looking good


this changing IDs seen with several devices are not used anymore in the mqtt topic and in HA :slight_smile:

image

I’m wondering if you could add a command, maybe MQTTtoRTL_433,
to remove this ID or not, by default keep it to not disturb the existing.
what do you think ?

thanks and regards
stephane

1 Like

Timestamping is supported since 1.5.0 IIRC, but then maybe only for BLE sensors, check

message_unixtimestamp
or
message_UTCtimestamp

in the sources
If it is not in the 433 part of the code, should not be too difficult to add it

1 Like

The root problem is that the same sensor is identified with many different IDs, like so:

Something I have reported here

I can’t use this solution (liligo + OpenMQTT).

1 Like

Hi MaxG
it’s exactly the issue I have too
the problem is not the lilygo but coming from zgatewayrtl_433 source. It uses the “id” of the device to create the MQTT topic as explained earlier. This ID is changing, not uniq in many cases despite it’s the same device so we see unexpected new entries in mqtt and in homeassistant. Previous record becomes obsolete… not really usable unfortunately
the rtl_sdr project does not use this ID, but apparently mqtt_hass.py uses it with a kind of filter to remove it later when reported in HA (complex)

for my personal case, the usage of this ID is the cause but NorthernMan54 and other users might have another opinion, I fully understand. The usage of this ID was implemented for a good reason probably. Always difficult to change a design without risk.

for the short term, the workaround is to use a self-made private version without this ID and check regularly how this project will evolve.
OMG is very cool; I’d love to see this issue addressed in the public release so I could use the GA version.

steph

Hi everyone
a little contribution and proposal to get the timestamp with RTL433 module
here are 2 “diff” for User_config.h and ZgatewayRTL-433.ino
careful, just tested during 1h, seems to work but
I don’t know if enabling UTCtimestamp can cause collateral damages elsewhere…

add these 8 lines (without the “+” of course) in ZgatewayRTL_433.ino around line #254
define the UTCtimestamp true in User_config.h (or unixtime if you prefer but it’s not human readable)

image

Enjoy.

1 Like

The best thing to do would be a PR :slight_smile:

1 Like

Steph,
trying your changes, I get an error

D:/Documents/ESP projects/OpenMQTTGateway/main/ZgatewayRTL_433.ino:252:16: error: ‘ESP8266’ was not declared in this scope

I’m not sure where to look to fix this.
Dave

Hi Dave
what device are you using ?
it’s not an ESP32 or ESP8266 based receiver ?

Hi ilgrank
I’m not yet “fluent” in github usage,
to create a PR it seems I need to create a branch, commit this change and then create a PR
can’t do it now…

The board is a LilyGo T3_V1.6.1.

I’ll try commentig out the if statements.

Steph,
sorted it out. I was missing the #'s at the start of the if statements.
Only thing now is to find where the OMG_VERSION is set, ESPHome wanted to update it because it was reporting “version_tag”.

1 Like

Hi Dave
jump to main/User_config.h

1 Like

I too have this problem of new ID topics of the same device.
Until now I did not known where it came from, but I read in this forum the ID of some (or all?) RF433 Mhz devices changes when you replace batteries.
This is indeed unusable then if the ID is located in the sensors MQTT topic name as discussed here.
I too see neigbour motion devices reappearing with other ID’s. So their batteries are probably almost dead which causes this ID changing.
Is there a global option available yet to toggle this ID ON/OFF in in MQTT topic. Because I don’t want it either in my MQTT topics. => Problem with auto creating new devices in HA, leaving old dead topics and devices.
I too use the LilyGo RTL 433 (Lora) board.
Going to check now if I can compile a custom version myself adding Steph’s fix. But it would be good to see this Feature added in the main branch so I can follow updates automatically.

edit: made a custom version compile myself with both ID remove out of MQTT topic & timestamp UTC addition & custom version_tag :slight_smile:
Thx Steph, hopefully these will be added in the main branch to I can keep up to date with main branch without having to make a custom compile each time.

Glad it helps you :slight_smile: