Esp32 home/OpenMQTTGateway/LWT offline

curious_bubble · April 8, 2022, 2:49pm

I’m want to scan ble devices using ESP32 and send it to the MQTT Broker. It looks like ESP32 is able to connect to the broker but sends “offline”. Serial monitor shows the scanned devices and data it is trying to send. I’m using MQTT fx tool to check the data received on all the topics using home/#. the broker only receives “offline” on home/OpenMQTTGateway/LWT every few seconds.
From the serial monitor logs, (at the end) it looks like the devices is reconnecting repeatedly in every loop, is it expected?

I’m using Arduino IDE and the only changes I have done are below (credentials are masked):

Logs from Serial monitor:

************* WELCOME TO OpenMQTTGateway **************
I: OpenMQTTGateway Version: version_tag
I: WiFi ok with manual config credentials
I: BLE scans interval: 55555
I: BLE scans number before connect: 10
I: Publishing only BLE sensors: false
I: minrssi: 100
I: Low Power Mode: 0
I: OpenMQTTGateway modules: ["BT"]
I: ************** Setup OpenMQTTGateway end **************
W: MQTT connection...
I: Connected to broker
I: Send on /SYStoMQTT msg {"uptime":8,"version":"version_tag","freemem":144112,"mqttport":"1883","mqttsecure":false,"freestack":6148,"rssi":-64,"SSID":"XXXX","BSSID":"XX:32:XX:59:XX:XX","ip":"192.168.1.129","mac":"84:XX:A8:7A:XX:XX","lowpowermode":0,"btqblck":0,"btqsum":0,"btqsnd":0,"btqavg":0,"interval":55555,"scanbcnct":10,"scnct":0,"modules":["BT"]}
W: MQTT connection...
I: Connected to broker
W: MQTT connection...
I: Connected to broker
W: MQTT connection...
I: Connected to broker
I: Scan begin
I: Device detected: XX:XX:CC:XX:C5:XX
I: Send on /BTtoMQTT/XXXXXXXXXX msg {"id":"XX:56:XX:FC:XX:XX","mac_type":1,"manufacturerdata":"xxxxxxxxxxx","rssi":-91,"txpower":12}
W: Client not connected, aborting thes publication
W: MQTT connection...
I: Device detected: XX:60:XX:E9:XX:XX
I: Device detected: XX:93:F1:XX:DF:XX
I: Device detected: 2B:XX:XX:FC:XX:XX
I: Device detected: AC:XX:XX:AB:81:XX
I: Connected to broker
I: Send on /BTtoMQTT/XXXXXXXXXX msg {"id":"XX:60:XX:XX:XX:XX","mac_type":1,"manufacturerdata":"xxxxxxxxxxxxxxxx","rssi":-97,"txpower":8}
I: Device detected: D8:01:00:05:29:4A
I: Send on /BTtoMQTT/xxxxxxxxxx msg {"id":"XX:93:XX:B9:XX:XX","mac_type":1,"manufacturerdata":"xxxxxxxx","rssi":-61}
I: Device detected: XX:XX:3F:XX:81:XX
I: Send on /BTtoMQTT/XXXXXXXXX msg {"id":"XX:XX:21:XX:XX:XX","mac_type":1,"manufacturerdata":"xxxxxxxxxxxxxx","rssi":-95}
I: Send on /BTtoMQTT/XXXXXXXXX msg {"id":"XX:XX:21:XX:XX:XX","mac_type":0,"name":"XXXX","manufacturerdata":"xxxxxx","rssi":-86}
I: Send on /BTtoMQTT/XXXXXXXXX msg {"id":"XX:XX:21:XX:XX:XX","mac_type":1,"rssi":-82,"servicedata":"xxxxxxxx","servicedatauuid":"xxxx"}
I: Send on /BTtoMQTT/XXXXXXXXX msg {"id":"XX:XX:21:XX:XX:XX","mac_type":0,"manufacturerdata":"xxxxxx","rssi":-78}
I: Device detected: XX:XX:XX:XX:XX:XX
I: Send on /BTtoMQTT/XXXXXXXXX msg {"id":"XX:XX:21:XX:XX:XX","mac_type":1,"name":"xxxx","manufacturerdata":"xxxxxx","rssi":-82}
W: MQTT connection...
I: Device detected: XX:XX:21:XX:XX:XX
I: Device detected: XX:XX:21:XX:XX:XX
I: Connected to broker
I: Send on /BTtoMQTT/XXXXXXXXX msg {"id":"XX:XX:21:XX:XX:XX","mac_type":0,"name":"xxxx","manufacturerdata":"xxxxxx","rssi":-78}
I: Send on /BTtoMQTT/XXXXXXXXX msg {"id":"XX:XX:21:XX:XX:XX","mac_type":0,"name":"xxxx","manufacturerdata":"xxxxxx","rssi":-78}
I: Found 8 devices, scan number 1 end
I: BLE Connect begin
I: BLE Connect end
W: MQTT connection...
I: Connected to broker
W: MQTT connection...
I: Connected to broker
W: MQTT connection...
I: Connected to broker
W: MQTT connection...
I: Connected to broker
W: MQTT connection...
I: Connected to broker
W: MQTT connection...
I: Connected to broker
W: MQTT connection...
I: Connected to broker
W: MQTT connection...
I: Connected to broker
W: MQTT connection...
I: Connected to broker
W: MQTT connection...
I: Connected to broker
W: MQTT connection...
I: Connected to broker
W: MQTT connection...
I: Connected to broker
W: MQTT connection...
I: Connected to broker
W: MQTT connection...
I: Connected to broker
W: MQTT connection...
I: Connected to broker

DigiH · April 8, 2022, 3:46pm

Hi @curious_bubble

looking at your logs it looks like you’re using a user name and password for your MQTT broker connection, but the MQTT_PORT is still defined for an insecure connection - 1833.

What happens when you define the secure TLS MQTT broker port 8883?

Either in the build_flags

'-DMQTT_PORT="8883"'

or in User_config.h

#ifndef MQTT_PORT
    # define MQTT_PORT "8883"
#endif

Hope this helps

1technophile · April 8, 2022, 5:54pm

Hello,

It could be due to having several gateways with the same name I think

curious_bubble · April 8, 2022, 6:27pm

Hi @DigiH,
No, I think my MQTT_PORT is correct, it is as per the recommendation of the MQTT broker service.
I have been using it with my other projects as well.

DigiH · April 8, 2022, 9:38pm

Yes, it really depends on the broker settings, but as user name and password usage often go together with encrypted connections only I thought that could have been the issue

Hope you can figure out what is causing it.

curious_bubble · April 10, 2022, 12:37pm

Solved!

The issue was, will_Retain flag.
The MQTT Broker I’m using doesn’t support retained messages (strange!). The will_Retain is “true” by default.
@1technophile would it make sense to make this will_Retain “false” by default and let developers change it to “true” if they need it? this won’t change much but I think having will_Retain “false” will be supported by all MQTT brokers, as I understand it is an optional flag. but some services might have a problem and it is tricky to find this out.

Thanks @1technophile & @DigiH for your support!

DigiH · April 10, 2022, 2:03pm

Hi @curious_bubble,

I’m glad you have solved the issue, but the fact that your MQTT broker doesn’t handle retained messages sounds as if it doesn’t (yet) fully support MQTT v3.1.1. Therefor the constant disconnect was actually default behaviour

If invalid flags are received, the receiver MUST close the Network Connection

Could it be that there is an update of your broker so that it supports MQTT v3.1.1??

curious_bubble · April 10, 2022, 2:17pm

Hi @DigiH ,
Makes sense, I’ll check if there’s an update. thanks.