Whitelist not persisted?

[stan23]

Hi all,
I started to use OMG two days ago.
My use-case is a simple BLE-sniffer: I want to glue cheap beacons to my garbage bins, and check by reception strength whether they are close to their default location or moved to the street for pickup.

My broker is the one embedded into ioBroker, which is based on MQTT.js.

I was able to set a whitelist like this, with the retain flag enabled:

sendTo ('mqtt.0', 
        'sendMessage2Client', {
            topic: 'ble-scanner/OpenMQTTGateway/commands/MQTTtoBT/config', 
            message: '{"white-list":["C5:68:8A:91:0A:A7","D6:2F:D0:06:17:23","D4:69:22:C5:82:C0","CF:ED:96:AA:5F:50"]}'}
        );

It results in a log like:

N: [ MQTT->OMG ]: {"white-list":["C5:68:8A:91:0A:A7","D6:2F:D0:06:17:23","D4:69:22:C5:82:C0","CF:ED:96:AA:5F:50"]}

But after few minutes to hours, the watchdog kicks in and reboots the ESP32, and then the whitelist is lost.

Is that expected? Is there a way to persist the whitelist? Can it be hard-coded?

HW:
ESP32-WROOM devboard

SW:
esp32-dev-ble (prebuilt) and built myself from latest dev-branch

Regards
Marco

[DigiH]

@stan23

Is that expected? Is there a way to persist the whitelist? Can it be hard-coded?

I had similar questions a while back

Query about white-list/black-list - specifically how to remove devices from the lists, or to fully clear the lists

Hi all, Having set up my BLE gateway on an ESP32, everything is working as expected apart from the white-list or black-list, when I want to remove devices from them or clear them fully. The last test I have done was (fully ignoring retaining messages for the time being, until I figure this out Add one device to the white-list, so as to only get messages from it send: {"white-list":["XX:XX:XX:XX:XX:XX"]} Great, only device XX:XX:XX:XX:XX:XX gets reported, while the monitoring …

and white/black-lists cannot be hard coded.

@stan23

I was able to set a whitelist like this, with the retain flag enabled:

sendTo ('mqtt.0', 
        'sendMessage2Client', {
            topic: 'ble-scanner/OpenMQTTGateway/commands/MQTTtoBT/config', 
            message: '{"white-list":["C5:68:8A:91:0A:A7","D6:2F:D0:06:17:23","D4:69:22:C5:82:C0","CF:ED:96:AA:5F:50"]}'}
        );

Not knowing much about ioBroker, where in your MQTT message is the retain flag set to true though? If the white-list is sent with the retain flag set to true the MQTT broker should send the white-list again after every restart of the ESP32.

If you connect to your MQTT broker with an app like MQTT.fx or MQTT Explorer, does the white-list message show up as retained there when you subsribe to your OpenMQTTGateway?

I actually resend different white-lists, depending on other variables like presence and security state, via OpenHAB, by monitoring for the LWT state of OMG, which sends an “online” after every restart (Actually also a retained message, along with another one containing the version number - both great for checking to see if retained messages correctly show up for you). I’m sure this would also be possible with ioBroker, but a correctly set retain=true with your white-list message should do the trick for you.

Hope this helps.

[stan23]

Thanks for your explanation. I was searching the last posts, but didn’t find your old question.

In ioBroker the retain flag is global for the whole MQTT adapter, but apparently it does not work as expected. When I subscribed to OpenMQTTGateway I got several thousands of (old) messages from previous BLE scans. It looks like all messages are retained internally, which is most likely not expected.

But your suggestion to monitor LWT is great: this is what I tried now and it seems to work nicely. Will check back in few days and see if any other devices that the ones on my whitelist were pushed to the MQTT server on ioBroker.

Thanks & regards
Marco

[stan23]

Three days alter only a single device “bypassed” the whitelist, i.e. was detected before the whitelist was published to the OMG.
That’s pretty good and I’m happy with the solution to monitor LWT and then publish the whitelist:

const devBleScannerTopic: string =  'mqtt.0.ble-scanner.OpenMQTTGateway';
const topicWhitelist: string =      'ble-scanner/OpenMQTTGateway/commands/MQTTtoBT/config';
const contentWhitelist: string =    '{"white-list":["C5:68:8A:91:0A:A7","D6:2F:D0:06:17:23","D4:69:22:C5:82:C0","CF:ED:96:AA:5F:50"]}';

subscribe ({id: `${devBleScannerTopic}.LWT`, val: 'online'}, async function (obj) {
    log ('BLE scanner has connected, sending the while-list.')
    await wait (10);
    sendTo ('mqtt.0', 'sendMessage2Client', {
            topic: topicWhitelist, 
            message: contentWhitelist}
        );
});

Next step is to glue the beacons to the garbage cans

[DigiH]

Good to hear that this is also working fine for you

@stan23

Next step is to glue the beacons to the garbage cans

Please also let us know how this is working out for you with the beacons attached to the garbage cans. Is there quite a substantial distance between their usual place and the street pick up location? As the reported distance/rssi can vary within a range, and also seems slightly weather/humidity sensitive, especially outdoors.

All the best.

[stan23]

The usual place is between 1 and 3 meters away form the OMG.
At the street for pick-up they are like 10 meters away.

I hope there’s a significant difference for the rssi. If not, the costs & effort was rather low and I learned something about MQTT

[DigiH]

@stan23

The usual place is between 1 and 3 meters away form the OMG.
At the street for pick-up they are like 10 meters away.

These 7-9 metres differences should be clearly discernible, even with various rssi fluctuations.