Create MQTT sensor for presence detection of BLE devices

Integrations Home Assistant
,
1

I have two OMGs and one TheengsGateway I want to use for presence detection for a specific BLE device.

For this I have this sensor (multiple times of course):

    - name: "E-Bike (OMG 1)"
      unique_id: xyzxyzxyz
      state_topic: "OpenMQTTGateway/presence/ESP_OMG_1"
      value_template: "{{ 'home' if value_json.id == 'XX:XX:XX:XX:XX:XX' else 'not_home' }}"
      #expire_after: 10
      icon: mdi:bicycle-electric
      device:
        identifiers: XYZ
      entity_category: diagnostic

While this seems (!) to basically work, it seems like the presence topic for OMG and TheengsGateway is very different to the one I used before in another ESP based project (where the state topic contained a list of all devices currenctly around, so as long as the device was active it was part of the state topic, that way it was discovered “home” as long as it was nearby):

the topic only stores the latest device found, for a very short period of time (depending on the next new/strongest one detected).

Multiple discovery doesn’t matter as I have one collection sensor (contains OMG 1 + OMG 2 + TG + WiFi based sensor + previous Bluetooth sensors).

In short: how to use the OMG/TheengsGateway provided presence topics for doing some half-serious (no device_tracker entities needed at the end, but reliable discovery and ideally signaling how long the device is around) device tracking?

I know you natively implemented this in a very professional way (meaning in the end there’s a autodiscovery provided device_tracker entity in Home Assistant, just perfect) for many devices (BM2, fitness trackers etc.). Would this be “the way to go to do it fully professional”?

I’m currently looking at the best way to go. Focus:

  • reliability
  • speed (I can’t wait months to have this natively implemented and shipped in a stable OMG/TG release, that’s why I tried to use what I can with the existing MQTT topics)

Thanks a lot for your help on this. Unfortunately I found that doing device tracking on my own is quite complicated with bluetooth devices and the way the presence topics are build (ultra short-term lifetime/no list as I was used to).

2

How about just creating and publishing your own device tracker discovery message for the mentioned BLE device (E-Bike??)?

Then you would have a proper BLE device tracker entry for the bike.

Do you think you can cobble this together yourself by looking at the topic and payload of your other already existing device tracker discovery entries?

Which E-Bike is it, a very popular one, and is the BLE integration standard with it?

Could you also post some sample BLE messages you receive from it, with advertising data turned on? As this might be interesting to others as well, and the advertising data might include some decodable data, like battery level or such?

Does the bike have a native app showing some received information?

The actual presence option in OMG and TGW is only really applicable for room presence, but device trackers are the way to go if you just want a home/away presence detection.

3

How to achieve this?

It’s just the on-board computer (BOSCH), a Nyon 2 attached specifically to that E-Bike. It just “sends” BLE messages.
See Nyon: der Bordcomputer mit Navigation fßr Elektrofahrräder

Yes it does: ‎Bosch eBike Connect im App Store
Maybe that’s inside the manufacturerdata part below? At least it is not clear text. But my smartphone app is paired, the OMGs/TheengsGateway obviously not.

That’s exactly what I’m looking for.

presence:

{"id":"XX:XX:XX:XX:XX:XX","mac_type":0,"adv_type":0,"name":"Nyon-XXX","manufacturerdata":"a60210eb002500","rssi":-73,"txpower":12,"distance":4.756481}

BTtoMQTT:

{"id":"XX:XX:XX:XX:XX:XX","mac_type":0,"adv_type":0,"name":"Nyon-XXX","manufacturerdata":"a60210eb002500","rssi":-73,"txpower":12,"distance":4.756481}

I can’t use the BTtoMQTT one as this seems to be kind of persistent (not removed once the device goes away/off) and the presence one only contains the latest fetched one as mentioned in the OP.

Can you help me on this / work with the information provided @DigiH ?

4

This was the information which was needed to allow me to add a device tracker decoder for the Bosch Nyon, and assuming this advertising data is sent out in regular intervals.

You should be able to test and use this additional decoder with your ESP32 OpenMQTTGateway gateway by installing the latest development version from

5

Oh great! Thanks. I‘ll give it a try asap.

Please note I masked things like the MAC and the latter part of the name (usually „Nyon-XXX“ contains the last 3 characters of the MAC address), I forgot to mention this in the previous post. Not sure if this is relevant for the decoder - just to be sure.

6

I already assumed as much, with all the Xs :slight_smile: It’s fine, they were not relevant for creating the decoder.

What you could monitor however is if the manufacturerdata changes at all over time, to see if there is any additional data encoded in it, like possibly the accumulated distance cycled or such.

7

I want to avoid using the web installer. Instead I tried to upgrade from the OMGs web interface (/up?):

grafik
grafik771×1139 40.2 KB

Unfortunately, it stays on stable 1.7.0. Tried several times, after restart still 1.7.0 is flashed. Do you have a OTA URL for the dev version I shall test?

8

I think that might be due to the certificate change.

@1technophile might have a direct OTA URL for you.

Otherwise, using the web installer and unselecting Erase Flash should keep your current settings, unless there have been changes which require a new set up for the current development version.

9

OK. I’ll wait for the OTA URL then. It’s just so much easier flashing/updating them remotely, considering where the ESP(s) are physically located.

To be ready to test after switching to dev channel: what is needed to provoke the “catch” and create the device tracker? I remember back in the days when we set this up for the BM2 I had tough times achieving the creation of the device_tracker entity and spent a lot of unnecessary time then.

I guess I need to:

  1. Enable “SYS: Auto discovery” temporarily
  2. Enable “BT: Publish advertisement data” temporarily
  3. Speed things up maybe by enabling “BT: Force scan” once?

Always enabled FYI: “BT: Publish HASS presence” as well as…

grafik
grafik582×1537 63.4 KB

10

You still need to do an upload from :

To get the new certificate, it will not work to enter the new url here due to the Github certificate change.

11

So I need to use (Option 1) Upload from the web | Theengs OpenMQTTGateway DEVELOPMENT SHA:c5b9bc TEST ONLY, no remote update?

Therefore: how could I switch back to latest stable (1.7.0 release)? Again a local web flash (but this time using the stable web flasher (Option 1) Upload from the web | OpenMQTTGateway v1.7.0)?

12

And then press the SYS: Restart Gateway button. After the restart, once the device Tracker for the Bosch Nyon has been auto discovered you can disable SYS: Auto discovery again.

BT: Force scan doesn’t really help here, nor is BT: Publish HASS presence required to be enabled if you only want to use Home/Away device trackers.

You can keep BT: Publish advertisement data enabled for a few days, to see if the manufacturerdata changes at all, especially after having used your eBike for a ride. You could also tell us what battery level the Nyon shows in the app, as this might also be encoded in the manufacturerdata.

If the manufacturerdata doesn’t change at all just let us know here, and you can dsable BT: Publish advertisement data again. If iot does change we should investigate some more to see which data might be encoded there, to create a more detailed decoder for the Nyon.

1 Like
13

Yes, no remote update.

You will be able to switch back easily as all the 1.7.0 binaries have been copied to the new OTA host.
But if you switch back you will still need to do a manual upload for the v1.8.0

14

@e-raser

Did you get a chance to test this with the development build at all?

Any feedback?

15

Not yet, scheduled for Sunday. A bit difficult to do remotely when you need to flash locally, so it took/takes longer than I was initially hoping for. I‘ll turn back.