BM2 to HomeAssistant. Getting % but not Voltages

Josh_Grubman · February 26, 2024, 2:29pm

Can someone help me get actual voltages into HomeAssistant? When I use the BM2 native app I get voltages down to two decimal places. But when it comes through OMG to MQTT to HomeAssistant, I’m only getting Home/Away and a battery reading as %. The Voltage says “unknown”.

Did I miss something in the setup for the gateway to pass voltages through to HomeAssistant?

Thank you!

DigiH · February 26, 2024, 2:32pm

Hi @Josh_Grubman

Just to make sure, you are using OpenMQTTGateway on an ESP32 or Theengs Gateway (Add-On) on Windows/Linux?

Josh_Grubman · February 26, 2024, 2:34pm

ESP32. I flashed the esp32dev-ble to an ESP32 that I had extra from a prior project, using the web-install page.

I am able to see both the gateway and the BM2 in HomeAssistant, so I feel like i’m close.

DigiH · February 26, 2024, 2:41pm

Good, as only OpenMQTTGateway is currently able to get the voltage property, as it needs to be fetched by conenction - different to the presence and battery level being received and decoded with every BLE bloadcasts of the BM2.

Because of the connection there is a different interval to the usual receiving passive or active scan interval.

You can see the connection interval (in ms) under BTtoMQTT as intervalcnct, with the default being 3600000, i e. 1 hour. So you should see your voltage value after an hour, with it then being updated every hour.

This longer connection interval is defaulted to an hour as connections most affect the battery on the devices themselves, and is usually ok for such values. You can set it to something lower or are ok with it updating with that interval - as long as the bleconnect setting is also true.

Hope this clarifies things.

Josh_Grubman · February 26, 2024, 2:44pm

Something’s wrong, then. I set this up last night. I’ve been logging the battery percents for the last 12 hours, but have yet to get a single voltage number come through.

DigiH · February 26, 2024, 2:48pm

Yes, voltage should have been fetched every hour since then as well.

The only thing I can think of then is that it might not be a BM2, but a BM6 battery monitor model, for which we do not know the connection function to retrieve the voltage yet, or a newer BM2 model which has also changed its voltage retrieval routine and/or key, as the device tracker and battery level seem to be functioning fine with the decoder BM2 model recognition.

A serial output with (notice/trace) logging could help to see what is happening during the connection attempt.

Josh_Grubman · February 26, 2024, 3:13pm

That’s the BM2 I just received. It is new.

How would I do a serial output?

DigiH · February 26, 2024, 3:29pm

For starters you can go into the WebUI and watch the logging under the Console option. With the default notice logging you might see

“Invalid notification data”

which would indicate a new/different connection value response for your BM2. For this you might want to set the connection interval quite low, so as the voltage retrieval happens more frequently.

Or if this doesn’t show any “Invalid notification data” logging, by activating trace logging in the WebUI a more detailed logging during the connection attempts should show up.

Josh_Grubman · February 26, 2024, 3:32pm

Nothing invalid found. I find this entry with my bluetooth mac address:
N: Send on /BTtoMQTT/50547B2283D8 msg {“id”:“50:54:7B:22:83:D8”,“rssi”:-89,“brand”:“GENERIC”,“model”:“BM2 Battery Monitor”,“model_id”:“BM2”,“type”:“BATT”,“batt”:97,“device”:“BM2 Tracker”}

DigiH · February 26, 2024, 3:37pm

This is only a publishing with the device tracker and battery level. Change the connection interval to 10000 so that OMG tries to connect to fetch the voltage very 10 seconds, by entering

commands/MQTTtoBT/config {"intervalcnct":10000}

into the Console entry field and hit Return.

Josh_Grubman · February 26, 2024, 3:37pm

I set the more verbose logging and found this chunk of code that mentions this device’s MAC. It also had my wifi SSID and its local 192.168 address. I changed those in the code below but otherwise this is a direct copy/paste from my logs. See anything useful?

Send on /BTtoMQTT/50547B2283D8 msg {“id”:“50:54:7B:22:83:D8”,“rssi”:-93}

T: jsonPubl - ON

T: [ OMG->MQTT ] topic: home/OMG_ESP32_BLE/BTtoMQTT/50547B2283D8 msg: {“id”:“50:54:7B:22:83:D8”,“rssi”:-93}

T: Enqueue JSON

T: Queue length: 1

T: Enqueue JSON

T: Queue length: 2

T: Enqueue JSON

T: Queue length: 3

T: Dequeue JSON

N: Send on /SYStoMQTT msg {“uptime”:48726,“version”:“v1.7.0”,“disc”:false,“ohdisc”:false,“env”:“esp32dev-ble”,“freemem”:88072,“mqttp”:“1883”,“mqtts”:false,“msgprc”:9354,“msgblck”:0,“maxq”:11,“minmem”:13692,“tempc”:55,“freestck”:536,“eth”:false,“rssi”:-40,“SSID”:“MYWIFI”,“BSSID”:“30:34:22:07:8C:66”,“ip”:“192.168.X.XYZ”,“mac”:“A0:B7:65:6B:2A:7C”,“lowpowermode”:-1,“modules”:[“WebUI”,“BT”]}

T: jsonPubl - ON

T: [ OMG->MQTT ] topic: home/OMG_ESP32_BLE/SYStoMQTT msg: {“uptime”:48726,“version”:“v1.7.0”,“disc”:false,“ohdisc”:false,“env”:“esp32dev-ble”,“freemem”:88072,“mqttp”:“1883”,“mqtts”:false,“msgprc”:9354,“msgblck”:0,“maxq”:11,“minmem”:13692,“tempc”:55,“freestck”:536,“eth”:false,“rssi”:-40,“SSID”:“MYWIFI”,“BSSID”:“30:34:22:07:8C:66”,“ip”:“192.168.X.XYZ”,“mac”:“A0:B7:65:6B:2A:7C”,“lowpowermode”:-1,“modules”:[“WebUI”,“BT”]}

T: Dequeue JSON

N: Send on /BTtoMQTT msg {“bleconnect”:true,“interval”:55555,“adaptivescan”:true,“intervalacts”:55555,“intervalcnct”:3600000,“scanduration”:10000,“onlysensors”:false,“randommacs”:false,“hasspresence”:false,“prestopic”:“presence/”,“presuseuuid”:false,“minrssi”:-100,“extDecoderEnable”:false,“extDecoderTopic”:“undecoded”,“filterConnectable”:false,“pubadvdata”:false,“pubuuid4topic”:false,“ignoreWBlist”:false,“presenceawaytimer”:120000,“movingtimer”:60000,“forcepscn”:false,“tskstck”:1360,“crstck”:2852,“enabled”:true,“scnct”:741}

T: jsonPubl - ON

T: [ OMG->MQTT ] topic: home/OMG_ESP32_BLE/BTtoMQTT msg: {“bleconnect”:true,“interval”:55555,“adaptivescan”:true,“intervalacts”:55555,“intervalcnct”:3600000,“scanduration”:10000,“onlysensors”:false,“randommacs”:false,“hasspresence”:false,“prestopic”:“presence/”,“presuseuuid”:false,“minrssi”:-100,“extDecoderEnable”:false,“extDecoderTopic”:“undecoded”,“filterConnectable”:false,“pubadvdata”:false,“pubuuid4topic”:false,“ignoreWBlist”:false,“presenceawaytimer”:120000,“movingtimer”:60000,“forcepscn”:false,“tskstck”:1360,“crstck”:2852,“enabled”:true,“scnct”:741}

T: Dequeue JSON

N: Send on /WebUItoMQTT msg {“displayMetric”:true,“webUISecure”:true,“displayQueue”:0}

T: jsonPubl - ON

T: [ OMG->MQTT ] topic: home/OMG_ESP32_BLE/WebUItoMQTT msg: {“displayMetric”:true,“webUISecure”:true,“displayQueue”:0}

Josh_Grubman · February 26, 2024, 3:43pm

I went to the console and entered the command you suggested. It said this:
N: [ MQTT->OMG ]: {“intervalcnct”:10000}
N: Config intervalcnct unchanged: false

Shortly thereafter I saw this:
Send on /BTtoMQTT msg {“bleconnect”:true,“interval”:55555,“adaptivescan”:true,“intervalacts”:55555,“intervalcnct”:10000,“scanduration”:10000,“onlysensors”:false,“randommacs”:false,“hasspresence”:false,“prestopic”:“presence/”,“presuseuuid”:false,“minrssi”:-100,“extDecoderEnable”:false,“extDecoderTopic”:“undecoded”,“filterConnectable”:false,“pubadvdata”:false,“pubuuid4topic”:false,“ignoreWBlist”:false,“presenceawaytimer”:120000,“movingtimer”:60000,“forcepscn”:false,“tskstck”:2936,“crstck”:3052,“enabled”:true,“scnct”:2}
N: Send on /SYStoMQTT msg {“uptime”:123,“version”:“v1.7.0”,“disc”:true,“ohdisc”:false,“env”:“esp32dev-ble”,“freemem”:88496,“mqttp”:“1883”,“mqtts”:false,“msgprc”:28,“msgblck”:0,“maxq”:4,“minmem”:35876,“tempc”:53.33,“freestck”:104,“eth”:false,“rssi”:-59,“SSID”:“MYWIFI”,“BSSID”:“30:34:22:09:8E:26”,“ip”:“192.168.X.XYZ8”,“mac”:“A0:B7:65:6B:2A:7C”,“lowpowermode”:-1,“modules”:[“WebUI”,“BT”]}
N: Send on /BTtoMQTT msg {“bleconnect”:true,“interval”:55555,“adaptivescan”:true,“intervalacts”:55555,“intervalcnct”:10000,“scanduration”:10000,“onlysensors”:false,“randommacs”:false,“hasspresence”:false,“prestopic”:“presence/”,“presuseuuid”:false,“minrssi”:-100,“extDecoderEnable”:false,“extDecoderTopic”:“undecoded”,“filterConnectable”:false,“pubadvdata”:false,“pubuuid4topic”:false,“ignoreWBlist”:false,“presenceawaytimer”:120000,“movingtimer”:60000,“forcepscn”:false,“tskstck”:2936,“crstck”:3052,“enabled”:true,“scnct”:2}
N: Send on /WebUItoMQTT msg {“displayMetric”:true,“webUISecure”:true,“displayQueue”:0}

Then later I see this with the device mac address:
Device detected: 50:54:7B:22:83:D8
N: Send on /BTtoMQTT/50547B2283D8 msg {“id”:“50:54:7B:22:83:D8”,“name”:“Battery Monitor”,“rssi”:-90,“txpower”:0}

While later I get this:
N: Send on /BTtoMQTT/50547B2283D8 msg {“id”:“50:54:7B:22:83:D8”,“rssi”:-91,“brand”:“GENERIC”,“model”:“BM2 Battery Monitor”,“model_id”:“BM2”,“type”:“BATT”,“batt”:97,“device”:“BM2 Tracker”}

Josh_Grubman · February 26, 2024, 3:49pm

Ok possible lead. Look at the first and last lines in this part of the log. It seems to read the device, then later say connection failed. Is this insightful? Note that I blahblahblah’d the irrelevant MAC addresses.

Send on /BTtoMQTT/50547B2283D8 msg {“id”:“50:54:7B:22:83:D8”,“name”:“Battery Monitor”,“rssi”:-91,“txpower”:0,“brand”:“GENERIC”,“model”:“BM2 Battery Monitor”,“model_id”:“BM2”,“type”:“BATT”,“batt”:97,“device”:“BM2 Tracker”}
N: Device detected: C9:B7:blahblahblah
N: Send on /BTtoMQTT/C9B7blahblahblah
{“id”:“C9:B7:blahblahblah”,“name”:“Nanoleaf A19 6FZ4”,“rssi”:-89}
N: Device detected: CD:CD:blahblahblah
N: Found 26 devices, scan number 9 end
N: BLE Connect begin
E: Connect to: 50:54:7b:22:83:d8 failed

DigiH · February 26, 2024, 4:01pm

So the 10 seconds connection interval change was registered correctly, with OMG now trying to fetch the voltage every 10 seconds. But with the trace logging not showing any trace log entries for the BM2 connection, but only your

So it seems that it already fails at

github.com

1technophile/OpenMQTTGateway/blob/development/main/ZgatewayBLEConnect.ino#L17


      
          #    define convertTemp_CtoF(c) ((c * 1.8) + 32)
          
          extern std::vector<BLEdevice*> devices;
          
          NimBLERemoteCharacteristic* zBLEConnect::getCharacteristic(const NimBLEUUID& service,
                                                                     const NimBLEUUID& characteristic) {
            BLERemoteCharacteristic* pRemoteCharacteristic = nullptr;
            if (!m_pClient) {
              Log.error(F("No BLE client" CR));
            } else if (!m_pClient->isConnected() && !m_pClient->connect()) {
              Log.error(F("Connect to: %s failed" CR), m_pClient->getPeerAddress().toString().c_str());
            } else {
              BLERemoteService* pRemoteService = m_pClient->getService(service);
              if (!pRemoteService) {
                Log.notice(F("Failed to find service UUID: %s" CR), service.toString().c_str());
              } else {
                Log.trace(F("Found service: %s" CR), service.toString().c_str());
                Log.trace(F("Client isConnected, freeHeap: %d" CR), ESP.getFreeHeap());
                pRemoteCharacteristic = pRemoteService->getCharacteristic(characteristic);
                if (!pRemoteCharacteristic) {
                  Log.notice(F("Failed to find characteristic UUID: %s" CR), characteristic.toString().c_str());

as otherwise some of the connection trace log entries, like at least Log.trace(F("Callback from … would have shown up.

github.com

1technophile/OpenMQTTGateway/blob/development/main/ZgatewayBLEConnect.ino#L261


      
                if (ulTaskNotifyTake(pdTRUE, pdMS_TO_TICKS(BLE_CNCT_TIMEOUT)) == pdFALSE) {
                  m_taskHandle = nullptr;
                }
              } else {
                Log.notice(F("Failed registering notification" CR));
              }
            }
          }
          
          /*-----------------------BM2 HANDLING-----------------------*/
          void BM2_connect::notifyCB(NimBLERemoteCharacteristic* pChar, uint8_t* pData, size_t length, bool isNotify) {
            if (m_taskHandle == nullptr) {
              return; // unexpected notification
            }
          
            if (!BTProcessLock) {
              Log.trace(F("Callback from %s characteristic" CR), pChar->getUUID().toString().c_str());
              if (length == 16) {
                Log.trace(F("Device identified creating BLE buffer" CR));
                DynamicJsonDocument BLEdataBuffer(JSON_MSG_BUFFER);
                JsonObject BLEdata = BLEdataBuffer.to<JsonObject>();

Definitely something to look into, if this also affects older BM2 for which this previously worked.

Could you also install the latest development version if this was currently with the 1.7.0 release, just to see that your issues remains the same.

Josh_Grubman · February 26, 2024, 4:06pm

I just flashed it yesterday and the version info says 1.7.0

Which firmware would you suggest? Is it one of the dropdowns on the web-install page? Which one?

Oh wait is this a whole new install page? So I should just use this site and grab the esp32dev-ble again?

DigiH · February 26, 2024, 4:13pm

Yes, this is the development web upload page

But I just heard from a team mate that his BM2 is working fine - with the 1.7.0 release and the latest development - so it really seems that this is something new to your hardware or formware version of the BM2.

Do you have the nRFConnect app on your mobile phone?

With it you could connect to your BM2 to see if at also has the service/characteristic fff0/fff4, which is the one where at least older BM2 models connect to to get the notification of the voltage.

Josh_Grubman · February 26, 2024, 4:21pm

Downloaded the app and walked over to car. I didn’t disable the OMGateway, but was able to see something pop up called “Battery Monitor”, which I assume is the device we want. I do see Services FFF0 before I even try to connect to it.

Does that answer the question?

I was unable to make this app connect to the “Battery Monitor” it found tho.

DigiH · February 26, 2024, 4:33pm

The service FFF0 could have several different characteristics, where only the FFF4 one would be important,

As I don’t have a BM2 myself it’s difficult for me to say if an nRFConnect connection should be possible or not, and if/how the connection characteristic FFF4 might have been reverse engineered thought GATT connections, along with its then required encryption key for correct decryption.

All I know now from your above logging, is that trying to connect to your BM2 also already fails at

…
  } else if (!m_pClient->isConnected() && !m_pClient->connect()) {
              Log.error(F("Connect to: %s failed" CR), m_pClient->getPeerAddress().toString().c_str());
            }
…

when the BM2 connection tries to get

NimBLERemoteCharacteristic* pChar = getCharacteristic(serviceUUID, charUUID);

with serviceUUID/charUUID being fff0/fff4

So this service/char might not be avaialble on your BM2, or could it be possibly different if you de-register your BM2 from the app, disconnect it in the app … so that then a connection might be possible with nRFConnect and/or OMG?

All conjecture from me at this point I’m afraid.

Josh_Grubman · February 26, 2024, 4:42pm

I was able to get a connection to the app you suggested.

The FFF4 does seem to be sending data. Have a look: https://photos.app.goo.gl/QW53umh8oxFU9Lpq9

Edit: Also I seem to have reached a first-day posting limit. Can you shoot me a message at josh.grubman@gmail.com to continue the convo?

Josh_Grubman · February 28, 2024, 5:13pm

I have tried a few things but none of it worked lol. Anyone have any further suggestions here? OMG seems to work the best in that it does connect and grabs some data; it just doesn’t seem to understand how to convert the BM2’s fff4 hex code into an actual voltage reading.