Request for Apple Watch, iPhone and iPad user feedback

[DigiH]

Hi all,

With our recent release of Theengs Gateway version 1.3.0 we have implemented, among other things, the detection of Apple Watch, iPhone and iPad for local Buethooth device presence tracking, as well as a security relevant unlocked property for these devices.

The Theengs Gateway installation is possible in several different ways, for Home Assistant users the most convenient is the Gateway HA Add-on. Auto-discovery is also new in this release.

To use any of the Apple devices you will need to enter the Identity Bluetooth MAC address and Identity Resolving Key for your devices, with instructions on how to do this at

Use | Theengs: BLE RF LoRa applications and MQTT gateways

There are a few things to consider with the unlocked property:
• The Apple Watch continuously reports unlocked true when worn on a user’s wrist and unlocked with its Passcode.
• For iPhone and iPad however unlocked really means - fully unlocked and with recent user interaction. As soon as there is no more user interaction it will go back into the a locked state, also indicated by the screen dimming before a complete lock, depending on the auto-lock setting on the device.
• Also an unlocked lock screen is not enough for an unlocked property change, it requires a complete unlocking of the device by either Passcode, Face ID or Touch ID.
• A phone or FaceTime call will also put the property into a locked state.

For Home Assistant users, after entering the Identity MAC address and IRK, with auto-discovered and added to a dashboard it will look something like this

While users of other controllers can use presence tracking by the now static MAC address and get the unlocked property boolean directly from the MQTT messages, like

{"id": “AA:BB:CC:DD:EE:FF”, "rssi": -64, "brand": "Apple", "model": "Apple iPhone/iPad", "model_id": "APPLEDEVICE", "type": "TRACK", "unlocked": false}

{"id": “FF:EE:DD:CC:BB:AA”, "rssi": -63, "brand": "Apple", "model": "Apple Watch", "model_id": "APPLEWATCH", "type": "BODY", "unlocked": true}

As there have been some issues with the correct unlocked property reporting on a device during internal testing, we would appreciate any feedback from Apple device users on this implementation, including hardware and OS version details. Obviously any other comments and suggestions are welcome.

We hope you enjoy these new features

[koobi]

Hi DigiH,
This would be a great feature! Thank you so much for working on this.

I see the MQTT messages from my iPhone (same as in your example). I am able to detect presence/absence based on the rssi field. However, the “unlocked” field stays false no matter what I try. I wonder if broadcasting this information is contingent on some iOS settings.

iOS Version: 17.4.1
Model Name: iPhone 13 (MLML3LL/A)

[DigiH]

Hi @koobi

This is already implemented in Theengs Gateway, as described above.

While the unlocked feature is working correctly at all times for the Apple Watch, unfortunately there are some iPhone users for whom unlocked isn’t working correctly.

I’m pretty sure it has to do with some general setting, but so far we haven’t been able to pin down exactly which setting it might be. The next thing I would ask you, or anyone else with this issue, to do is to transfer any file from the iPhone to a Mac (or another iPhone/iPad) via AirDrop - does the unlocked feature behave correctly after that?

For all the iPhones I have physical access to the unlocked feature is working as expected, so it’s a currently hard to pin down this issue.

@koobi

I am able to detect presence/absence based on the rssi field.

May I ask which controller you are using that you’re using the RSSI field for presence?

[koobi]

I interpret the raw MQTT messages directly in Node-RED.

[DigiH]

For non-HA-auto-discovery compatible controllers you might want to make the following changes to the Gateway configuration file theengsgw.conf in your home directory

"discovery": 0,
…
"general_presence": 1,

then restart Gateway.

This will then not create any superfluous HA discovery messages retained on your MQTT broker (you can delete any already existing ones through MQTT Explorer) but will also create additional "presence":"present" and "presence":"absent" JSON entries whenever the iPhone is coming within or leaving (after "tracker_timeout" ) Gateway’s Bluetooth range.

This still doesn’t get to the bottom of the unlocked issue however ;(

[koobi]

Thank you for the recommendation. I now see the presence field. For the records, this is how I start the Gateway Docker container:

docker run --restart=always -d -it  \
    --network host \
    -e MQTT_HOST=192.168.29.211 \
    -e IDENTITIES="{\"XX:XX:XX:XX:XX:XX\": \"XXXXXXXXXXXXXXXXX==\"}" \
    -e PUBLISH_ALL=true \
    -e GENERAL_PRESENCE=true \
    -e DISCOVERY=false \
    -e SCANNING_MODE=active \
    -v /var/run/dbus:/var/run/dbus \
    --name TheengsGateway \
    theengs/gateway

I airdropped a file to a MacBook and then tried again. Tried several times. Still only false.

[DigiH]

@koobi

For the records, this is how I start the Gateway Docker container:

This looks fine, you could also set PUBLISH_ALL=false, unless you also want to catch and publish broadcasts from devices for which we don’t have any decodings.

@koobi

I airdropped a file to a MacBook and then tried again. Tried several times. Still only false.

We can tick that off then for not being the setting affecting the unlocked property please keep trying with other settings you think might affect the continuity Nearby Info broadcast format. Unfortunately this is proving to be really elusive.

[DigiH]

@DigiH

Unfortunately this is proving to be really elusive.

Could you try streaming some music from your iPhone to a Bluetooth speaker?

Does this change anything with the unlock property for you?

Or possibly even better, if you have any AirPlay compatible device like HomePod, Apple TV or other, could you stream some music to them, and see how the unlocked property behaves afterwards.

[koobi]

I tried both AirPlay and Bluetooth speaker.
AirPlay changed nothing.
Playing on the Bluetooth speaker turned the device into an iBeacon with the same id:

{"id":"XX:XX:XX:XX:XX:XX","rssi":-61,"brand":"GENERIC","model":"iBeacon","model_id":"IBEACON","type":"BCON","mfid":"4c00","uuid":"XXXXXXXXXX","major":13596,"minor":1978,"volt":10.2}

The Gateway even reported “absent” after a while. It went back to normal operation a short while after I disconnected from the speaker.

[Johnnybegoode]

Is there any possibility that this could be brought into the Openmqtt gateway?

(Theengs not available where I am)

[DigiH]

We’ll be looking into also porting this to OpenMQTTGateway, but there is no final decision or time frame yet.

[koobi]

I added a 5-series Apple Watch today. It shows as type “BODY” compared to the iPhone, which shows as type “TRACK”.

Similar to the iPhone, the Apple Watch never shows unlocked true. It shows as unlocked false, whether it is worn on the wrist or not, unlocked or locked. Perhaps there is some account-wide setting that blocks this feature?

WatchOS Version: 10.3.1 (21S651)
Model Name: A2094

[DigiH]

Correct, the Apple Watch had type body, f being a wearable.

That the unlocked property isn’t working for you is very strange, it’s the first report I’m hearing this

Could you start Theengs Gateway with

 -padv PUBLISH_ADVDATA, --publish_advdata PUBLISH_ADVDATA
                        Publish advertising and advanced data (1) or not (0)
                        (default: 0)

set to 1, and then post an MQTT message for both the Watch being locked and unlocked?

Also, is the Apple Watch linked to the previously mentioned iPhone?

Any particular reason the Apple Watch is still on watchOS 10.3.1? Although I don’t recall any issues with that version at the beginning of the year myself.

Hopefully the MQTT messages will give more insight into what might be the issue.

[koobi]

I added -e PUBLISH_ADVDATA=true \ to the Docker command, but I don’t see a change in the MQTT messages. I see the option in the logs, though:

Creating config at /root/theengsgw.conf ...
{
    "host": "192.168.29.211",
    "pass": "",
    "user": "",
    "port": 1883,
    "publish_topic": "home/TheengsGateway/BTtoMQTT",
    "subscribe_topic": "home/+/BTtoMQTT/undecoded",
    "presence_topic": "home/presence/TheengsGateway",
    "presence": false,
    "general_presence": true,
    "publish_all": true,
    "publish_advdata": true,
    "ble_scan_time": 60,
    "ble_time_between_scans": 60,
    "tracker_timeout": 120,
    "log_level": "DEBUG",
    "lwt_topic": "home/TheengsGateway/LWT",
    "discovery": false,
    "hass_discovery": true,
    "discovery_topic": "homeassistant/sensor",
    "discovery_device_name": "TheengsGateway",
    "discovery_filter": "[IBEACON]",
    "scanning_mode": "active",
    "adapter": "hci0",
    "time_sync": "[]",
    "time_format": "0",
    "ble": true,
    "enable_tls": false,
    "enable_websocket": false
,    "identities": {"XXXXXXX": "XXXXX==", "XXXXX": "XXXXXX=="}
}

There is also no difference in the MQTT messages between locked and unlocked.

Locked, screen off, not on wrist:

{"topic":"home/TheengsGateway/BTtoMQTT/XXXXXXXXXXXX","payload":{"manufacturerdata":"XXXXXXXXXXXXXXX","id":"XX:XX:XX:XX:XX:XX","rssi":-83,"brand":"Apple","model":"Apple Watch","model_id":"APPLEWATCH","type":"BODY","track":true,"unlocked":false},"qos":0,"retain":false,"_topic":"home/TheengsGateway/BTtoMQTT/XXXXXXXXXXXX","_msgid":"f08cd75a9491e026"}

Unlocked, on wrist, screen on, interacted:

{"topic":"home/TheengsGateway/BTtoMQTT/XXXXXXXXXXXX","payload":{"manufacturerdata":"XXXXXXXXXXXXXXX","id":"XX:XX:XX:XX:XX:XX","rssi":-74,"brand":"Apple","model":"Apple Watch","model_id":"APPLEWATCH","type":"BODY","track":true,"unlocked":false},"qos":0,"retain":false,"_topic":"home/TheengsGateway/BTtoMQTT/XXXXXXXXXXXX","_msgid":"6bae01cdf5421ab7"}

The Apple Watch is linked to the same Family account as the previously mentioned iPhone, but operated independently from the iPhone via its own SIM.

No particular reason the watchOS is on this version. I’ll try to update and see if that changes anything.

[koobi]

I tried with an iPad Pro (4th Generation; MY2H2LL/A), iPadOS Version 16.2, linked to the same Apple account.

Same issue - stays false no matter what.

I realize this device also needs to be updated and I will try again once the update has completed.

[DigiH]

@koobi

but I don’t see a change in the MQTT messages.

There is the manufacturerdata included now, which I wanted to see, but you have obfuscated it

"manufacturerdata":"XXXXXXXXXXXXXXX"

Don’t worry, there is nothing identifiable within the manufacturerdata, but it could help in trying to pin down what your issue is. Please keep anonymising the ID/MAC address.

Also could you set your "ble_scan_time" and "ble_time_between_scans" both to the default 5 seconds, at least just this testing. In general though, the actual "ble_scan_time" doesn’t really require to be more than 10 seconds, as the "ble_time_between_scans" can be adjusted for less frequent scanning.

[koobi]

The Watch produces the same manufacturerdata whether unlocked or not:

• Locked, not on wrist, screen off: 4c001005091892b28d
• Unlocked, on wrist, screen on: 4c001005091892b28d

This is still on 10.3.1.

[DigiH]

@koobi

The Watch produces the same manufacturerdata whether unlocked or not:

That is very strange. Can you confirm that Wrist Detection is active for the Apple Watch and that a Passcode is set and required for unlocking.?