When I turn it off, the app continues to show “Synced” with the last temp reading.
Welcome @Jennifer_S
This is (sort of) expected behaviour - Theengs App, Theengs Gateway and OpenMQTTGateway do send MQTTQ messages or show the values received and decoded.
While generally with MQTT messages from all of these, especially with Theengs Gateway and OpenMQTTGateway, it is the controller’s task to reset ot set to 0 and values after a device has not been received for a certain time. This is usually done with some expire or timeout functionality.
(sort of) above, because this is not really something possible for you to achieve in the Theengs Mobile App.
What you can currently do however is to completely quit/stop the App and start it again, then you should correctly see your IBT-4XS as offline again.
Something we should look into to reset any BBQ thermometer too Offline after a certain timeout period, as this would also allow for a notification if the reception of the BBQ thermometer has been lost for that amount of time, instead of eventually only finding to some well burnt food 
Thanks for bringing this to our attention.