Error accessing device after a few hours working

[vjbutt]

I am getting the status from a WiFi Dual Meter every 10s using tinytuya. Everything works fine for a few hours. Then I start getting the following error:

Unexpected error: <class 'KeyError'>

I can still read the meter data using the Smart Life app whilst I am getting this error. However, in order to get the status working again using tinytuya, I have to power cycle the WiFi Dual Meter. Any idea what could be wrong?

[jasonacox]

Hi @vjbutt can you detail or link about the Tuya device (WiFi Dual Meter)?

I have had a few older devices that would stop responding to the local network requests but would continue to work with the cloud (Smart Life). Power cycling them was the only that that would fix them.

It would be interesting to see if you could run a monitoring loop with a persistent connect to see if that helps. See https://github.com/jasonacox/tinytuya/blob/master/examples/monitor.py as an example.

[vjbutt]

@jasonacox thanks for your suggestions, I will try that plus a few other things. It seems that I have bought one of these older devices :(.

The error that I reported above is wrong (that was actually due to a missing DP ID in one of the returned data). The actual error that I am getting when the WiFi Dual Meter gives up on the local connection is:

{'Error': 'Check device key or version', 'Err': '914', 'Payload': None}

The device key is definitely correct, because when I power cycle the device it works. I will next check the version. I am currently setting this to 3.4, as is reported by the device, but maybe I should set this to 3.3.

[vjbutt]

@jasonacox I have tried setting the version 3.3, but tinytuya does not even manage to get the status. So the correct value to set is actually 3.4.

I still need to find some time to check with a monitoring loop over a persistent connection.

BTW the device is this one - https://www.amazon.co.uk/Energy-Electricity-Monitor-Control-Management/dp/B0CDXF3J1K/ref=cm_cr_arp_d_product_top?ie=UTF8

Haven't managed to find a link to the manufacturer though :-(

[uzlonewolf]

I have some v3.4 devices which do this, and my suspicion is memory corruption within the device is causing it to "forget" what the correct local key is. How often it happens is pretty random; one day it may do it 3 times that day, and then it will be fine for a month. I haven't found a solution other than rebooting it.

[vjbutt]

@uzlonewolf that was exactly what I was thinking, that somehow there must be some memory corruption. However, today I was surprised to note that the device started to respond to status updates again after about 1 hour of giving {'Error': 'Check device key or version', 'Err': '914', 'Payload': None} (without necessitating a power cycle). So maybe it is not memory corruption.

Furthermore, just before it started to respond correctly again to status queries, it returned the following status:

{'protocol': 4, 't': 20, 'data': {'dps': {'2': 1887}}, 'dps': {'2': 1887}}

It seems to me that the device is somehow switching to protocol 4. Why should it do this is beyond me. The problem is that tinytuya does not support version 4, so maybe those error messages were being generated because the device switched to version 4. Note that these protocol 4 messages are being received also in between correctly received status updates. They seem to be returned quite at random.

One other thing to note is that the time stamp ('t':20) returned in the above status seems to indicate that the device had rebooted. From previous protocol 4 status messages, it is clear that this is a Unix timestamp. So a timestamp of 20 seems to indicate that not even the real-time clock on board of the device had yet been updated before this status was returned. This may indicate a more serious case of memory corruption.

Will keep monitoring.

[uzlonewolf]

The 'protocol' in the payload does not do anything as far as I can tell and is only present in packets which contain a "protocol version" header directly before the payload. Devices set it to 4 while the SmartLife app sets it to 5.

[vjbutt]

@uzlonewolf, indeed that is now my impression as well. The presence of these protocol 4 packets increases drastically when I've set set_socketPersistent(True). They basically contain a very small subset of the DPS data that eventually follows in a full data payload.

Interestingly, setting a persistent connection also seems to have solved my problem in that the device is still alive and supplying "valid" data after more than 24 hours of continuous operation. However, it has now created a new problem. The status command is returning repeated payloads. I still see the measurements being updated on the Smart Life app, but not in tinytuya. As a consequence the returned data is practically useless. I have tried to force the update of the DPS using the function updatedps() but this does not seem to resolve the issue.

Why would a persistent connection cause repeated payloads and not return the actual data being measured? Note that I am only reading the status every 10s, which is compatible with the value set in the report rate control.

[Dronec]

Got the same situation with this device:
https://www.ebay.com.au/itm/355760971428
Querying it every 5 min, after approximately 12 hours it stops responding with
"Check device key or version"

[dwmw2]

I have a similar problem with a Ciarra cooker hood. I was connected to it and it was happily reporting status, and suddenly it started returning the same error. I was running this:

#!/bin/python3

import tinytuya
d = tinytuya.OutletDevice(dev_id='xxx', address='172.31.31.209', local_key='xxx', version=3.4)


data = d.status()
print('Received Payload: %r' % data)
while (True):
      data = d.receive()
      print('Received Payload: %r' % data)
      if not data:
        print(" > Send Heartbeat Ping < ")
        d.heartbeat()
      print(" > Request Update for DPS indexes 18, 19 and 20 < ")
      result = d.updatedps([4, 5, 6, 18, 19, 20])
      data = d.status()
      print('Received Payload: %r' % data)

.. and it just stopped:

$ python3 hood.py 
Received Payload: {'dps': {'1': False, '2': '1', '6': False, '12': '0', '15': 66, '104': 'Level_1'}}
Received Payload: None
 > Send Heartbeat Ping < 
 > Request Update for DPS indexes 18, 19 and 20 < 
Received Payload: {'dps': {'1': False, '2': '1', '6': False, '12': '0', '15': 66, '104': 'Level_1'}}
Received Payload: None
 > Send Heartbeat Ping < 
 > Request Update for DPS indexes 18, 19 and 20 < 
Received Payload: {'dps': {'1': False, '2': '1', '6': False, '12': '0', '15': 66, '104': 'Level_1'}}
Received Payload: None
 > Send Heartbeat Ping < 
 > Request Update for DPS indexes 18, 19 and 20 < 
Received Payload: {'dps': {'1': False, '2': '1', '6': False, '12': '0', '15': 66, '104': 'Level_1'}}
Received Payload: None
 > Send Heartbeat Ping < 
 > Request Update for DPS indexes 18, 19 and 20 < 
Received Payload: {'Error': 'Check device key or version', 'Err': '914', 'Payload': None}
Received Payload: {'Error': 'Check device key or version', 'Err': '914', 'Payload': None}
 > Request Update for DPS indexes 18, 19 and 20 < 
Received Payload: {'Error': 'Check device key or version', 'Err': '914', 'Payload': None}
Received Payload: {'Error': 'Check device key or version', 'Err': '914', 'Payload': None}
 > Request Update for DPS indexes 18, 19 and 20 < 
Received Payload: {'Error': 'Check device key or version', 'Err': '914', 'Payload': None}
Received Payload: {'Error': 'Check device key or version', 'Err': '914', 'Payload': None}
 > Request Update for DPS indexes 18, 19 and 20 < 

Now it reports the same error every time I try to connect. The Android app was working, and a power cycle seems to have fixed it for tinytuya too.

[Dronec]

Android app is working via the cloud, which doesn't seem to be affected by this issue. I'm starting thinking that emulating the cloud locally might be the way forward https://github.com/kueblc/mocktuyacloud

[jasonacox]

@dwmw2 - If you do not send d.updatedps([4, 5, 6, 18, 19, 20]) does it still occur? That has always been a bit of a hack to try to get these devices to update those DPS values. I wonder if it is tripping some vulnerability in the firmware (e.g. memory leak) that eventually causes it to become unstable.

@Dronec - Yes, valid point. These devices were not designed for local access as we have seen again and again. They are designed for the cloud so that tends to be the most polished and reliable aspect. I think that project you reference has an interesting idea, but doesn't look like it has been touched in 5 years.

[Dronec]

@jasonacox I've tried that without sending updates, same result.

I'm going to play with that project over the weekend, just need to wait for a new device

[jasonacox]

I guess one experiment would be to remove heartbeat to see if that works. There is a high chance that it would hit heartbeat timeout and disconnect the TCP session.

And another experiment would be to stop sending anything at all and test to see if it is just a "running time" related issue vs some bug in their local API handler that traffic aggrevates. You are still having to test it every x minutes/hours, but it would at least tel lus that this is API usage related vs a firmware fundamental flaw.

Another experiment would be to write a timer in your loop to fully disconnect (tear down TCP link) and reconnect. Theory would be that whatever resource leakage that is causing the device to become unresponsive could be averted if it still does a cleanup after a connection close. Of course that also means that once it becomes unresponsive, it is no longer able to recover itself so picking the right timer for the reset before that threshold would be important.

[oeboema]

Same here, with this heater: https://eurom.nl/en/product/permanent-heating/bathroom-heaters-permanent/sani-bathroom-radiator-1000-wifi-white/

Stops responding after a while (several hours), and complains about the key not being valid. 2 Smartplugs that I'm polling work fine. A power cycle for the heater does resolve the situation (for a while).

[jasonacox]

Hi @oeboema can you share what you are using to access the device (script or just scanning)?