GAミント至上主義

Web Monomaniacal Developer.

【解決】 Python3のfirebase_adminでCollectionをon_snapshot()してるとhread-ConsumeBidirectionalStream caught unexpected exception

下記記事でやっていた処理だけど、寝る前に実行して、朝起きるころ見るとエラーを吐いて止まっている。
まだ解決してないけど、メモ。


Raspberry PiでPython3を使ってFirestoreにクエリする - GAミント至上主義

マシンはRaspberry Pi Zero WH。
OSは

pi@raspberrypi:~/led8 $ lsb_release  -a
No LSB modules are available.
Distributor ID: Raspbian
Description:    Raspbian GNU/Linux 9.9 (stretch)
Release:        9.9
Codename:       stretch

関係ありそうなパッケージのバージョンは下記。

firebase-admin==2.17.0
google-api-core==1.13.0
google-api-python-client==1.7.9
google-auth==1.6.3
google-auth-httplib2==0.0.3
google-cloud-core==1.0.2
google-cloud-firestore==1.2.0
google-cloud-storage==1.16.1
google-resumable-media==0.3.2
googleapis-common-protos==1.6.0

googleapis.github.io

エラー出力は下記。

hread-ConsumeBidirectionalStream caught unexpected exception <_Rendezvous of RPC that terminated with:                  
        status = StatusCode.INTERNAL                                                                                     
        details = "Received RST_STREAM with error code 0"                                                                
        debug_error_string = "{"created":"@1562627421.743478058","description":"Error received from peer ipv6:[2404:6800$
4004:801::200a]:443","file":"src/core/lib/surface/call.cc","file_line":1046,"grpc_message":"Received RST_STREAM with err$
r code 0","grpc_status":13}"                                                                                             
> and will exit.                                                                                                         
Traceback (most recent call last):                                                                                       
  File "/usr/local/lib/python3.5/dist-packages/google/api_core/bidi.py", line 633, in _thread_main                       
    response = self._bidi_rpc.recv()                                                                                     
  File "/usr/local/lib/python3.5/dist-packages/google/api_core/bidi.py", line 544, in recv                               
    return self._recoverable(self._recv)                                                                                 
  File "/usr/local/lib/python3.5/dist-packages/google/api_core/bidi.py", line 503, in _recoverable                       
    raise exc                                                                                                            
  File "/usr/local/lib/python3.5/dist-packages/google/api_core/bidi.py", line 493, in _recoverable                       
    return method(*args, **kwargs)                                                                                       
  File "/usr/local/lib/python3.5/dist-packages/google/api_core/bidi.py", line 541, in _recv                              
    return next(call)
  File "/usr/local/lib/python3.5/dist-packages/grpc/_channel.py", line 364, in __next__
    return self._next()
  File "/usr/local/lib/python3.5/dist-packages/grpc/_channel.py", line 358, in _next
    raise self
grpc._channel._Rendezvous: <_Rendezvous of RPC that terminated with:
        status = StatusCode.INTERNAL
        details = "Received RST_STREAM with error code 0"
        debug_error_string = "{"created":"@1562627421.743478058","description":"Error received from peer ipv6:[2404:6800:
4004:801::200a]:443","file":"src/core/lib/surface/call.cc","file_line":1046,"grpc_message":"Received RST_STREAM with erro
r code 0","grpc_status":13}"
>
Exception in thread Thread-OnRpcTerminated:
Traceback (most recent call last):
  File "/usr/lib/python3.5/threading.py", line 914, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.5/threading.py", line 862, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/local/lib/python3.5/dist-packages/google/cloud/firestore_v1/watch.py", line 290, in close
    raise reason
google.api_core.exceptions.InternalServerError: 500 Received RST_STREAM with error code 0

検索するとPub/Subなどでも同じようなエラーを出すようで、Firestore特有の問題ではなさそう。

https://github.com/googleapis/google-cloud-python/issues/4234

Threadingを使っているので、簡単にはtryでExceptionを受け取ることができない。

2019/7/10 追記

Firestore: what possible cause of this exception: InternalServerError: 500 Received RST_STREAM with error code 0 · Issue #282 · firebase/firebase-admin-python · GitHub
こちらのやり取りでwatchオブジェクトの_closedプロパティを見て、再接続をするハックを見つけたので試してみる。
みんなちょうど60分で切断されているらしい。

python - How to detect realtime listener errors in firebase firestore database? - Stack Overflow

ソースを確認したところ、_closed以外にもis_activeというプロパティもあったので、こちらも確認してみる。
google-cloud-python/watch.py at master · googleapis/google-cloud-python · GitHub

# 省略
def main():↲
    try:↲
        watcher = commands.collection_ref.on_snapshot(↲
            update_callback)↲
    except Exception as e:↲
        print(e)↲
        exit()↲
    while True:↲
        print(datetime.now())↲
        sleep(60)↲
↲
        if not watcher.is_active:↲
            print('is not active!!!')↲
            exit()↲
↲
        if watcher._closed:↲
            print('_closed!!!!!')↲
            exit()↲

7/11 解決。

_closedプロパティを確認して、Trueだったら再度開始することで数日以上動き続けている

# 省略
def main():↲
    try:↲
        watcher = collection_ref.on_snapshot(update_callback)↲
    except Exception as e:↲ 
        print(e)↲
        exit()↲
    while True:↲
        # print(datetime.now())↲
        sleep(60)↲
        if watcher._closed:↲
            print('_closed!!!!!')↲
            watcher = collection_ref.on_snapshot(update_callback