Websocket connection is not closed, when tcp client disconnects

[toewsar]

Some protocols do not keep the connection open. And if they open again they assume to have a clean session.
But because the c8ylp does not disconnect the websocket, the tcp connection on the device side remains open.

Therefore the c8ylp have to close the websocket conenction, if the tcp client disconnects. This would notify the device that the connection is closed and the device would then disconnect the tcp server, too.

I would be happy if you could fix this.

[reubenmiller]

Thanks @toewsar for the ticket. To make the change easier (and to add a test case for it), would you be able to answer the following questions?

1. I think this behaviour changed with the migration from v1 to v2. Could you please verify if it works for you in v1.5.0, and then stops working in v2?
2. Is there an example server/client that we could use to check the handling of this (and use to verify such a change in behaviour)?

[toewsar]

Yes, you can test it. I tried to tunnel the CODESYS IDE <-> CODESYS Gateway.
You need to install CODESYS Development System V3 and on the device you need to install the CODESYS Edge Gateway for Linux.

I'll try to test it today with 1.5.0.

[switschel]

@toewsar This depends in which mode you start the c8ylp. In server mode the c8ylp acts like a server so you have to shutdown it manually to terminate the websocket connection. So if the TCP-Client disconnects you have to make sure to also terminate the c8ylp server if this is desired.

@reubenmiller In 1.5 we had the script mode doing exactly that: Terminate c8ylp when tcp client disconnects. Not sure if we should add a flag to server or we can achieve that with connect command.

[reubenmiller]

@switschel There is already a retry settings which could be set to 0 to disable trying to reestablish the connection, for example:

c8ylp server --reconnects 1 device01

Though maybe even 1 reconnect is too much in this case, but it could help.

[reubenmiller]

Or using the in-built "command" plugin (in v2 only)

c8ylp plugin command --env-file .env --reconnects 1 device01 './runsomething $PORT'

Here is the plugin help

$ c8ylp plugin command -h
Usage: c8ylp plugin command [OPTIONS] DEVICE [REMOTE_COMMANDS]...

  Start once-off proxy and execute a (local) script/command

          DEVICE is the device's external identity
          REMOTE_COMMANDS is the script or command to run after the proxy has been started

      All additional arguments will be passed to the script/command. Use "--"
      before     the additional arguments to prevent the arguments from being
      interpreted by     c8ylp (i.e. to avoid clashes with c8ylp).

  Available ENV variables (use single quotes to prevent expansion):

      DEVICE - external device identity
      PORT   - port number of the local proxy
      C8Y_HOST - Cumulocity host/url

  Example 1: Use scp to copy a file to a device

          c8ylp plugin command --env-file .env device01 \
              -- /usr/bin/scp -P '$PORT' myfile.tar.gz admin@localhost:/tmp

  Example 2: Run a custom script (not included) to copy a file from the device
  to the current folder

          c8ylp plugin command --env-file .env -v device01 ./copyfrom.sh /var/log/dpkg.log ./

[switschel]

@reubenmiller reconnect 0 will set it to infinite reconnects
--reconnects INTEGER RANGE number of reconnects to the Cloud Remote Service. 0 for infinite reconnects [env var: C8YLP_RECONNECTS; default: 5; -1<=x<=10]
Maybe we can use -1 to disable any reconnect and shutdown the c8ylp after first websocket connection?

[reubenmiller]

@switschel Yes that sounds reasonable. Though I still want to make sure that this is the actual issue though, and not some other internal workings with the connection/disconnection of the sockets.

[switschel]

Btw. it is already implemented: https://github.com/SoftwareAG/cumulocity-remote-access-local-proxy/blob/27b7781df2342105f3aa90e2eee6f8d1664a3b88/c8ylp/websocket_client/ws_client.py#L177

@toewsar Can you try if adding '--reconnects -1' is the behavior you expect to have?

[reubenmiller]

Btw. it is already implemented:

https://github.com/SoftwareAG/cumulocity-remote-access-local-proxy/blob/27b7781df2342105f3aa90e2eee6f8d1664a3b88/c8ylp/websocket_client/ws_client.py#L177

@toewsar Can you try if adding '--reconnects -1' is the behavior you expect to have?

That is the right spot, however the logic is inverted. It should be (self.max_retries == -1), as the if statement is to decide if the connection should be terminated. I have made a change on this fix/disable-reconnects branch

You can try out the wip branch which supports the reconnect=-1 properly using:

pip install git+https://github.com/SoftwareAG/cumulocity-remote-access-local-proxy.git@fix/disable-reconnects

c8ylp server --reconnects -1 device01

I noticed that this might produce an other error when trying to write to a closed socket when it disconnects, but it should at least show that it is possible.

[toewsar]

I tried, but without success. The problem ist, that the tcp-server does not forward the close to the websocket client.
See https://github.com/SoftwareAG/cumulocity-remote-access-local-proxy/blob/27b7781df2342105f3aa90e2eee6f8d1664a3b88/c8ylp/tcp_socket/tcp_server.py#L81

[switschel]

@reubenmiller I added a WS stop if the TCP Client is disconnected in your fix branch.
Also I asked myself: Do we really need a reconnect of the web socket client? This should not be needed any longer as the web socket is established on request if a TCP Client is connecting. Reconnecting the ws makes only sense if no TCP Client is connected and the proxy should keep the connection open for any reason (like it was in v1.x). I commented the reconnect out.

[toewsar]

My connection between CODESYS <-> Gateway seems to be stable now.

[reubenmiller]

@switschel I've updated the PR #40 to hide the reconnects option and print out a warning that it is deprecated (so that we don't break any existing scripts).

[reubenmiller]

@toewsar This has been fixed and released in v2.1.2 v2.1.1