Make it easy to deploy a substrate-connect enabled bootnode to Kubernetes

[PierreBesson]

Substrate-connect light clients have been officially announced as ready to use for the general public (https://www.youtube.com/watch?v=TDbTCrDDO2U). However, from an operational point of view, an obstacle to adoption is that to work properly, the light client needs to access a bootnode which exposes it's p2p over websocket (typically --listen-addr /ip4/0.0.0.0/tcp/30444/ws --listen-addr /ip6/::/tcp/30444/ws). Moreover, as browsers will only allow connecting to a secured websocket, you need a reverse proxy in front such as nginx to add a letsencrypt certificate.

I believe it would be valuable to offer an easy way to deploy such a bootnode to kubernetes with the following:

• Option to auto-generate the relevant p2p-ws Ingress and Service on the helm-chart
• Example config showing how to set up an ingress controller and external-dns / cert-manager for automatic certificate management.

ping @tomaka

[tomaka]

To give more context, we're planning on integrating support for WebRTC in Substrate. WebRTC is basically an equivalent to WebSocket, but more suited to our use case.
Eventually you would use --listen-addr .../webrtc/... instead, and you wouldn't need any TLS certificate.
Also, because WebRTC uses UDP, we can make it use the same port as TCP, which makes configuration more simple.

However, the timeline for WebRTC is probably a few months, so it's probably still worth doing a helm chart for WebSocket.

[PierreBesson]

Service part handled in #165 by @bakhtin
As for the p2p-ws Ingress, I'm not sure if it should be part of the node chart or a different chart.

[bakhtin]

I would not include the Ingress part in the chart. The chart is using only standard Kubernetes primitives that enables it to work on different Kubernetes distributions: be it self-hosted or a managed cloud solution. I would leave the chart as generic as possible and allow users to customize it to their needs.

[PierreBesson]

I would not include the Ingress part in the chart. The chart is using only standard Kubernetes primitives that enables it to work on different Kubernetes distributions: be it self-hosted or a managed cloud solution. I would leave the chart as generic as possible and allow users to customize it to their needs.

I agree to leave our chart fully cloud agnostic, however it would be nice to have ingress examples to help third party teams set up connect enabled bootnodes. If they have to do multiple hours/days of research and config tweaking like us to do it, many teams might think it's too complex to set up/maintain.

[PierreBesson]

Also the bootnodes being the first node you set up for your network, it might be good to show some examples directly in the node chart readme. Alternatively this can be discussed in the devops guide

[bakhtin]

I created a separate issue to put some examples of using Ingress: #175

[lazam]

We can create a chart called polkadot-stack which is similar on how kube-prometheus-stack is designed. The polkadot-stack chart will use different charts as it's dependency to easily deploy a full blown solution with auxiliary services such as ingress, polkadot-introspector, substrate-connect, and others. This approach will help us to allow each individual stack to be cloud agnostic but still have a chart to allow the community to deploy a solution with the necessary configs and design.

[PierreBesson]

@lazam this sounds like an excellent idea, maybe it could be done as part as #97, where we could have polkadot-stack and polkadot-parachain-stack.

[dblane-digicatapult]

Would you envisage this being an umbrella chart (and thus using subcharts) or using something like helmfile to create those stacks?

The reason I ask is that using subcharts can offer some major advantages I.E being able to share subchart templated values and labels across the umbrella chart, as well as conditionally template in values based on enablements/env of your subcharts. There is some annoying overhead, however; chart dependencies require frequent version bumping, then helm dependency update and essentially publishing for every new published version of a subchart.

[PierreBesson]

To make it easy to manage a set of "wss bootnodes" we should implement node.perNodeIngress similar to node.perNodeServices, eg.:

perNodeIngress:
   relayP2pIngress:
        enabled: false
   paraP2pIngress:
        enabled: true
        annotation: [ingress  specific annotations, eg traefik/nginx...]
        domain: parity-testnet.parity.io
    apiIngress:
        enabled: false

Then this would create 1 ingress for each node with rules set up like this (eg. para p2p ingress):

    rules:
    - host: my-node-0.<domain>
      http:
        paths:
        - path: /
          pathType: Prefix
          backend:
            service:
              name: my-node-0-para-chain-p2p-ingress
              port:
                number: p2p-port-number

ping @bakhtin

[tomaka]

I'm no longer at Parity so feel free to ignore this comment, but I would suggest not waste time on this.
As soon as WebRTC support is merged (which is unfortunately taking a long time, but I guess should eventually happen), the next step is to deprecate WSS.