Providers
Migrating from Dshackle

Migrating from Dshackle

This guide is for dRPC providers who run Dshackle and move to NodeCore.

The migration has four steps:

  1. Convert your dshackle.yaml into a nodecore.yml
  2. Fix what the converter reports
  3. Run NodeCore next to your Dshackle, on its own port
  4. Check that everything is correct and contact the dRPC team

Dshackle and NodeCore work side by side on your server. Dshackle keeps serving requests all this time, and the dRPC team switches the traffic to NodeCore on its side.

What stays the same#

  • your provider name and the DNS name of your server
  • your SSL certificate and key
  • your authorization keys: the same key files work with NodeCore, there is nothing to generate or to send to us again
  • the ids of your nodes

Before you start#

NodeCore doesn't serve Ripple and Kadena for now. Nodes of these chains have to stay on Dshackle.

An upstream that points to another Dshackle instance is not supported by NodeCore for now either.

The converter skips such upstreams and lists them in its report.

Convert your config#

The converter is a part of the NodeCore repository. To run it you need Go (opens in a new tab) and a checkout of the repository:

git clone https://github.com/drpcorg/nodecore.git
cd nodecore
make convert-dshackle IN=/path/to/dshackle.yaml OUT=/path/to/nodecore.yml

IN is your Dshackle config, OUT is the NodeCore config to write. The converter doesn't change your Dshackle config and doesn't start anything.

For example, this Dshackle config:

version: v1
port: 12448
host: 0.0.0.0
 
tls:
  enabled: true
  server:
    certificate: /path_to_ssl_certs/your_ssl_cert.crt
    key: /path_to_ssl_certs/your_ssl_key.p8.key
 
auth:
  enabled: true
  publicKeyOwner: drpc # always drpc
  server:
    keys:
      provider-private-key: /path_to_keys/private-provider-name.p8.key
      external-public-key: /path_to_keys/public-drpc.pem
 
cluster:
  upstreams:
    - id: ethnode1
      chain: ethereum
      method-groups:
        enabled:
          - trace
          - filter
      connection:
        generic:
          rpc:
            url: "http://example1.com:8545"
          ws:
            url: "ws://example1.com:8545"
    - id: ethnode2
      chain: ethereum
      method-groups:
        enabled:
          - trace
      connection:
        generic:
          rpc:
            url: "http://example2.com:8545"
          ws:
            url: "ws://example2.com:8545"

becomes this NodeCore config:

server:
  grpc-port: 12448
  metrics-port: 8081
  tls:
    enabled: true
    certificate: /path_to_ssl_certs/your_ssl_cert.crt
    key: /path_to_ssl_certs/your_ssl_key.p8.key
  grpc-auth:
    enabled: true
    public-key-owner: drpc
    provider-private-key-path: /path_to_keys/private-provider-name.p8.key
    external-public-key-path: /path_to_keys/public-drpc.pem
upstream-config:
  upstreams:
    - id: ethnode1
      chain: ethereum
      connectors:
        - type: json-rpc
          url: http://example1.com:8545
        - type: websocket
          url: ws://example1.com:8545
      methods:
        enable:
          - trace
          - filter
    - id: ethnode2
      chain: ethereum
      connectors:
        - type: json-rpc
          url: http://example2.com:8545
        - type: websocket
          url: ws://example2.com:8545
      methods:
        enable:
          - trace
  balancing-strategy: base
  mode: strict

The converter always writes mode: strict and balancing-strategy: base. These two settings are required for dRPC providers, don't remove them.

Read the report#

Together with the config, the converter prints a report of everything it left out or changed. For the example above it looks like this:

converted 2 of 2 upstreams -> /path/to/nodecore.yml

needs manual action:
  auth.server.keys: nodecore does not start unless both key files exist; make them available at these paths where nodecore runs
  monitoring.prometheus.bind: dshackle listens on 127.0.0.1; nodecore listens on all interfaces, so port 8081 becomes reachable from outside

The report has up to three groups:

GroupMeaning
skipped upstreamsThe node is not in the new config at all. The line gives the reason
needs manual actionSomething you have to fix or decide before you use the new config
not carried overA Dshackle setting that has no place in a NodeCore config. There is nothing to fix

Go through the skipped upstreams and needs manual action lines one by one. The most common ones:

  • Key and certificate files. The converter copies the paths, not the files. The same files work with NodeCore, they only have to be available at these paths where NodeCore runs.
  • ${VAR} placeholders. NodeCore doesn't support placeholders. Replace each of them with the real value in nodecore.yml.
  • Ports that become reachable from outside. With Docker a port is reachable only if you publish it, see the NodeCore setup instructions.

If your new config has no server.metrics-port, add it, for example metrics-port: 8081. You need the metrics to check your nodes.

The complete list of what is converted, what is not, and what every report line means is in the converter documentation (opens in a new tab).

Run NodeCore next to Dshackle#

Start NodeCore next to your running Dshackle. Dshackle is not touched and keeps serving requests.

Add NodeCore to your docker-compose.yaml, or create a new one:

services:
  nodecore:
    container_name: nodecore
    image: drpcorg/nodecore:latest
    environment:
      - NODECORE_CONFIG_PATH=/etc/nodecore/nodecore.yml
    volumes:
      - "/path_to_config:/etc/nodecore"
      - "/path_to_ssl_certs:/path_to_ssl_certs"
      - "/path_to_keys:/path_to_keys"
    ports:
      - "12449:12448"
      - "127.0.0.1:9093:8081"

You can start it by docker compose up -d nodecore command.

A few things to keep in mind:

  • /path_to_config is the directory with the new nodecore.yml.
  • In 12449:12448 the right part is server.grpc-port from your new config, it is the same port Dshackle uses. The left part is the port dRPC will use to connect to NodeCore. It must be a free port of your server, because Dshackle keeps its own one. Make sure this port is reachable for dRPC the same way the Dshackle port is.
  • In 127.0.0.1:9093:8081 the right part is server.metrics-port from your new config, and the left part is any free port of your machine.
  • NodeCore runs inside the container as a non-root user with uid 65532, while Dshackle runs as root. The new config, the keys and the SSL certificate and key must be readable by this user, for example chown 65532 /path_to_config/nodecore.yml. Dshackle can still read these files after that. The converter writes nodecore.yml readable by its owner only, so don't forget about this file too.

If you run Dshackle and NodeCore on the same machine without Docker, NodeCore can't open the port that Dshackle already uses. Set another port in server.grpc-port of your new config instead.

If your Dshackle works behind a reverse proxy, add one more port to the proxy and pass it to NodeCore the same way, see NodeCore behind a reverse proxy.

All the details of the NodeCore deployment are in the NodeCore setup instructions.

Check and contact us#

Check the logs and the metrics as it's described in Check that it works:

docker logs nodecore
curl -s localhost:9093/metrics | grep nodecore_upstream_availability_status

Every node from your new config must be in the list with the value 1:

nodecore_upstream_availability_status{chain="ethereum",upstream="ethnode1"} 1
nodecore_upstream_availability_status{chain="ethereum",upstream="ethnode2"} 1

When everything is correct, contact the dRPC team and tell us the port of your NodeCore (12449 in the example above).

Keep both Dshackle and NodeCore running after that. The dRPC team connects to your NodeCore and switches the traffic on its side.

What behaves differently#

  • Methods. NodeCore detects which methods each node serves, including the trace, debug and filter methods, so there is no need to enable method groups by hand. The groups you enabled in Dshackle are carried over into methods.enable, and NodeCore keeps such methods enabled even if it finds out that the node doesn't serve them. You can remove these lines and let NodeCore decide.
  • Placeholders. NodeCore doesn't replace ${VAR} placeholders with environment variables.
  • User. NodeCore runs as a non-root user with uid 65532.
  • HTTP port. NodeCore opens an HTTP port (9090 by default) that serves JSON-RPC requests without authorization. Do not publish it, see the NodeCore setup instructions.
  • Metrics. All metrics have new names that start with nodecore_, so your dashboards and alerts have to be updated. See Prometheus metrics (opens in a new tab).
  • Health checks. NodeCore has its own health endpoints (opens in a new tab) on port 9096: /health, /ready and /status.
  • Reload. It works the same way, with the HUP signal, and applies changes of the list of nodes. See Reload config.