Dshackle setup instructions

Dshackle is a proxy used by dRPC as an entry point for your provider. It allows you to aggregate multiple nodes under one provider and to route JSON-RPC requests to specific nodes based on their capabilities or special requirements (e.g., trace calls).

Provider name#

Your provider name is a unique identifier for your nodes in dRPC. The convention is to name it using your-company-name + any-suffix.

Install Dshackle#

The easiest way to install Dshackle is by using the docker image (opens in a new tab).

Docker-compose example#

Create docker-compose.yaml file with the following content:

services:
  dshackle:
    container_name: dshackle
    image: drpcorg/dshackle:latest
    volumes:
      - "/path_to_config/dshackle.yaml:/etc/dshackle/dshackle.yaml"
      - "/path_to_ssl_certs:/path_to_ssl_certs"
      - "/path_to_keys:/path_to_keys"
    ports:
      - "12448:12448"

You can start it by docker compose up -d command.

Minimal provider config#

This configuration defines the simplest possible provider with two nodes that will serve requests. You can add any number of additional nodes as needed.

Note: We expect you to set up your Dshackle instance with TLS enabled and a valid SSL certificate.

ATTENTION: Connect your nodes to Dshackle ONLY directly, without any load balancers in between. Using a load balancer between your nodes and Dshackle can lead to errors and inconsistent node states. Dshackle acts as a load balancer itself.

version: v1
port: 12448
host: 0.0.0.0
 
tls:
  enabled: true
  server:
    certificate: /path_to_ssl_certs/your_ssl_cert.crt
    key: /path_to_ssl_certs/your_ssl_key.p8.key
 
auth:
  enabled: true
  publicKeyOwner: drpc # always drpc
  server:
    keys:
      provider-private-key: /path_to_keys/private-provider-name.p8.key
      external-public-key: /path_to_keys/public-drpc.pem
 
cluster:
  upstreams:
    - id: ethnode1
      chain: ethereum
      method-groups:
        enabled:
          - trace
          - filter # only if you are running nethermind or erigon node
      connection:
        generic:
          rpc:
            url: "http://example1.com:8545"
          ws:
            url: "ws://example1.com:8545"
    - id: ethnode2
      chain: ethereum
      method-groups:
        enabled:
          - trace
      connection:
        generic:
          rpc:
            url: "http://example2.com:8545"
          ws:
            url: "ws://example2.com:8545"

Archive nodes#

Dshackle automatically detects the node type (full or archive). If you have any concerns about how your node was detected on the dRPC side, please reach out to the dRPC developers in your dedicated Discord channel.

Filter nodes#

There is a group of methods that allow you to create custom filters:

eth_newFilter
eth_newBlockFilter
eth_newPendingTransactionFilter
eth_getFilterChanges
eth_getFilterLogs
eth_uninstallFilter

These methods are quite tricky to support because they require something like "sticky sessions" for nodes, which is implemented in dRPC. However, not all execution clients support all of them. Currently, we have tested and confirmed that everything works well with the following:

Nethermind
Erigon
Geth works, but it is very slow for eth_getFilterLogs, so we do not recommend using it for those requests

If you're confident that all the methods mentioned above are working correctly on your node, please mark it accordingly in the configuration:

id: ethnode1
chain: ethereum
method-groups:
  enabled:
    - filter 
connection:
  generic:
    rpc:
      url: "http://example1.com:8545"
    ws:
      url: "ws://example1.com:8545"

This way, dRPC will know it can route filter requests to your Dshackle server, which will handle routing those requests to your marked nodes.

Arbitrum and Optimism redirects#

As a client, if you try to ask Arbitrum node something like eth_call with a block tag before Nitro upgrade, your Arbitrum Nitro node won’t be able to perform this request on its own, and there is a redirect feature (opens in a new tab) that will redirect this request to an Arbitrum classic node.

Basically, the same happened (opens in a new tab) with Optimism Bedrock upgrade.

On our side, we want to distinguish providers who can serve requests from historical nodes, so there are two separate labels:

has_pre_bedrock_redirect: true // optimism 
has_pre_nitro_redirect: true // arbitrum

If you have Arbitrum/Optimism nodes which setup to handle historical requests, don’t forget to add a corresponding label to this node; otherwise, dRPC won’t route those requests to your provider.

Full configuration reference#

You can find detailed documentation on all available configuration options here (opens in a new tab).

Reload config#

You can change all configurations under cluster in your dshackle.yaml and then reload it in order to pick them up without dshackle restart.

You can do it via sending a kill command with HUP signal to your container, for example docker kill --signal SIGHUP ${containerId}. And then you will see the reloading logs.

Attention: your volume in docker with dshackle.yaml must not be with ro modifier otherwise reload config doesn't work.

Authorization#

Our new authorization algorithm is based on private/public key pairs and JWT tokens. We hold our private key and your public key, while you retain your own private key and our public key. Using these keys, both sides generate JWT tokens, verify their validity, and if confirmed, authorization is successfully established.

Currently, we support only RSA keys. You can use the following script to generate a key pair, specifying the key suffix as the first argument:

rsa-key-generator.sh:

#!/bin/sh
if [ -z "$1" ]
  then
    echo "Please, specify key suffix"
    exit 1
fi
 
echo "Generating RSA key pair"
 
openssl genrsa -out private-$1.pem 2048
openssl rsa -in private-$1.pem -pubout -out public-$1.pem
openssl pkcs8 -topk8 -inform PEM -outform PEM -in private-$1.pem -out private-$1.p8.key -nocrypt
rm private-$1.pem
 
echo "Key pair is generated"

For example, if you want to generate your pair with the provider-name suffix, you would execute ./rsa-key-generator.sh provider-name. After that, you will receive a pair of keys: a private key named private-provider-name.p8.key and a public key named public-provider-name.pem. Please share your public key with us. You can share it in your dedicated channel on our Discord server.

To enable authentication, add the following configuration to your dshackle.yaml:

auth:
  enabled: true
  publicKeyOwner: drpc # always drpc
  server:
    keys:
      provider-private-key: /path_to_keys/private-provider-name.p8.key
      external-public-key: /path_to_keys/public-drpc.pem

external-public-key is the dRPC public key public-drpc.pem:

Don't forget to add an additional volume (if it's necessary) with path_to_keys path.

Attention: Using this form of authorization does not imply that you can remove the TLS section from your configuration without taking further action. Your connection must remain secure. You can use your own SSL certificates (e.g., Let's Encrypt), or you can remove the TLS configuration and utilize TLS termination on your reverse proxy.

For example, you can add the following configuration for TLS with your own SSL certificate and key

tls:
  enabled: true
  server:
    certificate: /path_to_ssl_certs/your_ssl_cert.crt
    key: /path_to_ssl_certs/your_ssl_key.p8.key

Dshackle behind a reverse proxy

Dshackle can work behind a reverse proxy. You need to terminate SSL on your reverse proxy and run Dshackle with TLS disabled.

Example of a docker-compose configuration with Traefik that proxies port 12448 to 12448 on Dshackle:

version: '3'
services:
  traefik:
    image: traefik:latest
    ports:
      - "443:443"
      - "12448:12448"
    command:
      - "--providers.docker=true"
      - "--providers.docker.exposedbydefault=false"
      - "--entrypoints.websecure.address=:443"
      - "--entryPoints.grpc.address=:12448"
      - "--certificatesresolvers.myresolver.acme.tlschallenge=true"
      - "--certificatesresolvers.myresolver.acme.email=$MAIL"
      - "--certificatesresolvers.myresolver.acme.storage=/letsencrypt/acme.json"
    volumes:
      - "./traefik/letsencrypt:/letsencrypt"
      - "/var/run/docker.sock:/var/run/docker.sock:ro"
  dshackle:
    image: drpcorg/dshackle:latest
    volumes:
      - "/path_to_config/dshackle.yaml:/etc/dshackle/dshackle.yaml"
      - "/path_to_keys:/path_to_keys"
    labels:
      - "traefik.enable=true"
      - "traefik.http.services.drpc.loadbalancer.server.scheme=h2c"
      - "traefik.http.services.drpc.loadbalancer.server.port=12448"
      - "traefik.http.routers.drpc.entrypoints=grpc"
      - "traefik.http.routers.drpc.tls.certresolver=myresolver"
      - "traefik.http.routers.drpc.rule=Host(`$DOMAIN`)"

Example of a Nginx configuration:

location / {
    set $host_grpc_pass "grpc://<dshackle-server-address>:<dshackle-server-port>";
    grpc_pass $host_grpc_pass;
}

Example of a ingress-nginx configuration:

metadata:
  annotations:
    nginx.ingress.kubernetes.io/backend-protocol: GRPC

Creating dRPC account#

Providers can use dRPC for free, as long as they use it to query their own infrastructure. The benefits include:

A single interface for all your endpoints and keys
Detailed statistics
Automatic fallback mechanism

Also, we need to associate your account with a specific provider in our dRPC configuration. For now, this is done manually. The association is based on your Team ID.

Register on drpc.org

Go to https://drpc.org (opens in a new tab)
Select your preferred authorization method (MetaMask, email, etc)
Go to Settings -> Team
Copy your Team ID and send it to us in your dedicated channel on our Discord server

Heap dumps#

Now it's possible to collect heap dumps from dshackle during OutOfMemoryError. It could be very important for us for further troubleshooting and improve our service, send them to us as soon as possible. It's automated but it's necessary to do one thing - add another volume in your docker config - /path_to_dumps:/etc/dshackle/dumps.

If everything is ok you will see nothing, otherwise there will be the logs Heap dump creation is turned off or $path does not exist. Then heap dumps will be created in the folder you specify in your docker config if OutOfMemoryError is thrown. And if so there will be the log There are some heap dumps, please send them to dshackle developers during the next dshackle start. This message is printed periodically just to remind about your heap dumps.

Max number of heap dumps in the folder is 3. Dshackle monitors it and automatically clears extra dumps if necessary.

Additional info#

Since 1.10.0 version (for more info (opens in a new tab)) geth abandoned of indexing all transactions by default (they take transactions from 2,350,000 last blocks). Because of that tx methods like eth_getTransactionByHash and eth_getTransactionReceipt return null for old blocks. If you wish to disable transaction unindexing altogether, you can run Geth with --txlookuplimit=0. Please take this into account if you have geth nodes.

Dshackle Upgrade#

We release new versions of Dshackle quite often, improving stability and performance while adding support for new networks. Providers should stay up to date and install new versions as soon as possible, as using outdated versions may impact your earnings on our platform.

You can track updates through the following methods:

dRPC Discord – Channel #announcements in the Providers section
GitHub Releases – https://github.com/drpcorg/dshackle/releases (opens in a new tab)

You can subscribe to new releases on GitHub through various options, such as GitHub Notifications (Watching) or external services like https://newreleases.io (opens in a new tab)

Overview Price configuration