Local Restate Cluster with Docker

This guide shows how to deploy a distributed Restate cluster consisting of three nodes using Docker and Docker compose.

Deploy the Restate cluster using Docker

You can run a Restate Cluster using Docker and Docker Compose.To deploy a 3-node distributed Restate cluster, create a file docker-compose.yml and run docker compose up.

docker-compose.yml

x-environment: &common-env
  RESTATE_CLUSTER_NAME: "restate-cluster"
  # For more on logging, see: https://docs.restate.dev/operate/monitoring/logging
  RESTATE_LOG_FILTER: "restate=info"
  RESTATE_DEFAULT_REPLICATION: 2  # We require minimum of 2 nodes to accept writes
  # The addresses where nodes can reach each other over the "internal" Docker Compose network
  RESTATE_METADATA_CLIENT__ADDRESSES: '["http://restate-1:5122","http://restate-2:5122","http://restate-3:5122"]'
  # Partition snapshotting, see: https://docs.restate.dev/operate/snapshots
  RESTATE_WORKER__SNAPSHOTS__DESTINATION: "s3://restate/snapshots"
  RESTATE_WORKER__SNAPSHOTS__SNAPSHOT_INTERVAL_NUM_RECORDS: "1000"
  RESTATE_WORKER__SNAPSHOTS__AWS_REGION: "local"
  RESTATE_WORKER__SNAPSHOTS__AWS_ENDPOINT_URL: "http://minio:9000"
  RESTATE_WORKER__SNAPSHOTS__AWS_ALLOW_HTTP: true
  RESTATE_WORKER__SNAPSHOTS__AWS_ACCESS_KEY_ID: "minioadmin"
  RESTATE_WORKER__SNAPSHOTS__AWS_SECRET_ACCESS_KEY: "minioadmin"

x-defaults: &defaults
  image: docker.restate.dev/restatedev/restate:latest
  extra_hosts:
    - "host.docker.internal:host-gateway"
  volumes:
    - restate-data:/restate-data

services:
  restate-1:
    <<: *defaults
    ports:
      - "8080:8080"  # Ingress
      - "9070:9070"  # Admin
      - "5122:5122"  # Node-to-node communication
    environment:
      <<: *common-env
      RESTATE_NODE_NAME: restate-1
      RESTATE_FORCE_NODE_ID: 1
      RESTATE_ADVERTISED_ADDRESS: "http://restate-1:5122"  # Other Restate nodes must be able to reach us using this address
      RESTATE_AUTO_PROVISION: "true"                       # Only the first node provisions the cluster

  restate-2:
    <<: *defaults
    ports:
      - "25122:5122"
      - "29070:9070"
      - "28080:8080"
    environment:
      <<: *common-env
      RESTATE_NODE_NAME: restate-2
      RESTATE_FORCE_NODE_ID: 2
      RESTATE_ADVERTISED_ADDRESS: "http://restate-2:5122"
      RESTATE_AUTO_PROVISION: "false"

  restate-3:
    <<: *defaults
    ports:
      - "35122:5122"
      - "39070:9070"
      - "38080:8080"
    environment:
      <<: *common-env
      RESTATE_NODE_NAME: restate-3
      RESTATE_FORCE_NODE_ID: 3
      RESTATE_ADVERTISED_ADDRESS: "http://restate-3:5122"
      RESTATE_AUTO_PROVISION: "false"

  minio:
    image: quay.io/minio/minio
    entrypoint: "/bin/sh"
    # Ensure a bucket called "restate" exists on startup:
    command: "-c 'mkdir -p /data/restate && /usr/bin/minio server --quiet /data'"
    ports:
      - "9000:9000"

# We create a volume to persist data across container starts; delete it via `docker volume rm restate-data` if you want to start a fresh cluster
volumes:
  restate-data:

The cluster uses the replicated Bifrost provider and replicates log writes to a minimum of two nodes. Since we are running with 3 nodes, the cluster can tolerate one node failure without becoming unavailable. By default, partition state is replicated to all workers (though each partition has only one acting leader at a time).The replicated metadata cluster consists of all nodes since they all run the metadata-server role. Since the replicated metadata cluster requires a majority quorum to operate, the cluster can tolerate one node failure without becoming unavailable.Take a look at the cluster deployment documentation for more information on how to configure and deploy a distributed Restate cluster.In this example we also deployed a Minio server to host the cluster snapshots bucket. Visit Snapshots to learn more about whis is strongly recommended for all clusters.

Check the cluster status

You can check the status of the cluster by running the restatectl status command on any of the started Restate servers. Note, it might take a few seconds until the cluster has fully started and the status is available.

docker compose exec restate-1 restatectl status

Start a local Restate service

Follow the Quickstart guide to create a simple Restate service.

You can register the service endpoint at any of the started Restate nodes since they all run the admin role.

restate dp register http://host.docker.internal:9080

Or alternatively you can open the Restate UI at http://localhost:9080 and register the service endpoint there.

Invoke the service

You can invoke the registered service at any of the started Restate nodes since they all run the ingress.

curl localhost:8080/restate/call/Greeter/greet --json '"Sarah"' &&
curl localhost:28080/restate/call/Greeter/greet --json '"Bob"' &&
curl localhost:38080/restate/call/Greeter/greet --json '"Eve"'

Kill and restart Restate servers

Try killing and restarting one of the Restate nodes and see how the cluster reacts.

docker compose kill restate-1 &&
sleep 5 &&
docker compose up -d restate-1

Create snapshots

Try instructing the partition processors to create a snapshot of their state in the object store bucket:

docker compose exec restate-1 restatectl snapshot create

Navigate to the Minio console at http://localhost:9000 and browse the bucket contents (default credentials: minioadmin/minioadmin).

🎉Congratulations, you managed to run your first distributed Restate cluster and simulated some failures!

Here are some next steps for you to try:

Try to configure a 5-node Restate cluster that can tolerate up to two node failures.
Trim the logs (either manually, or by setting up automatic trimming) before adding more nodes.
Try to deploy a 3-node Restate cluster using Kubernetes.

Recipes

Development Guides

Deployment Guides

Integrations

Local Restate Cluster with Docker