diff --git a/DOCKER-README.md b/DOCKER-README.md deleted file mode 100644 index ed7a9a6..0000000 --- a/DOCKER-README.md +++ /dev/null @@ -1,56 +0,0 @@ -# Quilibrium Docker Instructions - -> [!TIP] -> For a comprehensive guide on building from source and deployment, see [DOCKER_GUIDE.md](DOCKER_GUIDE.md). - -## Build - -The only requirements are `git` (to checkout the repository) and docker (to build the image). -Golang does not have to be installed, the docker image build process uses a build stage that provides the -correct Go environment and compiles the node down to one command. - -In the repository root folder, where the [Dockerfile.source](Dockerfile.source) file is, build the docker image: -```shell -docker build -f Dockerfile.source --build-arg GIT_COMMIT=$(git log -1 --format=%h) -t quilibrium -t quilibrium:1.4.16 . -``` - -Use latest version instead of `1.4.16`. - -The image that is built is light and safe. It is based on Alpine Linux with the Quilibrium node binary, no -source code, nor the Go development environment. The image also has the `grpcurl` tool that can be used to -query the gRPC interface. - -### Task - -You can also use the [Task](https://taskfile.dev/) tool, it is a simple build tool that takes care of extracting -parameters and building the image. The tasks are all defined in [Taskfile.yaml](Taskfile.yaml). - -You can optionally create an `.env` file, in the same repository root folder to override specific parameters. Right now -only one optional env var is supported and that is `QUILIBRIUM_IMAGE_NAME`, if you want to change the default -image name from `quilibrium` to something else. If you are pushing your images to GitHub then you have to follow the -GitHub naming convention and use a name like `ghcr.io/mscurtescu/ceremonyclient`. - -Bellow there are example interactions with `Task`. - -The node version is extracted from [node/main.go](node/main.go). This version string is used to tag the image. The git -repo, branch and commit are read through the `git` command and depend on the current state of your working -directory (on what branch and at what commit you are). These last three values are used to label the image. - -List tasks: -```shell -task -l -``` - -Show what parameters, like image name, version etc, will be used: -```shell -task status -``` - -Build the image (aka run the `build` task): -```shell -task build -``` - -## Run - -In order to run a Quilibrium node using the docker image follow the instructions in the [docker](docker) subfolder. diff --git a/DOCKER_GUIDE.md b/DOCKER_GUIDE.md deleted file mode 100644 index 2f11971..0000000 --- a/DOCKER_GUIDE.md +++ /dev/null @@ -1,118 +0,0 @@ -# Quilibrium Node: Build from Source & Deployment Guide - -This guide covers the end-to-end process of building a Quilibrium (Q-Node) from source using Docker and optimizing it for hosting environments. - -## 1. System Preparation - -Before building, optimize the host network stack. Quilibrium relies on the QUIC protocol (UDP), which requires larger buffer sizes than the Linux default to prevent packet loss. - -### Increase UDP Buffer Sizes -Run these commands on your server: - -```bash -sudo sysctl -w net.core.rmem_max=7500000 -sudo sysctl -w net.core.wmem_max=7500000 -``` - -To make these changes permanent, add them to `/etc/sysctl.conf`. - -### Configure Firewall (UFW) -Open the ports required for P2P, streaming, and worker communication: - -```bash -sudo ufw allow 22/tcp -sudo ufw allow 443/tcp -sudo ufw allow 8336:8338/udp -sudo ufw allow 8340/udp -sudo ufw allow 50000:50010/tcp -sudo ufw allow 50000:50010/udp -sudo ufw reload -``` - -## 2. Configuration Generation - -Quilibrium nodes require a valid configuration and cryptographic keys to participate in the network. For containerized deployments, it is recommended to generate these separately from the main node execution. - -### Generating Config -You can use the `config-gen` utility included - -**Using Task (Recommended):** -```bash -task config:gen -``` - -You can override the directory using `CONFIG_DIR`: -```bash -task config:gen CONFIG_DIR=/path/to/config -``` - -## 3. Build from Source - -Navigate to your monorepo directory and execute: - -**Using Task:** -```bash -task build:node:source -``` - -**Using Docker directly:** -```bash -DOCKER_BUILDKIT=1 docker build \ - --target node-only \ - -f Dockerfile.source \ - --build-arg GIT_COMMIT=$(git log -1 --format=%h) \ - -t quilibrium:node-only . -``` - - -## 4. Deployment - -For servers with dedicated public IPs, **Host Networking** is the recommended deployment method. It eliminates Docker NAT overhead and ensures the node is easily reachable by peers. - -### Run the Container - -**Using Task:** -```bash -task deploy:node -``` - -**Using Docker directly:** -```bash -docker run -d --name q-node \ - --network host \ - --restart unless-stopped \ - -v $(pwd)/.config:/root/.config \ - quilibrium:node-only -signature-check=false -``` - -## 5. Managing the Container - -| Action | Command | -| --- | --- | -| **Stop** | `docker stop q-node` | -| **Start** | `docker start q-node` | -| **Restart** | `docker restart q-node` | -| **Status** | `docker ps --filter "name=q-node"` | -| **Logs** | `docker logs -f q-node` | - -## 6. Verification - -### Check Connectivity -Wait 5–10 minutes for the node to initialize workers and sync. Look for the "Reachable" status: -```bash -docker logs -f q-node | grep "reachable" -``` - -### Monitor Performance -Check how much data your node is processing: -```bash -docker stats q-node -``` - -## 7. Troubleshooting - -| Error | Meaning | Solution | -| --- | --- | --- | -| `connection refused ... :60000` | Master cannot talk to Workers. | Use `--network host`. | -| `YOUR NODE IS NOT REACHABLE` | External peers can't find you. | Ensure Ports 8336-8340 (UDP) are open. | -| `failed to increase buffer size` | UDP throttling. | Re-run the `sysctl` commands in Step 1. | diff --git a/Taskfile.yaml b/Taskfile.yaml index b8a3058..bf95704 100644 --- a/Taskfile.yaml +++ b/Taskfile.yaml @@ -5,9 +5,6 @@ version: '3' dotenv: - '.env' -env: - DOCKER_BUILDKIT: '1' - vars: VERSION: sh: cat config/version.go | grep -A 1 "func GetVersion() \[\]byte {" | grep -Eo '0x[0-9a-fA-F]+' | xargs printf "%d.%d.%d" @@ -23,7 +20,6 @@ tasks: status: desc: Display configuration info. cmds: - - echo -n "Image name:" && echo " ${QUILIBRIUM_IMAGE_NAME:-quilibrium}" - echo -n "Version :" && echo " {{.VERSION}}" - echo -n "Repo :" && echo " {{.GIT_REPO}}" - echo -n "Branch :" && echo " {{.GIT_BRANCH}}" @@ -55,148 +51,7 @@ tasks: - rpm/generate.sh - client/build.sh -o build/arm64_macos/qclient - build_node_arm64_linux: - desc: Build the Quilibrium node binary for ARM64 Linux. Outputs to node/build. - cmds: - - docker build --platform linux/arm64 -f Dockerfile.source --output node/build/arm64_linux --target=node . - - build_qclient_arm64_linux: - desc: Build the QClient node binary for ARM64 Linux. Outputs to client/build. - cmds: - - docker build --platform linux/arm64 -f Dockerfile.source --output client/build/arm64_linux --target=qclient . - - build_node_amd64_linux: - desc: Build the Quilibrium node binary for AMD64 Linux. Outputs to node/build. - cmds: - - docker build --platform linux/amd64 -f Dockerfile.source --output node/build/amd64_linux --target=node . - - build_conntest_amd64_linux: - desc: Build the Quilibrium node connection test binary for AMD64 Linux. Outputs to conntest/build. - cmds: - - docker build --platform linux/amd64 -f Dockerfile.conntest.source --output conntest/build/amd64_linux --target=conntest . - - build_node_amd64_avx512_linux: - desc: Build the Quilibrium node binary for AMD64 Linux with AVX-512 extensions. Outputs to node/build. - cmds: - - docker build --platform linux/amd64 -f Dockerfile.sourceavx512 --output node/build/amd64_avx512_linux --target=node . - - build_qclient_amd64_linux: - desc: Build the QClient node binary for AMD64 Linux. Outputs to client/build. - cmds: - - docker build --platform linux/amd64 -f Dockerfile.source --output client/build/amd64_linux --target=qclient . - - build_qclient_amd64_avx512_linux: - desc: Build the QClient node binary for AMD64 Linux with AVX-512 extensions. Outputs to client/build. - cmds: - - docker build --platform linux/amd64 -f Dockerfile.sourceavx512 --output client/build/amd64_avx512_linux --target=qclient . - - build_vdf_perf_analysis_amd64_linux: - cmds: - - docker build --platform linux/amd64 -f Dockerfile.vdf.source --output vdf/build/amd64_linux --target=vdf --progress=plain --no-cache . - - build_vdf_perf_analysis_amd64_avx512_linux: - cmds: - - docker build --platform linux/amd64 -f Dockerfile.vdf.sourceavx512 --output vdf/build/amd64_avx512_linux --target=vdf-avx512 . - - build_vdf_perf_analysis_amd64_zen3_linux: - cmds: - - docker build --platform linux/amd64 -f Dockerfile.vdf.sourcezen3 --output vdf/build/amd64_zen3_linux --target=vdf-zen3 --progress=plain --no-cache . - - build_vdf_perf_analysis_amd64_zen4_linux: - cmds: - - docker build --platform linux/amd64 -f Dockerfile.vdf.sourcezen4 --output vdf/build/amd64_zen4_linux --target=vdf-zen4 --progress=plain --no-cache . - - build_vdf_perf_analysis_arm64_linux: - cmds: - - docker build --platform linux/arm64 -f Dockerfile.vdf.source --output vdf/build/arm64_linux --target=vdf --progress=plain --no-cache . - - build:source: - desc: Build the Quilibrium docker image from source. - cmds: - - | - docker build \ - -f Dockerfile.source \ - --build-arg NODE_VERSION={{.VERSION}} \ - --build-arg GIT_REPO={{.GIT_REPO}} \ - --build-arg GIT_BRANCH={{.GIT_BRANCH}} \ - --build-arg GIT_COMMIT={{.GIT_COMMIT}} \ - -t ${QUILIBRIUM_IMAGE_NAME:-quilibrium}:{{.VERSION}}-source \ - -t ${QUILIBRIUM_IMAGE_NAME:-quilibrium}:source \ - . - status: - - | - docker image inspect \ - ${QUILIBRIUM_IMAGE_NAME:-quilibrium}:{{.VERSION}}-source \ - >/dev/null 2>/dev/null - - build:release: - desc: Build the Quilibrium docker image from release binaries. - aliases: - - build - cmds: - - | - docker build \ - -f Dockerfile.release \ - --build-arg NODE_VERSION={{.VERSION}} \ - --build-arg GIT_REPO={{.GIT_REPO}} \ - --build-arg GIT_BRANCH={{.GIT_BRANCH}} \ - --build-arg GIT_COMMIT={{.GIT_COMMIT}} \ - --build-arg MAX_KEY_ID={{.MAX_KEY_ID}} \ - -t ${QUILIBRIUM_IMAGE_NAME:-quilibrium}:{{.VERSION}}-release \ - -t ${QUILIBRIUM_IMAGE_NAME:-quilibrium}:release \ - . - status: - - | - docker image inspect \ - ${QUILIBRIUM_IMAGE_NAME:-quilibrium}:{{.VERSION}}-release \ - >/dev/null 2>/dev/null - - docker:login: - desc: Login to Docker hub - aliases: - - login - cmds: - - echo $DOCKER_TOKEN | docker login -u $DOCKER_USERNAME --password-stdin - - push: - desc: Push Quilibrium docker image to the container registry. - cmds: - - docker push ${QUILIBRIUM_IMAGE_NAME:-quilibrium}:{{.VERSION}} - - docker push ${QUILIBRIUM_IMAGE_NAME:-quilibrium}:latest - - test:qclient: - desc: Test the Quilibrium docker image. - cmds: - - client/test/run_tests.sh -d 'ubuntu' -v '24.04' - config:gen: desc: Generate configuration and keys using Go. cmds: - go run ./utils/config-gen --config {{.CONFIG_DIR | default ".config"}} - - - build:node:source: - desc: Build the optimized Quilibrium node-only docker image. - cmds: - - | - docker build \ - --target node-only \ - -f Dockerfile.source \ - --build-arg NODE_VERSION={{.VERSION}} \ - --build-arg GIT_REPO={{.GIT_REPO}} \ - --build-arg GIT_BRANCH={{.GIT_BRANCH}} \ - --build-arg GIT_COMMIT={{.GIT_COMMIT}} \ - -t ${QUILIBRIUM_IMAGE_NAME:-quilibrium}:{{.VERSION}}-node-only \ - -t ${QUILIBRIUM_IMAGE_NAME:-quilibrium}:node-only \ - . - - deploy:node: - desc: Run the Quilibrium node using host networking and external config. - cmds: - - | - docker run -d --name q-node \ - --network host \ - --restart unless-stopped \ - -v {{.CONFIG_DIR | default "$(pwd)/.config"}}:/root/.config \ - ${QUILIBRIUM_IMAGE_NAME:-quilibrium}:node-only \ - -signature-check=false diff --git a/.dockerignore b/docker/.dockerignore similarity index 100% rename from .dockerignore rename to docker/.dockerignore diff --git a/docker/.env.example b/docker/.env.example index f7337ca..3eb91b7 100644 --- a/docker/.env.example +++ b/docker/.env.example @@ -16,3 +16,7 @@ QUILIBRIUM_REST_PORT= # The public DNS name or IP address for this Quilibrium node. NODE_PUBLIC_NAME= + +# Use a custom configuration directory. +# Default: ../.config (monorepo root) +QUILIBRIUM_CONFIG_DIR= diff --git a/Dockerfile.conntest.source b/docker/Dockerfile.conntest.source similarity index 100% rename from Dockerfile.conntest.source rename to docker/Dockerfile.conntest.source diff --git a/Dockerfile.release b/docker/Dockerfile.release similarity index 100% rename from Dockerfile.release rename to docker/Dockerfile.release diff --git a/Dockerfile.source b/docker/Dockerfile.source similarity index 100% rename from Dockerfile.source rename to docker/Dockerfile.source diff --git a/Dockerfile.source.dockerignore b/docker/Dockerfile.source.dockerignore similarity index 100% rename from Dockerfile.source.dockerignore rename to docker/Dockerfile.source.dockerignore diff --git a/Dockerfile.sourceavx512 b/docker/Dockerfile.sourceavx512 similarity index 100% rename from Dockerfile.sourceavx512 rename to docker/Dockerfile.sourceavx512 diff --git a/Dockerfile.vdf.source b/docker/Dockerfile.vdf.source similarity index 100% rename from Dockerfile.vdf.source rename to docker/Dockerfile.vdf.source diff --git a/Dockerfile.vdf.sourceavx512 b/docker/Dockerfile.vdf.sourceavx512 similarity index 100% rename from Dockerfile.vdf.sourceavx512 rename to docker/Dockerfile.vdf.sourceavx512 diff --git a/Dockerfile.vdf.sourcezen3 b/docker/Dockerfile.vdf.sourcezen3 similarity index 100% rename from Dockerfile.vdf.sourcezen3 rename to docker/Dockerfile.vdf.sourcezen3 diff --git a/Dockerfile.vdf.sourcezen4 b/docker/Dockerfile.vdf.sourcezen4 similarity index 100% rename from Dockerfile.vdf.sourcezen4 rename to docker/Dockerfile.vdf.sourcezen4 diff --git a/docker/README.md b/docker/README.md index fa83b85..d0a9532 100644 --- a/docker/README.md +++ b/docker/README.md @@ -1,165 +1,293 @@ -# Quilibrium Docker Instructions +# Quilibrium Docker Guide -## Install Docker on a Server +This folder contains all the necessary files to build and run Quilibrium nodes using Docker. -> [!IMPORTANT] -> You have to install Docker Engine on your server, you don't want to install Docker Desktop. +## Prerequisites -The official Linux installation instructions start here: -https://docs.docker.com/engine/install/ +### Required Tools +- **Docker** (v20.10+) with BuildKit support +- **Docker Compose** (v2.0+) +- **Task** (optional but recommended) - [taskfile.dev](https://taskfile.dev) -For Ubuntu you can start here: -https://docs.docker.com/engine/install/ubuntu/ +### Install Task +```bash +# macOS +brew install go-task -While there are several installation methods, you really want to use the apt repository, this way you get -automatic updates. +# Linux (script) +sh -c "$(curl --location https://taskfile.dev/install.sh)" -- -d -b /usr/local/bin -Make sure you also follow the Linux post-installation steps: -https://docs.docker.com/engine/install/linux-postinstall/ +# Or via Go +go install github.com/go-task/task/v3/cmd/task@latest +``` -## Install Docker on a Desktop +### Docker Version Check +```bash +docker --version # Should be 20.10+ +docker compose version # Should be v2.0+ +``` -For a Linux desktop follow the server installation steps above, do not install Docker Desktop for Linux unless -you know what you are doing. +## 1. System Preparation -For Mac and Windows follow the corresponding Docker Desktop installation links from the top of: -https://docs.docker.com/engine/install/ +Before building or running, optimize the host network stack. Quilibrium relies on the QUIC protocol (UDP), which requires larger buffer sizes than the Linux default. -## Running a Node +### Increase UDP Buffer Sizes +```bash +sudo sysctl -w net.core.rmem_max=7500000 +sudo sysctl -w net.core.wmem_max=7500000 +``` +To make these changes permanent, add them to `/etc/sysctl.conf`. -Copy [docker-compose.yml](docker-compose.yml) to a new folder on a server. The official -Docker image provided by Quilibrium Network will be pulled. +### Configure Firewall (UFW) +Open the ports required for P2P, gRPC, REST, and worker communication: -A `.config/` subfolder will be created in this folder, this will hold both configuration -and the node storage. +```bash +sudo ufw allow 22/tcp +sudo ufw allow 443/tcp +sudo ufw allow 8336:8338/udp +sudo ufw allow 8340/udp +sudo ufw allow 50000:50010/tcp +sudo ufw allow 50000:50010/udp +sudo ufw reload +``` -Optionally you can also copy [Taskfile.yaml](Taskfile.yaml) and [.env.example](.env.example) to the -server, if you are planning to use them. See below. +## 2. File Structure -### New Instance +| File | Purpose | +| --- | --- | +| `Dockerfile.source` | Main multi-stage build for node + qclient | +| `Dockerfile.sourceavx512` | Optimized build with AVX-512 instructions | +| `Dockerfile.release` | Build from pre-compiled release binaries | +| `Dockerfile.conntest.source` | Connection test utility build | +| `Dockerfile.vdf.*` | VDF performance analysis builds (various CPU optimizations) | +| `docker-compose.yml` | Container orchestration config | +| `Taskfile.yaml` | Task automation commands | +| `.env.example` | Environment variable template | +| `rustup-init.sh` | Rust installer for build stages | -If you are starting a brand new node then simply run Quilibrium in a container with: -```shell +## 3. Dockerfile Variants + +### Production Dockerfiles + +| Dockerfile | Use Case | Target Stages | +| --- | --- | --- | +| `Dockerfile.source` | Standard build from source | `node-only`, `node`, `qclient`, `final` | +| `Dockerfile.sourceavx512` | AVX-512 optimized (Intel Xeon, AMD Zen4+) | Same as above | +| `Dockerfile.release` | Build from release binaries (faster) | Single stage | + +### Specialized Dockerfiles + +| Dockerfile | Use Case | +| --- | --- | +| `Dockerfile.conntest.source` | Network connectivity testing | +| `Dockerfile.vdf.source` | VDF performance benchmarking | +| `Dockerfile.vdf.sourceavx512` | VDF with AVX-512 optimizations | +| `Dockerfile.vdf.sourcezen3` | VDF optimized for AMD Zen3 | +| `Dockerfile.vdf.sourcezen4` | VDF optimized for AMD Zen4 | + +## 4. Build Targets & Cross-Compilation + +### Available Build Tasks + +| Task | Platform | Output | +| --- | --- | --- | +| `build_node_arm64_linux` | Linux ARM64 | `../node/build/arm64_linux/` | +| `build_node_amd64_linux` | Linux AMD64 | `../node/build/amd64_linux/` | +| `build_node_amd64_avx512_linux` | Linux AMD64 (AVX-512) | `../node/build/amd64_avx512_linux/` | +| `build_node_arm64_macos` | macOS ARM64 | `../node/build/arm64_macos/` | +| `build_qclient_arm64_linux` | Linux ARM64 | `../client/build/arm64_linux/` | +| `build_qclient_amd64_linux` | Linux AMD64 | `../client/build/amd64_linux/` | +| `build_qclient_amd64_avx512_linux` | Linux AMD64 (AVX-512) | `../client/build/amd64_avx512_linux/` | +| `build_qclient_arm64_macos` | macOS ARM64 | `../client/build/arm64_macos/` | + +### Cross-Compilation Examples + +**Build Linux ARM64 binary from any platform:** +```bash +task build_node_arm64_linux +``` + +**Build Linux AMD64 with AVX-512 optimizations:** +```bash +task build_node_amd64_avx512_linux +``` + +**Manual cross-compilation with Docker:** +```bash +# Build for ARM64 +docker build --platform linux/arm64 -f Dockerfile.source --output ../node/build/arm64_linux --target=node .. + +# Build for AMD64 +docker build --platform linux/amd64 -f Dockerfile.source --output ../node/build/amd64_linux --target=node .. +``` + +## 5. Docker Images + +### Build Images + +```bash +# Full image (node + qclient) +task build:source + +# Node-only optimized image +task build:node:source + +# From release binaries (faster) +task build:release +``` + +### Image Tags + +After building, images are tagged as: +- `quilibrium:2.1.0-source` / `quilibrium:source` +- `quilibrium:2.1.0-node-only` / `quilibrium:node-only` +- `quilibrium:2.1.0-release` / `quilibrium:release` + +## 6. Configuration + +### Configuration Directory +By default, the Docker configuration uses the `.config` directory at the **root of the repository** (`../.config`). This allows you to share configuration between native and containerized builds. + +### Generating Config +```bash +task config:gen +``` +This will generate the configuration in `../.config`. + +## 7. Running a Node + +### Quick Start (Docker Compose) +```bash +task up +``` +Or: +```bash docker compose up -d ``` -A `.config/` subfolder will be created under the current folder, this is mapped inside the container. +### Host Networking (Recommended for Servers) +For servers with dedicated public IPs, host networking eliminates Docker NAT overhead: + +```bash +task deploy:node +``` +Or: +```bash +docker run -d --name q-node \ + --network host \ + --restart unless-stopped \ + -v $(pwd)/../.config:/root/.config \ + quilibrium:node-only -signature-check=false +``` + +### New Instance +If you are starting a brand new node, a `.config/` folder will be created at the repository root. > [!IMPORTANT] -> Once the node is running (the `-node-info` command shows a balance) make sure you backup -> `config.yml` and `keys.yml`. +> Once the node is running (the `task node-info` command shows a balance), make sure you backup `config.yml` and `keys.yml`. ### Restore Previous Instance +1. Ensure your `config.yml` and `keys.yml` are in the `.config/` folder at the repository root. +2. Start the node: `task up` -If you have both `config.yml` and `keys.yml` backed up from a previous instance then follow these -steps to restore them: +## 8. gRPC & REST API Access -1. Create an empty `.config/` subfolder. -2. Copy `config.yml` and `keys.yml` to `.config/`. -3. Start the node with: - ```shell - docker compose up -d - ``` +The node exposes two APIs for programmatic access: -### Task +| API | Port | Protocol | Binding | +| --- | --- | --- | --- | +| **gRPC** | 8337 | TCP | `127.0.0.1` (localhost only) | +| **REST** | 8338 | TCP | `127.0.0.1` (localhost only) | -You can also use the [Task](https://taskfile.dev/) tool, it is a simple build tool that takes care of running -complex commands and intereacting with the container. The tasks are all defined in -[Taskfile.yaml](Taskfile.yaml). +### Using grpcurl +The container includes `grpcurl` for gRPC interaction: -You can optionally create an `.env` file, in the same folder to override specific parameters. Right now -only one optional env var is supported with `Task` and that is `QUILIBRIUM_IMAGE_NAME`, if you want to change the -default image name from `quilibrium` to something else. If you are pushing your images to GitHub, for example, then you -have to follow the GitHub naming convention and use a name like `ghcr.io/mscurtescu/ceremonyclient`. See the -[.env.example](.env.example) sample file, and keep in mind that `.env` is shared with -[docker-compose.yml](docker-compose.yml). +```bash +# List available services +docker compose exec node grpcurl -plaintext localhost:8337 list -Bellow there are example interactions with `Task`. +# Get node info +docker compose exec node grpcurl -plaintext localhost:8337 quilibrium.node.node.pb.NodeService/GetNodeInfo -Start the container through docker compose: -```shell -task up +# Get token balance +docker compose exec node grpcurl -plaintext localhost:8337 quilibrium.node.node.pb.NodeService/GetTokenInfo ``` -Show the logs through docker compose: -```shell -task logs +### Using qclient (inside container) +```bash +docker compose exec node qclient help +docker compose exec node qclient token balance +docker compose exec node qclient token info ``` -Drop into a shell inside the running container: -```shell -task shell -``` +### Exposing APIs Externally +By default, APIs are bound to localhost for security. To expose externally, modify `docker-compose.override.yml`: -Stop the running container(s): -```shell -task down -``` - -Backup the critical configuration: -```shell -task backup -``` - -The above command will create a `backup.tar.gz` archive in the current folder, you still have to copy this -file from the server into a safe location. The command adds the `config.yml` and `keys.yml` files from -the `.config/` subfolder to the archive, with the ownership of the current user. - - -## Customizing docker-compose.yml - -If you want to change certain parameters in [docker-compose.yml](docker-compose.yml) it is better not -to edit the file directly as new versions pushed through git would overwrite your changes. A more -flexible solution is to create another file called `docker-compose.override.yml` right next to it -and specifying the necessary overriding changes there. - -For example: ```yaml services: node: - image: ghcr.io/mscurtescu/ceremonyclient - restart: on-failure:7 + ports: + - '0.0.0.0:8337:8337/tcp' # gRPC (use with caution) + - '0.0.0.0:8338:8338/tcp' # REST (use with caution) ``` -The above will override the image name and also the restart policy. +> [!WARNING] +> Exposing gRPC/REST externally can be a security risk. Use a reverse proxy with authentication if needed. -You can optionally create an `.env` file, in the same folder to override specific parameters. See the -[.env.example](.env.example) sample file, and keep in mind that `.env` is shared with -[Taskfile.yaml](Taskfile.yaml). You can customize the image name and port mappings. +## 9. Management Commands -To check if your overrides are being picked up run the following command: -```shell -docker compose config +| Action | Task Command | Docker Command | +| --- | --- | --- | +| **Status** | `task status` | N/A | +| **Logs** | `task logs` | `docker compose logs -f` | +| **Stop** | `task down` | `docker compose down` | +| **Shell** | `task shell` | `docker compose exec -it node sh` | +| **Node Info** | `task node-info` | `docker compose exec node node -node-info` | +| **Backup** | `task backup` | N/A | +| **Restore** | `task restore` | N/A | +| **Update** | `task update` | Pull + restart | +| **Test Port** | `task test:port` | Requires `NODE_PUBLIC_NAME` | + +The `backup` task creates a `backup.tar.gz` archive in this `docker/` folder containing `config.yml` and `keys.yml` from `../.config`. Copy this file to a safe location. + +### Customizing Configuration +Create a `.env` file based on [.env.example](.env.example): + +| Variable | Default | Description | +| --- | --- | --- | +| `QUILIBRIUM_IMAGE_NAME` | `quilibrium` | Docker image name | +| `QUILIBRIUM_CONFIG_DIR` | `../.config` | Configuration directory | +| `QUILIBRIUM_P2P_PORT` | `8336` | P2P UDP port | +| `QUILIBRIUM_GRPC_PORT` | `8337` | gRPC TCP port | +| `QUILIBRIUM_REST_PORT` | `8338` | REST TCP port | +| `NODE_PUBLIC_NAME` | (none) | Public DNS/IP for port testing | + +## 10. Verification + +### Check Connectivity +Wait 5–10 minutes for the node to initialize. Look for the "Reachable" status: +```bash +docker logs -f q-node | grep "reachable" ``` -This will output the merged and canonical compose file that will be used to run the container(s). - - -## Interact with a running container - -Drop into a shell inside a running container: -```shell -docker compose exec -it node sh +### Test Port Visibility +```bash +NODE_PUBLIC_NAME=your.server.ip task test:port ``` -Watch the logs: -```shell -docker compose logs -f +### Monitor Performance +```bash +docker stats q-node ``` -Get the node related info (peer id, version, max frame and balance): -```shell -docker compose exec node node -node-info -``` +## 11. Troubleshooting -Run the DB console: -```shell -docker compose exec node node -db-console -``` - -Run the Quilibrium client: -```shell -docker compose exec node qclient help -docker compose exec node qclient token help -docker compose exec node qclient token balance -``` +| Issue | Solution | +| --- | --- | +| **Node Not Reachable** | Ensure ports 8336-8340 (UDP) are open in your firewall. | +| **Buffer Size Errors** | Re-run the `sysctl` commands in Step 1. | +| **Config Not Found** | Verify that `../.config` exists and contains your keys. | +| **Backup Fails** | Ensure `../.config/config.yml` and `../.config/keys.yml` exist. | +| **Connection Refused :60000** | Use `--network host` for worker communication. | +| **grpcurl not found** | Use the full image (`build:source`) not `node-only`. | +| **Build fails on Apple Silicon** | Use `--platform linux/amd64` for cross-compilation. | diff --git a/docker/Taskfile.yaml b/docker/Taskfile.yaml index d2115ba..01f289a 100644 --- a/docker/Taskfile.yaml +++ b/docker/Taskfile.yaml @@ -5,11 +5,34 @@ version: '3' dotenv: - '.env' +env: + DOCKER_BUILDKIT: '1' + vars: PROJECT_NAME: quilibrium SERVICE_NAME: node + VERSION: + sh: cat ../config/version.go | grep -A 1 "func GetVersion() \[\]byte {" | grep -Eo '0x[0-9a-fA-F]+' | xargs printf "%d.%d.%d" + GIT_REPO: + sh: git config --get remote.origin.url | sed 's/\.git$//' + GIT_BRANCH: + sh: git rev-parse --abbrev-ref HEAD + GIT_COMMIT: + sh: git log -1 --format=%h + MAX_KEY_ID: 17 tasks: + status: + desc: Display configuration info. + cmds: + - echo -n "Image name:" && echo " ${QUILIBRIUM_IMAGE_NAME:-quilibrium}" + - echo -n "Version :" && echo " {{.VERSION}}" + - echo -n "Repo :" && echo " {{.GIT_REPO}}" + - echo -n "Branch :" && echo " {{.GIT_BRANCH}}" + - echo -n "Commit :" && echo " {{.GIT_COMMIT}}" + - echo -n "Max Key ID:" && echo " {{.MAX_KEY_ID}}" + silent: true + up: desc: Run a new Quilibrium and related containers, through docker compose. cmds: @@ -56,25 +79,25 @@ tasks: desc: Create a backup file with the critical configuration files. prompt: You will be prompted for root access. Make sure you verify the generated backup file. Continue? preconditions: - - sh: 'test -d .config' + - sh: 'test -d ../.config' msg: '.config does not exists!' - - sh: 'test -f .config/config.yml' + - sh: 'test -f ../.config/config.yml' msg: '.config/config.yml does not exists!' - - sh: 'test -f .config/keys.yml' + - sh: 'test -f ../.config/keys.yml' msg: '.config/keys.yml does not exists!' - sh: '! test -f backup.tar.gz' msg: 'A previous backup.tar.gz found in the current folder!' sources: - - '.config/config.yml' - - '.config/keys.yml' + - '../.config/config.yml' + - '../.config/keys.yml' generates: - 'backup.tar.gz' cmds: - | export TMP_DIR=$(mktemp -d) export TASK_DIR=$(pwd) - sudo cp .config/config.yml $TMP_DIR - sudo cp .config/keys.yml $TMP_DIR + sudo cp ../.config/config.yml $TMP_DIR + sudo cp ../.config/keys.yml $TMP_DIR sudo chown $(whoami):$(id -gn) $TMP_DIR/* cd $TMP_DIR tar -czf $TASK_DIR/backup.tar.gz * @@ -87,19 +110,19 @@ tasks: restore: desc: Restores a backup file with the critical configuration files. preconditions: - - sh: '! test -d .config' + - sh: '! test -d ../.config' msg: '.config already exists, restore cannot be performed safely!' - sh: 'test -f backup.tar.gz' msg: 'backup.tar.gz not found in the current folder!' sources: - 'backup.tar.gz' generates: - - '.config/config.yml' - - '.config/keys.yml' + - '../.config/config.yml' + - '../.config/keys.yml' cmds: - | - mkdir .config - tar -xzf backup.tar.gz -C .config + mkdir ../.config + tar -xzf backup.tar.gz -C ../.config echo "Backup restored from: backup.tar.gz" silent: true @@ -112,3 +135,172 @@ tasks: msg: 'The public DNS name or IP address of the server must be set in NODE_PUBLIC_NAME.' cmds: - 'nc -vzu ${NODE_PUBLIC_NAME} ${QUILIBRIUM_P2P_PORT:=8336}' + + build_node_arm64_macos: + desc: Build the Quilibrium node binary for MacOS ARM. Assumes it's ran from the same platform. Outputs to node/build. + cmds: + - ../vdf/generate.sh + - ../bls48581/generate.sh + - ../verenc/generate.sh + - ../bulletproofs/generate.sh + - ../ferret/generate.sh + - ../channel/generate.sh + - ../rpm/generate.sh + - ../node/build.sh -o ../node/build/arm64_macos/node + + build_qclient_arm64_macos: + desc: Build the QClient node binary for MacOS ARM. Outputs to client/build + cmds: + - ../vdf/generate.sh + - ../bls48581/generate.sh + - ../verenc/generate.sh + - ../bulletproofs/generate.sh + - ../ferret/generate.sh + - ../channel/generate.sh + - ../rpm/generate.sh + - ../client/build.sh -o ../client/build/arm64_macos/qclient + + build_node_arm64_linux: + desc: Build the Quilibrium node binary for ARM64 Linux. Outputs to node/build. + cmds: + - docker build --platform linux/arm64 -f Dockerfile.source --output ../node/build/arm64_linux --target=node .. + + build_qclient_arm64_linux: + desc: Build the QClient node binary for ARM64 Linux. Outputs to client/build. + cmds: + - docker build --platform linux/arm64 -f Dockerfile.source --output ../client/build/arm64_linux --target=qclient .. + + build_node_amd64_linux: + desc: Build the Quilibrium node binary for AMD64 Linux. Outputs to node/build. + cmds: + - docker build --platform linux/amd64 -f Dockerfile.source --output ../node/build/amd64_linux --target=node .. + + build_conntest_amd64_linux: + desc: Build the Quilibrium node connection test binary for AMD64 Linux. Outputs to conntest/build. + cmds: + - docker build --platform linux/amd64 -f Dockerfile.conntest.source --output ../conntest/build/amd64_linux --target=conntest .. + + build_node_amd64_avx512_linux: + desc: Build the Quilibrium node binary for AMD64 Linux with AVX-512 extensions. Outputs to node/build. + cmds: + - docker build --platform linux/amd64 -f Dockerfile.sourceavx512 --output ../node/build/amd64_avx512_linux --target=node .. + + build_qclient_amd64_linux: + desc: Build the QClient node binary for AMD64 Linux. Outputs to client/build. + cmds: + - docker build --platform linux/amd64 -f Dockerfile.source --output ../client/build/amd64_linux --target=qclient .. + + build_qclient_amd64_avx512_linux: + desc: Build the QClient node binary for AMD64 Linux with AVX-512 extensions. Outputs to client/build. + cmds: + - docker build --platform linux/amd64 -f Dockerfile.sourceavx512 --output ../client/build/amd64_avx512_linux --target=qclient .. + + build_vdf_perf_analysis_amd64_linux: + cmds: + - docker build --platform linux/amd64 -f Dockerfile.vdf.source --output ../vdf/build/amd64_linux --target=vdf --progress=plain --no-cache .. + + build_vdf_perf_analysis_amd64_avx512_linux: + cmds: + - docker build --platform linux/amd64 -f Dockerfile.vdf.sourceavx512 --output ../vdf/build/amd64_avx512_linux --target=vdf-avx512 .. + + build_vdf_perf_analysis_amd64_zen3_linux: + cmds: + - docker build --platform linux/amd64 -f Dockerfile.vdf.sourcezen3 --output ../vdf/build/amd64_zen3_linux --target=vdf-zen3 --progress=plain --no-cache .. + + build_vdf_perf_analysis_amd64_zen4_linux: + cmds: + - docker build --platform linux/amd64 -f Dockerfile.vdf.sourcezen4 --output ../vdf/build/amd64_zen4_linux --target=vdf-zen4 --progress=plain --no-cache .. + + build_vdf_perf_analysis_arm64_linux: + cmds: + - docker build --platform linux/arm64 -f Dockerfile.vdf.source --output ../vdf/build/arm64_linux --target=vdf --progress=plain --no-cache .. + + build:source: + desc: Build the Quilibrium docker image from source. + cmds: + - | + docker build \ + -f Dockerfile.source \ + --build-arg NODE_VERSION={{.VERSION}} \ + --build-arg GIT_REPO={{.GIT_REPO}} \ + --build-arg GIT_BRANCH={{.GIT_BRANCH}} \ + --build-arg GIT_COMMIT={{.GIT_COMMIT}} \ + -t ${QUILIBRIUM_IMAGE_NAME:-quilibrium}:{{.VERSION}}-source \ + -t ${QUILIBRIUM_IMAGE_NAME:-quilibrium}:source \ + .. + status: + - | + docker image inspect \ + ${QUILIBRIUM_IMAGE_NAME:-quilibrium}:{{.VERSION}}-source \ + >/dev/null 2>/dev/null + + build:release: + desc: Build the Quilibrium docker image from release binaries. + aliases: + - build + cmds: + - | + docker build \ + -f Dockerfile.release \ + --build-arg NODE_VERSION={{.VERSION}} \ + --build-arg GIT_REPO={{.GIT_REPO}} \ + --build-arg GIT_BRANCH={{.GIT_BRANCH}} \ + --build-arg GIT_COMMIT={{.GIT_COMMIT}} \ + --build-arg MAX_KEY_ID={{.MAX_KEY_ID}} \ + -t ${QUILIBRIUM_IMAGE_NAME:-quilibrium}:{{.VERSION}}-release \ + -t ${QUILIBRIUM_IMAGE_NAME:-quilibrium}:release \ + .. + status: + - | + docker image inspect \ + ${QUILIBRIUM_IMAGE_NAME:-quilibrium}:{{.VERSION}}-release \ + >/dev/null 2>/dev/null + + docker:login: + desc: Login to Docker hub + aliases: + - login + cmds: + - echo $DOCKER_TOKEN | docker login -u $DOCKER_USERNAME --password-stdin + + push: + desc: Push Quilibrium docker image to the container registry. + cmds: + - docker push ${QUILIBRIUM_IMAGE_NAME:-quilibrium}:{{.VERSION}} + - docker push ${QUILIBRIUM_IMAGE_NAME:-quilibrium}:latest + + test:qclient: + desc: Test the Quilibrium docker image. + cmds: + - ../client/test/run_tests.sh -d 'ubuntu' -v '24.04' + + config:gen: + desc: Generate configuration and keys using Go. + cmds: + - go run ../utils/config-gen --config {{.CONFIG_DIR | default "../.config"}} + + build:node:source: + desc: Build the optimized Quilibrium node-only docker image. + cmds: + - | + docker build \ + --target node-only \ + -f Dockerfile.source \ + --build-arg NODE_VERSION={{.VERSION}} \ + --build-arg GIT_REPO={{.GIT_REPO}} \ + --build-arg GIT_BRANCH={{.GIT_BRANCH}} \ + --build-arg GIT_COMMIT={{.GIT_COMMIT}} \ + -t ${QUILIBRIUM_IMAGE_NAME:-quilibrium}:{{.VERSION}}-node-only \ + -t ${QUILIBRIUM_IMAGE_NAME:-quilibrium}:node-only \ + .. + + deploy:node: + desc: Run the Quilibrium node using host networking and external config. + cmds: + - | + docker run -d --name q-node \ + --network host \ + --restart unless-stopped \ + -v {{.CONFIG_DIR | default "$(pwd)/../.config"}}:/root/.config \ + ${QUILIBRIUM_IMAGE_NAME:-quilibrium}:node-only \ + -signature-check=false diff --git a/docker/docker-compose.yml b/docker/docker-compose.yml index 816b238..c3b098c 100644 --- a/docker/docker-compose.yml +++ b/docker/docker-compose.yml @@ -1,6 +1,6 @@ name: quilibrium -# See sysctl related warning in DOCKER-README.md. +# See sysctl related warning in README.md. # Host configuration changes are required. services: @@ -16,13 +16,13 @@ services: - '127.0.0.1:${QUILIBRIUM_GRPC_PORT:-8337}:8337/tcp' # gRPC - '127.0.0.1:${QUILIBRIUM_REST_PORT:-8338}:8338/tcp' # REST healthcheck: - test: ["CMD", "grpcurl", "-plaintext", "localhost:8337", "list", "quilibrium.node.node.pb.NodeService"] + test: [ "CMD", "grpcurl", "-plaintext", "localhost:8337", "list", "quilibrium.node.node.pb.NodeService" ] interval: 30s timeout: 5s retries: 3 start_period: 15m volumes: - - ./.config:/root/.config + - ${QUILIBRIUM_CONFIG_DIR:-../.config}:/root/.config logging: driver: "json-file" options: