diff --git a/README.md b/README.md index 96f437e0..68c94612 100644 --- a/README.md +++ b/README.md @@ -4,10 +4,11 @@ This repository contains the implementation of The Innovation Game (TIG). ## Important Links +* [Quick Start for Benchmarkers](./tig-benchmarker/README.md#quick-start) +* [Quick Start for Innovators](./tig-algorithms/README.md#quick-start) * [TIG Documentation](https://docs.tig.foundation/) * [TIG Whitepaper](docs/whitepaper.pdf) * [TIG Licensing Explainer](docs/guides/anatomy.md) -* [Getting Started with Innovating](docs/guides/innovating.md) * [Implementations vs Breakthroughs](docs/guides/breakthroughs.md) * [Voting Guidelines for Token Holders](docs/guides/voting.md) @@ -27,21 +28,44 @@ This repository contains the implementation of The Innovation Game (TIG). ## Docker Images -TIG docker images are hosted on [Github Packages](https://github.com/orgs/tig-foundation/packages): +TIG docker images are hosted on [Github Packages](https://github.com/orgs/tig-foundation/packages), supporting `linux/arm64` and `linux/amd64` platforms. -* [dev](https://github.com/orgs/tig-foundation/packages/container/package/tig-monorepo%2Fdev) - environment for Innovators who are developing algorithms -* [runtime](https://github.com/orgs/tig-foundation/packages/container/package/tig-monorepo%2Fruntime) - environment for Benchmarkers who are running slaves +For Innovators who are developing & compiling algorithms, there is a `dev` image for each challenge, containing the development environment: -## Useful Scripts +* [satisfiability/dev](https://github.com/tig-foundation/tig-monorepo/pkgs/container/tig-monorepo%2Fsatisfiability%2Fdev) +* [vehicle_routing/dev](https://github.com/tig-foundation/tig-monorepo/pkgs/container/tig-monorepo%2Fvehicle_routing%2Fdev) +* [knapsack/dev](https://github.com/tig-foundation/tig-monorepo/pkgs/container/tig-monorepo%2Fknapsack%2Fdev) +* [vector_search/dev](https://github.com/tig-foundation/tig-monorepo/pkgs/container/tig-monorepo%2Fvector_search%2Fdev) +* [hypergraph/dev](https://github.com/tig-foundation/tig-monorepo/pkgs/container/tig-monorepo%hypergraph%2Fdev) -Under `scripts/` folder is a bunch of useful scripts: +For Benchmarkers to spin up slaves, there is a `benchmarker\slave` image along with a `runtime` image for each challenge. It is intended that these images are spun up as part of [`docker-compose-slave.yml`](tig-benchmarker/docker-compose-slave.yml) (see [this README](tig-benchmarker/README.md) for more details): + +* [benchmarker/slave](https://github.com/orgs/tig-foundation/packages/container/package/tig-monorepo%2Fbenchmarker%2Fslave) +* [satisfiability/runtime](https://github.com/tig-foundation/tig-monorepo/pkgs/container/tig-monorepo%2Fsatisfiability%2Fruntime) +* [vehicle_routing/runtime](https://github.com/tig-foundation/tig-monorepo/pkgs/container/tig-monorepo%2Fvehicle_routing%2Fruntime) +* [knapsack/runtime](https://github.com/tig-foundation/tig-monorepo/pkgs/container/tig-monorepo%2Fknapsack%2Fruntime) +* [vector_search/runtime](https://github.com/tig-foundation/tig-monorepo/pkgs/container/tig-monorepo%2Fvector_search%2Fruntime) +* [hypergraph/runtime](https://github.com/tig-foundation/tig-monorepo/pkgs/container/tig-monorepo%hypergraph%2Fruntime) + +For Benchmarkers to spin up a master, there is a series of images that are spun up as part of [`docker-compose-master.yml`](tig-benchmarker/docker-compose-master.yml) (see [this README](tig-benchmarker/README.md) for more details): + +* [benchmarker/master](https://github.com/tig-foundation/tig-monorepo/pkgs/container/tig-monorepo%2Fbenchmarker%2Fmaster) +* [benchmarker/ui](https://github.com/tig-foundation/tig-monorepo/pkgs/container/tig-monorepo%2Fbenchmarker%2Fui) +* [benchmarker/postgres](https://github.com/tig-foundation/tig-monorepo/pkgs/container/tig-monorepo%2Fbenchmarker%2Fpostgres) +* [benchmarker/nginx](https://github.com/tig-foundation/tig-monorepo/pkgs/container/tig-monorepo%2Fbenchmarker%2Fnginx) + +### Useful Scripts + +As part of the `runtime` and `dev` images, there are a bunch of useful scripts available on `PATH`: -* `download_algorithm` * `list_algorithms` -* `list_challenges` -* `test_algorithms` +* `download_algorithm ` +* `test_algorithm ` -These are available on `PATH` in the `dev` and `runtime` docker images +Notes: +* The docker will automatically set a CHALLENGE environment variable used by these scripts + * e.g. `knapsack/runtime` docker will set `CHALLENGE=knapsack` +* Use `--testnet` option to target testnet ## License diff --git a/docs/guides/innovating.md b/docs/guides/innovating.md deleted file mode 100644 index 82943261..00000000 --- a/docs/guides/innovating.md +++ /dev/null @@ -1,262 +0,0 @@ -# Getting Started with Innovating - -## Developer Environment - -TIG compiles all algorithms into shared objects and ptx for verifiable execution. - -TIG currently requires all algorithms to be written in [Rust](https://www.rust-lang.org/tools/install). Algorithms can optionally have a .cu with Cuda kernels, which are launched from the Rust code. - -We recommend developing using [Visual Studio Code](https://code.visualstudio.com/) with Rust plugins: -* rust-analyzer -* Even Better TOML -* crates -* CodeLLDB - -## Checking out Existing Algorithms - -Every algorithm has its own `` with name `/`. - -To pull an existing algorithm from tig-monorepo repository, run the following command: -``` -git pull origin -``` - -Only algorithms that are successfully compiled have their branch pushed to this public repository. - -Each algorithm branch will have 6 key files (11 if there is CUDA code): -1. Rust code with TIG commercial license header @ `tig-algorithms/src//commercial.rs` -2. Rust code with TIG open data license header @ `tig-algorithms/src//open_data.rs` -3. Rust code with TIG benchmarker outbound license header @ `tig-algorithms/src//benchmarker_outbound.rs` -4. Rust code with TIG innovator outbound license header @ `tig-algorithms/src//innovator_outbound.rs` -5. Cuda code with TIG inbound license header @ `tig-algorithms/src//inbound.cu` -6. Tarball with shared objects and (optionally) ptx @ `tig-algorithms/lib/.tar.gz` -7. Cuda code with TIG commercial license header @ `tig-algorithms/src//commercial.cu` -8. Cuda code with TIG open data license header @ `tig-algorithms/src//open_data.cu` -9. Cuda code with TIG benchmarker outbound license header @ `tig-algorithms/src//benchmarker_outbound.cu` -10. Cuda code with TIG innovator outbound license header @ `tig-algorithms/src//innovator_outbound.cu` -11. Cuda code with TIG inbound license header @ `tig-algorithms/src//inbound.cu` - -## Developing Your Algorithm - -**READ THE IMPORTANT NOTES AT THE BOTTOM OF THIS SECTION** - -1. Pick a challenge (``) to develop an algorithm for -2. Make a copy of `tig-algorithms/src//template.rs` or an existing algorithm (see notes) - * (Optional) for Cuda, additionally make a copy of `tig-algorithms/src//template.cu` -3. Make sure your file has the following notice in its header if you intend to submit it to TIG: -``` -Copyright [year copyright work created] [name of copyright owner] - -Identity of Submitter [name of person or entity that submits the Work to TIG] - -UAI [UAI (if applicable)] - -Licensed under the TIG Inbound Game License v2.0 or (at your option) any later -version (the "License"); you may not use this file except in compliance with the -License. You may obtain a copy of the License at - -https://github.com/tig-foundation/tig-monorepo/tree/main/docs/licenses - -Unless required by applicable law or agreed to in writing, software distributed -under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR -CONDITIONS OF ANY KIND, either express or implied. See the License for the specific -language governing permissions and limitations under the License. -``` - * If your implementation is based on an algorithmic method submitted to TIG, you must attribute your implementation to it (example UAI: `c001_b001`) - * UAI of a method is detailed inside `tig-breakthroughs//.md` - * Methods have branch name `/method/` - * If your implementation is based on an algorithmic method outside of TIG, set UAI to `null` -4. Rename the file with your own `` -5. Edit `tig-algorithms//mod.rs` to export your algorithm and test it: - ``` - pub mod ; - - #[cfg(test)] - mod tests { - #[cfg(feature = "cuda")] - use cudarc::{driver::CudaContext, nvrtc::Ptx, runtime::result::device::get_device_prop}; - use super::::solve_challenge; - use tig_challenges::::*; - - #[cfg(not(feature = "cuda"))] - #[test] - fn test_algorithm() { - let difficulty = Difficulty { - // Uncomment the relevant fields. - // Modify the values for different difficulties - - // -- satisfiability -- - // num_variables: 2000, - // clauses_to_variables_percent: 415, - - // -- vehicle_routing -- - // num_nodes: 100, - // better_than_baseline: 15, - - // -- knapsack -- - // num_items: 100, - // better_than_baseline: 10, - - // -- vector_search -- - // num_queries: 1000, - // better_than_baseline: 350, - - // -- hypergraph -- - // num_hyperedges: 500, - // better_than_baseline: 150, - }; - let seed = [0u8; 32]; // change this to generate different instances - let challenge = Challenge::generate_instance(&seed, &difficulty).unwrap(); - match solve_challenge(&challenge) { - Ok(Some(solution)) => match challenge.verify_solution(&solution) { - Ok(_) => println!("Valid solution"), - Err(e) => println!("Invalid solution: {}", e), - }, - Ok(None) => println!("No solution"), - Err(e) => println!("Algorithm error: {}", e), - }; - } - - #[cfg(feature = "cuda")] - #[test] - fn test_algorithm() { - let difficulty = Difficulty { - // Uncomment the relevant fields. - // Modify the values for different difficulties - - // -- satisfiability -- - // num_variables: 2000, - // clauses_to_variables_percent: 415, - - // -- vehicle_routing -- - // num_nodes: 100, - // better_than_baseline: 15, - - // -- knapsack -- - // num_items: 100, - // better_than_baseline: 10, - - // -- vector_search -- - // num_queries: 1000, - // better_than_baseline: 350, - - // -- hypergraph -- - // num_hyperedges: 500, - // better_than_baseline: 150, - }; - let seed = [0u8; 32]; // change this to generate different instances - - let ptx_path = "/app/tig-algorithms/lib/vector_search/ptx/simple_search.ptx"; // change this to the path of your PTX file - let ptx = Ptx::from_file(ptx_path); - let gpu_device = 0; // change this to the desired GPU device ID - let ctx = CudaContext::new(gpu_device).unwrap(); - ctx.set_blocking_synchronize().unwrap(); - let module = ctx.load_module(ptx).unwrap(); - let stream = ctx.default_stream(); - let prop = get_device_prop(gpu_device as i32).unwrap(); - - let challenge = - Challenge::generate_instance(&seed, &difficulty, module.clone(), stream.clone(), &prop) - .unwrap(); - - match solve_challenge(&challenge, module.clone(), stream.clone(), &prop) { - Ok(Some(solution)) => { - match challenge.verify_solution(&solution, module.clone(), stream.clone(), &prop) { - Ok(_) => println!("Valid solution"), - Err(e) => println!("Invalid solution: {}", e), - } - } - Ok(None) => println!("No solution"), - Err(e) => println!("Algorithm error: {}", e), - }; - } - } - ``` -6. Run the appropiate [development docker image](https://github.com/tig-foundation/tig-monorepo/pkgs/container/tig-monorepo%2Fdev). Available flavours are: - * amd64 (x86_64 compatible) - * aarch64 - * amd64-cuda12.6.3 (x86_64 compatible) - * aarch64-cuda12.6.3 - ``` - # example - docker run -it -v $(pwd):/app --gpus all ghcr.io/tig-foundation/tig-monorepo/dev:0.0.1-amd64-cuda12.6.3 - ``` -6. If you have Cuda code, use `build_ptx` to compile it - ``` - build_ptx - ``` -7. Run the test - * No cuda: - ``` - cargo test -p tig-algorithms --release -- --nocapture - ``` - * With cuda: - ``` - cargo test -p tig-algorithms --release --features cuda -- --nocapture - ``` - -**IMPORTANT (READ THIS):** -* Not all challenge instances have solutions. -* You can find the current qualifying difficulties by: - * Query https://mainnet-api.tig.foundation/get-block for - * Query https://mainnet-api.tig.foundation/get-challenges?block_id= for -* If you are copying and modifying an algorithm that has been submitted to TIG, make sure to use the `innovator_outbound` version -* Do not include tests in your algorithm file. TIG will reject your algorithm submission. -* Only your algorithm's code gets submitted. You should not be modifying `Cargo.toml` in `tig-algorithms`. Any extra dependencies you add will not be available when TIG compiles your algorithm -* There are comments with more tips inside the templates! - * Rust: `tig-algorithms/src//template.rs` - * Cuda: `tig-algorithms/src//template.cu` - -## Locally Compiling Your Algorithm into Shared Object - -See the [README](../../tig-binary/README.md) for `tig-binary` - -## Testing Performance of Algorithms - -TODO - -## Checking CI Successfully Compiles Your Algorithm - -TIG pushes all algorithms to their own branch which triggers the CI (`.github/workflows/build_algorithm.yml`). - -To replicate this, Innovators will want to create a private fork: - -1. Create private repository on GitHub -2. Create empty git repository on your local machine - ``` - mkdir tig-monorepo - cd tig-monorepo - git init - ``` -3. Setup remotes with origin pointed to your private repository - ``` - git remote add origin - git remote add public https://github.com/tig-foundation/tig-monorepo.git - ``` - -4. Pulling `blank_slate` from TIG public repository (branch with no algorithms) - ``` - git fetch public - git checkout -b blank_slate - git pull public blank_slate - ``` - -5. Push to your private repository - ``` - git push origin blank_slate - ``` - -6. Trigger the CI on your private repo with your algorithm branch: - ``` - git checkout -b / blank_slate - git push origin / - ``` - -## Making Your Submission - -10 TIG will be deducted from your Available Fee Balance to make a submission. You can topup via the [Benchmarker page](https://play.tig.foundation/benchmarker) - -**IMPORTANT:** -* Submissions are final and cannot be modified after they are made -* Be sure to adhere to the notes in [developing your algorithm](#developing-your-algorithm) -* We highly recommend [checking the CI can compile your algorithm](#checking-ci-successfully-compiles-your-algorithm) diff --git a/tig-algorithms/README.md b/tig-algorithms/README.md index 6e5cd87d..4f225e5c 100644 --- a/tig-algorithms/README.md +++ b/tig-algorithms/README.md @@ -6,25 +6,145 @@ Each submissions is committed to their own branch with the naming pattern: `\` -## Getting Started with Innovating +## Table of Contents -See [the guide](../docs/guides/innovating.md) +1. [Quick Start](#quick-start) +2. [Developer Environment](#developer-environment) +3. [Improving Algorithms](#improving-algorithms) +4. [GPU Algorithms](#gpu-algorithms) +5. [Attributing Breakthroughs](#attributing-breakthroughs) +6. [License](#license) -## Downloading an Algorithm +# Quick Start -Shared objects and ptx for an algorithm are stored in the `lib/` subfolder and can be downloaded via: +1. Clone this repo + ``` + git clone https://github.com/tig-foundation/tig-monorepo + ``` -`https://raw.githubusercontent.com/tig-foundation/tig-monorepo//tig-algorithms/lib/.tar.gz` +2. Create an algorithm + * Create your algorithm file `tig-algorithms/src/satisfiability/my_algo.rs` and copy/paste [this code](https://github.com/tig-foundation/tig-monorepo/blob/test/satisfiability/schnoing/tig-algorithms/src/satisfiability/schnoing/benchmarker_outbound.rs) + * Edit `tig-algorithms/src/satisfiability/mod.rs` and add the line: + ``` + pub mod my_algo; + ``` -## Algorithm Submission Flow +3. Compile your algorithm: + ``` + cd tig-monorepo -1. New submissions get their branch pushed to a private version of this repository -2. CI will compile submissions into shared objects and ptx -3. A new submission made during round `X` will have its branch pushed to the public version of this repository at the start of round `X + 2` -4. At the start of round `X + 4`, the submission becomes active, where benchmarkers can use the algorithm for benchmarking -5. Every block, algorithms with at least 25% adoption earn rewards and a merge point -6. At the end of a round, a algorithm from each challenge with the most merge points, meeting the minimum threshold of 5040, gets merged to the `main` branch - * Merged algorithms receive rewards every block where their adoption is greater than 0% + CHALLENGE=satisfiability + VERSION=0.0.1 + docker run -it -v $(pwd):/app ghcr.io/tig-foundation/tig-monorepo/$CHALLENGE/dev:$VERSION + + # inside docker + build_algorithm my_algo + ``` + +4. Test your algorithm: + ``` + # inside docker + test_algorithm my_algo [100,300] + ``` + * Use `--help` to see more options + * Use `--verbose` to dump individual instance commands that you can run. Example: + ``` + /usr/local/bin/tig-runtime '{"algorithm_id":"","challenge_id":"c001","difficulty":[50,300],"block_id":"","player_id":""}' rand_hash 99 ./tig-algorithms/lib/satisfiability/arm64/my_algo.so --fuel 100000000000 + ``` + +5. Submitting your algorithm to Testnet: + * Add and edit the following license header to `my_algo.rs`: + ``` + Copyright [year copyright work created] [name of copyright owner] + + Identity of Submitter [name of person or entity that submits the Work to TIG] + + UAI [UAI (if applicable)] + + Licensed under the TIG Inbound Game License v2.0 or (at your option) any later + version (the "License"); you may not use this file except in compliance with the + License. You may obtain a copy of the License at + + https://github.com/tig-foundation/tig-monorepo/tree/main/docs/licenses + + Unless required by applicable law or agreed to in writing, software distributed + under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR + CONDITIONS OF ANY KIND, either express or implied. See the License for the specific + language governing permissions and limitations under the License. + ``` + + * See [Attributing Breakthroughs](#attributing-breakthroughs) for how to set UAI field + * Visit https://test.tig.foundation/innovation/submission and follow the instructions + * Each new address on testnet gets 10 TIG balance + * Algorithms on testnet can be used straight away after they are successfully compiled + +6. Try benchmarking! See [Benchmarking Quick Start](../tig-benchmarker/README.md#quick-start) + +# Developer Environment + +We recommend developing using [Visual Studio Code](https://code.visualstudio.com/) with Rust plugins: +* rust-analyzer +* Even Better TOML +* crates +* CodeLLDB + +# Improving Algorithms + +All algorithms in TIG are open! You can develop your own de-novo algorithm or improve upon existing ones. + +**Tip:** there is a `tig-algorithms/src//template.rs` for each challenge. Inside it contains valuable tips on how to make your algorithm deterministic. Determinism is important in TIG because results need to be reproducible by everyone! + +## Challenge Descriptions + +See the [README in tig-challenges](../tig-challenges/README.md) for links to descriptions for each challenge. + +## Obtaining Code + +Every algorithm has its own `` with name `/`. You should visit [the repository](https://github.com/tig-foundation/tig-monorepo) to look at code submitted by other Innovators (look in `tig-algorithm/src//`). + +## Compiling & Testing Your Algorithm + +See [Quick Start](README.md#quick-start) + +## Testing Existing Algorithms + +TIG has a [dev docker image](../README.md#docker-images) for each challenge. To test an existing algorithm, you can edit & follow this example: + +``` +CHALLENGE=satisfiability +VERSION=0.0.1 +docker run -it -v $(pwd):/app ghcr.io/tig-foundation/tig-monorepo/$CHALLENGE/dev:$VERSION + +# inside docker +list_algorithms --testnet +download_algorithm sat_global_opt --testnet +test_algorithm sat_global_opt [1000,300] +``` + +# GPU Algorithms + +For challenges `vector_search` and `hypergraph`, a GPU with CUDA 12.6.3+ is required. + +Additionally, you will need to run your docker with `--gpus all` option. Example: + +``` +CHALLENGE=vector_search +VERSION=0.0.1 +docker run -it --gpus all -v $(pwd):/app ghcr.io/tig-foundation/tig-monorepo/$CHALLENGE/dev:$VERSION + +# inside docker +nvidia-smi +``` + +To be competitive in these challenges, you should develop cuda kernels in a `.cu` file which can be launched from your rust `.rs` code. + +# Attributing Breakthroughs + +Innovators in TIG can also [submit breakthroughs](../tig-breakthroughs/README.md) (algorithmic methods). + +If your implementation is based on breakthrough that has been submitted to TIG, you must attribute your implementation to it (example UAI: `c001_b001`) +* UAI of a method is detailed inside `tig-breakthroughs//.md` +* If your implementation is based on an algorithmic method outside of TIG, set UAI to `null` # License diff --git a/tig-benchmarker/README.md b/tig-benchmarker/README.md index 0c7a153f..6a970f8f 100644 --- a/tig-benchmarker/README.md +++ b/tig-benchmarker/README.md @@ -1,89 +1,149 @@ # tig-benchmarker -Benchmarker for TIG. Expected setup is a single master and multiple slaves on different servers. +Benchmarker for TIG. Designed to run with a single master and multiple slaves distributed across servers. -## Overview +## Table of Contents -Benchmarking in TIG works as follows: +1. [Quick Start](#quick-start) +2. [Architecture Overview](#architecture-overview) +3. [Configuration Details](#configuration-details) + * [.env file](#env-file) + * [Master Config](#master-config) + * [Slave Config](#slave-config) +4. [Hard Resetting](#hard-resetting) +5. [Finding your API Key](#finding-your-api-key) +6. [License](#license) -1. Master submits a precommit to TIG protocol, with details of what they will benchmark: - * `block_id`: start of the benchmark - * `player_id`: address of the benchmarker - * `challenge_id`: challenge to benchmark - * `algorithm_id`: algorithm to benchmark - * `difficulty`: difficulty of instances to randomly generate - * `num_nonces`: number of instances to benchmark +# Quick Start -2. TIG protocol confirms the precommit, and assigns it a random string +1. Obtain a Testnet API Key (see last section) + * Each new address on testnet gets 10 TIG balance + * For more tokens, use testnet faucet at https://tigstats.com/faucet -3. Master starts benchmarking: - * polls TIG protocol for the confirmed precommit + random string - * creates a benchmark job, splitting it into batches - * slaves poll the master for batches to compute - * slaves do the computation, and send results back to master +2. Clone this repo + ``` + git clone https://github.com/tig-foundation/tig-monorepo + cd tig-monorepo/tig-benchmarker + ``` -4. After benchmarking is finished, master submits benchmark to TIG protocol: - * `solution_nonces`: list of nonces for which a solution was found - * `merkle_root`: Merkle root of the tree constructed using results as leafs +3. Start a master + * If port 80 is in use, modify `UI_PORT` in your `.env` file + ``` + docker-compose -f master.yml up + ``` -5. TIG protocol confirms the benchmark, and randomly samples nonces requiring proof +4. Start a slave (in a separate terminal) + ``` + docker-compose -f slave.yml up slave satisfiability + ``` -6. Master prepares the proof: - * polls TIG protocol for confirmed benchmark + sampled nonces - * creates proof jobs - * slaves poll the master for nonces requiring proof - * slaves send Merkle branches back to master +5. Configure the master: + * Visit `http://localhost/config` (or `http://localhost:/config` if `UI_PORT` was changed) + * Paste and edit the following config: + * Replace `player_id` with your wallet address + * Replace `api_key` with your API key from step 1 + * Press **Save** after updating + ``` + { + "player_id": "0x0000000000000000000000000000000000000000", + "api_key": "00000000000000000000000000000000", + "api_url": "https://testnet-api.tig.foundation", + "time_between_resubmissions": 60000, + "max_concurrent_benchmarks": 4, + "algo_selection": [ + { + "algorithm_id": "c001_a004", + "num_nonces": 40, + "difficulty_range": [0, 0.5], + "selected_difficulties": [], + "weight": 1, + "batch_size": 8 + } + ], + "time_before_batch_retry": 60000, + "slaves": [ + { + "name_regex": ".*", + "algorithm_id_regex": ".*", + "max_concurrent_batches": 1 + } + ] + } + ``` -7. Master submits proof to TIG protocol +6. Sit back and watch as benchmarks are submitted! + * You can list available algorithms using: + ``` + docker exec satisfiability list_algorithms --testnet + ``` -8. TIG protocol confirms the proof, calculating the block from which the solutions will become "active" (eligible to earn rewards) - * Verification is performed in parallel - * Solutions will be inactive 120 blocks from when the benchmark started - * The delay is determined by number of blocks between the start and when proof was confirmed - * Each block, active solutions which qualify will earn rewards for the Benchmarker +# Architecture Overview -# Starting Your Master +* The **Master** defines benchmark scheduling strategy: + * Algorithm selection + * Difficulty selection + * Batch sizing + * Slave assignment + ``` + docker-compose -f master.yml up + ``` -Simply run: +* A **Slave** benchmarks specific challenges + ``` + docker-compose -f slave.yml up slave [challenge] .. [challenge] + ``` + * CPU challenges include: `satisfiability`, `vehicle_routing`, and `knapsack`U + * GPU challenges (requires CUDA 12.6.3+) include: `vector_search`, and `hypergraph` + * `slave/config.json` controls how many algorithms are ran concurrently + +# Configuration Details + +## `.env` file + +Shared by both `master.yml` and `slave.yml`: ``` -docker-compose up --build -``` +# Version of all benchmarker containers +VERSION=0.0.1 +# Set to 1 to enable verbose logging +VERBOSE=1 -This uses the `.env` file: - -``` POSTGRES_USER=postgres POSTGRES_PASSWORD=mysecretpassword POSTGRES_DB=postgres UI_PORT=80 DB_PORT=5432 + +# This is used by both master and slave MASTER_PORT=5115 -VERBOSE= +# This is used by slave to connect to master. Set to 172.17.0.1 if master and slave are running on same server +MASTER_IP=172.17.0.1 + +# Path to config file for slave. Mounts to /app/config.json inside slave container +SLAVE_CONFIG=./slave/config.json +# Directory for slave to download algorithms. Mounts to /app/algorithms inside slave containers +ALGORITHMS_DIR=./algorithms +# Directory for slave to store results. Mounts to /app/results inside slave containers +RESULTS_DIR=./results +# Seconds for results to live +TTL=300 +# Name of the slave. Defaults to randomly generated name +SLAVE_NAME= +# How many worker threads to spawn in the slave container +NUM_WORKERS=8 ``` -See last section on how to find your player_id & api_key. - -**Notes:** -* Interaction with the master is via UI: `http://localhost` - * If your UI port is not 80, then your UI is accessed via `http://localhost:` - * If you are running on a server, then your UI is access via: `http://` - * Alternatively, you can [ssh port forward](https://www.ssh.com/academy/ssh/tunneling-example) -* The config of the master can be updated via the UI -* Recommend to run dockers in detached mode: `docker-compose up --detach` -* You can view the logs of each service individually: `docker-compose logs -f ` - * There are 4 services: `db`, `master`, `ui`, `nginx` -* To query the database, recommend to use [pgAdmin](https://www.pgadmin.org/) - -## Hard Resetting Your Master - -1. Kill the services: `docker-compose down` -2. Delete the database: `rm -rf db_data` -3. Start your master +Common variables to customise: +1. `VERBOSE=` (empty string for quieter logging) +2. `POSTGRES_PASSWORD` +3. `MASTER_IP` (set to `172.17.0.1` for slaves on same host as master) +4. `MASTER_PORT` +4. `SLAVE_NAME` (must be a unique name, or else master will assign duplicate batches) +5. `NUM_WORKERS` (number of worker threads on a slave) ## Master Config -The master config defines how benchmarking jobs are selected, scheduled, and distributed to slaves. This config can be edited via the master UI or via API (`http://localhost:/update-config`). +The master config defines how benchmarking jobs are selected, scheduled, and distributed to slaves. This config can be edited via the master UI `/config` or via API `/update-config`. ```json { @@ -121,102 +181,67 @@ The master config defines how benchmarking jobs are selected, scheduled, and dis * `time_between_resubmissions`: Time in milliseconds to wait before resubmitting an benchmark/proof which has not confirmed into a block * `max_concurrent_benchmarks`: Maximum number of benchmarks that can be "in flight" at once (i.e., benchmarks where the proof has not been computed yet). * `algo_selection`: list of algorithms that can be picked for benchmarking. Each entry has: - * `algorithm_id`: id for the algorithm (e.g., c001_a001). Tip: use [list_algorithms](../scripts/list_algorithms.py) script to get list of algorithm ids + * `algorithm_id`: id for the algorithm (e.g., c001_a001) * `num_nonces`: Number of instances to benchmark for this algorithm * `difficulty_range`: the bounds (0.0 = easiest, 1.0 = hardest) for a random difficulty sampling. Full range is `[0.0, 1.0]` * `selected_difficulties`: A list of difficulties `[[x1,y1], [x2, y2], ...]`. If any of the difficulties are in valid range, one will be randomly selected instead of sampling from the difficulty range * `weight`: Selection weight. An algorithm is chosen proportionally to `weight / total_weight` * `batch_size`: Number of nonces per batch. Must be a power of 2. For example, if num_nonces = 40 and batch_size = 8, the benchmark is split into 5 batches -# Connecting Slaves - -1. Run the appropiate [runtime docker image](https://github.com/tig-foundation/tig-monorepo/pkgs/container/tig-monorepo%2Fruntime) for your slave. Available flavours are: - * amd64 (x86_64 compatible) - * aarch64 - * amd64-cuda12.6.3 (x86_64 compatible) - * aarch64-cuda12.6.3 - ``` - # example - docker run -it --gpus all ghcr.io/tig-foundation/tig-monorepo/runtime:0.0.1-amd64-cuda12.6.3 - ``` - -2. Run `slave.py`: - ``` - # runtime docker container should start you in /app - python3 slave.py --help - ``` - -**Notes:** -* If your master is on a different server, add `--master ` -* Set a custom master port with `--port ` -* To see all options, use `--help` - ## Slave Config -You can control execution limits via a JSON config: +The slave config lives under `slave/config.json`. It controls concurrency for algorithms using cost limits: ```json { - "max_workers": 100, + "max_cost": 100, "algorithms": [ { "id_regex": ".*", - "cpu": 1.0, - "gpu": 0.0 + "cost": 1.0 } ] } ``` **Explanation:** -* By default, `slave.py` uses all CPUs and all GPUs. To impose limits you should set the environment variables: - * `CPU_VISIBLE_CORES`. e.g. you have 8 CPUs, to only expose the first and last: `export CPU_VISIBLE_CORES=0,7` - * `CUDA_VISIBLE_DEVICES`. e.g. you have 4 GPUs, to only expose the first and second: `export CUDA_VISIBLE_DEVICES=0,1` -* `max_workers`: maximum concurrent tig-runtime processes. +* `max_cost`: maximum total "cost" of running algorithms. * `algorithms`: rules for matching algorithms based on `id_regex`. * Regex matches algorithm ids (e.g., `c004_a[\d3]` matches all vector_search algorithms). - * An algorithm + nonce only starts being processed if: - ``` - TOTAL_USAGE["cpu"] + cpu <= len(VISIBLE_CPUS) and - TOTAL_USAGE["gpu"] + gpu <= len(VISIBLE_GPUS) - ``` - * Total costs gets adjusted when processing starts and ends - ``` - # when processing starts - TOTAL_USAGE["cpu"] += cpu - TOTAL_USAGE["gpu"] += gpu - - # processing logic - - # when processing ends - TOTAL_USAGE["cpu"] -= cpu - TOTAL_USAGE["gpu"] -= gpu - ``` + * An algorithm only starts running if the total cost is below the limit **Example:** -This example limits c001/c002/c003 to 2 concurrent instances per CPU. It also limits c004/c005 to 4 concurrent instances per GPU: +This example means that up to 10 satisfiability (c001) algorithms can be ran concurrently, or up to 5 vehicle_routing (c002) algorithms, or some combination of both: ```json { - "max_workers": 10, - "cpus": 4, - "gpus": 2, + "max_cost": 10, "algorithms": [ { - "id_regex": "c00[123].*", - "cpu_cost": 0.5, + "id_regex": "c001.*", + "cpu_cost": 1.0, "gpu_cost": 0.0 }, { - "id_regex": "c00[45].*", - "cpu_cost": 0.0, - "gpu_cost": 0.25 + "id_regex": "c002.*", + "cpu_cost": 2.0, + "gpu_cost": 0.0 } ] } ``` +# Hard Resetting + +To hard reset master: +1. Kill the services `docker-compose -f master.yml down` +2. Delete the database: `rm -rf db_data` + +To hard reset slave: +1. Kill the services `docker-compose -f slave.yml down` +2. Delete the data: `rm -rf algorithms results` + # Finding your API Key ## Mainnet diff --git a/tig-benchmarker/docker-compose-master.yml b/tig-benchmarker/master.yml similarity index 100% rename from tig-benchmarker/docker-compose-master.yml rename to tig-benchmarker/master.yml diff --git a/tig-benchmarker/docker-compose-slave.yml b/tig-benchmarker/slave.yml similarity index 100% rename from tig-benchmarker/docker-compose-slave.yml rename to tig-benchmarker/slave.yml diff --git a/tig-binary/README.md b/tig-binary/README.md index adf224e6..ee9af6bb 100644 --- a/tig-binary/README.md +++ b/tig-binary/README.md @@ -8,30 +8,47 @@ For CUDA, TIG uses `nvcc` to generate ptx using target version `sm_70/compute_70 # Getting Started -1. Run the appropiate [development docker image](https://github.com/tig-foundation/tig-monorepo/pkgs/container/tig-monorepo%2Fdev). Available flavours are: - * amd64 (x86_64 compatible) - * aarch64 - * amd64-cuda12.6.3 (x86_64 compatible) - * aarch64-cuda12.6.3 +1. Run the [development docker image](../README.md#docker-images) for your challenge. Example: ``` -# example -docker run -it -v $(pwd):/app ghcr.io/tig-foundation/tig-monorepo/dev:0.0.1-aarch64 -# scripts build_so and build_ptx are on PATH +# clone this repo +cd tig-monorepo +CHALLENGE=knapsack +VERSION=0.0.1 +docker run -it -v $(pwd):/app ghcr.io/tig-foundation/tig-monorepo/$CHALLENGE/dev:$VERSION ``` -2. Build shared object using `build_so` script: +2. Build shared object using `build_algorithm` script: * Expects `tig_algorithm::::::solve_challenge` to be importable - * Outputs to `tig-algorithms/lib///.so`, where `ARCH` is aarch64 or amd64 + * Outputs to `tig-algorithms/lib///.so`, where `ARCH` is arm64 or amd64 + * `CHALLENGE` is determined by your docker image ``` - # add '--cuda' flag if building cuda algorithm - build_so $CHALLENGE $ALGORITHM + build_algorithm $ALGORITHM ``` -3. (Optional) Build ptx using `build_ptx` script: +Notes: +* To specifically build the shared object, you can use `build_so` script which is on `PATH` +* To specifically build the ptx, you can use `build_ptx` script which is on `PATH` * Expects `tig_algorithm/src//.cu` or `tig_algorithm/src///benchmarker_outbound.cu` file to exist * Outputs to `tig-algorithms/lib//ptx/.ptx` +* To test your binary, you can use `test_algorithm` script which is on `PATH`. Example: + ``` + build_algorithm sat_global_opt + test_algorithm sat_global_opt [1000,300] + ``` + +## Complete Example + ``` -build_ptx $CHALLENGE $ALGORITHM +git clone https://github.com/tig-foundation/tig-monorepo -b test/satisfiability/sat_global_opt +cd tig-monorepo + +CHALLENGE=satisfiability +VERSION=0.0.1 +docker run -it -v $(pwd):/app ghcr.io/tig-foundation/tig-monorepo/$CHALLENGE/dev:$VERSION + +# inside docker +build_algorithm sat_global_opt +test_algorithm sat_global_opt [1000,300] ``` # License diff --git a/tig-challenges/README.md b/tig-challenges/README.md index 46f9af7c..24812f37 100644 --- a/tig-challenges/README.md +++ b/tig-challenges/README.md @@ -2,14 +2,12 @@ A Rust crate that contains the implementation of TIG's challenges (computational problems adapted for proof-of-work). -Presently, TIG features four challenges: -1. [Boolean satisfiability](../docs/challenges/satisfiability.md) -2. [Capacitated vehicle routing](../docs/challenges/vehicle_routing.md) -3. [Knapsack problem](../docs/challenges/knapsack.md) -4. [Vector range search](../docs/challenges/vector_search.md) -5. [Hypergraph partitioning](../docs/challenges/hypergraph.md) - -Over the coming year, an additional seven challenges from domains including artificial intelligence, biology, medicine, and climate science will be phased in. +Presently, TIG features the following challenges: +1. [Boolean satisfiability](./src/satisfiability.md) +2. [Capacitated vehicle routing](./src/vehicle_routing.md) +3. [Knapsack problem](./src/knapsack.md) +4. [Vector range search](./src/vector_search.md) +5. [Hypergraph partitioning](./src/hypergraph.md) # License diff --git a/docs/challenges/hypergraph.md b/tig-challenges/src/hypergraph.md similarity index 100% rename from docs/challenges/hypergraph.md rename to tig-challenges/src/hypergraph.md diff --git a/docs/challenges/knapsack.md b/tig-challenges/src/knapsack.md similarity index 100% rename from docs/challenges/knapsack.md rename to tig-challenges/src/knapsack.md diff --git a/docs/challenges/satisfiability.md b/tig-challenges/src/satisfiability.md similarity index 100% rename from docs/challenges/satisfiability.md rename to tig-challenges/src/satisfiability.md diff --git a/docs/challenges/vector_search.md b/tig-challenges/src/vector_search.md similarity index 100% rename from docs/challenges/vector_search.md rename to tig-challenges/src/vector_search.md diff --git a/docs/challenges/vehicle_routing.md b/tig-challenges/src/vehicle_routing.md similarity index 100% rename from docs/challenges/vehicle_routing.md rename to tig-challenges/src/vehicle_routing.md diff --git a/tig-runtime/README.md b/tig-runtime/README.md index 4d0fc412..a56f7d5c 100644 --- a/tig-runtime/README.md +++ b/tig-runtime/README.md @@ -4,50 +4,36 @@ A Rust crate that execute an algorithm (compiled from [`tig-binary`](../tig-bina # Getting Started -Users who don't intend to customise `tig-runtime` are recommended to download pre-compiled version available in [TIG's runtime docker images](https://github.com/tig-foundation/tig-monorepo/pkgs/container/tig-monorepo%2Fruntime). +Users who don't intend to customise `tig-runtime` are recommended to download pre-compiled version available in [TIG's runtime docker images](../README.md#docker-images). + +Note there is a different `tig-runtime` for each challenge. **Example:** ``` -docker run -it ghcr.io/tig-foundation/tig-monorepo/runtime:0.0.1-aarch64 -# tig-runtime is already on PATH +CHALLENGE=knapsack +VERSION=0.0.1 +docker run -it ghcr.io/tig-foundation/tig-monorepo/$CHALLENGE/runtime:$VERSION + +# inside docker +tig-runtime --help ``` -## Compiling (using dev docker image) +## Compiling -The required rust environment for development are available via [TIG's development docker images](https://github.com/tig-foundation/tig-monorepo/pkgs/container/tig-monorepo%2Fdev). +The required rust environment for development are available via [TIG's development docker images](../README.md#docker-images). +You will need to add `--features ` to compile for a specific challenge. **Example:** ``` -docker run -it -v $(pwd):/app ghcr.io/tig-foundation/tig-monorepo/dev:0.0.1-aarch64 -# cargo build -p tig-runtime --release -``` +# clone this repo +cd tig-monorepo +CHALLENGE=knapsack +VERSION=0.0.1 +docker run -it -v $(pwd):/app ghcr.io/tig-foundation/tig-monorepo/$CHALLENGE/dev:$VERSION -## Compiling (local setup) - -Users who intend to customise `tig-runtime` need to install a specific version of rust: - -1. Install rust version `nightly-2025-02-10` -``` -ARCH=$(uname -m) -RUST_TARGET=$(if [ "$ARCH" = "aarch64" ] || [ "$ARCH" = "arm64" ]; then - echo "aarch64-unknown-linux-gnu"; -else - echo "x86_64-unknown-linux-gnu"; -fi) -rustup install nightly-2025-02-10 -rustup default nightly-2025-02-10 -rustup component add rust-src -rustup target add $RUST_TARGET -RUST_LIBDIR=$(rustc --print target-libdir --target=$RUST_TARGET) -ln -s $RUST_LIBDIR /usr/local/lib/rust -echo "export LD_LIBRARY_PATH=\"${LD_LIBRARY_PATH}:/usr/local/lib/rust\"" >> ~/.bashrc -``` - -2. Compile `tig-runtime` -``` -# for cuda version, add --features cuda -cargo build -p tig-runtime --release --target $RUST_TARGET +# inside docker +cargo build -p tig-runtime --release --features knapsack ``` # Usage @@ -72,24 +58,37 @@ Options: The following exit codes indicate specific meanings: * 0 - solution found +* 84 - runtime error * 85 - no solution found -* 86 - runtime error +* 86 - invalid solution * 87 - out of fuel **Example:** ``` +CHALLENGE=satisfiability +VERSION=0.0.1 +docker run -it -v $(pwd):/app ghcr.io/tig-foundation/tig-monorepo/$CHALLENGE/runtime:$VERSION + +# inside docker +download_algorithm sat_global_opt --testnet + +ARCH=$(if [ "$(uname -i)" = "aarch64" ] || [ "$(uname -i)" = "arm64" ] || [ "$(arch 2>/dev/null || echo "")" = "aarch64" ] || [ "$(arch 2>/dev/null || echo "")" = "arm64" ]; then + echo "arm64" +else + echo "amd64" +fi) SETTINGS='{"challenge_id":"c001","difficulty":[50,300],"algorithm_id":"","player_id":"","block_id":""}' RANDHASH='rand_hash'$ NONCE=1337 FUEL=987654321123456789 -SO_PATH=./tig-algorithms/lib/satisfiability/aarch64/better_sat.so +SO_PATH=./tig-algorithms/lib/satisfiability/$ARCH/sat_global_opt.so tig-runtime $SETTINGS $RANDHASH $NONCE $SO_PATH --fuel $FUEL ``` **Example Output:** ``` -{"cpu_arch":"aarch64","fuel_consumed":95496,"nonce":1337,"runtime_signature":4125818588297548058,"solution":{"variables":[0,1,0,1,0,0,0,0,0,1,0,0,0,1,1,1,0,1,1,1,0,0,0,0,1,1,0,1,1,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,1,1,1,0,0]}} +{"cpu_arch":"arm64","fuel_consumed":97188,"nonce":1337,"runtime_signature":13607024390209669967,"solution":{"variables":[1,0,0,0,0,1,1,1,0,1,0,0,0,0,0,1,0,1,0,1,0,0,0,0,0,1,1,1,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,1,0,0,0,0]}} ``` # License diff --git a/tig-verifier/README.md b/tig-verifier/README.md index 1372c9bd..68787932 100644 --- a/tig-verifier/README.md +++ b/tig-verifier/README.md @@ -4,50 +4,36 @@ A Rust crate that verifies a single solution or Merkle proof. # Getting Started -Users who don't intend to customise `tig-verifier` are recommended to download pre-compiled version available in [TIG's runtime docker images](https://github.com/tig-foundation/tig-monorepo/pkgs/container/tig-monorepo%2Fruntime). +Users who don't intend to customise `tig-verifier` are recommended to download pre-compiled version available in [TIG's runtime docker images](../README.md#docker-images). + +Note there is a different `tig-verifier` for each challenge. **Example:** ``` -docker run -it ghcr.io/tig-foundation/tig-monorepo/runtime:0.0.1-aarch64 -# tig-verifier is already on PATH +CHALLENGE=knapsack +VERSION=0.0.1 +docker run -it ghcr.io/tig-foundation/tig-monorepo/$CHALLENGE/runtime:$VERSION + +# inside docker +tig-verifier --help ``` -## Compiling (using dev docker image) +## Compiling -The required rust environment for development are available via [TIG's development docker images](https://github.com/tig-foundation/tig-monorepo/pkgs/container/tig-monorepo%2Fdev). +The required rust environment for development are available via [TIG's development docker images](../README.md#docker-images). +You will need to add `--features ` to compile for a specific challenge. **Example:** ``` -docker run -it -v $(pwd):/app ghcr.io/tig-foundation/tig-monorepo/dev:0.0.1-aarch64 -# cargo build -p tig-verifier --release -``` +# clone this repo +cd tig-monorepo +CHALLENGE=knapsack +VERSION=0.0.1 +docker run -it -v $(pwd):/app ghcr.io/tig-foundation/tig-monorepo/$CHALLENGE/dev:$VERSION -## Compiling (local setup) - -Users who intend to customise `tig-verifier` need to install a specific version of rust: - -1. Install rust version `nightly-2025-02-10` -``` -ARCH=$(uname -m) -RUST_TARGET=$(if [ "$ARCH" = "aarch64" ] || [ "$ARCH" = "arm64" ]; then - echo "aarch64-unknown-linux-gnu"; -else - echo "x86_64-unknown-linux-gnu"; -fi) -rustup install nightly-2025-02-10 -rustup default nightly-2025-02-10 -rustup component add rust-src -rustup target add $RUST_TARGET -RUST_LIBDIR=$(rustc --print target-libdir --target=$RUST_TARGET) -ln -s $RUST_LIBDIR /usr/local/lib/rust -echo "export LD_LIBRARY_PATH=\"${LD_LIBRARY_PATH}:/usr/local/lib/rust\"" >> ~/.bashrc -``` - -2. Compile `tig-verifier` -``` -# for cuda version, add --features cuda -cargo build -p tig-verifier --release --target $RUST_TARGET +# inside docker +cargo build -p tig-verifier --release --features knapsack ``` # Usage @@ -71,6 +57,11 @@ Options: **Example:** ``` +CHALLENGE=satisfiability +VERSION=0.0.1 +docker run -it ghcr.io/tig-foundation/tig-monorepo/$CHALLENGE/runtime:$VERSION + +# inside docker SETTINGS='{"challenge_id":"c001","difficulty":[50,300],"algorithm_id":"","player_id":"","block_id":""}' RANDHASH='rand_hash' NONCE=1337