coosld/tig-monorepopool
1
0
Fork 0
mirror of https://github.com/tig-pool-nk/tig-monorepo.git synced 2026-02-21 17:07:20 +08:00
Code Issues Actions Packages Projects Releases Wiki Activity

Update documentation for native runtime.

Browse Source
This commit is contained in:
FiveMovesAhead 2025-05-06 06:48:27 +01:00
parent c85d0f79f1
commit 4135a6bfa1
10 changed files with 458 additions and 193 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

7
README.md
View File

@ -6,7 +6,6 @@ This repository contains the implementation of The Innovation Game (TIG).
* [TIG Documentation](https://docs.tig.foundation/)
* [TIG Whitepaper](docs/whitepaper.pdf)
* [TIG Tech Explainer](docs/tech/1_basics.md)
* [TIG Licensing Explainer](docs/guides/anatomy.md)
* [Getting Started with Innovating](docs/guides/innovating.md)
* [Implementations vs Breakthroughs](docs/guides/breakthroughs.md)
@ -16,14 +15,16 @@ This repository contains the implementation of The Innovation Game (TIG).
* [tig-algorithms](./tig-algorithms/README.md) - A Rust crate that hosts algorithm submissions made by Innovators in TIG
* [tig-benchmarker](./tig-benchmarker/README.md) - Python scripts for running TIG's benchmarker in master/slave configuration
* [tig-binary](./tig-binary/README.md) - A Rust crate that wraps an algorithm from [`tig-algorithm`](./tig-algorithms/README.md) for compilation into a shared object.
* [tig-breakthroughs](./tig-breakthroughs/README.md) - A folder that hosts submissions of algorithmic methods made by Innovators in TIG.
* [tig-challenges](./tig-challenges/README.md) - A Rust crate that contains the implementation of TIG's challenges (computational problems adapted for proof-of-work)
* [tig-protocol](./tig-protocol/README.md) - A Rust crate that contains the implementation of TIG's core protocol logic.
* [tig-runtime](./tig-runtime/README.md) - A Rust crate that execute an algorithm (compiled from [`tig-binary`](./tig-binary/README.md)) for a single nonce, generating runtime signature and fuel consumed for verification.
* [tig-structs](./tig-structs/README.md) - A Rust crate that contains the definitions of structs used throughout TIG
* [tig-token](./tig-token/README.md) - Solidity contract for TIG's ERC20 token that is deployed on Ethereum L2 Base chain
* [tig-utils](./tig-utils/README.md) - A Rust crate that contains utility functions used throughout TIG
* [tig-wasm](./tig-wasm/README.md) - A Rust crate for wrapping algorithm submissions for compilation into WASM with an exported `entry_point`
* [tig-worker](./tig-worker/README.md) - A Rust crate for verifying and computing solutions
* [tig-verifier](./tig-verifier/README.md) - A Rust crate that verifies a single solution or Merkle proof.
* [tig-worker](./tig-worker/README.md) - A Rust crate for executing a batch of instances using [`tig-runtime`](./tig-runtime/README.md), aggregating outputs, and calculating Merkle root.
## Useful Scripts

193
docs/guides/innovating.md
View File

@ -2,9 +2,9 @@
## Developer Environment
TIG compiles all algorithms into WASM for sandboxing and for verifiable execution.
TIG compiles all algorithms into shared objects and ptx for verifiable execution.
TIG currently requires all algorithms to be written in [Rust](https://www.rust-lang.org/tools/install).
TIG currently requires all algorithms to be written in [Rust](https://www.rust-lang.org/tools/install). Algorithms can optionally have a .cu with Cuda kernels, which are launched from the Rust code.
We recommend developing using [Visual Studio Code](https://code.visualstudio.com/) with Rust plugins:
* rust-analyzer
@ -12,54 +12,29 @@ We recommend developing using [Visual Studio Code](https://code.visualstudio.com
* crates
* CodeLLDB
## Setting up Private Fork
Innovators will want to create a private fork so that they can test that their algorithm can be successfully compiled into WASM by the CI.
1. Create private repository on GitHub
2. Create empty git repository on your local machine
```
mkdir tig-monorepo
cd tig-monorepo
git init
```
3. Setup remotes with origin pointed to your private repository
```
git remote add origin <your private repo>
git remote add public https://github.com/tig-foundation/tig-monorepo.git
```
4. Pulling `blank_slate` from TIG public repository (branch with no algorithms)
```
git fetch public
git checkout -b blank_slate
git pull public blank_slate
```
5. Push to your private repository
```
git push origin blank_slate
```
## Checking out Existing Algorithms
Every algorithm has its own `<branch>` with name `<challenge_name>/<algorithm_name>`.
Only algorithms that are successfully compiled into WASM have their branch pushed to this public repository.
To pull an existing algorithm from tig-monorepo repository, run the following command:
```
git pull origin <branch>
```
Each algorithm branch will have 6 key files:
Only algorithms that are successfully compiled have their branch pushed to this public repository.
Each algorithm branch will have 6 key files (11 if there is CUDA code):
1. Rust code with TIG commercial license header @ `tig-algorithms/src/<branch>/commercial.rs`
2. Rust code with TIG open data license header @ `tig-algorithms/src/<branch>/open_data.rs`
3. Rust code with TIG benchmarker outbound license header @ `tig-algorithms/src/<branch>/benchmarker_outbound.rs`
4. Rust code with TIG innovator outbound license header @ `tig-algorithms/src/<branch>/innovator_outbound.rs`
5. Rust code with TIG inbound license header @ `tig-algorithms/src/<branch>/inbound.rs`
6. Wasm blob @ `tig-algorithms/wasm/<branch>.wasm`
To pull an existing algorithm from TIG public repository, run the following command:
```
git fetch public
git pull public <branch>
```
5. Cuda code with TIG inbound license header @ `tig-algorithms/src/<branch>/inbound.cu`
6. Tarball with shared objects and (optionally) ptx @ `tig-algorithms/lib/<branch>.tar.gz`
7. Cuda code with TIG commercial license header @ `tig-algorithms/src/<branch>/commercial.cu`
8. Cuda code with TIG open data license header @ `tig-algorithms/src/<branch>/open_data.cu`
9. Cuda code with TIG benchmarker outbound license header @ `tig-algorithms/src/<branch>/benchmarker_outbound.cu`
10. Cuda code with TIG innovator outbound license header @ `tig-algorithms/src/<branch>/innovator_outbound.cu`
11. Cuda code with TIG inbound license header @ `tig-algorithms/src/<branch>/inbound.cu`
## Developing Your Algorithm
@ -67,6 +42,7 @@ git pull public <branch>
1. Pick a challenge (`<challenge_name>`) to develop an algorithm for
2. Make a copy of `tig-algorithms/<challenge_name>/template.rs` or an existing algorithm (see notes)
* (Optional) for Cuda, additionally make a copy of `tig-algorithms/<challenge_name>/template.cu`
3. Make sure your file has the following notice in its header if you intend to submit it to TIG:
```
Copyright [year copyright work created] [name of copyright owner]
@ -97,11 +73,14 @@ language governing permissions and limitations under the License.
#[cfg(test)]
mod tests {
use super::*;
use tig_challenges::{<challenge_name>::*, *};
#[cfg(feature = "cuda")]
use cudarc::{driver::CudaContext, nvrtc::Ptx, runtime::result::device::get_device_prop};
use super::<algorithm_name>::solve_challenge;
use tig_challenges::<challenge_name>::*;
#[cfg(not(feature = "cuda"))]
#[test]
fn test_<algorithm_name>() {
fn test_algorithm() {
let difficulty = Difficulty {
// Uncomment the relevant fields.
// Modify the values for different difficulties
@ -117,14 +96,14 @@ language governing permissions and limitations under the License.
// -- knapsack --
// num_items: 50,
// better_than_baseline: 10,
// -- vector_search --
// num_queries: 10,
// better_than_baseline: 350,
};
let seed = [0u8; 32]; // change this to generate different instances
let challenge = Challenge::generate_instance(seed, &difficulty).unwrap();
match <algorithm_name>::solve_challenge(&challenge) {
match solve_challenge(&challenge) {
Ok(Some(solution)) => match challenge.verify_solution(&solution) {
Ok(_) => println!("Valid solution"),
Err(e) => println!("Invalid solution: {}", e),
@ -133,50 +112,142 @@ language governing permissions and limitations under the License.
Err(e) => println!("Algorithm error: {}", e),
};
}
#[cfg(feature = "cuda")]
#[test]
fn test_algorithm() {
let difficulty = Difficulty {
// Uncomment the relevant fields.
// Modify the values for different difficulties
// -- satisfiability --
// num_variables: 50,
// clauses_to_variables_percent: 300,
// -- vehicle_routing --
// num_nodes: 40,
// better_than_baseline: 250,
// -- knapsack --
// num_items: 50,
// better_than_baseline: 10,
// -- vector_search --
// num_queries: 10,
// better_than_baseline: 350,
};
let seed = [0u8; 32]; // change this to generate different instances
let ptx_path = "/app/tig-algorithms/lib/vector_search/ptx/simple_search.ptx"; // change this to the path of your PTX file
let ptx = Ptx::from_file(ptx_path);
let gpu_device = 0; // change this to the desired GPU device ID
let ctx = CudaContext::new(gpu_device).unwrap();
ctx.set_blocking_synchronize().unwrap();
let module = ctx.load_module(ptx).unwrap();
let stream = ctx.default_stream();
let prop = get_device_prop(gpu_device as i32).unwrap();
let challenge =
Challenge::generate_instance(seed, &difficulty, module.clone(), stream.clone(), &prop)
.unwrap();
match solve_challenge(&challenge, module.clone(), stream.clone(), &prop) {
Ok(Some(solution)) => {
match challenge.verify_solution(&solution, module.clone(), stream.clone(), &prop) {
Ok(_) => println!("Valid solution"),
Err(e) => println!("Invalid solution: {}", e),
}
}
Ok(None) => println!("No solution"),
Err(e) => println!("Algorithm error: {}", e),
};
}
}
```
* See notes for CUDA tests
6. Use the above test to debug your algorithm:
6. Run the appropiate [development docker image](https://github.com/tig-foundation/tig-monorepo/pkgs/container/tig-monorepo%2Fdev). Available flavours are:
* amd64 (x86_64 compatible)
* aarch64
* amd64-cuda12.6.3 (x86_64 compatible)
* aarch64-cuda12.6.3
```
cargo test -p tig-algorithms -- --nocapture
# example
docker run -it -v $(pwd):/app --gpus all ghcr.io/tig-foundation/tig-monorepo/dev:0.0.1-amd64-cuda12.6.3
```
6. If you have Cuda code, use `build_ptx.py` to compile it
```
build_ptx.py <CHALLENGE> <ALGORITHM>
```
7. Run the test
* No cuda:
```
cargo test -p tig-algorithms --release -- --nocapture
```
* With cuda:
```
cargo test -p tig-algorithms --release --features cuda -- --nocapture
```
**IMPORTANT (READ THIS):**
* Not all challenge instances have solutions. Algorithms that can detect such cases and exit early (`return Ok(None)`) will potentially have better performance than algorithms that don't exit early
* Not all challenge instances have solutions.
* You can find the current qualifying difficulties by:
* Query https://mainnet-api.tig.foundation/get-block for <block_id>
* Query https://mainnet-api.tig.foundation/get-challenges?block_id=<block_id> for <qualifier_difficulties>
* If you are copying and modifying an algorithm that has been submitted to TIG, make sure to use the `innovator_outbound` version
* Do not include tests in your algorithm file. TIG will reject your algorithm submission.
* Only your algorithm's rust code gets submitted. You should not be modifying `Cargo.toml` in `tig-algorithms`. Any extra dependencies you add will not be available when TIG compiles your algorithm
* Only your algorithm's code gets submitted. You should not be modifying `Cargo.toml` in `tig-algorithms`. Any extra dependencies you add will not be available when TIG compiles your algorithm
* If you need to use random number generation, ensure that it is seeded so that your algorithm is deterministic.
* Suggest to use `let mut rng = SmallRng::from_seed(StdRng::from_seed(challenge.seed).gen())`
## Locally Compiling Your Algorithm into WASM
## Locally Compiling Your Algorithm into Shared Object
See the [README](../../tig-wasm/README.md) for `tig-wasm`
See the [README](../../tig-binary/README.md) for `tig-binary`
## Testing Performance of Algorithms
See the [README](../../tig-worker/README.md) for `tig-worker`
TODO
## Checking CI Successfully Compiles Your Algorithm
TIG pushes all algorithms to their own branch which triggers the CI (`.github/workflows/build_algorithm.yml`).
To trigger the CI on your private repo, your branch just needs to have a particular name:
```
git checkout -b <challenge_name>/<algorithm_name>
git push origin <challenge_name>/<algorithm_name>
```
To replicate this, Innovators will want to create a private fork:
1. Create private repository on GitHub
2. Create empty git repository on your local machine
```
mkdir tig-monorepo
cd tig-monorepo
git init
```
3. Setup remotes with origin pointed to your private repository
```
git remote add origin <your private repo>
git remote add public https://github.com/tig-foundation/tig-monorepo.git
```
4. Pulling `blank_slate` from TIG public repository (branch with no algorithms)
```
git fetch public
git checkout -b blank_slate
git pull public blank_slate
```
5. Push to your private repository
```
git push origin blank_slate
```
6. Trigger the CI on your private repo with your algorithm branch:
```
git checkout -b <challenge_name>/<algorithm_name> blank_slate
git push origin <challenge_name>/<algorithm_name>
```
## Making Your Submission
10 TIG will be deducted from your Available Fee Balance to make a submission. You can topup via the [Benchmarker page](https://play.tig.foundation/benchmarker)
**IMPORTANT:**
* Submissions are final and cannot be modified after they are made
* Be sure to adhere to the notes in [developing your algorithm](#developing-your-algorithm)
* We highly recommend [compiling your algorithm into WASM](#locally-compiling-your-algorithm-into-wasm) and [testing its performance](#testing-performance-of-algorithms) with `tig-worker`.
* We highly recommend [checking the CI can compile your algorithm](#checking-ci-successfully-compiles-your-algorithm)

14
tig-algorithms/README.md
View File

@ -12,9 +12,9 @@ See [the guide](../docs/guides/innovating.md)
## Downloading an Algorithm
WASM blobs for an algorithm are stored in the `wasm` subfolder and can be downloaded via:
Shared objects and ptx for an algorithm are stored in the `lib/<CHALLENGE>` subfolder and can be downloaded via:
`https://raw.githubusercontent.com/tig-foundation/tig-monorepo/<branch_name>/tig-algorithms/wasm/<branch_name>.wasm`
`https://raw.githubusercontent.com/tig-foundation/tig-monorepo/<branch_name>/tig-algorithms/lib/<branch_name>.tar.gz`
## Algorithm Submission Flow
@ -30,8 +30,8 @@ WASM blobs for an algorithm are stored in the `wasm` subfolder and can be downlo
Each algorithm submission will have 5 versions, each under a specific license:
* `commercial.rs` will be under TIG commercial license
* `open_data.rs` will be under TIG open data license
* `benchmarker_outbound.rs` will be under TIG benchmarker outbound license
* `innovator_outbound.rs` will be under TIG innovator outbound license
* `inbound.rs` will be under TIG inbound license
* `commercial` will be under TIG commercial license
* `open_data` will be under TIG open data license
* `benchmarker_outbound` will be under TIG benchmarker outbound license
* `innovator_outbound` will be under TIG innovator outbound license
* `inbound` will be under TIG inbound license

53
tig-benchmarker/README.md
View File

@ -43,14 +43,20 @@ See last section on how to find your player_id & api_key.
# Connecting Slaves
1. Compile `tig-worker`:
1. Run the appropiate [runtime docker image](https://github.com/tig-foundation/tig-monorepo/pkgs/container/tig-monorepo%2Fruntime) for your slave. Available flavours are:
* amd64 (x86_64 compatible)
* aarch64
* amd64-cuda12.6.3 (x86_64 compatible)
* aarch64-cuda12.6.3
```
cargo build -p tig-worker --release
# example
docker run -it --gpus all ghcr.io/tig-foundation/tig-monorepo/runtime:0.0.1-amd64-cuda12.6.3
```
2. Run `slave.py`:
```
# assume you are in tig-benchmarker folder
python3 slave.py ../target/release/tig-worker
# runtime docker container should start you in /app
python3 slave.py --help
```
**Notes:**
@ -59,38 +65,9 @@ See last section on how to find your player_id & api_key.
* To use a different port, use the option `--port <MASTER_PORT>`
* To see all options, use `--help`
## Hard Resetting Your Slave
1. Stop your slave
2. Remove the output folder (defaults to results): `rm -rf results`
3. Start your slave
# Optimising your Config
1. `difficulty_sampler_config` allows you to configure the difficulty your benchmarker selects for each challenge:
* `selected_difficulties` expects a list of difficulties. If any of those difficulties are within the valid range, they will be picked
* `difficulty_ranges` expects a range defined by 2 numbers: `[lower, higher]`. This is used to sample a random difficulty in the valid range. Examples:
* `[0.0, 1.0]` samples the full range of valid difficulties
* `[0.0, 0.1]` samples the easiest 10% of valid difficulties
2. `job_manager_config` allows you to set the `batch_size` for each challenge.
* `batch_size` is the number of nonces that are part of a batch. Must be a power of 2
* Recommend to pick a `batch_size` for your slave with lowest `num_workers` such that it takes a few seconds to compute (e.g. 5 seconds)
* `batch_size` shouldn't be too small, or else network latency between master and slave will affect performance
* Use `slave_manager_config` to support slaves with different `num_workers`
3. `precommit_manager_config` allows you to control your benchmarks:
* `max_pending_benchmarks` is the maximum number of pending benchmarks. You dont want benchmarks to take too long, nor do you want your slaves to be idle too much
* `num_nonces` is the number of nonces to compute per benchmark. See Discord #community-tools channel for stats to help pick
* `weight` affects how likely the challenge will be picked (weight of 0 will never be picked). Observe your cutoff and imbalance on the [benchmarker page](https://play.tig.foundation/benchmarker)
4. `slave_manager_config` allows you to control your slaves:
* Slave names are matched to regexes (or else rejected).
* `max_concurrent_batches` is the max number of concurrent batches a slave can work on
* `selected_challenges` is a whitelist of challenges for that slave. If you don't want a slave to benchmark a specific challenge, remove its entry from the list
See [docs.tig.foundation](https://docs.tig.foundation/benchmarking/benchmarker-config)
# Finding your API Key
@ -98,17 +75,13 @@ See last section on how to find your player_id & api_key.
1. Navigate to https://play.tig.foundation/
2. Connect your wallet
3. Run the following command in the console: `JSON.parse(Cookies.get("account"))`
* `address` is your Mainnet `player_id`
* `api_key` is your Mainnet API key
3. Your API key can be copied from the bottom left corner of the dashboard
## Testnet
1. Navigate to https://test.tig.foundation/
2. Connect your wallet
3. Run the following command in the console: `JSON.parse(Cookies.get("account"))`
* `address` is your Testnet `player_id`
* `api_key` is your Testnet API key
3. Your API key can be copied from the bottom left corner of the dashboard
# License

1
tig-benchmarker/common/structs.py
View File

@ -103,6 +103,7 @@ class OutputData(FromDict):
runtime_signature: int
fuel_consumed: int
solution: dict
cpu_arch: str
def calc_solution_signature(self) -> int:
return u64s_from_str(jsonify(self.solution))[0]

39
tig-binary/README.md Normal file
View File

@ -0,0 +1,39 @@
# tig-binary
A Rust crate that wraps an algorithm from [`tig-algorithm`](../tig-algorithms/README.md) for compilation into a shared object.
TIG uses a forked version of [LLVM compiler](https://github.com/tig-foundation/llvm) for injecting runtime signature and fuel tracking into the shared object.
For CUDA, TIG uses `nvcc` to generate ptx using target version `sm_70/compute_70`, before injecting runtime signature and fuel tracking into the ptx file.
# Getting Started
1. Run the appropiate [development docker image](https://github.com/tig-foundation/tig-monorepo/pkgs/container/tig-monorepo%2Fdev). Available flavours are:
* amd64 (x86_64 compatible)
* aarch64
* amd64-cuda12.6.3 (x86_64 compatible)
* aarch64-cuda12.6.3
```
# example
docker run -it -v $(pwd):/app ghcr.io/tig-foundation/tig-monorepo/dev:0.0.1-aarch64
# scripts build_so.sh and build_ptx.py are on PATH
```
2. Build shared object using `build_so.sh` script:
* Expects `tig_algorithm::<CHALLENGE>::<ALGORITHM>::solve_challenge` to be importable
* Outputs to `tig-algorithms/lib/<CHALLENGE>/<ARCH>/<ALGORITHM>.so`, where `ARCH` is aarch64 or amd64
```
# add '--cuda' flag if building cuda algorithm
build_so.sh $CHALLENGE $ALGORITHM
```
3. (Optional) Build ptx using `build_ptx.py` script:
* Expects `tig_algorithm/src/<CHALLENGE>/<ALGORITHM>.cu` or `tig_algorithm/src/<CHALLENGE>/<ALGORITHM>/benchmarker_outbound.cu` file to exist
* Outputs to `tig-algorithms/lib/<CHALLENGE>/ptx/<ALGORITHM>.ptx`
```
build_ptx.py $CHALLENGE $ALGORITHM
```
# License
[End User License Agreement](../docs/agreements/end_user_license_agreement.pdf)

97
tig-runtime/README.md Normal file
View File

@ -0,0 +1,97 @@
# tig-runtime
A Rust crate that execute an algorithm (compiled from [`tig-binary`](../tig-binary/README.md)) for a single nonce, generating runtime signature and fuel consumed for verification.
# Getting Started
Users who don't intend to customise `tig-runtime` are recommended to download pre-compiled version available in [TIG's runtime docker images](https://github.com/tig-foundation/tig-monorepo/pkgs/container/tig-monorepo%2Fruntime).
**Example:**
```
docker run -it ghcr.io/tig-foundation/tig-monorepo/runtime:0.0.1-aarch64
# tig-runtime is already on PATH
```
## Compiling (using dev docker image)
The required rust environment for development are available via [TIG's development docker images](https://github.com/tig-foundation/tig-monorepo/pkgs/container/tig-monorepo%2Fdev).
**Example:**
```
docker run -it -v $(pwd):/app ghcr.io/tig-foundation/tig-monorepo/dev:0.0.1-aarch64
# cargo build -p tig-runtime --release
```
## Compiling (local setup)
Users who intend to customise `tig-runtime` need to install a specific version of rust:
1. Install rust version `nightly-2025-02-10`
```
ARCH=$(uname -m)
RUST_TARGET=$(if [ "$ARCH" = "aarch64" ] || [ "$ARCH" = "arm64" ]; then
echo "aarch64-unknown-linux-gnu";
else
echo "x86_64-unknown-linux-gnu";
fi)
rustup install nightly-2025-02-10
rustup default nightly-2025-02-10
rustup component add rust-src
rustup target add $RUST_TARGET
RUST_LIBDIR=$(rustc --print target-libdir --target=$RUST_TARGET)
ln -s $RUST_LIBDIR /usr/local/lib/rust
echo "export LD_LIBRARY_PATH=\"${LD_LIBRARY_PATH}:/usr/local/lib/rust\"" >> ~/.bashrc
```
2. Compile `tig-runtime`
```
# for cuda version, add --features cuda
cargo build -p tig-runtime --release --target $RUST_TARGET
```
# Usage
```
Usage: tig-runtime [OPTIONS] <SETTINGS> <RAND_HASH> <NONCE> <BINARY>
Arguments:
<SETTINGS> Settings json string or path to json file
<RAND_HASH> A string used in seed generation
<NONCE> Nonce value
<BINARY> Path to a shared object (*.so) file
Options:
--ptx [<PTX>] Path to a CUDA ptx file
--fuel [<FUEL>] Optional maximum fuel parameter [default: 2000000000]
--output [<OUTPUT_FILE>] If set, the output data will be saved to this file path (default json)
--compress [<COMPRESS>] If output file is set, the output data will be compressed as zlib [default: false] [possible values: true, false]
--gpu [<GPU>] Which GPU device to use [default: 0]
-h, --help Print help
```
The following exit codes indicate specific meanings:
* 0 - solution found
* 85 - no solution found
* 86 - runtime error
* 87 - out of fuel
**Example:**
```
SETTINGS='{"challenge_id":"c001","difficulty":[50,300],"algorithm_id":"","player_id":"","block_id":""}'
RANDHASH='rand_hash'$
NONCE=1337
FUEL=987654321123456789
SO_PATH=./tig-algorithms/lib/satisfiability/aarch64/better_sat.so
tig-runtime $SETTINGS $RANDHASH $NONCE $SO_PATH --fuel $FUEL
```
**Example Output:**
```
{"cpu_arch":"aarch64","fuel_consumed":95496,"nonce":1337,"runtime_signature":4125818588297548058,"solution":{"variables":[0,1,0,1,0,0,0,0,0,1,0,0,0,1,1,1,0,1,1,1,0,0,0,0,1,1,0,1,1,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,1,1,1,0,0]}}
```
# License
[End User License Agreement](../docs/agreements/end_user_license_agreement.pdf)

2
tig-runtime/src/main.rs
View File

@ -42,7 +42,7 @@ fn cli() -> Command {
.value_parser(clap::value_parser!(u64)),
)
.arg(
arg!(--output [OUTPUT_FILE] "If set, the output data will be saved to this file path (default json")
arg!(--output [OUTPUT_FILE] "If set, the output data will be saved to this file path (default json)")
.value_parser(clap::value_parser!(PathBuf)),
)
.arg(

89
tig-verifier/README.md Normal file
View File

@ -0,0 +1,89 @@
# tig-verifier
A Rust crate that verifies a single solution or Merkle proof.
# Getting Started
Users who don't intend to customise `tig-verifier` are recommended to download pre-compiled version available in [TIG's runtime docker images](https://github.com/tig-foundation/tig-monorepo/pkgs/container/tig-monorepo%2Fruntime).
**Example:**
```
docker run -it ghcr.io/tig-foundation/tig-monorepo/runtime:0.0.1-aarch64
# tig-verifier is already on PATH
```
## Compiling (using dev docker image)
The required rust environment for development are available via [TIG's development docker images](https://github.com/tig-foundation/tig-monorepo/pkgs/container/tig-monorepo%2Fdev).
**Example:**
```
docker run -it -v $(pwd):/app ghcr.io/tig-foundation/tig-monorepo/dev:0.0.1-aarch64
# cargo build -p tig-verifier --release
```
## Compiling (local setup)
Users who intend to customise `tig-verifier` need to install a specific version of rust:
1. Install rust version `nightly-2025-02-10`
```
ARCH=$(uname -m)
RUST_TARGET=$(if [ "$ARCH" = "aarch64" ] || [ "$ARCH" = "arm64" ]; then
echo "aarch64-unknown-linux-gnu";
else
echo "x86_64-unknown-linux-gnu";
fi)
rustup install nightly-2025-02-10
rustup default nightly-2025-02-10
rustup component add rust-src
rustup target add $RUST_TARGET
RUST_LIBDIR=$(rustc --print target-libdir --target=$RUST_TARGET)
ln -s $RUST_LIBDIR /usr/local/lib/rust
echo "export LD_LIBRARY_PATH=\"${LD_LIBRARY_PATH}:/usr/local/lib/rust\"" >> ~/.bashrc
```
2. Compile `tig-verifier`
```
# for cuda version, add --features cuda
cargo build -p tig-verifier --release --target $RUST_TARGET
```
# Usage
`tig-verifier` will have multiple sub-commands, but currently only `verify_solution` is implemented:
```
Usage: tig-verifier verify_solution [OPTIONS] <SETTINGS> <RAND_HASH> <NONCE> <SOLUTION>
Arguments:
<SETTINGS> Settings json string or path to json file
<RAND_HASH> A string used in seed generation
<NONCE> Nonce value
<SOLUTION> Solution json string, path to json file, or '-' for stdin
Options:
--ptx [<PTX>] Path to a CUDA ptx file
--gpu [<GPU>] Which GPU device to use [default: 0]
-h, --help Print help
```
**Example:**
```
SETTINGS='{"challenge_id":"c001","difficulty":[50,300],"algorithm_id":"","player_id":"","block_id":""}'
RANDHASH='rand_hash'
NONCE=1337
SOLUTION='{"variables":[1,1,0,0,0,0,1,0,0,1,0,0,0,1,0,0,0,1,0,0,0,0,0,0,0,1,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,1,1,1,1,1,0,0]}'
tig-verifier verify_solution $SETTINGS $RANDHASH $NONCE $SOLUTION
```
**Example Output:**
```
Solution is valid
```
# License
[End User License Agreement](../docs/agreements/end_user_license_agreement.pdf)

156
tig-worker/README.md
View File

@ -1,51 +1,88 @@
# tig-worker
A Rust crate for verifying and computing solutions.
A Rust crate for executing a batch of instances using [`tig-runtime`](../tig-runtime/README.md), aggregating outputs, and calculating Merkle root.
Solutions are computed by executing an algorithm in a WASM virtual machine ([TIG's fork of wasmi](https://github.com/tig-foundation/wasmi)).
# Getting Started
# Compiling
`tig-worker` executes a number of `tig-runtime` concurrently. Each `tig-runtime` loads an algorithm shared object (compiled from `tig-binary`), which expects a specific version of rust standard libraries to be available on `LD_LIBRARY_PATH`.
Users who don't intend to customise `tig-worker` are recommended to download pre-compiled version available in [TIG's runtime docker images](https://github.com/tig-foundation/tig-monorepo/pkgs/container/tig-monorepo%2Fruntime).
**Example:**
```
cargo build -p tig-worker --release
./target/release/tig-worker --help
docker run -it ghcr.io/tig-foundation/tig-monorepo/runtime:0.0.1-aarch64
# tig-worker is already on PATH
```
## Compiling (using dev docker image)
The required rust environment for development are available via [TIG's development docker images](https://github.com/tig-foundation/tig-monorepo/pkgs/container/tig-monorepo%2Fdev).
**Example:**
```
docker run -it -v $(pwd):/app ghcr.io/tig-foundation/tig-monorepo/dev:0.0.1-aarch64
# cargo build -p tig-worker --release
```
## Compiling (local setup)
Users who intend to customise `tig-worker` need to install a specific version of rust:
1. Install rust version `nightly-2025-02-10`
```
ARCH=$(uname -m)
RUST_TARGET=$(if [ "$ARCH" = "aarch64" ] || [ "$ARCH" = "arm64" ]; then
echo "aarch64-unknown-linux-gnu";
else
echo "x86_64-unknown-linux-gnu";
fi)
rustup install nightly-2025-02-10
rustup default nightly-2025-02-10
rustup component add rust-src
rustup target add $RUST_TARGET
RUST_LIBDIR=$(rustc --print target-libdir --target=$RUST_TARGET)
ln -s $RUST_LIBDIR /usr/local/lib/rust
echo "export LD_LIBRARY_PATH=\"${LD_LIBRARY_PATH}:/usr/local/lib/rust\"" >> ~/.bashrc
```
2. Compile `tig-worker`
```
# for cuda version, add --features cuda
cargo build -p tig-worker --release --target $RUST_TARGET
```
# Usage
`tig-worker` has sub-commands `verify_solution`, `compute_solution` and `compute_batch`. These are used in 2 scripts:
* [Test algorithm performance](../scripts/test_algorithm_performance.sh)
* [Verify benchmark solutions](../scripts/verify_benchmark_solutions.sh)
## Verify Solution
Given settings, nonce and a solution, `tig-worker` verifies the solution is a valid solution for the challenge instance.
* If the solution is valid, `tig-worker` will terminate with exit code 0
* If the solution is invalid, `tig-worker` will terminate with exit code 1
```
Usage: tig-worker verify_solution <SETTINGS> <RAND_HASH> <NONCE> <SOLUTION>
Usage: tig-worker [OPTIONS] <RUNTIME> <SETTINGS> <RAND_HASH> <START_NONCE> <NUM_NONCES> <BATCH_SIZE> <BINARY>
Arguments:
<SETTINGS> Settings json string or path to json file
<RAND_HASH> A string used in seed generation
<NONCE> Nonce value
<SOLUTION> Solution json string or path to json file
<RUNTIME> Path to tig-runtime executable
<SETTINGS> Settings json string or path to json file
<RAND_HASH> A string used in seed generation
<START_NONCE> Starting nonce
<NUM_NONCES> Number of nonces to compute
<BATCH_SIZE> Batch size for Merkle tree
<BINARY> Path to a shared object (*.so) file
Options:
-h, --help Print help
--ptx [<PTX>] Path to a CUDA ptx file
--fuel [<FUEL>] Optional maximum fuel parameter for runtime [default: 2000000000]
--workers [<WORKERS>] Number of worker threads [default: 1]
--output [<OUTPUT_FOLDER>] If set, the data for nonce will be saved as '<nonce>.json' in this folder
-h, --help Print help
```
**Example:**
```
SETTINGS='{"challenge_id":"c001","difficulty":[50,300],"algorithm_id":"","player_id":"","block_id":""}'
NONCE=7
SOLUTION='{"variables":[1,0,0,0,0,0,1,0,0,0,1,1,1,1,0,1,1,0,0,1,1,1,1,0,1,0,1,1,0,0,1,1,0,0,0,0,1,0,0,1,1,1,1,1,1,0,1,1,1,0]}'
RAND_HASH=random_string
./target/release/tig-worker verify_solution $SETTINGS $RAND_HASH $NONCE $SOLUTION
SETTINGS='{"challenge_id":"c001","difficulty":[5000,415],"algorithm_id":"","player_id":"","block_id":""}'
RANDHASH='rand_hash'$
NONCE=1337
FUEL=987654321123456789
SO_PATH=./tig-algorithms/lib/satisfiability/aarch64/better_sat.so
tig-worker \tig-runtime $SETTINGS $RANDHASH $NONCE $SO_PATH --fuel $FUEL
```
## Compute Solution
@ -73,64 +110,21 @@ Options:
**Example:**
```
SETTINGS='{"challenge_id":"c001","difficulty":[50,300],"algorithm_id":"","player_id":"","block_id":""}'
NONCE=7
WASM=./tig-algorithms/wasm/satisfiability/schnoing.wasm
SETTINGS='{"challenge_id":"c001","difficulty":[5000,415],"algorithm_id":"","player_id":"","block_id":""}'
START=0
NUM_NONCES=8
BATCH_SIZE=8
SO_PATH=./tig-algorithms/lib/satisfiability/aarch64/better_sat.so
RAND_HASH=random_string
./target/release/tig-worker compute_solution $SETTINGS $RAND_HASH $NONCE $WASM
tig-worker \tig-runtime $SETTINGS $RAND_HASH $START $NUM_NONCES $BATCH_SIZE $SO_PATH --workers 8
```
# Compute Batch
Computes a batch of nonces and generates Merkle root and proofs.
**Example Output:**
```
Usage: tig-worker compute_batch [OPTIONS] <SETTINGS> <RAND_HASH> <START_NONCE> <NUM_NONCES> <BATCH_SIZE> <WASM>
Arguments:
<SETTINGS> Settings json string or path to json file
<RAND_HASH> A string used in seed generation
<START_NONCE> Starting nonce
<NUM_NONCES> Number of nonces to compute
<BATCH_SIZE> Batch size for Merkle tree
<WASM> Path to a wasm file
Options:
--fuel [<FUEL>] Optional maximum fuel parameter for WASM VM [default: 2000000000]
--mem [<MEM>] Optional maximum memory parameter for WASM VM [default: 1000000000]
--sampled <SAMPLED_NONCES>... Sampled nonces for which to generate proofs
--workers [<WORKERS>] Number of worker threads [default: 1]
-h, --help Print help
{"merkle_root":"ab3f7ea08a2b991217bd9b08299b063bc77a0239af3a826d3b0ea91ca3384f98","solution_nonces":[6,3,4,5,7,2,1,0]}
```
**Example:**
```
SETTINGS='{"challenge_id":"c001","difficulty":[50,300],"algorithm_id":"","player_id":"","block_id":""}'
START_NONCE=0
NUM_NONCES=1000
BATCH_SIZE=1024
WASM=./tig-algorithms/wasm/satisfiability/schnoing.wasm
RAND_HASH=random_string
./target/release/tig-worker compute_batch $SETTINGS $RAND_HASH $START_NONCE $NUM_NONCES $BATCH_SIZE $WASM
```
# Notes
* `challenge_id` must be set:
* `c001` is satisfiability
* `c002` is vehicle_routing
* `c003` is knapsack
* `c004` is vector_search
* Recommended low difficulties for testing are:
* satisfiability [50,300]
* vehicle_routing [40, 250]
* knapsack [50, 10]
* vector_search [10, 350]
* You can query the latest difficulties by using `scripts/list_challenges.sh`
* You can test the performance of an algorithm using `scripts/test_algorithm.sh`
* You can list all active benchmark ids with `scripts/list_benchmark_ids.sh`
* You can get benchmark data with `scripts/list_benchmark_ids.sh`
* You can verify a benchmark's solutions, runtime_signature and fuel_consumed with `scripts/verify_benchmark.sh`
# License
[End User License Agreement](../docs/agreements/end_user_license_agreement.pdf)