coosld/kubo
1
0
Fork 0
mirror of https://github.com/ipfs/kubo.git synced 2026-02-21 10:27:46 +08:00
Code Issues Actions 3 Packages Projects Releases Wiki Activity

docs: mark custom routing as experimental

Browse Source 
reorganize Routing.Type section for clarity, group production and
experimental options, consolidate DHT explanation, add limitations
section to delegated-routing.md documenting that HTTP-only routing
cannot provide content reliably
This commit is contained in:
Marcin Rataj 2025-12-16 17:20:55 +01:00
parent ab44726177
commit e36887ab32
No known key found for this signature in database
GPG Key ID: 222B6784D5A79E42
3 changed files with 125 additions and 77 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

140
docs/config.md
View File

@ -2542,55 +2542,50 @@ Contains options for content, peer, and IPNS routing mechanisms.
### `Routing.Type`
There are multiple routing options: "auto", "autoclient", "none", "dht", "dhtclient", "delegated", and "custom".
Controls how your node discovers content and peers on the network.
- **DEFAULT:** If unset, or set to "auto", your node will use the public IPFS DHT (aka "Amino")
and parallel [`Routing.DelegatedRouters`](#routingdelegatedrouters) for additional speed.
**Production options:**
- If set to "autoclient", your node will behave as in "auto" but without running a DHT server.
- **`auto`** (default): Uses both the public IPFS DHT (Amino) and HTTP routers
from [`Routing.DelegatedRouters`](#routingdelegatedrouters) for faster lookups.
Your node starts as a DHT client and automatically switches to server mode
when reachable from the public internet.
- If set to "none", your node will use _no_ routing system. You'll have to
explicitly connect to peers that have the content you're looking for.
- **`autoclient`**: Same as `auto`, but never runs a DHT server.
Use this if your node is behind a firewall or NAT.
- If set to "dht" (or "dhtclient"/"dhtserver"), your node will ONLY use the Amino DHT (no HTTP routers).
- **`dht`**: Uses only the Amino DHT (no HTTP routers). Automatically switches
between client and server mode based on reachability.
- If set to "custom", all default routers are disabled, and only ones defined in `Routing.Routers` will be used.
- **`dhtclient`**: DHT-only, always running as a client. Lower resource usage.
When the DHT is enabled, it can operate in two modes: client and server.
- **`dhtserver`**: DHT-only, always running as a server.
Only use this if your node is reachable from the public internet.
- In server mode, your node will query other peers for DHT records, and will
respond to requests from other peers (both requests to store records and
requests to retrieve records).
- **`none`**: Disables all routing. You must manually connect to peers.
- In client mode, your node will query the DHT as a client but will not respond
to requests from other peers. This mode is less resource-intensive than server
mode.
**About DHT client vs server mode:**
When the DHT is enabled, your node can operate as either a client or server.
In server mode, it queries other peers and responds to their queries - this helps
the network but uses more resources. In client mode, it only queries others without
responding, which is less resource-intensive. With `auto` or `dht`, your node starts
as a client and switches to server when it detects public reachability.
When `Routing.Type` is set to `auto` or `dht`, your node will start as a DHT client, and
switch to a DHT server when and if it determines that it's reachable from the
public internet (e.g., it's not behind a firewall).
> [!CAUTION]
> **`Routing.Type` Experimental options:**
>
> These modes are for research and testing only, not production use.
> They may change without notice between releases.
>
> - **`delegated`**: Uses only HTTP routers from [`Routing.DelegatedRouters`](#routingdelegatedrouters)
> and IPNS publishers from [`Ipns.DelegatedPublishers`](#ipnsdelegatedpublishers),
> without initializing the DHT. Useful when peer-to-peer connectivity is unavailable.
> Note: cannot provide content to the network (no DHT means no provider records).
>
> - **`custom`**: Disables all default routers. You define your own routing in
> [`Routing.Routers`](#routingrouters). See [delegated-routing.md](delegated-routing.md).
To force a specific Amino DHT-only mode, client or server, set `Routing.Type` to
`dhtclient` or `dhtserver` respectively. Please do not set this to `dhtserver`
unless you're sure your node is reachable from the public network.
When `Routing.Type` is set to `auto` or `autoclient` your node will accelerate some types of routing
by leveraging [`Routing.DelegatedRouters`](#routingdelegatedrouters) HTTP endpoints compatible with [Delegated Routing V1 HTTP API](https://specs.ipfs.tech/routing/http-routing-v1/)
introduced in [IPIP-337](https://github.com/ipfs/specs/pull/337)
in addition to the Amino DHT.
When `Routing.Type` is set to `delegated`, your node will use **only** HTTP delegated routers and IPNS publishers,
without initializing the Amino DHT at all. This mode is useful for environments where peer-to-peer DHT connectivity
is not available or desired, while still enabling content routing and IPNS publishing via HTTP APIs.
This mode requires configuring [`Routing.DelegatedRouters`](#routingdelegatedrouters) for content routing and
[`Ipns.DelegatedPublishers`](#ipnsdelegatedpublishers) for IPNS publishing.
**Note:** `delegated` mode operates as read-only for content providing - your node cannot announce content to the network
since there is no DHT connectivity. Content providing is automatically disabled when using this routing type.
[Advanced routing rules](https://github.com/ipfs/kubo/blob/master/docs/delegated-routing.md) can be configured in `Routing.Routers` after setting `Routing.Type` to `custom`.
Default: `auto` (DHT + [`Routing.DelegatedRouters`](#routingdelegatedrouters))
Default: `auto`
Type: `optionalString` (`null`/missing means the default)
@ -2623,27 +2618,28 @@ When it is enabled:
This is critical to maintain to not harm the network.
- The operations `ipfs stats dht` will default to showing information about the accelerated DHT client
**Caveats:**
1. Running the accelerated client likely will result in more resource consumption (connections, RAM, CPU, bandwidth)
- Users that are limited in the number of parallel connections their machines/networks can perform will likely suffer
- The resource usage is not smooth as the client crawls the network in rounds and reproviding is similarly done in rounds
- Users who previously had a lot of content but were unable to advertise it on the network will see an increase in
egress bandwidth as their nodes start to advertise all of their CIDs into the network. If you have lots of data
entering your node that you don't want to advertise, then consider using [Provide Strategies](#providestrategy)
to reduce the number of CIDs that you are reproviding. Similarly, if you are running a node that deals mostly with
short-lived temporary data (e.g. you use a separate node for ingesting data then for storing and serving it) then
you may benefit from using [Strategic Providing](experimental-features.md#strategic-providing) to prevent advertising
of data that you ultimately will not have.
2. Currently, the DHT is not usable for queries for the first 5-10 minutes of operation as the routing table is being
prepared. This means operations like searching the DHT for particular peers or content will not work initially.
- You can see if the DHT has been initially populated by running `ipfs stats dht`
3. Currently, the accelerated DHT client is not compatible with LAN-based DHTs and will not perform operations against
them
4. (⚠️ 0.39 limitation) When used with [`Provide.DHT.SweepEnabled`](#providedhtsweepenabled), the sweep provider may
fail to estimate DHT size during the accelerated client's network crawl, resulting in all CIDs grouped into a
single region. Content still gets reprovided, but without sweep efficiency gains. Consider disabling the
accelerated client when using sweep mode.
> [!CAUTION]
> **`Routing.AcceleratedDHTClient` Caveats:**
>
> 1. Running the accelerated client likely will result in more resource consumption (connections, RAM, CPU, bandwidth)
> - Users that are limited in the number of parallel connections their machines/networks can perform will likely suffer
> - The resource usage is not smooth as the client crawls the network in rounds and reproviding is similarly done in rounds
> - Users who previously had a lot of content but were unable to advertise it on the network will see an increase in
> egress bandwidth as their nodes start to advertise all of their CIDs into the network. If you have lots of data
> entering your node that you don't want to advertise, then consider using [Provide Strategies](#providestrategy)
> to reduce the number of CIDs that you are reproviding. Similarly, if you are running a node that deals mostly with
> short-lived temporary data (e.g. you use a separate node for ingesting data then for storing and serving it) then
> you may benefit from using [Strategic Providing](experimental-features.md#strategic-providing) to prevent advertising
> of data that you ultimately will not have.
> 2. Currently, the DHT is not usable for queries for the first 5-10 minutes of operation as the routing table is being
> prepared. This means operations like searching the DHT for particular peers or content will not work initially.
> - You can see if the DHT has been initially populated by running `ipfs stats dht`
> 3. Currently, the accelerated DHT client is not compatible with LAN-based DHTs and will not perform operations against
> them.
> 4. (⚠️ 0.39 limitation) When used with [`Provide.DHT.SweepEnabled`](#providedhtsweepenabled), the sweep provider may
> fail to estimate DHT size during the accelerated client's network crawl, resulting in all CIDs grouped into a
> single region. Content still gets reprovided, but without sweep efficiency gains. Consider disabling the
> accelerated client when using sweep mode.
Default: `false`
@ -2697,10 +2693,14 @@ Type: `array[string]` (URLs or `"auto"`)
Alternative configuration used when `Routing.Type=custom`.
> [!WARNING]
> **EXPERIMENTAL: `Routing.Routers` configuration may change in future release**
> [!CAUTION]
> **EXPERIMENTAL: `Routing.Routers` is for research and testing only, not production use.**
>
> Consider this advanced low-level config: Most users can simply use `Routing.Type=auto` or `autoclient` and set up basic config in user-friendly [`Routing.DelegatedRouters`](https://github.com/ipfs/kubo/blob/master/docs/config.md#routingdelegatedrouters).
> - The configuration format and behavior may change without notice between releases.
> - Bugs and regressions may not be prioritized.
> - HTTP-only configurations cannot reliably provide content. See [delegated-routing.md](delegated-routing.md#limitations).
>
> Most users should use `Routing.Type=auto` or `autoclient` with [`Routing.DelegatedRouters`](#routingdelegatedrouters).
Allows for replacing the default routing (Amino DHT) with alternative Router
implementations.
@ -2713,7 +2713,7 @@ Type: `object[string->object]`
#### `Routing.Routers: Type`
**EXPERIMENTAL: `Routing.Routers` configuration may change in future release**
**⚠️ EXPERIMENTAL: For research and testing only. May change without notice.**
It specifies the routing type that will be created.
@ -2727,7 +2727,7 @@ Type: `string`
#### `Routing.Routers: Parameters`
**EXPERIMENTAL: `Routing.Routers` configuration may change in future release**
**⚠️ EXPERIMENTAL: For research and testing only. May change without notice.**
Parameters needed to create the specified router. Supported params per router type:
@ -2768,10 +2768,14 @@ Type: `object[string->string]`
`Methods:map` will define which routers will be executed per method used when `Routing.Type=custom`.
> [!WARNING]
> **EXPERIMENTAL: `Routing.Routers` configuration may change in future release**
> [!CAUTION]
> **EXPERIMENTAL: `Routing.Methods` is for research and testing only, not production use.**
>
> Consider this advanced low-level config: Most users can simply use `Routing.Type=auto` or `autoclient` and set up basic config in user-friendly [`Routing.DelegatedRouters`](https://github.com/ipfs/kubo/blob/master/docs/config.md#routingdelegatedrouters).
> - The configuration format and behavior may change without notice between releases.
> - Bugs and regressions may not be prioritized.
> - HTTP-only configurations cannot reliably provide content. See [delegated-routing.md](delegated-routing.md#limitations).
>
> Most users should use `Routing.Type=auto` or `autoclient` with [`Routing.DelegatedRouters`](#routingdelegatedrouters).
The key will be the name of the method: `"provide"`, `"find-providers"`, `"find-peers"`, `"put-ipns"`, `"get-ipns"`. All methods must be added to the list.

55
docs/delegated-routing.md
View File

@ -1,4 +1,15 @@
# New multi-router configuration system
# Delegated Routing Notes
- Status Date: 2025-12
> [!IMPORTANT]
> Most users are best served by setting delegated HTTP router URLs in [`Routing.DelegatedRouters`](https://github.com/ipfs/kubo/blob/master/docs/config.md#routingdelegatedrouters) and `Routing.Type` to `auto` or `autoclient`, rather than using custom routing with `Routing.Routers` and `Routing.Methods` directly.
>
> The rest of this documentation describes experimental features intended only for researchers and advanced users.
----
# Custom Multi-Router Configuration (Experimental)
- Start Date: 2022-08-15
- Related Issues:
@ -6,6 +17,17 @@
- https://github.com/ipfs/kubo/issues/9079
- https://github.com/ipfs/kubo/pull/9877
> [!CAUTION]
> **`Routing.Type=custom` with `Routing.Routers` and `Routing.Methods` is EXPERIMENTAL.**
>
> This feature is provided for **research and testing purposes only**. It is **not suitable for production use**.
>
> - The configuration format and behavior may change without notice between Kubo releases.
> - Bugs and regressions affecting custom routing may not be prioritized or fixed promptly.
> - HTTP-only routing configurations (without DHT) cannot reliably provide content to the network (👉️ see [Limitations](#limitations) below).
>
> **For production deployments**, use `Routing.Type=auto` (default) or `Routing.Type=autoclient` with [`Routing.DelegatedRouters`](https://github.com/ipfs/kubo/blob/master/docs/config.md#routingdelegatedrouters).
## Summary
Previously we only used the Amino DHT for content routing and content
@ -16,14 +38,6 @@ which then got changed and standardized as [Routing V1 HTTP API](https://specs.i
Kubo 0.23.0 release added support for [self-hosting Routing V1 HTTP API server](https://github.com/ipfs/kubo/blob/master/docs/changelogs/v0.23.md#self-hosting-routingv1-endpoint-for-delegated-routing-needs).
> [!TIP]
> Kubo 0.35 added support for [`Routing.DelegatedRouters`](https://github.com/ipfs/kubo/blob/master/docs/config.md#routingdelegatedrouters).
>
> Most of users are best served by setting delegated HTTP router URLs there and `Routing.Type` to `auto` or `autoclient`, rather than custom routing with complex `Routing.Routers` and `Routing.Methods` directly.
>
> The rest of this documentation should be considered only by advanced users and researchers.
Now we need a better way to add different routers using different protocols
like [Routing V1](https://specs.ipfs.tech/routing/http-routing-v1/) or Amino
DHT, and be able to configure them (future routing systems to come) to cover different use cases.
@ -362,6 +376,29 @@ I got ideas from all of the following links to create this design document:
- https://www.notion.so/pl-strflt/Delegated-Routing-Thoughts-very-very-WIP-0543bc51b1bd4d63a061b0f28e195d38
- https://gist.github.com/guseggert/effa027ff4cbadd7f67598efb6704d12
### Limitations
#### HTTP-only routing cannot reliably provide content
Configurations that use only HTTP routers (without any DHT router) are unable to reliably announce content (provider records) to the network.
This limitation exists because:
1. **No standardized HTTP API for providing**: The [Routing V1 HTTP API](https://specs.ipfs.tech/routing/http-routing-v1/) spec only defines read operations (`GET /routing/v1/providers/{cid}`). The write operation (`PUT /routing/v1/providers`) was never standardized.
2. **Legacy experimental API**: The only available HTTP providing mechanism is an undocumented `PUT /routing/v1/providers` request format called `ProvideBitswap`, which is a historical experiment. See [IPIP-526](https://github.com/ipfs/specs/pull/526) for ongoing discussion about formalizing HTTP-based provider announcements.
3. **Provider system integration**: Kubo's default provider system (`Provide.DHT.SweepEnabled=true` since v0.38) is designed for DHT-based providing. When no DHT is configured, the provider system may silently skip HTTP routers or behave unexpectedly.
**Workarounds for testing:**
If you need to test HTTP providing, you can try:
- Setting `Provide.DHT.SweepEnabled=false` to use the legacy provider system
- Including at least one DHT router in your custom configuration alongside HTTP routers
These workarounds are not guaranteed to work across Kubo versions and should not be relied upon for production use.
### Copyright
Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).

7
routing/delegated.go
View File

@ -31,6 +31,13 @@ import (
var log = logging.Logger("routing/delegated")
// Parse creates a composed router from the custom routing configuration.
//
// EXPERIMENTAL: Custom routing (Routing.Type=custom with Routing.Routers and
// Routing.Methods) is for research and testing only, not production use.
// The configuration format and behavior may change without notice between
// releases. HTTP-only configurations cannot reliably provide content.
// See docs/delegated-routing.md for limitations.
func Parse(routers config.Routers, methods config.Methods, extraDHT *ExtraDHTParams, extraHTTP *ExtraHTTPParams) (routing.Routing, error) {
if err := methods.Check(); err != nil {
return nil, err