* feat: fast provide
* Check error from provideRoot
* do not provide if nil router
* fix(commands): prevent panic from typed nil DHTClient interface
Fixes panic when ipfsNode.DHTClient is a non-nil interface containing a
nil pointer value (typed nil). This happened when Routing.Type=delegated
or when using HTTP-only routing without DHT.
The panic occurred because:
- Go interfaces can be non-nil while containing nil pointer values
- Simple `if DHTClient == nil` checks pass, but calling methods panics
- Example: `(*ddht.DHT)(nil)` stored in interface passes nil check
Solution:
- Add HasActiveDHTClient() method to check both interface and concrete value
- Update all 7 call sites to use proper check before DHT operations
- Rename provideRoot → provideCIDSync for clarity
- Add structured logging with "fast-provide" prefix for easier filtering
- Add tests covering nil cases and valid DHT configurations
Fixes: https://github.com/ipfs/kubo/pull/11046#issuecomment-3525313349
* feat(add): split fast-provide into two flags for async/sync control
Renames --fast-provide to --fast-provide-root and adds --fast-provide-wait
to give users control over synchronous vs asynchronous providing behavior.
Changes:
- --fast-provide-root (default: true): enables immediate root CID providing
- --fast-provide-wait (default: false): controls whether to block until complete
- Default behavior: async provide (fast, non-blocking)
- Opt-in: --fast-provide-wait for guaranteed discoverability (slower, blocking)
- Can disable with --fast-provide-root=false to rely on background reproviding
Implementation:
- Async mode: launches goroutine with detached context for fire-and-forget
- Added 10 second timeout to prevent hanging on network issues
- Timeout aligns with other kubo operations (ping, DNS resolve, p2p)
- Sufficient for DHT with sweep provider or accelerated client
- Sync mode: blocks on provideCIDSync until completion (uses req.Context)
- Improved structured logging with "fast-provide-root:" prefix
- Removed redundant "root CID" from messages (already in prefix)
- Clear async/sync distinction in log messages
- Added FAST PROVIDE OPTIMIZATION section to ipfs add --help explaining:
- The problem: background queue takes time, content not immediately discoverable
- The solution: extra immediate announcement of just the root CID
- The benefit: peers can find content right away while queue handles rest
- Usage: async by default, --fast-provide-wait for guaranteed completion
Changelog:
- Added highlight section for fast root CID providing feature
- Updated TOC and overview
- Included usage examples with clear comments explaining each mode
- Emphasized this is extra announcement independent of background queue
The feature works best with sweep provider and accelerated DHT client
where provide operations are significantly faster.
* fix(add): respect Provide config in fast-provide-root
fast-provide-root should honor the same config settings as the regular
provide system:
- skip when Provide.Enabled is false
- skip when Provide.DHT.Interval is 0
- respect Provide.Strategy (all/pinned/roots/mfs/combinations)
This ensures fast-provide only runs when appropriate based on user
configuration and the nature of the content being added (pinned vs
unpinned, added to MFS or not).
* Update core/commands/add.go
---------
Co-authored-by: gammazero <11790789+gammazero@users.noreply.github.com>
Co-authored-by: Marcin Rataj <lidel@lidel.org>
New multi-router configuration system based on https://hackmd.io/G1KRDEX5T3qyfoBMkIrBew#Methods
- Added a new routing type: "custom"
- Added specific struct types for different Routers (instead of map[string]interface{})
- Added `Duration` config type, to make easier time string parsing
- Added config documentation.
- Use the latest go-delegated-routing library version with GET support.
- Added changelog notes for this feature.
It:
- closes#9157
- closes#9079
- closes#9186
* Delegated Routing.
Implementation of Reframe specs (https://github.com/ipfs/specs/blob/master/REFRAME.md) using go-delegated-routing library.
* Requested changes.
* Init using op string
* Separate possible ContentRouters for TopicDiscovery.
If we don't do this, we have a ciclic dependency creating TieredRouter.
Now we can create first all possible content routers, and after that,
create Routers.
* Set dht default routing type
* Add tests and remove uneeded code
* Add documentation.
* docs: Routing.Routers
* Requested changes.
Signed-off-by: Antonio Navarro Perez <antnavper@gmail.com>
* Add some documentation on new fx functions.
* Add changelog entry and integration tests
* test: sharness for 'dht' in 'routing' commands
Since 'routing' is currently the same as 'dht' (minus query command)
we need to test both, that way we won't have unnoticed divergence
in the default behavior.
* test(sharness): delegated routing via reframe URL
* Add more tests for delegated routing.
* If any put operation fails, the tiered router will fail.
* refactor: Routing.Routers: Parameters.Endpoint
As agreed in https://github.com/ipfs/kubo/pull/8997#issuecomment-1175684716
* Try to improve CHANGELOG entry.
* chore: update reframe spec link
* Update go-delegated-routing dependency
* Fix config error test
* use new changelog format
* Remove port conflict
* go mod tidy
* ProviderManyWrapper to ProviderMany
* Update docs/changelogs/v0.14.md
Co-authored-by: Adin Schmahmann <adin.schmahmann@gmail.com>
Co-authored-by: Marcin Rataj <lidel@lidel.org>
Co-authored-by: Adin Schmahmann <adin.schmahmann@gmail.com>
Switch _back_ to the 0.4.18 style of peer IDs while we figure things out. See
https://github.com/libp2p/specs/issues/138.
License: MIT
Signed-off-by: Steven Allen <steven@stebalien.com>
* go-datastore and friends: GetSize
* badger: new release, fewer allocations
* go-mplex: send fewer packets
* go-bitswap: pack multiple blocks in a single message, fewer allocations
* go-buffer-pool: replace the buffer pool from go-msgio
* yamux: fixed data race and uses go-buffer-pool for stream read-buffers to
reduce memory and allocations.
* go-libp2p-secio: get rid of a hot-spot allocation
* go-libp2p-peerstore: reduced allocations (at the cost of some memory)
More?
License: MIT
Signed-off-by: Steven Allen <steven@stebalien.com>
* Most of our datastores barf on non []byte values.
* We have to have a bunch of "is this a []byte" checks.
* Saves some allocations.
License: MIT
Signed-off-by: Steven Allen <steven@stebalien.com>
For the rest of the packages in util, move them to thirdparty
and update the references. util is gone!
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
