kubo/core/node/libp2p/host.go
Guillaume Michel d56fe3a026
Some checks are pending
CodeQL / codeql (push) Waiting to run
Docker Check / lint (push) Waiting to run
Docker Check / build (push) Waiting to run
Gateway Conformance / gateway-conformance (push) Waiting to run
Gateway Conformance / gateway-conformance-libp2p-experiment (push) Waiting to run
Go Build / go-build (push) Waiting to run
Go Check / go-check (push) Waiting to run
Go Lint / go-lint (push) Waiting to run
Go Test / go-test (push) Waiting to run
Interop / interop-prep (push) Waiting to run
Interop / helia-interop (push) Blocked by required conditions
Interop / ipfs-webui (push) Blocked by required conditions
Sharness / sharness-test (push) Waiting to run
Spell Check / spellcheck (push) Waiting to run
feat(cli/rpc/add): fast provide of root CID (#11046)
* feat: fast provide
* Check error from provideRoot
* do not provide if nil router
* fix(commands): prevent panic from typed nil DHTClient interface

Fixes panic when ipfsNode.DHTClient is a non-nil interface containing a
nil pointer value (typed nil). This happened when Routing.Type=delegated
or when using HTTP-only routing without DHT.

The panic occurred because:
- Go interfaces can be non-nil while containing nil pointer values
- Simple `if DHTClient == nil` checks pass, but calling methods panics
- Example: `(*ddht.DHT)(nil)` stored in interface passes nil check

Solution:
- Add HasActiveDHTClient() method to check both interface and concrete value
- Update all 7 call sites to use proper check before DHT operations
- Rename provideRoot → provideCIDSync for clarity
- Add structured logging with "fast-provide" prefix for easier filtering
- Add tests covering nil cases and valid DHT configurations

Fixes: https://github.com/ipfs/kubo/pull/11046#issuecomment-3525313349

* feat(add): split fast-provide into two flags for async/sync control

Renames --fast-provide to --fast-provide-root and adds --fast-provide-wait
to give users control over synchronous vs asynchronous providing behavior.

Changes:
- --fast-provide-root (default: true): enables immediate root CID providing
- --fast-provide-wait (default: false): controls whether to block until complete
- Default behavior: async provide (fast, non-blocking)
- Opt-in: --fast-provide-wait for guaranteed discoverability (slower, blocking)
- Can disable with --fast-provide-root=false to rely on background reproviding

Implementation:
- Async mode: launches goroutine with detached context for fire-and-forget
  - Added 10 second timeout to prevent hanging on network issues
  - Timeout aligns with other kubo operations (ping, DNS resolve, p2p)
  - Sufficient for DHT with sweep provider or accelerated client
- Sync mode: blocks on provideCIDSync until completion (uses req.Context)
- Improved structured logging with "fast-provide-root:" prefix
  - Removed redundant "root CID" from messages (already in prefix)
  - Clear async/sync distinction in log messages
- Added FAST PROVIDE OPTIMIZATION section to ipfs add --help explaining:
  - The problem: background queue takes time, content not immediately discoverable
  - The solution: extra immediate announcement of just the root CID
  - The benefit: peers can find content right away while queue handles rest
  - Usage: async by default, --fast-provide-wait for guaranteed completion

Changelog:
- Added highlight section for fast root CID providing feature
- Updated TOC and overview
- Included usage examples with clear comments explaining each mode
- Emphasized this is extra announcement independent of background queue

The feature works best with sweep provider and accelerated DHT client
where provide operations are significantly faster.

* fix(add): respect Provide config in fast-provide-root

fast-provide-root should honor the same config settings as the regular
provide system:
- skip when Provide.Enabled is false
- skip when Provide.DHT.Interval is 0
- respect Provide.Strategy (all/pinned/roots/mfs/combinations)

This ensures fast-provide only runs when appropriate based on user
configuration and the nature of the content being added (pinned vs
unpinned, added to MFS or not).

* Update core/commands/add.go

---------

Co-authored-by: gammazero <11790789+gammazero@users.noreply.github.com>
Co-authored-by: Marcin Rataj <lidel@lidel.org>
2025-11-14 11:08:29 -08:00

113 lines
3.4 KiB
Go

package libp2p
import (
"context"
"github.com/libp2p/go-libp2p"
record "github.com/libp2p/go-libp2p-record"
"github.com/libp2p/go-libp2p/core/host"
"github.com/libp2p/go-libp2p/core/peer"
"github.com/libp2p/go-libp2p/core/peerstore"
"github.com/libp2p/go-libp2p/core/routing"
routedhost "github.com/libp2p/go-libp2p/p2p/host/routed"
"github.com/ipfs/kubo/config"
"github.com/ipfs/kubo/core/node/helpers"
"github.com/ipfs/kubo/repo"
"go.uber.org/fx"
)
type P2PHostIn struct {
fx.In
Repo repo.Repo
Validator record.Validator
HostOption HostOption
RoutingOption RoutingOption
ID peer.ID
Peerstore peerstore.Peerstore
Opts [][]libp2p.Option `group:"libp2p"`
}
type P2PHostOut struct {
fx.Out
Host host.Host
Routing routing.Routing `name:"initialrouting"`
}
func Host(mctx helpers.MetricsCtx, lc fx.Lifecycle, params P2PHostIn) (out P2PHostOut, err error) {
opts := []libp2p.Option{libp2p.NoListenAddrs}
for _, o := range params.Opts {
opts = append(opts, o...)
}
ctx := helpers.LifecycleCtx(mctx, lc)
cfg, err := params.Repo.Config()
if err != nil {
return out, err
}
// Use auto-config resolution for actual connectivity
bootstrappers, err := cfg.BootstrapPeersWithAutoConf()
if err != nil {
return out, err
}
// Optimistic provide is enabled either via dedicated expierimental flag, or when DHT Provide Sweep is enabled.
// When DHT Provide Sweep is enabled, all provide operations go through the
// `SweepingProvider`, hence the provides don't use the optimistic provide
// logic. Provides use `SweepingProvider.StartProviding()` and not
// `IpfsDHT.Provide()`, which is where the optimistic provide logic is
// implemented. However, `IpfsDHT.Provide()` is used to quickly provide roots
// when user manually adds content with the `--fast-provide` flag enabled. In
// this case we want to use optimistic provide logic to quickly announce the
// content to the network. This should be the only use case of
// `IpfsDHT.Provide()` when DHT Provide Sweep is enabled.
optimisticProvide := cfg.Experimental.OptimisticProvide || cfg.Provide.DHT.SweepEnabled.WithDefault(config.DefaultProvideDHTSweepEnabled)
routingOptArgs := RoutingOptionArgs{
Ctx: ctx,
Datastore: params.Repo.Datastore(),
Validator: params.Validator,
BootstrapPeers: bootstrappers,
OptimisticProvide: optimisticProvide,
OptimisticProvideJobsPoolSize: cfg.Experimental.OptimisticProvideJobsPoolSize,
LoopbackAddressesOnLanDHT: cfg.Routing.LoopbackAddressesOnLanDHT.WithDefault(config.DefaultLoopbackAddressesOnLanDHT),
}
opts = append(opts, libp2p.Routing(func(h host.Host) (routing.PeerRouting, error) {
args := routingOptArgs
args.Host = h
r, err := params.RoutingOption(args)
out.Routing = r
return r, err
}))
out.Host, err = params.HostOption(params.ID, params.Peerstore, opts...)
if err != nil {
return P2PHostOut{}, err
}
routingOptArgs.Host = out.Host
// this code is necessary just for tests: mock network constructions
// ignore the libp2p constructor options that actually construct the routing!
if out.Routing == nil {
r, err := params.RoutingOption(routingOptArgs)
if err != nil {
return P2PHostOut{}, err
}
out.Routing = r
out.Host = routedhost.Wrap(out.Host, out.Routing)
}
lc.Append(fx.Hook{
OnStop: func(ctx context.Context) error {
return out.Host.Close()
},
})
return out, err
}