kubo/core/commands/dht.go
Guillaume Michel d56fe3a026
Some checks are pending
CodeQL / codeql (push) Waiting to run
Docker Check / lint (push) Waiting to run
Docker Check / build (push) Waiting to run
Gateway Conformance / gateway-conformance (push) Waiting to run
Gateway Conformance / gateway-conformance-libp2p-experiment (push) Waiting to run
Go Build / go-build (push) Waiting to run
Go Check / go-check (push) Waiting to run
Go Lint / go-lint (push) Waiting to run
Go Test / go-test (push) Waiting to run
Interop / interop-prep (push) Waiting to run
Interop / helia-interop (push) Blocked by required conditions
Interop / ipfs-webui (push) Blocked by required conditions
Sharness / sharness-test (push) Waiting to run
Spell Check / spellcheck (push) Waiting to run
feat(cli/rpc/add): fast provide of root CID (#11046)
* feat: fast provide
* Check error from provideRoot
* do not provide if nil router
* fix(commands): prevent panic from typed nil DHTClient interface

Fixes panic when ipfsNode.DHTClient is a non-nil interface containing a
nil pointer value (typed nil). This happened when Routing.Type=delegated
or when using HTTP-only routing without DHT.

The panic occurred because:
- Go interfaces can be non-nil while containing nil pointer values
- Simple `if DHTClient == nil` checks pass, but calling methods panics
- Example: `(*ddht.DHT)(nil)` stored in interface passes nil check

Solution:
- Add HasActiveDHTClient() method to check both interface and concrete value
- Update all 7 call sites to use proper check before DHT operations
- Rename provideRoot → provideCIDSync for clarity
- Add structured logging with "fast-provide" prefix for easier filtering
- Add tests covering nil cases and valid DHT configurations

Fixes: https://github.com/ipfs/kubo/pull/11046#issuecomment-3525313349

* feat(add): split fast-provide into two flags for async/sync control

Renames --fast-provide to --fast-provide-root and adds --fast-provide-wait
to give users control over synchronous vs asynchronous providing behavior.

Changes:
- --fast-provide-root (default: true): enables immediate root CID providing
- --fast-provide-wait (default: false): controls whether to block until complete
- Default behavior: async provide (fast, non-blocking)
- Opt-in: --fast-provide-wait for guaranteed discoverability (slower, blocking)
- Can disable with --fast-provide-root=false to rely on background reproviding

Implementation:
- Async mode: launches goroutine with detached context for fire-and-forget
  - Added 10 second timeout to prevent hanging on network issues
  - Timeout aligns with other kubo operations (ping, DNS resolve, p2p)
  - Sufficient for DHT with sweep provider or accelerated client
- Sync mode: blocks on provideCIDSync until completion (uses req.Context)
- Improved structured logging with "fast-provide-root:" prefix
  - Removed redundant "root CID" from messages (already in prefix)
  - Clear async/sync distinction in log messages
- Added FAST PROVIDE OPTIMIZATION section to ipfs add --help explaining:
  - The problem: background queue takes time, content not immediately discoverable
  - The solution: extra immediate announcement of just the root CID
  - The benefit: peers can find content right away while queue handles rest
  - Usage: async by default, --fast-provide-wait for guaranteed completion

Changelog:
- Added highlight section for fast root CID providing feature
- Updated TOC and overview
- Included usage examples with clear comments explaining each mode
- Emphasized this is extra announcement independent of background queue

The feature works best with sweep provider and accelerated DHT client
where provide operations are significantly faster.

* fix(add): respect Provide config in fast-provide-root

fast-provide-root should honor the same config settings as the regular
provide system:
- skip when Provide.Enabled is false
- skip when Provide.DHT.Interval is 0
- respect Provide.Strategy (all/pinned/roots/mfs/combinations)

This ensures fast-provide only runs when appropriate based on user
configuration and the nature of the content being added (pinned vs
unpinned, added to MFS or not).

* Update core/commands/add.go

---------

Co-authored-by: gammazero <11790789+gammazero@users.noreply.github.com>
Co-authored-by: Marcin Rataj <lidel@lidel.org>
2025-11-14 11:08:29 -08:00

133 lines
3.2 KiB
Go

package commands
import (
"context"
"errors"
"fmt"
"io"
cmds "github.com/ipfs/go-ipfs-cmds"
"github.com/ipfs/kubo/core/commands/cmdenv"
peer "github.com/libp2p/go-libp2p/core/peer"
routing "github.com/libp2p/go-libp2p/core/routing"
)
var ErrNotDHT = errors.New("routing service is not a DHT")
var DhtCmd = &cmds.Command{
Status: cmds.Deprecated,
Helptext: cmds.HelpText{
Tagline: "Issue commands directly through the DHT.",
ShortDescription: ``,
},
Subcommands: map[string]*cmds.Command{
"query": queryDhtCmd,
"findprovs": RemovedDHTCmd,
"findpeer": RemovedDHTCmd,
"get": RemovedDHTCmd,
"put": RemovedDHTCmd,
"provide": RemovedDHTCmd,
},
}
// kademlia extends the routing interface with a command to get the peers closest to the target
type kademlia interface {
routing.Routing
GetClosestPeers(ctx context.Context, key string) ([]peer.ID, error)
}
var queryDhtCmd = &cmds.Command{
Status: cmds.Deprecated,
Helptext: cmds.HelpText{
Tagline: "Find the closest Peer IDs to a given Peer ID by querying the DHT.",
ShortDescription: "Outputs a list of newline-delimited Peer IDs.",
},
Arguments: []cmds.Argument{
cmds.StringArg("peerID", true, true, "The peerID to run the query against."),
},
Options: []cmds.Option{
cmds.BoolOption(dhtVerboseOptionName, "v", "Print extra information."),
},
Run: func(req *cmds.Request, res cmds.ResponseEmitter, env cmds.Environment) error {
nd, err := cmdenv.GetNode(env)
if err != nil {
return err
}
if !nd.HasActiveDHTClient() {
return ErrNotDHT
}
id, err := peer.Decode(req.Arguments[0])
if err != nil {
return cmds.ClientError("invalid peer ID")
}
ctx, cancel := context.WithCancel(req.Context)
defer cancel()
ctx, events := routing.RegisterForQueryEvents(ctx)
client := nd.DHTClient
if nd.DHT != nil && client == nd.DHT {
client = nd.DHT.WAN
if !nd.DHT.WANActive() {
client = nd.DHT.LAN
}
}
if d, ok := client.(kademlia); !ok {
return errors.New("dht client does not support GetClosestPeers")
} else {
errCh := make(chan error, 1)
go func() {
defer close(errCh)
defer cancel()
closestPeers, err := d.GetClosestPeers(ctx, string(id))
for _, p := range closestPeers {
routing.PublishQueryEvent(ctx, &routing.QueryEvent{
ID: p,
Type: routing.FinalPeer,
})
}
if err != nil {
errCh <- err
return
}
}()
for e := range events {
if err := res.Emit(e); err != nil {
return err
}
}
return <-errCh
}
},
Encoders: cmds.EncoderMap{
cmds.Text: cmds.MakeTypedEncoder(func(req *cmds.Request, w io.Writer, out *routing.QueryEvent) error {
pfm := pfuncMap{
routing.FinalPeer: func(obj *routing.QueryEvent, out io.Writer, verbose bool) error {
fmt.Fprintf(out, "%s\n", obj.ID)
return nil
},
}
verbose, _ := req.Options[dhtVerboseOptionName].(bool)
return printEvent(out, w, verbose, pfm)
}),
},
Type: routing.QueryEvent{},
}
var RemovedDHTCmd = &cmds.Command{
Status: cmds.Removed,
Helptext: cmds.HelpText{
Tagline: "Removed, use 'ipfs routing' instead.",
},
Run: func(req *cmds.Request, res cmds.ResponseEmitter, env cmds.Environment) error {
return errors.New("removed, use 'ipfs routing' instead")
},
}