problem: `ipfs files write` in CIDv1 directories wrapped single-block
files in dag-pb even when raw-leaves was enabled, producing different
CIDs than `ipfs add --raw-leaves` for the same content.
fix: boxo now collapses single-block ProtoNode wrappers (with no
metadata) to RawNode in DagModifier.GetNode(). files with mtime/mode
stay as dag-pb since raw blocks cannot store UnixFS metadata.
also fixes sparse file writes where writing past EOF would lose data
because expandSparse didn't update the internal node pointer.
updates boxo to v0.36.1-0.20260203003133-7884ae23aaff
updates t0250-files-api.sh test hashes to match new behavior
`ipfs files` commands now respect these Import.* config options:
- UnixFSChunker: configures chunk size for `files write`
- UnixFSDirectoryMaxLinks: triggers HAMT sharding in `files mkdir`
- UnixFSHAMTDirectorySizeEstimation: controls size estimation mode
previously, MFS used hardcoded defaults ignoring user config.
changes:
- config/import.go: add UnixFSSplitterFunc() returning chunk.SplitterGen
- core/node/core.go: pass chunker, maxLinks, sizeEstimationMode to
mfs.NewRoot() via new boxo RootOption API
- core/commands/files.go: pass maxLinks and sizeEstimationMode to
mfs.Mkdir() and ensureContainingDirectoryExists(); document that
UnixFSFileMaxLinks doesn't apply to files write (trickle DAG limitation)
- test/cli/files_test.go: add tests for UnixFSDirectoryMaxLinks and
UnixFSChunker, including CID parity test with `ipfs add --trickle`
related: boxo@54e044f1b265
* fix http header when compress enabled for get command
Closes#2376
* fix(rpc): set Content-Type for ipfs get based on output format
- set application/x-tar when outputting tar (default and --archive)
- set application/gzip when compression is enabled (--compress)
- update go-ipfs-cmds with Tar encoding type and RFC 6713 compliant
MIME types (application/gzip instead of application/x-gzip)
* test(rpc): add Content-Type header tests for ipfs get
* feat(rpc): add Content-Type headers for binary responses
set proper Content-Type headers for RPC endpoints that return binary data:
- `dag export`: application/vnd.ipld.car
- `block get`: application/vnd.ipld.raw
- `diag profile`: application/zip
- `get`: application/x-tar or application/gzip (already worked, migrated to new API)
uses the new OctetStream encoding type and SetContentType() method
from go-ipfs-cmds to specify custom MIME types for binary responses.
refs: https://github.com/ipfs/kubo/issues/2376
* feat(rpc): add `ipfs name get` command for IPNS record retrieval
add dedicated command to retrieve raw signed IPNS records from the
routing system. returns protobuf-encoded IPNS record with Content-Type
`application/vnd.ipfs.ipns-record`.
this provides a more convenient alternative to `ipfs routing get /ipns/<name>`
which returns JSON with base64-encoded data. the raw output can be piped
directly to `ipfs name inspect`:
ipfs name get <name> | ipfs name inspect
spec: https://specs.ipfs.tech/ipns/ipns-record/
* feat(rpc): add `ipfs name put` command for IPNS record storage
adds `ipfs name put` to complement `ipfs name get`, allowing users to
store IPNS records obtained from external sources without needing the
private key. useful for backup, restore, and debugging workflows.
the command validates records by default (signature, sequence number).
use `--force` to bypass validation for testing how routing handles
malformed or outdated records.
also reorganizes test/cli files:
- rename http_rpc_* -> rpc_* to match existing convention
- merge name_get_put_test.go into name_test.go
- add file header comments documenting test purposes
* chore(deps): update go-ipfs-cmds to latest master
includes SetContentType() for dynamic Content-Type headers
---------
Co-authored-by: Marcin Rataj <lidel@lidel.org>
* fix(routing): update kad-dht with peerstore address clone fix
closes#11116
See https://github.com/ipfs/kubo/issues/11116 for context of this fix
* fix(routing): update kad-dht with CPL exploration fix
fixes an infinite loop when all peers share the same CPL during provider exploration
See https://github.com/libp2p/go-libp2p-kad-dht/pull/1216
* fix(routing): update kad-dht with shutdown loop check
https://github.com/libp2p/go-libp2p-kad-dht/pull/1217
* depend on latest kad-dht fix
* bump kad-dht to v0.37.0
---------
Co-authored-by: guillaumemichel <guillaume@michel.id>
Co-authored-by: Guillaume Michel <guillaumemichel@users.noreply.github.com>
* chore: exclude ancient +incompatible versions from go.mod
prevents Dependabot from failing when it tries to update
go-ipfs-cmds and go-libp2p directly and resolves to
pre-Go-modules v2.x/v6.x versions that reference deleted packages
* chore(deps): group opentelemetry, prometheus, and uber packages
reduces PR noise by batching related observability dependencies
* ci: add workflow to run make mod_tidy on Dependabot PRs
ensures all go.mod/go.sum files stay in sync when Dependabot
updates dependencies in the root module
supports manual dispatch with PR number for existing PRs
* feat(config): add Gateway.MaxRequestDuration option
exposes the previously hardcoded 1 hour gateway request deadline as a
configurable option, allowing operators to adjust it to fit deployment
needs. protects gateway from edge cases and slow client attacks.
boxo: https://github.com/ipfs/boxo/pull/1079
* test(gateway): add MaxRequestDuration integration test
verifies config is wired correctly and 504 is returned when exceeded
* docs: add MaxRequestDuration to gateway production guide
---------
Co-authored-by: Andrew Gillis <11790789+gammazero@users.noreply.github.com>
* datastore: upgrade go-ds-flatfs to v0.6.0
See: https://github.com/ipfs/go-ds-flatfs/pull/142
* docs(changelog): add go-ds-flatfs atomic batch writes
*documents the new flatfs batch implementation that uses atomic
operations via temp directory, preventing orphan blocks on interrupted
imports and reducing memory usage.
* includes improved tests, batch cleanup fixes, and docs
* docs(changelog): reframe go-ds-flatfs entry for users
focus on user benefits instead of implementation details
* fix: update go-libp2p to v0.46.0
- reduced WebRTC log noise (go-libp2p#3426)
- fixed mDNS discovery on Windows/macOS (go-libp2p#3434)
- includes quic-go v0.57.1 (v0.56.0 + v0.57.0)
* fix(example): kubo-as-a-library test timeout
- use custom ports (4010/4011) to avoid conflicts with default 4001
- add 2-minute context timeout to fail fast
- get peer addresses dynamically instead of hardcoding wrong port
- wait for peer connection synchronously instead of fire-and-forget
- update comments to reference autoconf.FallbackBootstrapPeers
* chore: update p2p-forge to v0.7.0
* fix(test): wait for DHT readiness in GetClosestPeers test
the test was failing for `routing_type=auto` because it only waited for
swarm connections but not for the DHT routing table to be populated.
added a separate probe loop that waits for GetClosestPeers to succeed
before running the actual test assertions.
This allows Kubo to respond to the GetClosestPeers() http routing v1 endpoint
as spec'ed here: https://github.com/ipfs/specs/pull/476
It is based on work from https://github.com/ipfs/boxo/pull/1021
We let IpfsNode implmement the contentRouter.Client interface with the new
method. We use our WAN-DHT to get the closest peers.
Additionally, Routing V1 HTTP API is exposed by default which enables light clients in browsers to use Kubo Gateway as delegated routing backend
Co-authored-by: Marcin Rataj <lidel@lidel.org>
* chore(deps): update go-libp2p to v0.44.0
- includes self-healing UPnP port mappings after router restarts
- update go-netroute to v0.3.0
- update quic-go to v0.55.0
- add changelog entry for UPnP fix
* docs: improve provide and UPnP clarity in changelog and docs
- add alert polling rationale to changelog
- add UPnP config note with default clarification
- clarify sweep timing and prefix length explanations
- add concrete examples for time offset and record holders
- improve workers stats formatting
- add See Also section to provide-stats.md
* docs: add RISC-V prebuilt binaries to changelog and README
- highlight linux-riscv64 availability with open hardware context
- update README with arm64 builds, remove 32-bit examples
* feat: provide stats
* added N/A
* format
* workers stats alignment
* ipfs provide stat --all --compact
* consolidating compact stat
* update column alignment
* flags combinations errors
* command description
* change schedule AvgPrefixLen to float
* changelog
* alignments
* provide stat description draft
* rephrased provide-stats.md
* linking provide-stats.md from command description
* documentation test
* fix: refactor provide stat command type handling
- add extractSweepingProvider() helper to reduce nested type switching
- extract lowWorkerThreshold constant for worker availability check
- fix --lan error handling to work with buffered providers
* docs: add clarifying comments
* fix(commands): improve provide stat compact mode
- prevent panic when both columns are empty
- fix column alignment with UTF-8 characters
- only track col0MaxWidth for first column (as intended)
* test: add tests for ipfs provide stat command
- test basic functionality, flags, JSON output
- test legacy provider behavior
- test integration with content scheduling
- test disabled provider configurations
- add parseSweepStats helper with t.Helper()
* docs: improve provide command help text
- update tagline to "Control and monitor content providing"
- simplify help descriptions
- make error messages more consistent
- update tests to match new error messages
* metrics rename
```
Next reprovide at:
Next prefix:
```
updated to:
```
Next region prefix:
Next region reprovide:
```
* docs: improve Provide system documentation clarity
Enhance documentation for the Provide system to better explain how provider
records work and the differences between sweep and legacy modes.
Changes to docs/config.md:
- Provide section: add clear explanation of provider records and their role
- Provide.DHT: add provider record lifecycle and two provider systems overview
- Provide.DHT.Interval: explain relationship to expiration, contrast sweep vs legacy behavior
- Provide.DHT.SweepEnabled: rewrite to explain legacy problem, sweep solution, and efficiency gains
- Monitoring section: prioritize command-line tools (ipfs provide stat) before Prometheus
Changes to core/commands/provide.go:
- ipfs provide stat help: add explanation of provider records, TTL expiration, and how sweep batching works
Changes to docs/changelogs/v0.39.md:
- Add context about why stats matter for monitoring provider health
- Emphasize real-time monitoring workflow with watch command
- Explain what users can observe (rates, queues, worker availability)
* depend on latest kad-dht master
* docs: nits
---------
Co-authored-by: Marcin Rataj <lidel@lidel.org>
* fix: add MFS operation limit for --flush=false
adds a global counter that tracks consecutive MFS operations performed
with --flush=false and fails with clear error after limit is reached.
this prevents unbounded memory growth while avoiding the data corruption
risks of auto-flushing.
- adds Internal.MFSNoFlushLimit config
- operations fail with actionable error at limit
- counter resets on successful flush or any --flush=true operation
- operations with --flush=true reset and don't count
this commit removes automatic flush from https://github.com/ipfs/kubo/pull/10971
and instead errors to encourage users of --flush=false to develop a habit
of calling 'ipfs files flush' periodically.
boxo will no longer auto-flush (https://github.com/ipfs/boxo/pull/1041) to
avoid corruption issues, and kubo applies the limit to 'ipfs files' commands
instead.
closes#10842
* test: add tests for MFSNoFlushLimit
tests verify the new Internal.MFSNoFlushLimit config option:
- default limit of 256 operations
- custom limit configuration
- counter reset on flush=true
- counter reset on explicit flush command
- limit=0 disables the feature
- multiple MFS command types count towards limit
* docs: explain why MFS operations fail instead of auto-flushing
addresses feedback from https://github.com/ipfs/kubo/pull/10985#pullrequestreview-3256250970
- clarify that automatic flushing at limit was considered but rejected
- explain the data corruption risks of auto-flushing
- guide users who want auto-flush to use --flush=true (default)
- document benefits of explicit failure for batch operations
* Filestore: provide Filestore nodes
When strategy is set to "all" (the blockstore does all the providing when a
block is written), no providing was happening to Filestore blocks that were
not written to the underlying blockstore (so, the DAG leaves, as they live in
the filesystem directly). This fixes that.
* docs: clarify filestore and urlstore fix in changelog
both filestore (local file references) and urlstore (HTTP/HTTPS URL
references) blocks are now properly provided shortly after initial add
* fix: use CheckIfPinnedWithType for pin ls with names
updates to use CheckIfPinnedWithType method from https://github.com/ipfs/boxo/pull/1035,
enabling efficient pin name retrieval for 'ipfs pin ls <cid> --names'
- uses new CheckIfPinnedWithType from boxo for type-specific pin checks
- pin names are now returned when listing specific CIDs with --names flag
* test: add CLI tests for pin ls with names
tests cover:
- pin ls with specific CIDs returning names
- pin ls without CID listing all pins with names
- pin ls with --type and --names combinations
- JSON output with and without names
- pin update preserving names
- error cases (invalid CID, unpinned CID)
* docs: add pin name improvements to v0.38 changelog
covers fix for ipfs pin ls --names with specific CIDs
and RPC pin name leak fix
* fix(rpc): support pin names in Add()
passes the Name field from PinAddSettings to the API request
adds test to verify pin names work via RPC
* test: add coverage for pin names functionality
- test special characters, unicode, long names
- test concurrent operations
- test persistence across daemon restarts
- test garbage collection preservation
- fix indirect pin test logic
* chore: boxo@main with boxo#1039
* fix(pin): improve pin ls robustness and validation
- add nil check for n.Pinning with early fail-fast validation
- use pin.StringToMode() for consistent type validation
- add edge case tests for invalid types and unpinned CIDs
* refactor: consolidate Provider/Reprovider into unified Provide config
- merge Provider and Reprovider configs into single Provide section
- add fs-repo-17-to-18 migration for config consolidation
- improve migration ergonomics with common package utilities
- convert deprecated "flat" strategy to "all" during migration
- improve Provide docs
* docs: add total_provide_count metric guidance
- document how to monitor provide success rates via prometheus metrics
- add performance comparison section to changelog
- explain how to evaluate sweep vs legacy provider effectiveness
* fix: add OpenTelemetry meter provider for metrics
- set up meter provider with Prometheus exporter in daemon
- enables metrics from external libs like go-libp2p-kad-dht
- fixes missing total_provide_count_total when SweepEnabled=true
- update docs to reflect actual metric names
---------
Co-authored-by: gammazero <11790789+gammazero@users.noreply.github.com>
Co-authored-by: guillaumemichel <guillaume@michel.id>
Co-authored-by: Daniel Norman <1992255+2color@users.noreply.github.com>
Co-authored-by: Hector Sanjuan <code@hector.link>
