skip '@helia/mfs - should have the same CID after creating a file' test
until helia implements IPIP-499 (tracking: https://github.com/ipfs/helia/issues/941)
the test fails because kubo now collapses single-block files to raw CIDs
while helia explicitly uses reduceSingleLeafToSelf: false
changes:
- run aegir directly instead of helia-interop binary (binary ignores --grep flags)
- cache node_modules keyed by @helia/interop version from npm registry
- skip npm install on cache hit (matches ipfs-webui caching pattern)
problem: `ipfs files write` in CIDv1 directories wrapped single-block
files in dag-pb even when raw-leaves was enabled, producing different
CIDs than `ipfs add --raw-leaves` for the same content.
fix: boxo now collapses single-block ProtoNode wrappers (with no
metadata) to RawNode in DagModifier.GetNode(). files with mtime/mode
stay as dag-pb since raw blocks cannot store UnixFS metadata.
also fixes sparse file writes where writing past EOF would lose data
because expandSparse didn't update the internal node pointer.
updates boxo to v0.36.1-0.20260203003133-7884ae23aaff
updates t0250-files-api.sh test hashes to match new behavior
`ipfs files` commands now respect these Import.* config options:
- UnixFSChunker: configures chunk size for `files write`
- UnixFSDirectoryMaxLinks: triggers HAMT sharding in `files mkdir`
- UnixFSHAMTDirectorySizeEstimation: controls size estimation mode
previously, MFS used hardcoded defaults ignoring user config.
changes:
- config/import.go: add UnixFSSplitterFunc() returning chunk.SplitterGen
- core/node/core.go: pass chunker, maxLinks, sizeEstimationMode to
mfs.NewRoot() via new boxo RootOption API
- core/commands/files.go: pass maxLinks and sizeEstimationMode to
mfs.Mkdir() and ensureContainingDirectoryExists(); document that
UnixFSFileMaxLinks doesn't apply to files write (trickle DAG limitation)
- test/cli/files_test.go: add tests for UnixFSDirectoryMaxLinks and
UnixFSChunker, including CID parity test with `ipfs add --trickle`
related: boxo@54e044f1b265
* fix http header when compress enabled for get command
Closes#2376
* fix(rpc): set Content-Type for ipfs get based on output format
- set application/x-tar when outputting tar (default and --archive)
- set application/gzip when compression is enabled (--compress)
- update go-ipfs-cmds with Tar encoding type and RFC 6713 compliant
MIME types (application/gzip instead of application/x-gzip)
* test(rpc): add Content-Type header tests for ipfs get
* feat(rpc): add Content-Type headers for binary responses
set proper Content-Type headers for RPC endpoints that return binary data:
- `dag export`: application/vnd.ipld.car
- `block get`: application/vnd.ipld.raw
- `diag profile`: application/zip
- `get`: application/x-tar or application/gzip (already worked, migrated to new API)
uses the new OctetStream encoding type and SetContentType() method
from go-ipfs-cmds to specify custom MIME types for binary responses.
refs: https://github.com/ipfs/kubo/issues/2376
* feat(rpc): add `ipfs name get` command for IPNS record retrieval
add dedicated command to retrieve raw signed IPNS records from the
routing system. returns protobuf-encoded IPNS record with Content-Type
`application/vnd.ipfs.ipns-record`.
this provides a more convenient alternative to `ipfs routing get /ipns/<name>`
which returns JSON with base64-encoded data. the raw output can be piped
directly to `ipfs name inspect`:
ipfs name get <name> | ipfs name inspect
spec: https://specs.ipfs.tech/ipns/ipns-record/
* feat(rpc): add `ipfs name put` command for IPNS record storage
adds `ipfs name put` to complement `ipfs name get`, allowing users to
store IPNS records obtained from external sources without needing the
private key. useful for backup, restore, and debugging workflows.
the command validates records by default (signature, sequence number).
use `--force` to bypass validation for testing how routing handles
malformed or outdated records.
also reorganizes test/cli files:
- rename http_rpc_* -> rpc_* to match existing convention
- merge name_get_put_test.go into name_test.go
- add file header comments documenting test purposes
* chore(deps): update go-ipfs-cmds to latest master
includes SetContentType() for dynamic Content-Type headers
---------
Co-authored-by: Marcin Rataj <lidel@lidel.org>
* feat(key): add 'ipfs key ls' as alias for 'ipfs key list'
Add 'ls' as an alias for the 'list' subcommand in 'ipfs key' to be
consistent with other ipfs commands like 'ipfs repo ls' and
'ipfs pin ls' which use 'ls' instead of 'list'.
Fixes#10976
Signed-off-by: Vedant Madane <6527493+VedantMadane@users.noreply.github.com>
* feat(key): make 'ipfs key ls' canonical, deprecate 'list'
aligns with other commands like 'ipfs pin ls' and 'ipfs files ls'.
'ipfs key list' still works but shows deprecation warning.
* fix(key): correct --key option description in verify command
was copy-pasted from sign command and said "signing" instead of "verifying"
---------
Signed-off-by: Vedant Madane <6527493+VedantMadane@users.noreply.github.com>
Co-authored-by: Marcin Rataj <lidel@lidel.org>
- docs/README.md: restructure to surface 20+ previously undiscoverable docs
- docs/README.md: fix broken github-issue-guide.md link (file was removed)
- docs/add-code-flow.md: rewrite with current code flow and mermaid diagrams
- docs/customizing.md, docs/gateway.md: use specs.ipfs.tech URLs
- README.md: fix orphan #nix anchor, use go.dev links, link to contributors graph
- remove stale docs/AUTHORS and docs/generate-authors.sh (last updated 2016)
* feat(dns): resolve libp2p.direct addresses locally without network I/O
p2p-forge hostnames encode IP addresses directly (e.g., 1-2-3-4.peerID.libp2p.direct -> 1.2.3.4),
so DNS queries are wasteful. kubo now parses these IPs in-memory.
- applies to both default libp2p.direct and custom AutoTLS.DomainSuffix
- TXT queries still delegate to network for ACME DNS-01 compatibility
- https://github.com/ipfs/kubo/pull/11140#discussion_r2683477754
use fallback to network DNS instead of returning errors when local
parsing fails, ensuring forward compatibility with future DNS records
- https://github.com/ipfs/kubo/pull/11140#discussion_r2683512408
add peerID validation using peer.Decode(), matching libp2p.direct
server behavior, with fallback on invalid peerID
- https://github.com/ipfs/kubo/pull/11140#discussion_r2683521930
document interaction with DNS.Resolvers in config.md
- https://github.com/ipfs/kubo/pull/11140#discussion_r2683526647
add AutoTLS.SkipDNSLookup config flag to disable local resolution
(useful for debugging or custom DNS override scenarios)
- https://github.com/ipfs/kubo/pull/11140#discussion_r2683533462
add E2E test verifying libp2p.direct resolves locally even when
DNS.Resolvers points to a broken server
additional improvements:
- use madns.BasicResolver interface instead of custom basicResolver
- add compile-time interface checks for p2pForgeResolver and madns.Resolver
- refactor tests: merge IPv4/IPv6, add helpers, use config.DefaultDomainSuffix
- improve changelog to explain public good benefit (reducing DNS load)
Fixes#11136
* ci: add caching to ipfs-webui interop tests
cache node_modules, Playwright browsers, and test build output
to speed up repeated CI runs. also use node version from
ipfs-webui/.tool-versions instead of hardcoding, and upload
test artifacts on failure.
* docs(ci): add header comment to interop workflow
explain what helia-interop and ipfs-webui jobs do
remove duplicate profile threshold tests from add_test.go since they
are fully covered by the data-driven tests in cid_profiles_test.go.
changes:
- improve test names to describe what threshold is being tested
- add inline documentation explaining each test's purpose
- add byte-precise helper IPFSAddDeterministicBytes for threshold tests
- remove ~200 lines of duplicated test code from add_test.go
- keep non-profile tests (pinning, symlinks, hidden files) in add_test.go
- add UnixFSDataType() helper to directly check UnixFS type via protobuf
- refactor threshold tests to use exact +1 byte calculations instead of +1 file
- verify directory type directly (ft.TDirectory vs ft.THAMTShard) instead of
inferring from link count
- clean up helper function signatures by removing unused cidLength parameter
switches to 6141039ad8
changes since 5cf22196ad0b:
- refactor(unixfs): use arithmetic for exact block size calculation
- refactor(unixfs): unify size tracking and make SizeEstimationMode immutable
- feat(unixfs): optimize SizeEstimationBlock and add mode/mtime tests
also clarifies that directory sharding globals affect both `ipfs add` and MFS.
* fix(routing): update kad-dht with peerstore address clone fix
closes#11116
See https://github.com/ipfs/kubo/issues/11116 for context of this fix
* fix(routing): update kad-dht with CPL exploration fix
fixes an infinite loop when all peers share the same CPL during provider exploration
See https://github.com/libp2p/go-libp2p-kad-dht/pull/1216
* fix(routing): update kad-dht with shutdown loop check
https://github.com/libp2p/go-libp2p-kad-dht/pull/1217
* depend on latest kad-dht fix
* bump kad-dht to v0.37.0
---------
Co-authored-by: guillaumemichel <guillaume@michel.id>
Co-authored-by: Guillaume Michel <guillaumemichel@users.noreply.github.com>
* chore: exclude ancient +incompatible versions from go.mod
prevents Dependabot from failing when it tries to update
go-ipfs-cmds and go-libp2p directly and resolves to
pre-Go-modules v2.x/v6.x versions that reference deleted packages
* chore(deps): group opentelemetry, prometheus, and uber packages
reduces PR noise by batching related observability dependencies
* ci: add workflow to run make mod_tidy on Dependabot PRs
ensures all go.mod/go.sum files stay in sync when Dependabot
updates dependencies in the root module
supports manual dispatch with PR number for existing PRs
experimental setup to surface missed ecosystem updates via monthly PRs
groups ipfs, libp2p, multiformats, ipld, and golang.org/x dependencies
to reduce noise while ensuring visibility into available updates
add test that confirms kubo uses balanced layout (all leaves at same
depth) rather than balanced-packed (varying depths). creates 45MiB file
to trigger multi-level DAG and walks it to verify leaf depth uniformity.
includes trickle subtest to validate test logic can detect varying depths.
supports CAR export via DAG_LAYOUT_CAR_OUTPUT env var for test vectors.
Previously, resolving symlinks required two flags:
- --dereference-args: resolved symlinks passed as CLI arguments
- --dereference-symlinks: resolved symlinks inside directories
Now --dereference-symlinks handles both cases. Users only need one flag
to fully dereference symlinks when adding files to IPFS.
The deprecated --dereference-args still works for backwards compatibility
but is no longer necessary.
add comprehensive test suite for UnixFS CID determinism per IPIP-499:
- verify exact HAMT threshold boundary for both estimation modes:
- v0-2015 (links): sum(name_len + cid_len) == 262144
- v1-2025 (block): serialized block size == 262144
- verify HAMT triggers at threshold + 1 byte for both profiles
- add all deterministic CIDs for cross-implementation testing
also wires SizeEstimationMode through CLI/API, allowing
Import.UnixFSHAMTSizeEstimation config to take effect.
bumps boxo to ipfs/boxo@6707376 which aligns HAMT threshold with
JS implementation (uses > instead of >=), fixing CID determinism
at the exact 256 KiB boundary.
add CLI flags for controlling file collection behavior during ipfs add:
- `--dereference-symlinks`: recursively resolve symlinks to their target
content (replaces deprecated --dereference-args which only worked on
CLI arguments). wired through go-ipfs-cmds to boxo's SerialFileOptions.
- `--empty-dirs` / `-E`: include empty directories (default: true)
- `--hidden` / `-H`: include hidden files (default: false)
these flags are CLI-only and not wired to Import.* config options because
go-ipfs-cmds library handles input file filtering before the directory
tree is passed to kubo. removed unused Import.UnixFSSymlinkMode config
option that was defined but never actually read by the CLI.
also:
- wire --trickle to Import.UnixFSDAGLayout config default
- update go-ipfs-cmds to v0.15.1-0.20260117043932-17687e216294
- add SYMLINK HANDLING section to ipfs add help text
- add CLI tests for all three flags
ref: https://github.com/ipfs/specs/pull/499
* Implements the -l/--long flag for the ipfs ls command to display Unix-style
file permissions and modification times, similar to the traditional ls -l.
When the --long flag is used, the output includes:
- File mode/permissions in Unix format (e.g., -rw-r--r--, drwxr-xr-x)
- File hash (CID)
- File size (when --size is also specified)
- Modification time in human-readable format
- File name
The permission string implementation handles all file types and special bits:
- File types: regular (-), directory (d), symlink (l), named pipe (p),
socket (s), character device (c), block device (b)
- Special permission bits: setuid (s/S), setgid (s/S), sticky (t/T)
- Lowercase when execute bit is set, uppercase when not set
The timestamp format follows Unix ls conventions:
- Recent files (within 6 months): "Jan 02 15:04"
- Older files: "Jan 02 2006"
Signed-off-by: sneax <paladesh600@gmail.com>
* fix(ls): correct --long flag header order and help text
- fix header column order: was "Mode Hash Size Name ModTime" but data
outputs "Mode Hash Size ModTime Name", now headers match data order
- remove redundant if/else branch in directory output that had
identical code in both branches
- add example output to help text showing format with mode, hash,
size, mtime, and name columns
- document that files without preserved metadata show '----------'
for mode and '-' for mtime
- add changelog entry for v0.40
* test(ls): add format stability tests for --long flag
add tests to prevent formatting regressions in ipfs ls --long output:
unit tests (core/commands/ls_test.go):
- TestFormatMode: 20 cases covering all file types (regular, dir,
symlink, pipe, socket, block/char devices) and special permission
bits (setuid, setgid, sticky with/without execute)
- TestFormatModTime: zero time, old time (year format), future time,
format length consistency
integration tests (test/cli/ls_test.go):
- explicit full output comparison with deterministic CIDs to catch
any formatting changes
- header column order verification for --long with --size=true/false
- files without preserved metadata (---------- and - placeholders)
- directory output (trailing slash, d prefix in mode)
requested in: https://github.com/ipfs/kubo/pull/11103#issuecomment-3745043561
* fix(ls): improve --long flag docs and fix minor issues
- improved godocs for formatMode and formatModTime functions
- fixed permBit signature: char rune → char byte (avoids unnecessary cast)
- clarified help text: mode/mtime are optional UnixFS metadata
- documented that times are displayed in UTC
- fixed flaky time test by using 1 month ago instead of 1 hour
- removed hardcoded CID assertion that would break on DAG changes
* fix(ls): show "-" for missing mode in --long output
display "-" instead of "----------" when mode metadata is not preserved.
this avoids ambiguity with Unix mode 0000 and matches how missing mtime
is already displayed. follows common Unix tool conventions (ps, netstat)
where "-" indicates "not available".
---------
Signed-off-by: sneax <paladesh600@gmail.com>
Co-authored-by: Marcin Rataj <lidel@lidel.org>
* feat(pubsub): persistent seqno validation and diagnostic commands
- upgrade go-libp2p-pubsub to v0.15.0
- add persistent seqno validator using BasicSeqnoValidator
stores max seen seqno per peer at /pubsub/seqno/<peerid>
survives daemon restarts, addresses message cycling in large networks (#9665)
- add `ipfs pubsub reset` command to clear validator state
- add `ipfs diag datastore get/count` commands for datastore inspection
requires daemon to be stopped, useful for debugging
- change pubsub status from Deprecated to Experimental
- add CLI tests for pubsub and diag datastore commands
- remove flaky pubsub_msg_seen_cache_test.go (replaced by CLI tests)
* fix(pubsub): improve reset command and add deprecation warnings
- use batched delete for efficient bulk reset
- check key existence before reporting deleted count
- sync datastore after deletions to ensure persistence
- show "no validator state found" when resetting non-existent peer
- log deprecation warnings when using --enable-pubsub-experiment
or --enable-namesys-pubsub CLI flags
* refactor(test): add datastore helpers to test harness
---------
Co-authored-by: Andrew Gillis <11790789+gammazero@users.noreply.github.com>
* feat(config): add Gateway.MaxRequestDuration option
exposes the previously hardcoded 1 hour gateway request deadline as a
configurable option, allowing operators to adjust it to fit deployment
needs. protects gateway from edge cases and slow client attacks.
boxo: https://github.com/ipfs/boxo/pull/1079
* test(gateway): add MaxRequestDuration integration test
verifies config is wired correctly and 504 is returned when exceeded
* docs: add MaxRequestDuration to gateway production guide
---------
Co-authored-by: Andrew Gillis <11790789+gammazero@users.noreply.github.com>
