15753 Commits

This Branch

This Branch

All Branches

Author	SHA1	Message	Date
Marcin Rataj	4c6dab73cf	feat: update ipfs-webui to v4.11.0 (#11182) ... https://github.com/ipfs/ipfs-webui/releases/tag/v4.11.0	2026-02-05 05:02:21 +01:00
Marcin Rataj	67c89bbd7e	feat(config): add Import.* for CID Profiles from IPIP-499 (#11148) ... * feat(config): Import.* and unixfs-v1-2025 profile implements IPIP-499: add config options for controlling UnixFS DAG determinism and introduces `unixfs-v1-2025` and `unixfs-v0-2015` profiles for cross-implementation CID reproducibility. changes: - add Import.* fields: HAMTDirectorySizeEstimation, SymlinkMode, DAGLayout, IncludeEmptyDirectories, IncludeHidden - add validation for all Import.* config values - add unixfs-v1-2025 profile (recommended for new data) - add unixfs-v0-2015 profile (alias: legacy-cid-v0) - remove deprecated test-cid-v1 and test-cid-v1-wide profiles - wire Import.HAMTSizeEstimationMode() to boxo globals - update go.mod to use boxo with SizeEstimationMode support ref: https://specs.ipfs.tech/ipips/ipip-0499/ * feat(add): add --dereference-symlinks, --empty-dirs, --hidden CLI flags add CLI flags for controlling file collection behavior during ipfs add: - `--dereference-symlinks`: recursively resolve symlinks to their target content (replaces deprecated --dereference-args which only worked on CLI arguments). wired through go-ipfs-cmds to boxo's SerialFileOptions. - `--empty-dirs` / `-E`: include empty directories (default: true) - `--hidden` / `-H`: include hidden files (default: false) these flags are CLI-only and not wired to Import.* config options because go-ipfs-cmds library handles input file filtering before the directory tree is passed to kubo. removed unused Import.UnixFSSymlinkMode config option that was defined but never actually read by the CLI. also: - wire --trickle to Import.UnixFSDAGLayout config default - update go-ipfs-cmds to v0.15.1-0.20260117043932-17687e216294 - add SYMLINK HANDLING section to ipfs add help text - add CLI tests for all three flags ref: https://github.com/ipfs/specs/pull/499 * test(add): add CID profile tests and wire SizeEstimationMode add comprehensive test suite for UnixFS CID determinism per IPIP-499: - verify exact HAMT threshold boundary for both estimation modes: - v0-2015 (links): sum(name_len + cid_len) == 262144 - v1-2025 (block): serialized block size == 262144 - verify HAMT triggers at threshold + 1 byte for both profiles - add all deterministic CIDs for cross-implementation testing also wires SizeEstimationMode through CLI/API, allowing Import.UnixFSHAMTSizeEstimation config to take effect. bumps boxo to ipfs/boxo@6707376 which aligns HAMT threshold with JS implementation (uses > instead of >=), fixing CID determinism at the exact 256 KiB boundary. * feat(add): --dereference-symlinks now resolves all symlinks Previously, resolving symlinks required two flags: - --dereference-args: resolved symlinks passed as CLI arguments - --dereference-symlinks: resolved symlinks inside directories Now --dereference-symlinks handles both cases. Users only need one flag to fully dereference symlinks when adding files to IPFS. The deprecated --dereference-args still works for backwards compatibility but is no longer necessary. * chore: update boxo and improve changelog - update boxo to ebdaf07c (nil filter fix, thread-safety docs) - simplify changelog for IPIP-499 section - shorten test names, move context to comments * chore: update boxo to 5cf22196 * chore: apply suggestions from code review Co-authored-by: Andrew Gillis <11790789+gammazero@users.noreply.github.com> * test(add): verify balanced DAG layout produces uniform leaf depth add test that confirms kubo uses balanced layout (all leaves at same depth) rather than balanced-packed (varying depths). creates 45MiB file to trigger multi-level DAG and walks it to verify leaf depth uniformity. includes trickle subtest to validate test logic can detect varying depths. supports CAR export via DAG_LAYOUT_CAR_OUTPUT env var for test vectors. * chore(deps): update boxo to 6141039ad8ef switches to 6141039ad8 changes since 5cf22196ad0b: - refactor(unixfs): use arithmetic for exact block size calculation - refactor(unixfs): unify size tracking and make SizeEstimationMode immutable - feat(unixfs): optimize SizeEstimationBlock and add mode/mtime tests also clarifies that directory sharding globals affect both `ipfs add` and MFS. * test(cli): improve HAMT threshold tests with exact +1 byte verification - add UnixFSDataType() helper to directly check UnixFS type via protobuf - refactor threshold tests to use exact +1 byte calculations instead of +1 file - verify directory type directly (ft.TDirectory vs ft.THAMTShard) instead of inferring from link count - clean up helper function signatures by removing unused cidLength parameter * test(cli): consolidate profile tests into cid_profiles_test.go remove duplicate profile threshold tests from add_test.go since they are fully covered by the data-driven tests in cid_profiles_test.go. changes: - improve test names to describe what threshold is being tested - add inline documentation explaining each test's purpose - add byte-precise helper IPFSAddDeterministicBytes for threshold tests - remove ~200 lines of duplicated test code from add_test.go - keep non-profile tests (pinning, symlinks, hidden files) in add_test.go * chore: update to rebased boxo and go-ipfs-cmds PRs * docs: add HAMT threshold fix details to changelog * feat(mfs): use Import config for CID version and hash function make MFS commands (files cp, files write, files mkdir, files chcid) respect Import.CidVersion and Import.HashFunction config settings when CLI options are not explicitly provided. also add tests for: - files write respects Import.UnixFSRawLeaves=true - single-block file: files write produces same CID as ipfs add - updated comments clarifying CID parity with ipfs add * feat(files): wire Import.UnixFSChunker and UnixFSDirectoryMaxLinks to MFS `ipfs files` commands now respect these Import.* config options: - UnixFSChunker: configures chunk size for `files write` - UnixFSDirectoryMaxLinks: triggers HAMT sharding in `files mkdir` - UnixFSHAMTDirectorySizeEstimation: controls size estimation mode previously, MFS used hardcoded defaults ignoring user config. changes: - config/import.go: add UnixFSSplitterFunc() returning chunk.SplitterGen - core/node/core.go: pass chunker, maxLinks, sizeEstimationMode to mfs.NewRoot() via new boxo RootOption API - core/commands/files.go: pass maxLinks and sizeEstimationMode to mfs.Mkdir() and ensureContainingDirectoryExists(); document that UnixFSFileMaxLinks doesn't apply to files write (trickle DAG limitation) - test/cli/files_test.go: add tests for UnixFSDirectoryMaxLinks and UnixFSChunker, including CID parity test with `ipfs add --trickle` related: boxo@54e044f1b265 * feat(files): wire Import.UnixFSHAMTDirectoryMaxFanout and UnixFSHAMTDirectorySizeThreshold wire remaining HAMT config options to MFS root: - Import.UnixFSHAMTDirectoryMaxFanout via mfs.WithMaxHAMTFanout - Import.UnixFSHAMTDirectorySizeThreshold via mfs.WithHAMTShardingSize add CLI tests: - files mkdir respects Import.UnixFSHAMTDirectoryMaxFanout - files mkdir respects Import.UnixFSHAMTDirectorySizeThreshold - config change takes effect after daemon restart add UnixFSHAMTFanout() helper to test harness update boxo to ac97424d99ab90e097fc7c36f285988b596b6f05 * fix(mfs): single-block files in CIDv1 dirs now produce raw CIDs problem: `ipfs files write` in CIDv1 directories wrapped single-block files in dag-pb even when raw-leaves was enabled, producing different CIDs than `ipfs add --raw-leaves` for the same content. fix: boxo now collapses single-block ProtoNode wrappers (with no metadata) to RawNode in DagModifier.GetNode(). files with mtime/mode stay as dag-pb since raw blocks cannot store UnixFS metadata. also fixes sparse file writes where writing past EOF would lose data because expandSparse didn't update the internal node pointer. updates boxo to v0.36.1-0.20260203003133-7884ae23aaff updates t0250-files-api.sh test hashes to match new behavior * chore(test): use Go 1.22+ range-over-int syntax * chore: update boxo to c6829fe26860 - fix typo in files write help text - update boxo with CI fixes (gofumpt, race condition in test) * chore: update go-ipfs-cmds to 192ec9d15c1f includes binary content types fix: gzip, zip, vnd.ipld.car, vnd.ipld.raw, vnd.ipfs.ipns-record * chore: update boxo to 0a22cde9225c includes refactor of maxLinks check in addLinkChild (review feedback). * ci: fix helia-interop and improve caching skip '@helia/mfs - should have the same CID after creating a file' test until helia implements IPIP-499 (tracking: https://github.com/ipfs/helia/issues/941) the test fails because kubo now collapses single-block files to raw CIDs while helia explicitly uses reduceSingleLeafToSelf: false changes: - run aegir directly instead of helia-interop binary (binary ignores --grep flags) - cache node_modules keyed by @helia/interop version from npm registry - skip npm install on cache hit (matches ipfs-webui caching pattern) * chore: update boxo to 1e30b954 includes latest upstream changes from boxo main * chore: update go-ipfs-cmds to 1b2a641ed6f6 * chore: update boxo to f188f79fd412 switches to boxo@main after merging https://github.com/ipfs/boxo/pull/1088 * chore: update go-ipfs-cmds to af9bcbaf5709 switches to go-ipfs-cmds@master after merging https://github.com/ipfs/go-ipfs-cmds/pull/315 --------- Co-authored-by: Andrew Gillis <11790789+gammazero@users.noreply.github.com>	2026-02-04 22:03:15 +01:00
Marcin Rataj	ff4bb10989	chore: replace libp2p.io URL with Internet Archive (#11181) ... swap to last working snapshots until libp2p project fixes their websites Right now its either broken due to Fleek shutting down: https://web.archive.org/web/20260201232813/https://docs.libp2p.io/concepts/pubsub/overview/ or URLs changed by being moved to new place at: https://web.archive.org/web/20260204171903/https://libp2p.io/guides/circuit-relay/ we can undo/switch to real URLs once things stabilize for a few months	2026-02-04 18:50:58 +01:00
Marcin Rataj	df9574e090	test: IPIP-523 format query precedence over Accept header (#11086) ... * test: IPIP-523 format query precedence over Accept header update boxo to ipfs/boxo#1074 and gateway-conformance tests to ipfs/gateway-conformance#252 for testing IPIP-523 changes where ?format= URL query parameter takes precedence over Accept HTTP header * chore: update boxo and gateway-conformance for IPIP-523 testing - boxo@9aeb0c835899fedb4f886040a4ccf3ba09fd97d4 - gateway-conformance@c82a2a9bc79303e0f07216a80cf454ef2a1e042c * chore(ci): switch to gateway-conformance@v0.9 * chore: update boxo with IPIP-523 changes * chore: update boxo to main after ipfs/boxo#1074 merge * docs: add gateway-conformance v0.9 to changelog	2026-02-04 05:48:54 +01:00
Marcin Rataj	ea8a1ced79	docs: update go-libp2p changelog entry to v0.47.0 ... add missing fixes from v0.47.0 release	2026-02-04 02:31:29 +01:00
Andrew Gillis	dd882c0bdb	fix: recognize content-type application/x-tar (#11180) ... * fix: recognize content-type application/x-tar For context, see: https://github.com/ipfs/go-ipfs-cmds/pull/320 Closes: #11179	2026-02-03 07:32:21 -08:00
dependabot[bot]	22ca729eb3	chore(deps): bump github.com/hashicorp/go-version from 1.7.0 to 1.8.0 (#11160) ... * chore(deps): bump github.com/hashicorp/go-version from 1.7.0 to 1.8.0 Bumps [github.com/hashicorp/go-version](https://github.com/hashicorp/go-version) from 1.7.0 to 1.8.0. - [Release notes](https://github.com/hashicorp/go-version/releases) - [Changelog](https://github.com/hashicorp/go-version/blob/main/CHANGELOG.md) - [Commits](https://github.com/hashicorp/go-version/compare/v1.7.0...v1.8.0) --- updated-dependencies: - dependency-name: github.com/hashicorp/go-version dependency-version: 1.8.0 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> * chore: run make mod_tidy --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Andrew Gillis <11790789+gammazero@users.noreply.github.com>	2026-02-02 22:08:57 -08:00
dependabot[bot]	3522136c4f	chore(deps): bump github.com/multiformats/go-multiaddr-dns from 0.4.1 to 0.5.0 in the ipfs-ecosystem group (#11177) ... * chore(deps): bump github.com/multiformats/go-multiaddr-dns Bumps the ipfs-ecosystem group with 1 update: [github.com/multiformats/go-multiaddr-dns](https://github.com/multiformats/go-multiaddr-dns). Updates `github.com/multiformats/go-multiaddr-dns` from 0.4.1 to 0.5.0 - [Release notes](https://github.com/multiformats/go-multiaddr-dns/releases) - [Commits](https://github.com/multiformats/go-multiaddr-dns/compare/v0.4.1...v0.5.0) --- updated-dependencies: - dependency-name: github.com/multiformats/go-multiaddr-dns dependency-version: 0.5.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: ipfs-ecosystem ... Signed-off-by: dependabot[bot] <support@github.com> * chore: run make mod_tidy --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Andrew Gillis <11790789+gammazero@users.noreply.github.com>	2026-02-02 08:41:16 -08:00
dependabot[bot]	07ad431b00	chore(deps): bump github.com/coreos/go-systemd/v22 from 22.5.0 to 22.7.0 (#11178) ... Bumps [github.com/coreos/go-systemd/v22](https://github.com/coreos/go-systemd) from 22.5.0 to 22.7.0. - [Release notes](https://github.com/coreos/go-systemd/releases) - [Commits](https://github.com/coreos/go-systemd/compare/v22.5.0...v22.7.0) --- updated-dependencies: - dependency-name: github.com/coreos/go-systemd/v22 dependency-version: 22.7.0 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>	2026-02-02 08:24:30 -08:00
Andrew Gillis	77ed3dd0ef	feat(rpc): Content-Type headers and IPNS record get/put (#11067) ... * fix http header when compress enabled for get command Closes #2376 * fix(rpc): set Content-Type for ipfs get based on output format - set application/x-tar when outputting tar (default and --archive) - set application/gzip when compression is enabled (--compress) - update go-ipfs-cmds with Tar encoding type and RFC 6713 compliant MIME types (application/gzip instead of application/x-gzip) * test(rpc): add Content-Type header tests for ipfs get * feat(rpc): add Content-Type headers for binary responses set proper Content-Type headers for RPC endpoints that return binary data: - `dag export`: application/vnd.ipld.car - `block get`: application/vnd.ipld.raw - `diag profile`: application/zip - `get`: application/x-tar or application/gzip (already worked, migrated to new API) uses the new OctetStream encoding type and SetContentType() method from go-ipfs-cmds to specify custom MIME types for binary responses. refs: https://github.com/ipfs/kubo/issues/2376 * feat(rpc): add `ipfs name get` command for IPNS record retrieval add dedicated command to retrieve raw signed IPNS records from the routing system. returns protobuf-encoded IPNS record with Content-Type `application/vnd.ipfs.ipns-record`. this provides a more convenient alternative to `ipfs routing get /ipns/<name>` which returns JSON with base64-encoded data. the raw output can be piped directly to `ipfs name inspect`: ipfs name get <name> | ipfs name inspect spec: https://specs.ipfs.tech/ipns/ipns-record/ * feat(rpc): add `ipfs name put` command for IPNS record storage adds `ipfs name put` to complement `ipfs name get`, allowing users to store IPNS records obtained from external sources without needing the private key. useful for backup, restore, and debugging workflows. the command validates records by default (signature, sequence number). use `--force` to bypass validation for testing how routing handles malformed or outdated records. also reorganizes test/cli files: - rename http_rpc_* -> rpc_* to match existing convention - merge name_get_put_test.go into name_test.go - add file header comments documenting test purposes * chore(deps): update go-ipfs-cmds to latest master includes SetContentType() for dynamic Content-Type headers --------- Co-authored-by: Marcin Rataj <lidel@lidel.org>	2026-01-30 23:41:55 +01:00
Vedant Madane	59b5d6ab4d	feat(key): add 'ipfs key ls' as alias for 'ipfs key list' (#11147) ... * feat(key): add 'ipfs key ls' as alias for 'ipfs key list' Add 'ls' as an alias for the 'list' subcommand in 'ipfs key' to be consistent with other ipfs commands like 'ipfs repo ls' and 'ipfs pin ls' which use 'ls' instead of 'list'. Fixes #10976 Signed-off-by: Vedant Madane <6527493+VedantMadane@users.noreply.github.com> * feat(key): make 'ipfs key ls' canonical, deprecate 'list' aligns with other commands like 'ipfs pin ls' and 'ipfs files ls'. 'ipfs key list' still works but shows deprecation warning. * fix(key): correct --key option description in verify command was copy-pasted from sign command and said "signing" instead of "verifying" --------- Signed-off-by: Vedant Madane <6527493+VedantMadane@users.noreply.github.com> Co-authored-by: Marcin Rataj <lidel@lidel.org>	2026-01-30 19:39:26 +01:00
Marcin Rataj	9539b4d8b8	docs: cleanup broken links and outdated content (#11100) ... - docs/README.md: restructure to surface 20+ previously undiscoverable docs - docs/README.md: fix broken github-issue-guide.md link (file was removed) - docs/add-code-flow.md: rewrite with current code flow and mermaid diagrams - docs/customizing.md, docs/gateway.md: use specs.ipfs.tech URLs - README.md: fix orphan #nix anchor, use go.dev links, link to contributors graph - remove stale docs/AUTHORS and docs/generate-authors.sh (last updated 2016)	2026-01-30 19:38:42 +01:00
Marcin Rataj	7de7af0820	feat(dns): skip DNS lookups for AutoTLS hostnames (#11140) ... * feat(dns): resolve libp2p.direct addresses locally without network I/O p2p-forge hostnames encode IP addresses directly (e.g., 1-2-3-4.peerID.libp2p.direct -> 1.2.3.4), so DNS queries are wasteful. kubo now parses these IPs in-memory. - applies to both default libp2p.direct and custom AutoTLS.DomainSuffix - TXT queries still delegate to network for ACME DNS-01 compatibility - https://github.com/ipfs/kubo/pull/11140#discussion_r2683477754 use fallback to network DNS instead of returning errors when local parsing fails, ensuring forward compatibility with future DNS records - https://github.com/ipfs/kubo/pull/11140#discussion_r2683512408 add peerID validation using peer.Decode(), matching libp2p.direct server behavior, with fallback on invalid peerID - https://github.com/ipfs/kubo/pull/11140#discussion_r2683521930 document interaction with DNS.Resolvers in config.md - https://github.com/ipfs/kubo/pull/11140#discussion_r2683526647 add AutoTLS.SkipDNSLookup config flag to disable local resolution (useful for debugging or custom DNS override scenarios) - https://github.com/ipfs/kubo/pull/11140#discussion_r2683533462 add E2E test verifying libp2p.direct resolves locally even when DNS.Resolvers points to a broken server additional improvements: - use madns.BasicResolver interface instead of custom basicResolver - add compile-time interface checks for p2pForgeResolver and madns.Resolver - refactor tests: merge IPv4/IPv6, add helpers, use config.DefaultDomainSuffix - improve changelog to explain public good benefit (reducing DNS load) Fixes #11136	2026-01-30 17:20:56 +01:00
Marcin Rataj	ef99e0a0f7	refactor(ci): add caching to ipfs-webui interop tests (#11173) ... * ci: add caching to ipfs-webui interop tests cache node_modules, Playwright browsers, and test build output to speed up repeated CI runs. also use node version from ipfs-webui/.tool-versions instead of hardcoding, and upload test artifacts on failure. * docs(ci): add header comment to interop workflow explain what helia-interop and ipfs-webui jobs do	2026-01-29 02:56:42 +01:00
dependabot[bot]	b57278017a	chore(deps): bump github.com/tidwall/gjson from 1.16.0 to 1.18.0 (#11168) ... Bumps [github.com/tidwall/gjson](https://github.com/tidwall/gjson) from 1.16.0 to 1.18.0. - [Commits](https://github.com/tidwall/gjson/compare/v1.16.0...v1.18.0) --- updated-dependencies: - dependency-name: github.com/tidwall/gjson dependency-version: 1.18.0 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>	2026-01-27 21:43:52 -08:00
Andrew Gillis	a137401272	Upgrade to Boxo v0.36.0 (#11175) ... * Upgrade to Boxo v0.36.0 * sharness: add missing metrics	2026-01-27 15:51:01 -08:00
Marcin Rataj	5ccdcdd4fc	chore(deps): ignore datastore backends in dependabot (#11169) ... pebble, leveldb, and badger should be updated via go-ds-* wrappers to ensure compatibility	2026-01-24 01:32:56 +01:00
Andrew Gillis	f6a5c5f5c5	chore: upgrade go-ds-pebble to v0.5.9 (#11170) ... - upgrade go-ds-pebble to [v0.5.9](https://github.com/ipfs/go-ds-pebble/releases/tag/v0.5.9) - git ignore ipfswatch binary	2026-01-22 23:38:59 -08:00
Marcin Rataj	bc9b388610	fix(kad-dht): peerstore and exploration loop (#11139) ... * fix(routing): update kad-dht with peerstore address clone fix closes #11116 See https://github.com/ipfs/kubo/issues/11116 for context of this fix * fix(routing): update kad-dht with CPL exploration fix fixes an infinite loop when all peers share the same CPL during provider exploration See https://github.com/libp2p/go-libp2p-kad-dht/pull/1216 * fix(routing): update kad-dht with shutdown loop check https://github.com/libp2p/go-libp2p-kad-dht/pull/1217 * depend on latest kad-dht fix * bump kad-dht to v0.37.0 --------- Co-authored-by: guillaumemichel <guillaume@michel.id> Co-authored-by: Guillaume Michel <guillaumemichel@users.noreply.github.com>	2026-01-23 06:41:47 +01:00
dependabot[bot]	4a80af4b54	chore(deps): bump actions/checkout from 4 to 6 (#11166) ... Bumps [actions/checkout](https://github.com/actions/checkout) from 4 to 6. - [Release notes](https://github.com/actions/checkout/releases) - [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md) - [Commits](https://github.com/actions/checkout/compare/v4...v6) --- updated-dependencies: - dependency-name: actions/checkout dependency-version: '6' dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>	2026-01-23 06:35:20 +01:00
dependabot[bot]	b7a35a4bd0	chore(deps): bump actions/setup-go from 5 to 6 (#11165) ... Bumps [actions/setup-go](https://github.com/actions/setup-go) from 5 to 6. - [Release notes](https://github.com/actions/setup-go/releases) - [Commits](https://github.com/actions/setup-go/compare/v5...v6) --- updated-dependencies: - dependency-name: actions/setup-go dependency-version: '6' dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>	2026-01-23 06:34:24 +01:00
Marcin Rataj	6d253a6b80	chore(ci): dependabot fixes (#11164) ... * chore: exclude ancient +incompatible versions from go.mod prevents Dependabot from failing when it tries to update go-ipfs-cmds and go-libp2p directly and resolves to pre-Go-modules v2.x/v6.x versions that reference deleted packages * chore(deps): group opentelemetry, prometheus, and uber packages reduces PR noise by batching related observability dependencies * ci: add workflow to run make mod_tidy on Dependabot PRs ensures all go.mod/go.sum files stay in sync when Dependabot updates dependencies in the root module supports manual dispatch with PR number for existing PRs	2026-01-23 06:17:32 +01:00
Marcin Rataj	1128d81042	chore: add gomod dependabot config (#11153) ... experimental setup to surface missed ecosystem updates via monthly PRs groups ipfs, libp2p, multiformats, ipld, and golang.org/x dependencies to reduce noise while ensuring visibility into available updates	2026-01-23 05:27:30 +01:00
Guillaume Michel	8848b537cf	fix(commands/reprovide): update manual reprovide error message (#11151) ... Reprovide Sweep is now the default behaviour since v0.39.0. Update manual reprovide error message to reflect this change.	2026-01-21 05:59:56 +01:00
segfault_bits	c1fd4d70f5	feat(cli): ls --long (#11103) ... * Implements the -l/--long flag for the ipfs ls command to display Unix-style file permissions and modification times, similar to the traditional ls -l. When the --long flag is used, the output includes: - File mode/permissions in Unix format (e.g., -rw-r--r--, drwxr-xr-x) - File hash (CID) - File size (when --size is also specified) - Modification time in human-readable format - File name The permission string implementation handles all file types and special bits: - File types: regular (-), directory (d), symlink (l), named pipe (p), socket (s), character device (c), block device (b) - Special permission bits: setuid (s/S), setgid (s/S), sticky (t/T) - Lowercase when execute bit is set, uppercase when not set The timestamp format follows Unix ls conventions: - Recent files (within 6 months): "Jan 02 15:04" - Older files: "Jan 02 2006" Signed-off-by: sneax <paladesh600@gmail.com> * fix(ls): correct --long flag header order and help text - fix header column order: was "Mode Hash Size Name ModTime" but data outputs "Mode Hash Size ModTime Name", now headers match data order - remove redundant if/else branch in directory output that had identical code in both branches - add example output to help text showing format with mode, hash, size, mtime, and name columns - document that files without preserved metadata show '----------' for mode and '-' for mtime - add changelog entry for v0.40 * test(ls): add format stability tests for --long flag add tests to prevent formatting regressions in ipfs ls --long output: unit tests (core/commands/ls_test.go): - TestFormatMode: 20 cases covering all file types (regular, dir, symlink, pipe, socket, block/char devices) and special permission bits (setuid, setgid, sticky with/without execute) - TestFormatModTime: zero time, old time (year format), future time, format length consistency integration tests (test/cli/ls_test.go): - explicit full output comparison with deterministic CIDs to catch any formatting changes - header column order verification for --long with --size=true/false - files without preserved metadata (---------- and - placeholders) - directory output (trailing slash, d prefix in mode) requested in: https://github.com/ipfs/kubo/pull/11103#issuecomment-3745043561 * fix(ls): improve --long flag docs and fix minor issues - improved godocs for formatMode and formatModTime functions - fixed permBit signature: char rune → char byte (avoids unnecessary cast) - clarified help text: mode/mtime are optional UnixFS metadata - documented that times are displayed in UTC - fixed flaky time test by using 1 month ago instead of 1 hour - removed hardcoded CID assertion that would break on DAG changes * fix(ls): show "-" for missing mode in --long output display "-" instead of "----------" when mode metadata is not preserved. this avoids ambiguity with Unix mode 0000 and matches how missing mtime is already displayed. follows common Unix tool conventions (ps, netstat) where "-" indicates "not available". --------- Signed-off-by: sneax <paladesh600@gmail.com> Co-authored-by: Marcin Rataj <lidel@lidel.org>	2026-01-16 02:27:32 +01:00
Marcin Rataj	824a47ae11	feat(pubsub): persistent validation and diagnostic commands (#11110) ... * feat(pubsub): persistent seqno validation and diagnostic commands - upgrade go-libp2p-pubsub to v0.15.0 - add persistent seqno validator using BasicSeqnoValidator stores max seen seqno per peer at /pubsub/seqno/<peerid> survives daemon restarts, addresses message cycling in large networks (#9665) - add `ipfs pubsub reset` command to clear validator state - add `ipfs diag datastore get/count` commands for datastore inspection requires daemon to be stopped, useful for debugging - change pubsub status from Deprecated to Experimental - add CLI tests for pubsub and diag datastore commands - remove flaky pubsub_msg_seen_cache_test.go (replaced by CLI tests) * fix(pubsub): improve reset command and add deprecation warnings - use batched delete for efficient bulk reset - check key existence before reporting deleted count - sync datastore after deletions to ensure persistence - show "no validator state found" when resetting non-existent peer - log deprecation warnings when using --enable-pubsub-experiment or --enable-namesys-pubsub CLI flags * refactor(test): add datastore helpers to test harness --------- Co-authored-by: Andrew Gillis <11790789+gammazero@users.noreply.github.com>	2026-01-16 00:27:09 +01:00
Marcin Rataj	edb7056747	feat(config): add Gateway.MaxRequestDuration option (#11138) ... * feat(config): add Gateway.MaxRequestDuration option exposes the previously hardcoded 1 hour gateway request deadline as a configurable option, allowing operators to adjust it to fit deployment needs. protects gateway from edge cases and slow client attacks. boxo: https://github.com/ipfs/boxo/pull/1079 * test(gateway): add MaxRequestDuration integration test verifies config is wired correctly and 504 is returned when exceeded * docs: add MaxRequestDuration to gateway production guide --------- Co-authored-by: Andrew Gillis <11790789+gammazero@users.noreply.github.com>	2026-01-15 19:31:16 +01:00
Guillaume Michel	698354342e	feat(provider): info log AcceleratedDHTClient crawl (#11143) ... * feat(provider): log fullrt crawl * fix(provider): improve AcceleratedDHTClient log messages use config option name instead of internal "fullrt" jargon, align terminology with daemon startup message, add time estimate --------- Co-authored-by: Marcin Rataj <lidel@lidel.org>	2026-01-15 17:11:35 +01:00
Andrew Gillis	4bdc9ad220	ipfswatch: fix panic on broken link (#11145) ... Fix panic when broken link is created in watch directory. Closes #10017	2026-01-13 20:44:28 -08:00
Andrew Gillis	6a595c27a9	upgrade go-libp2p-pubsub to v0.15.0 (#11144)	2026-01-13 20:27:58 -08:00
Lucas Molas	39c609b3db	feat(mfs): chroot command to change the root (#8648) ... Co-authored-by: Marcin Rataj <lidel@lidel.org>	2026-01-13 21:47:36 +01:00
Andrew Gillis	ec973aeb38	feat: improved go-ds-flatfs (#11092) ... * datastore: upgrade go-ds-flatfs to v0.6.0 See: https://github.com/ipfs/go-ds-flatfs/pull/142 * docs(changelog): add go-ds-flatfs atomic batch writes *documents the new flatfs batch implementation that uses atomic operations via temp directory, preventing orphan blocks on interrupted imports and reducing memory usage. * includes improved tests, batch cleanup fixes, and docs * docs(changelog): reframe go-ds-flatfs entry for users focus on user benefits instead of implementation details	2026-01-13 02:42:23 -08:00
Andrew Gillis	56bf782cc6	test: fix flaky ipfswatch test (#11142) ... Gracefully shutdown the ipfs node to ensure data is saved. Forceful shutdown is done if there is an error sending the interrupt signal to the process, such as on Windows.	2026-01-13 02:38:44 -08:00
Marcin Rataj	447109df64	docs: clarify Routing.Type=custom as experimental (#11111) ... * docs: mark custom routing as experimental reorganize Routing.Type section for clarity, group production and experimental options, consolidate DHT explanation, add limitations section to delegated-routing.md documenting that HTTP-only routing cannot provide content reliably * chore(config): reorder Routing sections and improve callout formatting move DelegatedRouters after Type, add config option names to CAUTION headers * docs: address reviewer feedback on config.md - clarify that `auto` can be combined with custom URLs in `Routing.DelegatedRouters` - rename headers for consistency: `Routing.Routers.[name].Type`, `Routing.Routers.[name].Parameters`, `Routing.Methods` - replace deprecated Strategic Providing reference with `Provide.*` config - remove outdated caveat about 0.39 sweep limitation - wording: "likely suffer" → "will be most affected" * docs: remove redundant Summary section from delegated-routing.md the IMPORTANT callout and Motivation section already cover what users need to know. historical version info was noise for researchers trying to configure custom routing. addresses reviewer feedback from #11111. --------- Co-authored-by: Daniel Norman <2color@users.noreply.github.com> Co-authored-by: Andrew Gillis <11790789+gammazero@users.noreply.github.com>	2026-01-11 00:39:30 +01:00
Marcin Rataj	de20a78a1f	fix(routing): defensive clone of AddrInfo from provider channel (#11120) ... Belt-and-suspenders defense against data races where routing subsystem (DHT or delegated routing) may reuse backing array. Clones AddrInfo before publishing to QueryEvent to ensure isolated copy. Closes https://github.com/ipfs/kubo/issues/11116	2026-01-09 21:40:38 +01:00
Guillaume Michel	07ea37e99f	fix(provider): wait for fullrt crawl completion before providing (#11137) ... Co-authored-by: Marcin Rataj <lidel@lidel.org>	2026-01-09 21:17:37 +01:00
Andrew Gillis	ac9ae9bf5d	fix(provide): do not output keystore error on shutdown (#11130) ... * Do not output keystore error on shutdown. Closes #11127 * fix: add debug log for keystore sync interrupted by shutdown log at DEBUG level when keystore sync is interrupted during shutdown, preserving error details for debugging while keeping normal output clean --------- Co-authored-by: Marcin Rataj <lidel@lidel.org>	2026-01-09 20:45:46 +01:00
Marcin Rataj	25ebab9dae	feat(p2p): add --foreground flag to listen and forward commands (#11099) ... * feat(p2p): add --foreground flag to listen and forward commands adds `-f/--foreground` option that keeps the command running until interrupted (SIGTERM/Ctrl+C) or closed via `ipfs p2p close`. the listener/forwarder is automatically removed when the command exits. useful for systemd services and scripts that need cleanup on exit. * docs: add p2p-tunnels.md with systemd examples - add dedicated docs/p2p-tunnels.md covering: - why p2p tunnels (NAT traversal, no public IP needed) - quick start with netcat - background and foreground modes - systemd integration with path-based activation - security considerations and troubleshooting - document Experimental.Libp2pStreamMounting in docs/config.md - simplify docs/experimental-features.md, link to new doc - add "Learn more" links to ipfs p2p listen/forward --help - update changelog entry with doc link - add cross-reference in misc/README.md * chore: reference kubo#5460 for p2p config Ref. https://github.com/ipfs/kubo/issues/5460 * fix(daemon): write api/gateway files only after HTTP server is ready fixes race condition where $IPFS_PATH/api and $IPFS_PATH/gateway files were written before the HTTP servers were ready to accept connections. this caused issues for tools like systemd path units that immediately try to connect when these files appear. changes: - add corehttp.ServeWithReady() that signals when server is ready - wait for ready signal before writing address files - use sync.WaitGroup.Go() (Go 1.25) for cleaner goroutine management - add TestAddressFileReady to verify both api and gateway files * fix(daemon): buffer errc channel and wait for all listeners - buffer error channel with len(listeners) to prevent deadlock when multiple servers write errors simultaneously - wait for ALL listeners to be ready before writing api/gateway file, not just the first one Feedback-from: https://github.com/ipfs/kubo/pull/11099#pullrequestreview-3593885839 * docs(changelog): improve p2p tunnel section clarity reframe to lead with user benefit and add example output * docs(p2p): remove obsolete race condition caveat the "First launch fails but restarts work" troubleshooting section described a race where the api file was written before the daemon was ready. this was fixed in 80b703a which ensures api/gateway files are only written after HTTP servers are ready to accept connections. --------- Co-authored-by: Andrew Gillis <11790789+gammazero@users.noreply.github.com>	2026-01-09 19:22:43 +01:00
Marcin Rataj	5288946fd1	feat(cli): improve ipfs dag stat output UX (#11097) ... - add TTY auto-detection for progress display (matching `dag export`) - use single-line progress with carriage return instead of flooding - show human-readable sizes alongside raw bytes in summary - update --progress flag to be auto-detected by default progress format: `Fetched/Processed N blocks, M bytes (X MB)` summary format: `Total Size: 99 (99 B)`	2026-01-09 18:41:10 +01:00
Marcin Rataj	3e85793b58	docs: add production deployment guidance for gateway (#11117) ... - add "Running in Production" section to gateway.md - link to specs.ipfs.tech instead of github - update "Protocol Labs" to "IPFS Foundation" - add reverse proxy requirement to config.md PublicGateways section - reference reverse proxy, timeout, rate limiting, and CDN caveats	2026-01-09 18:26:32 +01:00
Marcin Rataj	23ba660ef0	fix(routing): use LegacyProvider for HTTP-only custom routing (#11112) ... * fix(routing): use LegacyProvider for HTTP-only custom routing when `Routing.Type=custom` with only HTTP routers and no DHT, fall back to LegacyProvider instead of SweepingProvider. SweepingProvider requires a DHT client which is unavailable in HTTP-only configurations, causing it to return NoopProvider and breaking provider record announcements to HTTP routers. fixes #11089 * test(routing): verify provide stat works with HTTP-only routing * docs(config): clarify SweepEnabled fallback for HTTP-only routing --------- Co-authored-by: Andrew Gillis <11790789+gammazero@users.noreply.github.com>	2026-01-09 18:23:31 +01:00
Andrew Gillis	aa3c88dcdd	shutdown daemon after test (#11135)	2026-01-07 20:51:19 -08:00
Marcin Rataj	1301710a91	fix(ci): parallelize gotest, cleanup output, flakiness (#11113) ... * ci: parallelize gotest by separating test/cli into own job split the Go Test workflow into two parallel jobs: - `unit-tests`: runs unit tests (excluding test/cli) - `cli-tests`: runs test/cli end-to-end tests test/cli takes ~3 minutes (~50% of total gotest time), so running it in parallel should reduce wall-clock CI time by ~1.5-2.5 minutes. both jobs produce JUnit XML and HTML reports for consistent debugging. * ci(gotest): reduce noise on test timeout panics add GOTRACEBACK=single to show only one goroutine stack instead of all when a test timeout panic occurs. this makes CI output much cleaner when tests hang. * fix(ci): prevent stderr from corrupting test JSON output - remove 2>&1 which mixed "go: downloading" stderr messages into JSON - add JSON validation before parsing - print failed test names for easier debugging * ci(gotest): use gotestsum for human-readable test output - replace per-package coverage loop with single gotestsum invocation - both unit-tests and cli-tests now show human-readable output - simplified coverage collection (single -coverprofile, no gocovmerge) - clarified step names to indicate they run tests * ci: fix codecov uploads by adding token - add CODECOV_TOKEN to gotest.yml and sharness.yml - update codecov-action to v5.5.2 - add fail_ci_if_error: false for robustness codecov stopped receiving coverage data ~1 year ago when they started requiring tokens for public repos * refactor(make): add test_unit and test_cli targets - add `make test_unit` for unit tests with coverage (used by CI) - add `make test_cli` for CLI integration tests (used by CI) - only disable colors when CI env var is set (local dev gets colors) - remove legacy targets: test_go_test, test_go_short, test_go_race, test_go_expensive - update gotest.yml to use make targets instead of inline commands - add test artifacts to .gitignore * fix(ci): move client/rpc tests to cli-tests job client/rpc tests use test/cli/harness which requires the ipfs binary. Move them from test_unit to test_cli where the binary is built. also: - update gotestsum to v1.13.0 - simplify workflow step names * fix(ci): use build tags when listing test packages go list needs build tags to properly exclude packages like fuse/mfs when running with TEST_FUSE=0 (nofuse tag). * fix(ci): move test/integration to cli-tests job test/integration tests need the ipfs binary, move them from test_unit to test_cli. * fix(test): fix flaky kubo-as-a-library and GetClosestPeers tests kubo-as-a-library: use `Bootstrap()` instead of raw `Swarm().Connect()` to fix race condition between swarm connection and bitswap peer discovery. `Bootstrap()` properly integrates peers into the routing system, ensuring bitswap learns about connected peers synchronously. GetClosestPeers: simplify retry logic using `EventuallyWithT` with 10-minute timeout. tests all 4 routing types (`auto`, `autoclient`, `dht`, `dhtclient`) against real bootstrap peers with patient polling. * fix(example): use bidirectional Swarm().Connect() for reliable bitswap - connect nodes bidirectionally (A→B and B→A) to simulate mutual peering - mutual peering protects connection from resource manager culling - use port 0 for random available ports (avoids CI conflicts) - enable LoopbackAddressesOnLanDHT for local testing - move retry logic to test file using require.Eventually * fix(ci): add test_examples target and parallel example-tests job - add `make test_examples` target to mk/golang.mk for consistency with test_unit/test_cli - move example tests to separate parallel CI job (example-tests) - example: use Bootstrap() with autoconf.FallbackBootstrapPeers for reliable bitswap - example: increase context timeout to 10 minutes - test: add 60s per-request timeout to GetClosestPeers (server has 30s routing timeout) - test: reduce EventuallyWithT to 3 minutes (locally passes in under 1 minute) * fix(ci): improve test targets, exclusion patterns, and artifact naming - define COVERPKG_EXCLUDE and UNIT_EXCLUDE as documented variables - use grep -vE with single regex instead of multiple grep -v calls - add mkdir -p before rm to ensure directories exist - add DEPS_GO dependency to test_cli target - make CLI test timeout configurable via TEST_CLI_TIMEOUT (default 10m) - fix test_examples cleanup on failure using subshell - reduce GetClosestPeers test wait time from 3m to 2m - rename artifacts to match job names: unit-tests-{junit,html}, cli-tests-{junit,html} - update cli-tests upload-artifact from v5 to v6 * fix(ci): fix unit test exclusion and speed up example test - fix UNIT_EXCLUDE regex to match client/rpc at end of path - remove public bootstrap peers from example (only connect to nodeA) - example test now runs in ~3s instead of timing out * fix(test): fix flaky TestAddMultipleGCLive race condition added time.Sleep after spawning GC goroutines to ensure they reach GCLock() before the test proceeds. without this, the adder's maybePauseForGC() might check GCRequested() before GC has even requested the lock, causing the lock to not be released and GC to block indefinitely. this matches the existing pattern in TestAddGCLive which already had this sleep. also replaced context.Background() with t.Context() in both TestAddMultipleGCLive and TestAddGCLive for proper test lifecycle management. * fix(example): use test harness settings for reliable CI the kubo-as-a-library example was flaky on CI. applied test-harness-like settings that match what transports_test.go uses: - TCP-only on 127.0.0.1 with random port (no QUIC/UDP) - explicitly disable non-TCP transports (QUIC, Relay, WebTransport, etc) - use NilRouterOption (no routing) since we connect peers directly - bitswap works with directly connected peers without DHT lookups - 2-minute context timeout - streaming output in test for debugging	2026-01-08 05:07:08 +01:00
Andrew Gillis	55b94751cc	test: replace go-clock with testing/synctest (#11131) ... Use testing/synctest instead of go-clock for artificial time control.	2026-01-07 13:30:15 -08:00
Marcin Rataj	584025bb69	docs: improve README for first-time users (#11133) ... * docs: improve README for first-time users - add Quick Taste section with real CIDv1 example near top - rewrite "What is Kubo?" with technical concepts (CIDs, DAGs, UnixFS, Bitswap) - reorder features to follow user journey (CLI before advanced HTTP features) - streamline install section with links to docs.ipfs.tech - organize package managers in tables with Repology version badges - add supply chain security warning for third-party packages - surface important docs (metrics, debug guide, customizing) - update maintainer info with Shipyard branding Closes #11125 Closes #7298 Closes #5471 Closes #5087 * docs(readme): add changelogs link and fix docs directory URL * docs(readme): add mDNS/DHT links and clarify build steps - link LAN discovery to mDNS spec and WAN to Amino DHT glossary - show both make build and make install with output paths	2026-01-07 20:33:16 +01:00
dependabot[bot]	663a4a78fa	chore(deps): bump actions/upload-artifact from 5 to 6 (#11109) ... Bumps [actions/upload-artifact](https://github.com/actions/upload-artifact) from 5 to 6. - [Release notes](https://github.com/actions/upload-artifact/releases) - [Commits](https://github.com/actions/upload-artifact/compare/v5...v6) --- updated-dependencies: - dependency-name: actions/upload-artifact dependency-version: '6' dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>	2026-01-07 11:25:01 -08:00
dependabot[bot]	9b99dc6f45	chore(deps): bump codecov/codecov-action from 5.5.1 to 5.5.2 (#11106) ... Bumps [codecov/codecov-action](https://github.com/codecov/codecov-action) from 5.5.1 to 5.5.2. - [Release notes](https://github.com/codecov/codecov-action/releases) - [Changelog](https://github.com/codecov/codecov-action/blob/main/CHANGELOG.md) - [Commits](5a1091511a...671740ac38) --- updated-dependencies: - dependency-name: codecov/codecov-action dependency-version: 5.5.2 dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>	2026-01-07 11:24:08 -08:00
Andrew Gillis	566f8ba63f	keys: skip bad keys when listing (#11115) ... Change the `ipfs key list` behavior to log an error and continue listing keys when a key cannot be read from the keystore or decoded. Closes: #11102	2026-01-07 09:56:11 -08:00
Marcin Rataj	f4d6253ae2	docs: add developer guide for local development workflow (#11128) ... addresses https://discuss.ipfs.tech/t/19933 - add docs/developer-guide.md with prerequisites, build, test, and troubleshooting - link from README.md, docs/README.md, and CONTRIBUTING.md - document test suite differences (unit vs e2e, test/cli vs test/sharness) - include tips for running specific tests during development	2026-01-07 17:09:13 +01:00
dependabot[bot]	3d0e7c8465	chore(deps): bump actions/download-artifact from 6 to 7 (#11107) ... Bumps [actions/download-artifact](https://github.com/actions/download-artifact) from 6 to 7. - [Release notes](https://github.com/actions/download-artifact/releases) - [Commits](https://github.com/actions/download-artifact/compare/v6...v7) --- updated-dependencies: - dependency-name: actions/download-artifact dependency-version: '7' dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>	2026-01-06 22:50:47 -08:00