add "Understanding the Metrics" section explaining three types:
- per-worker rates (multiply by active workers for total throughput)
- per-region averages (do NOT multiply by worker count)
- system totals (cumulative across all workers)
enhance metric descriptions with:
- explicit calculation examples showing which worker counts to use
- warnings about when NOT to multiply by worker count
- cross-references to relevant sections
add "Capacity Planning" section with:
- step-by-step throughput capacity calculations
- diagnostic guidance for common scenarios
- worked examples for estimating required vs actual capacity
addresses confusion from PR #11034 comments about when to multiply
metrics by worker count and how to interpret per-worker rates
addresses stream frame memory pooling issue where StreamFrame objects
weren't properly returned to sync.Pool during stream cancellation
see quic-go/quic-go#5327
* chore(deps): update go-libp2p to v0.44.0
- includes self-healing UPnP port mappings after router restarts
- update go-netroute to v0.3.0
- update quic-go to v0.55.0
- add changelog entry for UPnP fix
* docs: improve provide and UPnP clarity in changelog and docs
- add alert polling rationale to changelog
- add UPnP config note with default clarification
- clarify sweep timing and prefix length explanations
- add concrete examples for time offset and record holders
- improve workers stats formatting
- add See Also section to provide-stats.md
* docs: add RISC-V prebuilt binaries to changelog and README
- highlight linux-riscv64 availability with open hardware context
- update README with arm64 builds, remove 32-bit examples
* feat: provide stats
* added N/A
* format
* workers stats alignment
* ipfs provide stat --all --compact
* consolidating compact stat
* update column alignment
* flags combinations errors
* command description
* change schedule AvgPrefixLen to float
* changelog
* alignments
* provide stat description draft
* rephrased provide-stats.md
* linking provide-stats.md from command description
* documentation test
* fix: refactor provide stat command type handling
- add extractSweepingProvider() helper to reduce nested type switching
- extract lowWorkerThreshold constant for worker availability check
- fix --lan error handling to work with buffered providers
* docs: add clarifying comments
* fix(commands): improve provide stat compact mode
- prevent panic when both columns are empty
- fix column alignment with UTF-8 characters
- only track col0MaxWidth for first column (as intended)
* test: add tests for ipfs provide stat command
- test basic functionality, flags, JSON output
- test legacy provider behavior
- test integration with content scheduling
- test disabled provider configurations
- add parseSweepStats helper with t.Helper()
* docs: improve provide command help text
- update tagline to "Control and monitor content providing"
- simplify help descriptions
- make error messages more consistent
- update tests to match new error messages
* metrics rename
```
Next reprovide at:
Next prefix:
```
updated to:
```
Next region prefix:
Next region reprovide:
```
* docs: improve Provide system documentation clarity
Enhance documentation for the Provide system to better explain how provider
records work and the differences between sweep and legacy modes.
Changes to docs/config.md:
- Provide section: add clear explanation of provider records and their role
- Provide.DHT: add provider record lifecycle and two provider systems overview
- Provide.DHT.Interval: explain relationship to expiration, contrast sweep vs legacy behavior
- Provide.DHT.SweepEnabled: rewrite to explain legacy problem, sweep solution, and efficiency gains
- Monitoring section: prioritize command-line tools (ipfs provide stat) before Prometheus
Changes to core/commands/provide.go:
- ipfs provide stat help: add explanation of provider records, TTL expiration, and how sweep batching works
Changes to docs/changelogs/v0.39.md:
- Add context about why stats matter for monitoring provider health
- Emphasize real-time monitoring workflow with watch command
- Explain what users can observe (rates, queues, worker availability)
* depend on latest kad-dht master
* docs: nits
---------
Co-authored-by: Marcin Rataj <lidel@lidel.org>
Increase default Provide.DHT.MaxProvideConnsPerWorker value to match the
DHT replication factor (16 -> 20). A similar value is used in legacy
systems (with and without accelerated DHT client).
- clarify staging environment step for FINAL releases
- mark infrastructure updates (collab cluster, bootstrappers) as FINAL only
- improve ipfs-desktop release step wording
- update discourse topic examples to v0.38.0
- reference v0.38.0 release issue in metadata comment
* test: add migration tests for Windows and macOS
- add dedicated CI workflow for migration tests on Windows/macOS
- workflow triggers on migration-related file changes only
* build: remove redundant go version checks
- remove GO_MIN_VERSION and check_go_version scripts
- go.mod already enforces minimum version (go 1.25)
- fixes make build on Windows
* fix: windows migration panic by reading config into memory
fixes migration panic on Windows when upgrading from v0.37 to v0.38
by reading the entire config file into memory before performing atomic
operations. this avoids file locking issues on Windows where open files
cannot be renamed.
also fixes:
- TestRepoDir to set USERPROFILE on Windows (not just HOME)
- CLI migration tests to sanitize directory names (remove colons)
minimal fix that solves the "panic: error can't be dealt with
transactionally: Access is denied" error without adding unnecessary
platform-specific complexity.
* fix: set PATH for CLI migration tests in CI
the CLI tests need the built ipfs binary to be in PATH
* fix: use ipfs shutdown for graceful daemon termination in tests
replaces platform-specific signal handling with ipfs shutdown command
which works consistently across all platforms including Windows
* fix: isolate PATH modifications in parallel migration tests
tests running in parallel with t.Parallel() were interfering with each
other through global PATH modifications via os.Setenv(). this caused
tests to download real migration binaries instead of using mocks,
leading to Windows failures due to path separator issues in external tools.
now each test builds its own custom PATH and passes it explicitly to
commands, preventing interference between parallel tests.
* chore: improve error messages in WithBackup
* fix: Windows CI migration test failures
- add .exe extension to mock migration binaries on Windows
- handle repo lock file properly in mock migration binary
- ensure lock is created and removed to prevent conflicts
* refactor: align atomicfile error handling with fs-repo-migrations
- check close error in Abort() before attempting removal
- leave temp file on rename failure for debugging (like fs-repo-15-to-16)
- improves consistency with external migration implementations
* fix: use req.Context in repo migrate to avoid double-lock
The repo migrate command was calling cctx.Context() which has a hidden
side effect: it lazily constructs the IPFS node by calling GetNode(),
which opens the repository and acquires repo.lock. When migrations then
tried to acquire the same lock, it failed with "lock is already held by us"
because go4.org/lock tracks locks per-process in a global map.
The fix uses req.Context instead, which is a plain context.Context with
no side effects. This provides what migrations need (cancellation handling)
without triggering node construction or repo opening.
Context types explained:
- req.Context: Standard Go context for request lifetime, cancellation,
and timeouts. No side effects.
- cctx.Context(): Kubo-specific method that lazily constructs the full
IPFS node (opens repo, acquires lock, initializes subsystems). Returns
the node's internal context.
Why req.Context is correct here:
- Migrations work on raw filesystem (only need ConfigRoot path)
- Command has SetDoesNotUseRepo(true) - doesn't need running node
- Migrations handle their own locking via lockfile.Lock()
- Need cancellation support but not node lifecycle
The bug only appeared with embedded migrations (v16+) because they run
in-process. External migrations (pre-v16) were separate processes, so
each had isolated state. Sequential migrations (forward then backward)
in the same process exposed this latent double-lock issue.
Also adds repo.lock acquisition to RunEmbeddedMigrations to prevent
concurrent migration access, and removes the now-unnecessary daemon
lock check from the migrate command handler.
* fix: use req.Context for migrations and autoconf in daemon startup
daemon.go was incorrectly using cctx.Context() in two critical places:
1. Line 337: migrations call - cctx.Context() triggers GetNode() which
opens the repo and acquires repo.lock BEFORE migrations run, causing
"lock is already held by us" errors when migrations try to lock
2. Line 390: autoconf client.Start() - uses context for HTTP timeouts
and background updater lifecycle, doesn't need node construction
Both now use req.Context (plain Go context) which provides:
- request lifetime and cancellation
- no side effects (doesn't construct node or open repo)
- correct lifecycle for HTTP requests and background goroutines
(cherry picked from commit f4834e797d)
* test: add migration tests for Windows and macOS
- add dedicated CI workflow for migration tests on Windows/macOS
- workflow triggers on migration-related file changes only
* build: remove redundant go version checks
- remove GO_MIN_VERSION and check_go_version scripts
- go.mod already enforces minimum version (go 1.25)
- fixes make build on Windows
* fix: windows migration panic by reading config into memory
fixes migration panic on Windows when upgrading from v0.37 to v0.38
by reading the entire config file into memory before performing atomic
operations. this avoids file locking issues on Windows where open files
cannot be renamed.
also fixes:
- TestRepoDir to set USERPROFILE on Windows (not just HOME)
- CLI migration tests to sanitize directory names (remove colons)
minimal fix that solves the "panic: error can't be dealt with
transactionally: Access is denied" error without adding unnecessary
platform-specific complexity.
* fix: set PATH for CLI migration tests in CI
the CLI tests need the built ipfs binary to be in PATH
* fix: use ipfs shutdown for graceful daemon termination in tests
replaces platform-specific signal handling with ipfs shutdown command
which works consistently across all platforms including Windows
* fix: isolate PATH modifications in parallel migration tests
tests running in parallel with t.Parallel() were interfering with each
other through global PATH modifications via os.Setenv(). this caused
tests to download real migration binaries instead of using mocks,
leading to Windows failures due to path separator issues in external tools.
now each test builds its own custom PATH and passes it explicitly to
commands, preventing interference between parallel tests.
* chore: improve error messages in WithBackup
* fix: Windows CI migration test failures
- add .exe extension to mock migration binaries on Windows
- handle repo lock file properly in mock migration binary
- ensure lock is created and removed to prevent conflicts
* refactor: align atomicfile error handling with fs-repo-migrations
- check close error in Abort() before attempting removal
- leave temp file on rename failure for debugging (like fs-repo-15-to-16)
- improves consistency with external migration implementations
* fix: use req.Context in repo migrate to avoid double-lock
The repo migrate command was calling cctx.Context() which has a hidden
side effect: it lazily constructs the IPFS node by calling GetNode(),
which opens the repository and acquires repo.lock. When migrations then
tried to acquire the same lock, it failed with "lock is already held by us"
because go4.org/lock tracks locks per-process in a global map.
The fix uses req.Context instead, which is a plain context.Context with
no side effects. This provides what migrations need (cancellation handling)
without triggering node construction or repo opening.
Context types explained:
- req.Context: Standard Go context for request lifetime, cancellation,
and timeouts. No side effects.
- cctx.Context(): Kubo-specific method that lazily constructs the full
IPFS node (opens repo, acquires lock, initializes subsystems). Returns
the node's internal context.
Why req.Context is correct here:
- Migrations work on raw filesystem (only need ConfigRoot path)
- Command has SetDoesNotUseRepo(true) - doesn't need running node
- Migrations handle their own locking via lockfile.Lock()
- Need cancellation support but not node lifecycle
The bug only appeared with embedded migrations (v16+) because they run
in-process. External migrations (pre-v16) were separate processes, so
each had isolated state. Sequential migrations (forward then backward)
in the same process exposed this latent double-lock issue.
Also adds repo.lock acquisition to RunEmbeddedMigrations to prevent
concurrent migration access, and removes the now-unnecessary daemon
lock check from the migrate command handler.
* fix: use req.Context for migrations and autoconf in daemon startup
daemon.go was incorrectly using cctx.Context() in two critical places:
1. Line 337: migrations call - cctx.Context() triggers GetNode() which
opens the repo and acquires repo.lock BEFORE migrations run, causing
"lock is already held by us" errors when migrations try to lock
2. Line 390: autoconf client.Start() - uses context for HTTP timeouts
and background updater lifecycle, doesn't need node construction
Both now use req.Context (plain Go context) which provides:
- request lifetime and cancellation
- no side effects (doesn't construct node or open repo)
- correct lifecycle for HTTP requests and background goroutines
* feat: add docker stub for deprecated ipfs/go-ipfs name
implements docker part of #10941 by creating a stub image that redirects
users from ipfs/go-ipfs to ipfs/kubo
changes:
- add stub dockerfile and script in .github/legacy/
- modify docker-image.yml to push stub to ipfs/go-ipfs with same tags as ipfs/kubo
- remove ipfs/go-ipfs from get-docker-tags.sh to prevent docker-hub job from pushing to legacy name
- stub displays clear deprecation message directing users to ipfs/kubo:release
* docs: add v0.39 changelog highlight for go-ipfs deprecation
* fix: add MFS operation limit for --flush=false
adds a global counter that tracks consecutive MFS operations performed
with --flush=false and fails with clear error after limit is reached.
this prevents unbounded memory growth while avoiding the data corruption
risks of auto-flushing.
- adds Internal.MFSNoFlushLimit config
- operations fail with actionable error at limit
- counter resets on successful flush or any --flush=true operation
- operations with --flush=true reset and don't count
this commit removes automatic flush from https://github.com/ipfs/kubo/pull/10971
and instead errors to encourage users of --flush=false to develop a habit
of calling 'ipfs files flush' periodically.
boxo will no longer auto-flush (https://github.com/ipfs/boxo/pull/1041) to
avoid corruption issues, and kubo applies the limit to 'ipfs files' commands
instead.
closes#10842
* test: add tests for MFSNoFlushLimit
tests verify the new Internal.MFSNoFlushLimit config option:
- default limit of 256 operations
- custom limit configuration
- counter reset on flush=true
- counter reset on explicit flush command
- limit=0 disables the feature
- multiple MFS command types count towards limit
* docs: explain why MFS operations fail instead of auto-flushing
addresses feedback from https://github.com/ipfs/kubo/pull/10985#pullrequestreview-3256250970
- clarify that automatic flushing at limit was considered but rejected
- explain the data corruption risks of auto-flushing
- guide users who want auto-flush to use --flush=true (default)
- document benefits of explicit failure for batch operations
(cherry picked from commit a688b7eeac)
* Filestore: provide Filestore nodes
When strategy is set to "all" (the blockstore does all the providing when a
block is written), no providing was happening to Filestore blocks that were
not written to the underlying blockstore (so, the DAG leaves, as they live in
the filesystem directly). This fixes that.
* docs: clarify filestore and urlstore fix in changelog
both filestore (local file references) and urlstore (HTTP/HTTPS URL
references) blocks are now properly provided shortly after initial add
(cherry picked from commit f63887ae96)
* fix: add MFS operation limit for --flush=false
adds a global counter that tracks consecutive MFS operations performed
with --flush=false and fails with clear error after limit is reached.
this prevents unbounded memory growth while avoiding the data corruption
risks of auto-flushing.
- adds Internal.MFSNoFlushLimit config
- operations fail with actionable error at limit
- counter resets on successful flush or any --flush=true operation
- operations with --flush=true reset and don't count
this commit removes automatic flush from https://github.com/ipfs/kubo/pull/10971
and instead errors to encourage users of --flush=false to develop a habit
of calling 'ipfs files flush' periodically.
boxo will no longer auto-flush (https://github.com/ipfs/boxo/pull/1041) to
avoid corruption issues, and kubo applies the limit to 'ipfs files' commands
instead.
closes#10842
* test: add tests for MFSNoFlushLimit
tests verify the new Internal.MFSNoFlushLimit config option:
- default limit of 256 operations
- custom limit configuration
- counter reset on flush=true
- counter reset on explicit flush command
- limit=0 disables the feature
- multiple MFS command types count towards limit
* docs: explain why MFS operations fail instead of auto-flushing
addresses feedback from https://github.com/ipfs/kubo/pull/10985#pullrequestreview-3256250970
- clarify that automatic flushing at limit was considered but rejected
- explain the data corruption risks of auto-flushing
- guide users who want auto-flush to use --flush=true (default)
- document benefits of explicit failure for batch operations
* Filestore: provide Filestore nodes
When strategy is set to "all" (the blockstore does all the providing when a
block is written), no providing was happening to Filestore blocks that were
not written to the underlying blockstore (so, the DAG leaves, as they live in
the filesystem directly). This fixes that.
* docs: clarify filestore and urlstore fix in changelog
both filestore (local file references) and urlstore (HTTP/HTTPS URL
references) blocks are now properly provided shortly after initial add
* docs: improve slow reprovide warning messages
simplify warning text and provide actionable solutions in order of preference
* feat(config): add validation for Provide.DHT settings
- validate interval doesn't exceed DHT record validity (48h)
- validate worker counts and other parameters are within valid ranges
- improve slow reprovide warning messages to reference config parameter
- add tests for all validation cases
* docs: add reprovide cycle visualization
shows traffic patterns of legacy vs sweep vs accelerated DHT
