Commit Graph

15698 Commits

Author SHA1 Message Date
Guillaume Michel
9bb2eae5e8
Merge branch 'master' into reprovide-sweep-enabled 2025-11-01 12:39:53 +01:00
Guillaume Michel
ae86672964
fix(provider/stats): number format (#11045)
Some checks failed
CodeQL / codeql (push) Has been cancelled
Docker Check / lint (push) Has been cancelled
Docker Check / build (push) Has been cancelled
Gateway Conformance / gateway-conformance (push) Has been cancelled
Gateway Conformance / gateway-conformance-libp2p-experiment (push) Has been cancelled
Go Build / go-build (push) Has been cancelled
Go Check / go-check (push) Has been cancelled
Go Lint / go-lint (push) Has been cancelled
Go Test / go-test (push) Has been cancelled
Interop / interop-prep (push) Has been cancelled
Sharness / sharness-test (push) Has been cancelled
Spell Check / spellcheck (push) Has been cancelled
Interop / helia-interop (push) Has been cancelled
Interop / ipfs-webui (push) Has been cancelled
* fix: reprovide alert bug

* number formatting

* show full number for peer count
2025-11-01 08:20:41 +01:00
guillaumemichel
9a1f83b153
bump kad-dht to test last (unmerged) PRs 2025-10-31 23:42:45 +01:00
Guillaume Michel
a29a66fc8a
Merge branch 'master' into reprovide-sweep-enabled 2025-10-31 13:59:32 +01:00
Guillaume Michel
e05357ed19
provider: protect libp2p connections (#11028)
Some checks are pending
CodeQL / codeql (push) Waiting to run
Docker Check / lint (push) Waiting to run
Docker Check / build (push) Waiting to run
Gateway Conformance / gateway-conformance (push) Waiting to run
Gateway Conformance / gateway-conformance-libp2p-experiment (push) Waiting to run
Go Build / go-build (push) Waiting to run
Go Check / go-check (push) Waiting to run
Go Lint / go-lint (push) Waiting to run
Go Test / go-test (push) Waiting to run
Interop / interop-prep (push) Waiting to run
Interop / helia-interop (push) Blocked by required conditions
Interop / ipfs-webui (push) Blocked by required conditions
Sharness / sharness-test (push) Waiting to run
Spell Check / spellcheck (push) Waiting to run
* provider: protect libp2p connections

Use latest kad-dht version, introducing connection protection and
retention of addresses in peerstore during provide operations.

* depend on kad-dht master
2025-10-31 13:59:21 +01:00
guillaumemichel
1ed9b69181
fix: reprovide alert nil check 2025-10-31 12:18:33 +01:00
guillaumemichel
50873297ef
depend on latest (unmerged!) kad-dht changes
Depends on current state of unmerged kad-dht PRs combined:
* https://github.com/libp2p/go-libp2p-kad-dht/pull/1181
* https://github.com/libp2p/go-libp2p-kad-dht/pull/1187
* https://github.com/libp2p/go-libp2p-kad-dht/pull/1188
* https://github.com/libp2p/go-libp2p-kad-dht/pull/1189

Combined at
https://github.com/libp2p/go-libp2p-kad-dht/tree/provider/recent-changes-tmp
(22a7685)
2025-10-30 23:33:18 +01:00
Guillaume Michel
4d2e9c7e5c
Merge branch 'master' into reprovide-sweep-enabled 2025-10-30 23:28:50 +01:00
guillaumemichel
1bf85300d4
address review 2025-10-30 09:44:46 +01:00
Marcin Rataj
cf4b1f41db
Merge pull request #11044 from ipfs/merge-release-v0.38.2
Some checks failed
CodeQL / codeql (push) Has been cancelled
Docker Check / lint (push) Has been cancelled
Docker Check / build (push) Has been cancelled
Gateway Conformance / gateway-conformance (push) Has been cancelled
Gateway Conformance / gateway-conformance-libp2p-experiment (push) Has been cancelled
Go Build / go-build (push) Has been cancelled
Go Check / go-check (push) Has been cancelled
Go Lint / go-lint (push) Has been cancelled
Go Test / go-test (push) Has been cancelled
Interop / interop-prep (push) Has been cancelled
Sharness / sharness-test (push) Has been cancelled
Spell Check / spellcheck (push) Has been cancelled
Interop / helia-interop (push) Has been cancelled
Interop / ipfs-webui (push) Has been cancelled
Merge release v0.38.2
2025-10-30 04:27:31 +01:00
Marcin Rataj
044db8253b merge: master into release-v0.38.2 2025-10-30 04:25:11 +01:00
Marcin Rataj
9fd105a1ba
Merge pull request #11042 from ipfs/release-v0.38.2
Release v0.38.2
2025-10-30 02:14:57 +01:00
Andrew Gillis
a4323abc10
Upgrade to Boxo v0.35.1 (#11043)
Some checks are pending
CodeQL / codeql (push) Waiting to run
Docker Check / lint (push) Waiting to run
Docker Check / build (push) Waiting to run
Gateway Conformance / gateway-conformance (push) Waiting to run
Gateway Conformance / gateway-conformance-libp2p-experiment (push) Waiting to run
Go Build / go-build (push) Waiting to run
Go Check / go-check (push) Waiting to run
Go Lint / go-lint (push) Waiting to run
Go Test / go-test (push) Waiting to run
Interop / interop-prep (push) Waiting to run
Interop / helia-interop (push) Blocked by required conditions
Interop / ipfs-webui (push) Blocked by required conditions
Sharness / sharness-test (push) Waiting to run
Spell Check / spellcheck (push) Waiting to run
* Upgrade to Boxo v0.35.1
* use tagged boxo release
* fix lint error
2025-10-29 17:18:34 -07:00
Marcin Rataj
ccc4ed7fb3
docs: changelog v0.38.2 2025-10-30 00:49:21 +01:00
Marcin Rataj
adf58023d8
chore: boxo and kad-dht updates
https://github.com/ipfs/boxo/releases/tag/v0.35.1
https://github.com/libp2p/go-libp2p-kad-dht/releases/tag/v0.35.1
2025-10-30 00:34:00 +01:00
Marcin Rataj
0060f343a1
fix: update quic-go to v0.55.0
addresses stream frame memory pooling issue where StreamFrame objects
weren't properly returned to sync.Pool during stream cancellation

see quic-go/quic-go#5327
2025-10-29 23:20:59 +01:00
Guillaume Michel
c2bf0f9515
feat(provider): resume cycle (#11031)
Some checks are pending
CodeQL / codeql (push) Waiting to run
Docker Check / lint (push) Waiting to run
Docker Check / build (push) Waiting to run
Gateway Conformance / gateway-conformance (push) Waiting to run
Gateway Conformance / gateway-conformance-libp2p-experiment (push) Waiting to run
Go Build / go-build (push) Waiting to run
Go Check / go-check (push) Waiting to run
Go Lint / go-lint (push) Waiting to run
Go Test / go-test (push) Waiting to run
Interop / interop-prep (push) Waiting to run
Interop / helia-interop (push) Blocked by required conditions
Interop / ipfs-webui (push) Blocked by required conditions
Sharness / sharness-test (push) Waiting to run
Spell Check / spellcheck (push) Waiting to run
* bump kad-dht: resume reprovide cycle

* daemon: --provide-fresh-start flag

* changelog

* docs

* go-fmt

* chore: latest go-libp2p-kad-dht#1170

after conflict resolution, to confirm CI is still green

* kad-dht: depend on latest master

* move daemon flag to Provider.DHT.ResumeEnabled config

* refactor: sweep provider datastore

* bump kad-dht

* bump kad-dht

* bump kad-dht

* make datastore keys constant

* use kad-dht master

* add emoji to changelog entry

* go-fmt

* bump kad-dht

* test(provider): add tests for resume cycle feature

validates Provide.DHT.ResumeEnabled behavior:
- preserves cycle state when enabled (default)
- resets cycle when disabled

tests verify current_time_offset across restarts using JSON output

---------

Co-authored-by: Marcin Rataj <lidel@lidel.org>
2025-10-29 11:07:46 +01:00
Andrew Gillis
2e9c4ec500
chore: upgrade pebble to v2.1.1 (#11040)
Some checks failed
CodeQL / codeql (push) Waiting to run
Docker Check / lint (push) Waiting to run
Docker Check / build (push) Waiting to run
Gateway Conformance / gateway-conformance (push) Waiting to run
Gateway Conformance / gateway-conformance-libp2p-experiment (push) Waiting to run
Go Build / go-build (push) Waiting to run
Go Check / go-check (push) Waiting to run
Go Lint / go-lint (push) Waiting to run
Go Test / go-test (push) Waiting to run
Interop / interop-prep (push) Waiting to run
Interop / helia-interop (push) Blocked by required conditions
Interop / ipfs-webui (push) Blocked by required conditions
Sharness / sharness-test (push) Waiting to run
Spell Check / spellcheck (push) Waiting to run
Migrations / test (macos-latest) (push) Has been cancelled
Migrations / test (ubuntu-latest) (push) Has been cancelled
Migrations / test (windows-latest) (push) Has been cancelled
2025-10-28 23:15:32 +01:00
dependabot[bot]
7d0534b76c
chore(deps): bump actions/setup-node from 5 to 6 (#11030)
Bumps [actions/setup-node](https://github.com/actions/setup-node) from 5 to 6.
- [Release notes](https://github.com/actions/setup-node/releases)
- [Commits](https://github.com/actions/setup-node/compare/v5...v6)

---
updated-dependencies:
- dependency-name: actions/setup-node
  dependency-version: '6'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-10-28 23:05:16 +01:00
dependabot[bot]
513ee699ab
chore(deps): bump actions/download-artifact from 5 to 6 (#11037)
Bumps [actions/download-artifact](https://github.com/actions/download-artifact) from 5 to 6.
- [Release notes](https://github.com/actions/download-artifact/releases)
- [Commits](https://github.com/actions/download-artifact/compare/v5...v6)

---
updated-dependencies:
- dependency-name: actions/download-artifact
  dependency-version: '6'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-10-28 22:55:04 +01:00
dependabot[bot]
fee76b66c9
chore(deps): bump actions/upload-artifact from 4 to 5 (#11038)
Bumps [actions/upload-artifact](https://github.com/actions/upload-artifact) from 4 to 5.
- [Release notes](https://github.com/actions/upload-artifact/releases)
- [Commits](https://github.com/actions/upload-artifact/compare/v4...v5)

---
updated-dependencies:
- dependency-name: actions/upload-artifact
  dependency-version: '5'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-10-28 22:54:46 +01:00
Guillaume Michel
8c5f302d25
fix(cli): provide stat cosmetics (#11034) 2025-10-28 22:38:37 +01:00
Marcin Rataj
ae78c7821c
fix: go-libp2p v0.44 with self-healing UPnP port mappings (#11032)
Some checks failed
CodeQL / codeql (push) Has been cancelled
Docker Check / lint (push) Has been cancelled
Docker Check / build (push) Has been cancelled
Gateway Conformance / gateway-conformance (push) Has been cancelled
Gateway Conformance / gateway-conformance-libp2p-experiment (push) Has been cancelled
Go Build / go-build (push) Has been cancelled
Go Check / go-check (push) Has been cancelled
Go Lint / go-lint (push) Has been cancelled
Go Test / go-test (push) Has been cancelled
Interop / interop-prep (push) Has been cancelled
Sharness / sharness-test (push) Has been cancelled
Spell Check / spellcheck (push) Has been cancelled
Interop / helia-interop (push) Has been cancelled
Interop / ipfs-webui (push) Has been cancelled
* chore(deps): update go-libp2p to v0.44.0

- includes self-healing UPnP port mappings after router restarts
- update go-netroute to v0.3.0
- update quic-go to v0.55.0
- add changelog entry for UPnP fix

* docs: improve provide and UPnP clarity in changelog and docs

- add alert polling rationale to changelog
- add UPnP config note with default clarification
- clarify sweep timing and prefix length explanations
- add concrete examples for time offset and record holders
- improve workers stats formatting
- add See Also section to provide-stats.md

* docs: add RISC-V prebuilt binaries to changelog and README

- highlight linux-riscv64 availability with open hardware context
- update README with arm64 builds, remove 32-bit examples
2025-10-25 01:13:34 +02:00
Guillaume Michel
5d4c40e1f1
feat(provide): slow reprovide alerts when SweepEnabled (#11021)
Co-authored-by: Marcin Rataj <lidel@lidel.org>
2025-10-24 23:46:06 +02:00
Guillaume Michel
e9ed71f8dc
Merge branch 'resume-reprovide-cycle' into reprovide-sweep-enabled 2025-10-24 18:34:18 +02:00
Guillaume Michel
ec0646a407
Merge branch 'master' into resume-reprovide-cycle 2025-10-24 18:32:12 +02:00
guillaumemichel
957589366a
bump kad-dht 2025-10-24 18:28:42 +02:00
Guillaume Michel
f9ffda0df3
Merge branch 'resume-reprovide-cycle' into reprovide-sweep-enabled 2025-10-24 18:08:16 +02:00
guillaumemichel
1eb61e0b43
bump kad-dht 2025-10-24 18:07:36 +02:00
Guillaume Michel
c75d2ccd01
Merge branch 'resume-reprovide-cycle' into reprovide-sweep-enabled 2025-10-24 11:59:47 +02:00
guillaumemichel
1c914aaff7
bump kad-dht 2025-10-24 11:58:58 +02:00
guillaumemichel
6c15a941ae
limit optimistic provide test to legacy provider 2025-10-24 11:57:17 +02:00
Guillaume Michel
46bf19211c
Merge branch 'resume-reprovide-cycle' into reprovide-sweep-enabled 2025-10-24 10:31:49 +02:00
guillaumemichel
20f3ae9da7
bump kad-dht 2025-10-24 10:31:10 +02:00
Dennis Trautwein
886ac22005
feat: trace delegated routing http client (#11017)
Some checks are pending
CodeQL / codeql (push) Waiting to run
Docker Check / lint (push) Waiting to run
Docker Check / build (push) Waiting to run
Gateway Conformance / gateway-conformance (push) Waiting to run
Gateway Conformance / gateway-conformance-libp2p-experiment (push) Waiting to run
Go Build / go-build (push) Waiting to run
Go Check / go-check (push) Waiting to run
Go Lint / go-lint (push) Waiting to run
Go Test / go-test (push) Waiting to run
Interop / interop-prep (push) Waiting to run
Interop / helia-interop (push) Blocked by required conditions
Interop / ipfs-webui (push) Blocked by required conditions
Sharness / sharness-test (push) Waiting to run
Spell Check / spellcheck (push) Waiting to run
2025-10-24 01:21:42 -07:00
guillaumemichel
10263645ef
bump kad-dht 2025-10-24 09:35:18 +02:00
guillaumemichel
731d092e76
optimistic provide test: increase swarm size 2025-10-24 01:32:23 +02:00
guillaumemichel
443b0d8bad
fix: prometheus metric 2025-10-24 01:25:18 +02:00
guillaumemichel
6b9ee7626e
indent 2025-10-24 00:48:14 +02:00
Guillaume Michel
d39695b68a
Merge branch 'resume-reprovide-cycle' into reprovide-sweep-enabled 2025-10-24 00:27:55 +02:00
guillaumemichel
708bcb8c91
bump kad-dht 2025-10-24 00:26:31 +02:00
Guillaume Michel
4ebb77f292
Merge branch 'resume-reprovide-cycle' into reprovide-sweep-enabled 2025-10-24 00:04:35 +02:00
guillaumemichel
82e0c3095f
bump kad-dht 2025-10-24 00:04:19 +02:00
guillaumemichel
e43ec5ef58
fix(sharness): prometheus provide metrics 2025-10-23 23:44:06 +02:00
guillaumemichel
90801e0cee
sharness: 'ipfs add' with an unregistered hash and wrapped leaves fails without crashing 2025-10-23 23:44:06 +02:00
guillaumemichel
1aafcd929e
attempt at fixing TestRoutingProvide 2025-10-23 21:44:54 +02:00
guillaumemichel
4076744cee
fix cli tests with enabled Provide Sweep 2025-10-23 21:18:37 +02:00
guillaumemichel
7b3d098a40
enable Provide Sweep by default 2025-10-23 21:18:22 +02:00
Guillaume Michel
2141c898da
Merge branch 'master' into resume-reprovide-cycle 2025-10-23 20:32:08 +02:00
Guillaume Michel
16479ec692
feat(provide): detailed ipfs provide stat (#11019)
Some checks are pending
CodeQL / codeql (push) Waiting to run
Docker Check / lint (push) Waiting to run
Docker Check / build (push) Waiting to run
Gateway Conformance / gateway-conformance (push) Waiting to run
Gateway Conformance / gateway-conformance-libp2p-experiment (push) Waiting to run
Go Build / go-build (push) Waiting to run
Go Check / go-check (push) Waiting to run
Go Lint / go-lint (push) Waiting to run
Go Test / go-test (push) Waiting to run
Interop / interop-prep (push) Waiting to run
Interop / helia-interop (push) Blocked by required conditions
Interop / ipfs-webui (push) Blocked by required conditions
Sharness / sharness-test (push) Waiting to run
Spell Check / spellcheck (push) Waiting to run
* feat: provide stats

* added N/A

* format

* workers stats alignment

* ipfs provide stat --all --compact

* consolidating compact stat

* update column alignment

* flags combinations errors

* command description

* change schedule AvgPrefixLen to float

* changelog

* alignments

* provide stat description draft

* rephrased provide-stats.md

* linking provide-stats.md from command description

* documentation test

* fix: refactor provide stat command type handling

- add extractSweepingProvider() helper to reduce nested type switching
- extract lowWorkerThreshold constant for worker availability check
- fix --lan error handling to work with buffered providers

* docs: add clarifying comments

* fix(commands): improve provide stat compact mode

- prevent panic when both columns are empty
- fix column alignment with UTF-8 characters
- only track col0MaxWidth for first column (as intended)

* test: add tests for ipfs provide stat command

- test basic functionality, flags, JSON output
- test legacy provider behavior
- test integration with content scheduling
- test disabled provider configurations
- add parseSweepStats helper with t.Helper()

* docs: improve provide command help text

- update tagline to "Control and monitor content providing"
- simplify help descriptions
- make error messages more consistent
- update tests to match new error messages

* metrics rename

```
Next reprovide at:
Next prefix:
```
updated to:
```
Next region prefix:
Next region reprovide:
```

* docs: improve Provide system documentation clarity

Enhance documentation for the Provide system to better explain how provider
records work and the differences between sweep and legacy modes.

Changes to docs/config.md:
- Provide section: add clear explanation of provider records and their role
- Provide.DHT: add provider record lifecycle and two provider systems overview
- Provide.DHT.Interval: explain relationship to expiration, contrast sweep vs legacy behavior
- Provide.DHT.SweepEnabled: rewrite to explain legacy problem, sweep solution, and efficiency gains
- Monitoring section: prioritize command-line tools (ipfs provide stat) before Prometheus

Changes to core/commands/provide.go:
- ipfs provide stat help: add explanation of provider records, TTL expiration, and how sweep batching works

Changes to docs/changelogs/v0.39.md:
- Add context about why stats matter for monitoring provider health
- Emphasize real-time monitoring workflow with watch command
- Explain what users can observe (rates, queues, worker availability)

* depend on latest kad-dht master

* docs: nits

---------

Co-authored-by: Marcin Rataj <lidel@lidel.org>
2025-10-23 20:29:36 +02:00