Commit Graph

346 Commits

Author SHA1 Message Date
guillaumemichel
0dbf157919
restore dual dht provider 2025-08-23 00:02:18 +02:00
guillaumemichel
812f7fcf6d
docs 2025-08-22 22:40:26 +02:00
guillaumemichel
79038738aa
don't provide LAN DHT 2025-08-22 13:34:36 +02:00
Guillaume Michel
814085f76c
Merge branch 'master' into reprovide-sweep 2025-08-22 12:00:34 +02:00
Hector Sanjuan
846e219ead
feat: Reprovider.Strategy: rename "flat" to "all" (#10928)
* Reprovider strategy: rename "flat" to "all".

Value "flat" now parses to "all". Behaviour from "all" removed.

Fixes #10864 which has detailed explanation.

* core/node/provider.go: remove unused function mfsRootProvider

It was used in the "all" strategy.

* docs: improve reprovider.strategy=all changelog framing

- highlight memory efficiency improvements
- clarify this removes v0.28 workaround
- update config.md memory requirements
- fix announce-on profile typo

* feat: deprecate Reprovider.Strategy=flat

- add deprecation warning in daemon.go when flat strategy is detected
- document that flat is deprecated in ParseReproviderStrategy comment
- add explicit test case for flat -> all mapping
- flat continues to work but users are warned to migrate to all

---------

Co-authored-by: Marcin Rataj <lidel@lidel.org>
2025-08-21 19:02:29 +02:00
guillaumemichel
dc2bb67d22
extended tests 2025-08-21 16:30:45 +02:00
guillaumemichel
c9fbe62069
fix merge issues 2025-08-20 15:15:08 +02:00
guillaumemichel
07189277e2
fx cyclic deps 2025-08-20 14:08:31 +02:00
guillaumemichel
a44d3f57e5
fix fx 2025-08-20 10:42:57 +02:00
guillaumemichel
55df2329e4
Merge branch 'master' into reprovide-sweep 2025-08-20 10:26:25 +02:00
guillaumemichel
c571318428
using latest kad-dht commit 2025-08-20 10:07:59 +02:00
Marcin Rataj
ccb49de852
feat(config): AutoConf with "auto" placeholders (#10883)
Some checks failed
CodeQL / codeql (push) Has been cancelled
Docker Build / docker-build (push) Has been cancelled
Gateway Conformance / gateway-conformance (push) Has been cancelled
Gateway Conformance / gateway-conformance-libp2p-experiment (push) Has been cancelled
Go Build / go-build (push) Has been cancelled
Go Check / go-check (push) Has been cancelled
Go Lint / go-lint (push) Has been cancelled
Go Test / go-test (push) Has been cancelled
Interop / interop-prep (push) Has been cancelled
Sharness / sharness-test (push) Has been cancelled
Spell Check / spellcheck (push) Has been cancelled
Interop / helia-interop (push) Has been cancelled
Interop / ipfs-webui (push) Has been cancelled
https://github.com/ipfs/kubo/pull/10883
https://github.com/ipshipyard/config.ipfs-mainnet.org/issues/3

---------

Co-authored-by: gammazero <gammazero@users.noreply.github.com>
2025-08-20 05:59:11 +02:00
Andrew Gillis
7109534464
Upgrade to Boxo v0.34.0 (#10917)
Some checks failed
CodeQL / codeql (push) Has been cancelled
Docker Build / docker-build (push) Has been cancelled
Gateway Conformance / gateway-conformance (push) Has been cancelled
Gateway Conformance / gateway-conformance-libp2p-experiment (push) Has been cancelled
Go Build / go-build (push) Has been cancelled
Go Check / go-check (push) Has been cancelled
Go Lint / go-lint (push) Has been cancelled
Go Test / go-test (push) Has been cancelled
Interop / interop-prep (push) Has been cancelled
Sharness / sharness-test (push) Has been cancelled
Spell Check / spellcheck (push) Has been cancelled
Interop / helia-interop (push) Has been cancelled
Interop / ipfs-webui (push) Has been cancelled
* Use ValidatingBlockstore when HashOnRead is configured
* fix ValidatingBlockstore creation
* upgrade to boxo v0.34.0
* Use tagged boxo release
2025-08-15 14:43:26 -07:00
Piotr Galar
2b858bb3e6
feat(ci): reusable spellcheck from unified CI (#10873)
* ci: use spellcheck from unified CI
* chore: fix spelling

---------

Co-authored-by: Marcin Rataj <lidel@lidel.org>
2025-08-14 00:35:35 +00:00
guillaumemichel
9764140cc9
boxo PR fix
Some checks failed
Spell Check / spellcheck (push) Has been cancelled
2025-08-12 15:53:02 +02:00
guillaumemichel
6788e6e351
updated boxo provide interface 2025-08-12 15:31:57 +02:00
guillaumemichel
b650b9f4f4
make linter happy 2025-08-12 14:36:36 +02:00
guillaumemichel
49f74e8cd5
use latest kad-dht commit 2025-08-12 14:10:38 +02:00
gammazero
5ad8e3a944 Add Clear to Provider interface 2025-08-11 22:54:52 -10:00
guillaumemichel
107a0bb25a moved Provider interface from kad-dht to kubo/core/node 2025-08-11 22:17:10 -10:00
guillaumemichel
688944ae33 move Provider interface to from kad-dht to node 2025-08-11 22:10:36 -10:00
guillaumemichel
d67173ae27 merged IpfsNode provider and reprovider 2025-08-11 21:58:36 -10:00
guillaumemichel
d140885902 config docs 2025-08-11 21:43:12 -10:00
guillaumemichel
82d6feeb49 config 2025-08-11 21:43:12 -10:00
guillaumemichel
5a57c4f051 initialize MHStore 2025-08-11 21:43:12 -10:00
guillaumemichel
d13eaa6b6c fix missing KeyChanFunc 2025-08-11 21:43:12 -10:00
guillaumemichel
d7d3e2bcfa restore boxo KeyChanFunc 2025-08-11 21:43:12 -10:00
guillaumemichel
2dfcd3d8a2 update KeyChanFunc type alias 2025-08-11 21:43:12 -10:00
guillaumemichel
4025316c0d add noop reprovider 2025-08-11 21:43:12 -10:00
guillaumemichel
f7ab84f22b removing SweepingProvider 2025-08-11 21:43:12 -10:00
guillaumemichel
7b7e528d5f reverting changes in core/node/libp2p/routing.go 2025-08-11 21:43:12 -10:00
guillaumemichel
6644657ff0 new attempt at integrating 2025-08-11 21:43:09 -10:00
guillaumemichel
4333fcf4ca another iteration 2025-08-11 21:29:19 -10:00
guillaumemichel
c2d6065682 disabled initial network estimation 2025-08-11 21:29:19 -10:00
guillaumemichel
92710b3f2c noop provider for nil dht 2025-08-11 21:29:19 -10:00
guillaumemichel
99f05503ac SweepingReprovider test 2025-08-11 21:29:19 -10:00
guillaumemichel
b62ac647c8 back to start 2025-08-11 21:28:44 -10:00
guillaumemichel
456c69beb6 revert to provider.System 2025-08-11 21:28:41 -10:00
guillaumemichel
a60eaee7f2 dual reprovider 2025-08-11 21:22:09 -10:00
guillaumemichel
7b11dfccc5 change router type 2025-08-11 21:18:52 -10:00
guillaumemichel
f406fdea42 fix provider type 2025-08-11 21:18:51 -10:00
guillaumemichel
302cfa81ae update reprovider dep 2025-08-11 21:11:30 -10:00
guillaumemichel
caa37ccf15 reprovide sweep draft 2025-08-11 21:09:32 -10:00
Marcin Rataj
c513e1adfa
fix(relay): feed connected peers to AutoRelay discovery (#10901)
Some checks failed
Docker Build / docker-build (push) Waiting to run
Gateway Conformance / gateway-conformance (push) Waiting to run
Gateway Conformance / gateway-conformance-libp2p-experiment (push) Waiting to run
Go Build / go-build (push) Waiting to run
Go Check / go-check (push) Waiting to run
Go Lint / go-lint (push) Waiting to run
Go Test / go-test (push) Waiting to run
Interop / interop-prep (push) Waiting to run
Interop / helia-interop (push) Blocked by required conditions
Interop / ipfs-webui (push) Blocked by required conditions
Sharness / sharness-test (push) Waiting to run
Spell Check / spellcheck (push) Waiting to run
CodeQL / codeql (push) Has been cancelled
* fix(relay): feed connected peers to AutoRelay discovery

Feed all connected swarm peers to AutoRelay as potential relay
candidates. This allows peers from HTTP routing and manual connections
to serve as relays, not just DHT-discovered peers.

Fixes #10899

* docs: changelog
2025-08-11 23:59:00 +02:00
Hector Sanjuan
a673c2ec95
fix: Provide according to Reprovider.Strategy (#10886)
* Provide according to strategy

Updates boxo to a version with the changes from https://github.com/ipfs/boxo/pull/976, which decentralize the providing responsibilities (from a central providing.Exchange to blockstore, pinner, mfs).

The changes consist in initializing the Pinner, MFS and the blockstore with the provider.System, which is created first.

Since the provider.System is created first, the reproviding KeyChanFunc is set
later when we can create it once we have the Pinner, MFS and the blockstore.

Some additional work applies to the Add() workflow. Normally, blocks would get provided at the Blockstore or the Pinner, but when adding blocks AND a "pinned" strategy is used, the blockstore does not provide, and the
pinner does not traverse the DAG (and thus doesn't provide either), so we need to provide directly from the Adder. This is resolved by wrapping the DAGService in a "providingDAGService" which provides every added block, when using the "pinned" strategy.

`ipfs --offline add` when the ONLINE daemon is running will now announce blocks per the chosen strategy, where before it did not announce them. This is documented in the changelog. A couple of releases ago, adding with `ipfs --offline add` was faster, but this is no longer the case so we are not incurring in any penalties by sticking to the fact that the daemon is online and has a providing strategy that we follow.

Co-authored-by: gammazero <11790789+gammazero@users.noreply.github.com>
Co-authored-by: Marcin Rataj <lidel@lidel.org>
2025-08-08 10:56:44 +02:00
Hector Sanjuan
58ad11b573
Merge pull request #10889 from ipfs/reuse-connevtmanager
Some checks are pending
CodeQL / codeql (push) Waiting to run
Docker Build / docker-build (push) Waiting to run
Gateway Conformance / gateway-conformance (push) Waiting to run
Gateway Conformance / gateway-conformance-libp2p-experiment (push) Waiting to run
Go Build / go-build (push) Waiting to run
Go Check / go-check (push) Waiting to run
Go Lint / go-lint (push) Waiting to run
Go Test / go-test (push) Waiting to run
Interop / interop-prep (push) Waiting to run
Interop / helia-interop (push) Blocked by required conditions
Interop / ipfs-webui (push) Blocked by required conditions
Sharness / sharness-test (push) Waiting to run
Spell Check / spellcheck (push) Waiting to run
Bitswap: use a single ConnectEventManager.
2025-08-06 22:39:25 +02:00
Andrew Gillis
90b73d2ad2
refactor: remove goprocess (#10872)
* refactor: remove goprocess

The `goprocess` package is no longer needed. It can be replaces by modern `context` and `context.AfterFunc`.

* mod tidy

* log unmount errors on shutdown

* Do not log non-mounted errors on shutdown

* Use WaitGroup associated with IPFS node to wait for services to whutdown

* Prefer explicit Close to context.ArterFunc

* Do not use node-level WaitGroup

* Unmount for non-supported platforms

* fix return values

* test: daemon shuts down gracefully

make sure ongoing operations dont block shutdown

* test(cli): add TestFUSE

* test: smarter RequiresFUSE

opportunistically run FUSE tests if env has fusermount
and TEST_FUSE was not explicitly set

* docs: changelog

---------

Co-authored-by: gammazero <gammazero@users.noreply.github.com>
Co-authored-by: Marcin Rataj <lidel@lidel.org>
2025-08-06 00:33:45 +02:00
Hector Sanjuan
19300f2d3f core: Add a ContentDiscovery field
No behaviour changes.

Currently we are using ProvideManyRouter for Bitswap, which is only meant to
use ContentDiscovery. This makes things more clear in that there is a
designated ContentDiscovery instance.
2025-08-01 12:22:13 +02:00
Hector Sanjuan
017d5003f2 Bitswap: use a single ConnectEventManager.
After boxo v0.33.1, this is a recommended step to fix http retrieval bugs.

Having a single ConnectEventManager prevents misdirected operations in the
network.Router to change the Connectedness state in a way that the counterpart
(httpnet or bsnet) can later correct.
2025-07-31 16:13:47 +02:00
Andrew Gillis
a22efea6f3
provider: clear provide queue when reprovide strategy changes (#10863)
* provider: clear reprovide queue when reprovide strategy changes

When the currently configured reprovide strategy does not match the previous strategy read from the datastore, then clear the reprovide queue and update the reprovide strategy that is stored in the datastore.

Depends on https://github.com/ipfs/boxo/pull/978

Closes #10829

* Update docs/changelogs/v0.36.md

Co-authored-by: Guillaume Michel <guillaumemichel@users.noreply.github.com>

* update log message
* update boxo
* Move change log to v0.37.md
* Add `provide clear` command to clear provide queue

The `provide clear` command clears all items from the provide queue and prints out the number of items removed from the queue. The `quiet` option tells the command not to print output.

* refactor(cmds): ipfs provide clear

moving to new namespace to avoid conflicts, and also document other
commands

* docs: clarify Reprovider.Strategy
* chore: remove undesired md link
2025-07-16 01:28:27 -07:00