Commit Graph

185 Commits

Author SHA1 Message Date
Marcin Rataj
ccb49de852
feat(config): AutoConf with "auto" placeholders (#10883)
Some checks failed
CodeQL / codeql (push) Has been cancelled
Docker Build / docker-build (push) Has been cancelled
Gateway Conformance / gateway-conformance (push) Has been cancelled
Gateway Conformance / gateway-conformance-libp2p-experiment (push) Has been cancelled
Go Build / go-build (push) Has been cancelled
Go Check / go-check (push) Has been cancelled
Go Lint / go-lint (push) Has been cancelled
Go Test / go-test (push) Has been cancelled
Interop / interop-prep (push) Has been cancelled
Sharness / sharness-test (push) Has been cancelled
Spell Check / spellcheck (push) Has been cancelled
Interop / helia-interop (push) Has been cancelled
Interop / ipfs-webui (push) Has been cancelled
https://github.com/ipfs/kubo/pull/10883
https://github.com/ipshipyard/config.ipfs-mainnet.org/issues/3

---------

Co-authored-by: gammazero <gammazero@users.noreply.github.com>
2025-08-20 05:59:11 +02:00
Piotr Galar
2b858bb3e6
feat(ci): reusable spellcheck from unified CI (#10873)
* ci: use spellcheck from unified CI
* chore: fix spelling

---------

Co-authored-by: Marcin Rataj <lidel@lidel.org>
2025-08-14 00:35:35 +00:00
Marcin Rataj
c513e1adfa
fix(relay): feed connected peers to AutoRelay discovery (#10901)
Some checks failed
Docker Build / docker-build (push) Waiting to run
Gateway Conformance / gateway-conformance (push) Waiting to run
Gateway Conformance / gateway-conformance-libp2p-experiment (push) Waiting to run
Go Build / go-build (push) Waiting to run
Go Check / go-check (push) Waiting to run
Go Lint / go-lint (push) Waiting to run
Go Test / go-test (push) Waiting to run
Interop / interop-prep (push) Waiting to run
Interop / helia-interop (push) Blocked by required conditions
Interop / ipfs-webui (push) Blocked by required conditions
Sharness / sharness-test (push) Waiting to run
Spell Check / spellcheck (push) Waiting to run
CodeQL / codeql (push) Has been cancelled
* fix(relay): feed connected peers to AutoRelay discovery

Feed all connected swarm peers to AutoRelay as potential relay
candidates. This allows peers from HTTP routing and manual connections
to serve as relays, not just DHT-discovered peers.

Fixes #10899

* docs: changelog
2025-08-11 23:59:00 +02:00
Hector Sanjuan
19300f2d3f core: Add a ContentDiscovery field
No behaviour changes.

Currently we are using ProvideManyRouter for Bitswap, which is only meant to
use ContentDiscovery. This makes things more clear in that there is a
designated ContentDiscovery instance.
2025-08-01 12:22:13 +02:00
Hector Sanjuan
eb6cc02c06
feat(config): connmgr: expose silence period (#10827) 2025-06-17 15:55:57 +02:00
sukun
d70f6421d4
feat: AutoTLS with AutoNATv2 client (#10835)
* update autotls; libp2p to use autonatv2
* chore: p2p-forge@main with autonat fix

https://github.com/ipshipyard/p2p-forge/pull/70

---------

Co-authored-by: Marcin Rataj <lidel@lidel.org>
2025-06-17 00:29:50 +02:00
Andrew Gillis
20d9660a64
chore: use go-log/v2 (#10801)
Some checks are pending
CodeQL / codeql (push) Waiting to run
Docker Build / docker-build (push) Waiting to run
Gateway Conformance / gateway-conformance (push) Waiting to run
Gateway Conformance / gateway-conformance-libp2p-experiment (push) Waiting to run
Go Build / go-build (push) Waiting to run
Go Check / go-check (push) Waiting to run
Go Lint / go-lint (push) Waiting to run
Go Test / go-test (push) Waiting to run
Interop / interop-prep (push) Waiting to run
Interop / helia-interop (push) Blocked by required conditions
Interop / ipfs-webui (push) Blocked by required conditions
Sharness / sharness-test (push) Waiting to run
Spell Check / spellcheck (push) Waiting to run
* chore: update to go-log/v2

go-log v2 has been out for quite a while now and it is time to deprecate v1.

Replace all use of go-log with go-log/v2
Makes /api/v0/log/tail useful over HTTP
Updates dependencies that have moved to go-lov/v2
Removes support for ContextWithLoggable as this is not needed for tracing-like functionality
- Replaces: PR #8765
- Closes issue #8753
- Closes issue #9245
- Closes issue #10809

Other fixes:
* update go-ipfs-cmds
* update http logs test
* fix test
* Read/send one line of log data at a time
* Update -log-level docs
2025-05-19 13:04:05 -07:00
Hector Sanjuan
b5d73695ba
feat: opt-in http retrieval client (#10772)
* Feat: http retrieval as experimental feature

This introduces the http-retrieval capability as an experimental feature.

It can be enabled in the configuration `Experimental.HTTPRetrieval.Enabled = true`.

Documentation and changelog to be added later.

* refactor: HTTPRetrieval.Enabled as Flag

* docs(config): HTTPRetrieval section

* refactor: reusable MockHTTPContentRouter

* feat: HTTPRetrieval.TLSInsecureSkipVerify

allows self-signed certificates in tests

* feat(config): HTTPRetrieval.MaxBlockSize

* test: end-to-end HTTPRetrieval.Enabled

this spawns two http services on localhost:
1. HTTP router that returns HTTP provider when /routing/v1/providers/cid  i queried
2. HTTP provider that returns a block when /ipfs/cid is queried
3. Configures Kubo to use (1) instead of cid.contact

this seems to work (running test with DEBUG=true shows (1) was queried
for the test CID and returned multiaddr of (2), but Kubo never requested
test CID block from (2) – needs investigation

* fix: enable /routing/v1/peers for non-cid.contact

we artificially limited every delegated routing endpoint because of
cid.contact being limited to one endpoint

* feat: Routing.DelegatedRouters

make it easy to override the hardcoded implicit HTTP routeur URL
without having to set the entire custom Router.Routers and
Router.Methods

(http_retrieval_client_test.go still needs to be fixed in future commit)

* test: flag remaining work

* docs: review feedback

* refactor: providerQueryMgr with bitswapNetworks

this fixes two regressions:

(1) introduced in https://github.com/ipfs/kubo/issues/10717
    where we only used bitswapLib2p query manager
    (this is why E2E did not act on http provider)

(2) introduced in https://github.com/ipfs/kubo/pull/10765
    where it was not possible to set binary peerID in IgnoreProviders
    (we changed to []string)

* refactor: Bitswap.Libp2pEnabled

replaces Bitswap.Enabled with Bitswap.Libp2pEnabled
adds tests that confirm it is possible to disable libp2p bitswap fully
and only keep http in client mode

also, removes the need for passing empty blockstore in client-only mode

* docs: changelog

---------

Co-authored-by: Marcin Rataj <lidel@lidel.org>
2025-05-06 19:06:40 +02:00
Marcin Rataj
d6a5bc0b68
feat(AutoTLS): enabled by default with 1h RegistrationDelay (#10724)
Enables AutoTLS by default, but delays registration by 1h
if user did not explicitly set `AutoTLS.Enabled` to `true`
2025-02-25 22:25:58 +01:00
Marcin Rataj
256a739e3b
fix(autotls): renewal and AutoTLS.ShortAddrs (#10669)
Some checks are pending
CodeQL / codeql (push) Waiting to run
Docker Build / docker-build (push) Waiting to run
Gateway Conformance / gateway-conformance (push) Waiting to run
Gateway Conformance / gateway-conformance-libp2p-experiment (push) Waiting to run
Go Build / go-build (push) Waiting to run
Go Check / go-check (push) Waiting to run
Go Lint / go-lint (push) Waiting to run
Go Test / go-test (push) Waiting to run
Interop / interop-prep (push) Waiting to run
Interop / helia-interop (push) Blocked by required conditions
Interop / ipfs-webui (push) Blocked by required conditions
Sharness / sharness-test (push) Waiting to run
* fix(autotls): renewal and AutoTLS.ShortAddrs

updates to p2p-forge/client with fix from
https://github.com/ipshipyard/p2p-forge/pull/42

we also add AutoTLS.ShortAddrs flag and enable it by default
to benefit from shorter addrs discusses in
https://github.com/ipshipyard/p2p-forge/pull/40

* test: fix flaky NewRandPort

reducing chance of bind: address already in use
2025-01-22 17:26:15 +01:00
Andrew Gillis
104b661134
update dependencies (#10664) 2025-01-21 13:00:32 -08:00
Andrew Gillis
7383e1ecbb
chore: fix typos and comment formatting (#10653)
Some checks failed
CodeQL / codeql (push) Has been cancelled
Docker Build / docker-build (push) Has been cancelled
Gateway Conformance / gateway-conformance (push) Has been cancelled
Gateway Conformance / gateway-conformance-libp2p-experiment (push) Has been cancelled
Go Build / go-build (push) Has been cancelled
Go Check / go-check (push) Has been cancelled
Go Lint / go-lint (push) Has been cancelled
Go Test / go-test (push) Has been cancelled
Interop / interop-prep (push) Has been cancelled
Sharness / sharness-test (push) Has been cancelled
Interop / helia-interop (push) Has been cancelled
Interop / ipfs-webui (push) Has been cancelled
* fix typos and comment formatting
2025-01-06 18:17:39 -08:00
Adin Schmahmann
397c346ae0
feat(libp2p): shared TCP listeners and AutoTLS.AutoWSS (#10565)
Some checks are pending
CodeQL / codeql (push) Waiting to run
Docker Build / docker-build (push) Waiting to run
Gateway Conformance / gateway-conformance (push) Waiting to run
Gateway Conformance / gateway-conformance-libp2p-experiment (push) Waiting to run
Go Build / go-build (push) Waiting to run
Go Check / go-check (push) Waiting to run
Go Lint / go-lint (push) Waiting to run
Go Test / go-test (push) Waiting to run
Interop / interop-prep (push) Waiting to run
Interop / helia-interop (push) Blocked by required conditions
Interop / ipfs-webui (push) Blocked by required conditions
Sharness / sharness-test (push) Waiting to run
* feat(libp2p): enable shared TCP listeners

* docs: switch mentions of /ws to /tcp/4001

* feat: AutoTLS.AutoWSS

This adds AutoTLS.AutoWSS flag that is set to true by default.

It will check if Addresses.Swarm contain explicit /ws listener,
and if not found, it will append one per every /tcp listener

This way existing TCP ports are reused without any extra configuration,
but we don't break user's who have custom / explicit /ws listener
already.

I also moved logger around, to include Addresses.Swarm inspection
results in `autotls` logger.

* chore: go-libp2p v0.38.1

https://github.com/libp2p/go-libp2p/releases/tag/v0.38.0
https://github.com/libp2p/go-libp2p/releases/tag/v0.38.1

* docs: AutoTLS.AutoWSS and go-libp2p v0.38.x

* chore: p2p-forge/client v0.2.0

https://github.com/ipshipyard/p2p-forge/releases/tag/v0.2.0

* fix: disable libp2p.ShareTCPListener() in PNET

* chore(ci): timeout sharness after 15m

average successful run is  <9 minutes, no need to wait for 20
https://github.com/ipfs/kubo/actions/workflows/sharness.yml?query=is%3Asuccess

---------

Co-authored-by: Andrew Gillis <11790789+gammazero@users.noreply.github.com>
Co-authored-by: Marcin Rataj <lidel@lidel.org>
2024-12-20 18:41:25 +01:00
Andi
642e58ef3b
chore: use errors.New to replace fmt.Errorf with no parameters (#10617)
Some checks failed
CodeQL / codeql (push) Has been cancelled
Docker Build / docker-build (push) Has been cancelled
Gateway Conformance / gateway-conformance (push) Has been cancelled
Gateway Conformance / gateway-conformance-libp2p-experiment (push) Has been cancelled
Go Build / go-build (push) Has been cancelled
Go Check / go-check (push) Has been cancelled
Go Lint / go-lint (push) Has been cancelled
Go Test / go-test (push) Has been cancelled
Interop / interop-prep (push) Has been cancelled
Sharness / sharness-test (push) Has been cancelled
Interop / helia-interop (push) Has been cancelled
Interop / ipfs-webui (push) Has been cancelled
2024-12-10 13:50:35 -08:00
Marten Seemann
778a418952
refactor(rcmgr): use default libp2p rcmgr metrics (#9947)
Some checks failed
CodeQL / codeql (push) Has been cancelled
Docker Build / docker-build (push) Has been cancelled
Gateway Conformance / gateway-conformance (push) Has been cancelled
Gateway Conformance / gateway-conformance-libp2p-experiment (push) Has been cancelled
Go Build / go-build (push) Has been cancelled
Go Check / go-check (push) Has been cancelled
Go Lint / go-lint (push) Has been cancelled
Go Test / go-test (push) Has been cancelled
Interop / interop-prep (push) Has been cancelled
Sharness / sharness-test (push) Has been cancelled
Interop / helia-interop (push) Has been cancelled
Interop / ipfs-webui (push) Has been cancelled
Co-authored-by: Marcin Rataj <lidel@lidel.org>
2024-11-27 20:50:43 +01:00
Marcin Rataj
e18fdf28d2
chore: label implicit loggers
this should help with debugging
https://github.com/ipshipyard/p2p-forge/issues/8
on boxes with debug log level
(and hide noise from normal users)
2024-11-07 20:53:05 +01:00
Adin Schmahmann
84345afcde fix(autotls): store certificates at the location from the repo path (#10566)
* fix(autotls): store certificates at the location from the repo path
* docs(autotls): cert storale and other caveats

---------

Co-authored-by: Marcin Rataj <lidel@lidel.org>
(cherry picked from commit 1ca0ae0af6)
2024-10-30 23:24:20 +01:00
Adin Schmahmann
3134fd246e
feat(AutoTLS): opt-in WSS certs from p2p-forge at libp2p.direct (#10521)
Some checks are pending
CodeQL / codeql (push) Waiting to run
Docker Build / docker-build (push) Waiting to run
Gateway Conformance / gateway-conformance (push) Waiting to run
Gateway Conformance / gateway-conformance-libp2p-experiment (push) Waiting to run
Go Build / go-build (push) Waiting to run
Go Check / go-check (push) Waiting to run
Go Lint / go-lint (push) Waiting to run
Go Test / go-test (push) Waiting to run
Interop / interop-prep (push) Waiting to run
Interop / helia-interop (push) Blocked by required conditions
Interop / ipfs-webui (push) Blocked by required conditions
Sharness / sharness-test (push) Waiting to run
Co-authored-by: Marcin Rataj <lidel@lidel.org>
2024-10-29 22:37:33 +01:00
Marco Munizaga
56c68a1657
refactor: update to go-libp2p v0.37.0 (#10554)
Some checks failed
CodeQL / codeql (push) Has been cancelled
Docker Build / docker-build (push) Has been cancelled
Gateway Conformance / gateway-conformance (push) Has been cancelled
Gateway Conformance / gateway-conformance-libp2p-experiment (push) Has been cancelled
Go Build / go-build (push) Has been cancelled
Go Check / go-check (push) Has been cancelled
Go Lint / go-lint (push) Has been cancelled
Go Test / go-test (push) Has been cancelled
Interop / interop-prep (push) Has been cancelled
Sharness / sharness-test (push) Has been cancelled
Interop / helia-interop (push) Has been cancelled
Interop / ipfs-webui (push) Has been cancelled
https://github.com/libp2p/go-libp2p/releases/tag/v0.37.0

Co-authored-by: Marcin Rataj <lidel@lidel.org>
2024-10-23 01:31:40 +02:00
Marcin Rataj
52ca370759
feat(routing/http): support IPIP-484 and streaming (#10534) 2024-10-03 22:58:25 +00:00
fengzie
a17830754c
Fix issue in ResourceManager and nopfsPlugin about repo path (#10492) 2024-09-28 20:58:24 -07:00
Andrew Gillis
5c91fe655c
Merge branch 'master' into tls-by-default 2024-08-06 20:16:57 -07:00
Marcin Rataj
ffab7b271a
feat: run AutoNAT V2 service in addition to V1 (#10468)
* feat: libp2p.EnableAutoNATv2

Part of https://github.com/ipfs/kubo/issues/10091
We include a flag that allows shutting down V2 in case there are issues
with it.

* docs: EnableAutoNATv2
2024-08-06 21:51:45 +02:00
Marcin Rataj
feef0851bd
feat: go-libp2p 0.36 and /webrtc-direct listener (#10463)
Co-authored-by: Marco Munizaga <git@marcopolo.io>
2024-08-06 21:25:27 +02:00
Marco Munizaga
121c6023c3
chore: go-libp2p 0.35.1 (#10430)
https://github.com/libp2p/go-libp2p/releases/tag/v0.35.1

Co-authored-by: Marcin Rataj <lidel@lidel.org>
2024-06-14 21:33:00 +02:00
Lukáš Lukáč
40cf630874
fix(libp2p): streams config validation in resource manager (#10435) 2024-06-03 21:13:30 +02:00
Henrique Dias
b79cedec73
chore: libp2p 0.34.1 (#10429) 2024-05-27 10:16:13 +02:00
Henrique Dias
11183bb2f5
chore: upgrade go-libp2p-kad-dht (#10378)
* chore: upgrade go-libp2p-kad-dht
* config: make LoopbackAddressesOnLanDHT a Flag
* config: add DefaultLoopbackAddressesOnLanDHT
* docs(config): Routing.LoopbackAddressesOnLanDHT

---------

Co-authored-by: Marcin Rataj <lidel@lidel.org>
2024-04-04 12:56:19 +00:00
Henrique Dias
cd78f2eae3
chore(config): make Routing.AcceleratedDHTClient a Flag (#10384) 2024-04-04 13:59:31 +02:00
Henrique Dias
d1db95c447 config: remove all options that are marked as REMOVED
Most of the removed options are many years old. In addition, they've all been removed in past iterations of Kubo. Some options were marked as removed in the config.md, but we still had a warning in the code to let users know they have been removed.

I think it's been long enough for all of this options, and enough Kubo iterations in order to alert the users. It is good to keep it in the config.md for now so that people can actually check. However, I think it's time to remove them from the code itself.
2024-01-22 15:33:27 +01:00
Jorropo
6a51849c29 libp2p: remove mplex
Fixes: #10069
2023-11-22 08:06:39 +03:00
Jorropo
a617c52f57 libp2p: default to preffering TLS
See https://github.com/libp2p/go-libp2p/pull/2650.
2023-11-22 06:06:15 +01:00
Henrique Dias
9f3251c1b0 feat: add WebRTC Direct support 2023-11-03 14:43:42 +01:00
Alejandro Criado-Pérez
cc3c224c62
docs: fix typos 2023-09-22 11:08:26 +00:00
Icarus9913
9fdb085605
refactor: stop using go-libp2p deprecated peer.ID.Pretty
Signed-off-by: Icarus9913 <icaruswu66@qq.com>
2023-09-18 15:58:15 +02:00
Kay
2c66ea6995
refactor: if statement (#10105) 2023-09-05 15:35:51 +02:00
Henrique Dias
4606586207
feat(gateway): expose /routing/v1 server (opt-in) (#9877) 2023-08-25 17:30:04 +02:00
Henrique Dias
5ea58251d5
chore: update boxo, go-libp2p, and internalize mplex (#10095) 2023-08-24 06:11:02 +00:00
Kay
f12b372af9
style: gofumpt and godot [skip changelog] (#10081) 2023-08-17 14:02:08 +02:00
Jorropo
f805b9fcda feat: allow users to optin again into mplex
This is a partial revert of 7220409394.

Closes #9958
2023-08-15 14:53:44 +02:00
Jorropo
7220409394 feat: remove Mplex
Mplex does not implement backpressure, our implementation will randomly reset streams if buffers overflow instead of risking deadlocks.

In the past we had a bug where kubo nodes would prefer mplex over yamux. Turning off mplex make our connections to thoses nodes negociate yamux.

Closes #9958
2023-08-15 11:35:45 +02:00
Marco Munizaga
9013cf9956 Fix usage numbers 2023-07-31 16:16:00 +02:00
Mohamed MHAMDI
4c35289556 fix(relay): apply user provider options 2023-07-26 15:57:52 +02:00
Marco Munizaga
649283bb6c chore: update go-libp2p to v0.29.0 2023-07-26 14:48:19 +02:00
Jorropo
9c02eecad7
fix: mark all routers DoNotWaitForSearchValue (#10020)
* fix: mark ipns pubsub router DoNotWaitForSearchValue

That means if the DHT has finished searching and no one responded over pubsub *yet*, we will not spend 1 minute searching for no reason.

This also include other error handling bug fixes inside `go-libp2p-routing-helpers`.

Fixes: #9927

* routing: bring back the old IPNS behaviour

Stop making this configurable let everything race like it used to do.
2023-07-25 19:39:21 +02:00
Jorropo
b55cd72657 fix: print rcmgr to logger 2023-06-12 16:25:22 +02:00
Jorropo
e7294cbdff feat: warn users who are falling behind reprovides
Fixes: #9704
Fixes: #9702
Fixes: #9703
Fixes: #9419
2023-06-08 10:05:30 +02:00
Ivan Schasny
e5b33924a7
fix: correct list of addresses for delegated routing (#9920) 2023-06-07 17:59:55 +02:00
Henrique Dias
c178c51835
fix: deadlock on retrieving WebTransport addresses (#9857)
Co-authored-by: Marco Polo <git@marcopolo.io>
2023-05-08 16:11:03 +02:00
Gus Eggert
4ca36c41b6
fix: use default HTTP routers when FullRT DHT client is used (#9841) 2023-05-01 15:29:34 -04:00