Commit Graph

1083 Commits

Author SHA1 Message Date
Marcin Rataj
033de22e3b fix: quic-go v0.49.0 (#10673)
https://github.com/quic-go/quic-go/releases/tag/v0.49.0
(cherry picked from commit 82945a9392)
2025-01-23 19:11:59 +01:00
Andrew Gillis
62ff2e6d37 Upgrade to Boxo v0.27.2 (#10672)
* Upgrade to Boxo v0.27.2

(cherry picked from commit a26aee1a11)
2025-01-23 16:49:36 +01:00
Marcin Rataj
6426e191bc
chore: 0.33.0-rc2 2025-01-22 22:43:44 +01:00
Andrew Gillis
5fdd87c850 Upgrade to Boxo v0.27.1 (#10671)
* Upgrade to Boxo v0.27.1
* update changelog
* use tagged release of boxo

(cherry picked from commit fcfdbfb32e)
2025-01-22 22:32:22 +01:00
Marcin Rataj
00cb38865f fix(autotls): renewal and AutoTLS.ShortAddrs (#10669)
* fix(autotls): renewal and AutoTLS.ShortAddrs

updates to p2p-forge/client with fix from
https://github.com/ipshipyard/p2p-forge/pull/42

we also add AutoTLS.ShortAddrs flag and enable it by default
to benefit from shorter addrs discusses in
https://github.com/ipshipyard/p2p-forge/pull/40

* test: fix flaky NewRandPort

reducing chance of bind: address already in use

(cherry picked from commit 256a739e3b)
2025-01-22 17:41:03 +01:00
Andrew Gillis
c51d7cd684 update changelog for boxo and go-libp2p (#10668)
(cherry picked from commit d0bc598e05)
2025-01-22 17:40:45 +01:00
Andrew Gillis
7d878bb5c9 Upgrade to go-ds-pebble v0.4.2 (#10667)
Upgrade pebble to v1.1.4

(cherry picked from commit 22466fa051)
2025-01-22 17:40:35 +01:00
Andrew Gillis
3dd1232637 Upgrade to Boxo v0.27.0 (#10665)
* Upgrade to Boxo v0.27.0

(cherry picked from commit 332a9484eb)
2025-01-22 17:39:57 +01:00
Andrew Gillis
2e15a20c1c update dependencies (#10664)
(cherry picked from commit 104b661134)
2025-01-22 17:39:44 +01:00
Marcin Rataj
6aa7d99fe4 fix(dns): update default DNSLink resolvers (#10655)
* fix(dns): update default DNSLink resolvers

Depends on
- https://github.com/ipfs/boxo/pull/781
- https://github.com/ipfs/boxo/pull/782

* chore: latest boxo main and go-doh-resolver v0.5.0

makes error message better, informing which URL failled

* chore: p2p-forge v0.2.1+

this removes acmez/v2 and fixes go vet

* chore: latest boxo with DoH fixes

commit from main branch that includes:
https://github.com/ipfs/boxo/pull/645
https://github.com/ipfs/boxo/pull/782
https://github.com/ipfs/boxo/pull/781

(cherry picked from commit b021a00bf6)
2025-01-14 22:24:04 +01:00
Marcin Rataj
89c6e7eb89 chore: p2p-forge v0.2.2 + go-libp2p-kad-dht v0.28.2 (#10663)
* chore: p2p-forge v0.2.2

https://github.com/ipshipyard/p2p-forge/releases/tag/v0.2.2

* chore: go-libp2p-kad-dht v0.28.2

https://github.com/libp2p/go-libp2p-kad-dht/releases/tag/v0.28.2
(cherry picked from commit 5f8dd91c3c)
2025-01-14 01:57:03 +01:00
Marcin Rataj
9386fdcdfb fix(cli): support HTTPS in ipfs --api (#10659)
* fix(cli): support HTTPS in ipfs --api

Closes #10539

* chore: go-ipfs-cmds v0.14.1

https://github.com/ipfs/go-ipfs-cmds/releases/tag/v0.14.1

* docs: ipfs --api example

* test(cli): https rpc support

makes sure we dont have regression where HTTPS endpoint
starts getting cleartext requests

(cherry picked from commit 3b098b969a)
2025-01-13 18:58:03 +01:00
Andrew Gillis
a0c57cba22 chore: fix typos and comment formatting (#10653)
* fix typos and comment formatting

(cherry picked from commit 7383e1ecbb)
2025-01-13 18:56:53 +01:00
Andrew Gillis
5e1bacc2a8 fix/gateway: escape directory redirect url (#10649)
* fix/gateway: escape directory redirect URLs

When a director gets redirected to a URL with a treailing slash, special chars in the directory name must be escaped in the redirect URL. This upgrades to a version of box that has that fix.

Closes #10536

* Fix sharness test for new redirect URLs
* Update to latest boxo
* Use latest gateway-conformance

(cherry picked from commit b7cc47aef5)
2025-01-13 18:56:20 +01:00
Andrew Gillis
53792003bd collection of typo fixes (#10647)
(cherry picked from commit 9425421114)
2025-01-13 18:55:10 +01:00
Marcin Rataj
9282f08188
fix: ipfs-webui v4.4.2 (#10635)
https://github.com/ipfs/ipfs-webui/releases/tag/v4.4.2
2024-12-20 19:24:21 +01:00
Adin Schmahmann
397c346ae0
feat(libp2p): shared TCP listeners and AutoTLS.AutoWSS (#10565)
Some checks are pending
CodeQL / codeql (push) Waiting to run
Docker Build / docker-build (push) Waiting to run
Gateway Conformance / gateway-conformance (push) Waiting to run
Gateway Conformance / gateway-conformance-libp2p-experiment (push) Waiting to run
Go Build / go-build (push) Waiting to run
Go Check / go-check (push) Waiting to run
Go Lint / go-lint (push) Waiting to run
Go Test / go-test (push) Waiting to run
Interop / interop-prep (push) Waiting to run
Interop / helia-interop (push) Blocked by required conditions
Interop / ipfs-webui (push) Blocked by required conditions
Sharness / sharness-test (push) Waiting to run
* feat(libp2p): enable shared TCP listeners

* docs: switch mentions of /ws to /tcp/4001

* feat: AutoTLS.AutoWSS

This adds AutoTLS.AutoWSS flag that is set to true by default.

It will check if Addresses.Swarm contain explicit /ws listener,
and if not found, it will append one per every /tcp listener

This way existing TCP ports are reused without any extra configuration,
but we don't break user's who have custom / explicit /ws listener
already.

I also moved logger around, to include Addresses.Swarm inspection
results in `autotls` logger.

* chore: go-libp2p v0.38.1

https://github.com/libp2p/go-libp2p/releases/tag/v0.38.0
https://github.com/libp2p/go-libp2p/releases/tag/v0.38.1

* docs: AutoTLS.AutoWSS and go-libp2p v0.38.x

* chore: p2p-forge/client v0.2.0

https://github.com/ipshipyard/p2p-forge/releases/tag/v0.2.0

* fix: disable libp2p.ShareTCPListener() in PNET

* chore(ci): timeout sharness after 15m

average successful run is  <9 minutes, no need to wait for 20
https://github.com/ipfs/kubo/actions/workflows/sharness.yml?query=is%3Asuccess

---------

Co-authored-by: Andrew Gillis <11790789+gammazero@users.noreply.github.com>
Co-authored-by: Marcin Rataj <lidel@lidel.org>
2024-12-20 18:41:25 +01:00
Marcin Rataj
f2c190535c
feat(flatfs): default to sync=false (#10632)
Some checks are pending
CodeQL / codeql (push) Waiting to run
Docker Build / docker-build (push) Waiting to run
Gateway Conformance / gateway-conformance (push) Waiting to run
Gateway Conformance / gateway-conformance-libp2p-experiment (push) Waiting to run
Go Build / go-build (push) Waiting to run
Go Check / go-check (push) Waiting to run
Go Lint / go-lint (push) Waiting to run
Go Test / go-test (push) Waiting to run
Interop / interop-prep (push) Waiting to run
Interop / helia-interop (push) Blocked by required conditions
Interop / ipfs-webui (push) Blocked by required conditions
Sharness / sharness-test (push) Waiting to run
2024-12-19 16:14:45 -08:00
Marcin Rataj
295fd96bd0
docs: clarify Swarm.ResourceMgr.MaxMemory (#10622)
related to ux problem described in #10621
2024-12-20 00:48:21 +01:00
Hector Sanjuan
519ae27dce
feat: expose BlockKeyCacheSize and enable WriteThrough datastore options (#10614)
* feat: expose BlockKeyCacheSize and enable WriteThrough when bloom filter disabled
* import/config: add BatchMaxSize and BatchMaxNodes
* config: make BlockKeyCacheSize an OptionalInteger
* config: add and wire datastore.WriteThrough option
* config: omitempty on BlockKeyCacheSize
* changelog: rewrite entry about new options for the datastore
* config: add docs for BatchMaxNodes and BatchMaxSize
* config: make WriteThrough an optional Flag
* changelog: improve description of new datastore/import options
* refactor: DefaultWriteThrough as bool
* chore: boxo v0.26.0
* docs: config and changelog fixes
2024-12-19 23:12:05 +00:00
Hector Sanjuan
ecb25581c6
cmd/files: flush parent folders (#10630)
* cmd/files: flush parent folders

This is a mitigation to increased MFS memory usage in the course of many writes operations.

The underlying issue is the unbounded growth of the mfs directory cache in
boxo. In the latest boxo version, this cache can be cleared by calling Flush()
on the folder. In order to trigger that, we call Flush() on the parent folder
of the file/folder where the write-operations are happening.

To flushing the parent folder allows it to grow unbounded. Then, any read
operation to that folder or parents (i.e. stat), will trigger a sync-operation to match
the cache to the underlying unixfs structure (and obtain the correct node-cid).

This sync operation must visit every item in the cache. When the cache has grown too much,
and the underlying unixfs-folder has switched into a HAMT, the operation can take minutes.

Thus, we should clear the cache often and the Flush flag is a good indicator
that we can let it go. Users can always run with --flush=false and flush at
regular intervals during their MFS writes if they want to extract some performance.

Fixes #8694, #10588.

* cmd/files: docs and changelog for --flush changes
2024-12-19 14:07:17 -08:00
Andrew Gillis
7c49860061
Upgrade to Boxo v0.26.0 (#10631)
* Upgrade to Boxo v0.26.0
* Update docs/changelogs/v0.33.md
2024-12-19 13:54:02 -08:00
Andrew Gillis
335a27820d
Merge branch 'master' into update-nopfs 2024-12-13 02:16:57 -10:00
gammazero
09d5789946 mod tidy examples 2024-12-13 02:16:38 -10:00
Andrew Gillis
b42eb5a8f9
Update deps (#10624)
Some checks are pending
CodeQL / codeql (push) Waiting to run
Docker Build / docker-build (push) Waiting to run
Gateway Conformance / gateway-conformance (push) Waiting to run
Gateway Conformance / gateway-conformance-libp2p-experiment (push) Waiting to run
Go Build / go-build (push) Waiting to run
Go Check / go-check (push) Waiting to run
Go Lint / go-lint (push) Waiting to run
Go Test / go-test (push) Waiting to run
Interop / interop-prep (push) Waiting to run
Interop / helia-interop (push) Blocked by required conditions
Interop / ipfs-webui (push) Blocked by required conditions
Sharness / sharness-test (push) Waiting to run
2024-12-13 04:12:55 -08:00
Andrew Gillis
19dc712498
chore: boxo v0.25.0 (#10619)
Some checks are pending
CodeQL / codeql (push) Waiting to run
Docker Build / docker-build (push) Waiting to run
Gateway Conformance / gateway-conformance (push) Waiting to run
Gateway Conformance / gateway-conformance-libp2p-experiment (push) Waiting to run
Go Build / go-build (push) Waiting to run
Go Check / go-check (push) Waiting to run
Go Lint / go-lint (push) Waiting to run
Go Test / go-test (push) Waiting to run
Interop / interop-prep (push) Waiting to run
Interop / helia-interop (push) Blocked by required conditions
Interop / ipfs-webui (push) Blocked by required conditions
Sharness / sharness-test (push) Waiting to run
https://github.com/ipfs/boxo/releases/tag/v0.25.0
2024-12-09 23:25:44 +01:00
Hector Sanjuan
433444b609
fix(cmds/add): disallow --wrap with --to-files (#10612)
Some checks failed
CodeQL / codeql (push) Has been cancelled
Docker Build / docker-build (push) Has been cancelled
Gateway Conformance / gateway-conformance (push) Has been cancelled
Gateway Conformance / gateway-conformance-libp2p-experiment (push) Has been cancelled
Go Build / go-build (push) Has been cancelled
Go Check / go-check (push) Has been cancelled
Go Lint / go-lint (push) Has been cancelled
Go Test / go-test (push) Has been cancelled
Interop / interop-prep (push) Has been cancelled
Sharness / sharness-test (push) Has been cancelled
Interop / helia-interop (push) Has been cancelled
Interop / ipfs-webui (push) Has been cancelled
Close #10611
Co-authored-by: Marcin Rataj <lidel@lidel.org>
2024-12-03 21:49:18 +01:00
Andrew Gillis
224d6a3ba4
refactor(cmds): do not return errors embedded in result type (#10527)
incl. https://github.com/ipfs/boxo/pull/738
2024-12-03 20:15:33 +01:00
Marcin Rataj
53e793afdd
fix: ipfs-webui v4.4.1 (#10608)
https://github.com/ipfs/ipfs-webui/releases/tag/v4.4.1
2024-12-03 20:11:34 +01:00
Marten Seemann
778a418952
refactor(rcmgr): use default libp2p rcmgr metrics (#9947)
Some checks failed
CodeQL / codeql (push) Has been cancelled
Docker Build / docker-build (push) Has been cancelled
Gateway Conformance / gateway-conformance (push) Has been cancelled
Gateway Conformance / gateway-conformance-libp2p-experiment (push) Has been cancelled
Go Build / go-build (push) Has been cancelled
Go Check / go-check (push) Has been cancelled
Go Lint / go-lint (push) Has been cancelled
Go Test / go-test (push) Has been cancelled
Interop / interop-prep (push) Has been cancelled
Sharness / sharness-test (push) Has been cancelled
Interop / helia-interop (push) Has been cancelled
Interop / ipfs-webui (push) Has been cancelled
Co-authored-by: Marcin Rataj <lidel@lidel.org>
2024-11-27 20:50:43 +01:00
Hector Sanjuan
9433105a85
docs(changelog/v0.33): bitswap reprovide changes (#10604) 2024-11-27 19:40:16 +01:00
Marcin Rataj
3b2c2698e7
chore: p2p-forge/client v0.1.0 (#10605) 2024-11-27 19:28:11 +01:00
Marcin Rataj
23ef1d70e7
fix: go-libp2p v0.37.2 (#10603)
Some checks are pending
CodeQL / codeql (push) Waiting to run
Docker Build / docker-build (push) Waiting to run
Gateway Conformance / gateway-conformance (push) Waiting to run
Gateway Conformance / gateway-conformance-libp2p-experiment (push) Waiting to run
Go Build / go-build (push) Waiting to run
Go Check / go-check (push) Waiting to run
Go Lint / go-lint (push) Waiting to run
Go Test / go-test (push) Waiting to run
Interop / interop-prep (push) Waiting to run
Interop / helia-interop (push) Blocked by required conditions
Interop / ipfs-webui (push) Blocked by required conditions
Sharness / sharness-test (push) Waiting to run
* chore: go-libp2p v0.37.1

https://github.com/libp2p/go-libp2p/releases/tag/v0.37.1

* chore: go-libp2p v0.37.2

https://github.com/libp2p/go-libp2p/releases/tag/v0.37.2
2024-11-26 19:49:02 +01:00
Hector Sanjuan
466f72f518
docs: typos (#10602) 2024-11-26 19:19:48 +01:00
Hector Sanjuan
37c5060742
Update to boxo with refactored providerQueryManager. (#10595)
Some checks are pending
CodeQL / codeql (push) Waiting to run
Docker Build / docker-build (push) Waiting to run
Gateway Conformance / gateway-conformance (push) Waiting to run
Gateway Conformance / gateway-conformance-libp2p-experiment (push) Waiting to run
Go Build / go-build (push) Waiting to run
Go Check / go-check (push) Waiting to run
Go Lint / go-lint (push) Waiting to run
Go Test / go-test (push) Waiting to run
Interop / interop-prep (push) Waiting to run
Interop / helia-interop (push) Blocked by required conditions
Interop / ipfs-webui (push) Blocked by required conditions
Sharness / sharness-test (push) Waiting to run
2024-11-26 03:34:06 -08:00
Andrew Gillis
ef58568964
fix some typos in docs (#10598)
Some checks are pending
CodeQL / codeql (push) Waiting to run
Docker Build / docker-build (push) Waiting to run
Gateway Conformance / gateway-conformance (push) Waiting to run
Gateway Conformance / gateway-conformance-libp2p-experiment (push) Waiting to run
Go Build / go-build (push) Waiting to run
Go Check / go-check (push) Waiting to run
Go Lint / go-lint (push) Waiting to run
Go Test / go-test (push) Waiting to run
Interop / interop-prep (push) Waiting to run
Interop / helia-interop (push) Blocked by required conditions
Interop / ipfs-webui (push) Blocked by required conditions
Sharness / sharness-test (push) Waiting to run
2024-11-25 15:15:43 -08:00
Marcin Rataj
0bd0edc782
feat(bootstrap): add JS-based va1.bootstrap.libp2p.io (#10575)
Some checks failed
Docker Build / docker-build (push) Waiting to run
Gateway Conformance / gateway-conformance (push) Waiting to run
Gateway Conformance / gateway-conformance-libp2p-experiment (push) Waiting to run
Go Build / go-build (push) Waiting to run
Go Check / go-check (push) Waiting to run
Go Lint / go-lint (push) Waiting to run
Go Test / go-test (push) Waiting to run
Interop / interop-prep (push) Waiting to run
Interop / helia-interop (push) Blocked by required conditions
Interop / ipfs-webui (push) Blocked by required conditions
Sharness / sharness-test (push) Waiting to run
CodeQL / codeql (push) Has been cancelled
* feat(bootstrap): /dnsaddr/va1.bootstrap.libp2p.io

this adds
https://github.com/libp2p/js-libp2p-amino-dht-bootstrapper
instance at /dnsaddr/va1.bootstrap.libp2p.io
to ensure Kubo does not depend on single language stack
for initial bootstrap

* docs: document where defaults live

* test: updated bootstrappers
2024-11-25 09:04:53 -08:00
Bryer
d506003913
Typos Update config.md (#10591)
Some checks failed
CodeQL / codeql (push) Has been cancelled
Docker Build / docker-build (push) Has been cancelled
Gateway Conformance / gateway-conformance (push) Has been cancelled
Gateway Conformance / gateway-conformance-libp2p-experiment (push) Has been cancelled
Go Build / go-build (push) Has been cancelled
Go Check / go-check (push) Has been cancelled
Go Lint / go-lint (push) Has been cancelled
Go Test / go-test (push) Has been cancelled
Interop / interop-prep (push) Has been cancelled
Sharness / sharness-test (push) Has been cancelled
Interop / helia-interop (push) Has been cancelled
Interop / ipfs-webui (push) Has been cancelled
Fix Typos and Improve Clarity in Documentation
2024-11-19 09:40:25 -08:00
Andrew Gillis
3a1b8eed0f
refactor: update to boxo without goprocess (#10567)
* chore: update to boxo without goprocess
* Use boxo fix for registering metrics
* chore: switch to boxo main with PR 723

---------

Co-authored-by: Marcin Rataj <lidel@lidel.org>
2024-11-19 18:04:22 +01:00
Marcin Rataj
83b06f14d2
fix: go-libp2p-kad-dht v0.28.1 (#10581)
Some checks are pending
CodeQL / codeql (push) Waiting to run
Docker Build / docker-build (push) Waiting to run
Gateway Conformance / gateway-conformance (push) Waiting to run
Gateway Conformance / gateway-conformance-libp2p-experiment (push) Waiting to run
Go Build / go-build (push) Waiting to run
Go Check / go-check (push) Waiting to run
Go Lint / go-lint (push) Waiting to run
Go Test / go-test (push) Waiting to run
Interop / interop-prep (push) Waiting to run
Interop / helia-interop (push) Blocked by required conditions
Interop / ipfs-webui (push) Blocked by required conditions
Sharness / sharness-test (push) Waiting to run
https://github.com/libp2p/go-libp2p-kad-dht/releases/tag/v0.28.1
2024-11-15 17:50:18 +01:00
Marcin Rataj
720663d7c8
docs: update RELEASE_CHECKLIST.md (#10564)
Some checks failed
CodeQL / codeql (push) Has been cancelled
Docker Build / docker-build (push) Has been cancelled
Gateway Conformance / gateway-conformance (push) Has been cancelled
Gateway Conformance / gateway-conformance-libp2p-experiment (push) Has been cancelled
Go Build / go-build (push) Has been cancelled
Go Check / go-check (push) Has been cancelled
Go Lint / go-lint (push) Has been cancelled
Go Test / go-test (push) Has been cancelled
Interop / interop-prep (push) Has been cancelled
Sharness / sharness-test (push) Has been cancelled
Interop / helia-interop (push) Has been cancelled
Interop / ipfs-webui (push) Has been cancelled
Based on 0.32 iteration
2024-11-14 04:04:18 +01:00
Marcin Rataj
7ca106353e
chore: changelog for 0.33 2024-11-14 03:17:31 +01:00
Marcin Rataj
970fcb58df
chore: 0.32.0 2024-11-14 00:57:03 +01:00
Marcin Rataj
136ed3ca6d fix: go-libp2p-kad-dht v0.28.0 (#10578)
https://github.com/libp2p/go-libp2p-kad-dht/releases/tag/v0.28.0
(cherry picked from commit e80e821cf8)
2024-11-13 23:58:22 +01:00
Marcin Rataj
4f06b6a005
chore: 0.32.0-rc2 2024-11-08 00:15:49 +01:00
Marcin Rataj
b87d512b31 feat: ipfs-webui v4.4.0 (#10574)
https://github.com/ipfs/ipfs-webui/releases/tag/v4.4.0
(cherry picked from commit 5d2f5369c7)
2024-11-07 22:13:29 +01:00
Marcin Rataj
5d88447d27 chore: boxo v0.24.3 and p2p-forge v0.0.2 (#10572)
https://github.com/ipfs/boxo/releases/tag/v0.24.3
https://github.com/ipshipyard/p2p-forge/releases/tag/v0.0.2
(cherry picked from commit 1512ec5fa5)
2024-11-07 03:34:57 +01:00
Andrew Gillis
c53f3c552d chore: stop using go-homedir (#10568)
* chore: stop using go-homedir

The `github.com/mitchellh/go-homedir` repo is archived, no longer needed, and no longer maintained.

- `homedir.Dir` is replaced by the stdlib `os.UserHomeDir`
- `homedir.Expand` is replaced by fsutil.ExpandHome` in the `github.com/ipfs/kubo/misc/fsutil` package.

Additional functionality, such as `DirWritable` and `FileExists` was moved into or included in the `github.com/ipfs/kubo/misc/fsutil` package.

(cherry picked from commit 4009ad3e5a)
2024-11-05 17:02:55 +01:00
Adin Schmahmann
468eebaeac chore: update deps (#10569)
(cherry picked from commit d4ae7fa812)
2024-11-04 20:25:22 +01:00
Adin Schmahmann
84345afcde fix(autotls): store certificates at the location from the repo path (#10566)
* fix(autotls): store certificates at the location from the repo path
* docs(autotls): cert storale and other caveats

---------

Co-authored-by: Marcin Rataj <lidel@lidel.org>
(cherry picked from commit 1ca0ae0af6)
2024-10-30 23:24:20 +01:00