Commit Graph

1169 Commits

Author SHA1 Message Date
Andrew Gillis
8077dbea61
use go-datastore without go-process (#10736)
* use go-datastore without go-process
* update go-ds-xxx dependencies
* update go-libp2p-kad-dht
* bitswap api changes
* Do not use multiple multi-error packages, pick one
* update boxo
* update expected metrics
2025-03-04 18:52:26 -08:00
Marco Munizaga
baa94fcb2f
chore: update go-libp2p to v0.41.0 (#10733)
Some checks failed
CodeQL / codeql (push) Has been cancelled
Docker Build / docker-build (push) Has been cancelled
Gateway Conformance / gateway-conformance (push) Has been cancelled
Gateway Conformance / gateway-conformance-libp2p-experiment (push) Has been cancelled
Go Build / go-build (push) Has been cancelled
Go Check / go-check (push) Has been cancelled
Go Lint / go-lint (push) Has been cancelled
Go Test / go-test (push) Has been cancelled
Interop / interop-prep (push) Has been cancelled
Sharness / sharness-test (push) Has been cancelled
Spell Check / spellcheck (push) Has been cancelled
Interop / helia-interop (push) Has been cancelled
Interop / ipfs-webui (push) Has been cancelled
https://github.com/libp2p/go-libp2p/releases/tag/v0.41.0

Co-authored-by: Marcin Rataj <lidel@lidel.org>
2025-02-25 23:50:45 +01:00
Marcin Rataj
d6a5bc0b68
feat(AutoTLS): enabled by default with 1h RegistrationDelay (#10724)
Enables AutoTLS by default, but delays registration by 1h
if user did not explicitly set `AutoTLS.Enabled` to `true`
2025-02-25 22:25:58 +01:00
Russell Dempsey
40a7a388a4
fix: switch away from IPFS_LOG_LEVEL (#10694)
Some checks failed
CodeQL / codeql (push) Has been cancelled
Docker Build / docker-build (push) Has been cancelled
Gateway Conformance / gateway-conformance (push) Has been cancelled
Gateway Conformance / gateway-conformance-libp2p-experiment (push) Has been cancelled
Go Build / go-build (push) Has been cancelled
Go Check / go-check (push) Has been cancelled
Go Lint / go-lint (push) Has been cancelled
Go Test / go-test (push) Has been cancelled
Interop / interop-prep (push) Has been cancelled
Sharness / sharness-test (push) Has been cancelled
Spell Check / spellcheck (push) Has been cancelled
Interop / helia-interop (push) Has been cancelled
Interop / ipfs-webui (push) Has been cancelled
2025-02-14 18:25:31 +00:00
Marcin Rataj
3b098b969a
fix(cli): support HTTPS in ipfs --api (#10659)
Some checks failed
CodeQL / codeql (push) Has been cancelled
Docker Build / docker-build (push) Has been cancelled
Gateway Conformance / gateway-conformance (push) Has been cancelled
Gateway Conformance / gateway-conformance-libp2p-experiment (push) Has been cancelled
Go Build / go-build (push) Has been cancelled
Go Check / go-check (push) Has been cancelled
Go Lint / go-lint (push) Has been cancelled
Go Test / go-test (push) Has been cancelled
Interop / interop-prep (push) Has been cancelled
Sharness / sharness-test (push) Has been cancelled
Interop / helia-interop (push) Has been cancelled
Interop / ipfs-webui (push) Has been cancelled
* fix(cli): support HTTPS in ipfs --api

Closes #10539

* chore: go-ipfs-cmds v0.14.1

https://github.com/ipfs/go-ipfs-cmds/releases/tag/v0.14.1

* docs: ipfs --api example

* test(cli): https rpc support

makes sure we dont have regression where HTTPS endpoint
starts getting cleartext requests
2025-01-10 22:34:56 +01:00
Adin Schmahmann
397c346ae0
feat(libp2p): shared TCP listeners and AutoTLS.AutoWSS (#10565)
Some checks are pending
CodeQL / codeql (push) Waiting to run
Docker Build / docker-build (push) Waiting to run
Gateway Conformance / gateway-conformance (push) Waiting to run
Gateway Conformance / gateway-conformance-libp2p-experiment (push) Waiting to run
Go Build / go-build (push) Waiting to run
Go Check / go-check (push) Waiting to run
Go Lint / go-lint (push) Waiting to run
Go Test / go-test (push) Waiting to run
Interop / interop-prep (push) Waiting to run
Interop / helia-interop (push) Blocked by required conditions
Interop / ipfs-webui (push) Blocked by required conditions
Sharness / sharness-test (push) Waiting to run
* feat(libp2p): enable shared TCP listeners

* docs: switch mentions of /ws to /tcp/4001

* feat: AutoTLS.AutoWSS

This adds AutoTLS.AutoWSS flag that is set to true by default.

It will check if Addresses.Swarm contain explicit /ws listener,
and if not found, it will append one per every /tcp listener

This way existing TCP ports are reused without any extra configuration,
but we don't break user's who have custom / explicit /ws listener
already.

I also moved logger around, to include Addresses.Swarm inspection
results in `autotls` logger.

* chore: go-libp2p v0.38.1

https://github.com/libp2p/go-libp2p/releases/tag/v0.38.0
https://github.com/libp2p/go-libp2p/releases/tag/v0.38.1

* docs: AutoTLS.AutoWSS and go-libp2p v0.38.x

* chore: p2p-forge/client v0.2.0

https://github.com/ipshipyard/p2p-forge/releases/tag/v0.2.0

* fix: disable libp2p.ShareTCPListener() in PNET

* chore(ci): timeout sharness after 15m

average successful run is  <9 minutes, no need to wait for 20
https://github.com/ipfs/kubo/actions/workflows/sharness.yml?query=is%3Asuccess

---------

Co-authored-by: Andrew Gillis <11790789+gammazero@users.noreply.github.com>
Co-authored-by: Marcin Rataj <lidel@lidel.org>
2024-12-20 18:41:25 +01:00
Hector Sanjuan
898f024f3c
[skip changelog] pinmfs: mitigate slow mfs writes when it triggers (#10623)
Some checks failed
CodeQL / codeql (push) Has been cancelled
Docker Build / docker-build (push) Has been cancelled
Gateway Conformance / gateway-conformance (push) Has been cancelled
Gateway Conformance / gateway-conformance-libp2p-experiment (push) Has been cancelled
Go Build / go-build (push) Has been cancelled
Go Check / go-check (push) Has been cancelled
Go Lint / go-lint (push) Has been cancelled
Go Test / go-test (push) Has been cancelled
Interop / interop-prep (push) Has been cancelled
Sharness / sharness-test (push) Has been cancelled
Interop / helia-interop (push) Has been cancelled
Interop / ipfs-webui (push) Has been cancelled
* pinmfs: mitigate slow mfs writes when it triggers

This mitigates slow mfs writes when the pinmfs daemon calls mfs.RootNode()

When writing lots of files to MFS, this call triggers a mfs directory cache
sync operations. The cache grows forever and is unbounded. The more files
added to a directory, the longer it takes. In the meantime, writing to mfs is
locked.

The pinmfs, even when no remote pinning services are configured, will trigger
this issue. When RootNode() takes more than 30 seconds, the issue will be triggered
continuously causing a write-deadlock onto MFS.

This commit does not fix the fact that if you write 1M items into an MFS
directory, the first time that the directory is traversed it will still have
to sync those 1M items. It does at least prevent writes from stalling after
about ~6000 items.

* pinmfs: fix test
2024-12-17 12:59:12 -08:00
Andi
642e58ef3b
chore: use errors.New to replace fmt.Errorf with no parameters (#10617)
Some checks failed
CodeQL / codeql (push) Has been cancelled
Docker Build / docker-build (push) Has been cancelled
Gateway Conformance / gateway-conformance (push) Has been cancelled
Gateway Conformance / gateway-conformance-libp2p-experiment (push) Has been cancelled
Go Build / go-build (push) Has been cancelled
Go Check / go-check (push) Has been cancelled
Go Lint / go-lint (push) Has been cancelled
Go Test / go-test (push) Has been cancelled
Interop / interop-prep (push) Has been cancelled
Sharness / sharness-test (push) Has been cancelled
Interop / helia-interop (push) Has been cancelled
Interop / ipfs-webui (push) Has been cancelled
2024-12-10 13:50:35 -08:00
Andrew Gillis
224d6a3ba4
refactor(cmds): do not return errors embedded in result type (#10527)
incl. https://github.com/ipfs/boxo/pull/738
2024-12-03 20:15:33 +01:00
Andrew Gillis
3a1b8eed0f
refactor: update to boxo without goprocess (#10567)
* chore: update to boxo without goprocess
* Use boxo fix for registering metrics
* chore: switch to boxo main with PR 723

---------

Co-authored-by: Marcin Rataj <lidel@lidel.org>
2024-11-19 18:04:22 +01:00
Andrew Gillis
4009ad3e5a
chore: stop using go-homedir (#10568)
Some checks failed
CodeQL / codeql (push) Has been cancelled
Docker Build / docker-build (push) Has been cancelled
Gateway Conformance / gateway-conformance (push) Has been cancelled
Gateway Conformance / gateway-conformance-libp2p-experiment (push) Has been cancelled
Go Build / go-build (push) Has been cancelled
Go Check / go-check (push) Has been cancelled
Go Lint / go-lint (push) Has been cancelled
Go Test / go-test (push) Has been cancelled
Interop / interop-prep (push) Has been cancelled
Sharness / sharness-test (push) Has been cancelled
Interop / helia-interop (push) Has been cancelled
Interop / ipfs-webui (push) Has been cancelled
* chore: stop using go-homedir

The `github.com/mitchellh/go-homedir` repo is archived, no longer needed, and no longer maintained.

- `homedir.Dir` is replaced by the stdlib `os.UserHomeDir`
- `homedir.Expand` is replaced by fsutil.ExpandHome` in the `github.com/ipfs/kubo/misc/fsutil` package.

Additional functionality, such as `DirWritable` and `FileExists` was moved into or included in the `github.com/ipfs/kubo/misc/fsutil` package.
2024-11-05 07:45:11 -08:00
Marcin Rataj
6305932b4e
fix(daemon): webui URL when rpc is catch-all (#10520)
Closes #10515
2024-10-04 00:51:45 +02:00
Marcin Rataj
a8ecf014a9
feat: explicit announce-on/off profiles (#10524)
moving reprovide on/off to separate profile to avoid footgun
where node no longer announces to DHT + ipfs daemon check
that prints warning on start if reprovide system is disabled
2024-10-03 21:39:52 +02:00
Andrew Gillis
ca4f486781
refactor: simplify logic for MFS pinning (#10506) 2024-09-27 15:37:47 +02:00
Andrew Gillis
5fe960474c
fix(cli): preserve hostname specified with --api in http request headers (#10497)
Preserve hostname specified with --api in http request headers

- Replaces PR #10233
- Add test to check for hostname in HTTP header
- Update docs/changelogs/v0.30.md
2024-08-28 10:03:26 -07:00
Marcin Rataj
fdfd2bc4a1
docs: document rpc over unix socket 2024-08-20 17:04:20 +02:00
Marcin Rataj
34008b6ffa
chore(daemon): sort listeners (#10480) 2024-08-19 14:23:14 +02:00
Marcin Rataj
0d428310b2
feat(daemon): improve stdout on startup (#10472) 2024-08-14 16:42:24 +02:00
Marcin Rataj
a339e6e807
fix(daemon): panic in kubo/daemon.go:595 (#10473) 2024-08-12 23:00:56 +02:00
Patryk
225dbe6c03
feat: periodic version check and json config (#10438)
Co-authored-by: Lucas Molas <schomatis@gmail.com>
Co-authored-by: Marcin Rataj <lidel@lidel.org>
2024-07-24 23:42:19 +02:00
Marcin Rataj
f08d585a5a
chore: update deps incl. boxo v0.21.0 (#10444)
* chore: update dependencies
* refactor: go lint
* chore: boxo v0.21.0

https://github.com/ipfs/boxo/releases/tag/v0.21.0
2024-06-21 02:57:36 +02:00
Henrique Dias
cd78f2eae3
chore(config): make Routing.AcceleratedDHTClient a Flag (#10384) 2024-04-04 13:59:31 +02:00
Marco Munizaga
9a5f5e7352
chore: cleanup old workaround (#10369) 2024-03-19 12:26:20 +01:00
Marcin Rataj
5af3cc1c02
docs: updated links and refs to external resources (#10368) 2024-03-16 00:59:22 +01:00
Henrique Dias
e22f47ae4b
core/corehttp!: remove /api/v0 from gateway port 2024-03-05 08:21:13 +00:00
Henrique Dias
e166af9756
core/corehttp: wrap gateway with headers, deprecate gateway /api/v0 2024-01-24 09:33:23 +00:00
Henrique Dias
d1db95c447 config: remove all options that are marked as REMOVED
Most of the removed options are many years old. In addition, they've all been removed in past iterations of Kubo. Some options were marked as removed in the config.md, but we still had a warning in the code to let users know they have been removed.

I think it's been long enough for all of this options, and enough Kubo iterations in order to alert the users. It is good to keep it in the config.md for now so that people can actually check. However, I think it's time to remove them from the code itself.
2024-01-22 15:33:27 +01:00
Henrique Dias
a8a6bbe929
feat: support optional pin names (#10261) 2024-01-04 14:25:06 +01:00
Adin Schmahmann
e503c843b8 feat: unexport unneeded functions and add comments 2023-12-19 19:52:49 -05:00
Steven Allen
0e83c3f2f9 remove old main function 2023-12-19 16:23:58 -05:00
Łukasz Magiera
a8a12789f2 feat: cmd/ipfs: Nicer to use BuildEnv 2023-12-19 16:23:58 -05:00
Łukasz Magiera
b5dddf67a2 feat: cmd/ipfs: Make it possible to depend on cmd/ipfs 2023-12-19 16:23:53 -05:00
Łukasz Magiera
86b73f61fc feat: cmd/ipfs: Allow passing custom BuildEnv to main 2023-12-19 15:36:23 -05:00
Jorropo
8c4bdd8556
fix: allow daemon to start correctly if the API is null (#10062) 2023-12-11 11:45:08 +01:00
Henrique Dias
58c29399cf
chore: clean migration 2023-11-29 12:29:59 +01:00
Henrique Dias
01cc5eab57
feat(rpc): Opt-in HTTP RPC API Authorization (#10218)
Context: https://github.com/ipfs/kubo/issues/10187
Co-authored-by: Marcin Rataj <lidel@lidel.org>
2023-11-17 01:29:29 +01:00
Marcin Rataj
0770702289 docs: clarify ipfs id agent version 2023-11-15 16:14:16 +01:00
Henrique Dias
a7c6518497
feat: path consolidation (#10063) 2023-10-06 16:14:44 +02:00
Adin Schmahmann
f2c43d5bbf feat(config): Add gateway-over-libp2p experiment 2023-09-01 16:52:09 -04:00
Adin Schmahmann
bf548a3cc6 fix(gateway): close http-over-libp2p host when the node is ready to shutdown 2023-08-31 04:54:39 -04:00
Adin Schmahmann
bc6bee1585 rename Libp2pGatewayOption and hard code its gateway configuration 2023-08-31 03:51:48 -04:00
Adin Schmahmann
9d32f71e22 move the gateway-over-libp2p mountpoint to the root 2023-08-31 03:49:35 -04:00
Adin Schmahmann
5b0da08333 fix: use /ipfs/gateway as the protocol ID for serving the gateway over libp2p 2023-08-31 03:48:23 -04:00
Jorropo
f7aa1204b1 feat: add gateway to http over libp2p 2023-08-30 18:51:22 +02:00
Henrique Dias
4606586207
feat(gateway): expose /routing/v1 server (opt-in) (#9877) 2023-08-25 17:30:04 +02:00
Kay
2b7c20fc66
refactor: using error is instead of == (#10093) 2023-08-22 14:53:29 +00:00
Henrique Dias
1e5ce93659 fix: use %-encoded headers in most compatible way 2023-08-22 15:43:01 +02:00
Henrique Dias
5e9bad9804 fix: open /dev/null with read write permissions
The way we create the kubo binary for coverage is very hacky.
It uses the testing tool. In order to simulate a Kubo binary,
we need to supress all the output that would otherwise be printed
by 'go test'.

So far, we were setting os.Stdout and os.Stderr as a read-only
/dev/null file descriptor. This is causing issues with the new
versions of Go:

    error generating coverage report: write /dev/null: bad file descriptor
    exit status 2

Updating it to a Read-Write file descriptor solves the problem.
I did not try looking into what is causing this issue now. There have
been some updates to the 'go test' tool in Go 1.20 and it is likely
that some error is now being checked for that hasn't been checked
before. Writing to a read-only file descriptor always failed. But
the error was just supressed somehow.
2023-08-22 15:43:01 +02:00
Henrique Dias
2aa721b121 chore: bump to go 1.20
- Bumps golangci-lint to work for the new Go version
- Removes rand.Seed, which has been deprecated. It is seeded by default
  with a random value since Go 1.20.
- Replaces deprecated Fuse errors with syscall.Errno
2023-08-22 15:43:01 +02:00
Kay
f12b372af9
style: gofumpt and godot [skip changelog] (#10081) 2023-08-17 14:02:08 +02:00