* include pebble as built-in plugin
Pebble provides a high-performance alternative to leveldb as the datastore, and will serve as a replacement for badger1.
There are a number of tuning parameters available for tuning pebble's performance to your specific needs. Default values are used for any that are not configured or are set to the parameter's zero-value.
Requires https://github.com/ipfs/go-ds-pebble/pull/39Closes#10347
* docs: remove mention of ipfs-ds-convert. Rationale: https://github.com/ipfs/ipfs-ds-convert/issues/50
* docs: pebbleds profile
* test: meaningful t0025-datastores.sh
* Update config/init.go
* Update docs/config.md
* Do not hard-code zero values into pebble config
Allow configuration of the bitswap server's replace WantHave with WantBlock maximum block size using the Internal.Bitswap.WantHaveReplaceSize config item. This sets the maximum size of a block in bytes up to which we will replace a want-have with a want-block. Setting a size of 0 disables this replacement and means that block sizes are not read for WantHave requests.
See ipfs/boxo#672 for more details
Updated boxo to version with PR 672
---------
Co-authored-by: Marcin Rataj <lidel@lidel.org>
Updates: #9396Closes: #6831Closes: #6208
Currently the Graphsync server is not widely used due to lack of compatible software.
There have been many years yet we are unable to find any production software making use of the graphsync server in Kubo.
There exists some in the filecoin ecosystem but we are not aware of uses with Kubo.
Even in filecoin graphsync is not the only datatransfer solution available like it could have been in the past.
`go-graphsync` is also developped on many concurrent branches.
The specification for graphsync are less clear than the trustless gateway one and lack a complete conformance test suite any implementation can run.
It is not easily extansible either because selectors are too limited for interesting queries without sideloading ADLs, which for now are hardcoded solutions.
Finaly Kubo is consistently one of the fastest software to update to a new go-libp2p release.
This means the burden to track go-libp2p changes in go-graphsync falls on us, else Kubo cannot compile even if almost all users do not use this feature.
We are then removing the graphsync server experiment.
For people who want alternatives we would like you to try the Trustless-Gateway-over-Libp2p experiment instead, the protocol is simpler (request-response-based) and let us reuse both clients and servers with minimal injection in the network layer.
If you think this is a mistake and we should put it back you should try to answer theses points:
- Find a piece of opensource code which uses a graphsync client to download data from Kubo.
- Why is Trustless-Gateway-over-Libp2p not suitable instead ?
- Why is bitswap not suitable instead ?
Implementation details such as go-graphsync performance vs boxo/gateway is not very interesting to us in this discussion unless they are really huge (in the range of 10x~100x+ more) because the gateway code is under high development and we would be interested in fixing theses.
Fixes#8492
This introduces "nopfs" as a preloaded plugin into Kubo
with support for denylists from https://github.com/ipfs/specs/pull/383
It automatically makes Kubo watch *.deny files found in:
- /etc/ipfs/denylists
- $XDG_CONFIG_HOME/ipfs/denylists
- $IPFS_PATH/denylists
* test: Gateway.NoFetch and GatewayOverLibp2p
adds missing tests for "no fetch" gateways one can expose,
in both cases the offline mode is done by passing custom
blockservice/exchange into path resolver, which means
global path resolver that has nopfs intercept is not used,
and the content blocking does not happen on these gateways.
* fix: use offline path resolvers where appropriate
this fixes the problem described in
https://github.com/ipfs/kubo/pull/10161#issuecomment-1782175955
by adding explicit offline path resolvers that are backed
by offline exchange, and using them in NoFetch gateways
instead of the default online ones
---------
Co-authored-by: Henrique Dias <hacdias@gmail.com>
Co-authored-by: Marcin Rataj <lidel@lidel.org>
I only updated otel to 1.17.0 since we need to handle breaking changes with newer releases (it doesn't build).
I also didn't update go-multistream since it's touched by libp2p and break stuff.
- Bumps golangci-lint to work for the new Go version
- Removes rand.Seed, which has been deprecated. It is seeded by default
with a random value since Go 1.20.
- Replaces deprecated Fuse errors with syscall.Errno
This is a Q&D fix to get kubo building again, this is completely equivalent to what we used to have.
I'll submit a patch that DI the verifcid.Allowlist interface with fx later but this is needed to unblock something else.
Mplex does not implement backpressure, our implementation will randomly reset streams if buffers overflow instead of risking deadlocks.
In the past we had a bug where kubo nodes would prefer mplex over yamux. Turning off mplex make our connections to thoses nodes negociate yamux.
Closes#9958
* fix: mark ipns pubsub router DoNotWaitForSearchValue
That means if the DHT has finished searching and no one responded over pubsub *yet*, we will not spend 1 minute searching for no reason.
This also include other error handling bug fixes inside `go-libp2p-routing-helpers`.
Fixes: #9927
* routing: bring back the old IPNS behaviour
Stop making this configurable let everything race like it used to do.
Streams used to be blocked on ping IO because we didn't handled the DHT ping check asynchronously.
Include fixes from libp2p/go-libp2p-kad-dht#851
Fixes#9957
