This adds a plugin interface that lets the plugin modify the fx
options that are passed to fx when the app is initialized. This means
plugins can inject their own implementations of IPFS interfaces. This
enables granular customization of go-ipfs behavior by plugins, such
as:
- Bitswap with custom filters (e.g. for CID blocking) Custom interface
- implementations such as Pinner or DAGService
- Dynamic configuration of libp2p ...
One downside of this is that we're exposing the entire dependency
graph, init hooks, initialization, etc. to users, so this comes with a
caveat that we reserve the right to make breaking changes to the graph
structure and initialization logic (although this historically happens
rarely). If these things are changed, we should mention them in
release notes and changelogs though, since they could impact users of
this plugin interface.
I'm not particularly fond of DI frameworks (and neither are some of
the folks work on/near go-ipfs), but it seems unlikely that somebody
will rewrite the dependency wiring and lifecycle hooks of go-ipfs, and
add dynamic extension points, so this seems like a palatable
compromise.
There are also problems that we should clean up in how model the
go-ipfs app in fx, such as:
- We make extensive use of nested fx.Options, which fx itself
discourages because it "limits the user's ability to customize their
application". It should be easy to flatten these out into a single
[]fx.Option slice.
- We pass around a list of opaque libp2p opts, which makes it hard to
customize after-the-fact...we should consider naming each of these
opts and providing them to fx as proper dependencies, so that they can
be explicitly overridden.
- We call fx.Invoke() in some places with anonymous functions. We
should instead only pass exported functions to fx.Invoke(), so that
they have exported names, which would make it easier to remove/augment
the invocations that happen when the app is initialized.
These aren't blocking issues, they just make it harder and more
brittle to customize go-ipfs with this plugin.
This fixes a regression around directory listing and index.html hosting.
Seems that during one of recent refactors code changed and we no longer
check for trailing slash in HTTP request path, but look at content path
instead.
This cleans this up and also ensures dir behavior is the same for
both index.html hosting and dir-index-html (generated listing).
It also adds more tests so we catch any future regressions.
This fixes a regression introduced in 0.13.0, where websites hosted via
index.html placed in UnixFS directory were always returned with
Cache-Control: public, max-age=29030400, immutable
even when loaded from mutable /ipns/ contentPath.
* Delegated Routing.
Implementation of Reframe specs (https://github.com/ipfs/specs/blob/master/REFRAME.md) using go-delegated-routing library.
* Requested changes.
* Init using op string
* Separate possible ContentRouters for TopicDiscovery.
If we don't do this, we have a ciclic dependency creating TieredRouter.
Now we can create first all possible content routers, and after that,
create Routers.
* Set dht default routing type
* Add tests and remove uneeded code
* Add documentation.
* docs: Routing.Routers
* Requested changes.
Signed-off-by: Antonio Navarro Perez <antnavper@gmail.com>
* Add some documentation on new fx functions.
* Add changelog entry and integration tests
* test: sharness for 'dht' in 'routing' commands
Since 'routing' is currently the same as 'dht' (minus query command)
we need to test both, that way we won't have unnoticed divergence
in the default behavior.
* test(sharness): delegated routing via reframe URL
* Add more tests for delegated routing.
* If any put operation fails, the tiered router will fail.
* refactor: Routing.Routers: Parameters.Endpoint
As agreed in https://github.com/ipfs/kubo/pull/8997#issuecomment-1175684716
* Try to improve CHANGELOG entry.
* chore: update reframe spec link
* Update go-delegated-routing dependency
* Fix config error test
* use new changelog format
* Remove port conflict
* go mod tidy
* ProviderManyWrapper to ProviderMany
* Update docs/changelogs/v0.14.md
Co-authored-by: Adin Schmahmann <adin.schmahmann@gmail.com>
Co-authored-by: Marcin Rataj <lidel@lidel.org>
Co-authored-by: Adin Schmahmann <adin.schmahmann@gmail.com>
* fix: remove mdns_legacy
We've been running both implementations for a long, long time.
It is time to remove legacy version and lower the number of LAN packets
IPFS node produces.
See https://github.com/ipfs/go-ipfs/pull/9048#discussion_r906814717
for the Interval removal rational.
* feat: upgrade to go-libp2p-kad-dht@v0.16.0
* rename "cid format --codec" to "cid format --mc"
* refactor(test): from --codec to --mc
Co-authored-by: Marcin Rataj <lidel@lidel.org>
* feat: disable resource manager by default
We are disabling this by default for v0.13 as we work to improve the
UX around Resource Manager. It is still usable and can be enabled in
the IPFS config with "ipfs config --bool Swarm.ResourceMgr.Enabled true".
We intend to enable Resource Manager by default in a subsequent
release.
* docs(config): Swarm.ResourceMgr disabled by default
Co-authored-by: Marcin Rataj <lidel@lidel.org>
* fix: adjust rcmgr limits for accelerated DHT client rt refresh
The Accelerated DHT client periodically refreshes its routing table,
including at startup, and if Resource Manager throttling causes the
client's routing table to be incomplete, then content routing may be
degraded or broken for users.
This adjusts the default limits to a level that empirically doesn't
cause Resource Manager throttling during initial DHT client
bootstrapping. Ideally the Accelerated DHT client would handle this
scenario more gracefully, but this works for now to unblock the 0.13
release.
* Set default outbound conns unconditionally
This also sets the default overall conns as a function of the outbound
and inbound conns, since they are adjusted dynamically, and it makes
the intention of the value clear.
* increase min FD limit
This periodically logs how many times Resource Manager limits were
exceeded. If they aren't exceeded, then nothing is logged. The log
levels are at ERROR log level so that they are shown by default.
The motivation is so that users know when they have exceeded resource
manager limits. To find what is exceeding the limits, they'll need to
turn on debug logging and inspect the errors being logged. This could
collect the specific limits being reached, but that's more complicated
to implement and could result in much longer log messages.
This fix safelists additional headers allowing JS running on websites to
read them when IPFS resource is downloaded via Fetch API.
These headers provide metadata necessary for making smart caching
decisions when IPFS resources are downloaded via Service Worker or a
similar middleware on the edge.
Fixes#8957
The context was only checked while reading data.
Not while writing data to the http connection.
So since the data flow through an io.Pipe the closing didn't flowed through and left the writer open hanging.
Co-authored-by: Antonio Navarro Perez <antnavper@gmail.com>
* Add 'ipfs repo migrate' command
This PR replaces #7658 that was originally contributed by zaibons, in order to move code into a branch and avoid some CI problem.
The command allows the user to run the repo migration without starting the daemon.
resolves#7471
* return non-ErrNeedMigration errors from fsrepo.Open()
Co-authored-by: Gus Eggert <gus@gus.dev>
* pubsub multibase encoding
Adds clarification for pubsub multibase encoding over HTTP RPC for issue https://github.com/ipfs/ipfs-docs/issues/1007
* Grammatical change
* Moved period
