This also means that rb-pinning-service-api is no longer required for
running remote pinning tests. This alone saves at least 3 minutes in
test runtime in CI because we don't need to checkout the repo, build
the Docker image, run it, etc.
Instead this implements a simple pinning service in Go that the test
runs in-process, with a callback that can be used to control the async
behavior of the pinning service (e.g. simulate work happening
asynchronously like transitioning from "queued" -> "pinning" ->
"pinned").
This also adds an environment variable to Kubo to control the MFS
remote pin polling interval, so that we don't have to wait 30 seconds
in the test for MFS changes to be repinned. This is purely for tests
so I don't think we should document this.
This entire test suite runs in around 2.5 sec on my laptop, compared to
the existing 3+ minutes in CI.
New multi-router configuration system based on https://hackmd.io/G1KRDEX5T3qyfoBMkIrBew#Methods
- Added a new routing type: "custom"
- Added specific struct types for different Routers (instead of map[string]interface{})
- Added `Duration` config type, to make easier time string parsing
- Added config documentation.
- Use the latest go-delegated-routing library version with GET support.
- Added changelog notes for this feature.
It:
- closes#9157
- closes#9079
- closes#9186
The file contains the gateway your node is hosting in the http://<host>:<port> RFC 3986 format.
Structurally it works exactly the same as the API file.
* Delegated Routing.
Implementation of Reframe specs (https://github.com/ipfs/specs/blob/master/REFRAME.md) using go-delegated-routing library.
* Requested changes.
* Init using op string
* Separate possible ContentRouters for TopicDiscovery.
If we don't do this, we have a ciclic dependency creating TieredRouter.
Now we can create first all possible content routers, and after that,
create Routers.
* Set dht default routing type
* Add tests and remove uneeded code
* Add documentation.
* docs: Routing.Routers
* Requested changes.
Signed-off-by: Antonio Navarro Perez <antnavper@gmail.com>
* Add some documentation on new fx functions.
* Add changelog entry and integration tests
* test: sharness for 'dht' in 'routing' commands
Since 'routing' is currently the same as 'dht' (minus query command)
we need to test both, that way we won't have unnoticed divergence
in the default behavior.
* test(sharness): delegated routing via reframe URL
* Add more tests for delegated routing.
* If any put operation fails, the tiered router will fail.
* refactor: Routing.Routers: Parameters.Endpoint
As agreed in https://github.com/ipfs/kubo/pull/8997#issuecomment-1175684716
* Try to improve CHANGELOG entry.
* chore: update reframe spec link
* Update go-delegated-routing dependency
* Fix config error test
* use new changelog format
* Remove port conflict
* go mod tidy
* ProviderManyWrapper to ProviderMany
* Update docs/changelogs/v0.14.md
Co-authored-by: Adin Schmahmann <adin.schmahmann@gmail.com>
Co-authored-by: Marcin Rataj <lidel@lidel.org>
Co-authored-by: Adin Schmahmann <adin.schmahmann@gmail.com>
* feat: add block profiling to collect-profiles.sh
* feat: add more profiles to 'ipfs diag profile'
This adds mutex and block profiles, and brings the command up-to-par
with 'collect-profiles.sh', so that we can remove it.
Profiles are also now collected concurrently, which improves the
runtime from (profile_time * num_profiles) to just (profile_time).
Note that this has a backwards-incompatible change, removing
--cpu-profile-time in favor of the more general --profile-time, which
covers all sampling profiles.
* docs(cli): ipfs diag profile
* add CLI flag to select specific diag collectors
Co-authored-by: Marcin Rataj <lidel@lidel.org>
* add deprecation warning when tracer plugins are loaded
* add response format attribute to span in gateway handler
* add note about tracing's experimental status in godoc
* add nil check for TTL when adding name span attrs
* add basic sharness test for integration with otel collector
* add nil check in UnixFSAPI.processLink
* test: sharness check all json objs for swarm span
* add env var docs to docs/environment-variables.md
* chore: pin the otel collector version
* add tracing spans per response type (#8841)
* docs: tracing with jaeger-ui
Co-authored-by: Marcin Rataj <lidel@lidel.org>
* fix: installation without sudo
I think it's good practice that users don't need to run `sudo`
to install something just to try it out. With this change, the
local bin directory is tried first, which usually is also in the
`PATH`. This way the installation script can be run without
`sudo` and should still work.
* fix: try local path last and support spaces in $HOME
Try the `$HOME/.local/bin` path last, so that the script is backwards
compatible. Also make sure that it works even if there are spaces in
the directory set by `$HOME`.
* Allow pubsub and namesys-pubsub to be enabled via config
Signed-off-by: Joe Holden <jwh@zorins.us>
Co-authored-by: Marcin Rataj <lidel@lidel.org>
Co-authored-by: Adin Schmahmann <adin.schmahmann@gmail.com>
* Expose additional migration APIs
Expose migration APIs for reading migration config and creating migration fetchers. This allows implementation of commands and external applications that want to retrieve migrations according to the Migration portion of the IPFS config.
This change also moves some functionality that is specific to fetching migrations via IPFS into the `ipfsfetcher` package.
