kubo/config
Marten Seemann 514411bedb
feat: opt-in Swarm.ResourceMgr (go-libp2p v0.18) (#8680)
* update go-libp2p to v0.18.0

* initialize the resource manager

* add resource manager stats/limit commands

* load limit file when building resource manager

* log absent limit file

* write rcmgr to file when IPFS_DEBUG_RCMGR is set

* fix: mark swarm limit|stats as experimental

* feat(cfg): opt-in Swarm.ResourceMgr

This ensures we can safely test the resource manager without impacting
default behavior.

- Resource manager is disabled by default
    - Default for Swarm.ResourceMgr.Enabled is false for now
- Swarm.ResourceMgr.Limits allows user to tweak limits per specific
  scope in a way that is persisted across restarts
- 'ipfs swarm limit system' outputs human-readable json
- 'ipfs swarm limit system new-limits.json' sets new runtime limits
  (but does not change Swarm.ResourceMgr.Limits in the config)

Conventions to make libp2p devs life easier:
- 'IPFS_RCMGR=1 ipfs daemon' overrides the config and enables resource manager
- 'limit.json' overrides implicit defaults from libp2p (if present)

* docs(config): small tweaks

* fix: skip libp2p.ResourceManager if disabled

This ensures 'ipfs swarm limit|stats' work only when enabled.

* fix: use NullResourceManager when disabled

This reverts commit b19f7c9eca.
after clarification feedback from
https://github.com/ipfs/go-ipfs/pull/8680#discussion_r841680182

* style: rename IPFS_RCMGR to LIBP2P_RCMGR

preexisting libp2p toggles use LIBP2P_ prefix

* test: Swarm.ResourceMgr

* fix: location of opt-in limit.json and rcmgr.json.gz

Places these files inside of IPFS_PATH

* Update docs/config.md

* feat: expose rcmgr metrics when enabled (#8785)

* add metrics for the resource manager
* export protocol and service name in Prometheus metrics
* fix: expose rcmgr metrics only when enabled

Co-authored-by: Marcin Rataj <lidel@lidel.org>

* refactor: rcmgr_metrics.go

* refactor: rcmgr_defaults.go

This file defines implicit limit defaults used when Swarm.ResourceMgr.Enabled

We keep vendored copy to ensure go-ipfs is not impacted when go-libp2p
decides to change defaults in any of the future releases.

* refactor: adjustedDefaultLimits

Cleans up the way we initialize defaults and adds a fix for case
when connection manager runs with high limits.

It also hides `Swarm.ResourceMgr.Limits` until we have a better
understanding what syntax makes sense.

* chore: cleanup after a review

* fix: restore go-ipld-prime v0.14.2

* fix: restore go-ds-flatfs v0.5.1

Co-authored-by: Lucas Molas <schomatis@gmail.com>
Co-authored-by: Marcin Rataj <lidel@lidel.org>
2022-04-07 21:06:35 -04:00
..
serialize fix: rewrite dependencies over the go-ipfs-config package 2022-03-04 09:45:16 +01:00
addresses.go Fix typos 2022-03-18 00:58:36 +01:00
api.go go-ipfs-config: Added API + Gateway support for arbitrary HTTP headers 2015-07-28 23:08:09 -07:00
autonat.go go-ipfs-config: feat: add an OptionalDuration type (#148) 2021-10-27 18:23:35 +02:00
bootstrap_peers_test.go go-ipfs-config: fix string formatting of bootstrap peers 2019-06-01 09:03:11 -07:00
bootstrap_peers.go go-ipfs-config: [LINT] error strings should not end with punctuation or a newline (ST1005) 2021-05-14 14:31:24 -07:00
config_test.go go-ipfs-config: Add one more test for config.Clone 2018-12-13 16:03:29 -08:00
config.go go-ipfs-config: feat: add Internal and Internal.Bitswap config options 2021-08-18 08:14:25 -04:00
datastore.go go-ipfs-config: cleanup and bug fixes 2017-09-03 14:08:52 -07:00
discovery.go go-ipfs-config: config: apply review to lowpower profile 2018-03-24 16:11:42 +01:00
dns.go go-ipfs-config: feat: add DNS.MaxCacheTTL for DNS-over-HTTPS resolvers (#161) 2022-01-20 20:04:53 +01:00
experiments.go go-ipfs-config: chore: omitempty Experimental.ShardingEnabled (#158) 2021-11-23 18:44:35 +01:00
gateway.go Fix typos 2022-03-18 00:58:36 +01:00
identity.go go-ipfs-config: migrate to the consolidated libp2p 2019-05-31 12:54:12 -07:00
init_test.go go-ipfs-config: error if bit size specified with ed25519 keys (#105) 2020-07-14 09:50:10 -07:00
init.go go-ipfs-config: feat: add Addresses.AppendAnnounce (#135) 2021-11-23 19:09:58 +01:00
internal.go go-ipfs-config: feat: Swarm.RelayService (circuit v2) (#146) 2021-11-13 03:07:14 -08:00
ipns.go go-ipfs-config: feat: pubsub and ipns pubsub flags (#145) 2021-10-27 21:50:10 +02:00
migration_test.go go-ipfs-config: Removed Peers from migration config 2021-05-03 09:08:16 -07:00
migration.go go-ipfs-config: Removed Peers from migration config 2021-05-03 09:08:16 -07:00
mounts.go go-ipfs-config: Added fuse allow_other option 2015-04-02 15:11:27 -07:00
peering.go go-ipfs-config: doc: improve wording for peering config 2020-05-25 19:14:40 -07:00
plugins.go go-ipfs-config: plugins: don't omit empty config values 2019-08-29 13:56:13 -07:00
profile.go go-ipfs-config: docs: updated flatfs/badger profile helptext (#167) 2022-01-28 23:39:10 +01:00
provider.go go-ipfs-config: Add very basic (possibly temporary) Provider configs 2019-06-27 12:20:44 -07:00
pubsub.go go-ipfs-config: feat: pubsub and ipns pubsub flags (#145) 2021-10-27 21:50:10 +02:00
remotepin.go go-ipfs-config: add support for pinning mfs (#116) 2021-01-28 18:05:47 -05:00
reprovider.go go-ipfs-config: Reprovider strategies 2017-08-13 00:02:23 +02:00
routing.go go-ipfs-config: feat: remove Routing.PrivateType 2020-04-09 13:24:03 -07:00
swarm.go feat: opt-in Swarm.ResourceMgr (go-libp2p v0.18) (#8680) 2022-04-07 21:06:35 -04:00
types_test.go go-ipfs-config: feat: Swarm.RelayService (circuit v2) (#146) 2021-11-13 03:07:14 -08:00
types.go go-ipfs-config: feat: Swarm.RelayService (circuit v2) (#146) 2021-11-13 03:07:14 -08:00