kubo/core/node/libp2p
Marten Seemann 514411bedb
feat: opt-in Swarm.ResourceMgr (go-libp2p v0.18) (#8680)
* update go-libp2p to v0.18.0

* initialize the resource manager

* add resource manager stats/limit commands

* load limit file when building resource manager

* log absent limit file

* write rcmgr to file when IPFS_DEBUG_RCMGR is set

* fix: mark swarm limit|stats as experimental

* feat(cfg): opt-in Swarm.ResourceMgr

This ensures we can safely test the resource manager without impacting
default behavior.

- Resource manager is disabled by default
    - Default for Swarm.ResourceMgr.Enabled is false for now
- Swarm.ResourceMgr.Limits allows user to tweak limits per specific
  scope in a way that is persisted across restarts
- 'ipfs swarm limit system' outputs human-readable json
- 'ipfs swarm limit system new-limits.json' sets new runtime limits
  (but does not change Swarm.ResourceMgr.Limits in the config)

Conventions to make libp2p devs life easier:
- 'IPFS_RCMGR=1 ipfs daemon' overrides the config and enables resource manager
- 'limit.json' overrides implicit defaults from libp2p (if present)

* docs(config): small tweaks

* fix: skip libp2p.ResourceManager if disabled

This ensures 'ipfs swarm limit|stats' work only when enabled.

* fix: use NullResourceManager when disabled

This reverts commit b19f7c9eca.
after clarification feedback from
https://github.com/ipfs/go-ipfs/pull/8680#discussion_r841680182

* style: rename IPFS_RCMGR to LIBP2P_RCMGR

preexisting libp2p toggles use LIBP2P_ prefix

* test: Swarm.ResourceMgr

* fix: location of opt-in limit.json and rcmgr.json.gz

Places these files inside of IPFS_PATH

* Update docs/config.md

* feat: expose rcmgr metrics when enabled (#8785)

* add metrics for the resource manager
* export protocol and service name in Prometheus metrics
* fix: expose rcmgr metrics only when enabled

Co-authored-by: Marcin Rataj <lidel@lidel.org>

* refactor: rcmgr_metrics.go

* refactor: rcmgr_defaults.go

This file defines implicit limit defaults used when Swarm.ResourceMgr.Enabled

We keep vendored copy to ensure go-ipfs is not impacted when go-libp2p
decides to change defaults in any of the future releases.

* refactor: adjustedDefaultLimits

Cleans up the way we initialize defaults and adds a fix for case
when connection manager runs with high limits.

It also hides `Swarm.ResourceMgr.Limits` until we have a better
understanding what syntax makes sense.

* chore: cleanup after a review

* fix: restore go-ipld-prime v0.14.2

* fix: restore go-ds-flatfs v0.5.1

Co-authored-by: Lucas Molas <schomatis@gmail.com>
Co-authored-by: Marcin Rataj <lidel@lidel.org>
2022-04-07 21:06:35 -04:00
..
addrs.go feat: Addresses.AppendAnnounce (#8177) 2021-11-30 17:58:19 +01:00
discovery.go fix(discovery): fix daemon not starting due to mdns startup failure (#8704) 2022-03-09 07:08:24 -08:00
dns.go explicit construction of DNS resolver 2021-05-10 21:03:27 -07:00
filters.go use a ConnectionManager for filtering addresses 2021-08-19 18:04:48 +02:00
host.go feat: go-libp2p 0.16, UnixFS autosharding and go-datastore with contexts (#8563) 2021-11-29 19:58:05 +01:00
hostopt.go feat: go-libp2p 0.16, UnixFS autosharding and go-datastore with contexts (#8563) 2021-11-29 19:58:05 +01:00
libp2p_test.go add a test case for libp2p option prioritization 2022-02-28 15:26:06 +04:00
libp2p.go feat: opt-in Swarm.ResourceMgr (go-libp2p v0.18) (#8680) 2022-04-07 21:06:35 -04:00
nat.go fix: rewrite dependencies over the go-ipfs-config package 2022-03-04 09:45:16 +01:00
peerstore.go feat: go-libp2p 0.16, UnixFS autosharding and go-datastore with contexts (#8563) 2021-11-29 19:58:05 +01:00
pnet.go update to new private network interfaces 2020-03-09 19:39:06 -07:00
pubsub.go fix(pubsub): flood publish 2020-05-29 16:42:10 -07:00
rcmgr_defaults.go feat: opt-in Swarm.ResourceMgr (go-libp2p v0.18) (#8680) 2022-04-07 21:06:35 -04:00
rcmgr_metrics.go feat: opt-in Swarm.ResourceMgr (go-libp2p v0.18) (#8680) 2022-04-07 21:06:35 -04:00
rcmgr.go feat: opt-in Swarm.ResourceMgr (go-libp2p v0.18) (#8680) 2022-04-07 21:06:35 -04:00
relay.go fix: rewrite dependencies over the go-ipfs-config package 2022-03-04 09:45:16 +01:00
routing.go integrate experimental AcceleratedDHTClient 2021-05-14 04:31:06 -04:00
routingopt.go integrate experimental AcceleratedDHTClient 2021-05-14 04:31:06 -04:00
sec.go fix: rewrite dependencies over the go-ipfs-config package 2022-03-04 09:45:16 +01:00
smux.go feat: opt-in Swarm.ResourceMgr (go-libp2p v0.18) (#8680) 2022-04-07 21:06:35 -04:00
topicdiscovery.go ipns(pubsub): utilize persistent pubsub value store 2019-12-11 11:52:05 -05:00
transport.go fix: rewrite dependencies over the go-ipfs-config package 2022-03-04 09:45:16 +01:00