kubo/config/swarm.go
Marten Seemann 514411bedb
feat: opt-in Swarm.ResourceMgr (go-libp2p v0.18) (#8680)
* update go-libp2p to v0.18.0

* initialize the resource manager

* add resource manager stats/limit commands

* load limit file when building resource manager

* log absent limit file

* write rcmgr to file when IPFS_DEBUG_RCMGR is set

* fix: mark swarm limit|stats as experimental

* feat(cfg): opt-in Swarm.ResourceMgr

This ensures we can safely test the resource manager without impacting
default behavior.

- Resource manager is disabled by default
    - Default for Swarm.ResourceMgr.Enabled is false for now
- Swarm.ResourceMgr.Limits allows user to tweak limits per specific
  scope in a way that is persisted across restarts
- 'ipfs swarm limit system' outputs human-readable json
- 'ipfs swarm limit system new-limits.json' sets new runtime limits
  (but does not change Swarm.ResourceMgr.Limits in the config)

Conventions to make libp2p devs life easier:
- 'IPFS_RCMGR=1 ipfs daemon' overrides the config and enables resource manager
- 'limit.json' overrides implicit defaults from libp2p (if present)

* docs(config): small tweaks

* fix: skip libp2p.ResourceManager if disabled

This ensures 'ipfs swarm limit|stats' work only when enabled.

* fix: use NullResourceManager when disabled

This reverts commit b19f7c9eca.
after clarification feedback from
https://github.com/ipfs/go-ipfs/pull/8680#discussion_r841680182

* style: rename IPFS_RCMGR to LIBP2P_RCMGR

preexisting libp2p toggles use LIBP2P_ prefix

* test: Swarm.ResourceMgr

* fix: location of opt-in limit.json and rcmgr.json.gz

Places these files inside of IPFS_PATH

* Update docs/config.md

* feat: expose rcmgr metrics when enabled (#8785)

* add metrics for the resource manager
* export protocol and service name in Prometheus metrics
* fix: expose rcmgr metrics only when enabled

Co-authored-by: Marcin Rataj <lidel@lidel.org>

* refactor: rcmgr_metrics.go

* refactor: rcmgr_defaults.go

This file defines implicit limit defaults used when Swarm.ResourceMgr.Enabled

We keep vendored copy to ensure go-ipfs is not impacted when go-libp2p
decides to change defaults in any of the future releases.

* refactor: adjustedDefaultLimits

Cleans up the way we initialize defaults and adds a fix for case
when connection manager runs with high limits.

It also hides `Swarm.ResourceMgr.Limits` until we have a better
understanding what syntax makes sense.

* chore: cleanup after a review

* fix: restore go-ipld-prime v0.14.2

* fix: restore go-ds-flatfs v0.5.1

Co-authored-by: Lucas Molas <schomatis@gmail.com>
Co-authored-by: Marcin Rataj <lidel@lidel.org>
2022-04-07 21:06:35 -04:00

191 lines
6.9 KiB
Go

package config
type SwarmConfig struct {
// AddrFilters specifies a set libp2p addresses that we should never
// dial or receive connections from.
AddrFilters []string
// DisableBandwidthMetrics disables recording of bandwidth metrics for a
// slight reduction in memory usage. You probably don't need to set this
// flag.
DisableBandwidthMetrics bool
// DisableNatPortMap turns off NAT port mapping (UPnP, etc.).
DisableNatPortMap bool
// DisableRelay explicitly disables the relay transport.
//
// Deprecated: This flag is deprecated and is overridden by
// `Swarm.Transports.Relay` if specified.
DisableRelay bool `json:",omitempty"`
// EnableRelayHop makes this node act as a public relay v1
//
// Deprecated: The circuit v1 protocol is deprecated.
// Use `Swarm.RelayService` to configure the circuit v2 relay.
EnableRelayHop bool `json:",omitempty"`
// EnableAutoRelay enables the "auto relay user" feature.
// Node will find and use advertised public relays when it determines that
// it's not reachable from the public internet.
//
// Deprecated: This flag is deprecated and is overridden by
// `Swarm.RelayClient.Enabled` if specified.
EnableAutoRelay bool `json:",omitempty"`
// RelayClient controls the client side of "auto relay" feature.
// When enabled, the node will use relays if it is not publicly reachable.
RelayClient RelayClient
// RelayService.* controls the "relay service".
// When enabled, node will provide a limited relay service to other peers.
RelayService RelayService
// EnableHolePunching enables the hole punching service.
EnableHolePunching Flag `json:",omitempty"`
// Transports contains flags to enable/disable libp2p transports.
Transports Transports
// ConnMgr configures the connection manager.
ConnMgr ConnMgr
// ResourceMgr configures the libp2p Network Resource Manager
ResourceMgr ResourceMgr
}
type RelayClient struct {
// Enables the auto relay feature: will use relays if it is not publicly reachable.
Enabled Flag `json:",omitempty"`
// StaticRelays configures static relays to use when this node is not
// publicly reachable. If set, auto relay will not try to find any
// other relay servers.
StaticRelays []string `json:",omitempty"`
}
// RelayService configures the resources of the circuit v2 relay.
// For every field a reasonable default will be defined in go-ipfs.
type RelayService struct {
// Enables the limited relay service for other peers (circuit v2 relay).
Enabled Flag `json:",omitempty"`
// ConnectionDurationLimit is the time limit before resetting a relayed connection.
ConnectionDurationLimit *OptionalDuration `json:",omitempty"`
// ConnectionDataLimit is the limit of data relayed (on each direction) before resetting the connection.
ConnectionDataLimit *OptionalInteger `json:",omitempty"`
// ReservationTTL is the duration of a new (or refreshed reservation).
ReservationTTL *OptionalDuration `json:",omitempty"`
// MaxReservations is the maximum number of active relay slots.
MaxReservations *OptionalInteger `json:",omitempty"`
// MaxCircuits is the maximum number of open relay connections for each peer; defaults to 16.
MaxCircuits *OptionalInteger `json:",omitempty"`
// BufferSize is the size of the relayed connection buffers.
BufferSize *OptionalInteger `json:",omitempty"`
// MaxReservationsPerPeer is the maximum number of reservations originating from the same peer.
MaxReservationsPerPeer *OptionalInteger `json:",omitempty"`
// MaxReservationsPerIP is the maximum number of reservations originating from the same IP address.
MaxReservationsPerIP *OptionalInteger `json:",omitempty"`
// MaxReservationsPerASN is the maximum number of reservations origination from the same ASN.
MaxReservationsPerASN *OptionalInteger `json:",omitempty"`
}
type Transports struct {
// Network specifies the base transports we'll use for dialing. To
// listen on a transport, add the transport to your Addresses.Swarm.
Network struct {
// All default to on.
QUIC Flag `json:",omitempty"`
TCP Flag `json:",omitempty"`
Websocket Flag `json:",omitempty"`
Relay Flag `json:",omitempty"`
}
// Security specifies the transports used to encrypt insecure network
// transports.
Security struct {
// Defaults to 100.
TLS Priority `json:",omitempty"`
// Defaults to 200.
SECIO Priority `json:",omitempty"`
// Defaults to 300.
Noise Priority `json:",omitempty"`
}
// Multiplexers specifies the transports used to multiplex multiple
// connections over a single duplex connection.
Multiplexers struct {
// Defaults to 100.
Yamux Priority `json:",omitempty"`
// Defaults to 200.
Mplex Priority `json:",omitempty"`
}
}
// ConnMgr defines configuration options for the libp2p connection manager
type ConnMgr struct {
Type string
LowWater int
HighWater int
GracePeriod string
}
// ResourceMgr defines configuration options for the libp2p Network Resource Manager
// <https://github.com/libp2p/go-libp2p-resource-manager#readme>
type ResourceMgr struct {
// Enables the Network Resource Manager feature
Enabled Flag `json:",omitempty"`
/* TODO: decide if and how we want to expose limits in our config
Limits *ResourceMgrScopeConfig `json:",omitempty"` */
}
const (
ResourceMgrSystemScope = "system"
ResourceMgrTransientScope = "transient"
ResourceMgrServiceScopePrefix = "svc:"
ResourceMgrProtocolScopePrefix = "proto:"
ResourceMgrPeerScopePrefix = "peer:"
)
/* TODO: decide if and how we want to expose limits in our config
type ResourceMgrLimitsConfig struct {
System *ResourceMgrScopeConfig `json:",omitempty"`
Transient *ResourceMgrScopeConfig `json:",omitempty"`
ServiceDefault *ResourceMgrScopeConfig `json:",omitempty"`
ServicePeerDefault *ResourceMgrScopeConfig `json:",omitempty"`
Service map[string]ResourceMgrScopeConfig `json:",omitempty"`
ServicePeer map[string]ResourceMgrScopeConfig `json:",omitempty"`
ProtocolDefault *ResourceMgrScopeConfig `json:",omitempty"`
ProtocolPeerDefault *ResourceMgrScopeConfig `json:",omitempty"`
Protocol map[string]ResourceMgrScopeConfig `json:",omitempty"`
ProtocolPeer map[string]ResourceMgrScopeConfig `json:",omitempty"`
PeerDefault *ResourceMgrScopeConfig `json:",omitempty"`
Peer map[string]ResourceMgrScopeConfig `json:",omitempty"`
Conn *ResourceMgrScopeConfig `json:",omitempty"`
Stream *ResourceMgrScopeConfig `json:",omitempty"`
}
*/
// libp2p Network Resource Manager config for a scope
type ResourceMgrScopeConfig struct {
Dynamic bool `json:",omitempty"`
// set if Dynamic is false
Memory int64 `json:",omitempty"`
// set if Dynamic is true
MemoryFraction float64 `json:",omitempty"`
MinMemory int64 `json:",omitempty"`
MaxMemory int64 `json:",omitempty"`
Streams, StreamsInbound, StreamsOutbound int
Conns, ConnsInbound, ConnsOutbound int
FD int
}