kubo/docs/plugins.md
Hector Sanjuan 4255cc3889
feat: telemetry plugin (#10866)
* Initial pass at Telemetry plugin

Currently, IP Shipyard, with the help of Probelab, monitor and extract
Amino/IPFS public network metrics with the use of DHT crawlers and
bootstrappers (via peerlog plugin). For example, we log all peer IDs seen and
their AgentVersion/Addresses obtained from the `identify` protocol, which
provides insights into protocol usage, total number of peers etc.

We would like to increase the ability to obtain more insights from the network
by collecting some more information in the future, but also to give users more
control over this collection (i.e. opt-out). The information collected will
not allow unique identification of anyone and is only used for aggregation.

Now, this PR explores a way of moving in this direction:

* A new "telemetry" fx plugin is in charge of dealing with telemetry
* The FX plugin allows to plug and make decisions / take actions during the setup phase:
  * We can inspect whether we are using Private Networks before the libp2p.Host has been initialized.
  * We can send telemetry after the libp2p Host is initialized.
  * Everything is self-contained. Custom builds can remove the plugin altogether without needing to surgically edit the code.

As for behaviour:

* The user can opt-in/out via EnvVar, file in the repo path or plugin configuration.
* Users on private networks or with custom bootstrappers are detected, offered a wall of text explaining why we need telemetry and invited to opt-in. Opt-out happens otherwise on a timeout (with no input). Their preferences are stored.
* Users on standard settings are opted-in by default. This is the status quo in Kubo already, except they don't get a chance to opt out.

The telemetry libp2p protocol is yet to be defined, but expect something similar to identify, with a protobuf being pushed to bootstrappers or to a specific telemetry node that we define. In the case of pnets, this will be done with a temporary peer.

* checkpoint

* telemetry plugin: second pass

* On first run it generates a UUID and shows a message to the user.
* UUID is persistend to "telemetry_uuid"
* Sends telemetry 1 minute after boot and every 24h
* LogEvent is the thing containing all the telemetry that is sent
* Opt-out possible via env-var or plugin configuration

* Telemetry: add changelog and environment variable documentation

* docs: improved daemon message

making it more obvious nothing was sent yet
and that user had 15m to out-out

plus some debug logs that confirm opt-out

* refactor: rename IPFS_TELEMETRY_MODE to IPFS_TELEMETRY

* fix: add User-Agent header to telemetry requests

---------

Co-authored-by: Andrew Gillis <11790789+gammazero@users.noreply.github.com>
Co-authored-by: Marcin Rataj <lidel@lidel.org>
2025-08-18 20:46:05 +02:00

6.7 KiB

Plugins

Since 0.4.11 Kubo has an experimental plugin system that allows augmenting the daemons functionality without recompiling.

When an IPFS node is started, it will load plugins from the $IPFS_PATH/plugins directory (by default ~/.ipfs/plugins).

Table of Contents

Plugin Types

Plugins can implement one or more plugin types, defined in the plugin package.

IPLD

IPLD plugins add support for additional formats to ipfs dag and other IPLD related commands.

Datastore

Datastore plugins add support for additional datastore backends.

Tracer

(experimental)

Tracer plugins allow injecting an opentracing backend into Kubo.

Daemon

Daemon plugins are started when the Kubo daemon is started and are given an instance of the CoreAPI. This should make it possible to build an ipfs-based application without IPC and without forking Kubo.

Note: We eventually plan to make Kubo usable as a library. However, this plugin type is likely the best interim solution.

fx (experimental)

Fx plugins let you customize the fx dependency graph and configuration, by customizing thefx.Options that are passed to fx when the Kubo node is initialized.

For example, you can override an interface such as exchange.Interface or pin.Pinner with a custom implementation by appending an option like fx.Decorate(func() exchange.Interface { return customExchange }).

Fx supports some advanced customization. Simple interface replacements like above are unlikely to break in the future, but the more invasive your changes, the more likely they are to break between releases. Kubo cannot guarantee backwards compatibility for fx customizations.

Fx options are applied across every execution of the ipfs binary, including:

  • Repo initialization
  • Daemon
  • Applying migrations
  • etc.

So if you plug in a blockservice that disallows non-allowlisted CIDs, then this may break migrations that fetch migration code over the IPFS network.

Internal

(never stable)

Internal plugins are like daemon plugins except that they can access, replace, and modify all internal state. Use this plugin type to extend Kubo in arbitrary ways. However, be aware that your plugin will likely break every time Kubo updated.

Configuration

Plugins can be configured in the Plugins section of the config file. Here, plugins can be:

  1. Passed an arbitrary config object via the Config field.
  2. Disabled via the Disabled field.

Example:

{
  // ...
  "Plugins": {
    "Plugins": {
      // plugin named "plugin-foo"
      "plugin-foo": {
        "Config": { /* arbitrary json */ }
      },
      // plugin named "plugin-bar"
      "plugin-bar": {
        "Disabled": true // "plugin-bar" will not be loaded
      }
    }
  }
}

Available Plugins

Name Type Preloaded Description
git IPLD x An IPLD format for git objects.
badgerds Datastore x A high performance but experimental datastore.
flatfs Datastore x A stable filesystem-based datastore.
levelds Datastore x A stable, flexible datastore backend.
jaeger Tracing An opentracing backend.
telemetry Telemetry x Collects anonymized usage data for Kubo development.
  • Preloaded plugins are built into the Kubo binary and do not need to be installed separately. At the moment, all in-tree plugins are preloaded.

Installing Plugins

Kubo supports two types of plugins: External and Preloaded.

  • External plugins must be installed in $IPFS_PATH/plugins/ (usually ~/.ipfs/plugins/).
  • Preloaded plugins are built-into the Kubo when it's compiled.

External Plugin

The advantage of an external plugin is that it can be built, packaged, and installed independently of Kubo. Unfortunately, this method is only supported on Linux and MacOS at the moment. Users of other operating systems should follow the instructions for preloaded plugins.

In-tree

To build plugins included in plugin/plugins, run:

kubo$ make build_plugins
kubo$ ls plugin/plugins/*.so

To install, copy desired plugins to $IPFS_PATH/plugins. For example:

kubo$ mkdir -p ~/.ipfs/plugins/
kubo$ cp plugin/plugins/git.so ~/.ipfs/plugins/
kubo$ chmod +x ~/.ipfs/plugins/git.so # ensure plugin is executable

Finally, restart daemon if it is running.

Out-of-tree

To build out-of-tree plugins, use the plugin's Makefile if provided. Otherwise, you can manually build the plugin by running:

myplugin$ go build -buildmode=plugin -o myplugin.so myplugin.go

Finally, as with in-tree plugins:

  1. Install the plugin in $IPFS_PATH/plugins.
  2. Mark the plugin as executable (chmod +x $IPFS_PATH/plugins/myplugin.so).
  3. Restart your IPFS daemon (if running).

Preloaded Plugins

The advantages of preloaded plugins are:

  1. They're bundled with the Kubo binary.
  2. They work on all platforms.

To preload a Kubo plugin:

  1. Add the plugin to the preload list: plugin/loader/preload_list
  2. Build ipfs
kubo$ make build

You can also preload an in-tree but disabled-by-default plugin by adding it to the IPFS_PLUGINS variable. For example, to enable plugins foo, bar, and baz:

kubo$ make build IPFS_PLUGINS="foo bar baz"

Creating A Plugin

To create your own out-of-tree plugin, use the example plugin as a starting point. When you're ready, submit a PR adding it to the list of available plugins.