From d405dfd1aebfc589659ed03aa981ce68f9272c16 Mon Sep 17 00:00:00 2001 From: Marcin Rataj Date: Mon, 9 Feb 2026 22:00:17 +0100 Subject: [PATCH] docs: loud deprecation of badger v1 datastore (#11187) * docs: loud deprecation of badger v1 datastore badger v1 (go-ds-badger) has not been maintained by its upstream maintainers for years and has known bugs (startup timeouts, shutdown hangs, fd exhaustion). make the deprecation loud and unavoidable: - print ERROR log line and bordered stderr message on every daemon start when a badger-based datastore is detected - mark badgerds and badgerds-measure init profiles as DEPRECATED - update docs/datastores.md and docs/config.md with migration guidance - add changelog highlight for v0.40 badger v1 support will be removed later in 2026. part of https://github.com/ipfs/kubo/issues/11186 * docs(changelog): link badger deprecation to #11186 --- config/profile.go | 17 +++++++++++++--- docs/changelogs/v0.40.md | 7 +++++++ docs/config.md | 23 ++++++++++++++++------ docs/datastores.md | 23 ++++++++++++++++------ plugin/plugins/badgerds/badgerds.go | 30 ++++++++++++++++++++++++++++- 5 files changed, 84 insertions(+), 16 deletions(-) diff --git a/config/profile.go b/config/profile.go index 7832c7599..cfcc828c2 100644 --- a/config/profile.go +++ b/config/profile.go @@ -210,7 +210,9 @@ NOTE: This profile may only be applied when first initializing node at IPFS_PATH }, }, "badgerds": { - Description: `Configures the node to use the legacy badgerv1 datastore. + Description: `DEPRECATED: Configures the node to use the legacy badgerv1 datastore. +This profile will be removed in a future Kubo release. +New deployments should use 'flatfs' or 'pebbleds' instead. NOTE: this is badger 1.x, which has known bugs and is no longer supported by the upstream team. It is provided here only for pre-existing users, allowing them to migrate away to more modern datastore. @@ -225,6 +227,14 @@ Other caveats: * Good for medium-size datastores, but may run into performance issues if your dataset is bigger than a terabyte. +To migrate: create a new IPFS_PATH with 'ipfs init --profile=flatfs', +move pinned data via 'ipfs dag export/import' or 'ipfs pin ls -t recursive|add', +and decommission the old badger-based node. +When it comes to block storage, use experimental 'pebbleds' only if you are sure +modern 'flatfs' does not serve your use case (most users will be perfectly fine +with flatfs, it is also possible to keep flatfs for blocks and replace leveldb +with pebble if preferred over leveldb). + See configuration documentation at: https://github.com/ipfs/kubo/blob/master/docs/datastores.md#badgerds @@ -239,8 +249,9 @@ NOTE: This profile may only be applied when first initializing node at IPFS_PATH }, }, "badgerds-measure": { - Description: `Configures the node to use the legacy badgerv1 datastore with metrics wrapper. -Additional '*_datastore_*' metrics will be exposed on /debug/metrics/prometheus + Description: `DEPRECATED: Configures the node to use the legacy badgerv1 datastore with metrics wrapper. +This profile will be removed in a future Kubo release. +New deployments should use 'flatfs' or 'pebbleds' instead. NOTE: This profile may only be applied when first initializing node at IPFS_PATH via 'ipfs init --profile badgerds-measure' diff --git a/docs/changelogs/v0.40.md b/docs/changelogs/v0.40.md index 0f5041859..143180a7b 100644 --- a/docs/changelogs/v0.40.md +++ b/docs/changelogs/v0.40.md @@ -31,6 +31,7 @@ This release was brought to you by the [Shipyard](https://ipshipyard.com/) team. - [๐Ÿ“‹ Long listing format for `ipfs ls`](#-long-listing-format-for-ipfs-ls) - [๐Ÿ–ฅ๏ธ WebUI Improvements](#-webui-improvements) - [๐Ÿ“ข libp2p announces all interface addresses](#-libp2p-announces-all-interface-addresses) + - [๐Ÿ—‘๏ธ Badger v1 datastore slated for removal this year](#-badger-v1-datastore-slated-for-removal-this-year) - [๐Ÿ“ฆ๏ธ Dependency updates](#-dependency-updates) - [๐Ÿ“ Changelog](#-changelog) - [๐Ÿ‘จโ€๐Ÿ‘ฉโ€๐Ÿ‘งโ€๐Ÿ‘ฆ Contributors](#-contributors) @@ -313,6 +314,12 @@ $ ipfs config --json Addresses.NoAnnounce '["/ip4/172.17.0.0/ipcidr/16"]' The [`server` profile](https://github.com/ipfs/kubo/blob/master/docs/config.md#server-profile) already [filters common private ranges](https://github.com/ipfs/kubo/blob/master/config/profile.go#L24-L43) via `Addresses.NoAnnounce`. +#### ๐Ÿ—‘๏ธ Badger v1 datastore slated for removal this year + +The `badgerds` datastore (based on badger 1.x) is slated for removal. Badger v1 has not been maintained by its upstream maintainers for years and has known bugs including startup timeouts, shutdown hangs, and file descriptor exhaustion. Starting with this release, every daemon start with a badger-based repository prints a loud deprecation error on stderr. + +See the [`badgerds` profile documentation](https://github.com/ipfs/kubo/blob/master/docs/config.md#badgerds-profile) for migration guidance, and [#11186](https://github.com/ipfs/kubo/issues/11186) for background. + #### ๐Ÿ“ฆ๏ธ Dependency updates - update `go-libp2p` to [v0.47.0](https://github.com/libp2p/go-libp2p/releases/tag/v0.47.0) (incl. [v0.46.0](https://github.com/libp2p/go-libp2p/releases/tag/v0.46.0)) diff --git a/docs/config.md b/docs/config.md index 290550d47..bf7f29353 100644 --- a/docs/config.md +++ b/docs/config.md @@ -4066,9 +4066,21 @@ Configures the node to use the pebble datastore with metrics. This is the same a Configures the node to use the **legacy** badgerv1 datastore. > [!CAUTION] -> This is based on very old badger 1.x, which has known bugs and is no longer supported by the upstream team. -> It is provided here only for pre-existing users, allowing them to migrate away to more modern datastore. -> Do not use it for new deployments, unless you really, really know what you are doing. +> **Badger v1 datastore is deprecated and will be removed in a future Kubo release.** +> +> This is based on very old badger 1.x, which has not been maintained by its +> upstream maintainers for years and has known bugs (startup timeouts, shutdown +> hangs, file descriptor +> exhaustion, and more). Do not use it for new deployments. +> +> **To migrate:** create a new `IPFS_PATH` with `flatfs` +> (`ipfs init --profile=flatfs`), move pinned data via +> `ipfs dag export/import` or `ipfs pin ls -t recursive|add`, and decommission the +> old badger-based node. When it comes to block storage, use experimental +> `pebbleds` only if you are sure modern `flatfs` does not serve your use case +> (most users will be perfectly fine with `flatfs`, it is also possible to keep +> `flatfs` for blocks and replace `leveldb` with `pebble` if preferred over +> `leveldb`). Also, be aware that: @@ -4078,17 +4090,16 @@ Also, be aware that: `flatfs`. - This datastore uses up to several gigabytes of memory. - Good for medium-size datastores, but may run into performance issues if your dataset is bigger than a terabyte. -- The current implementation is based on old badger 1.x which is no longer supported by the upstream team. > [!WARNING] > This profile may only be applied when first initializing the node via `ipfs init --profile badgerds` > [!NOTE] -> See other caveats and configuration options at [`datastores.md#pebbleds`](datastores.md#pebbleds) +> See other caveats and configuration options at [`datastores.md#badgerds`](datastores.md#badgerds) ### `badgerds-measure` profile -Configures the node to use the **legacy** badgerv1 datastore with metrics. This is the same as [`badgerds` profile](#badger-profile) with the addition of the `measure` datastore wrapper. +Configures the node to use the **legacy** badgerv1 datastore with metrics. This is the same as [`badgerds` profile](#badger-profile) with the addition of the `measure` datastore wrapper. This profile will be removed in a future Kubo release. ### `lowpower` profile diff --git a/docs/datastores.md b/docs/datastores.md index 9ba500a59..f1dcf90d1 100644 --- a/docs/datastores.md +++ b/docs/datastores.md @@ -93,13 +93,24 @@ When installing a new version of kubo when `"formatMajorVersion"` is configured, Uses [badger](https://github.com/dgraph-io/badger) as a key-value store. > [!CAUTION] -> This is based on very old badger 1.x, which has known bugs and is no longer supported by the upstream team. -> It is provided here only for pre-existing users, allowing them to migrate away to more modern datastore. -> Do not use it for new deployments, unless you really, really know what you are doing. +> **Badger v1 datastore is deprecated and will be removed in a future Kubo release.** +> +> This is based on very old badger 1.x, which has not been maintained by its +> upstream maintainers for years and has known bugs (startup timeouts, shutdown +> hangs, file descriptor +> exhaustion, and more). Do not use it for new deployments. +> +> **To migrate:** create a new `IPFS_PATH` with `flatfs` +> (`ipfs init --profile=flatfs`), move pinned data via +> `ipfs dag export/import` or `ipfs pin ls -t recursive|add`, and decommission the +> old badger-based node. When it comes to block storage, use experimental +> `pebbleds` only if you are sure modern `flatfs` does not serve your use case +> (most users will be perfectly fine with `flatfs`, it is also possible to keep +> `flatfs` for blocks and replace `leveldb` with `pebble` if preferred over +> `leveldb`). - -* `syncWrites`: Flush every write to disk before continuing. Setting this to false is safe as kubo will automatically flush writes to disk before and after performing critical operations like pinning. However, you can set this to true to be extra-safe (at the cost of a 2-3x slowdown when adding files). -* `truncate`: Truncate the DB if a partially written sector is found (defaults to true). There is no good reason to set this to false unless you want to manually recover partially written (and unpinned) blocks if kubo crashes half-way through a write operation. +- `syncWrites`: Flush every write to disk before continuing. Setting this to false is safe as kubo will automatically flush writes to disk before and after performing critical operations like pinning. However, you can set this to true to be extra-safe (at the cost of a 2-3x slowdown when adding files). +- `truncate`: Truncate the DB if a partially written sector is found (defaults to true). There is no good reason to set this to false unless you want to manually recover partially written (and unpinned) blocks if kubo crashes half-way through a write operation. ```json { diff --git a/plugin/plugins/badgerds/badgerds.go b/plugin/plugins/badgerds/badgerds.go index 2410f196c..c931db793 100644 --- a/plugin/plugins/badgerds/badgerds.go +++ b/plugin/plugins/badgerds/badgerds.go @@ -5,6 +5,7 @@ import ( "os" "path/filepath" + logging "github.com/ipfs/go-log/v2" "github.com/ipfs/kubo/plugin" "github.com/ipfs/kubo/repo" "github.com/ipfs/kubo/repo/fsrepo" @@ -13,6 +14,8 @@ import ( badgerds "github.com/ipfs/go-ds-badger" ) +var log = logging.Logger("plugin/badgerds") + // Plugins is exported list of plugins that will be loaded. var Plugins = []plugin.Plugin{ &badgerdsPlugin{}, @@ -108,7 +111,32 @@ func (c *datastoreConfig) DiskSpec() fsrepo.DiskSpec { } func (c *datastoreConfig) Create(path string) (repo.Datastore, error) { - fmt.Fprintln(os.Stderr, "โš ๏ธ badgerds is based on badger 1.x, which has known bugs and is no longer supported by the upstream team. Please switch to a newer datastore such as pebbleds or flatfs.") + log.Error("badger v1 datastore is deprecated and will be removed later in 2026, migrate to flatfs or experimental pebbleds: https://github.com/ipfs/kubo/issues/11186") + fmt.Fprintf(os.Stderr, ` +โ•”โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•— +โ•‘ โ•‘ +โ•‘ ERROR: BADGER v1 DATASTORE IS DEPRECATED โ•‘ +โ•‘ โ•‘ +โ•‘ This datastore is based on badger 1.x which has not been maintained โ•‘ +โ•‘ by its upstream maintainers for years and has known bugs (startup โ•‘ +โ•‘ timeouts, shutdown hangs, file descriptor exhaustion, and more). โ•‘ +โ•‘ โ•‘ +โ•‘ Badger v1 support will be REMOVED later in 2026. โ•‘ +โ•‘ โ•‘ +โ•‘ To migrate: โ•‘ +โ•‘ 1. Create a new IPFS_PATH with flatfs (or experimental pebbleds โ•‘ +โ•‘ if flatfs does not serve your use case): โ•‘ +โ•‘ export IPFS_PATH=/path/to/new/repo โ•‘ +โ•‘ ipfs init --profile=flatfs โ•‘ +โ•‘ 2. Move pinned data via ipfs dag export/import โ•‘ +โ•‘ or ipfs pin ls -t recursive|add โ•‘ +โ•‘ 3. Decommission the old badger-based node โ•‘ +โ•‘ โ•‘ +โ•‘ See https://github.com/ipfs/kubo/blob/master/docs/datastores.md โ•‘ +โ•‘ https://github.com/ipfs/kubo/issues/11186 โ•‘ +โ•‘ โ•‘ +โ•šโ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ• +`) p := c.path if !filepath.IsAbs(p) { p = filepath.Join(path, p)