diff --git a/docs/config.md b/docs/config.md index 290550d47..e41a842d2 100644 --- a/docs/config.md +++ b/docs/config.md @@ -910,6 +910,14 @@ storage system. A soft upper limit for the size of the ipfs repository's datastore. With `StorageGCWatermark`, is used to calculate whether to trigger a gc run (only if `--enable-gc` flag is set). +> [!NOTE] +> This only controls when automatic GC of raw blocks is triggered. It is not a +> hard limit on total disk usage. The metadata stored alongside blocks (pins, +> MFS, provider system state, pubsub message ID tracking, and other internal +> data) is not counted against this limit. Always include extra headroom to +> account for metadata overhead. See [datastores.md](datastores.md) for details +> on how different datastore backends handle disk space reclamation. + Default: `"10GB"` Type: `string` (size) diff --git a/docs/datastores.md b/docs/datastores.md index 9ba500a59..3de1cbca2 100644 --- a/docs/datastores.md +++ b/docs/datastores.md @@ -34,7 +34,9 @@ The shardFunc is prefixed with `/repo/flatfs/shard/v1` then followed by a descri NOTE: flatfs must only be used as a block store (mounted at `/blocks`) as it only partially implements the datastore interface. You can mount flatfs for /blocks only using the mount datastore (described below). ## levelds -Uses a leveldb database to store key-value pairs. + +Uses a [leveldb](https://github.com/syndtr/goleveldb) database to store key-value +pairs via [go-ds-leveldb](https://github.com/ipfs/go-ds-leveldb). ```json { @@ -44,6 +46,26 @@ Uses a leveldb database to store key-value pairs. } ``` +> [!NOTE] +> LevelDB uses a log-structured merge-tree (LSM) storage engine. When keys are +> deleted, the data is not removed immediately. Instead, a tombstone marker is +> written, and the actual data is removed later by background compaction. +> +> LevelDB's compaction decides what to compact based on file counts (L0) and +> total level size (L1+), without considering how many tombstones a file +> contains. This means that after bulk deletions (such as pin removals or the +> periodic provider keystore sync), disk space may not be reclaimed promptly. +> The `datastore/` directory can grow significantly larger than the live data it +> holds, especially on long-running nodes with many CIDs. +> +> Unlike flatfs (which deletes files immediately) or pebble (which has +> tombstone-aware compaction), LevelDB has no way to prioritize reclaiming +> space from deleted keys. Restarting the daemon may trigger some compaction, +> but this is not guaranteed. +> +> If slow compaction is a problem, consider using the `pebbleds` datastore +> instead (see below), which handles this workload more efficiently. + ## pebbleds Uses [pebble](https://github.com/cockroachdb/pebble) as a key-value store.