kubo/docs
Hector Sanjuan ecb25581c6
cmd/files: flush parent folders (#10630)
* cmd/files: flush parent folders

This is a mitigation to increased MFS memory usage in the course of many writes operations.

The underlying issue is the unbounded growth of the mfs directory cache in
boxo. In the latest boxo version, this cache can be cleared by calling Flush()
on the folder. In order to trigger that, we call Flush() on the parent folder
of the file/folder where the write-operations are happening.

To flushing the parent folder allows it to grow unbounded. Then, any read
operation to that folder or parents (i.e. stat), will trigger a sync-operation to match
the cache to the underlying unixfs structure (and obtain the correct node-cid).

This sync operation must visit every item in the cache. When the cache has grown too much,
and the underlying unixfs-folder has switched into a HAMT, the operation can take minutes.

Thus, we should clear the cache often and the Flush flag is a good indicator
that we can let it go. Users can always run with --flush=false and flush at
regular intervals during their MFS writes if they want to extract some performance.

Fixes #8694, #10588.

* cmd/files: docs and changelog for --flush changes
2024-12-19 14:07:17 -08:00
..
changelogs cmd/files: flush parent folders (#10630) 2024-12-19 14:07:17 -08:00
examples Upgrade to Boxo v0.26.0 (#10631) 2024-12-19 13:54:02 -08:00
production doc: document reverse proxy bug 2020-06-15 17:01:40 -07:00
specifications Docs fix spelling issues (#10493) 2024-08-26 11:32:07 -07:00
add-code-flow.md fix some typos in docs (#10598) 2024-11-25 15:15:43 -08:00
AUTHORS Updated authors doc 2016-08-18 11:54:48 -07:00
cli-http-api-core-diagram.png docs: README refresh, add cli-http-api-core diagram 2018-08-22 16:09:51 -07:00
command-completion.md feat: add zsh completions (#10040) 2023-08-17 16:43:27 +02:00
config.md docs: typos (#10602) 2024-11-26 19:19:48 +01:00
content-blocking.md docs: add detail to NOpfs instructions in content-blocking.md 2023-12-14 08:56:53 +01:00
customizing.md docs(customizing.md): add kubo binary imports description 2023-12-19 19:52:50 -05:00
datastores.md docs: typos (#10602) 2024-11-26 19:19:48 +01:00
debug-guide.md refactor: kubo in ./docs 2022-07-06 23:45:25 +02:00
delegated-routing.md chore: finish reframe removal 2024-03-19 12:25:35 +01:00
developer-certificate-of-origin Add developer certificate of origin document 2017-06-26 15:36:35 -07:00
EARLY_TESTERS.md docs: update RELEASE_CHECKLIST.md (#10496) 2024-09-11 19:06:54 +02:00
environment-variables.md feat(routing/http): support IPIP-484 and streaming (#10534) 2024-10-03 22:58:25 +00:00
experimental-features.md docs: clarify pnet limitations 2024-06-24 22:19:20 +02:00
file-transfer.md core/commands: remove 'ipfs dht' commands, except 'query' (#10328) 2024-02-07 09:56:30 +00:00
fuse.md docs: "error mounting: could not resolve name" (#10449) 2024-06-21 16:56:58 +02:00
gateway.md docs: updated links and refs to external resources (#10368) 2024-03-16 00:59:22 +01:00
generate-authors.sh added AUTHORS file 2015-02-11 07:42:21 -08:00
http-rpc-clients.md docs: updated links and refs to external resources (#10368) 2024-03-16 00:59:22 +01:00
implement-api-bindings.md core/commands!: remove deprecated object APIs (#10375) 2024-03-22 09:32:30 +01:00
libp2p-resource-management.md Docs fix spelling issues (#10493) 2024-08-26 11:32:07 -07:00
plugins.md docs: use fx.Decorate instead of fx.Replace in examples (#9725) 2023-03-20 17:27:45 -04:00
README.md docs: improve docs/README (#9539) 2023-01-23 12:02:37 +01:00
RELEASE_CHECKLIST.md docs: update RELEASE_CHECKLIST.md (#10564) 2024-11-14 04:04:18 +01:00
RELEASE_ISSUE_TEMPLATE.md docs: update RELEASE_CHECKLIST.md (#10496) 2024-09-11 19:06:54 +02:00
releases_thunderdome.md docs: update RELEASE_CHECKLIST.md (#10496) 2024-09-11 19:06:54 +02:00
releases.md docs: incorporate the RC into the main checklist 2022-09-22 10:41:10 +02:00
transports.md refactor: kubo in ./docs 2022-07-06 23:45:25 +02:00
windows.md docs: updated links and refs to external resources (#10368) 2024-03-16 00:59:22 +01:00

Developer Documentation and Guides

If you are looking for User Documentation & Guides, please visit docs.ipfs.tech or check General Documentation.

If youre experiencing an issue with IPFS, please follow our issue guide when filing an issue!

Otherwise, check out the following guides to using and developing IPFS:

General Documentation

Developing kubo

Guides

Advanced User Guides

Other