Commit Graph

311 Commits

Author SHA1 Message Date
Marcin Rataj
a81cc29282
fix: resolve TestAddMultipleGCLive race condition (#10916)
test was expecting immediate GC lock acquisition after pipe close,
but timing wasn't guaranteed. replaced blocking wait with 5-second
timeout to handle timing variations while still detecting deadlocks.
2025-08-18 20:49:26 +02:00
Hector Sanjuan
a673c2ec95
fix: Provide according to Reprovider.Strategy (#10886)
* Provide according to strategy

Updates boxo to a version with the changes from https://github.com/ipfs/boxo/pull/976, which decentralize the providing responsibilities (from a central providing.Exchange to blockstore, pinner, mfs).

The changes consist in initializing the Pinner, MFS and the blockstore with the provider.System, which is created first.

Since the provider.System is created first, the reproviding KeyChanFunc is set
later when we can create it once we have the Pinner, MFS and the blockstore.

Some additional work applies to the Add() workflow. Normally, blocks would get provided at the Blockstore or the Pinner, but when adding blocks AND a "pinned" strategy is used, the blockstore does not provide, and the
pinner does not traverse the DAG (and thus doesn't provide either), so we need to provide directly from the Adder. This is resolved by wrapping the DAGService in a "providingDAGService" which provides every added block, when using the "pinned" strategy.

`ipfs --offline add` when the ONLINE daemon is running will now announce blocks per the chosen strategy, where before it did not announce them. This is documented in the changelog. A couple of releases ago, adding with `ipfs --offline add` was faster, but this is no longer the case so we are not incurring in any penalties by sticking to the fact that the daemon is online and has a providing strategy that we follow.

Co-authored-by: gammazero <11790789+gammazero@users.noreply.github.com>
Co-authored-by: Marcin Rataj <lidel@lidel.org>
2025-08-08 10:56:44 +02:00
Kapil Sareen
10abb9073d
feat(add): add support for naming pinned CIDs (#10877)
Some checks are pending
CodeQL / codeql (push) Waiting to run
Docker Build / docker-build (push) Waiting to run
Gateway Conformance / gateway-conformance (push) Waiting to run
Gateway Conformance / gateway-conformance-libp2p-experiment (push) Waiting to run
Go Build / go-build (push) Waiting to run
Go Check / go-check (push) Waiting to run
Go Lint / go-lint (push) Waiting to run
Go Test / go-test (push) Waiting to run
Interop / interop-prep (push) Waiting to run
Interop / helia-interop (push) Blocked by required conditions
Interop / ipfs-webui (push) Blocked by required conditions
Sharness / sharness-test (push) Waiting to run
Spell Check / spellcheck (push) Waiting to run
* feat(add): add support for naming pinned CID

Signed-off-by: kapil <kapilsareen584@gmail.com>

* fix(add): no double pinning and simplify pin-name

- modify PinRoot to accept name parameter, eliminating double pinning
- remove automatic filename fallback logic for cleaner behavior
- only create named pins when explicitly requested via --pin-name=value
- replace NoPinName constant with idiomatic empty string literals
- Update help text and tests to reflect explicit-only behavior

* docs: changelog

* chore: lint

* test: negative case for empty pin-name

* chore: gofmt

---------

Signed-off-by: kapil <kapilsareen584@gmail.com>
Co-authored-by: Marcin Rataj <lidel@lidel.org>
2025-08-06 02:16:16 +02:00
Andrew Gillis
20d9660a64
chore: use go-log/v2 (#10801)
Some checks are pending
CodeQL / codeql (push) Waiting to run
Docker Build / docker-build (push) Waiting to run
Gateway Conformance / gateway-conformance (push) Waiting to run
Gateway Conformance / gateway-conformance-libp2p-experiment (push) Waiting to run
Go Build / go-build (push) Waiting to run
Go Check / go-check (push) Waiting to run
Go Lint / go-lint (push) Waiting to run
Go Test / go-test (push) Waiting to run
Interop / interop-prep (push) Waiting to run
Interop / helia-interop (push) Blocked by required conditions
Interop / ipfs-webui (push) Blocked by required conditions
Sharness / sharness-test (push) Waiting to run
Spell Check / spellcheck (push) Waiting to run
* chore: update to go-log/v2

go-log v2 has been out for quite a while now and it is time to deprecate v1.

Replace all use of go-log with go-log/v2
Makes /api/v0/log/tail useful over HTTP
Updates dependencies that have moved to go-lov/v2
Removes support for ContextWithLoggable as this is not needed for tracing-like functionality
- Replaces: PR #8765
- Closes issue #8753
- Closes issue #9245
- Closes issue #10809

Other fixes:
* update go-ipfs-cmds
* update http logs test
* fix test
* Read/send one line of log data at a time
* Update -log-level docs
2025-05-19 13:04:05 -07:00
Hector Sanjuan
6b55e64918
feat(config): ipfs add and Import options for controling UnixFS DAG Width (#10774)
Some checks failed
CodeQL / codeql (push) Has been cancelled
Docker Build / docker-build (push) Has been cancelled
Gateway Conformance / gateway-conformance (push) Has been cancelled
Gateway Conformance / gateway-conformance-libp2p-experiment (push) Has been cancelled
Go Build / go-build (push) Has been cancelled
Go Check / go-check (push) Has been cancelled
Go Lint / go-lint (push) Has been cancelled
Go Test / go-test (push) Has been cancelled
Interop / interop-prep (push) Has been cancelled
Sharness / sharness-test (push) Has been cancelled
Spell Check / spellcheck (push) Has been cancelled
Interop / helia-interop (push) Has been cancelled
Interop / ipfs-webui (push) Has been cancelled
Co-authored-by: Marcin Rataj <lidel@lidel.org>
2025-04-15 22:56:38 +02:00
Andrew Gillis
263edb251e
feat: Support storing UnixFS 1.5 Mode and ModTime (#10478)
Co-authored-by: Marcin Rataj <lidel@lidel.org>
2024-08-21 02:02:46 +02:00
Marcin Rataj
feef0851bd
feat: go-libp2p 0.36 and /webrtc-direct listener (#10463)
Co-authored-by: Marco Munizaga <git@marcopolo.io>
2024-08-06 21:25:27 +02:00
Henrique Dias
a8a6bbe929
feat: support optional pin names (#10261) 2024-01-04 14:25:06 +01:00
Henrique Dias
58c29399cf
chore: clean migration 2023-11-29 12:29:59 +01:00
Henrique Dias
a7c6518497
feat: path consolidation (#10063) 2023-10-06 16:14:44 +02:00
Kay
f12b372af9
style: gofumpt and godot [skip changelog] (#10081) 2023-08-17 14:02:08 +02:00
Jorropo
3ab1086f71 chore: migrate go-libipfs to boxo
Resolves #9677, #9676, #9675, #9736
2023-03-28 22:05:25 -04:00
Michael Muré
86da181f7f
feat(pinning): connect some missing go context (#9557)
Co-authored-by: Henrique Dias <hacdias@gmail.com>
2023-02-22 16:56:59 +01:00
Jorropo
f20c980f2d chore: bump go-libipfs to replace go-block-format
Includes changes from:
- https://github.com/ipfs/go-block-format/pull/37
- https://github.com/ipfs/go-libipfs/pull/58
2023-01-26 16:55:48 +01:00
Henrique Dias
255e64e49e
chore: migrate from go-ipfs-files to go-libipfs/files (#9535) 2023-01-13 14:27:03 +01:00
Rod Vagg
cb1ba70a9b
fix: handle new ProtoNode errors 2022-12-02 10:16:10 +11:00
Marcin Rataj
82467bc936 refactor: rename to kubo 2022-07-06 18:40:37 +02:00
Jorropo
a433064d72
chore: replace ioutil with io and os (#8969)
Co-authored-by: Håvard Anda Estensen <haavard.ae@gmail.com>
2022-06-14 12:37:02 -04:00
Gus Eggert
f855bfe6ef
feat: add basic gateway tracing (#8595)
* add deprecation warning when tracer plugins are loaded
* add response format attribute to span in gateway handler
* add note about tracing's experimental status in godoc
* add nil check for TTL when adding name span attrs
* add basic sharness test for integration with otel collector
* add nil check in UnixFSAPI.processLink
* test: sharness check all json objs for swarm span
* add env var docs to docs/environment-variables.md
* chore: pin the otel collector version
* add tracing spans per response type (#8841)
* docs: tracing with jaeger-ui

Co-authored-by: Marcin Rataj <lidel@lidel.org>
2022-04-04 19:24:05 +02:00
Laurent Senta
8d549f03f3 fix: rewrite dependencies over the go-ipfs-config package 2022-03-04 09:45:16 +01:00
Adin Schmahmann
52c177ced9
feat: go-libp2p 0.16, UnixFS autosharding and go-datastore with contexts (#8563)
* plumb through go-datastore context changes

* update go-libp2p to v0.16.0
* use LIBP2P_TCP_REUSEPORT instead of IPFS_REUSEPORT
* use relay config
* making deprecation notice match the go-ipfs-config key
* docs(config): circuit relay v2
* docs(config): fix links and headers
* feat(config): Internal.Libp2pForceReachability

This switches to config that supports setting and reading
Internal.Libp2pForceReachability OptionalString flag

* use configuration option for static relays

* chore: go-ipfs-config v0.18.0

https://github.com/ipfs/go-ipfs-config/releases/tag/v0.18.0

* feat: circuit v1 migration prompt when Swarm.EnableRelayHop is set (#8559)
* exit when Swarm.EnableRelayHop is set
* docs: Experimental.ShardingEnabled migration

This ensures existing users of global sharding experiment get notified
that the flag no longer works + that autosharding happens automatically.

For people who NEED to keep the old behavior (eg. have no time to
migrate today) there is a note about restoring it with
`UnixFSShardingSizeThreshold`.

* chore: add dag-jose code to the cid command output

* add support for setting automatic unixfs sharding threshold from the config
* test: have tests use low cutoff for sharding to mimic old behavior
* test: change error message to match the current error
* test: Add automatic sharding/unsharding tests (#8547)
* test: refactored naming in the sharding sharness tests to make more sense

* ci: set interop test executor to convenience image for Go1.16 + Node
* ci: use interop master

Co-authored-by: Marcin Rataj <lidel@lidel.org>
Co-authored-by: Marten Seemann <martenseemann@gmail.com>
Co-authored-by: Marcin Rataj <lidel@lidel.org>
Co-authored-by: Gus Eggert <gus@gus.dev>
Co-authored-by: Lucas Molas <schomatis@gmail.com>
2021-11-29 19:58:05 +01:00
Dimitris Apostolou
1e437c7e97
Fix typos and cleanup 2020-04-20 22:00:01 +03:00
Adin Schmahmann
3a0d0e817a support async datastores 2019-12-19 00:59:31 -05:00
Michael Muré
c3692a5913 extract the pinner to go-ipfs-pinner and dagutils into go-merkledag 2019-12-02 15:45:44 -05:00
Michael Muré
097b684b42
pin: add context and error return to most of the Pinner functions 2019-11-18 18:32:48 +01:00
Steven Allen
9738d81c0c switch to new merkledag walk functions
EnumerateChildrenAsync has been renamed to WalkParallel to reflect the fact
that:

1. It visits the root.
2. It's parallel, not async.

To mirror this change, EnumerateChildren has also been renamed to Walk and now
behaves the same (except that it's not parallel).
2019-07-16 16:34:21 -07:00
Steven Allen
4924b80e10 chore: fix linter nits
License: MIT
Signed-off-by: Steven Allen <steven@stebalien.com>
2019-05-14 09:38:26 -07:00
Erik Ingenito
4427d7e8e3 Test and fix GC/pin bug #6252
License: MIT
Signed-off-by: Erik Ingenito <erik.ingenito@protocol.ai>
2019-05-01 14:25:31 -07:00
Łukasz Magiera
667b7f9927 coreiface: updates for moving path to subpackage
License: MIT
Signed-off-by: Łukasz Magiera <magik6k@gmail.com>
2019-04-17 12:03:37 -07:00
Łukasz Magiera
740b03a43e gomod: update interface to v0.0.6
License: MIT
Signed-off-by: Łukasz Magiera <magik6k@gmail.com>
2019-04-09 15:58:32 +02:00
Steven Allen
74d07eff35
Merge pull request #6156 from ipfs/fix/6139
fix add error handling
2019-03-29 15:31:14 +00:00
Steven Allen
1f293eff1b chore: fix a bunch of issues caught by golangci-lint
Most of these are probably harmless but a few looked like they might actually be
bugs. Most of them are just faulty tests.

License: MIT
Signed-off-by: Steven Allen <steven@stebalien.com>
2019-03-29 11:57:45 +00:00
Steven Allen
337bf0f4aa handle errors from commit on add
fixes #6139

License: MIT
Signed-off-by: Steven Allen <steven@stebalien.com>
2019-03-29 11:52:44 +00:00
Łukasz Magiera
95906d3201 coreapi: remove Unixfs.Wrap
License: MIT
Signed-off-by: Łukasz Magiera <magik6k@gmail.com>
2019-03-25 19:00:51 +01:00
Łukasz Magiera
152829d1fa adder: pin the right node
License: MIT
Signed-off-by: Łukasz Magiera <magik6k@gmail.com>
2019-03-21 19:20:24 +01:00
Łukasz Magiera
964ef0b2e7 unixfs add: remove StdinName
License: MIT
Signed-off-by: Łukasz Magiera <magik6k@gmail.com>
2019-03-21 19:20:24 +01:00
Łukasz Magiera
c5b81e918f coreapi: remove hidden file handling in add
License: MIT
Signed-off-by: Łukasz Magiera <magik6k@gmail.com>
2019-03-21 19:17:58 +01:00
Łukasz Magiera
b8bc26a700 WIP almost working ipfs add
License: MIT
Signed-off-by: Łukasz Magiera <magik6k@gmail.com>
2019-03-21 19:17:58 +01:00
Łukasz Magiera
332d4aeab1 coreapi unixfs: change Wrap logic to make more sense
License: MIT
Signed-off-by: Łukasz Magiera <magik6k@gmail.com>
2019-03-21 19:17:58 +01:00
Jakub Sztandera
42e191c017 gx: unrewrite
License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protonmail.ch>
2019-03-05 18:33:56 +01:00
Steven Allen
f227862e88 gx: update go-ipfs-cmds, go-bitswap, go-libp2p-kad-dht, and go-mplex
Fixes the latest batch of bugs found in RC testing.

License: MIT
Signed-off-by: Steven Allen <steven@stebalien.com>
2019-02-27 12:39:24 -08:00
Hector Sanjuan
6972a9aa4d Gx Bubble. libp2p-6.0.38
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2019-02-27 01:10:59 +00:00
Steven Allen
fea7ae727f gx: update go-cid
License: MIT
Signed-off-by: Steven Allen <steven@stebalien.com>
2019-02-20 20:29:06 -08:00
Steven Allen
f924f57c61 gx: update go-bitswap and go-libp2p-kad-dht
* go-bitswap: fix some race conditions.
* go-libp2p-kad-dht: fix a goroutine leak.

License: MIT
Signed-off-by: Steven Allen <steven@stebalien.com>
2019-02-20 17:19:54 -08:00
Steven Allen
91ca5911be coreapi: return coreiface.ErrNotSupported when "catting" symlinks.
License: MIT
Signed-off-by: Steven Allen <steven@stebalien.com>
2019-02-19 03:48:04 -08:00
Jakub Sztandera
28cf3de0f9 Update protobuf
License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protonmail.ch>
2019-02-18 20:46:22 +01:00
Steven Allen
cf0d4706e2 gx: update libp2p stuff
License: MIT
Signed-off-by: Steven Allen <steven@stebalien.com>
2019-02-14 14:58:35 -08:00
Steven Allen
e97a60b073 gx: update go-ipfs-files
fix compatibility issue with js-ipfs

License: MIT
Signed-off-by: Steven Allen <steven@stebalien.com>
2019-02-11 10:48:58 -08:00
Łukasz Magiera
648cc40ec3 coreapi: fix import grouping after extracting iface
License: MIT
Signed-off-by: Łukasz Magiera <magik6k@gmail.com>
2019-02-09 01:57:26 +01:00
Łukasz Magiera
7ed6b518b6 coreapi: update imports to updated interface
License: MIT
Signed-off-by: Łukasz Magiera <magik6k@gmail.com>
2019-02-09 01:42:32 +01:00