Commit Graph

163 Commits

Author SHA1 Message Date
Marcin Rataj
7871a0beb7
feat(cmds): add support for CAR v2 imports (#8854)
* feat: add support for carv2 import
* update to multicodec 0.4.0
* add sharness test for carv2

Co-authored-by: Keenan Nemetz <keenan.nemetz@gmail.com>
2022-04-07 04:33:13 +02:00
Marcin Rataj
a61c53f87f
feat(cmds): document deprecated RPC API commands (#8802)
* feat(cmds): add deprecated and experimental status

Added programmatic state annotation introduced in
https://github.com/ipfs/go-ipfs-cmds/pull/225
for already deprecated / experimental commands.

* chore: go-ipfs-cmds v0.7.0

Co-authored-by: Lucas Molas <schomatis@gmail.com>
2022-03-22 20:46:32 +01:00
Jorropo
09aed2d63b
Merge pull request #8757 from ipfs/feat/err-not-found
Bubble ErrNotFound improvements.

The is a really nice UX improvement because it know tell you which ipld block is missing instead of a generic "not found".
2022-03-18 00:42:40 +01:00
Lucas Molas
04e7e9502e
feat(cmds): extend block size check for dag|block put (#8751)
* feat(cmds): extend block size check for dag|block put
* feat(cmds): block size check for dag import
* style: dag-pb → UnixFS, 1MB → 1MiB

Co-authored-by: Marcin Rataj <lidel@lidel.org>
2022-03-11 23:23:11 +01:00
Hector Sanjuan
7bb3bfbc33 Bubble ErrNotFound improvements 2022-03-02 18:29:29 +01:00
Adin Schmahmann
52c177ced9
feat: go-libp2p 0.16, UnixFS autosharding and go-datastore with contexts (#8563)
* plumb through go-datastore context changes

* update go-libp2p to v0.16.0
* use LIBP2P_TCP_REUSEPORT instead of IPFS_REUSEPORT
* use relay config
* making deprecation notice match the go-ipfs-config key
* docs(config): circuit relay v2
* docs(config): fix links and headers
* feat(config): Internal.Libp2pForceReachability

This switches to config that supports setting and reading
Internal.Libp2pForceReachability OptionalString flag

* use configuration option for static relays

* chore: go-ipfs-config v0.18.0

https://github.com/ipfs/go-ipfs-config/releases/tag/v0.18.0

* feat: circuit v1 migration prompt when Swarm.EnableRelayHop is set (#8559)
* exit when Swarm.EnableRelayHop is set
* docs: Experimental.ShardingEnabled migration

This ensures existing users of global sharding experiment get notified
that the flag no longer works + that autosharding happens automatically.

For people who NEED to keep the old behavior (eg. have no time to
migrate today) there is a note about restoring it with
`UnixFSShardingSizeThreshold`.

* chore: add dag-jose code to the cid command output

* add support for setting automatic unixfs sharding threshold from the config
* test: have tests use low cutoff for sharding to mimic old behavior
* test: change error message to match the current error
* test: Add automatic sharding/unsharding tests (#8547)
* test: refactored naming in the sharding sharness tests to make more sense

* ci: set interop test executor to convenience image for Go1.16 + Node
* ci: use interop master

Co-authored-by: Marcin Rataj <lidel@lidel.org>
Co-authored-by: Marten Seemann <martenseemann@gmail.com>
Co-authored-by: Marcin Rataj <lidel@lidel.org>
Co-authored-by: Gus Eggert <gus@gus.dev>
Co-authored-by: Lucas Molas <schomatis@gmail.com>
2021-11-29 19:58:05 +01:00
Rod Vagg
9e8b6e5b50
chore: replace go-merkledag walk with go-ipld-prime traversal for dag export (#8506)
* chore: replace go-merkledag walk with go-ipld-prime traversal for dag export

This is "safe" now because we can limit duplicate block loads like
go-merkledag does and won't get trapped taking a long time for complex
DAGs. We can do this while we're using an exhaustive selector (like
ExploreAll here) but will need an alternative strategy when we go for
arbitrary selectors.
2021-10-26 14:07:31 -04:00
Adin Schmahmann
56a2ae92dc change ipfs dag get flag name from format to output-codec 2021-09-27 16:15:49 -04:00
Adin Schmahmann
b83651b220 remove dag put option shortcuts 2021-09-27 14:39:03 -04:00
Adin Schmahmann
2920679056 change names of ipfs dag put flags to make changes clearer 2021-09-27 14:39:03 -04:00
Rod Vagg
0057199af4
feat: dag import --stats (#8237)
* feat: report block count on `dag import`
* fix: clean-up dag import message format
* Only print stats when --stats flag is passed

This applies to both text and json output encoding.

- Stats data is now contained within a Stats datastructure
- Stats are printed after root so that first line of output is the same as previously, even when stats are output using --stats

* fix sharness test

* Add PayloadBytesCount to stats

* Attempt to stabilize flaky tests

* Rename PayloadBytesCount to BlockBytesCount

* Correctly calculate size or imported dag

* Use RawSize of original block for import bytes calc

* test: dag import without --stats

basic regression tests for the default output (text and json)

Co-authored-by: gammazero <gammazero@users.noreply.github.com>
Co-authored-by: Marcin Rataj <lidel@lidel.org>
2021-09-23 16:23:57 +02:00
Hannah Howard
f63a997c35
IPLD Prime In IPFS: Target Merge Branch (#7976)
* feat: switch to using go-ipld-prime for codecs, path resolution, and the `dag put/get` commands
* fix: `dag put/get` not roundtripping due to an extra new line being added (https://github.com/ipfs/go-ipfs/issues/3503)

More detailed information is in the CHANGELOG.md file. Very high level:
* IPLD codecs (and their plugins) must use go-ipld-prime
* Added support for the dag-json codec
* `dag get/put` use IPLD codec names from the multicodec table
* `dag get` defaults to dag-json output instead of json, but may output with other codecs
* Data model pathing can be achieved using the /ipld prefix. For example, you can use `/ipld/QmFoo/Links/0/Hash` to traverse through a DagPB node
* With `dag get/put` the DagPB field names have been changed to match the ones in the protobuf listed in the specification

Co-authored-by: hannahhoward <hannah@hannahhoward.net>
Co-authored-by: Daniel Martí <mvdan@mvdan.cc>
Co-authored-by: acruikshank <acruikshank@example.com>
Co-authored-by: Steven Allen <steven@stebalien.com>
Co-authored-by: Will Scott <will.scott@protocol.ai>
Co-authored-by: Will Scott <will@cypherpunk.email>
Co-authored-by: Rod Vagg <rod@vagg.org>
Co-authored-by: Adin Schmahmann <adin.schmahmann@gmail.com>
Co-authored-by: Eric Myhre <hash@exultant.us>
2021-08-17 13:32:49 -04:00
Marcin Rataj
0ff1cf33aa
docs: cosmetic fixes of help text
Unfortunately, API docs are genereted from go-ipfs sources
via https://github.com/ipfs/http-api-docs
so it all gets lost when API docs for new version are re-generated.

This re-applies manual fixes from:
https://docs.ipfs.io/reference/http/api/
so the next time http-api-docs are re-generated, those changes stay.
2021-03-24 17:41:14 +01:00
Will Scott
6e2b166773
split core/commands/dag into individual files for different subcommands 2021-03-10 12:14:50 -08:00
Adin Schmahmann
6e82b53f03
Merge pull request #7753 from ipfs/fix/dag-import-export-coreapi
fix: ipfs dag export uses the CoreAPI and respects the offline flag
2020-11-24 13:14:53 -05:00
Andrew Nesbitt
37a637e8c1
Fix typo in ipfs dag stat command 2020-11-11 13:04:52 +00:00
Adin Schmahmann
9c5304a21d fix: ipfs dag export uses the CoreAPI and respects the offline flag 2020-11-04 21:14:00 -05:00
Fazlul Shahriar
cac753bb18
fix build on Plan 9
Fixes #7575
Fixes #7671
2020-09-23 22:27:01 -04:00
Adin Schmahmann
8ae5aa56d1
add ipfs dag stat command (#7553)
* commands: add ipfs dag stat command
* sharness: add ipfs dag stat tests
2020-08-17 14:29:36 -04:00
Dimitris Apostolou
1e437c7e97
Fix typos and cleanup 2020-04-20 22:00:01 +03:00
Peter Rabbitson
fbe40883e6 Remove PresentInImport tracking as per conversation 2020-04-08 14:54:49 -07:00
Peter Rabbitson
90c58d6cac Dag import functionality only ( no progress )
This still works over "loosely defined" .car files
Please refer to the sharness tests for extra info

We can tighten this up if the sentiment is "Postel was wrong"
2020-04-08 14:54:49 -07:00
Peter Rabbitson
4772ca6134 Add simple byte-counting export progress-bar 2020-04-08 14:54:49 -07:00
Peter Rabbitson
ae5a9e0549 Bring back pipe closure from other end 2020-04-08 14:54:49 -07:00
Peter Rabbitson
380bffa146 Move commented out code together 2020-04-08 14:54:49 -07:00
Peter Rabbitson
73bb0e8353 Check against error veriable 2020-04-08 14:54:49 -07:00
Peter Rabbitson
c2f9849a2b Fastify with go-car version taking a session 2020-04-08 14:54:49 -07:00
Peter Rabbitson
f129f24bd7 Bring back error-prettifyer removed in f6b707d85f 2020-04-08 14:54:49 -07:00
Peter Rabbitson
3dca025f05 Remove pipe close since we are already in error 2020-04-08 14:54:49 -07:00
Peter Rabbitson
7a749ea7cb Less confusing CLI helptext 2020-04-08 14:54:49 -07:00
Peter Rabbitson
a5f19c4f9c Remove unnecessay prefetch 2020-04-08 14:54:49 -07:00
Peter Rabbitson
b0a461ca5c Typo fix 2020-04-08 14:54:49 -07:00
Peter Rabbitson
635e333d60 Dag export command, silent, no progress 2020-04-08 14:54:49 -07:00
Steven Allen
5f64b270f8 cmdkit -> cmds
License: MIT
Signed-off-by: Steven Allen <steven@stebalien.com>
2019-05-10 23:00:20 -07:00
Łukasz Magiera
21c6e076f5 Update coreiface
License: MIT
Signed-off-by: Łukasz Magiera <magik6k@gmail.com>
2019-04-17 12:04:25 -07:00
Łukasz Magiera
a54b64bede coreiface: updates for path name refactor
License: MIT
Signed-off-by: Łukasz Magiera <magik6k@gmail.com>
2019-04-17 12:03:37 -07:00
Łukasz Magiera
667b7f9927 coreiface: updates for moving path to subpackage
License: MIT
Signed-off-by: Łukasz Magiera <magik6k@gmail.com>
2019-04-17 12:03:37 -07:00
Łukasz Magiera
2e77df04ca coreapi: Update path error handling
License: MIT
Signed-off-by: Łukasz Magiera <magik6k@gmail.com>
2019-04-17 12:03:37 -07:00
Jakub Sztandera
42e191c017 gx: unrewrite
License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protonmail.ch>
2019-03-05 18:33:56 +01:00
Steven Allen
3fa1bfe1bc gx: update cmds and flatfs
fixes #6028, fixes crash when writing after closing on flatfs.

License: MIT
Signed-off-by: Steven Allen <steven@stebalien.com>
2019-03-01 12:17:49 -08:00
Steven Allen
f227862e88 gx: update go-ipfs-cmds, go-bitswap, go-libp2p-kad-dht, and go-mplex
Fixes the latest batch of bugs found in RC testing.

License: MIT
Signed-off-by: Steven Allen <steven@stebalien.com>
2019-02-27 12:39:24 -08:00
Hector Sanjuan
6972a9aa4d Gx Bubble. libp2p-6.0.38
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2019-02-27 01:10:59 +00:00
Steven Allen
3c2536dd81 gx: update go-ipfs-cmds
fixes #6021

License: MIT
Signed-off-by: Steven Allen <steven@stebalien.com>
2019-02-25 16:45:43 -07:00
Steven Allen
fea7ae727f gx: update go-cid
License: MIT
Signed-off-by: Steven Allen <steven@stebalien.com>
2019-02-20 20:29:06 -08:00
Steven Allen
f924f57c61 gx: update go-bitswap and go-libp2p-kad-dht
* go-bitswap: fix some race conditions.
* go-libp2p-kad-dht: fix a goroutine leak.

License: MIT
Signed-off-by: Steven Allen <steven@stebalien.com>
2019-02-20 17:19:54 -08:00
Steven Allen
2f17b951c2 gx: update deps
* Updates go-ipfs-cmds to try to get the tests to pass on travis.
* While we're at it, fix duplicate gx deps.

License: MIT
Signed-off-by: Steven Allen <steven@stebalien.com>
2019-02-19 13:12:21 -08:00
Steven Allen
91ca5911be coreapi: return coreiface.ErrNotSupported when "catting" symlinks.
License: MIT
Signed-off-by: Steven Allen <steven@stebalien.com>
2019-02-19 03:48:04 -08:00
Jakub Sztandera
28cf3de0f9 Update protobuf
License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protonmail.ch>
2019-02-18 20:46:22 +01:00
Steven Allen
cf0d4706e2 gx: update libp2p stuff
License: MIT
Signed-off-by: Steven Allen <steven@stebalien.com>
2019-02-14 14:58:35 -08:00
Steven Allen
e97a60b073 gx: update go-ipfs-files
fix compatibility issue with js-ipfs

License: MIT
Signed-off-by: Steven Allen <steven@stebalien.com>
2019-02-11 10:48:58 -08:00