* feat(cmd): add silent option repo gc command
closes#7129
* test(cmd): add test case for silent option for command repo gc
* fix: no emit on server with --silent
This removes unnecessary send to the client that does not care
Co-authored-by: Marcin Rataj <lidel@lidel.org>
* feat: add warning message when the local node is offline for remote pinning
* feat: check is node is online
* feat: add test in t0700-remotepin.sh
* feat: use grep and fix the test
* fix: print to stdout only when --enc=text
This ensures we don't break JSON produced by --enc=json
Co-authored-by: Marcin Rataj <lidel@lidel.org>
This sharness test provides 2 datastore blocks directly.
One of them (corresponding to an "insecure block") is not a CIDv0 therefore it
needs to be migrated and stored with its raw-multihash ID.
The (raw) CID of the block used in the test remains the same.
The other block needs no changes.
This fixes some tests which expect "refs local" and "repo gc" outputs to match
the CIDs produced when adding data. These operations are now outputting CIDv1-raw
hashes, regardless of the original CIDs used to address those blocks, so some tests
fail.
The fix is usually:
* To use "block stat" to check if a block was correctly gc'ed
* To convert the CIDs to multihash (using cid-fmt) and compare those instead
* Allow pubsub and namesys-pubsub to be enabled via config
Signed-off-by: Joe Holden <jwh@zorins.us>
Co-authored-by: Marcin Rataj <lidel@lidel.org>
Co-authored-by: Adin Schmahmann <adin.schmahmann@gmail.com>
* multibase encoding on pubsub
* emit multibase for json clients
* refactor(pubsub): base64url for all URL args
This makes it easier to reason about.
Also added better helptext to each command explaining how the binary
data is encoded on the wire, and how to process it in userland.
* refactor: remove ndpayload and lenpayload
Those output formats are undocumented and seem to be only used in tests.
This change removes their implementation and replaces it with error
message to use JSON instead.
I also refactored tests to test the --enc=json response format instead
of imaginary one, making tests more useful as they also act as
regression tests for HTTP RPC.
* test(pubsub): go-ipfs-api
Testing against compatible version from
https://github.com/ipfs/go-ipfs-api/pull/255
* refactor: safeTextListEncoder
Making it clear what it does and why
* refactor(pubsub): unify peerids
This ensures `ipfs pubsub sub` returns the same peerids in the `From`
field as `ipfs pubsub peers`.
libp2p already uses base encoding, no need to double wrap or use custom
multibase.
* test(pubsub): go-ipfs-http-client
* refactor(pubsub): make pub command read from a file
We want to send payload in the body as multipart so users can use
existing tools like curl for publishing arbitrary bytes to a topic.
StringArg was created for "one message per line" use case, and if data
has `\n` or `\r\n` byte sequences, it will cause payload to be split. It
is not possible to undo this, because mentioned sequences are lost, so
we are not able to tell if it was `\n` or `\r\n`
We already avoid this problem in `block put` and `dht put` by reading
payload via FileArg which does not mangle binary data and send it as-is.
It feel like `pubsub pub` should be using it in the first place anyway,
so this commit replaces StringArg with FileArg.
This also closes https://github.com/ipfs/go-ipfs/issues/8454
and makes rpc in go-ipfs easier to code against.
* test(pubsub): publishing with line breaks
Making sure we don't see regressions in the future.
Ref. https://github.com/ipfs/go-ipfs/issues/7939
* chore: disable pubsub interop for now
See
344f692d8c
* test: t0322-pubsub-http-rpc.sh
- Adds HTTP RPC regression test that ensures topic is encoded as URL-safe
multibase.
- Moves pubsub tests to live in unique range ./t032x
* fix(ci): js-ipfs with fixed pubsub wire format
uses js-ipfs from https://github.com/ipfs/js-ipfs/pull/3922
until js-ipfs release can ship with dependency on go-ipfs 0.11.0-rc1
Co-authored-by: Marcin Rataj <lidel@lidel.org>
Co-authored-by: Adin Schmahmann <adin.schmahmann@gmail.com>
* plumb through go-datastore context changes
* update go-libp2p to v0.16.0
* use LIBP2P_TCP_REUSEPORT instead of IPFS_REUSEPORT
* use relay config
* making deprecation notice match the go-ipfs-config key
* docs(config): circuit relay v2
* docs(config): fix links and headers
* feat(config): Internal.Libp2pForceReachability
This switches to config that supports setting and reading
Internal.Libp2pForceReachability OptionalString flag
* use configuration option for static relays
* chore: go-ipfs-config v0.18.0
https://github.com/ipfs/go-ipfs-config/releases/tag/v0.18.0
* feat: circuit v1 migration prompt when Swarm.EnableRelayHop is set (#8559)
* exit when Swarm.EnableRelayHop is set
* docs: Experimental.ShardingEnabled migration
This ensures existing users of global sharding experiment get notified
that the flag no longer works + that autosharding happens automatically.
For people who NEED to keep the old behavior (eg. have no time to
migrate today) there is a note about restoring it with
`UnixFSShardingSizeThreshold`.
* chore: add dag-jose code to the cid command output
* add support for setting automatic unixfs sharding threshold from the config
* test: have tests use low cutoff for sharding to mimic old behavior
* test: change error message to match the current error
* test: Add automatic sharding/unsharding tests (#8547)
* test: refactored naming in the sharding sharness tests to make more sense
* ci: set interop test executor to convenience image for Go1.16 + Node
* ci: use interop master
Co-authored-by: Marcin Rataj <lidel@lidel.org>
Co-authored-by: Marten Seemann <martenseemann@gmail.com>
Co-authored-by: Marcin Rataj <lidel@lidel.org>
Co-authored-by: Gus Eggert <gus@gus.dev>
Co-authored-by: Lucas Molas <schomatis@gmail.com>
* fix(cli): object add-link: do not allow blocks over BS limit
* refactor: allow-big-block
- renamed override flag to --allow-big-block
- separate tests for default and override behavior
Co-authored-by: Marcin Rataj <lidel@lidel.org>
* feat: report block count on `dag import`
* fix: clean-up dag import message format
* Only print stats when --stats flag is passed
This applies to both text and json output encoding.
- Stats data is now contained within a Stats datastructure
- Stats are printed after root so that first line of output is the same as previously, even when stats are output using --stats
* fix sharness test
* Add PayloadBytesCount to stats
* Attempt to stabilize flaky tests
* Rename PayloadBytesCount to BlockBytesCount
* Correctly calculate size or imported dag
* Use RawSize of original block for import bytes calc
* test: dag import without --stats
basic regression tests for the default output (text and json)
Co-authored-by: gammazero <gammazero@users.noreply.github.com>
Co-authored-by: Marcin Rataj <lidel@lidel.org>
* Add a transcoder command to multibase
In order to more easily facilitate the conversion
between multibase formats, include a transcode command
to avoid `multibase decode | multibase encode`
* Example code needed go mod tidy
Co-authored-by: gammazero <gammazero@users.noreply.github.com>
This preloads the peerlog plugin in the ipfs binary, but keeps it
disabled by default. To enabled it, set Enabled=true in its config.
The motivation is to simplify building and deploying gateways, and for
them to use binaries that are more similar to release bins.
* feat: extract Bitswap fx initialization to its own file
* chore: bump go-bitswap dependency
* feat: bump go-ipfs-config dependency and utilize the new Internal.Bitswap configuration options. Add documentation around the new OptionalInteger config type as well as the Internal.Bitswap options.
* docs(docs/config.md): move the table of contents towards the top of the document and update it
Co-authored-by: Petar Maymounkov <petarm@gmail.com>
Co-authored-by: Marcin Rataj <lidel@lidel.org>
Co-authored-by: Gus Eggert <877588+guseggert@users.noreply.github.com>
* feat: switch to using go-ipld-prime for codecs, path resolution, and the `dag put/get` commands
* fix: `dag put/get` not roundtripping due to an extra new line being added (https://github.com/ipfs/go-ipfs/issues/3503)
More detailed information is in the CHANGELOG.md file. Very high level:
* IPLD codecs (and their plugins) must use go-ipld-prime
* Added support for the dag-json codec
* `dag get/put` use IPLD codec names from the multicodec table
* `dag get` defaults to dag-json output instead of json, but may output with other codecs
* Data model pathing can be achieved using the /ipld prefix. For example, you can use `/ipld/QmFoo/Links/0/Hash` to traverse through a DagPB node
* With `dag get/put` the DagPB field names have been changed to match the ones in the protobuf listed in the specification
Co-authored-by: hannahhoward <hannah@hannahhoward.net>
Co-authored-by: Daniel Martí <mvdan@mvdan.cc>
Co-authored-by: acruikshank <acruikshank@example.com>
Co-authored-by: Steven Allen <steven@stebalien.com>
Co-authored-by: Will Scott <will.scott@protocol.ai>
Co-authored-by: Will Scott <will@cypherpunk.email>
Co-authored-by: Rod Vagg <rod@vagg.org>
Co-authored-by: Adin Schmahmann <adin.schmahmann@gmail.com>
Co-authored-by: Eric Myhre <hash@exultant.us>
This should replace the "collect-profiles.sh" script and allow users to
easily collect profiles.
At the moment, it just dumps all profiles into a single zip file. It
does this server-side so it's easy fetch them with curl.
In the future, it would be nice to add:
1. Progress indicators (cpu profiles take 30 seconds).
2. An option to specify which profiles to collect.
But we can handle that later.
Unfortunately, this command doesn't produce symbolized svgs as I didn't
want to depend on having a local go compiler.
DOCKER_HOST is a special environment variable of the Docker CLI
so there's risk of creating conflicts whenever attempting to use `docker`
in the test suite; it also makes it difficult to run the same full suite
locally using the same setup since DOCKER_HOST is a fixed variable in
t0700-remotepin.sh
Write a `sharness` test that expects failure documenting the issue
raised in #3503. The issue may get resolved in the refactoring of
`ipfs dag get` command to support go-ipld-prime (e.g. #8171). For now
add a `sharness` test that expects failure. We will then flip the
expectation in the success in the rewriting PR to check if #3503 gets
resolved.
Relates to:
- https://github.com/ipfs/go-ipfs/issues/3503
