kubo/core/commands
Cory Schwartz a43e506d7f
fix: multibase in pubsub http rpc (#8183)
* multibase encoding on pubsub
* emit multibase for json clients
* refactor(pubsub): base64url for all URL args

This makes it easier to reason about.
Also added better helptext to each command explaining how the binary
data is encoded on the wire, and how to process it in userland.

* refactor: remove ndpayload and lenpayload

Those output formats are undocumented and seem to be only used in tests.
This change removes their implementation and replaces it with error
message to use JSON instead.

I also refactored tests to test the --enc=json response format instead
of imaginary one, making tests more useful as they also act as
regression tests for HTTP RPC.

* test(pubsub): go-ipfs-api

Testing against compatible version from
https://github.com/ipfs/go-ipfs-api/pull/255

* refactor: safeTextListEncoder

Making it clear what it does and why

* refactor(pubsub): unify peerids

This ensures `ipfs pubsub sub` returns the same peerids in the `From`
field as `ipfs pubsub peers`.

libp2p already uses base encoding, no need to double wrap or use custom
multibase.

* test(pubsub): go-ipfs-http-client

* refactor(pubsub): make pub command read from a file

We want to send payload in the body as multipart so users can use
existing tools like curl for publishing arbitrary bytes to a topic.

StringArg was created for "one message per line" use case, and if data
has `\n` or `\r\n` byte sequences, it will cause payload to be split. It
is not possible to undo this, because mentioned sequences are lost, so
we are not able to tell if it was `\n` or `\r\n`

We already avoid this problem in `block put` and `dht put` by reading
payload via FileArg which does not mangle binary data and send it as-is.
It feel like `pubsub pub` should be using it in the first place anyway,
so this commit replaces StringArg with FileArg.

This also closes https://github.com/ipfs/go-ipfs/issues/8454
and makes rpc in go-ipfs easier to code against.

* test(pubsub): publishing with line breaks

Making sure we don't see regressions in the future.
Ref. https://github.com/ipfs/go-ipfs/issues/7939

* chore: disable pubsub interop for now

See
344f692d8c

* test: t0322-pubsub-http-rpc.sh

- Adds HTTP RPC regression test that ensures topic is encoded as URL-safe
  multibase.
- Moves pubsub tests to live in unique range ./t032x

* fix(ci):  js-ipfs with  fixed pubsub wire format

uses js-ipfs from https://github.com/ipfs/js-ipfs/pull/3922 
until js-ipfs release can ship with dependency on go-ipfs 0.11.0-rc1

Co-authored-by: Marcin Rataj <lidel@lidel.org>
Co-authored-by: Adin Schmahmann <adin.schmahmann@gmail.com>
2021-11-29 23:06:54 +01:00
..
cmdenv code review changes 2021-01-14 02:29:11 -08:00
dag feat: go-libp2p 0.16, UnixFS autosharding and go-datastore with contexts (#8563) 2021-11-29 19:58:05 +01:00
e Fix typo in an error.go comment: "is adds" -> "adds". 2018-01-17 15:25:54 -08:00
keyencode fix: remove some deprecated calls 2021-08-13 14:09:43 -07:00
name docs: cosmetic fixes of help text 2021-03-24 17:41:14 +01:00
object fix(cli): object add-link: do not allow blocks over BS limit (#8414) 2021-09-28 15:27:15 +02:00
pin feat: go-libp2p 0.16, UnixFS autosharding and go-datastore with contexts (#8563) 2021-11-29 19:58:05 +01:00
unixfs Escape non-printable characters in user output 2020-12-17 18:16:34 -08:00
active.go refactor: get rid of cmdDetails awkwardness 2020-08-04 18:28:40 -04:00
add.go Apply suggestions from code review 2021-10-11 20:36:06 +02:00
bitswap.go fix: move away from deprecated peer ID functions 2019-12-14 19:48:31 +01:00
block.go fix: block put helptext 2019-06-12 12:44:01 -04:00
bootstrap.go chore: deprecate go-ipfs-addr 2019-05-31 23:46:11 -07:00
cat.go fix: migrate from deprecated warning function 2020-01-28 21:20:21 -08:00
cid.go refactor: get rid of cmdDetails awkwardness 2020-08-04 18:28:40 -04:00
commands_test.go feat: multibase transcode command (#8403) 2021-09-21 20:19:40 +02:00
commands.go feat: programmatic shell completions 2021-07-01 08:46:49 -07:00
completion.go feat: programmatic shell completions 2021-07-01 08:46:49 -07:00
config_test.go add remote pinning policy for mfs (#7798) 2021-01-28 18:58:44 -05:00
config.go Fix typo in comment 2021-04-22 20:30:41 +02:00
dht_test.go Fix typos (#8548) 2021-11-17 21:16:06 +01:00
dht.go feat: go-libp2p 0.16, UnixFS autosharding and go-datastore with contexts (#8563) 2021-11-29 19:58:05 +01:00
diag.go feat: add an "ipfs diag profile" command 2021-07-21 14:51:47 -07:00
dns.go explicit construction of DNS resolver 2021-05-10 21:03:27 -07:00
external.go refactor: get rid of cmdDetails awkwardness 2020-08-04 18:28:40 -04:00
extra.go refactor: get rid of cmdDetails awkwardness 2020-08-04 18:28:40 -04:00
files.go Add support for multiple files to ipfs files rm. 2021-11-12 12:07:58 -05:00
filestore.go feat: go-libp2p 0.16, UnixFS autosharding and go-datastore with contexts (#8563) 2021-11-29 19:58:05 +01:00
get_test.go cmdkit -> cmds 2019-05-10 23:00:20 -07:00
get.go chore: switch tar-utils dep to ipfs org 2021-06-22 14:35:55 -07:00
helptext_test.go gx: unrewrite 2019-03-05 18:33:56 +01:00
id.go feat(cli): add daemon option --agent-version-suffix (#8419) 2021-09-21 20:31:08 +02:00
keystore.go refactor: key export requires exact repo ver 2021-05-11 20:11:12 +02:00
log.go refactor: get rid of cmdDetails awkwardness 2020-08-04 18:28:40 -04:00
ls.go Escape non-printable characters in user output 2020-12-17 18:16:34 -08:00
mount_nofuse.go cmdkit -> cmds 2019-05-10 23:00:20 -07:00
mount_unix.go Escape non-printable characters in user output 2020-12-17 18:16:34 -08:00
mount_windows.go cmdkit -> cmds 2019-05-10 23:00:20 -07:00
multibase.go feat: multibase transcode command (#8403) 2021-09-21 20:19:40 +02:00
p2p.go docs: cosmetic fixes of help text 2021-03-24 17:41:14 +01:00
ping.go fix: move away from deprecated peer ID functions 2019-12-14 19:48:31 +01:00
profile.go doc(profile): improve profiling docs 2021-07-21 18:04:11 -07:00
pubsub.go fix: multibase in pubsub http rpc (#8183) 2021-11-29 23:06:54 +01:00
refs.go fix: use bitswap sessions for ipfs refs 2020-05-28 17:15:56 -07:00
repo.go feat: go-libp2p 0.16, UnixFS autosharding and go-datastore with contexts (#8563) 2021-11-29 19:58:05 +01:00
resolve.go Extract the namesys and the keystore submodules 2021-03-12 14:09:36 -05:00
root_test.go pre-populate required arguments from request body 2018-03-18 13:04:25 -07:00
root.go feat: add 'ipfs multibase' commands (#8180) 2021-08-18 16:48:39 -04:00
shutdown.go docs: cosmetic fixes of help text 2021-03-24 17:41:14 +01:00
stat_dht.go integrate experimental AcceleratedDHTClient 2021-05-14 04:31:06 -04:00
stat_provide.go refactor: improved humanNumber and humanSI 2021-05-27 13:35:19 -07:00
stat.go added support for an experimental batched provider system 2021-05-14 01:47:48 -07:00
swarm.go fix: multibase in pubsub http rpc (#8183) 2021-11-29 23:06:54 +01:00
sysdiag.go feat: add an "ipfs diag profile" command 2021-07-21 14:51:47 -07:00
tar.go docs: cosmetic fixes of help text 2021-03-24 17:41:14 +01:00
urlstore.go Update imports to go-filestore 2019-07-15 15:52:34 +02:00
version.go feat(ipfs diag profile): output version info only 2021-07-21 15:50:21 -07:00