Commit Graph

422 Commits

Author SHA1 Message Date
Gus Eggert
f855bfe6ef
feat: add basic gateway tracing (#8595)
* add deprecation warning when tracer plugins are loaded
* add response format attribute to span in gateway handler
* add note about tracing's experimental status in godoc
* add nil check for TTL when adding name span attrs
* add basic sharness test for integration with otel collector
* add nil check in UnixFSAPI.processLink
* test: sharness check all json objs for swarm span
* add env var docs to docs/environment-variables.md
* chore: pin the otel collector version
* add tracing spans per response type (#8841)
* docs: tracing with jaeger-ui

Co-authored-by: Marcin Rataj <lidel@lidel.org>
2022-04-04 19:24:05 +02:00
Jorropo
09aed2d63b
Merge pull request #8757 from ipfs/feat/err-not-found
Bubble ErrNotFound improvements.

The is a really nice UX improvement because it know tell you which ipld block is missing instead of a generic "not found".
2022-03-18 00:42:40 +01:00
Laurent Senta
8d549f03f3 fix: rewrite dependencies over the go-ipfs-config package 2022-03-04 09:45:16 +01:00
Hector Sanjuan
7bb3bfbc33 Bubble ErrNotFound improvements 2022-03-02 18:29:29 +01:00
Adin Schmahmann
52c177ced9
feat: go-libp2p 0.16, UnixFS autosharding and go-datastore with contexts (#8563)
* plumb through go-datastore context changes

* update go-libp2p to v0.16.0
* use LIBP2P_TCP_REUSEPORT instead of IPFS_REUSEPORT
* use relay config
* making deprecation notice match the go-ipfs-config key
* docs(config): circuit relay v2
* docs(config): fix links and headers
* feat(config): Internal.Libp2pForceReachability

This switches to config that supports setting and reading
Internal.Libp2pForceReachability OptionalString flag

* use configuration option for static relays

* chore: go-ipfs-config v0.18.0

https://github.com/ipfs/go-ipfs-config/releases/tag/v0.18.0

* feat: circuit v1 migration prompt when Swarm.EnableRelayHop is set (#8559)
* exit when Swarm.EnableRelayHop is set
* docs: Experimental.ShardingEnabled migration

This ensures existing users of global sharding experiment get notified
that the flag no longer works + that autosharding happens automatically.

For people who NEED to keep the old behavior (eg. have no time to
migrate today) there is a note about restoring it with
`UnixFSShardingSizeThreshold`.

* chore: add dag-jose code to the cid command output

* add support for setting automatic unixfs sharding threshold from the config
* test: have tests use low cutoff for sharding to mimic old behavior
* test: change error message to match the current error
* test: Add automatic sharding/unsharding tests (#8547)
* test: refactored naming in the sharding sharness tests to make more sense

* ci: set interop test executor to convenience image for Go1.16 + Node
* ci: use interop master

Co-authored-by: Marcin Rataj <lidel@lidel.org>
Co-authored-by: Marten Seemann <martenseemann@gmail.com>
Co-authored-by: Marcin Rataj <lidel@lidel.org>
Co-authored-by: Gus Eggert <gus@gus.dev>
Co-authored-by: Lucas Molas <schomatis@gmail.com>
2021-11-29 19:58:05 +01:00
Steven Allen
9f03f1ff14 fix(unixfs): check for errors before dereferencing the link
If there's an error, the link will be nil, and this will panic.
2021-10-12 12:39:39 -07:00
Adin Schmahmann
6bf4429082 test: check behavior of loading UnixFS sharded directories with missing shards 2021-09-27 15:50:23 -04:00
Hannah Howard
f63a997c35
IPLD Prime In IPFS: Target Merge Branch (#7976)
* feat: switch to using go-ipld-prime for codecs, path resolution, and the `dag put/get` commands
* fix: `dag put/get` not roundtripping due to an extra new line being added (https://github.com/ipfs/go-ipfs/issues/3503)

More detailed information is in the CHANGELOG.md file. Very high level:
* IPLD codecs (and their plugins) must use go-ipld-prime
* Added support for the dag-json codec
* `dag get/put` use IPLD codec names from the multicodec table
* `dag get` defaults to dag-json output instead of json, but may output with other codecs
* Data model pathing can be achieved using the /ipld prefix. For example, you can use `/ipld/QmFoo/Links/0/Hash` to traverse through a DagPB node
* With `dag get/put` the DagPB field names have been changed to match the ones in the protobuf listed in the specification

Co-authored-by: hannahhoward <hannah@hannahhoward.net>
Co-authored-by: Daniel Martí <mvdan@mvdan.cc>
Co-authored-by: acruikshank <acruikshank@example.com>
Co-authored-by: Steven Allen <steven@stebalien.com>
Co-authored-by: Will Scott <will.scott@protocol.ai>
Co-authored-by: Will Scott <will@cypherpunk.email>
Co-authored-by: Rod Vagg <rod@vagg.org>
Co-authored-by: Adin Schmahmann <adin.schmahmann@gmail.com>
Co-authored-by: Eric Myhre <hash@exultant.us>
2021-08-17 13:32:49 -04:00
Steven Allen
34be171765
Merge pull request #8296 from ipfs/fix/deprecated
fix: remove some deprecated calls
2021-08-13 14:43:58 -07:00
Steven Allen
2138798ea5 fix: remove some deprecated calls 2021-08-13 14:09:43 -07:00
vallder
0a6d78c66a perf: set an appropriate capacity 2021-07-05 19:48:59 +07:00
vyzo
eb7456d38f update for the new namesys api 2021-05-10 21:04:57 -07:00
vyzo
543c5da34a explicit construction of DNS resolver 2021-05-10 21:03:27 -07:00
Steven Allen
99309df9e1
Merge pull request #7883 from ipfs/chore/pin-ls-all-faster
Do not fetch recursive pins from pinner unnecessarily
2021-03-29 16:04:37 -07:00
gammazero
fb55f09882 Add comment about reading until channel closed 2021-03-29 15:11:38 -07:00
Hector Sanjuan
3db9551f79 Extract the namesys and the keystore submodules
Namesys is a very useful submodule. Given a ValueStore and a Datastore it can
resolve and publish /ipns/ paths.

This functionality does not need to be sequestered inside go-ipfs as it can
and should be used without IPFS, for example, for implementing lightweight
IPNS publishing services or for resolving /ipns/ paths.

"keystore" extraction was necessary, as there is a dependency to it in
namesys. Keystore is also a useful module by itself within the stack.

Fixes #6537
2021-03-12 14:09:36 -05:00
gammazero
07aa00dfa6 Do not fetch recursive pins from pinner unnecessarily
When fetching all pins, the recursive pins are fetched from the pinner two times.  The second fetch is unnecessary and copies all recursive pins into a slice again.

Additionally, the output channel is now buffered.  This allows the goroutine to exit in the case the pinner returns an error and there is no reader for the output channel.  This might be possible if a canceled context causes the caller to abandon waiting to read the output of Ls().
2021-01-27 10:17:53 -08:00
Steven Allen
7a50636f9a fix: return an error when an unknown object type is passed
fixes #7785
2020-11-30 17:59:37 -08:00
Steven Allen
5cd013ed38 update libp2p for stream closure refactor 2020-11-13 13:59:03 -08:00
Hector Sanjuan
7d95359c3e Fix #7624: Do not fetch dag nodes when checking if a pin exists 2020-08-26 09:13:15 +02:00
Petar Maymounkov
e384b5964a IPNS format keys in b36cid 2020-08-04 00:18:29 -04:00
Adin Schmahmann
19d6230cde Merge branch 'release' into master 2020-06-19 20:04:30 -04:00
Steven Allen
5f19f3bbe8 fix: cancel resolve search context
We probably don't need to do this, but it can't hurt.
2020-06-17 19:43:20 -07:00
Steven Allen
cee60700b9 fix: use bitswap sessions for ipfs refs
This isn't perfect (we only use sessions after resolving the root cid) but it's
better than what we have. The real solution is #7198 so we can use sessions
everywhere.

(cherry picked from commit 62f61c588d)
2020-06-02 17:36:30 -07:00
Steven Allen
0cdde03824
Merge pull request #7389 from ipfs/fix/refs-sessions
fix: use bitswap sessions for ipfs refs
2020-06-02 11:17:12 -07:00
Steven Allen
c58e3e4c7d fix: remove pubsub discovery hack
Pubsub handles this internally now.
2020-05-29 16:42:10 -07:00
Steven Allen
62f61c588d fix: use bitswap sessions for ipfs refs
This isn't perfect (we only use sessions after resolving the root cid) but it's
better than what we have. The real solution is #7198 so we can use sessions
everywhere.
2020-05-28 17:15:56 -07:00
Michael Muré
6388f5a1c2
pin: honor the context more accurately 2020-05-05 14:19:38 +02:00
Michael Muré
6ef190f5a6 pin: implement pin/ls with only CoreApi 2020-05-04 18:02:50 -07:00
Steven Allen
e55c688fca fix: run tests in "dht server" mode
Otherwise, we'd need to wait a delay on start for nodes to start becoming
servers.
2020-04-09 23:07:27 -07:00
Steven Allen
bb08f7fe56 assign public IP addresses for tests that need them 2020-04-09 21:41:42 -07:00
Steven Allen
b5ad3be060 feat: improve key lookup
* Support peer IDs encoded as CIDs.
* Support looking up the identity key by peer ID.
2020-03-23 15:40:48 -07:00
flowed
07cbba7c40 Fix Typos 2019-12-25 03:12:41 +00:00
Adin Schmahmann
3a0d0e817a support async datastores 2019-12-19 00:59:31 -05:00
Steven Allen
8c5c933559 chore: ignore pubsub deprecation warnings 2019-12-05 13:26:46 -05:00
Michael Muré
c3692a5913 extract the pinner to go-ipfs-pinner and dagutils into go-merkledag 2019-12-02 15:45:44 -05:00
Adin Schmahmann
0906d7f3b1 fix (pinning): pin ls traverses all indirect pins. pin ls pin type precedence change - a direct/recursive pin is now labeled as such even if also indirectly pinned. 2019-12-02 13:53:14 -05:00
Michael Muré
097b684b42
pin: add context and error return to most of the Pinner functions 2019-11-18 18:32:48 +01:00
Hucg
bc8a329a74
Update unixfs.go
use sync.Once instead
2019-11-18 17:26:14 +08:00
hucg
6a171a0b61 fix issue 6760, adding with hash-only, high CPU usage. 2019-11-15 14:58:59 +08:00
Steven Allen
ec748a7b5b
Merge pull request #6747 from ipfs/fix/only-hash-leak
fix(coreapi/add): close the fake repo used when adding with hash-only
2019-11-08 10:30:59 +00:00
hucg
6155fd17c1 fix bug 6748
ipfs add whit only hash, don't need to announce cid to other peer
2019-11-08 15:12:42 +08:00
Steven Allen
e9e32f8ac6 fix(coreapi/add): close the fake repo used when adding with hash-only
fixes #6744
2019-11-04 18:05:35 +00:00
Cole Brown
1bcf51e02d Update go-libp2p, fix tests with weak RSA keys 2019-08-23 14:26:20 -07:00
Steven Allen
2c222741d1 fix: update for merkledag API changes 2019-07-25 19:26:06 -07:00
Steven Allen
41f301a3fd fix: parallel walk in gc & pin ls 2019-07-22 16:43:58 -07:00
Steven Allen
9738d81c0c switch to new merkledag walk functions
EnumerateChildrenAsync has been renamed to WalkParallel to reflect the fact
that:

1. It visits the root.
2. It's parallel, not async.

To mirror this change, EnumerateChildren has also been renamed to Walk and now
behaves the same (except that it's not parallel).
2019-07-16 16:34:21 -07:00
Łukasz Magiera
3f7927f1d8 Update imports to go-filestore 2019-07-15 15:52:34 +02:00
Michael Avila
10de165644
Change to construct provider from go-ipfs-provider 2019-07-03 14:22:55 -07:00
Dirk McCormick
4ead108a80 fix: ensure pin rm takes a lock 2019-06-12 14:59:06 -04:00