Turns out that `pool.Put(buf)` had to *allocate* because we needed to turn
`[]byte` into `interface{}`. Apparently, we've never done this correctly we just
never noticed because we never really used buffer pools extensively.
However, since migrating yamux to a buffer-pool backed buffer, this started
showing up in allocation profiles.
License: MIT
Signed-off-by: Steven Allen <steven@stebalien.com>
* go-datastore and friends: GetSize
* badger: new release, fewer allocations
* go-mplex: send fewer packets
* go-bitswap: pack multiple blocks in a single message, fewer allocations
* go-buffer-pool: replace the buffer pool from go-msgio
* yamux: fixed data race and uses go-buffer-pool for stream read-buffers to
reduce memory and allocations.
* go-libp2p-secio: get rid of a hot-spot allocation
* go-libp2p-peerstore: reduced allocations (at the cost of some memory)
More?
License: MIT
Signed-off-by: Steven Allen <steven@stebalien.com>
Allows to dynamically change the MutexProfileFraction to enable and
disable mutex profiling. It should be very useful for detecting
deadlocks, lock contention and general concurrency problems.
How to use:
To enable run: curl -X POST -v 'localhost:5001/debug/pprof-mutex/?fraction=10
To disable: curl -X POST -v 'localhost:5001/debug/pprof-mutex/?fraction=0'
Fraction defines which fraction of events will be profiled. Higher it is
the lower performance impact but less reliable the result.
To fetch the result use:
go tool pprof $PATH_TO_IPFS_BIN http://localhost:5001/debug/pprof/mutex
License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protonmail.ch>
excerpt of commit messages:
- update postrun functions in core/commands
- sharness: allow setting -i with TEST_IMMEDIATE=1
- cmds Run func returns error now
- gx update cmdkit to 1.1.2 and cmds to 2.0.0-beta1
License: MIT
Signed-off-by: keks <keks@cryptoscope.co>
Unfortunately, this doesn't really test the *actual* gateway config as it uses
the options specified by the test. However, it's a step in the right direction.
License: MIT
Signed-off-by: Steven Allen <steven@stebalien.com>
* Most of our datastores barf on non []byte values.
* We have to have a bunch of "is this a []byte" checks.
* Saves some allocations.
License: MIT
Signed-off-by: Steven Allen <steven@stebalien.com>
The progressbar should now correctly calculate the size of a directory (by
ignoring the directory sizes).
fixes#5288
License: MIT
Signed-off-by: Steven Allen <steven@stebalien.com>
Using httpServer.Shutdown will:
1. Close the listener (preventing new connections).
2. Close each connection as outstanding requests finish.
This prevent us from shutting down before outstanding requests get a chance to
respond.
fixes#4055
License: MIT
Signed-off-by: Steven Allen <steven@stebalien.com>
Namesys returns `ErrResolveRecursion` when it stops recursing due to a depth
limit. It doesn't return success.
Alternative to #5199.
License: MIT
Signed-off-by: Steven Allen <steven@stebalien.com>
If the domain has a DNS-Link, we want to use it even if it points to, e.g., an
IPNS address that doesn't resolve.
fixes#4973
License: MIT
Signed-off-by: Steven Allen <steven@stebalien.com>
otherwise, we block on required arguments from stdin before processing the help
flag.
fixes#4837
License: MIT
Signed-off-by: Steven Allen <steven@stebalien.com>
This way, we can always assume that indexing a required argument works.
Also:
* test that the command tree doesn't have any obvious bugs (duplicate options,
arguments in the wrong order, etc).
* simplify the usage ParseBodyArgs.
* remove unnecessary check in the get command.
fixes#4823
License: MIT
Signed-off-by: Steven Allen <steven@stebalien.com>
Currently the "path" module does two very different things:
* Defines how ipfs paths look like and provides tools to parse/split etc.
* Provides a resolver to resolve paths.
This moves the resolver stuff to `path/resolver` and leaves the
path utilities in `path`.
The result is that now the IPFS `path` package just defines what a path
looks like and becomes a module that can be exported/re-used without problems.
Currently there are circular dependency cycles (resolve_test -> merkledag/utils,
merkledag->path), which the prevent the export of merkledag itself.
License: MIT
Signed-off-by: Hector Sanjuan <hector@protocol.ai>
Also:
* Update the blockstore/blockservice methods to match.
* Construct a new temporary offline dag instead of having a
GetOfflineLinkService method.
License: MIT
Signed-off-by: Steven Allen <steven@stebalien.com>
- add comments, trim api path prefix
- corehttp: add option to set HTTP header "Server"
- daemon: use new corehttp options
License: MIT
Signed-off-by: keks <keks@cryptoscope.co>
Commits:
namesys: pubsub Publisher and Resolver
namesys/pubsub: pacify code climate.
namesys/pubsub: timeout for rendezvous
namesys/pubsub: filter self in bootstrap connections
namesys/pubsub: Publish to the correct topic
License: MIT
Signed-off-by: vyzo <vyzo@hackzen.org>
namesys/pubsub: unit test
Commits:
namesys/pubsub: test
namesys/pubsub_test: pacify code climate
namesys/pubsub: update test to use extant mock routing
License: MIT
Signed-off-by: vyzo <vyzo@hackzen.org>
namesys/pubsub: integrate namesys pubsub
namesys: integrate pubsub resolvers
namesys/pubsub_test: tweak delays
- trying to make travis happy.
namesys/pubsub: fix duplicate bootstraps
- subscription key is topic, not ipnskey.
namesys/pubsub: no warning needed on cancellation
namesys/pubsub: warning for receive errors
- and more informative error messages at that.
namesys/pubsub_test: smaller test
- make it work with seemingly low fdlimits in travis/macosx.
also, more informative test failures.
namesys/pubsub: add delay to let pubsub perform handshake
namesys/pubsub: update gx imports
namesys/pubsub_test: preconnect publisher, reduce delays
- preconnects the publisher to the receivers in order to avoid bootstrap
flakiness with connectivity problems in travis.
reduces sleeps to 1s for flood propagation (3s seems excessive with 5 hosts).
namesys/pubsub: drop named return values in resolveOnce
- per review comment.
namesys/pubsub: check errors
namesys/pubsub: store bytes in resolver datastore
namesys/pubsub: resolver Cancel
- for canceling subscriptions, pre whyrusleeping's request.
namesys/pubsub: fix resolution without /ipns prefix
- also improve the logging a bit.
namesys/pubsub: don't resolve own keys through pubsub
namesys/pubsub: signal ErrResolveFailed on resolution failure
namesys/pubsub: use sync datastore, resolver lock only for subs
namesys/pubsub_test: coverage for Cancel
License: MIT
Signed-off-by: vyzo <vyzo@hackzen.org>
namesys/pubsub: parallelize dht and pubsub publishing
Commits:
namesys/pubsub: code cosmetics
namesys: parallelize publishing with dht and pubsub
namesys/pubsub: periodically reprovide topic rendezvous
namesys/pubsub: cancelation for rendezvous goroutine
namesys/pubsub: log ipns record seqno on publish
License: MIT
Signed-off-by: vyzo <vyzo@hackzen.org>
namesys/pubsub: error checking
License: MIT
Signed-off-by: vyzo <vyzo@hackzen.org>
namesys/pubsub: --enable-namesys-pubsub option and management
Commits:
package.json: update go-libp2p-blankhost
namesys: fix stale package imports
update go-testutil
namesys/pubsub: reduce bootstrap provide period to 8hr
namesys/pubsub: try to extract the key from id first
option to enable ipns pubsub: --enable-namesys-pubsub
ipfs name pubsub management subcommands
corehttp/gateway_test: mockNamesys needs to implement GetResolver
pacify code climate
License: MIT
Signed-off-by: vyzo <vyzo@hackzen.org>
namesys/pubsub: pubsub sharness test
test/sharness: test for ipns pubsub
namesys/pubsub: return boolean indicator on Cancel
package.json: remove duplicate entry for go-testutil
update gx deps, testutil to 1.1.12
fix jenkins failure: use tabs in t0183-namesys-pubsub
t0183: use 4 spaces for tabification
License: MIT
Signed-off-by: vyzo <vyzo@hackzen.org>
namesys/pubsub: update for new command interface
License: MIT
Signed-off-by: vyzo <vyzo@hackzen.org>
namesys/pubsub: fix sharness test for broken MacOS echo
echo -n "" should print -n, but hey it's a mac.
License: MIT
Signed-off-by: vyzo <vyzo@hackzen.org>
Note: This commit is technically broken. However, I need to make a bunch of
cmds changes to make this work and I'd rather not bundle both changes into a
single commit.
License: MIT
Signed-off-by: Steven Allen <steven@stebalien.com>
Introduces a new Reset method on streams that kills both sides of the
connection. Close now officially just closes the write side (what it did all
along...)
* Also pull through shiny new go-multiplexer fixes.
* Also pull in go-reuseport update.
License: MIT
Signed-off-by: Steven Allen <steven@stebalien.com>
The `Content-Range` header is necessary for extracting (only)
the metadata of audio files.
License: MIT
Signed-off-by: Steven Vandevelde <icid.asset@gmail.com>
This enables `go get` to parse go-import meta tags from index.html
files stored in IPFS. One tiny step toward whyrusleeping/gx-go#2.
For an import like `ipfs.io/ipfs/QmFoo/mypkg`, the gateway would
previously redirect to `/ipfs/QmFoo/mypkg/` (note the trailing slash),
which the `go get` tool can't deal with.
Thankfully, `go get` sets a URL query parameter (`?go-get=1`) which
we can use to switch off the redirect in this case.
License: MIT
Signed-off-by: Lars Gierth <larsg@systemli.org>
This existed before but was disabled in 912a972 because the Suborigin spec
changed and it became incompatible.
This commit updates the generated Suborigin header to be conformant with the
latest spec.
License: MIT
Signed-off-by: James Stanley <james@incoherency.co.uk>
Instead of resolving a node, we resolve a path. This resolved path
is then re-used for Cat and Ls. This way, a resolve operation is
only done once.
The error messages for a failed resolve is changed from `ipfs cat …`
to `ipfs resolve …` to better reflect the API calls. The test is
updated accordingly.
License: MIT
Signed-off-by: Remco Bloemen <remco@2π.com>
* Always use the fully resolved CID from api.ResolveNode
as the ETag (also for IPNS).
* Format the result as a valid "Strong Validator"
(double quotes around the encoded CID).
Fixes#3868
License: MIT
Signed-off-by: Remco Bloemen <remco@2π.com>
The new coreiface.Path maps a path to the cid.Cid
resulting from a full path resolution.
The path is internally represented as a go-ipfs/path.Path,
but that doesn't matter to the outside.
Apart from the path-to-CID mapping, it also aims to hold all
resolved segment CIDs of the path. Right now it only exposes
Root(), and only for flat paths a la /ipfs/Qmfoo. In other cases,
the root is nil.
In the future, resolution will internally use
go-ipfs/path.Resolver.ResolvePathComponents and thus always return
the proper resolved segments, via Root(), or a future Segments() func.
- Add coreiface.Path with Cid() and Root().
- Add CoreAPI.ResolvePath() for getting a coreiface.Path.
- All functions now expect and return coreiface.Path.
- Add ParsePath() and ParseCid() for constructing a coreiface.Path.
- Add coreiface.Node and Link which are simply go-ipld-node.Node and Link.
- Add CoreAPI.ResolveNode() for getting a Node from a Path.
License: MIT
Signed-off-by: Lars Gierth <larsg@systemli.org>
The Suborigins spec was changed and we have to adjust, the spec is still
unstable and it might change in future. Currently the only browser
supporting it (Chrome) errors out on it as it doesn't confront spec it
uses.
See https://github.com/ipfs/specs/issues/131
License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protonmail.ch>
