This avoids revisiting already-searched branches and cut down
the IsPinned() check times considerably when recursive pins
share big underlying DAGs.
A test has been added which double-checks that pinned and unpinned items
lookups respond as expected with shared branches.
License: MIT
Signed-off-by: Hector Sanjuan <hector@protocol.ai>
Also change existing 'Node' type to 'ProtoNode' and use that most
everywhere for now. As we move forward with the integration we will try
and use the Node interface in more places that we're currently using
ProtoNode.
License: MIT
Signed-off-by: Jeromy <why@ipfs.io>
This change adds the /ipfs/bitswap/1.1.0 protocol. The new protocol
adds a 'payload' field to the protobuf message and deprecates the
existing 'blocks' field. The 'payload' field is an array of pairs of cid
prefixes and block data. The cid prefixes are used to ensure the correct
codecs and hash functions are used to handle the block on the receiving
end.
License: MIT
Signed-off-by: Jeromy <why@ipfs.io>
Author: Kevin Atkinson <k@kevina.org>
Fix EnumerateChildren & hasChild to take a *cid.Cid instead of []*mdag.Link
Author: Jeromy Johnson <why@ipfs.io>
make FetchGraph use a cid
pin: fix TestPinRecursiveFail
License: MIT
Signed-off-by: Jeromy <why@ipfs.io>
License: MIT
Signed-off-by: Kevin Atkinson <k@kevina.org>
For the rest of the packages in util, move them to thirdparty
and update the references. util is gone!
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
dont GC blocks used by pinner
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
comment GC algo
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
add lock to blockstore to prevent GC from eating wanted blocks
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
improve FetchGraph
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
separate interfaces for blockstore and GCBlockstore
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
reintroduce indirect pinning, add enumerateChildren dag method
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
This commit adds a new set of sharness tests for pinning, and addresses
bugs that were pointed out by said tests.
test/sharness: added more pinning tests
Pinning is currently broken. See issue #1051. This commit introduces
a few more pinning tests. These are by no means exhaustive, but
definitely surface the present problems going on. I believe these
tests are correct, but not sure. Pushing them as failing so that
pinning is fixed in this PR.
make pinning and merkledag.Get take contexts
improve 'add' commands usage of pinning
FIXUP: fix 'pin lists look good'
ipfs-pin-stat simple script to help check pinning
This is a simple shell script to help check pinning.
We ought to strive towards making adding commands this easy.
The http api is great and powerful, but our setup right now
gets in the way. Perhaps we can clean up that area.
updated t0081-repo-pinning
- fixed a couple bugs with the tests
- made it a bit clearer (still a lot going on)
- the remaining tests are correct and highlight a problem with
pinning. Namely, that recursive pinning is buggy. At least:
towards the end of the test, $HASH_DIR4 and $HASH_FILE4 should
be pinned indirectly, but they're not. And thus get gc-ed out.
There may be other problems too.
cc @whyrusleeping
fix grep params for context deadline check
fix bugs in pin and pin tests
check for block local before checking recursive pin
All the datastores used by pinners and so on should be mutex
wrapped. One issue with changing all of them from
ds.Datastore -> ds.ThreadSafeDatastore is that we wrap the
incoming ds.ThreadSafeDatastore with other datastores, which
do not implement the interface. Re-wrapping again causes
double locking. (which may be ok..., but...) any ideas?
