Commit Graph

92 Commits

Author SHA1 Message Date
Steven Allen
953b610470 add node to dagserv under lock
Otherwise, we could run a GC between adding and pinning.

License: MIT
Signed-off-by: Steven Allen <steven@stebalien.com>
2017-12-03 19:02:09 -08:00
Łukasz Magiera
6401a9191e gx: Update go-datastore to 1.4.0
License: MIT
Signed-off-by: Łukasz Magiera <magik6k@gmail.com>
2017-12-02 14:55:26 -08:00
Steven Allen
70d6629940 gx: update go-cid, go-multibase, base32
License: MIT
Signed-off-by: Steven Allen <steven@stebalien.com>
2017-09-01 17:46:49 -07:00
Jeromy
cccc6a94a3 update go-multihash and bubble up changes
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
2017-07-11 19:27:43 -07:00
Łukasz Magiera
8ab93aaeb2 Update go-datastore to 1.2.2, go-cid to 0.7.16
License: MIT
Signed-off-by: Łukasz Magiera <magik6k@gmail.com>
2017-07-04 20:18:57 +02:00
Łukasz Magiera
4b8a9a3f9f Update go-datastore to 1.2.1
License: MIT
Signed-off-by: Łukasz Magiera <magik6k@gmail.com>
2017-07-03 20:17:03 +02:00
Steven Allen
13636bef50 blocks: gx import go-block-format
And updated related dependencies.

License: MIT
Signed-off-by: Steven Allen <steven@stebalien.com>
2017-06-29 22:12:14 +02:00
zramsay
c5df8f0796 apply the megacheck tool to improve code quality
License: MIT
Signed-off-by: Zach Ramsay <zach.ramsay@gmail.com>
2017-05-31 16:56:11 -04:00
Jeromy
0d3a86e8fc address code review, add comments
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
2017-05-17 22:09:33 -07:00
Jeromy
315849639d implement ipfs pin update
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
2017-05-17 21:31:47 -07:00
Jeromy
b15470d548
bubble up updates from go-multihash changes
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
2017-04-20 20:28:04 +02:00
Hector Sanjuan
71a7a90f62 Fix #3783: Improve IsPinned() lookups for indirect pins
This avoids revisiting already-searched branches and cut down
the IsPinned() check times considerably when recursive pins
share big underlying DAGs.

A test has been added which double-checks that pinned and unpinned items
lookups respond as expected with shared branches.

License: MIT
Signed-off-by: Hector Sanjuan <hector@protocol.ai>
2017-03-22 20:29:11 +01:00
Jeromy
b4eeff2d84 update go-multihash and bubble up deps
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
2017-02-12 12:25:41 -08:00
Jeromy
53d47669da bubble up go-datastore deps
License: MIT
Signed-off-by: Jeromy <why@ipfs.io>
2016-11-28 22:29:38 -08:00
Jeromy
396c629301 update to newer ipld node interface with Copy and better Tree
License: MIT
Signed-off-by: Jeromy <why@ipfs.io>
2016-11-15 18:00:49 -08:00
Jeromy
eac13abaec update to new cid and ipld node packages
License: MIT
Signed-off-by: Jeromy <why@ipfs.io>
2016-10-25 09:54:04 -07:00
Jeromy
48f7e14277 extract node interface
License: MIT
Signed-off-by: Jeromy <why@ipfs.io>
2016-10-14 07:53:48 -07:00
Jeromy
01aee44679 merkledag: change 'Node' to be an interface
Also change existing 'Node' type to 'ProtoNode' and use that most
everywhere for now. As we move forward with the integration we will try
and use the Node interface in more places that we're currently using
ProtoNode.

License: MIT
Signed-off-by: Jeromy <why@ipfs.io>
2016-10-12 08:16:03 -07:00
Jeromy
f4d7369c4a bitswap: protocol extension to handle cids
This change adds the /ipfs/bitswap/1.1.0 protocol. The new protocol
adds a 'payload' field to the protobuf message and deprecates the
existing 'blocks' field. The 'payload' field is an array of pairs of cid
prefixes and block data. The cid prefixes are used to ensure the correct
codecs and hash functions are used to handle the block on the receiving
end.

License: MIT
Signed-off-by: Jeromy <why@ipfs.io>
2016-10-10 08:19:31 -07:00
Kevin Atkinson
772164cc7a Fix EnumerateChildren & hasChild to take a *cid.Cid instead of []*mdag.Link
Author: Kevin Atkinson <k@kevina.org>

Fix EnumerateChildren & hasChild to take a *cid.Cid instead of []*mdag.Link

Author: Jeromy Johnson <why@ipfs.io>

make FetchGraph use a cid

pin: fix TestPinRecursiveFail

License: MIT
Signed-off-by: Jeromy <why@ipfs.io>

License: MIT
Signed-off-by: Kevin Atkinson <k@kevina.org>
2016-10-06 11:55:31 -04:00
Kevin Atkinson
721df367a2 Don't use a separate LinkService for DAGService.GetLinks()
Instead make LinkService a part of DAGService.  The LinkService is now
simply an interface that DAGService implements.  Also provide a
GetOfflineLinkService() method that the GC uses to get an offline
instance.

License: MIT
Signed-off-by: Kevin Atkinson <k@kevina.org>
2016-10-06 11:52:39 -04:00
Kevin Atkinson
3899194cb0 Add DAGService.GetLinks() method and use it in the GC and elsewhere.
This method will use the (also new) LinkService if it is available to
retrieving just the links for a MerkleDAG without necessary having to
retrieve the underlying block.

For now the main benefit is that the pinner will not break when a block
becomes invalid due to a change in the backing file.  This is possible
because the metadata for a block (that includes the Links) is stored
separately and thus always available even if the backing file changes.

License: MIT
Signed-off-by: Kevin Atkinson <k@kevina.org>
2016-10-06 11:38:41 -04:00
Jeromy
1f9ec4e3ed update to libp2p 4.0.1 and propogate other changes
License: MIT
Signed-off-by: Jeromy <why@ipfs.io>
2016-10-05 22:12:43 -07:00
Kevin Atkinson
0a459f65ac Pinner: Provide Pinned.String() method and use it in "block rm"
License: MIT
Signed-off-by: Kevin Atkinson <k@kevina.org>
2016-09-15 04:15:36 -04:00
George Antoniadis
6859b8ccd8 Extract key and datastore
License: MIT
Signed-off-by: George Antoniadis <george@noodles.gr>
2016-09-09 15:52:25 +01:00
Jeromy
c8fe495934 integrate CIDv0
License: MIT
Signed-off-by: Jeromy <why@ipfs.io>
2016-09-07 13:32:32 -07:00
Jeromy
05b197df2d update deps for libp2p 3.4.0
License: MIT
Signed-off-by: Jeromy <why@ipfs.io>
2016-08-25 19:02:55 -07:00
Jeromy
db228e1054 pin: use separate dagservice for storing pinsets
License: MIT
Signed-off-by: Jeromy <why@ipfs.io>
2016-08-19 17:45:49 -07:00
Kevin Atkinson
efeb789878 Check for multiple pinned blocks in a single pass.
Provide a new method, Pinner.CheckIfPinned(), which will check if
any of the arguments are pinned.  Previously IsPinned would need to be
called once for each block.  The new method will speed up the checking
of multiple pinned blocks from O(p*n) to O(p) (where p is the number
of pinned blocks and n is the number of blocks to be check)

Use the new method in "block rm".

License: MIT
Signed-off-by: Kevin Atkinson <k@kevina.org>
2016-08-15 13:17:43 -04:00
Jeromy
e8addf5ee3 use batching datastore for providers storage
License: MIT
Signed-off-by: Jeromy <why@ipfs.io>
2016-07-26 10:48:25 -07:00
Jeromy
0aafa6db8c update go-datastore changes 0.1.2
License: MIT
Signed-off-by: Jeromy <why@ipfs.io>
2016-07-01 22:40:57 -07:00
Jeromy
0782c4dc03 encode keys to datastore with base32 standard encoding
Fixes #2601

Also bump version to 0.4.3-dev

License: MIT
Signed-off-by: Jeromy <why@ipfs.io>
2016-07-01 14:15:06 -07:00
Jakub Sztandera
ce8c8a7a71 Update go-log in whole dependency tree (#2898)
* Update golog in go-ipfs

License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protonmail.ch>

* Update go-libp2p for go-log

License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protonmail.ch>

* Update go-libp2p-secio for go-log

License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protonmail.ch>

* Update go-libp2p-crypto for go-log

License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protonmail.ch>

* Update go-libp2p-peer for go-log

License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protonmail.ch>

* Import peersore, it wasn't imported

License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protonmail.ch>

* Update peerstore

License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protonmail.ch>

* Update peer

License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protonmail.ch>

* Update secio

License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protonmail.ch>

* Update go-libp2p

License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protonmail.ch>
2016-06-24 09:38:07 -07:00
Jakub Sztandera
cafd81c8c6 Import go-datastore to gx
License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protonmail.ch>
2016-06-12 08:47:15 +02:00
Jakub Sztandera
6217e1f141 Remove go-datastore from Godeps
License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protonmail.ch>
2016-06-12 08:23:43 +02:00
Jakub Sztandera
eaccd07505 Update go-log
https://github.com/ipfs/go-log/pull/3

License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protonmail.ch>
2016-06-09 23:37:09 +02:00
Jeromy
7274c6be35 update libp2p to v3.3.1
License: MIT
Signed-off-by: Jeromy <why@ipfs.io>
2016-06-01 16:45:46 -07:00
Christian Couder
39f23677f5 pin: use new constants instead of literal values
License: MIT
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
2016-05-15 18:18:29 +02:00
Christian Couder
c4195e5328 pin: add missing consts and convertion functions
License: MIT
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
2016-05-15 18:15:28 +02:00
Hector Sanjuan
57b494a70c Update go-log to 1.1.0 and fix calls to go-log.Uuid
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2016-05-04 23:17:21 +02:00
Jeromy
5474e15e93 blockstore locks return unlocker object now
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
2016-02-08 14:42:07 -08:00
Jeromy
0e8a6700f9 correct go-log dep
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
2016-01-30 09:34:10 -08:00
Jeromy
0e312f5caf initial vendoring of libp2p outside of the repo with gx
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
2016-01-30 09:34:06 -08:00
Christian Couder
b770c881a7 pin/pin: replace isPinned() with isPinnedWithType()
It is more generic to be able to pass a pin type argument.

License: MIT
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
2016-01-18 21:37:44 +01:00
Jeromy
4d5af93f43 vendor in new go-datastore
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
2016-01-12 08:22:55 -08:00
Tommi Virtanen
b76581d6c7 fsrepo: Refactor to extract datastore internals
License: MIT
Signed-off-by: Tommi Virtanen <tv@eagain.net>
2016-01-12 08:22:55 -08:00
Jeromy
859de51406 dont use searchset for indirect pin checking
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
2016-01-12 08:21:13 -08:00
Jeromy
879cfeeec9 pin rm fails appropriately for indirect pins
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
2016-01-12 08:21:13 -08:00
Jeromy
16c4d8cdcb addressing comments from CR
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
2016-01-12 08:21:13 -08:00
Jeromy
b12ee40aba implement mark and sweep GC
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>

dont GC blocks used by pinner

License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>

comment GC algo

License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>

add lock to blockstore to prevent GC from eating wanted blocks

License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>

improve FetchGraph

License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>

separate interfaces for blockstore and GCBlockstore

License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>

reintroduce indirect pinning, add enumerateChildren dag method

License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
2016-01-12 08:21:13 -08:00