Commit Graph

156 Commits

Author SHA1 Message Date
Jakub Sztandera
51bf1b6cd4 Significanly improve GC UX with verifcid
License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protonmail.ch>
2018-03-04 01:43:42 +01:00
Hector Sanjuan
758e00bc4c Extract blocks/blockstore package to go-ipfs-blockstore
This extracts the blocks/blockstore package and renames the
blocks/blockstore/util package to /blocks/blockstoreutil
(because util depends on Pin and I don't plan to extract
Pin and its depedencies).

The history of blocks/blockstore has been preserved. It has
been gx'ed and imported. Imports have been rewritten accordingly
and re-ordered.

License: MIT
Signed-off-by: Hector Sanjuan <hector@protocol.ai>
2018-02-15 18:03:41 +01:00
Hector Sanjuan
d7cd059165 Doc: golint: remove stuttering in pin package
License: MIT
Signed-off-by: Hector Sanjuan <hector@protocol.ai>
2018-02-13 13:00:48 +01:00
Hector Sanjuan
ee7a8f3482 Docs: golint-ify pin package
License: MIT
Signed-off-by: Hector Sanjuan <hector@protocol.ai>
2018-02-13 12:53:20 +01:00
Whyrusleeping
52301ce841
Merge pull request #4578 from ipfs/feat/datastore-gc
Make repo gc call CollectGarbage on datastore
2018-02-03 23:38:08 -08:00
Łukasz Magiera
04e43b855b make repo gc call CollectGarbage on datastore
License: MIT
Signed-off-by: Łukasz Magiera <magik6k@gmail.com>
2018-02-04 01:16:06 +01:00
Łukasz Magiera
f1c4df8999 fix pin test on windows
License: MIT
Signed-off-by: Łukasz Magiera <magik6k@gmail.com>
2018-02-04 01:10:43 +01:00
Steven Allen
b2cbfd299c gx: update go-log
License: MIT
Signed-off-by: Steven Allen <steven@stebalien.com>
2018-01-31 21:54:22 -08:00
Steven Allen
f9d935b984 rename import of go-ipld-format from node/format to ipld
License: MIT
Signed-off-by: Steven Allen <steven@stebalien.com>
2018-01-29 11:55:34 -08:00
Steven Allen
d154b4a990 merkledag: switch to new dag interface
Also:

* Update the blockstore/blockservice methods to match.
* Construct a new temporary offline dag instead of having a
  GetOfflineLinkService method.

License: MIT
Signed-off-by: Steven Allen <steven@stebalien.com>
2018-01-25 15:13:19 -08:00
Steven Allen
fe8846fcd7 gx: mass update
License: MIT
Signed-off-by: Steven Allen <steven@stebalien.com>
2018-01-24 15:58:44 -08:00
ForrestWeston
83ccc7e44b docs for pin method impls
License: MIT
Signed-off-by: ForrestWeston <forrest@protocol.ai>
2018-01-22 16:25:00 -08:00
Jeromy
ce57a76687 interface docs for pinner
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
2018-01-21 11:31:47 -08:00
Steven Allen
dfe7ef4fcf gx: update go-multihash
License: MIT
Signed-off-by: Steven Allen <steven@stebalien.com>
2017-12-14 13:09:51 -08:00
Steven Allen
953b610470 add node to dagserv under lock
Otherwise, we could run a GC between adding and pinning.

License: MIT
Signed-off-by: Steven Allen <steven@stebalien.com>
2017-12-03 19:02:09 -08:00
Łukasz Magiera
6401a9191e gx: Update go-datastore to 1.4.0
License: MIT
Signed-off-by: Łukasz Magiera <magik6k@gmail.com>
2017-12-02 14:55:26 -08:00
Steven Allen
70d6629940 gx: update go-cid, go-multibase, base32
License: MIT
Signed-off-by: Steven Allen <steven@stebalien.com>
2017-09-01 17:46:49 -07:00
Jakub Sztandera
5e3fd5e708
gc: add events for profiling GC
License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protonmail.ch>
2017-08-09 23:18:50 +02:00
Jeromy
cccc6a94a3 update go-multihash and bubble up changes
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
2017-07-11 19:27:43 -07:00
Łukasz Magiera
8ab93aaeb2 Update go-datastore to 1.2.2, go-cid to 0.7.16
License: MIT
Signed-off-by: Łukasz Magiera <magik6k@gmail.com>
2017-07-04 20:18:57 +02:00
Łukasz Magiera
4b8a9a3f9f Update go-datastore to 1.2.1
License: MIT
Signed-off-by: Łukasz Magiera <magik6k@gmail.com>
2017-07-03 20:17:03 +02:00
Steven Allen
13636bef50 blocks: gx import go-block-format
And updated related dependencies.

License: MIT
Signed-off-by: Steven Allen <steven@stebalien.com>
2017-06-29 22:12:14 +02:00
zramsay
c5df8f0796 apply the megacheck tool to improve code quality
License: MIT
Signed-off-by: Zach Ramsay <zach.ramsay@gmail.com>
2017-05-31 16:56:11 -04:00
Jeromy
0d3a86e8fc address code review, add comments
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
2017-05-17 22:09:33 -07:00
Jeromy
315849639d implement ipfs pin update
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
2017-05-17 21:31:47 -07:00
Jeromy
b15470d548
bubble up updates from go-multihash changes
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
2017-04-20 20:28:04 +02:00
Jakub Sztandera
6546aeade7
fix: multiple govet warnings
License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protonmail.ch>
2017-03-24 01:02:50 +01:00
Jeromy Johnson
4e2e537caa Merge pull request #3809 from ipfs/faster-pin-ls
Fix #3783: Improve IsPinned() lookups for indirect pins
2017-03-22 20:54:51 -07:00
Hector Sanjuan
71a7a90f62 Fix #3783: Improve IsPinned() lookups for indirect pins
This avoids revisiting already-searched branches and cut down
the IsPinned() check times considerably when recursive pins
share big underlying DAGs.

A test has been added which double-checks that pinned and unpinned items
lookups respond as expected with shared branches.

License: MIT
Signed-off-by: Hector Sanjuan <hector@protocol.ai>
2017-03-22 20:29:11 +01:00
Kevin Atkinson
d39d9ed60b gc: address CR comments
License: MIT
Signed-off-by: Kevin Atkinson <k@kevina.org>
2017-03-20 14:46:02 -04:00
Kevin Atkinson
96f8c17b0f gc: return Result instead of two channels
License: MIT
Signed-off-by: Kevin Atkinson <k@kevina.org>
2017-03-06 14:26:14 -05:00
Kevin Atkinson
f1938f3cbd gc: output all errors to a channel
Errors from ColoredSet are now reported as encountered and errors
encountered when deleting blocks are no longer ignored.

License: MIT
Signed-off-by: Kevin Atkinson <k@kevina.org>

gc: report errors from ColoredSet as encountered

License: MIT
Signed-off-by: Kevin Atkinson <k@kevina.org>
2017-03-06 14:26:09 -05:00
Kevin Atkinson
70eccb03e0 gc: collect all errors during ColoredSet phase
License: MIT
Signed-off-by: Kevin Atkinson <k@kevina.org>
2017-03-06 14:24:47 -05:00
Jeromy
2884c84343 Implement basic filestore 'no-copy' functionality
License: MIT
Signed-off-by: Jeromy <why@ipfs.io>
2017-03-06 00:37:24 -08:00
Kevin Atkinson
6e4fd937fd Refactor EnumerateChildren to avoid need for bestEffort parameter.
License: MIT
Signed-off-by: Kevin Atkinson <k@kevina.org>
2017-03-02 05:19:41 -05:00
Jeromy
b4eeff2d84 update go-multihash and bubble up deps
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
2017-02-12 12:25:41 -08:00
Jeromy Johnson
4028e89016 Merge pull request #3640 from ipfs/fix/pinset-obj-explosion
Make pinset sharding deterministic
2017-02-12 12:05:32 -08:00
Jakub Sztandera
73a711b2df
make: revert the panic change in pin
License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protonmail.ch>
2017-02-12 01:18:40 +01:00
Jakub Sztandera
f630222580
make: rework makefiles for non-recursive make and add sharness coverage
This commit introduces non-recursive Makefile infrastructure that replaces current Makefile infrastructure.
It also generally cleanups the Makefiles, separates them into nicer sub-modules and centralizes common operations into single definitions.

It allows to depend on any target that is defined in the makefile, this means that for example `gx install` is called once when `make build test_expensive_sharness` is called instead of 4 or 5 times.

It also makes the dependencies much cleaner and allows for reuse of modules. For example sharness coverage collection (WIP) uses sharness target with amended PATH, previously it might have been possible but not without wiring in the coverage collection into sharness make runner code.

Yes, it is more complex but not much more. There are few rules that have to be followed and few complexities added but IMHO it is worth it.

How to NR-make:
1. If make is to generate some file via a target, it MUST be defined in Rules.mk file in the directory of the target.
2. `Rules.mk` file MUST have `include mk/header.mk` statement as the first line and `include mk/footer.mk` statement as the last line (apart from project root `Rules.mk`).
3. It then MUST be included by the closest `Rules.mk` file up the directory tree.
4. Inside a `Rules.mk` special variable accessed as `$(d)` is defined. Its value is current directory, use it so if the `Rules.mk` file is moved in the tree it still works without a problem. Caution: this variable is not available in the recipe part and MUST NOT be used. Use name of the target or prerequisite to extract it if you need it.
5. Make has only one global scope, this means that name conflicts are a thing. Names SHOULD  follow `VAR_NAME_$(d)` convention. There are exceptions from this rule in form of well defined global variables. Examples: General lists `TGT_BIN`, `CLEAN`; General targets: `TEST`, `COVERAGE`; General variables: `GOFLAGS`, `DEPS_GO`.
3. Any rules, definitions or variables that fit some family SHOULD be defined in `mk/$family.mk` file and included from project root `Rules.mk`

License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protonmail.ch>
2017-02-12 01:18:40 +01:00
Jeromy
728ff6dd21 Make pinset sharding deterministic
Making this deterministic keeps us from creating an exponential amount
of objects as the number of pins in the set increases.

License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
2017-01-27 22:04:26 -08:00
Jeromy
53d47669da bubble up go-datastore deps
License: MIT
Signed-off-by: Jeromy <why@ipfs.io>
2016-11-28 22:29:38 -08:00
Jeromy
f0f88adea0 completely remove go-key dep
License: MIT
Signed-off-by: Jeromy <why@ipfs.io>
2016-11-28 21:42:32 -08:00
Jeromy
396c629301 update to newer ipld node interface with Copy and better Tree
License: MIT
Signed-off-by: Jeromy <why@ipfs.io>
2016-11-15 18:00:49 -08:00
Jeromy
eac13abaec update to new cid and ipld node packages
License: MIT
Signed-off-by: Jeromy <why@ipfs.io>
2016-10-25 09:54:04 -07:00
Jeromy
48f7e14277 extract node interface
License: MIT
Signed-off-by: Jeromy <why@ipfs.io>
2016-10-14 07:53:48 -07:00
Jeromy
01aee44679 merkledag: change 'Node' to be an interface
Also change existing 'Node' type to 'ProtoNode' and use that most
everywhere for now. As we move forward with the integration we will try
and use the Node interface in more places that we're currently using
ProtoNode.

License: MIT
Signed-off-by: Jeromy <why@ipfs.io>
2016-10-12 08:16:03 -07:00
Jeromy
f4d7369c4a bitswap: protocol extension to handle cids
This change adds the /ipfs/bitswap/1.1.0 protocol. The new protocol
adds a 'payload' field to the protobuf message and deprecates the
existing 'blocks' field. The 'payload' field is an array of pairs of cid
prefixes and block data. The cid prefixes are used to ensure the correct
codecs and hash functions are used to handle the block on the receiving
end.

License: MIT
Signed-off-by: Jeromy <why@ipfs.io>
2016-10-10 08:19:31 -07:00
Jeromy Johnson
7962903a2a Merge pull request #3290 from ipfs/feat/el-cid-2
The conquest of El Cid, Pt. 2
2016-10-10 06:37:00 -07:00
Jeromy Johnson
391b78a23c Merge pull request #3273 from ipfs/fix/pin-fail
fix bug in pinsets and add a stress test for the scenario
2016-10-08 17:09:06 -07:00
Jeromy
282bdc4816 cid: integrate cid into bitswap and blockstores
License: MIT
Signed-off-by: Jeromy <why@ipfs.io>
2016-10-08 12:14:58 -07:00