Commit Graph

81 Commits

Author SHA1 Message Date
Jeromy
f209763357 blockservice.New doesnt need to return an error
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
2015-08-14 16:25:51 -07:00
Jeromy
4d287bce51 randomly getting a bad data layout shouldnt fail the tests
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
2015-08-10 14:44:06 -07:00
Jeromy
39a23392c1 use rabin fingerprinting for a chunker
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>

implement rabin fingerprinting as a chunker for ipfs

License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>

vendor correctly

License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>

refactor chunking interface a little

License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>

work chunking interface changes up into importer

License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>

move chunker type parsing into its own file in chunk

License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
2015-08-08 15:33:10 -07:00
Jeromy
137c0ac4ac use batching transaction interface from datastore
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
2015-07-09 16:37:30 -07:00
rht
e671ab2f42 Swap all 'crypto/rand' rng in tests with 'math/rand' 2015-06-03 18:12:34 +07:00
Jeromy
ef294431d4 move util.Key into its own package under blocks 2015-06-01 16:10:08 -07:00
Jeromy
a0aa07e39f rename root to last 2015-05-29 13:33:45 -07:00
Jeromy
38479ff1d3 docs 2015-05-28 08:52:20 -07:00
Jeromy
b660f937bd make callback take a node instead of a key 2015-05-28 08:52:20 -07:00
Jeromy
dd928a2b1d change pinning to happen in a callback 2015-05-28 08:52:20 -07:00
Jeromy Johnson
95f01537a3 correct the blocksize calculation for link blocks
update hash for bigfile
2015-05-06 01:04:15 -07:00
Jeromy
0a6b880bee fix for #1008 and other pinning fixes
This commit adds a new set of sharness tests for pinning, and addresses
bugs that were pointed out by said tests.

test/sharness: added more pinning tests

Pinning is currently broken. See issue #1051. This commit introduces
a few more pinning tests. These are by no means exhaustive, but
definitely surface the present problems going on. I believe these
tests are correct, but not sure. Pushing them as failing so that
pinning is fixed in this PR.

make pinning and merkledag.Get take contexts

improve 'add' commands usage of pinning

FIXUP: fix 'pin lists look good'

ipfs-pin-stat simple script to help check pinning

This is a simple shell script to help check pinning.

We ought to strive towards making adding commands this easy.
The http api is great and powerful, but our setup right now
gets in the way. Perhaps we can clean up that area.

updated t0081-repo-pinning

- fixed a couple bugs with the tests
- made it a bit clearer (still a lot going on)
- the remaining tests are correct and highlight a problem with
  pinning. Namely, that recursive pinning is buggy. At least:
  towards the end of the test, $HASH_DIR4 and $HASH_FILE4 should
  be pinned indirectly, but they're not. And thus get gc-ed out.
  There may be other problems too.

cc @whyrusleeping

fix grep params for context deadline check

fix bugs in pin and pin tests

check for block local before checking recursive pin
2015-04-19 23:40:25 -07:00
Ho-Sheng Hsiao
bf22aeec0a Reorged imports from jbenet/go-ipfs to ipfs/go-ipfs
- Modified Godeps/Godeps.json by hand
- [TEST] Updated welcome docs hash to sharness
- [TEST] Updated contact doc
- [TEST] disabled breaking test (t0080-repo refs local)
2015-03-31 12:52:25 -07:00
Jeromy
50f0b06bdf Correct pinning for dagmodifier, and a bunch more tests 2015-03-11 19:38:01 -07:00
Jeromy
63e15abd8f refactor dagmodifier to work with trickledag format 2015-03-11 19:38:01 -07:00
Jeromy
e539024ebb some code cleanup 2015-03-07 15:58:57 -08:00
Jeromy
298305e457 implement append to trickledag 2015-03-07 15:58:57 -08:00
Jeromy
0e9772c57e refactoring unixfs node and importer/helpers to match 2015-03-06 22:12:03 -08:00
Henry
92d08db7a5 rewrote import paths of go.net/context to use golang.org/x/context
- updated go-ctxgroup and goprocess
ctxgroup: AddChildGroup was changed to AddChild. Used in two files:
- p2p/net/mock/mock_net.go
- routing/dht/dht.go

- updated context from hg repo to git
prev. commit in hg was ad01a6fcc8a19d3a4478c836895ffe883bd2ceab. (context: make parentCancelCtx iterative)
represents commit 84f8955a887232b6308d79c68b8db44f64df455c in git repo

- updated context to master (b6fdb7d8a4ccefede406f8fe0f017fb58265054c)

Aaron Jacobs (2):
net/context: Don't accept a context in the DoSomethingSlow example.
context: Be clear that users must cancel the result of WithCancel.

Andrew Gerrand (1):
go.net: use golang.org/x/... import paths

Bryan C. Mills (1):
net/context: Don't leak goroutines in Done example.

Damien Neil (1):
context: fix removal of cancelled timer contexts from parent

David Symonds (2):
context: Fix WithValue example code.
net: add import comments.

Sameer Ajmani (1):
context: fix TestAllocs to account for ints in interfaces
2015-02-25 11:58:19 +01:00
Jeromy
1e93ee00c0 clean up benchmarks, implement WriterTo on DAGReader, and optimize DagReader 2015-02-04 21:59:51 +00:00
Jeromy
414bdc78c0 fix benchmarks 2015-02-04 21:59:51 +00:00
Jeromy
bc79ae17a1 refactor importer package with trickle and balanced dag generation 2015-02-04 21:59:51 +00:00
Jeromy
b3e74fabb5 implement trickledag for faster unixfs operations 2015-02-04 21:59:50 +00:00
Juan Batiz-Benet
58f39687cf logs: removed all log.Errors unhelpful to users
Let's save log.Error for things the user can take action on.
Moved all our diagnostics to log.Debug. We can ideally reduce them
even further.
2015-02-03 01:06:07 -08:00
Jeromy
2eca66e6d2 off by one error seeking to end of single block file 2015-01-27 07:41:51 +00:00
Jeromy
6a6ede85d3 TODO -> Background 2015-01-27 03:27:26 +00:00
Jeromy
ba4ea139b9 address concerns from PR 2015-01-27 01:33:09 +00:00
Jeromy
0023cb3097 refactor and clean up dagreader 2015-01-27 01:33:09 +00:00
Jeromy
26826bd55e implement seeking in the dagreader 2015-01-27 01:33:09 +00:00
Jeromy
1244fd251d update tests, and ensure 'default' blocks are pinned 2015-01-22 00:16:30 +00:00
Juan Batiz-Benet
16690d4af2 race fix: pinner loads with a threadsafe datastore
All the datastores used by pinners and so on should be mutex
wrapped. One issue with changing all of them from
ds.Datastore -> ds.ThreadSafeDatastore is that we wrap the
incoming ds.ThreadSafeDatastore with other datastores, which
do not implement the interface. Re-wrapping again causes
double locking. (which may be ok..., but...) any ideas?
2015-01-14 16:15:38 -08:00
Juan Batiz-Benet
136ef70f00 calc_test for picking link block size 2015-01-10 01:22:45 -08:00
Jeromy
e3cf893616 implement recursive indirect blocks
improve efficiency of multilayered indirect blocks

clean up tests

panic cleanup

clean up logic, improve readability

add final root node to the dagservice upon creation

importer: simplified dag generation

test: updated hashes using latest code

@whyrusleeping this is why the sharness tests
were failing: the hashes are added manually to
make sure our generation doesn't change.

cleanup after CR

fix merkledag tests

fix small block generation (no subblocks!)
2015-01-06 19:43:56 +00:00
Brian Tiger Chow
8e0c8a7a7e refactor(mdag, bserv, bs) mocks, etc.
License: MIT
Signed-off-by: Brian Tiger Chow <brian@perfmode.com>
2014-12-13 05:17:16 -08:00
Brian Tiger Chow
74eb03231d fix(core, peer) helpers to testutil, err handling
License: MIT
Signed-off-by: Brian Tiger Chow <brian@perfmode.com>
2014-12-08 14:32:52 -08:00
Jeromy
d53deebada wire GetBlocks into blockservice 2014-12-05 20:53:26 +00:00
Juan Batiz-Benet
f6235c5cc6 importer: simplified splitter
The splitter is simplified using io.ReadFull, as this function
does exactly what we wanted.

I believe io.ErrUnexpectedEOF should be handled as an EOF here,
but please correct me if I'm wrong.
2014-11-25 06:12:35 -08:00
Juan Batiz-Benet
9ae420abb8 SizeSplitter fix: keep-reading until chunk full
if the underlying reader is buffered with a smaller buffer
it would force the chunk sizes to come out smaller than
intended.

cc @whyrusleeping @mappum
2014-11-18 06:13:07 -08:00
Brian Tiger Chow
ed4a8eb782 chore(tests) add Short() -> SkipNow() to slowest tests
vanilla:
    21.57 real        45.14 user         8.51 sys

short:
    14.40 real        31.13 user         5.56 sys

License: MIT
Signed-off-by: Brian Tiger Chow <brian@perfmode.com>
2014-11-15 02:10:53 -08:00
Jeromy
4f083d54d6 write a few package doc strings to improve look of godoc 2014-11-08 21:42:36 -08:00
Jeromy
88bf39118c comment comment comment comment 2014-11-03 03:02:56 +00:00
Jeromy
027920540f more memory tweaks 2014-11-01 16:07:56 -07:00
Jeromy
317ca2f865 benchmark secure channel 2014-11-01 16:07:56 -07:00
Jeromy
e9779b1c7f address comments from PR 2014-10-31 05:57:47 +00:00
Jeromy
ae339a0320 remove dagwriter in favor of new importer function 2014-10-31 05:03:42 +00:00
Juan Batiz-Benet
37ce1863c0 test splitting is deterministic. (it is) 2014-10-30 02:49:42 -07:00
Jeromy
71ac117546 Address concerns in PR 2014-10-30 06:23:31 +00:00
Jeromy
18ada93ec3 rewrite add command to use dagwriter, moved a pinner into the dagwriter for inline pinning 2014-10-30 04:40:48 +00:00
Juan Batiz-Benet
184c25430b go-vet friendly codebase
- distinguish log.Error and log.Errorf functions
- Initialize structs with field names
- A bit of unreachable code (defers)
2014-10-25 03:46:39 -07:00
Jeromy
093c8fb04d Rework package structure for unixfs and subpackage
cc @jbenet
2014-10-08 21:14:18 +00:00