Commit Graph

91 Commits

Author SHA1 Message Date
Jakub Sztandera
f13506c11a blocks/blockstore: style cleanup of bloomcache
License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protonmail.ch>
2016-07-04 20:34:46 +02:00
Jakub Sztandera
e92e6662a7 test: fix races in bloomcache tests
License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protonmail.ch>
2016-07-04 20:34:07 +02:00
Jakub Sztandera
98f877af81 blocks/blockstorage: use automic for bloom.active
License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protonmail.ch>
2016-07-04 20:12:13 +02:00
Jakub Sztandera
5d83d89f36 blocks/blockstore: Add bloom filter
Replace write_cache with bloom_cache
Improve ARC caching
Fix small issue in case of AllKeysChan fails
deps: Update go-datastore
blocks/blockstore: Invalidate ARC cache before deletin block
deps: Update go-datastore

License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protonmail.ch>
2016-07-03 22:16:28 +02:00
Jeromy
0aafa6db8c update go-datastore changes 0.1.2
License: MIT
Signed-off-by: Jeromy <why@ipfs.io>
2016-07-01 22:40:57 -07:00
Jeromy
0782c4dc03 encode keys to datastore with base32 standard encoding
Fixes #2601

Also bump version to 0.4.3-dev

License: MIT
Signed-off-by: Jeromy <why@ipfs.io>
2016-07-01 14:15:06 -07:00
Jakub Sztandera
f2686965ac tests: Add test to RuntimeHashing option of blockstore
License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protonmail.ch>
2016-06-24 22:21:01 +02:00
Jakub Sztandera
989e91b1c4 blockstore: add fetch rehashing
License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protonmail.ch>
2016-06-24 22:20:58 +02:00
Jakub Sztandera
ce8c8a7a71 Update go-log in whole dependency tree (#2898)
* Update golog in go-ipfs

License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protonmail.ch>

* Update go-libp2p for go-log

License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protonmail.ch>

* Update go-libp2p-secio for go-log

License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protonmail.ch>

* Update go-libp2p-crypto for go-log

License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protonmail.ch>

* Update go-libp2p-peer for go-log

License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protonmail.ch>

* Import peersore, it wasn't imported

License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protonmail.ch>

* Update peerstore

License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protonmail.ch>

* Update peer

License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protonmail.ch>

* Update secio

License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protonmail.ch>

* Update go-libp2p

License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protonmail.ch>
2016-06-24 09:38:07 -07:00
Jakub Sztandera
5a08e9e08a Remove failing blockstore test with context
Why is it failing:
process is started, cancel() is called,
between we satart listening to the channels
in select statemnet there is race of three things
that can happent:
 1. Task can complete
 2. Task can start closing <- expected
 3. Task already closed

This race causes failures of the test.

It is basing heavily on race of conditions where
the task not closing, nor the task is completed
before channels are being listened.

It is quite impossible to resolve without
adding bunch of timings in there, which we
want to avoid, as there is no atomic
"send message on channel and select" in Golang

License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protonmail.ch>
2016-06-15 21:06:34 +02:00
Jakub Sztandera
a14bd71749 Move golang-lru to gx
License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protonmail.ch>
2016-06-12 10:02:44 +02:00
Jakub Sztandera
cafd81c8c6 Import go-datastore to gx
License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protonmail.ch>
2016-06-12 08:47:15 +02:00
Jakub Sztandera
6217e1f141 Remove go-datastore from Godeps
License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protonmail.ch>
2016-06-12 08:23:43 +02:00
Jakub Sztandera
eaccd07505 Update go-log
https://github.com/ipfs/go-log/pull/3

License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protonmail.ch>
2016-06-09 23:37:09 +02:00
Kevin Atkinson
dbabcf9681 Rename blocks.RawBlock to blocks.BasicBlock.
License: MIT
Signed-off-by: Kevin Atkinson <k@kevina.org>
2016-06-01 17:14:16 -04:00
Jakub Sztandera
e86bf92f31 Migrate hamming to gx from Godeps
License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protonmail.ch>
2016-05-19 06:45:19 +02:00
jbenet
0ddafe603a add error checking for nil keys
Checks in:
- blockstore
- blockservice
- dagservice
- bitswap

Do not anger the pokemans #2715

License: MIT
Signed-off-by: Juan Benet <juan@benet.ai>
2016-05-16 23:03:28 -07:00
Kevin Atkinson
b84cbec2b6 Make blocks.Block an interface.
License: MIT
Signed-off-by: Kevin Atkinson <k@kevina.org>
2016-05-05 18:02:11 -04:00
Hector Sanjuan
57b494a70c Update go-log to 1.1.0 and fix calls to go-log.Uuid
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2016-05-04 23:17:21 +02:00
Jeromy
e79e1d31e9 remove a ton of unused godeps
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
2016-04-13 11:23:10 -07:00
Jeromy Johnson
b0a613fd71 Merge pull request #1892 from MichaelMure/blockstore_log
Add log events when blocks are added/removed to/from the blockstore
2016-04-12 18:59:20 -07:00
Michael Muré
c88e46cb52 clean deprecated Key.Pretty()
License: MIT
Signed-off-by: Michael Muré <batolettre@gmail.com>
2016-03-25 12:05:45 +01:00
Jeromy
d7dab3afea Use gx vendored go-ipfs-utils where possible
For the rest of the packages in util, move them to thirdparty
and update the references. util is gone!

License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
2016-02-12 17:21:40 -08:00
Jeromy
5474e15e93 blockstore locks return unlocker object now
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
2016-02-08 14:42:07 -08:00
Michael Muré
e86c82c68d Add log events when blocks are added/removed from the blockstore
License: MIT
Signed-off-by: Michael Muré <mure.michael@gmail.com>
2016-01-31 19:31:39 +01:00
Jeromy
0e8a6700f9 correct go-log dep
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
2016-01-30 09:34:10 -08:00
Jeromy
0e312f5caf initial vendoring of libp2p outside of the repo with gx
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
2016-01-30 09:34:06 -08:00
Jeromy
4d5af93f43 vendor in new go-datastore
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
2016-01-12 08:22:55 -08:00
Jeromy
c49dcffce2 Allow for gc during adds
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
2016-01-12 08:22:55 -08:00
Jeromy
c6ee2058d9 comments from CR
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
2016-01-12 08:22:55 -08:00
Jeromy
45d4b1a8bc fixup datastore interfaces
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
2016-01-12 08:22:55 -08:00
Tommi Virtanen
b76581d6c7 fsrepo: Refactor to extract datastore internals
License: MIT
Signed-off-by: Tommi Virtanen <tv@eagain.net>
2016-01-12 08:22:55 -08:00
Juan Batiz-Benet
27f34b4311 renamed {R,}Lock -> {Pin,GC}Lock
License: MIT
Signed-off-by: Juan Batiz-Benet <juan@benet.ai>
2016-01-12 08:21:13 -08:00
Jeromy
da0d48e6c5 Add locking interface to blockstore
The addition of a locking interface to the blockstore allows us to
perform atomic operations on the underlying datastore without having to
worry about different operations happening in the background, such as
garbage collection.

License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
2016-01-12 08:21:13 -08:00
Tommi Virtanen
c4d2988c11 blocks/set: Remove now-unused NewDBWrapperSet
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
2016-01-12 08:17:17 -08:00
Jeromy
94bdce63a7 vendor logging lib update
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
2015-11-05 15:57:21 -08:00
Jeromy
c023d187b5 update code to use new logging changes
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
2015-10-27 11:04:56 -07:00
Jeromy
8f0623255d replace imports with absolute path instead of using symlink
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
2015-10-03 14:30:50 -07:00
Jeromy
35a5ca0ef5 update go-datastore to latest
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
2015-09-15 17:15:29 -07:00
Jeromy
e5a2896c0e extract logging
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
2015-09-14 17:35:50 -07:00
Jeromy
137c0ac4ac use batching transaction interface from datastore
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
2015-07-09 16:37:30 -07:00
Jeromy
ef294431d4 move util.Key into its own package under blocks 2015-06-01 16:10:08 -07:00
Tommi Virtanen
a743290316 blocks: Don't re-Put blocks we already have
Commit 1192be196b tried to do this, but
had a simple mistake. Functions returning `bool, error` pretty much
never return `true, anError`, so that branch was never taken.

Also fix the partial sentence in the
2015-04-28 16:05:52 -07:00
Tommi Virtanen
24daeec70c Use flatfs to store objects under /blocks outside of LevelDB
WARNING: No migration performed! That needs to come in a separate
commit, perhaps amended into this one.

Migration must move keyspace "/b" from leveldb to the flatfs subdir,
while removing the "b" prefix (keys should start with just "/").
2015-04-20 02:24:23 -07:00
Kristoffer Ström
2c3f9f2419 Add hamming distance calculation to bloom filters 2015-04-15 17:13:47 +02:00
Kristoffer Ström
3d8e96a22e Make bloom filters simpler
These did not work before, and had some unnecessary complexity.

Now the filters use only one hashing function, no bignum arithmetic, and gets the additional bit positions by repeatedly hashing the result of prior hash.

Since we're not concerned about crypto hashing here, this should be a win.

External interfaces unchanged.
2015-04-07 18:03:15 +02:00
Ho-Sheng Hsiao
bf22aeec0a Reorged imports from jbenet/go-ipfs to ipfs/go-ipfs
- Modified Godeps/Godeps.json by hand
- [TEST] Updated welcome docs hash to sharness
- [TEST] Updated contact doc
- [TEST] disabled breaking test (t0080-repo refs local)
2015-03-31 12:52:25 -07:00
Tommi Virtanen
63c25285b2 Dead code cleanup: remove AllKeys from blockstore
Nothing uses it.
2015-03-11 21:46:57 -07:00
Tommi Virtanen
1a48e6a91f Dead code cleanup: remove Range support from blockstore
Nothing uses it, and offset+limit is a bad query mechanism for
mutating data.
2015-03-11 21:46:57 -07:00
Henry
92d08db7a5 rewrote import paths of go.net/context to use golang.org/x/context
- updated go-ctxgroup and goprocess
ctxgroup: AddChildGroup was changed to AddChild. Used in two files:
- p2p/net/mock/mock_net.go
- routing/dht/dht.go

- updated context from hg repo to git
prev. commit in hg was ad01a6fcc8a19d3a4478c836895ffe883bd2ceab. (context: make parentCancelCtx iterative)
represents commit 84f8955a887232b6308d79c68b8db44f64df455c in git repo

- updated context to master (b6fdb7d8a4ccefede406f8fe0f017fb58265054c)

Aaron Jacobs (2):
net/context: Don't accept a context in the DoSomethingSlow example.
context: Be clear that users must cancel the result of WithCancel.

Andrew Gerrand (1):
go.net: use golang.org/x/... import paths

Bryan C. Mills (1):
net/context: Don't leak goroutines in Done example.

Damien Neil (1):
context: fix removal of cancelled timer contexts from parent

David Symonds (2):
context: Fix WithValue example code.
net: add import comments.

Sameer Ajmani (1):
context: fix TestAllocs to account for ints in interfaces
2015-02-25 11:58:19 +01:00