Commit Graph

98 Commits

Author SHA1 Message Date
Jeromy Johnson
4e67003f5a Merge pull request #2953 from ipfs/feature/blocks-bloom-no-rebuild
core: do not run bloom in case of ephemeral node
2016-07-10 13:34:44 -07:00
Kevin Atkinson
9e6cc1f112 Increase channel buffer size in blockstore.AllKeysChan().
License: MIT
Signed-off-by: Kevin Atkinson <k@kevina.org>
2016-07-10 15:41:58 -04:00
Jakub Sztandera
58526b25b5 blocks/blockstore: shift insertion of TODO context to tests
License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protonmail.ch>
2016-07-10 15:16:42 +02:00
Jakub Sztandera
af7778213c block/blockstore: bloomcache PutMany logic was not adding to ARC
License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protonmail.ch>
2016-07-08 21:17:32 +02:00
Jakub Sztandera
61a3d127e7 blocks/blockstore: improve logic a bit
License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protonmail.ch>
2016-07-08 21:17:32 +02:00
Jakub Sztandera
016d3d97ef blocks/blockstore: introduce context passing to blockstore
License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protonmail.ch>
2016-07-08 21:17:32 +02:00
Jakub Sztandera
3035aa8935 blocks/blockstore: add CacheOpts - structure of cache config
License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protonmail.ch>
2016-07-08 20:53:11 +02:00
Jakub Sztandera
f13506c11a blocks/blockstore: style cleanup of bloomcache
License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protonmail.ch>
2016-07-04 20:34:46 +02:00
Jakub Sztandera
e92e6662a7 test: fix races in bloomcache tests
License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protonmail.ch>
2016-07-04 20:34:07 +02:00
Jakub Sztandera
98f877af81 blocks/blockstorage: use automic for bloom.active
License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protonmail.ch>
2016-07-04 20:12:13 +02:00
Jakub Sztandera
5d83d89f36 blocks/blockstore: Add bloom filter
Replace write_cache with bloom_cache
Improve ARC caching
Fix small issue in case of AllKeysChan fails
deps: Update go-datastore
blocks/blockstore: Invalidate ARC cache before deletin block
deps: Update go-datastore

License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protonmail.ch>
2016-07-03 22:16:28 +02:00
Jeromy
0aafa6db8c update go-datastore changes 0.1.2
License: MIT
Signed-off-by: Jeromy <why@ipfs.io>
2016-07-01 22:40:57 -07:00
Jeromy
0782c4dc03 encode keys to datastore with base32 standard encoding
Fixes #2601

Also bump version to 0.4.3-dev

License: MIT
Signed-off-by: Jeromy <why@ipfs.io>
2016-07-01 14:15:06 -07:00
Jakub Sztandera
f2686965ac tests: Add test to RuntimeHashing option of blockstore
License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protonmail.ch>
2016-06-24 22:21:01 +02:00
Jakub Sztandera
989e91b1c4 blockstore: add fetch rehashing
License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protonmail.ch>
2016-06-24 22:20:58 +02:00
Jakub Sztandera
ce8c8a7a71 Update go-log in whole dependency tree (#2898)
* Update golog in go-ipfs

License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protonmail.ch>

* Update go-libp2p for go-log

License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protonmail.ch>

* Update go-libp2p-secio for go-log

License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protonmail.ch>

* Update go-libp2p-crypto for go-log

License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protonmail.ch>

* Update go-libp2p-peer for go-log

License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protonmail.ch>

* Import peersore, it wasn't imported

License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protonmail.ch>

* Update peerstore

License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protonmail.ch>

* Update peer

License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protonmail.ch>

* Update secio

License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protonmail.ch>

* Update go-libp2p

License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protonmail.ch>
2016-06-24 09:38:07 -07:00
Jakub Sztandera
5a08e9e08a Remove failing blockstore test with context
Why is it failing:
process is started, cancel() is called,
between we satart listening to the channels
in select statemnet there is race of three things
that can happent:
 1. Task can complete
 2. Task can start closing <- expected
 3. Task already closed

This race causes failures of the test.

It is basing heavily on race of conditions where
the task not closing, nor the task is completed
before channels are being listened.

It is quite impossible to resolve without
adding bunch of timings in there, which we
want to avoid, as there is no atomic
"send message on channel and select" in Golang

License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protonmail.ch>
2016-06-15 21:06:34 +02:00
Jakub Sztandera
a14bd71749 Move golang-lru to gx
License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protonmail.ch>
2016-06-12 10:02:44 +02:00
Jakub Sztandera
cafd81c8c6 Import go-datastore to gx
License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protonmail.ch>
2016-06-12 08:47:15 +02:00
Jakub Sztandera
6217e1f141 Remove go-datastore from Godeps
License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protonmail.ch>
2016-06-12 08:23:43 +02:00
Jakub Sztandera
eaccd07505 Update go-log
https://github.com/ipfs/go-log/pull/3

License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protonmail.ch>
2016-06-09 23:37:09 +02:00
Kevin Atkinson
dbabcf9681 Rename blocks.RawBlock to blocks.BasicBlock.
License: MIT
Signed-off-by: Kevin Atkinson <k@kevina.org>
2016-06-01 17:14:16 -04:00
Jakub Sztandera
e86bf92f31 Migrate hamming to gx from Godeps
License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protonmail.ch>
2016-05-19 06:45:19 +02:00
jbenet
0ddafe603a add error checking for nil keys
Checks in:
- blockstore
- blockservice
- dagservice
- bitswap

Do not anger the pokemans #2715

License: MIT
Signed-off-by: Juan Benet <juan@benet.ai>
2016-05-16 23:03:28 -07:00
Kevin Atkinson
b84cbec2b6 Make blocks.Block an interface.
License: MIT
Signed-off-by: Kevin Atkinson <k@kevina.org>
2016-05-05 18:02:11 -04:00
Hector Sanjuan
57b494a70c Update go-log to 1.1.0 and fix calls to go-log.Uuid
License: MIT
Signed-off-by: Hector Sanjuan <code@hector.link>
2016-05-04 23:17:21 +02:00
Jeromy
e79e1d31e9 remove a ton of unused godeps
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
2016-04-13 11:23:10 -07:00
Jeromy Johnson
b0a613fd71 Merge pull request #1892 from MichaelMure/blockstore_log
Add log events when blocks are added/removed to/from the blockstore
2016-04-12 18:59:20 -07:00
Michael Muré
c88e46cb52 clean deprecated Key.Pretty()
License: MIT
Signed-off-by: Michael Muré <batolettre@gmail.com>
2016-03-25 12:05:45 +01:00
Jeromy
d7dab3afea Use gx vendored go-ipfs-utils where possible
For the rest of the packages in util, move them to thirdparty
and update the references. util is gone!

License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
2016-02-12 17:21:40 -08:00
Jeromy
5474e15e93 blockstore locks return unlocker object now
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
2016-02-08 14:42:07 -08:00
Michael Muré
e86c82c68d Add log events when blocks are added/removed from the blockstore
License: MIT
Signed-off-by: Michael Muré <mure.michael@gmail.com>
2016-01-31 19:31:39 +01:00
Jeromy
0e8a6700f9 correct go-log dep
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
2016-01-30 09:34:10 -08:00
Jeromy
0e312f5caf initial vendoring of libp2p outside of the repo with gx
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
2016-01-30 09:34:06 -08:00
Jeromy
4d5af93f43 vendor in new go-datastore
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
2016-01-12 08:22:55 -08:00
Jeromy
c49dcffce2 Allow for gc during adds
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
2016-01-12 08:22:55 -08:00
Jeromy
c6ee2058d9 comments from CR
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
2016-01-12 08:22:55 -08:00
Jeromy
45d4b1a8bc fixup datastore interfaces
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
2016-01-12 08:22:55 -08:00
Tommi Virtanen
b76581d6c7 fsrepo: Refactor to extract datastore internals
License: MIT
Signed-off-by: Tommi Virtanen <tv@eagain.net>
2016-01-12 08:22:55 -08:00
Juan Batiz-Benet
27f34b4311 renamed {R,}Lock -> {Pin,GC}Lock
License: MIT
Signed-off-by: Juan Batiz-Benet <juan@benet.ai>
2016-01-12 08:21:13 -08:00
Jeromy
da0d48e6c5 Add locking interface to blockstore
The addition of a locking interface to the blockstore allows us to
perform atomic operations on the underlying datastore without having to
worry about different operations happening in the background, such as
garbage collection.

License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
2016-01-12 08:21:13 -08:00
Tommi Virtanen
c4d2988c11 blocks/set: Remove now-unused NewDBWrapperSet
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
2016-01-12 08:17:17 -08:00
Jeromy
94bdce63a7 vendor logging lib update
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
2015-11-05 15:57:21 -08:00
Jeromy
c023d187b5 update code to use new logging changes
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
2015-10-27 11:04:56 -07:00
Jeromy
8f0623255d replace imports with absolute path instead of using symlink
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
2015-10-03 14:30:50 -07:00
Jeromy
35a5ca0ef5 update go-datastore to latest
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
2015-09-15 17:15:29 -07:00
Jeromy
e5a2896c0e extract logging
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
2015-09-14 17:35:50 -07:00
Jeromy
137c0ac4ac use batching transaction interface from datastore
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
2015-07-09 16:37:30 -07:00
Jeromy
ef294431d4 move util.Key into its own package under blocks 2015-06-01 16:10:08 -07:00
Tommi Virtanen
a743290316 blocks: Don't re-Put blocks we already have
Commit 1192be196b tried to do this, but
had a simple mistake. Functions returning `bool, error` pretty much
never return `true, anError`, so that branch was never taken.

Also fix the partial sentence in the
2015-04-28 16:05:52 -07:00