Commit Graph

346 Commits

Author SHA1 Message Date
Juan Batiz-Benet
10072c149d bitswap: log superfluous messages 2015-01-05 04:55:02 -08:00
Juan Batiz-Benet
aeb2e07198 bitswap: remove DialPeer from interface
Bitswap doesn't usually care about dialing. the underlying
network adapter can make sure of that.
2015-01-05 04:55:02 -08:00
Brian Tiger Chow
05d8c80c39 fix(bitswap/network): return when context is done
@jbenet @whyrusleeping

This bug (missing return) could tie up the client worker and cause
operations to come to a halt.
2015-01-05 00:16:53 -08:00
Juan Batiz-Benet
9c6228d18f bitswap and dht: lots of debugging logs 2015-01-05 00:16:53 -08:00
Juan Batiz-Benet
71ada4696f bitswap net: always close 2015-01-05 00:16:53 -08:00
Juan Batiz-Benet
d357b0ac00 bitswap debug logging 2015-01-05 00:16:53 -08:00
Juan Batiz-Benet
c100390a49 bitswap engine: signal in own func 2015-01-05 00:16:52 -08:00
Juan Batiz-Benet
5639042df5 bitswap: send wantlist code reuse + debug logs 2015-01-05 00:16:52 -08:00
Juan Batiz-Benet
7629ad7ec7 bitswap: add self peer.ID 2015-01-05 00:16:52 -08:00
Juan Batiz-Benet
8fb5cf9578 swap net2 -> net 2015-01-02 08:46:46 -08:00
Juan Batiz-Benet
e2698a8733 net -> p2p/net
The net package is the next to move. It will be massaged
a bit still to fix the Network / "NetworkBackend" conflict.
2015-01-02 08:46:45 -08:00
Juan Batiz-Benet
89f5cd4c94 introducing p2p pkg
I think it's time to move a lot of the peer-to-peer networking
but-not-ipfs-specific things into its own package: p2p.
This could in the future be split off into its own library.
The first thing to go is the peer.
2015-01-02 08:46:45 -08:00
Brian Tiger Chow
0172e4f2bb style(testutil) rename testutil.Peer -> testutil.Identity
cc @jbenet
2014-12-24 09:59:24 -05:00
Brian Tiger Chow
237e594a8f don't link when creating network client. rely on caller 2014-12-24 09:31:19 -05:00
Brian Tiger Chow
ca32a83394 wip with DHT
@whyrusleeping @jbenet this is a WIP with the DHT.

wip

License: MIT
Signed-off-by: Brian Tiger Chow <brian@perfmode.com>

Conflicts:
	epictest/addcat_test.go
	exchange/bitswap/testnet/peernet.go
	exchange/bitswap/testutils.go
	routing/mock/centralized_server.go
	routing/mock/centralized_test.go
	routing/mock/interface.go

fix(routing/mock) fill in function definition
2014-12-24 09:31:19 -05:00
Brian Tiger Chow
6da57c6764 feat(bitswap/testnet) impl a version of bitswap testnet that uses mockpeernet under the hood
License: MIT
Signed-off-by: Brian Tiger Chow <brian@perfmode.com>
2014-12-24 09:31:18 -05:00
Brian Tiger Chow
85ce94c510 refactor(bitswap/testnet) rename to virtual
License: MIT
Signed-off-by: Brian Tiger Chow <brian@perfmode.com>
2014-12-24 09:31:18 -05:00
Brian Tiger Chow
d390e31c11 refactor(bitswap/testnet) extract interface in prep for mockpeernet version
License: MIT
Signed-off-by: Brian Tiger Chow <brian@perfmode.com>
2014-12-24 09:31:18 -05:00
Brian Tiger Chow
6cdb4c23b4 pass peer into testnet adapter method 2014-12-24 09:31:18 -05:00
Brian Tiger Chow
e017d6edc6 refactor(bitswap/testnet) slim down interface
so it'll be easier to create another implementation using the new
mocknet
2014-12-24 09:31:17 -05:00
Brian Tiger Chow
b551e45ae9 use testutil peer in sess 2014-12-24 09:31:17 -05:00
Brian Tiger Chow
81a67338f1 fix(bitswap) remove peerstore 2014-12-24 09:31:17 -05:00
Brian Tiger Chow
c933d439b7 misc docs and fmting 2014-12-24 09:31:17 -05:00
Brian Tiger Chow
bd45adfb89 chore(bitswap) remove Peerstore() methods from bitswap.Network interface 2014-12-23 09:22:17 -08:00
Brian Tiger Chow
c34132e080 refactor(bitswap) change PeerInfo to ID in bitswap package
@jbenet @whyrusleeping

This commit replaces peer.PeerInfo with peer.ID in the bitswap package
2014-12-23 09:21:08 -08:00
Brian Tiger Chow
27dc9594ba refactor(bitswap) bitswap.Network now abstracts ipfs.Network + ipfs.Routing
@jbenet @whyrusleeping

the next commit will change bitswap.Network.FindProviders to only deal
with IDs
2014-12-23 09:19:21 -08:00
Brian Tiger Chow
c6294646e9 fix(bitswap) always add addresses 2014-12-23 09:17:40 -08:00
Brian Tiger Chow
070b633df8 style(bitswap) rename to network 2014-12-23 09:16:00 -08:00
Brian Tiger Chow
60f3a874c2 feat(bitswap/network) expose peerstore 2014-12-23 09:14:54 -08:00
Brian Tiger Chow
a225568ff6 style(bitswap) public methods at top 2014-12-23 09:13:17 -08:00
Brian Tiger Chow
4ab4da7e2b style(bitswap) rename 2014-12-23 09:10:44 -08:00
Brian Tiger Chow
fb75184c35 fix(bitswap) always use prvivate send method to send
cc @whyrusleeping
2014-12-23 09:09:34 -08:00
Brian Tiger Chow
d0d220f9de fix(bitswap) always dial 2014-12-23 09:08:11 -08:00
Brian Tiger Chow
55c8f8f505 Revert "bitswap: network interface changed"
This reverts commit bf88f1aec5.
2014-12-23 09:06:44 -08:00
Juan Batiz-Benet
d970f538e5 bitswap: network interface changed
Had to change the network interface from DialPeer(peer.ID) to
DialPeer(peer.PeerInfo), so that addresses of a provider are
handed to the network.

@maybebtc and I are discussing whether this should go all the
way down to the network, or whether the network _should always
work_ with just an ID (which means the network needs to be
able to resolve ID -> Addresses, using the routing system.
This latter point might mean that "routing" might need to
break down into subcomponents. It's a bit sketchy that
the Network would become smarter than just dial/listen and
I/O, but maybe there's a distinction between net.Network,
and something like a peernet.Network that has routing
built in...)
2014-12-23 09:05:15 -08:00
Brian Tiger Chow
3f9c1263ec fix: data race in test
https://build.protocol-dev.com/job/race/9352/console

@jbenet @whyrusleeping

pinging you guys to spread awareness about the delay.D type for
configurable delays

License: MIT
Signed-off-by: Brian Tiger Chow <brian@perfmode.com>
2014-12-23 08:59:40 -08:00
Juan Batiz-Benet
c84a714b16 peer change: peer.Peer -> peer.ID
this is a major refactor of the entire codebase
it changes the monolithic peer.Peer into using
a peer.ID and a peer.Peerstore.

Other changes:
- removed handshake3.
-	testutil vastly simplified peer
-	secio bugfix + debugging logs
-	testutil: RandKeyPair
-	backpressure bugfix: w.o.w.
-	peer: added hex enc/dec
-	peer: added a PeerInfo struct
  PeerInfo is a small struct used to pass around a peer with
 	a set of addresses and keys. This is not meant to be a
 	complete view of the system, but rather to model updates to
 	the peerstore. It is used by things like the routing system.
-	updated peer/queue + peerset
-	latency metrics
-	testutil: use crand for PeerID gen
 	RandPeerID generates random "valid" peer IDs. it does not
 	NEED to generate keys because it is as if we lost the key
 	right away. fine to read some randomness and hash it. to
 	generate proper keys and an ID, use:
 	  sk, pk, _ := testutil.RandKeyPair()
 	  id, _ := peer.IDFromPublicKey(pk)
 	Also added RandPeerIDFatal helper
- removed old spipe
- updated seccat
- core: cleanup initIdentity
- removed old getFromPeerList
2014-12-23 08:33:32 -08:00
Jeromy
9328fbaf81 clean peerset constructor names 2014-12-17 23:44:46 -08:00
Brian Tiger Chow
8100582e38 fix: batches of blocks have equal priority
addresses...

https://github.com/jbenet/go-ipfs/pull/438/files#r21878994

License: MIT
Signed-off-by: Brian Tiger Chow <brian@perfmode.com>
2014-12-17 23:44:46 -08:00
Brian Tiger Chow
bd3ee739b9 doc: some comments about the future of the decision engine
License: MIT
Signed-off-by: Brian Tiger Chow <brian@perfmode.com>
2014-12-17 23:44:46 -08:00
Brian Tiger Chow
ce2d0a2258 fix: add lock to taskQueue
@whyrusleeping may wanna have a look and make sure i didn't screw
anything up here

BenchmarkInstantaneousAddCat1MB-4            200          10763761 ns/op
97.42 MB/s
BenchmarkInstantaneousAddCat2MB-4       panic: runtime error: invalid
memory address or nil pointer dereference
[signal 0xb code=0x1 addr=0x0 pc=0xbedd]

goroutine 14297 [running]:
github.com/jbenet/go-ipfs/exchange/bitswap/decision.(*taskQueue).Remove(0xc2087553a0,
        0xc2085ef200, 0x22, 0x56f570, 0xc208367a40)
    /Users/btc/go/src/github.com/jbenet/go-ipfs/exchange/bitswap/decision/taskqueue.go:66
    +0x82
github.com/jbenet/go-ipfs/exchange/bitswap/decision.(*Engine).MessageSent(0xc20871b5c0,
        0x56f570, 0xc208367a40, 0x570040, 0xc208753d40, 0x0, 0x0)
    /Users/btc/go/src/github.com/jbenet/go-ipfs/exchange/bitswap/decision/engine.go:177
    +0x29e
github.com/jbenet/go-ipfs/exchange/bitswap.(*bitswap).send(0xc20871b7a0,
        0x56f4d8, 0xc208379800, 0x56f570, 0xc208367a40,
        0x570040, 0xc208753d40, 0x0, 0x0)
    /Users/btc/go/src/github.com/jbenet/go-ipfs/exchange/bitswap/bitswap.go:352
    +0x11c
github.com/jbenet/go-ipfs/exchange/bitswap.(*bitswap).taskWorker(0xc20871b7a0,
        0x56f4d8, 0xc208379800)
    /Users/btc/go/src/github.com/jbenet/go-ipfs/exchange/bitswap/bitswap.go:238
    +0x165
    created by
    github.com/jbenet/go-ipfs/exchange/bitswap.New
    /Users/btc/go/src/github.com/jbenet/go-ipfs/exchange/bitswap/bitswap.go:66
    +0x49e
2014-12-17 23:44:46 -08:00
Brian Tiger Chow
175513e281 refactor(bs/decision.Engine): pass in Entry
License: MIT
Signed-off-by: Brian Tiger Chow <brian@perfmode.com>
2014-12-17 23:44:46 -08:00
Brian Tiger Chow
4bcfe094a1 extract constants
License: MIT
Signed-off-by: Brian Tiger Chow <brian@perfmode.com>
2014-12-17 23:44:46 -08:00
Brian Tiger Chow
0545c4d15d refactor: *Entry -> Entry
in many places, entries are assigned from one slice to another and in
different goroutines. In one place, entries were modified (in the
queue). To avoid shared mutable state, probably best to handle entries
by value.

License: MIT
Signed-off-by: Brian Tiger Chow <brian@perfmode.com>
2014-12-17 23:44:46 -08:00
Brian Tiger Chow
e36d656632 log unusual event
License: MIT
Signed-off-by: Brian Tiger Chow <brian@perfmode.com>
2014-12-17 23:44:45 -08:00
Brian Tiger Chow
2b60b641c4 fix: check blockstore before adding task
addresses https://github.com/jbenet/go-ipfs/pull/438#discussion_r21953742

License: MIT
Signed-off-by: Brian Tiger Chow <brian@perfmode.com>
2014-12-17 23:44:45 -08:00
Brian Tiger Chow
6e7c46a6e2 unexport functions
License: MIT
Signed-off-by: Brian Tiger Chow <brian@perfmode.com>
2014-12-17 23:44:45 -08:00
Brian Tiger Chow
9c301a2d77 add comment
License: MIT
Signed-off-by: Brian Tiger Chow <brian@perfmode.com>
2014-12-17 23:44:45 -08:00
Brian Tiger Chow
1b1260b658 rm unused method
License: MIT
Signed-off-by: Brian Tiger Chow <brian@perfmode.com>
2014-12-17 23:44:45 -08:00
Brian Tiger Chow
8d4d5b86ef fix: don't sort the output of Entries()
only sort SortedEntries()

License: MIT
Signed-off-by: Brian Tiger Chow <brian@perfmode.com>
2014-12-17 23:44:45 -08:00
Brian Tiger Chow
acc714823b rename to peerRequestQueue
this opens up the possibility of having multiple queues. And for all
outgoing messages to be managed by the decision engine

License: MIT
Signed-off-by: Brian Tiger Chow <brian@perfmode.com>
2014-12-17 23:44:45 -08:00
Brian Tiger Chow
1d23e94f16 rm empty file
License: MIT
Signed-off-by: Brian Tiger Chow <brian@perfmode.com>
2014-12-17 23:44:45 -08:00
Brian Tiger Chow
5bd0b95462 rename to strategy.LedgerManager to decision.Engine
License: MIT
Signed-off-by: Brian Tiger Chow <brian@perfmode.com>
2014-12-17 23:44:44 -08:00
Brian Tiger Chow
bef622222d refactor: wantlist splits into WL and ThreadSafe WL
bitswap keeps the threadsafe version. observing the ledger shows that it
doesn't need it anymore (ledgermanager is protected and safe).

License: MIT
Signed-off-by: Brian Tiger Chow <brian@perfmode.com>
2014-12-17 23:44:44 -08:00
Brian Tiger Chow
d069ae11f4 refactor: put mutex next to the things it protects
If we put the lock next to the fields it protects, it can sometimes make
it easier to reason about threadsafety.

In this case, it reveals that the task queue (not threadsafe) isn't protected by the
mutex, yet shared between the worker and callers.

@whyrusleeping

License: MIT
Signed-off-by: Brian Tiger Chow <brian@perfmode.com>
2014-12-17 23:44:44 -08:00
Brian Tiger Chow
962a9477cc refactor: remove ledgerMap type
it's only used in two places, but i think we've been using maps on IPFS
types so much now that the specificity is no longer necessary

License: MIT
Signed-off-by: Brian Tiger Chow <brian@perfmode.com>
2014-12-17 23:44:44 -08:00
Brian Tiger Chow
39d71931a8 mv comment
License: MIT
Signed-off-by: Brian Tiger Chow <brian@perfmode.com>
2014-12-17 23:44:44 -08:00
Brian Tiger Chow
7fdbae1306 refactor: separate responsibilties
Before, priority carried two pieces of information.

One: priority as defined by remote peer
Two: whether task is trashed

This assumes the protocol is defined for natural numbers instead of
integers. That may not always be the case. Better to leave that
assumption outside so this package isn't coupled to the whims of the
protocol.

The protocol may be changed to allow any integer value to be used.
Hopefully by that time, new responsibilties weren't added to the
Priority variable.

License: MIT
Signed-off-by: Brian Tiger Chow <brian@perfmode.com>
2014-12-17 23:44:44 -08:00
Brian Tiger Chow
fedcebf67a refactor: re-use wantlist.Entry type wherever it makes sense
it seems to make sense since, in each place, the Key and Priority
represent the same information

b/c you know the saying...

"It is better to have 100 functions operate on one data structure than
10 functions on 10 data structures."

License: MIT
Signed-off-by: Brian Tiger Chow <brian@perfmode.com>
2014-12-17 23:44:44 -08:00
Brian Tiger Chow
1d715956f5 doc: add comment to Envelope
License: MIT
Signed-off-by: Brian Tiger Chow <brian@perfmode.com>
2014-12-17 23:44:44 -08:00
Brian Tiger Chow
e11f099c42 privatize Task
License: MIT
Signed-off-by: Brian Tiger Chow <brian@perfmode.com>
2014-12-17 23:44:44 -08:00
Brian Tiger Chow
181ff4eab1 tq.Cancel -> tq.Remove
License: MIT
Signed-off-by: Brian Tiger Chow <brian@perfmode.com>
2014-12-17 23:44:43 -08:00
Brian Tiger Chow
198aa1959a it's not a queue yet but it's okay to name it as such
License: MIT
Signed-off-by: Brian Tiger Chow <brian@perfmode.com>
2014-12-17 23:44:43 -08:00
Brian Tiger Chow
7280aac83f perf: avoid lots of communication by signaling once at end of method
License: MIT
Signed-off-by: Brian Tiger Chow <brian@perfmode.com>
2014-12-17 23:44:43 -08:00
Brian Tiger Chow
cc2a7312a2 add comment to fix race
License: MIT
Signed-off-by: Brian Tiger Chow <brian@perfmode.com>
2014-12-17 23:44:43 -08:00
Brian Tiger Chow
b41fef2fff refactor: remove peerKey type
we've been using maps with peers long enough now that this probably is
no longer necessary

License: MIT
Signed-off-by: Brian Tiger Chow <brian@perfmode.com>
2014-12-17 23:44:43 -08:00
Brian Tiger Chow
286723d888 unexport task and taskList
the less bitswap has to know about, the easier it'll be for readers.
(This now returns Messages.)

License: MIT
Signed-off-by: Brian Tiger Chow <brian@perfmode.com>
2014-12-17 23:44:43 -08:00
Brian Tiger Chow
93fde86d18 refactor: taskKey := p.Key() + block.Key()
for clarity and to avoid errors, define a function

License: MIT
Signed-off-by: Brian Tiger Chow <brian@perfmode.com>
2014-12-17 23:44:43 -08:00
Brian Tiger Chow
460967839e doc: comment
License: MIT
Signed-off-by: Brian Tiger Chow <brian@perfmode.com>
2014-12-17 23:44:43 -08:00
Brian Tiger Chow
10e970c0cf refactor: context first in argument list
(merely by convention)

License: MIT
Signed-off-by: Brian Tiger Chow <brian@perfmode.com>
2014-12-17 23:44:43 -08:00
Brian Tiger Chow
f66d94aaf1 fix: move the check into the function.
function should be a no-op when passed an empty slice

License: MIT
Signed-off-by: Brian Tiger Chow <brian@perfmode.com>
2014-12-17 23:44:42 -08:00
Brian Tiger Chow
8c05c444a5 refactor: avoid loop reuse
License: MIT
Signed-off-by: Brian Tiger Chow <brian@perfmode.com>
2014-12-17 23:44:42 -08:00
Brian Tiger Chow
2ea8ed81ac refactor: change Tasks to Outbox
notice that moving the blockstore fetch into the manager removes the
weird error handling case.

License: MIT
Signed-off-by: Brian Tiger Chow <brian@perfmode.com>
2014-12-17 23:44:42 -08:00
Brian Tiger Chow
a495a014af style constify variables
good to const until it's required for them to be variable.

TODO pass them in as configuration options
2014-12-17 23:44:42 -08:00
Brian Tiger Chow
8bef1dce80 fix: move to callsite so public callers don't experience the internal timeout rule
License: MIT
Signed-off-by: Brian Tiger Chow <brian@perfmode.com>
2014-12-17 23:44:42 -08:00
Brian Tiger Chow
5603b2e0a4 style: line wrapping
License: MIT
Signed-off-by: Brian Tiger Chow <brian@perfmode.com>
2014-12-17 23:44:42 -08:00
Brian Tiger Chow
c0a18d9c08 fix(test): nil Blockstore
License: MIT
Signed-off-by: Brian Tiger Chow <brian@perfmode.com>
2014-12-17 23:44:42 -08:00
Jeromy
69dd260236 refactor peerSet 2014-12-17 23:44:42 -08:00
Brian Tiger Chow
f533678724 avoid attaching context to object when it's not necessary.
License: MIT
Signed-off-by: Brian Tiger Chow <brian@perfmode.com>
2014-12-17 23:44:42 -08:00
Brian Tiger Chow
f028c44fd2 name findOrCreate
License: MIT
Signed-off-by: Brian Tiger Chow <brian@perfmode.com>
2014-12-17 23:44:41 -08:00
Brian Tiger Chow
1bced710b1 queue-like naming
License: MIT
Signed-off-by: Brian Tiger Chow <brian@perfmode.com>
2014-12-17 23:44:41 -08:00
Brian Tiger Chow
29ef238fa3 remove dead code
License: MIT
Signed-off-by: Brian Tiger Chow <brian@perfmode.com>
2014-12-17 23:44:41 -08:00
Brian Tiger Chow
86c438b6e1 refactor() message API
performing CR in the form of a PR. Let me know what you think.

License: MIT
Signed-off-by: Brian Tiger Chow <brian@perfmode.com>
2014-12-17 23:44:41 -08:00
Jeromy
6389bfda6b some cleanup before CR 2014-12-17 23:44:41 -08:00
Jeromy
90a30961de renaming and removing empty strategy file 2014-12-17 23:44:41 -08:00
Jeromy
029e305f19 tasklist queue for bitswap tasks 2014-12-17 23:44:41 -08:00
Jeromy
cfbe92bc8b rewrite sendWantlistToProviders 2014-12-17 23:44:40 -08:00
Jeromy
946d2a96f2 add locks to wantlist to avoid race condition 2014-12-17 23:44:40 -08:00
Brian Tiger Chow
59f0ffb8c6 remove noisy statement
License: MIT
Signed-off-by: Brian Tiger Chow <brian@perfmode.com>
2014-12-17 23:44:40 -08:00
Jeromy
50aa37fec4 blockstore.ErrNotFound, and proper wantlist sorting 2014-12-17 23:44:40 -08:00
Jeromy
e7bba82dcb add priorities to GetBlocks requests, and add waitgroup to sendWantListTo 2014-12-17 23:44:40 -08:00
Jeromy
3778eedff0 dont spawn so many goroutines when rebroadcasting wantlist 2014-12-17 23:44:40 -08:00
Jeromy
57e7dd7b8b extracted ledgerset from strategy, cleaned up a few comments from the PR 2014-12-17 23:44:40 -08:00
Jeromy
5b6a5e807f implement bitswap roundWorker
make vendor
2014-12-17 23:44:39 -08:00
Jeromy
12b296ee1a create wantlist object 2014-12-17 23:44:39 -08:00
Juan Batiz-Benet
ba323c3e48 make vendor 2014-12-17 23:25:38 -08:00
Juan Batiz-Benet
41751b4938 Integrated new network into ipfs 2014-12-16 14:47:29 -08:00
Brian Tiger Chow
5b5ece07ba doc TODO
License: MIT
Signed-off-by: Brian Tiger Chow <brian@perfmode.com>
2014-12-14 16:37:42 -08:00
Brian Tiger Chow
9a85f8fa72 style: Stop -> Close() error for Closer interface
License: MIT
Signed-off-by: Brian Tiger Chow <brian@perfmode.com>
2014-12-14 16:35:09 -08:00
Jeromy
c79fad2e12 give sessiongenerator a master context for easy cancelling 2014-12-14 16:33:05 -08:00