f630222580
This commit introduces non-recursive Makefile infrastructure that replaces current Makefile infrastructure. It also generally cleanups the Makefiles, separates them into nicer sub-modules and centralizes common operations into single definitions. It allows to depend on any target that is defined in the makefile, this means that for example `gx install` is called once when `make build test_expensive_sharness` is called instead of 4 or 5 times. It also makes the dependencies much cleaner and allows for reuse of modules. For example sharness coverage collection (WIP) uses sharness target with amended PATH, previously it might have been possible but not without wiring in the coverage collection into sharness make runner code. Yes, it is more complex but not much more. There are few rules that have to be followed and few complexities added but IMHO it is worth it. How to NR-make: 1. If make is to generate some file via a target, it MUST be defined in Rules.mk file in the directory of the target. 2. `Rules.mk` file MUST have `include mk/header.mk` statement as the first line and `include mk/footer.mk` statement as the last line (apart from project root `Rules.mk`). 3. It then MUST be included by the closest `Rules.mk` file up the directory tree. 4. Inside a `Rules.mk` special variable accessed as `$(d)` is defined. Its value is current directory, use it so if the `Rules.mk` file is moved in the tree it still works without a problem. Caution: this variable is not available in the recipe part and MUST NOT be used. Use name of the target or prerequisite to extract it if you need it. 5. Make has only one global scope, this means that name conflicts are a thing. Names SHOULD follow `VAR_NAME_$(d)` convention. There are exceptions from this rule in form of well defined global variables. Examples: General lists `TGT_BIN`, `CLEAN`; General targets: `TEST`, `COVERAGE`; General variables: `GOFLAGS`, `DEPS_GO`. 3. Any rules, definitions or variables that fit some family SHOULD be defined in `mk/$family.mk` file and included from project root `Rules.mk` License: MIT Signed-off-by: Jakub Sztandera <kubuxu@protonmail.ch> |
||
|---|---|---|
| .. | ||
| decision | ||
| message | ||
| network | ||
| notifications | ||
| testnet | ||
| wantlist | ||
| bitswap_test.go | ||
| bitswap.go | ||
| README.md | ||
| stat.go | ||
| testutils.go | ||
| wantmanager.go | ||
| workers.go | ||
Bitswap
Protocol
Bitswap is the data trading module for ipfs, it manages requesting and sending blocks to and from other peers in the network. Bitswap has two main jobs, the first is to acquire blocks requested by the client from the network. The second is to judiciously send blocks in its posession to other peers who want them.
Bitswap is a message based protocol, as opposed to response-reply. All messages contain wantlists, or blocks. Upon receiving a wantlist, a node should consider sending out wanted blocks if they have them. Upon receiving blocks, the node should send out a notification called a 'Cancel' signifying that they no longer want the block. At a protocol level, bitswap is very simple.
go-ipfs Implementation
Internally, when a message with a wantlist is received, it is sent to the decision engine to be considered, and blocks that we have that are wanted are placed into the peer request queue. Any block we possess that is wanted by another peer has a task in the peer request queue created for it. The peer request queue is a priority queue that sorts available tasks by some metric, currently, that metric is very simple and aims to fairly address the tasks of each other peer. More advanced decision logic will be implemented in the future. Task workers pull tasks to be done off of the queue, retreive the block to be sent, and send it off. The number of task workers is limited by a constant factor.
Client requests for new blocks are handled by the want manager, for every new block (or set of blocks) wanted, the 'WantBlocks' method is invoked. The want manager then ensures that connected peers are notified of the new block that we want by sending the new entries to a message queue for each peer. The message queue will loop while there is work available and do the following: 1) Ensure it has a connection to its peer, 2) grab the message to be sent, and 3) send it. If new messages are added while the loop is in steps 1 or 3, the messages are combined into one to avoid having to keep an actual queue and send multiple messages. The same process occurs when the client receives a block and sends a cancel message for it.