Commit Graph

668 Commits

Author SHA1 Message Date
Jeromy
ec5276c29c really ugly impl of 'ipfs dht query' command 2015-01-24 03:56:45 +00:00
Matt Bell
ccaabbc06d core/commands: Rearranged command list sections 2015-01-23 19:17:09 -08:00
Matt Bell
452f54a48e core/commands: Enabled stdin arguments for 'bootstrap {add, rm} 2015-01-23 19:11:56 -08:00
Matt Bell
71301d2c63 core/commands: Updated root command helptext 2015-01-23 19:11:56 -08:00
Matt Bell
5706471897 commands: Made PostRun signature match Run 2015-01-23 18:29:31 -08:00
Matt Bell
79741438d9 core/commands: config: Simplified some syntax 2015-01-23 18:29:31 -08:00
Matt Bell
3fc9bedb0b commands: Made Std{in|out|err} accessible in Request/Response 2015-01-23 18:29:31 -08:00
Matt Bell
b77d1c21b6 core/commands: add: Don't show progress bar when using --quiet option 2015-01-23 18:29:31 -08:00
Matt Bell
121dfb10b4 core/commands: Fixed progress bar line clear race condition in 'add' 2015-01-23 18:29:31 -08:00
Matt Bell
2363cb63e6 core/commands: Fixed 'add' progress bar sometimes not clearing progress bar output line 2015-01-23 18:29:31 -08:00
Matt Bell
c3ea164f64 core/commands: Added progress bars for 'add'
squash! core/commands: Added progress bars for 'add'

Use vendored progress bar lib in 'add'
2015-01-23 18:29:30 -08:00
Matt Bell
487ef33e67 core/commands: Added progress bar to 'cat'
squash! core/commands: Added progress bar to 'cat'

Vendored progress bar lib
2015-01-23 18:29:30 -08:00
Matt Bell
7b4de230eb commands: Refactored Command#Run function signature to (req Request, res Response) 2015-01-23 18:29:29 -08:00
Juan Batiz-Benet
c4f9a92e55 Merge pull request #628 from jbenet/feat/more-bootstraps
Add Bootstrap Peers: Mercury, Earth, Venus, Jupiter
2015-01-23 17:08:56 -08:00
Juan Batiz-Benet
bb2d81ace2 Merge pull request #622 from jbenet/webui-commands
Webui Commands
2015-01-23 16:51:11 -08:00
Brian Tiger Chow
bbc282bfab more bootstrap nodes 2015-01-23 16:19:50 -08:00
Matt Bell
68b2a3f15c core/commands: Renamed 'log read' to 'log tail' 2015-01-23 15:08:20 -08:00
Juan Batiz-Benet
95d58b2a4a core: cleaned up bootstrap process 2015-01-23 05:25:31 -08:00
Juan Batiz-Benet
dd9c1b6243 core/bootstrap: CR comments 2015-01-23 02:08:30 -08:00
Juan Batiz-Benet
d6ce837d72 core/bootstrap: cleaned up bootstrapping
Moved it to its own package to isolate scope.
2015-01-23 02:08:29 -08:00
Juan Batiz-Benet
010cedf0a9 ipfs swarm peers: sort output 2015-01-23 02:08:29 -08:00
Juan Batiz-Benet
ec848c486b core: call dht bootstrap 2015-01-23 02:08:28 -08:00
Juan Batiz-Benet
898b9696ca diag/net: add timeout param to cmd 2015-01-23 02:08:28 -08:00
Juan Batiz-Benet
82d38a2692 routing/dht: periodic bootstrapping #572 2015-01-23 02:08:28 -08:00
Jeromy
eb6afd3edc fix interface for coreunix.Cat, now takes a path 2015-01-23 06:39:20 +00:00
Matt Bell
81d17e0843 cmd/ipfs: gatewayHandler: Fixed directory listing getting appended to index.html pages 2015-01-22 16:29:00 -08:00
Matt Bell
926b96463d core/commands: Added 'log read' command
squash! core/commands: Added 'log read' command

core/commands: Use a unidirectional channel for 'log read' output
2015-01-22 16:28:59 -08:00
Matt Bell
8d412e48cf core/commands: Added an 'ipfs config replace' command 2015-01-22 16:28:59 -08:00
Brian Tiger Chow
fadede6cb2 separate concerns 2015-01-22 01:28:40 -08:00
Brian Tiger Chow
fadedf9e68 refactor(core) extract corehttp package
PACKAGE DOCUMENTATION

package corehttp

    FUNCTIONS

    func GatewayOption(n *core.IpfsNode, mux *http.ServeMux) error

    func ListenAndServe(n *core.IpfsNode, addr ma.Multiaddr, options
    ...ServeOption) error

    func WebUIOption(n *core.IpfsNode, mux *http.ServeMux) error

    TYPES

    type ServeOption func(*core.IpfsNode, *http.ServeMux) error

    func DaemonOption(cctx commands.Context) ServeOption
2015-01-22 01:08:59 -08:00
Juan Batiz-Benet
f6111d82fe Merge pull request #558 from jbenet/feat/gc
initial implementation of repo-gc command
2015-01-21 16:48:28 -08:00
Jeromy
8e7d984751 updates from PR, tests tests tests! 2015-01-21 08:55:50 +00:00
Brian Tiger Chow
721ff57263 fix(blockservice) fully async exchange.HasBlock 2015-01-20 23:43:21 -08:00
Jeromy
4de881a185 move GC code into core/repo, and add sharness test 2015-01-21 04:28:02 +00:00
Matt Bell
31ae178078 commands: Support outputting <-chan interface{} 2015-01-21 04:28:02 +00:00
Jeromy
9adcfe7bc2 address concerns from PR 2015-01-21 04:28:02 +00:00
Jeromy
ccb36277dd fix pinning UX, and add tests to match 2015-01-21 04:28:01 +00:00
Jeromy
b91db68056 update pinning to new semantics, and fix a couple bugs 2015-01-21 04:28:01 +00:00
Jeromy
3601a93d17 fix pin command output 2015-01-21 04:28:01 +00:00
Jeromy
f0781d8119 initial implementation of repo-gc command 2015-01-21 04:28:01 +00:00
Juan Batiz-Benet
56ae2fd0a8 routing: record validation into record/
This commit moves the record validation/verification
from dht/ into the new record/ packaage. Validator object
-- which is merely a map of ValidatorFuncs -- with a
VerifyRecord

cc @whyrusleeping
2015-01-19 08:10:37 +00:00
Jeromy
a5233faeae some comments 2015-01-19 08:06:51 +00:00
Jeromy
feeada0d90 fix fuse mounting issues
this time, without loading the private key on every startup
2015-01-19 08:05:52 +00:00
Brian Tiger Chow
797aad9137 refactor(core) extract const 2015-01-18 14:41:43 -08:00
Brian Tiger Chow
b0596cd4ff rename coreio to coreunix 2015-01-18 14:41:43 -08:00
Brian Tiger Chow
a025fc9adf refactor(core) switch style to type assertions
@jbenet cool with this?
2015-01-18 14:41:43 -08:00
Brian Tiger Chow
c1ff2149f1 doc(core) add TODO 2015-01-18 14:41:43 -08:00
Brian Tiger Chow
15a6237ffc fix(core) improve documentation and describe behavior of function 2015-01-18 14:41:43 -08:00
Brian Tiger Chow
19c2c6556f fix(core) don't assume Repo won't be nil 2015-01-18 14:41:43 -08:00
Brian Tiger Chow
de45277883 doc(core, main) 2015-01-18 14:41:43 -08:00
Brian Tiger Chow
1edc5c0a65 refactor(core) Close in teardown
This declarative style is simpler to compose than the imperative wiring
up of objects.

+ pass context to StartOnlineServices as parameter. one by one, trying
to remove dependencies on node state so these initialization steps can
be broken down.
2015-01-18 14:41:43 -08:00
Brian Tiger Chow
ffe6f9a949 doc(core_io/Add)
@jbenet wasn't sure about the terminology here. I'm certain what I wrote
is not entirely correct. Feel free to edit.
2015-01-18 14:41:42 -08:00
Brian Tiger Chow
3dd68ba5c5 fix(core_io/Add): by default, use pinner
@whyrusleeping @jbenet does passing this argument effectively instruct
the dagbuilder to pin recursively?
2015-01-18 14:41:42 -08:00
Brian Tiger Chow
4b5ff960c9 feat(core): use repo.Repo in core constructor 2015-01-18 14:41:42 -08:00
Brian Tiger Chow
884143bf9c move generic packages to thirdparty (see thirdparty/README.md) 2015-01-18 14:04:45 -08:00
Juan Batiz-Benet
132add2ffd core/mount: make non-darwin archs see fuseversion
See the note:

// this file is only here to prevent go src tools (like godep) from
// thinking fuseversion is not a required package by non-darwin archs.
2015-01-18 12:38:18 -08:00
Juan Batiz-Benet
2963f48f62 Merge pull request #554 from jbenet/feat/reprovide
basic reprovider implementation
2015-01-17 03:03:01 -08:00
Juan Batiz-Benet
00b099c71f core/cmds: fix go-fuse-version path 2015-01-17 02:45:17 -08:00
Jeromy
acfc35212e add backoff retry to reprovider 2015-01-17 04:31:50 +00:00
Jeromy
a7650b259d add test for reprovider and slight refactor 2015-01-14 22:14:52 +00:00
Juan Batiz-Benet
01283b92f7 Merge pull request #556 from jbenet/net-diag-viewer
diag/net: visualizing in d3 and dot
2015-01-14 10:32:56 -08:00
Brian Tiger Chow
40e41d24f7 feat(fsrepo): protect with a repo lockfile
NB: daemon is one spot the repo lock is typically acquired
2015-01-14 03:02:23 -08:00
Juan Batiz-Benet
b8757d18ef diag/net: visualizing in d3 and dot
Try it out:

```
ipfs net diag --vis=d3 | diagnostics/d3/d3view
```

Notes: this is not the best way to do it, because it
breaks `--encoding=json`. Not sure what the best way is, and
right now this provides more utility than the other.
2015-01-13 23:49:35 -08:00
Juan Batiz-Benet
920ddc7fc3 Merge pull request #553 from jbenet/serial-file
Serial File Opening
2015-01-13 23:30:08 -08:00
Matt Bell
86bb8ca7a7 core/commands: Enabled stdin for various command arguments 2015-01-13 22:51:11 -08:00
Matt Bell
70adc21d08 core/commands: Enable stdin arguments for 'add' and 'cat', resolves #542 2015-01-13 22:51:10 -08:00
Matt Bell
7a579bbf79 commands: Moved files code into 'commands/files' subpackage 2015-01-13 22:50:53 -08:00
Juan Batiz-Benet
f068c89147 reprovide: move core's reprovider into online services 2015-01-13 22:12:37 -08:00
Jeromy
3269986e42 basic reprovider implementation
make vendor
2015-01-13 22:12:37 -08:00
Brian Tiger Chow
3eafb3e5ab style(repo): repo.Interface -> repo.Repo
The pkg.Interface style is modeled after heap.Interface. Generally, I
find it helpful for interfaces that have many implementations. It
provides clear distinction between the generic interface and the |n|
implementations that implement it (which may be interface types
themselves).

For clients who cannot keep the repo name, one can imagine that the most
likely rename is `ipfsrepo`. In that case, `ipfsrepo.Interface` remains
meaningful.

This is low-pri so it doesn't matter than much. But for the record, the
repo.Interface feels appropriate in this use-case.
2015-01-13 03:09:33 -08:00
Brian Tiger Chow
86258face6 refactor(config, repo): use repo.Interface to (Get|Set)ConfigKey 2015-01-13 03:09:32 -08:00
Brian Tiger Chow
1b700864f9 refactor(repo): all config writes must go through repo 2015-01-13 03:09:31 -08:00
Brian Tiger Chow
405afd2a02 refactor(config, repo): all writes go through FSRepo. next: privatize these 2015-01-13 03:09:31 -08:00
Brian Tiger Chow
e28a235f77 it's unclear what this test case is meant to protect, but passing an empty datastore value no longer has bearing on node construction. Datastore setup now occurs in repo. 2015-01-13 03:09:31 -08:00
Brian Tiger Chow
6e730f4c73 bad datastore type won't cause failure 2015-01-13 03:09:31 -08:00
Brian Tiger Chow
20e1610075 nil will no longer be a valid argument 2015-01-13 03:09:31 -08:00
Brian Tiger Chow
5b3164de5c refactor(repo/config) move config under repo 2015-01-13 03:09:30 -08:00
Juan Batiz-Benet
7aa4a83f2e addr: proper filter + subtract 2015-01-12 20:49:06 -08:00
Juan Batiz-Benet
ceadfbe791 core: some logging for addresses 2015-01-12 19:15:02 -08:00
Juan Batiz-Benet
0cb0f27b68 bootstrap: print error when failed 2015-01-12 19:15:02 -08:00
Juan Batiz-Benet
7ec1a674e3 p2p/net/swarm: do not usre link local addrs 2015-01-12 19:15:02 -08:00
Jeromy
a768e841bc update printout for net diag 2015-01-12 02:55:15 -08:00
Juan Batiz-Benet
7867e01c90 ipfs ping wording 2015-01-11 20:52:34 -08:00
Juan Batiz-Benet
f7941e9841 ping: use context 2015-01-11 20:52:34 -08:00
Juan Batiz-Benet
374a75b612 refs: tie the contexts together 2015-01-11 20:52:33 -08:00
Juan Batiz-Benet
92e8a7bcd5 updated datastore for proper query handling
Queries now can be cancelled and the resources collected
2015-01-11 20:52:33 -08:00
Juan Batiz-Benet
708e47fcbc ipfs blocks local command
ipfs blocks local returns _all_ local refs. For now this is one
long op. future commits will make it async.
2015-01-11 16:59:22 -08:00
Brian Tiger Chow
63c0d41614 fix(core): perform sophisticated boostrap operation 2015-01-11 01:23:27 -08:00
Brian Tiger Chow
033e33dadf feat(core): expose IpfsNode.Resolve 2015-01-11 01:23:27 -08:00
Brian Tiger Chow
223ee4df1a refactor(core): move Add, Cat to core/io 2015-01-11 01:23:27 -08:00
Brian Tiger Chow
ddf14bee78 refactor(core): replace online bool with mode type 2015-01-11 01:23:27 -08:00
Brian Tiger Chow
130532c532 rm errTODO 2015-01-11 01:23:27 -08:00
Brian Tiger Chow
b0f7143250 refactor(core): distinguish repo.Repo components 2015-01-11 01:23:27 -08:00
Brian Tiger Chow
9d438843bd refactor(core): distinguish Online services 2015-01-11 01:23:27 -08:00
Brian Tiger Chow
ca8190a889 refactor(core): isolate complex DHT initialization 2015-01-11 01:23:27 -08:00
Brian Tiger Chow
eb6c40d5dd refactor(core): isolate the complex initialization of PeerHost construction 2015-01-11 01:23:27 -08:00
Brian Tiger Chow
8e42e86d87 hack(core): instantiate peerstore at the end if it hasn't already been instantiated
this is a dirty hack
2015-01-11 01:23:26 -08:00
Brian Tiger Chow
707874c3a5 refactor(core): init node.Blocks in shared NewIPFSNode constructor 2015-01-11 01:23:26 -08:00
Brian Tiger Chow
34e8f3ee26 refactor(core): init DAG and its dependents in shared constructor
fix: remove merkledag import

may need to squash this commit into the merkledag move commit
2015-01-11 01:23:26 -08:00
Brian Tiger Chow
7fa5d81093 refactor(core): rename 2015-01-11 01:23:26 -08:00
Brian Tiger Chow
efb75ee5ba refactor: move add and cat to the core 2015-01-11 01:23:25 -08:00
Brian Tiger Chow
2c3fb43350 feat: expose IpfsNode.Bootstrap() method 2015-01-11 01:23:25 -08:00
Brian Tiger Chow
4c2eda2123 refactor: remove deprecated function 2015-01-11 01:23:25 -08:00
Brian Tiger Chow
57b3ffa533 feat: new core constructor + config options (Standard, Online, Offline) 2015-01-11 01:23:25 -08:00
Juan Batiz-Benet
456719ede7 Merge pull request #523 from jbenet/feat/ping
Implement ipfs ping
2015-01-10 08:11:03 -08:00
Juan Batiz-Benet
1bd69b19ac core: make sure to call swarm.FilterAddrs
also move network setup into its own self-contained func
2015-01-10 08:07:19 -08:00
Juan Batiz-Benet
538550f8f8 fix osx mounting version checks 2015-01-10 01:07:43 -08:00
Jeromy
0f4f91c2f3 restore ability to ping by ID alone 2015-01-10 08:31:17 +00:00
Jeromy
1561291d61 improve UI of ping 2015-01-10 07:27:49 +00:00
Jeromy
9fc0f86a3a cleanup from PR 2015-01-10 07:27:49 +00:00
Jeromy
50c8561bdd add peer info after FindPeer RPC
fix ping test
2015-01-10 07:27:49 +00:00
Jeromy
566a86f5d4 Address PR comments and add in more user feedback 2015-01-10 07:27:48 +00:00
Jeromy
0794d5b46a a little cleanup 2015-01-10 07:27:48 +00:00
Brian Tiger Chow
069966d55a ping WIP
License: MIT
Signed-off-by: Brian Tiger Chow <brian@perfmode.com>

Conflicts:
	core/commands/root.go

begin ping command, WIP

finish initial ping implementation
2015-01-10 07:27:45 +00:00
Juan Batiz-Benet
d1bff333a5 refs: log writer error 2015-01-07 13:49:13 -08:00
Juan Batiz-Benet
180b9b32d3 refs: added --format option 2015-01-07 13:42:06 -08:00
Juan Batiz-Benet
f46120b194 refs edges + stream fix 2015-01-07 12:56:14 -08:00
Juan Batiz-Benet
8307572777 ipfs block stat bugfix: cast 2015-01-07 06:26:34 -08:00
Juan Batiz-Benet
6b2795338f ipfs object has learned stat
'ipfs object stat' is a plumbing command to print DAG
node statistics. <key> is a base58 encoded multihash.
It outputs to stdout:

  NumLinks        int number of links in link table
  BlockSize       int size of the raw, encoded data
  LinksSize       int size of the links segment
  DataSize        int size of the data segment
  CumulativeSize  int cumulative size of object and references
2015-01-07 02:31:29 -08:00
Juan Batiz-Benet
07b923d7a4 ipfs block stat cmd 2015-01-07 00:55:27 -08:00
Matt Bell
6236ef7fdb core/commands: Made add command output a channel
core/commands: Don't use pointers for Type field
2015-01-06 15:55:16 -08:00
Matt Bell
abd390b892 core/commands: Made add command show streamed output 2015-01-06 15:53:48 -08:00
Matt Bell
fd40702f73 commands: Changed Marshaler to return a io.Reader instead of a []byte
core/commands: Refactored command marshalers
2015-01-06 15:53:42 -08:00
Juan Batiz-Benet
0395a7af1e ipfs add output not sorted, cmds files sorted
I made the commands lib dir listing sort the contents
so we get the same sequence of files from it repeatably.
2015-01-06 14:02:49 -08:00
Christian Couder
d490a85706 Make sure ipfs add output is sorted by name
License: MIT
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
2015-01-06 13:12:22 -08:00
Juan Batiz-Benet
64bf493b54 swarm peers print pid.Pretty 2015-01-06 10:13:49 -08:00
Juan Batiz-Benet
fbba7c8f39 bootstrap: only dht bootstrap with live conns 2015-01-06 10:10:20 -08:00
Juan Batiz-Benet
9e67acf51a bootstrap add --default option 2015-01-06 04:30:06 -08:00
Juan Batiz-Benet
517d3e99b8 bootstrap addrs: moved parsing around
this commit:
* moves parsing of bootstrap peers into config
* moves location of bootstrap addrs into core/commands
* refactor `*BootstrapPeer -> BootstrapPeer
2015-01-06 00:25:17 -08:00
Juan Batiz-Benet
6eb32aef35 bootstrap err: info 2015-01-05 09:00:58 -08:00
Juan Batiz-Benet
730793e2f7 cmds/bootstrap: rm --all 2015-01-05 07:59:53 -08:00
Juan Batiz-Benet
dfcea4c6f1 bootstrap: not error to not have enough bootstrap peers
use dht bootstrap. there is an edge case where the dht
is tiny (1?) and we have 0 bootstrap peers. we should
probably _inform_ the user, but this may be more a
webui or command thing.
2015-01-05 07:13:51 -08:00
Brian Tiger Chow
30eb8f213f adjust const
to test jenkins

License: MIT
Signed-off-by: Brian Tiger Chow <brian@perfmode.com>
2015-01-05 06:25:56 -08:00
Brian Tiger Chow
9e4b9714ce feat(core) dht.Bootstrap
License: MIT
Signed-off-by: Brian Tiger Chow <brian@perfmode.com>
2015-01-05 06:25:25 -08:00
Juan Batiz-Benet
b599e28b88 core/mock: use mock routing
this fixes a failing ipns test which didnt have
a "working" routing system
2015-01-05 00:16:53 -08:00
Juan Batiz-Benet
e21b1f662b fuse mount lifecycle fixes
This commit cleans up how mounting was being done.
It now successfully signals when it is properly mounted
and listen to close signals correctly.
2015-01-04 00:47:27 -08:00
Juan Batiz-Benet
1fa14335b9 core: rearranged initialization a bit 2015-01-04 00:47:27 -08:00
Juan Batiz-Benet
8fb5cf9578 swap net2 -> net 2015-01-02 08:46:46 -08:00
Juan Batiz-Benet
e2698a8733 net -> p2p/net
The net package is the next to move. It will be massaged
a bit still to fix the Network / "NetworkBackend" conflict.
2015-01-02 08:46:45 -08:00
Juan Batiz-Benet
cc0d7c9b57 crypto -> p2p/crypto
The crypto package moves into p2p. Nothing in it so far is ipfs
specific; everything is p2p-general.
2015-01-02 08:46:45 -08:00
Juan Batiz-Benet
89f5cd4c94 introducing p2p pkg
I think it's time to move a lot of the peer-to-peer networking
but-not-ipfs-specific things into its own package: p2p.
This could in the future be split off into its own library.
The first thing to go is the peer.
2015-01-02 08:46:45 -08:00
Juan Batiz-Benet
49cb135ca2 ipfsnet -> swarmnet
swarmnet is a better name for the package, because
it's just a Network implemented with a Swarm.
(ipfsnet will be something slightly different).
2015-01-02 08:46:44 -08:00
Juan Batiz-Benet
4807127def net: move Network implementation to own pkg
I needed the network implementation in its own
package, because I'll be writing several services that
will plug into _it_ that shouldn't be part of the core net
package. and then there were dependency conflicts. yay.
mux + identify are good examples of what i mean.
2015-01-02 08:46:43 -08:00
Jeromy
14fc4188be address comments from PR 2015-01-02 07:42:08 +00:00
Jeromy
07b064010e rewrite of provides to better select peers to send RPCs to
refactor test peer creation to be deterministic and reliable

a bit of cleanup trying to figure out TestGetFailure

add test to verify deterministic peer creation

switch put RPC over to use getClosestPeers

rm 0xDEADC0DE

fix queries not searching peer if its not actually closer
2015-01-02 07:42:07 +00:00
Brian Tiger Chow
07fa9eecbd hotfix(core/mock) set Network field 2014-12-25 15:20:54 -05:00
Brian Tiger Chow
ca32a83394 wip with DHT
@whyrusleeping @jbenet this is a WIP with the DHT.

wip

License: MIT
Signed-off-by: Brian Tiger Chow <brian@perfmode.com>

Conflicts:
	epictest/addcat_test.go
	exchange/bitswap/testnet/peernet.go
	exchange/bitswap/testutils.go
	routing/mock/centralized_server.go
	routing/mock/centralized_test.go
	routing/mock/interface.go

fix(routing/mock) fill in function definition
2014-12-24 09:31:19 -05:00
Brian Tiger Chow
27dc9594ba refactor(bitswap) bitswap.Network now abstracts ipfs.Network + ipfs.Routing
@jbenet @whyrusleeping

the next commit will change bitswap.Network.FindProviders to only deal
with IDs
2014-12-23 09:19:21 -08:00
Juan Batiz-Benet
64a4abcd6c listen addresses: add Interface addresses
network.ListenAddresses() are general.
Interface addresses are specific.
2014-12-23 09:01:54 -08:00
Juan Batiz-Benet
7952d95bbf added bootstrap logging 2014-12-23 08:48:58 -08:00