daemon output now includes initial swarm addresses. this is not a
full solution, as a change in network will not trigger re-printing.
We need a good way to do that.
This made me re-think how we're outputting these messages, perhaps
we should be throwing them as log.Events, and capturing some with
a special keyword to output to the user on stdout. Things like
network addresses being rebound, NATs being holepunched, external
network addresses being figured out, connections established, etc
may be valuable events to show the user. Of course, these should be
very few, as a noisy daemon is an annoying daemon.
License: MIT
Signed-off-by: Juan Batiz-Benet <juan@benet.ai>
Except when there is an explicit os.Exit(1) after the Critical line,
then replace with Fatal{,f}.
golang's log and logrus already call os.Exit(1) by default with Fatal.
License: MIT
Signed-off-by: rht <rhtbot@gmail.com>
This configuration is supported since GoBuilder 1.16.0 so now we should be able to build windows binaries in GoBuilder
License: MIT
Signed-off-by: Knut Ahlers <knut@ahlers.me>
I think we should lower the default rsa key size to 2048 for now -- until we have a proper focus on securing everything. It's always a pain for new users to get hung on 4096 rsa key gen, when we have not even made sure we're using the keys perfectly correctly yet. (And 2048 is still considered secure)
If the daemon is running we do not want to proceed with an
an initialization.
Return a client error telling the user to kill the daemon
before proceeding with the command.
Instead of just terminating right there and then, we cancel the
context, and let the daemon exit cleanly. This make take a few
seconds, as the node builder and its child processes do not
care too much about the context state while building nodes,
but this can be improved by injecting checks for ctx.Done()
before time-consuming steps.
Instead of assuming the command is the daemon command and closing
the node, which resulted in bugs like #1053, we cancel the context
and let the context children detect the cancellation and gracefully
clean up after themselves.
The shutdown logging has been moved into the daemon command, where
it makes more sense, so that commands like ping will not print out
the same output on cancellation.
If a command invocation such as 'daemon' is interrupted, the interrupt
handler asks the node to close. The closing of the node will result in
the command invocation finishing, and possibly returning from main()
before the interrupt handler is done. In particular, the info logging
that a graceful shutdown was completed may never reach reach stdout.
As the whole point of logging "Gracefully shut down." is to give
confidence when debugging that the shutdown was clean, this is
slightly unfortunate.
The interrupt handler needs to be set up in main() instead of Run(),
so that we can defer the closing of the interrupt handler until just
before returning from main, not when Run() returns with a streaming
result reader.
This reverts commit f74e71f965.
The 'Online' flag of the command context does not seem to be set in
any code paths, at least not when running commands such as 'ipfs daemon'
or 'ipfs ping'. The result after f74e71f9 is that we never shutdown
cleanly, as we'll always os.Exit(0) from the interrupt handler.
The os.Exit(0) itself is also dubious, as conceptually the interrupt
handler should ask whatever is stalling to stop stalling, so that
main() can return like normal. Exiting with -1 in error cases where
the interrupt handler is unable to stop the stall is fine, but the
normal case of interrupting cleanly should exit through main().
We now consider debugerrors harmful: we've run into cases where
debugerror.Wrap() hid valuable error information (err == io.EOF?).
I've removed them from the main code, but left them in some tests.
Go errors are lacking, but unfortunately, this isn't the solution.
It is possible that debugerros.New or debugerrors.Errorf should
remain still (i.e. only remove debugerrors.Wrap) but we don't use
these errors often enough to keep.
This commit adds a new set of sharness tests for pinning, and addresses
bugs that were pointed out by said tests.
test/sharness: added more pinning tests
Pinning is currently broken. See issue #1051. This commit introduces
a few more pinning tests. These are by no means exhaustive, but
definitely surface the present problems going on. I believe these
tests are correct, but not sure. Pushing them as failing so that
pinning is fixed in this PR.
make pinning and merkledag.Get take contexts
improve 'add' commands usage of pinning
FIXUP: fix 'pin lists look good'
ipfs-pin-stat simple script to help check pinning
This is a simple shell script to help check pinning.
We ought to strive towards making adding commands this easy.
The http api is great and powerful, but our setup right now
gets in the way. Perhaps we can clean up that area.
updated t0081-repo-pinning
- fixed a couple bugs with the tests
- made it a bit clearer (still a lot going on)
- the remaining tests are correct and highlight a problem with
pinning. Namely, that recursive pinning is buggy. At least:
towards the end of the test, $HASH_DIR4 and $HASH_FILE4 should
be pinned indirectly, but they're not. And thus get gc-ed out.
There may be other problems too.
cc @whyrusleeping
fix grep params for context deadline check
fix bugs in pin and pin tests
check for block local before checking recursive pin
this more clearly explains that the daemon is listening on the network. it also hints as to how to change the listening ports. this is also related to #874
- updated go-ctxgroup and goprocess
ctxgroup: AddChildGroup was changed to AddChild. Used in two files:
- p2p/net/mock/mock_net.go
- routing/dht/dht.go
- updated context from hg repo to git
prev. commit in hg was ad01a6fcc8a19d3a4478c836895ffe883bd2ceab. (context: make parentCancelCtx iterative)
represents commit 84f8955a887232b6308d79c68b8db44f64df455c in git repo
- updated context to master (b6fdb7d8a4ccefede406f8fe0f017fb58265054c)
Aaron Jacobs (2):
net/context: Don't accept a context in the DoSomethingSlow example.
context: Be clear that users must cancel the result of WithCancel.
Andrew Gerrand (1):
go.net: use golang.org/x/... import paths
Bryan C. Mills (1):
net/context: Don't leak goroutines in Done example.
Damien Neil (1):
context: fix removal of cancelled timer contexts from parent
David Symonds (2):
context: Fix WithValue example code.
net: add import comments.
Sameer Ajmani (1):
context: fix TestAllocs to account for ints in interfaces
This allows someone to host a static site by pointing a TXT record at their
content in IPFS, and a CNAME record at an IPFS gateway.
Note that such a setup technically violates RFC1912 (section 2.4; "A CNAME
record is not allowed to coexist with any other data."), but tends to work in
practice.
We may want to consider changing the DNS->IPFS resolution scheme to allow this
scenario to be RFC-compliant (e.g. store the mapping on a well-known subdomain
to allow CNAME records on the domain itself).
License: MIT
Signed-off-by: Kevin Wallace <kevin@pentabarf.net>
fix(cmd/daemon) set node on req context since it expects daemon to be
present when handling commands over the API
@jbenet revenge of the request context redux
- core: daemon stdout print to cmd + daemon init checks
- core: fixed bug where the gateway was printed as "API"
- sharness/test-lib: daemon init checks
- sharness/test-lib: portable TCP port check
- sharness/init: fix test bits output
- sharness: use common hashes in one place.
- move t0100-http-gateway -> t0111-gateway-writable
- sharness: test-lib funcs for gateway config
- sharness/t0111-gateway-writable: use sh funcs
- sharness/t0111-gateway-writable: fixes
- escape all vars (always `cmd "$VAR"` never `cmd $VAR`)
- use $FILEPATH, not $path
- last test seems to fail
improve efficiency of multilayered indirect blocks
clean up tests
panic cleanup
clean up logic, improve readability
add final root node to the dagservice upon creation
importer: simplified dag generation
test: updated hashes using latest code
@whyrusleeping this is why the sharness tests
were failing: the hashes are added manually to
make sure our generation doesn't change.
cleanup after CR
fix merkledag tests
fix small block generation (no subblocks!)
this commit:
* moves parsing of bootstrap peers into config
* moves location of bootstrap addrs into core/commands
* refactor `*BootstrapPeer -> BootstrapPeer
I think it's time to move a lot of the peer-to-peer networking
but-not-ipfs-specific things into its own package: p2p.
This could in the future be split off into its own library.
The first thing to go is the peer.
this is a major refactor of the entire codebase
it changes the monolithic peer.Peer into using
a peer.ID and a peer.Peerstore.
Other changes:
- removed handshake3.
- testutil vastly simplified peer
- secio bugfix + debugging logs
- testutil: RandKeyPair
- backpressure bugfix: w.o.w.
- peer: added hex enc/dec
- peer: added a PeerInfo struct
PeerInfo is a small struct used to pass around a peer with
a set of addresses and keys. This is not meant to be a
complete view of the system, but rather to model updates to
the peerstore. It is used by things like the routing system.
- updated peer/queue + peerset
- latency metrics
- testutil: use crand for PeerID gen
RandPeerID generates random "valid" peer IDs. it does not
NEED to generate keys because it is as if we lost the key
right away. fine to read some randomness and hash it. to
generate proper keys and an ID, use:
sk, pk, _ := testutil.RandKeyPair()
id, _ := peer.IDFromPublicKey(pk)
Also added RandPeerIDFatal helper
- removed old spipe
- updated seccat
- core: cleanup initIdentity
- removed old getFromPeerList
Mistake: Saturn referred to the NY node and vice versa.
Correction: Fittingly, Uranus is in NY and _S_aturn is _S_ingapore.
I'll have to reboot the testnet to rectify the changes.
License: MIT
Signed-off-by: Brian Tiger Chow <brian@perfmode.com>
This commit applies the logging settings to the logging system upon
initialization `ipfs init` and `ipfs daemon --init`.
License: MIT
Signed-off-by: Brian Tiger Chow <brian@perfmode.com>
It looks like files are not ignored when they are listed
starting with ./ in a .gitignore file.
At least this is true for me on Linux for the "ipfs"
binary in "cmd/ipfs/.gitignore".
License: MIT
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
I broke out multiaddr/net to its own package so
that multiaddr parsing could remain lean. multiaddr-net
will vendor special implementations of protocols (like utp)
Warning: during normal execution node teardown must be the
last thing that happens because command requests return
io.Readers, which may still be constructing or processing
their output. The node (and its subservices) is needed for
this. good night and good luck.
@jbenet
now, ipfs can be built and executed in one step:
```
docker run jbenet/go-ipfs daemon --initialized
```
License: MIT
Signed-off-by: Brian Tiger Chow <brian@perfmode.com>
@jbenet @whyrusleeping @mappum
very helpful for tracking down errors. the stack traces are only
shown when debug mode is visible. They function best when caught at the
source.
I propose we use this errors package as a drop-in replacement for
fmt.Errorf and errors.New in all of our code, and use errors.Wrap for
external errors as they emerge from others' libraries.
License: MIT
Signed-off-by: Brian Tiger Chow <brian@perfmode.com>
Signed-off-by: Brian Tiger Chow <brian.holderchow@gmail.com>
@jbenet this commit re-introduces the peer identity line. It's very
useful. I understand it may have been removed because of the clashing of
the IDs. To alleviate this, this commit places some negative space
between the two lines.
'to test' -> 'to get started' as a stronger call to action
```
λ. ipfs2 init -f
initializing ipfs node at /Users/btc/.go-ipfs
generating key pair...done.
peer identity: QmWzjxNEYKjDAxuHJqvtLP1dZTDjreBSUsArWoHai1v9yP
to get started, enter: ipfs cat QmYpv2VEsxzTTXRYX3PjDg961cnJE3kY1YDXLycHGQ3zZB
```
License: MIT
Signed-off-by: Brian Tiger Chow <brian@perfmode.com>
TODO use a writer to get the command output
resulting appearance:
```
go-ipfs (feat/machine-readable-logging) λ. make install_2; ipfs2 init -f
cd cmd/ipfs2 && go install
initializing ipfs node at /Users/btc/.go-ipfs
generating key pair...
peer identity: %s QmcPLgrqfvnneJ3vr3oGpkTiS9Psx8p3HxevjnoZYToDRm
```
License: MIT
Signed-off-by: Brian Tiger Chow <brian@perfmode.com>
TODO allow user dir override
License: MIT
Signed-off-by: Brian Tiger Chow <brian@perfmode.com>
# TYPES
# feat
# fix
# docs
# style (formatting, missing semi colons, etc; no code change):
# refactor
# test (adding missing tests, refactoring tests; no production code change)
# chore (updating grunt tasks etc; no production code change)
Signed-off-by: Brian Tiger Chow <brian.holderchow@gmail.com>
This commit changes the signal handler to be added once
the command is executing. this is because the daemon
has its own signal handler, that must try to shut down
the node gracefully first. You know, just in case.
This commit adds a Mount abstraction (which is really just
a wrapped context closer). It makes sure to bind the mount
to the fate of the Node (i.e. close it if the node ends).
This fixes#350
@jbenet @mappum
Yeah, there's some duplicated work. But there's also a separation of
concerns. In one case, we check to determine where the command should
run. In the other case, we check to determine which hooks should run.
Having these actions separated reduces complexity in a nice way.
License: MIT
Signed-off-by: Brian Tiger Chow <brian@perfmode.com>
License: MIT
Signed-off-by: Brian Tiger Chow <brian@perfmode.com>
impl errorf
License: MIT
Signed-off-by: Brian Tiger Chow <brian@perfmode.com>
return a debug error
License: MIT
Signed-off-by: Brian Tiger Chow <brian@perfmode.com>
not immediately useful, but nice to have
tagging you to make sure i didn't make a mistake here @jbenet
License: MIT
Signed-off-by: Brian Tiger Chow <brian@perfmode.com>
eg.
```
ipfs2 (cmd-ref-part2) λ. go build ./...; ./ipfs2 tour 0
Tour 0 - Hello Mars
Hello Mars
ipfs2 (cmd-ref-part2) λ. go build ./...; ./ipfs2 tour 10
ERROR
no topic with id: 10
TOPICS
0 - Hello Mars
0.1 - Hello Mars 2
```
This commit adds the pretty-complicated decision function
to check whether a command should run on the daemon.
@maybebtc @mappum double check the logic?
Attention @maybebtc @mappum
I cleaned up + simplified the main flow. Now, all printing
is contained inside main itself! (:cheer:). I do this with
the help of a cmdInvocation struct that has both
a Parse and Run. The only major clunkiness left is that the
"CallCommand" is still its own function. But *shrug*.
Please test it works as we would expect. i changed much of
the flow, so likely that i missed a complicated edge case.
main roadmap:
- parse the commandline to get a cmdInvocation
- if user requests, help, print it and exit.
- run the command invocation
- output the response
- if anything fails, print error, maybe with help
