Use fsrepo.IsInitialized to test for initialization instead of just
testing if the directory exists.
Also add test cases for '--init' option, including one the creates an
empty directory.
License: MIT
Signed-off-by: Kevin Atkinson <k@kevina.org>
This commit introduces non-recursive Makefile infrastructure that replaces current Makefile infrastructure.
It also generally cleanups the Makefiles, separates them into nicer sub-modules and centralizes common operations into single definitions.
It allows to depend on any target that is defined in the makefile, this means that for example `gx install` is called once when `make build test_expensive_sharness` is called instead of 4 or 5 times.
It also makes the dependencies much cleaner and allows for reuse of modules. For example sharness coverage collection (WIP) uses sharness target with amended PATH, previously it might have been possible but not without wiring in the coverage collection into sharness make runner code.
Yes, it is more complex but not much more. There are few rules that have to be followed and few complexities added but IMHO it is worth it.
How to NR-make:
1. If make is to generate some file via a target, it MUST be defined in Rules.mk file in the directory of the target.
2. `Rules.mk` file MUST have `include mk/header.mk` statement as the first line and `include mk/footer.mk` statement as the last line (apart from project root `Rules.mk`).
3. It then MUST be included by the closest `Rules.mk` file up the directory tree.
4. Inside a `Rules.mk` special variable accessed as `$(d)` is defined. Its value is current directory, use it so if the `Rules.mk` file is moved in the tree it still works without a problem. Caution: this variable is not available in the recipe part and MUST NOT be used. Use name of the target or prerequisite to extract it if you need it.
5. Make has only one global scope, this means that name conflicts are a thing. Names SHOULD follow `VAR_NAME_$(d)` convention. There are exceptions from this rule in form of well defined global variables. Examples: General lists `TGT_BIN`, `CLEAN`; General targets: `TEST`, `COVERAGE`; General variables: `GOFLAGS`, `DEPS_GO`.
3. Any rules, definitions or variables that fit some family SHOULD be defined in `mk/$family.mk` file and included from project root `Rules.mk`
License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protonmail.ch>
Tried to check all instances of IPFS and make sure they werent referring to the CLI tool. See #2910.
License: MIT
Signed-off-by: Richard Littauer <richard.littauer@gmail.com>
Return to the old definition of the mv command since there is no `-t`
parameter in `mv` of BSD. Cf.
https://www.freebsd.org/cgi/man.cgi?query=mv&sektion=1
License: MIT
Signed-off-by: Stephan Kulla <git.mail@kulla.me>
I removed the comment about using $PATH since it leads to long
installation scripts (which violates the KISS principle). Cf. the
discussion on https://github.com/ipfs/go-ipfs/pull/2504
License: MIT
Signed-off-by: Stephan Kulla <git.mail@kulla.me>
Show error message that the user shall try running this script with sudo
in case write permissions are missing. Implement proposal of comment
https://github.com/ipfs/go-ipfs/pull/3194#issuecomment-245376993
License: MIT
Signed-off-by: Stephan Kulla <git.mail@kulla.me>
I moved the mv command into the if-condition so that the script only
succeeds when mv command ran properly. Thus, there is no need to check
whether the mv command will succeed beforehand.
License: MIT
Signed-off-by: Stephan Kulla <git.mail@kulla.me>
In case $binpath/$bin is an already existing directory, the command
mv "$bin" "$binpath/$bin"
would store the binary in the place $binpath/$bin/$bin. I guess this is
not the expected behavior. Therefore I used the -t option of `mv' to
specify the target directory explicitly.
License: MIT
Signed-off-by: Stephan Kulla <git.mail@kulla.me>
* Update golog in go-ipfs
License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protonmail.ch>
* Update go-libp2p for go-log
License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protonmail.ch>
* Update go-libp2p-secio for go-log
License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protonmail.ch>
* Update go-libp2p-crypto for go-log
License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protonmail.ch>
* Update go-libp2p-peer for go-log
License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protonmail.ch>
* Import peersore, it wasn't imported
License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protonmail.ch>
* Update peerstore
License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protonmail.ch>
* Update peer
License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protonmail.ch>
* Update secio
License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protonmail.ch>
* Update go-libp2p
License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protonmail.ch>
This patch is in preparation for the gateway's extraction.
It's interesting to trace technical debt back to its
origin, understanding the circumstances in which it
was introduced and built up, and then cutting it back
at exactly the right places.
- Clean up the gateway's surface
The option builder GatewayOption() now takes only
arguments which are relevant for HTTP handler muxing,
i.e. the paths where the gateway should be mounted.
All other configuration happens through the
GatewayConfig object.
- Remove BlockList
I know why this was introduced in the first place,
but it never ended up fulfilling that purpose.
Somehow it was only ever used by the API server,
not the gateway, which really doesn't make sense.
It was also never wired up with CLI nor fs-repo.
Eventually @krl started punching holes into it
to make the Web UI accessible.
- Remove --unrestricted-api
This was holes being punched into BlockList too,
for accessing /ipfs and /ipn on the API server.
With BlockList removed and /ipfs and /ipns freely
accessible, putting this option out of action
is safe. With the next major release,
the option can be removed for good.
License: MIT
Signed-off-by: Lars Gierth <larsg@systemli.org>
According to golang standards, these should not be capitalized nor having a trailing period, AFAIK.
License: MIT
Signed-off-by: Richard Littauer <richard.littauer@gmail.com>
Modified init command to receive default configuration from stdin. The changes enable us to use existing key-pair, datastore configuration while initializing new ipfs node.
License: MIT
Signed-off-by: Sivachandran <sivachandran.p@gmail.com>
* reduces help indent from 4 to 2 spaces
License: MIT
Signed-off-by: Stephen Whitmore <noffle@ipfs.io>
* reduces horz/vert space taken by "ipfs" cmd
License: MIT
Signed-off-by: Stephen Whitmore <noffle@ipfs.io>
* show subcommands on shorthelp
License: MIT
Signed-off-by: Stephen Whitmore <noffle@ipfs.io>
* Drops colons at the end of cmd headings.
This makes command headings consistent with the output of 'ipfs', which
does not include colons.
License: MIT
Signed-off-by: Stephen Whitmore <noffle@ipfs.io>
* more consistent output between short-/long-help
License: MIT
Signed-off-by: Stephen Whitmore <noffle@ipfs.io>
* Puts DESCRIPTION before SUBCOMMANDS.
Users likely want to understand what a command does before worrying
about its subcommands.
License: MIT
Signed-off-by: Stephen Whitmore <noffle@ipfs.io>
* Keeps ipfs cmd from outputting its subcmds twice.
License: MIT
Signed-off-by: Stephen Whitmore <noffle@ipfs.io>
* Removes redundant synopsis from "file"
License: MIT
Signed-off-by: Stephen Whitmore <noffle@ipfs.io>
* Removes extra whitespace from longhelp
License: MIT
Signed-off-by: Stephen Whitmore <noffle@ipfs.io>
* Consistent spacing whether SUBCMDS or not.
License: MIT
Signed-off-by: Stephen Whitmore <noffle@ipfs.io>
* Removes redundant SUBCMD output from ipfs object.
License: MIT
Signed-off-by: Stephen Whitmore <noffle@ipfs.io>
* Removes redundant synopsis from "name"
License: MIT
Signed-off-by: Stephen Whitmore <noffle@ipfs.io>
* Newline after Description only if it exists.
License: MIT
Signed-off-by: Stephen Whitmore <noffle@ipfs.io>
* Removes redundant synopsis from "bootstrap"
License: MIT
Signed-off-by: Stephen Whitmore <noffle@ipfs.io>
* Removes redundant synopsis from "swarm"
License: MIT
Signed-off-by: Stephen Whitmore <noffle@ipfs.io>
* Removes trailing newline in ping help.
License: MIT
Signed-off-by: Stephen Whitmore <noffle@ipfs.io>
* Prints shorthelp on parse error.
License: MIT
Signed-off-by: Stephen Whitmore <noffle@ipfs.io>
* tiny comment fixes
License: MIT
Signed-off-by: Stephen Whitmore <noffle@ipfs.io>
* updates README usage
License: MIT
Signed-off-by: Stephen Whitmore <noffle@ipfs.io>
* Don't include extra whitespace if no .MoreHelp
License: MIT
Signed-off-by: Stephen Whitmore <noffle@ipfs.io>
* description improvements
License: MIT
Signed-off-by: Stephen Whitmore <noffle@ipfs.io>
* Hides the obscure 'file' subcommand.
License: MIT
Signed-off-by: Stephen Whitmore <noffle@ipfs.io>
* Splits 'ipfs daemon' into Short and Long help.
License: MIT
Signed-off-by: Stephen Whitmore <noffle@ipfs.io>
* Removes redundant synopsis from "config"
License: MIT
Signed-off-by: Stephen Whitmore <noffle@ipfs.io>
* Wraps lines to keep from going over 80.
License: MIT
Signed-off-by: Stephen Whitmore <noffle@ipfs.io>
* specify repo separately
License: MIT
Signed-off-by: Stephen Whitmore <noffle@ipfs.io>
* s/structure/hierarchy
License: MIT
Signed-off-by: Stephen Whitmore <noffle@ipfs.io>
* missing .
License: MIT
Signed-off-by: Stephen Whitmore <noffle@ipfs.io>
* Removes trailing colon from 'usage' test.
License: MIT
Signed-off-by: Stephen Whitmore <noffle@ipfs.io>
* Updates sharness test error messages.
License: MIT
Signed-off-by: Stephen Whitmore <noffle@ipfs.io>
* Removes trailing colon from 'usage' test.
License: MIT
Signed-off-by: Stephen Whitmore <noffle@ipfs.io>
* Updates add-symlink to use /bin/sh.
License: MIT
Signed-off-by: Stephen Whitmore <noffle@ipfs.io>
* Removes "hierarchy".
License: MIT
Signed-off-by: Stephen Whitmore <noffle@ipfs.io>
* Updates "ipfs ping" synopsis.
* Updates t0040 with latest wording.
* Removes unnecessary daemon setup.
License: MIT
Signed-off-by: Stephen Whitmore <noffle@ipfs.io>
Move the go commands that should run under cmd/ipfs in the Makefile in
cmd/ipfs rather than doing a "cd cmd/ipfs && go ..." in the root
Makefile.
The "cd cmd/ipfs && go ..." lines causes problems with GNU Emacs's
compilation mode. With the current setup Emacs is unable to jump to
the location of the error outputted by go compiler as it can not find
the source file. The problem is that the embedded "cd" command causes
Emacs's compilation mode to lose track of the current directory and
thus attempts to look for the source file in the wrong directory.
License: MIT
Signed-off-by: Kevin Atkinson <k@kevina.org>
* Adds repo fsck subcommand
Fixes#2457
License: MIT
Signed-off-by: Mike Pfister <pfista@gmail.com>
* Checks for error on file deletion
License: MIT
Signed-off-by: Mike Pfister <pfista@gmail.com>
* Checks if node is online
License: MIT
Signed-off-by: Mike Pfister <pfista@gmail.com>
* Update error checking
License: MIT
Signed-off-by: Michael Pfister <pfista@gmail.com>
* Prevents command from running while daemon is running
License: MIT
Signed-off-by: Michael Pfister <pfista@gmail.com>
* Add newline to command output message
License: MIT
Signed-off-by: Michael Pfister <pfista@gmail.com>
* removing superfluous error
License: MIT
Signed-off-by: Michael Pfister <pfista@gmail.com>
* Adds sharness test for repo fsck command
License: MIT
Signed-off-by: Michael Pfister <pfista@gmail.com>
* Ignore warning if file doesn't exist
License: MIT
Signed-off-by: Michael Pfister <pfista@gmail.com>
* Updating message output
License: MIT
Signed-off-by: Michael Pfister <pfista@gmail.com>
* adding debug statements
License: MIT
Signed-off-by: Michael Pfister <pfista@gmail.com>
* update and add fsck sharness tests
License: MIT
Signed-off-by: Michael Pfister <pfista@gmail.com>
* updating comments
License: MIT
Signed-off-by: Michael Pfister <pfista@gmail.com>
* Use printf in test
Using printf prevents a newline from being printed to the api test file. When
the newline was present, multiaddr threw errors trying to parse the api address
to an integer since the newline character was present.
License: MIT
Signed-off-by: Michael Pfister <pfista@gmail.com>
* updating tests
License: MIT
Signed-off-by: Michael Pfister <pfista@gmail.com>
* removing commented code
License: MIT
Signed-off-by: Michael Pfister <pfista@gmail.com>
With a proper IpfsCollector object and tests, this time.
The collector object makes it easy to add further metrics,
like e.g. bitswap wants/provs.
License: MIT
Signed-off-by: Lars Gierth <larsg@systemli.org>
The gateway accepts an X-Ipfs-Path-Prefix header,
and assumes that it is mounted in a reverse proxy
like nginx, at this path. Links in directory listings,
as well as trailing-slash redirects need to be rewritten
with that prefix in mind.
We don't want a potential attacker to be able to
pass in arbitrary path prefixes, which would end up
in redirects and directory listings, which is why
every prefix has to be explicitly allowed in the config.
Previously, we'd accept *any* X-Ipfs-Path-Prefix header.
Example:
We mount blog.ipfs.io (a dnslink page) at ipfs.io/blog.
nginx_ipfs.conf:
location /blog/ {
rewrite "^/blog(/.*)$" $1 break;
proxy_set_header Host blog.ipfs.io;
proxy_set_header X-Ipfs-Gateway-Prefix /blog;
proxy_pass http://127.0.0.1:8080;
}
.ipfs/config:
"Gateway": {
"PathPrefixes": ["/blog"],
// ...
},
dnslink:
> dig TXT _dnslink.blog.ipfs.io
dnslink=/ipfs/QmWcBjXPAEdhXDATV4ghUpkAonNBbiyFx1VmmHcQe9HEGd
License: MIT
Signed-off-by: Lars Gierth <larsg@systemli.org>
For the rest of the packages in util, move them to thirdparty
and update the references. util is gone!
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
This lets FUSE mounts to track whether they are active or not by
tracking when fs.Serve terminates.
License: MIT
Signed-off-by: Stephen Whitmore <noffle@ipfs.io>
misc/completion/ipfs-completion.bash: add `ipfs stats` to BASH completion
core/commands/mount_unix.go: ensure error is not nil before printing it
contribute.md: fix bibliography indexing in example
core/commands/swarm.go: change tabs to spaces in USAGE message
*: 80-column readability improvements
License: MIT
Signed-off-by: Thomas Gardner <tmg@fastmail.com>
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
rewrite path to filepath in fsrepo
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
remove api file on repo close
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
update function to check normal net.OpErrors
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
use go1.5 syntax to ensure builds on older versions fail
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
fix t0230
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
The first run does not try to connect to API, as an address is not
yet saved in configuration. The second run did not recognize
corrently a refused connection.
Fixes#1661.
License: MIT
Signed-off-by: Marcin Janczyk <marcinjanczyk@gmail.com>
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
use NewNode instead of NewIPFSNode in most of the codebase
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
make mocknet work with node constructor better
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
finish cleanup of old construction method
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
blockservice.New doesnt return an error anymore
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
break up node construction into separate function
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
add error case to default filling on node constructor
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
This commit adds support for the --api option, which allows users
to specify an API endpoint to run the cli command against. It enables
much easier control of remote daemons.
It also
- ensures the API server version matches the API client
- implements support for the $IPFS_PATH/api file
Still TODO:
- tests!
- multiaddr to support /dns/
License: MIT
Signed-off-by: Juan Batiz-Benet <juan@benet.ai>
ServeOptions take the node and muxer, they should get the listener
too as sometimes they need to operate on the listener address.
License: MIT
Signed-off-by: Juan Batiz-Benet <juan@benet.ai>
License: MIT
Signed-off-by: Henry <cryptix@riseup.net>
t0080-repo.sh: added gateway assets to pinning tests
License: MIT
Signed-off-by: Henry <cryptix@riseup.net>
This commit adds an option to turn off all encryption. This is a mode
used for tests, debugging, achieving protocol implementation interop,
learning about how the protocol works (nc ftw), and worst case
networks which _demand_ to be able to snoop on all the traffic.
(sadly, there are some private intranets like this...). (We should
consider at least _signing_ all this traffic.)
Because of the severity of this sort of thing, this is an
all-or-nothing deal. Either encryption is ON or OFF _fully_.
This way, partially unencrypted nodes cannot be accidentally left
running without the user's understanding. Nodes without encrypted
connections will simply not be able to speak to any of the global
bootstrap nodes, or anybody in the public network.
License: MIT
Signed-off-by: Juan Batiz-Benet <juan@benet.ai>
daemon output now includes initial swarm addresses. this is not a
full solution, as a change in network will not trigger re-printing.
We need a good way to do that.
This made me re-think how we're outputting these messages, perhaps
we should be throwing them as log.Events, and capturing some with
a special keyword to output to the user on stdout. Things like
network addresses being rebound, NATs being holepunched, external
network addresses being figured out, connections established, etc
may be valuable events to show the user. Of course, these should be
very few, as a noisy daemon is an annoying daemon.
License: MIT
Signed-off-by: Juan Batiz-Benet <juan@benet.ai>
Except when there is an explicit os.Exit(1) after the Critical line,
then replace with Fatal{,f}.
golang's log and logrus already call os.Exit(1) by default with Fatal.
License: MIT
Signed-off-by: rht <rhtbot@gmail.com>
This configuration is supported since GoBuilder 1.16.0 so now we should be able to build windows binaries in GoBuilder
License: MIT
Signed-off-by: Knut Ahlers <knut@ahlers.me>
I think we should lower the default rsa key size to 2048 for now -- until we have a proper focus on securing everything. It's always a pain for new users to get hung on 4096 rsa key gen, when we have not even made sure we're using the keys perfectly correctly yet. (And 2048 is still considered secure)
If the daemon is running we do not want to proceed with an
an initialization.
Return a client error telling the user to kill the daemon
before proceeding with the command.
Instead of just terminating right there and then, we cancel the
context, and let the daemon exit cleanly. This make take a few
seconds, as the node builder and its child processes do not
care too much about the context state while building nodes,
but this can be improved by injecting checks for ctx.Done()
before time-consuming steps.
Instead of assuming the command is the daemon command and closing
the node, which resulted in bugs like #1053, we cancel the context
and let the context children detect the cancellation and gracefully
clean up after themselves.
The shutdown logging has been moved into the daemon command, where
it makes more sense, so that commands like ping will not print out
the same output on cancellation.
If a command invocation such as 'daemon' is interrupted, the interrupt
handler asks the node to close. The closing of the node will result in
the command invocation finishing, and possibly returning from main()
before the interrupt handler is done. In particular, the info logging
that a graceful shutdown was completed may never reach reach stdout.
As the whole point of logging "Gracefully shut down." is to give
confidence when debugging that the shutdown was clean, this is
slightly unfortunate.
The interrupt handler needs to be set up in main() instead of Run(),
so that we can defer the closing of the interrupt handler until just
before returning from main, not when Run() returns with a streaming
result reader.
This reverts commit f74e71f965.
The 'Online' flag of the command context does not seem to be set in
any code paths, at least not when running commands such as 'ipfs daemon'
or 'ipfs ping'. The result after f74e71f9 is that we never shutdown
cleanly, as we'll always os.Exit(0) from the interrupt handler.
The os.Exit(0) itself is also dubious, as conceptually the interrupt
handler should ask whatever is stalling to stop stalling, so that
main() can return like normal. Exiting with -1 in error cases where
the interrupt handler is unable to stop the stall is fine, but the
normal case of interrupting cleanly should exit through main().
We now consider debugerrors harmful: we've run into cases where
debugerror.Wrap() hid valuable error information (err == io.EOF?).
I've removed them from the main code, but left them in some tests.
Go errors are lacking, but unfortunately, this isn't the solution.
It is possible that debugerros.New or debugerrors.Errorf should
remain still (i.e. only remove debugerrors.Wrap) but we don't use
these errors often enough to keep.
This commit adds a new set of sharness tests for pinning, and addresses
bugs that were pointed out by said tests.
test/sharness: added more pinning tests
Pinning is currently broken. See issue #1051. This commit introduces
a few more pinning tests. These are by no means exhaustive, but
definitely surface the present problems going on. I believe these
tests are correct, but not sure. Pushing them as failing so that
pinning is fixed in this PR.
make pinning and merkledag.Get take contexts
improve 'add' commands usage of pinning
FIXUP: fix 'pin lists look good'
ipfs-pin-stat simple script to help check pinning
This is a simple shell script to help check pinning.
We ought to strive towards making adding commands this easy.
The http api is great and powerful, but our setup right now
gets in the way. Perhaps we can clean up that area.
updated t0081-repo-pinning
- fixed a couple bugs with the tests
- made it a bit clearer (still a lot going on)
- the remaining tests are correct and highlight a problem with
pinning. Namely, that recursive pinning is buggy. At least:
towards the end of the test, $HASH_DIR4 and $HASH_FILE4 should
be pinned indirectly, but they're not. And thus get gc-ed out.
There may be other problems too.
cc @whyrusleeping
fix grep params for context deadline check
fix bugs in pin and pin tests
check for block local before checking recursive pin
this more clearly explains that the daemon is listening on the network. it also hints as to how to change the listening ports. this is also related to #874
- updated go-ctxgroup and goprocess
ctxgroup: AddChildGroup was changed to AddChild. Used in two files:
- p2p/net/mock/mock_net.go
- routing/dht/dht.go
- updated context from hg repo to git
prev. commit in hg was ad01a6fcc8a19d3a4478c836895ffe883bd2ceab. (context: make parentCancelCtx iterative)
represents commit 84f8955a887232b6308d79c68b8db44f64df455c in git repo
- updated context to master (b6fdb7d8a4ccefede406f8fe0f017fb58265054c)
Aaron Jacobs (2):
net/context: Don't accept a context in the DoSomethingSlow example.
context: Be clear that users must cancel the result of WithCancel.
Andrew Gerrand (1):
go.net: use golang.org/x/... import paths
Bryan C. Mills (1):
net/context: Don't leak goroutines in Done example.
Damien Neil (1):
context: fix removal of cancelled timer contexts from parent
David Symonds (2):
context: Fix WithValue example code.
net: add import comments.
Sameer Ajmani (1):
context: fix TestAllocs to account for ints in interfaces
