fix the nc wait. the issue was that stdin needs to remain _open_
but not receive any input for some time. If stdin receives (invalid)
input or closes, the other side terminates the connection before
writing out the muxer frames + identify handshake.
This commit also changes the use of `!` for `test_must_fail`
License: MIT
Signed-off-by: Juan Batiz-Benet <juan@benet.ai>
daemon output now includes initial swarm addresses. this is not a
full solution, as a change in network will not trigger re-printing.
We need a good way to do that.
This made me re-think how we're outputting these messages, perhaps
we should be throwing them as log.Events, and capturing some with
a special keyword to output to the user on stdout. Things like
network addresses being rebound, NATs being holepunched, external
network addresses being figured out, connections established, etc
may be valuable events to show the user. Of course, these should be
very few, as a noisy daemon is an annoying daemon.
License: MIT
Signed-off-by: Juan Batiz-Benet <juan@benet.ai>
IPFS_PATH should really be exported to make sure it is
available to the ipfs binary.
It looks like sharness tests fail otherwise on CircleCi.
License: MIT
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
We don't want to prefix these results with the argument. If there was
only one argument, the unprefixed results are still explicit.
License: MIT
Signed-off-by: W. Trevor King <wking@tremily.us>
Discussion with Juan on IRC ([1] through [2]) lead to this adjusted
JSON output. Benefits over the old output include:
* deduplication (we only check the children of a given Merkle node
once, even if multiple arguments resolve to that hash)
* alphabetized output (like POSIX's ls). As a side-effect of this
change, I'm also matching GNU Coreutils' ls output (maybe in POSIX?)
by printing an alphabetized list of non-directories (one per line)
first, with alphabetized directory lists afterwards.
[1]: https://botbot.me/freenode/ipfs/2015-06-12/?msg=41725570&page=5
[2]: https://botbot.me/freenode/ipfs/2015-06-12/?msg=41726547&page=5
License: MIT
Signed-off-by: W. Trevor King <wking@tremily.us>
This doesn't affect the text output, which was already using a
stringified name. The earlier stringification does change the JSON
output from an enumeration integer (e.g. 2) to the string form
(e.g. "File"). If/when we transition to Merkle-object types named by
their hash, we will probably want to revisit this and pass both the
type hash and human-readable-but-collision-prone name on to clients.
License: MIT
Signed-off-by: W. Trevor King <wking@tremily.us>
Change the approach to the directory-header control so we can set the
Argument value in the JSON response.
Stripping the trailing newline from the JSON output is annoying, but
looking over [1] I saw no easy way to add a newline to the JSON
output. And with the general framework that commands/ attempts to be,
it feels a bit funny to customize the JSON output for a command-line
program. Perhaps a workable solution is to have the command-line
client append newlines to any output that otherwise lacks them? But
that seems like a change best left to a separate series.
[1]: http://golang.org/pkg/encoding/json/
License: MIT
Signed-off-by: W. Trevor King <wking@tremily.us>
Instead of raising "keychains not yet implemented" whenever we have an
explicit node ID, only raise the error when the given node ID isn't
the local node. This allows folks to use the more-general
explicit-node-ID form in scripts and such now, as long as they use the
local node name when calling those scripts.
Also add a test for this case, and update the comment for the
one-argument case to match the current syntax for extracting a
multihash name string.
License: MIT
Signed-off-by: W. Trevor King <wking@tremily.us>
Folks operating at the Unix-filesystem level shouldn't care about that
level of Merkle-DAG detail. Before this commit we had:
$ ipfs unixfs ls /ipfs/QmSRCHG21Sbqm3EJG9aEBo4vS7Fqu86pAjqf99MyCdNxZ4/busybox
/ipfs/QmSRCHG21Sbqm3EJG9aEBo4vS7Fqu86pAjqf99MyCdNxZ4/busybox:
... several lines of empty-string names ...
And with this commit we have:
$ ipfs unixfs ls /ipfs/QmSRCHG21Sbqm3EJG9aEBo4vS7Fqu86pAjqf99MyCdNxZ4/busybox
/ipfs/QmSRCHG21Sbqm3EJG9aEBo4vS7Fqu86pAjqf99MyCdNxZ4/busybox
I also reworked the argument-prefixing (object.Argument) in the output
marshaller to avoid redundancies like:
$ ipfs unixfs ls /ipfs/QmSRCHG21Sbqm3EJG9aEBo4vS7Fqu86pAjqf99MyCdNxZ4/busybox
/ipfs/QmSRCHG21Sbqm3EJG9aEBo4vS7Fqu86pAjqf99MyCdNxZ4/busybox:
/ipfs/QmSRCHG21Sbqm3EJG9aEBo4vS7Fqu86pAjqf99MyCdNxZ4/busybox
As a side-effect of this rework, we no longer have the trailing blank
line that we used to have after the final directory listing.
The new ErrImplementation is like Python's NotImplementedError, and is
mostly a way to guard against external changes that would need
associated updates in this code. For example, once we see something
that's neither a file nor a directory, we'll have to update the switch
statement to handle those objects.
License: MIT
Signed-off-by: W. Trevor King <wking@tremily.us>
This is similar to 'ipfs ls ...', but it:
* Lists file sizes that match the content size:
$ ipfs --encoding=json unixfs ls /ipfs/QmSRCHG21Sbqm3EJG9aEBo4vS7Fqu86pAjqf99MyCdNxZ4
{
"Objects": [
{
"Argument": "/ipfs/QmSRCHG21Sbqm3EJG9aEBo4vS7Fqu86pAjqf99MyCdNxZ4",
"Links": [
{
"Name": "busybox",
"Hash": "QmPbjmmci73roXf9VijpyQGgRJZthiQfnEetaMRGoGYV5a",
"Size": 1947624,
"Type": 2
}
]
}
]
}
$ ipfs cat /ipfs/QmSRCHG21Sbqm3EJG9aEBo4vS7Fqu86pAjqf99MyCdNxZ4/busybox | wc -c
1947624
'ipfs ls ...', on the other hand, is using the Merkle-descendant
size, which also includes fanout links and the typing information
unixfs objects store in their Data:
$ ipfs --encoding=json ls /ipfs/QmSRCHG21Sbqm3EJG9aEBo4vS7Fqu86pAjqf99MyCdNxZ4
{
"Objects": [
{
"Hash": "/ipfs/QmSRCHG21Sbqm3EJG9aEBo4vS7Fqu86pAjqf99MyCdNxZ4",
"Links": [
{
"Name": "busybox",
"Hash": "QmPbjmmci73roXf9VijpyQGgRJZthiQfnEetaMRGoGYV5a",
"Size": 1948128,
"Type": 2
}
]
}
]
}
* Has a simpler text output corresponding to POSIX ls [1]:
$ ipfs unixfs ls /ipfs/QmV2FrBtvue5ve7vxbAzKz3mTdWq8wfMNPwYd8d9KHksCF/gentoo/stage3/amd64/2015-04-02
bin
dev
etc
proc
run
sys
$ ipfs ls /ipfs/QmV2FrBtvue5ve7vxbAzKz3mTdWq8wfMNPwYd8d9KHksCF/gentoo/stage3/amd64/2015-04-02
QmSRCHG21Sbqm3EJG9aEBo4vS7Fqu86pAjqf99MyCdNxZ4 1948183 bin/
QmUNLLsPACCz1vLxQVkXqqLX5R1X345qqfHbsf67hvA3Nn 4 dev/
QmUz1Z5jnQEjwr78fiMk5babwjJBDmhN5sx5HvPiTGGGjM 1207 etc/
QmUNLLsPACCz1vLxQVkXqqLX5R1X345qqfHbsf67hvA3Nn 4 proc/
QmUNLLsPACCz1vLxQVkXqqLX5R1X345qqfHbsf67hvA3Nn 4 run/
QmUNLLsPACCz1vLxQVkXqqLX5R1X345qqfHbsf67hvA3Nn 4 sys/
The minimal output allows us to start off with POSIX compliance and
then add options (which may or may not be POSIX compatible) to
adjust the output format as we get a better feel for what we need
([2] through [3]).
[1]: http://pubs.opengroup.org/onlinepubs/9699919799/utilities/ls.html
[2]: https://botbot.me/freenode/ipfs/2015-06-12/?msg=41724727&page=5
[3]: https://botbot.me/freenode/ipfs/2015-06-12/?msg=41725146&page=5
License: MIT
Signed-off-by: W. Trevor King <wking@tremily.us>
the iptb tests are failing all the time on travis. It's possible
this is still a port problem, and it may be something else.
Regardless, right now they're just adding noise.
This PR moves the addition of new blocks to our wantlist (and their
subsequent broadcast to the network) outside of the clientWorker loop.
This allows blocks to more quickly propogate to peers we are already
connected to, where before we had to wait for the previous findProviders
call in clientworker to complete before we could notify our partners of
the next blocks that we want. I then changed the naming of the
clientWorker and related variables to be a bit more appropriate to the
model. Although the clientWorker (now named providerConnector) feels a
bit awkward and should probably be changed.
fix test assumption
Previously we had a confusing situation, with:
* single-arg doc: published name <name> to <value>
* double-arg doc: published name <value> to <name>
* implementation: Published name <name> to <value>
Now we have the uniform:
Published to <name>: <value>
With the following goals:
1. It's clear that we're writing <value> to <name>'s IPNS slot in the
DHT.
2. We preserve the order of arguments from the command-line
invocation:
$ ipfs name publish <name> <value>
Published to <name>: <value>
commands/object: remove objectData() and objectLinks() helpers
resolver: added context parameters
sharness: $HASH carried the \r from the http protocol with
sharness: write curl output to individual files
http gw: break PUT handler until PR#1191
As test directories contain a space, we need to
properly quote paths, otherwise we get errors like:
```
umount: /home/christian/gocode/src/github.com/ipfs/go-ipfs/test/sharness/trash is not mounted (according to mtab)
umount: /home/christian/gocode/src/github.com/ipfs/go-ipfs/test/sharness/trash is not mounted (according to mtab)
```
instead of:
```
umount: /home/christian/gocode/src/github.com/ipfs/go-ipfs/test/sharness/trash directory.t0030-mount.sh/ipfs is not mounted (according to mtab)
umount: /home/christian/gocode/src/github.com/ipfs/go-ipfs/test/sharness/trash directory.t0030-mount.sh/ipns is not mounted (according to mtab)
```
License: MIT
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
As the ipfs mount call is now encapsulated in a
temporary function (see previous commit) its
output should not be tempered with by
test_must_fail.
License: MIT
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
As described in issue #1109 on OSX some output from
test_must_fail unfortunately goes into the "output"
file that we use to test the output from "ipfs mount".
This patch avoids the above by encapsulating the call
to "ipfs mount" into a temporary function.
License: MIT
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
sharness should only send the kill signal once, as that is
what a graceful shutdown should do. in the event that doesn't
happen, we should send it again, and then kill -9 to prevent it
lingering and messing with other tests.
This changes .go-ipfs to .ipfs everywhere.
And by the way this defines a DefaultPathName const
for this name.
License: MIT
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
This commit adds a new set of sharness tests for pinning, and addresses
bugs that were pointed out by said tests.
test/sharness: added more pinning tests
Pinning is currently broken. See issue #1051. This commit introduces
a few more pinning tests. These are by no means exhaustive, but
definitely surface the present problems going on. I believe these
tests are correct, but not sure. Pushing them as failing so that
pinning is fixed in this PR.
make pinning and merkledag.Get take contexts
improve 'add' commands usage of pinning
FIXUP: fix 'pin lists look good'
ipfs-pin-stat simple script to help check pinning
This is a simple shell script to help check pinning.
We ought to strive towards making adding commands this easy.
The http api is great and powerful, but our setup right now
gets in the way. Perhaps we can clean up that area.
updated t0081-repo-pinning
- fixed a couple bugs with the tests
- made it a bit clearer (still a lot going on)
- the remaining tests are correct and highlight a problem with
pinning. Namely, that recursive pinning is buggy. At least:
towards the end of the test, $HASH_DIR4 and $HASH_FILE4 should
be pinned indirectly, but they're not. And thus get gc-ed out.
There may be other problems too.
cc @whyrusleeping
fix grep params for context deadline check
fix bugs in pin and pin tests
check for block local before checking recursive pin
GNU Make's wildcard function does not recurse into subdirectories when
passed the '**' glob, which results in adding a dependency only to .go
files in the first level of subdirectories under the source root.
We shell out to 'find' instead, which catches all .go files in the
given directory.
Running make -jN would result in the tests starting to execute
before the tests binaries were built, resulting in the error:
"Cannot find the tests' local ipfs tool"
Each test now depends on the deps. They also depend on a new
target for cleaning the test results, so that the tests can
write new clean results.
The aggregate target also needs to depend on the same test
results clean target, as well as the tests themselves, so
that the aggregation happens when all tests have finished
running.
By introducing a separate target for cleaning test results we
also ensure that we don't end up removing and rebuilding
the binary on each test run.
The result is that the tests *can* be run with with -jN > 1,
but individual tests may still not supports this, so to get
stable test results it's still recommended to run them in
sequence.
The GOFLAGS variable makes it possible to run all sharness
tests with go binaries built with some special flags.
The "race" target makes it easy run the sharness tests
with go binaries built with the -race flag.
License: MIT
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
...and remove bin/* stuff from the .gitignore
as /test/bin is already in the root .gitignore.
License: MIT
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
This makes it possible to build binaries with
different flags.
The content of the GOFLAGS variable is stored
in a IPFS-BUILD-OPTIONS file, so that if GOFLAGS
changes a rebuild of the binaries with the new
flags is forced.
License: MIT
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
This script can be used in a Makefile to detect flag changes
and to save the new flags in a file.
License: MIT
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
As `ipfs version` and `ipfs help` are basic commands
it is ok to test them online in t0060.
License: MIT
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
It looks like some tests that were expected failures
are now always succeeding, so let's mark them as such.
License: MIT
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
To do that the current tests are move inside a
test_object_cmd() function. This function is then first
called before running the daemon for offline tests
and then after starting the daemon for online tests.
License: MIT
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
To do that the current tests are move inside a
test_ls_cmd() function. This function is then first
called before running the daemon for offline tests
and then after starting the daemon for online tests.
License: MIT
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
To do that the current tests are move inside a
test_get_cmd() function. This function is then first
called before running the daemon for offline tests
and then after starting the daemon for online tests.
License: MIT
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
It looks like there were two problems:
- "read" command probably needs the name of the variable
it reads into
- [[ didn't work
This might be because on Linux /bin/sh is sometimes not
bash. For example on Ubuntu it is a symlink to dash.
License: MIT
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
Tests in this script had been deactivated because
there was no good way to wait until the daemon was
ready. Now that we can use pollEndpoint for that
it's time to reactivate the tests.
License: MIT
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
Add information from the discussion of PR #891.
While at it, suggest using -v and -i options to debug one test.
License: MIT
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
This makes the following changes:
- remove some superfluous \ at the end of some lines
- use `cat <<-\EOF >expected` to ignore the starting tabs
in the here document
- add missing && after some `cat`
- use test_cmp to compare files
License: MIT
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
we have a problem where initializing daemons with the same api port
often fails-- it hangs indefinitely. The proper solution is to make
ipfs pick an unused port for the api on startup, and then use that.
Unfortunately, ipfs doesnt yet know how to do this-- the api port
must be specified. Until ipfs learns how to do this, we must use
specific port numbers, which may still fail but less frequently
if we at least use different ones.
There was basically:
curl ... "$URL 2>curl.out"
instead of:
curl ... "$URL" 2>curl.out
So "curl.out" was not properly overwritten by the curl command.
License: MIT
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
- updated go-ctxgroup and goprocess
ctxgroup: AddChildGroup was changed to AddChild. Used in two files:
- p2p/net/mock/mock_net.go
- routing/dht/dht.go
- updated context from hg repo to git
prev. commit in hg was ad01a6fcc8a19d3a4478c836895ffe883bd2ceab. (context: make parentCancelCtx iterative)
represents commit 84f8955a887232b6308d79c68b8db44f64df455c in git repo
- updated context to master (b6fdb7d8a4ccefede406f8fe0f017fb58265054c)
Aaron Jacobs (2):
net/context: Don't accept a context in the DoSomethingSlow example.
context: Be clear that users must cancel the result of WithCancel.
Andrew Gerrand (1):
go.net: use golang.org/x/... import paths
Bryan C. Mills (1):
net/context: Don't leak goroutines in Done example.
Damien Neil (1):
context: fix removal of cancelled timer contexts from parent
David Symonds (2):
context: Fix WithValue example code.
net: add import comments.
Sameer Ajmani (1):
context: fix TestAllocs to account for ints in interfaces
add automatic add and cat for debugging purposes
retry cats
cast string
argh! @whyrusleepgin
use mock repo to avoid clashing
larger files for more variety FIXUP into gc-client
fix: change in both places FIXUP gc-client
fix(gc-client) start with size 1 (unit)
fix(gc-client) start at 1 FIXUP prev
gc-client rm comment
gc-client larger files
fix(gc-client) cat the data out to dev null to ensure entire file is fetched
perf(gc-client) use io.Pipe to handle io transfer
fixup
As shasum is not installed on all machines and
we use multihash anyway in the code base, it
removes one dependency to use shasum instead of
shasum in the tests.
Now that there are sharness tests in multihash
it is also safe to use it.
License: MIT
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
The new ipfs-test-lib.sh file contains generic test
functions.
We also start replacing fsh with a shell function named
test_fsh() in ipfs-test-lib.sh.
And we move our custom test_cmp in ipfs-test-lib.sh.
License: MIT
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
The pattern "cat FILE | COMMAND" can be simplified
to just "COMMAND FILE" when COMMAND accepts files as
arguments.
License: MIT
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
The following changes are made to make t0080
cleaner, safer and more canonical:
- remove useless stuff,
- don't use a pipe after ipfs commands,
- use test_must_fail before ipfs commands instead of !
License: MIT
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
⚠️ this commit makes your current configs unusable, as the
default bootstrap peers. You may need to edit your config.
Go from:
```js
Bootstrap: [
{
"Address": "/ip4/104.131.131.82/tcp/4001",
"PeerID": "QmaCpDMGvV2BGHeYERUEnRQAwe3N8SzbUtfsmvsqQLuvuJ"
}
]
```
To:
```js
Bootstrap: [
"/ip4/104.131.131.82/tcp/4001/ipfs/QmaCpDMGvV2BGHeYERUEnRQAwe3N8SzbUtfsmvsqQLuvuJ"
]
```
- core: daemon stdout print to cmd + daemon init checks
- core: fixed bug where the gateway was printed as "API"
- sharness/test-lib: daemon init checks
- sharness/test-lib: portable TCP port check
- sharness/init: fix test bits output
- sharness: use common hashes in one place.
- move t0100-http-gateway -> t0111-gateway-writable
- sharness: test-lib funcs for gateway config
- sharness/t0111-gateway-writable: use sh funcs
- sharness/t0111-gateway-writable: fixes
- escape all vars (always `cmd "$VAR"` never `cmd $VAR`)
- use $FILEPATH, not $path
- last test seems to fail
In OSX, the `echo -n` was _somehow_ printing "-n" out... not
really sure why, as the bsd `echo` program supports -n. I changed
this to printf -- which is safer.
improve efficiency of multilayered indirect blocks
clean up tests
panic cleanup
clean up logic, improve readability
add final root node to the dagservice upon creation
importer: simplified dag generation
test: updated hashes using latest code
@whyrusleeping this is why the sharness tests
were failing: the hashes are added manually to
make sure our generation doesn't change.
cleanup after CR
fix merkledag tests
fix small block generation (no subblocks!)
Make sharness tests' output helpful when verbose.
This means cating certain files, or running diagnostic
commands. I used a construction like:
test_expect_success ".go-ipfs/ has been created" '
test -d ".go-ipfs" &&
test -f ".go-ipfs/config" &&
test -d ".go-ipfs/datastore" ||
fsh ls -al .go-ipfs
'
The `|| ...` is a diagnostic run when the preceding command
fails. `fsh` is a trivial script that echoes the args, runs
the cmd, and then also fails, making sure the test case fails.
(wouldnt want the diagnostic accidentally returning true and
making it _seem_ like the test case succeeded).
These new tests are not marked EXPENSIVE as they
should be fast enough even on Travis CI.
License: MIT
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
this is a hack around travis key-gen being really slow.
One option would be to add the --key-bits option to daemon
as well. Another option would be to find a better way to
wait for the output, rather than waiting n seconds.
cc @chriscool thoughts?
The ouput from "ipfs help" changed in ipfs2.
With the change in this commit, this output change doesn't
prevent the test to pass.
Test t0010 now passes on my Linux machine.
License: MIT
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
The ouput from "ipfs block" changed in ipfs2.
With the change in this commit, this output change doesn't
prevent the test to pass.
Test t0050 now passes on my Linux machine.
License: MIT
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
The ouput from "ipfs add" changed in ipfs2.
With the changes in this commit, this output change doesn't
prevent the test to pass.
With TEST_NO_FUSE=1 test t0040 now passes on my Linux machine.
License: MIT
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
@jbenet @chriscool
(not to be merged into master) This is a hack to run sharness tests on
th ipfs2 binary. Instead of compiling cmd/ipfs, it compiles cmd/ipfs2
and copies this into test/bin/ipfs.
I thought this would be enough to pass the `basic-commands` test, but
it's not.
Although the output is fairly similar, the `ipfs version` test fails.
```
test (feat/test2) λ. diff version1 version2
1c1
< ipfs version 0.1.7
---
> ipfs version 0.1.5
```
I'm not very experienced with `sh` scripting, so perhaps I'm missing a
key ingredient or maybe misunderstanding the the tests are meant to
work.
Would like to get input on this.
Thanks, @maybebtc
This adds test_launch_ipfs_mount() and test_kill_ipfs_mount()
to avoid duplicating tests to launch "ipfs mount" and to
kill it.
License: MIT
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
You can use it like this to launch all the
test scripts in order:
$ cd test
$ make
rm -r test-results
*** t0010-basic-commands.sh ***
ok 1 - current dir is writable
ok 2 - ipfs version succeeds
ok 3 - ipfs version output looks good
ok 4 - ipfs help succeeds
ok 5 - ipfs help output looks good
# passed all 5 test(s)
1..5
./test-aggregate-results.sh
fixed 0
success 5
failed 0
broken 0
total 5
Or you can just run one test like this:
$ make t0010-basic-commands.sh
*** t0010-basic-commands.sh ***
ok 1 - current dir is writable
ok 2 - ipfs version succeeds
ok 3 - ipfs version output looks good
ok 4 - ipfs help succeeds
ok 5 - ipfs help output looks good
# passed all 5 test(s)
1..5
This checks a little bit the installation and some
basic commands.
You can run it like that:
$ cd test
$ ./t0010-basic-commands.sh
ok 1 - current dir is writable
ok 2 - ipfs version succeeds
ok 3 - ipfs version output looks good
ok 4 - ipfs help succeeds
ok 5 - ipfs help output looks good
# passed all 5 test(s)
1..5
