The ldd command used for detection doesn't seem to have a
--version flag on Alpine Linux. It would print the expected
output, but instead of stdout, it would print it on stderr.
The musl detection code would only scan stdout for mentions
of "musl", and would thus *not* download the musl version
of the fs-repo-migrations executable.
This manifested in the well-known "fs-repo-migrations: not found"
error, which you get when executing something that was linked
against a different libc than the one present on the system.
License: MIT
Signed-off-by: Lars Gierth <larsg@systemli.org>
This commit introduces non-recursive Makefile infrastructure that replaces current Makefile infrastructure.
It also generally cleanups the Makefiles, separates them into nicer sub-modules and centralizes common operations into single definitions.
It allows to depend on any target that is defined in the makefile, this means that for example `gx install` is called once when `make build test_expensive_sharness` is called instead of 4 or 5 times.
It also makes the dependencies much cleaner and allows for reuse of modules. For example sharness coverage collection (WIP) uses sharness target with amended PATH, previously it might have been possible but not without wiring in the coverage collection into sharness make runner code.
Yes, it is more complex but not much more. There are few rules that have to be followed and few complexities added but IMHO it is worth it.
How to NR-make:
1. If make is to generate some file via a target, it MUST be defined in Rules.mk file in the directory of the target.
2. `Rules.mk` file MUST have `include mk/header.mk` statement as the first line and `include mk/footer.mk` statement as the last line (apart from project root `Rules.mk`).
3. It then MUST be included by the closest `Rules.mk` file up the directory tree.
4. Inside a `Rules.mk` special variable accessed as `$(d)` is defined. Its value is current directory, use it so if the `Rules.mk` file is moved in the tree it still works without a problem. Caution: this variable is not available in the recipe part and MUST NOT be used. Use name of the target or prerequisite to extract it if you need it.
5. Make has only one global scope, this means that name conflicts are a thing. Names SHOULD follow `VAR_NAME_$(d)` convention. There are exceptions from this rule in form of well defined global variables. Examples: General lists `TGT_BIN`, `CLEAN`; General targets: `TEST`, `COVERAGE`; General variables: `GOFLAGS`, `DEPS_GO`.
3. Any rules, definitions or variables that fit some family SHOULD be defined in `mk/$family.mk` file and included from project root `Rules.mk`
License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protonmail.ch>
It conflicts with already running instance on the Dev's machine so
running this test requires disabling the normal ipfs.
The launch of ipfs here doesn't test much as we compare the config,
after initing from it and we also launch from default config in many
other cases (with few variables changed to not conflict with already
running standalone ipfs instance).
License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protonmail.ch>
Tried to check all instances of IPFS and make sure they werent referring to the CLI tool. See #2910.
License: MIT
Signed-off-by: Richard Littauer <richard.littauer@gmail.com>
This change adds the /ipfs/bitswap/1.1.0 protocol. The new protocol
adds a 'payload' field to the protobuf message and deprecates the
existing 'blocks' field. The 'payload' field is an array of pairs of cid
prefixes and block data. The cid prefixes are used to ensure the correct
codecs and hash functions are used to handle the block on the receiving
end.
License: MIT
Signed-off-by: Jeromy <why@ipfs.io>
Add back still useful test from reverted commit:
fe7b01f14e
Resolve symlink if it is directly referenced in cli (#2897)
License: MIT
Signed-off-by: Kevin Atkinson <k@kevina.org>
This reverts commit fe7b01f14e.
Conflicts:
commands/cli/parse.go
Revert "Merge pull request #3023 from ipfs/feature/eval-symlink-windows"
This reverts commit 16c5a89dd4, reversing
changes made to 8c77ff8188.
Conflicts:
commands/cli/parse.go
License: MIT
Signed-off-by: Kevin Atkinson <k@kevina.org>
Add test that removes a combination of pinned, valid, and non-existent
blocks in one command.
License: MIT
Signed-off-by: Kevin Atkinson <k@kevina.org>
Provide a new method, Pinner.CheckIfPinned(), which will check if
any of the arguments are pinned. Previously IsPinned would need to be
called once for each block. The new method will speed up the checking
of multiple pinned blocks from O(p*n) to O(p) (where p is the number
of pinned blocks and n is the number of blocks to be check)
Use the new method in "block rm".
License: MIT
Signed-off-by: Kevin Atkinson <k@kevina.org>
test_fsh() should quote its arguments before passing them
to `eval` otherwise there are problems when the arguments
contain spaces.
For example when running the following program:
```
#!/bin/sh
. ./ipfs-test-lib.sh
die () {
printf >&2 "%s\n" "$@"
exit 1
}
DIR1="test dir 1"
DIR2="test dir 2"
mkdir "$DIR1" "$DIR2" || die "Could not mkdir '$DIR1' '$DIR2'"
echo "in dir 1" >"$DIR1/file1" || die "Could not write into '$DIR1/file1'"
echo "in dir 2" >"$DIR2/file2" || die "Could not write into '$DIR2/file2'"
if test_cmp "$DIR1/file1" "$DIR2/file2"
then
echo "test_cmp succeeded!"
else
echo "test_cmp failed!"
fi
rm -rf "$DIR1" "$DIR2" || die "Could not rm -rf '$DIR1' '$DIR2'"
```
we get:
```
> diff -u test dir 1/file1 test dir 2/file2
diff: extra operand '1/file1'
diff: Try 'diff --help' for more information.
test_cmp failed!
```
License: MIT
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
Only print "Reading from /dev/stdin" message when we actually read from
stdin (and not in other cases such as ipfs add --help).
License: MIT
Signed-off-by: Jeromy <why@ipfs.io>
1. Test that ipfs --version has a 0 exit status
2. Check the ipfs --version output matches ipfs version
License: MIT
Signed-off-by: Chris Sasarak <chris.sasarak@gmail.com>
* Resolve symlink if it is directly referenced in cli
test: Directly referenced symlink should be resolved
License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protonmail.ch>
* sharness: add test for symlink in the middle
License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protonmail.ch>
With verbose flag:
* remove EnableStdin() flags on all StringArg,
* remove all unneeded parsing code for StringArg, and print an
* informative message if `ipfs` begins reading from a CharDevice,
* remove broken go tests for EnableStdin cli parsing, and add some
* trivial test cases for reading FileArg from stdin,
* add a panic to prevent EnableStdin from being set on
* StringArg in the future.
Resolves: #2877, #2870
License: MIT
Signed-off-by: Thomas Gardner <tmg@fastmail.com>
* Update golog in go-ipfs
License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protonmail.ch>
* Update go-libp2p for go-log
License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protonmail.ch>
* Update go-libp2p-secio for go-log
License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protonmail.ch>
* Update go-libp2p-crypto for go-log
License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protonmail.ch>
* Update go-libp2p-peer for go-log
License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protonmail.ch>
* Import peersore, it wasn't imported
License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protonmail.ch>
* Update peerstore
License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protonmail.ch>
* Update peer
License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protonmail.ch>
* Update secio
License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protonmail.ch>
* Update go-libp2p
License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protonmail.ch>
This patch is in preparation for the gateway's extraction.
It's interesting to trace technical debt back to its
origin, understanding the circumstances in which it
was introduced and built up, and then cutting it back
at exactly the right places.
- Clean up the gateway's surface
The option builder GatewayOption() now takes only
arguments which are relevant for HTTP handler muxing,
i.e. the paths where the gateway should be mounted.
All other configuration happens through the
GatewayConfig object.
- Remove BlockList
I know why this was introduced in the first place,
but it never ended up fulfilling that purpose.
Somehow it was only ever used by the API server,
not the gateway, which really doesn't make sense.
It was also never wired up with CLI nor fs-repo.
Eventually @krl started punching holes into it
to make the Web UI accessible.
- Remove --unrestricted-api
This was holes being punched into BlockList too,
for accessing /ipfs and /ipn on the API server.
With BlockList removed and /ipfs and /ipns freely
accessible, putting this option out of action
is safe. With the next major release,
the option can be removed for good.
License: MIT
Signed-off-by: Lars Gierth <larsg@systemli.org>
previously, paths were not supported as link values.
this would not work, and now can:
ipfs object patch $root add-link foo /ipfs/$hash/foo/bar
License: MIT
Signed-off-by: Juan Benet <juan@benet.ai>
startup_cluster() already contains some test_expect_success, so
it should not be inside one.
License: MIT
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
Modified init command to receive default configuration from stdin. The changes enable us to use existing key-pair, datastore configuration while initializing new ipfs node.
License: MIT
Signed-off-by: Sivachandran <sivachandran.p@gmail.com>
Now that iptb has been added as a gx dependency, we can replace
existing dependencies with the one from gx.
License: MIT
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
* reduces help indent from 4 to 2 spaces
License: MIT
Signed-off-by: Stephen Whitmore <noffle@ipfs.io>
* reduces horz/vert space taken by "ipfs" cmd
License: MIT
Signed-off-by: Stephen Whitmore <noffle@ipfs.io>
* show subcommands on shorthelp
License: MIT
Signed-off-by: Stephen Whitmore <noffle@ipfs.io>
* Drops colons at the end of cmd headings.
This makes command headings consistent with the output of 'ipfs', which
does not include colons.
License: MIT
Signed-off-by: Stephen Whitmore <noffle@ipfs.io>
* more consistent output between short-/long-help
License: MIT
Signed-off-by: Stephen Whitmore <noffle@ipfs.io>
* Puts DESCRIPTION before SUBCOMMANDS.
Users likely want to understand what a command does before worrying
about its subcommands.
License: MIT
Signed-off-by: Stephen Whitmore <noffle@ipfs.io>
* Keeps ipfs cmd from outputting its subcmds twice.
License: MIT
Signed-off-by: Stephen Whitmore <noffle@ipfs.io>
* Removes redundant synopsis from "file"
License: MIT
Signed-off-by: Stephen Whitmore <noffle@ipfs.io>
* Removes extra whitespace from longhelp
License: MIT
Signed-off-by: Stephen Whitmore <noffle@ipfs.io>
* Consistent spacing whether SUBCMDS or not.
License: MIT
Signed-off-by: Stephen Whitmore <noffle@ipfs.io>
* Removes redundant SUBCMD output from ipfs object.
License: MIT
Signed-off-by: Stephen Whitmore <noffle@ipfs.io>
* Removes redundant synopsis from "name"
License: MIT
Signed-off-by: Stephen Whitmore <noffle@ipfs.io>
* Newline after Description only if it exists.
License: MIT
Signed-off-by: Stephen Whitmore <noffle@ipfs.io>
* Removes redundant synopsis from "bootstrap"
License: MIT
Signed-off-by: Stephen Whitmore <noffle@ipfs.io>
* Removes redundant synopsis from "swarm"
License: MIT
Signed-off-by: Stephen Whitmore <noffle@ipfs.io>
* Removes trailing newline in ping help.
License: MIT
Signed-off-by: Stephen Whitmore <noffle@ipfs.io>
* Prints shorthelp on parse error.
License: MIT
Signed-off-by: Stephen Whitmore <noffle@ipfs.io>
* tiny comment fixes
License: MIT
Signed-off-by: Stephen Whitmore <noffle@ipfs.io>
* updates README usage
License: MIT
Signed-off-by: Stephen Whitmore <noffle@ipfs.io>
* Don't include extra whitespace if no .MoreHelp
License: MIT
Signed-off-by: Stephen Whitmore <noffle@ipfs.io>
* description improvements
License: MIT
Signed-off-by: Stephen Whitmore <noffle@ipfs.io>
* Hides the obscure 'file' subcommand.
License: MIT
Signed-off-by: Stephen Whitmore <noffle@ipfs.io>
* Splits 'ipfs daemon' into Short and Long help.
License: MIT
Signed-off-by: Stephen Whitmore <noffle@ipfs.io>
* Removes redundant synopsis from "config"
License: MIT
Signed-off-by: Stephen Whitmore <noffle@ipfs.io>
* Wraps lines to keep from going over 80.
License: MIT
Signed-off-by: Stephen Whitmore <noffle@ipfs.io>
* specify repo separately
License: MIT
Signed-off-by: Stephen Whitmore <noffle@ipfs.io>
* s/structure/hierarchy
License: MIT
Signed-off-by: Stephen Whitmore <noffle@ipfs.io>
* missing .
License: MIT
Signed-off-by: Stephen Whitmore <noffle@ipfs.io>
* Removes trailing colon from 'usage' test.
License: MIT
Signed-off-by: Stephen Whitmore <noffle@ipfs.io>
* Updates sharness test error messages.
License: MIT
Signed-off-by: Stephen Whitmore <noffle@ipfs.io>
* Removes trailing colon from 'usage' test.
License: MIT
Signed-off-by: Stephen Whitmore <noffle@ipfs.io>
* Updates add-symlink to use /bin/sh.
License: MIT
Signed-off-by: Stephen Whitmore <noffle@ipfs.io>
* Removes "hierarchy".
License: MIT
Signed-off-by: Stephen Whitmore <noffle@ipfs.io>
* Updates "ipfs ping" synopsis.
* Updates t0040 with latest wording.
* Removes unnecessary daemon setup.
License: MIT
Signed-off-by: Stephen Whitmore <noffle@ipfs.io>
* Adds repo fsck subcommand
Fixes#2457
License: MIT
Signed-off-by: Mike Pfister <pfista@gmail.com>
* Checks for error on file deletion
License: MIT
Signed-off-by: Mike Pfister <pfista@gmail.com>
* Checks if node is online
License: MIT
Signed-off-by: Mike Pfister <pfista@gmail.com>
* Update error checking
License: MIT
Signed-off-by: Michael Pfister <pfista@gmail.com>
* Prevents command from running while daemon is running
License: MIT
Signed-off-by: Michael Pfister <pfista@gmail.com>
* Add newline to command output message
License: MIT
Signed-off-by: Michael Pfister <pfista@gmail.com>
* removing superfluous error
License: MIT
Signed-off-by: Michael Pfister <pfista@gmail.com>
* Adds sharness test for repo fsck command
License: MIT
Signed-off-by: Michael Pfister <pfista@gmail.com>
* Ignore warning if file doesn't exist
License: MIT
Signed-off-by: Michael Pfister <pfista@gmail.com>
* Updating message output
License: MIT
Signed-off-by: Michael Pfister <pfista@gmail.com>
* adding debug statements
License: MIT
Signed-off-by: Michael Pfister <pfista@gmail.com>
* update and add fsck sharness tests
License: MIT
Signed-off-by: Michael Pfister <pfista@gmail.com>
* updating comments
License: MIT
Signed-off-by: Michael Pfister <pfista@gmail.com>
* Use printf in test
Using printf prevents a newline from being printed to the api test file. When
the newline was present, multiaddr threw errors trying to parse the api address
to an integer since the newline character was present.
License: MIT
Signed-off-by: Michael Pfister <pfista@gmail.com>
* updating tests
License: MIT
Signed-off-by: Michael Pfister <pfista@gmail.com>
* removing commented code
License: MIT
Signed-off-by: Michael Pfister <pfista@gmail.com>
* Cleans up 'ipfs dht findpeer' output
License: MIT
Signed-off-by: Stephen Whitmore <noffle@ipfs.io>
* Adds more docs for 'ipfs dht put'.
License: MIT
Signed-off-by: Stephen Whitmore <noffle@ipfs.io>
* Write pretty peer ids for ipfs dht put.
License: MIT
Signed-off-by: Stephen Whitmore <noffle@ipfs.io>
* Writes pretty peer ids for ipfs dht query.
License: MIT
Signed-off-by: Stephen Whitmore <noffle@ipfs.io>
* Suppresses unrecognized event type for FinalPeer.
License: MIT
Signed-off-by: Stephen Whitmore <noffle@ipfs.io>
* Improves helptext on dht commands.
License: MIT
Signed-off-by: Stephen Whitmore <noffle@ipfs.io>
* Adds 'ipfs dht findpeer' sharness test.
License: MIT
Signed-off-by: Stephen Whitmore <noffle@ipfs.io>
* Adds sharness tests for remaining DHT commands.
License: MIT
Signed-off-by: Stephen Whitmore <noffle@ipfs.io>
* Uses bash tests rather than 'test' command.
License: MIT
Signed-off-by: Stephen Whitmore <noffle@ipfs.io>
* Removes commented code.
License: MIT
Signed-off-by: Stephen Whitmore <noffle@ipfs.io>
* Removes unneeded init_ipfs.
License: MIT
Signed-off-by: Stephen Whitmore <noffle@ipfs.io>
* Tweaks iptb setup.
License: MIT
Signed-off-by: Stephen Whitmore <noffle@ipfs.io>
* Tweaks wording on dht 'put' and 'get'.
License: MIT
Signed-off-by: Stephen Whitmore <noffle@ipfs.io>
* Removes extraneous ).
License: MIT
Signed-off-by: Stephen Whitmore <noffle@ipfs.io>
* Removes apostrophe.
License: MIT
Signed-off-by: Stephen Whitmore <noffle@ipfs.io>
* Tests the expected peer addresses.
License: MIT
Signed-off-by: Stephen Whitmore <noffle@ipfs.io>
* Gets peer id using iptb.
License: MIT
Signed-off-by: Stephen Whitmore <noffle@ipfs.io>
* Checks explicitly for common put/findprovs peers.
License: MIT
Signed-off-by: Stephen Whitmore <noffle@ipfs.io>
* Sorts expected/actual findpeer results.
License: MIT
Signed-off-by: Stephen Whitmore <noffle@ipfs.io>
* Fix disconnect argument description
License: MIT
Signed-off-by: Richard Littauer <richard.littauer@gmail.com>
* Fixes sort order in t0170-dht.sh.
License: MIT
Signed-off-by: Stephen Whitmore <noffle@ipfs.io>
The gateway accepts an X-Ipfs-Path-Prefix header,
and assumes that it is mounted in a reverse proxy
like nginx, at this path. Links in directory listings,
as well as trailing-slash redirects need to be rewritten
with that prefix in mind.
We don't want a potential attacker to be able to
pass in arbitrary path prefixes, which would end up
in redirects and directory listings, which is why
every prefix has to be explicitly allowed in the config.
Previously, we'd accept *any* X-Ipfs-Path-Prefix header.
Example:
We mount blog.ipfs.io (a dnslink page) at ipfs.io/blog.
nginx_ipfs.conf:
location /blog/ {
rewrite "^/blog(/.*)$" $1 break;
proxy_set_header Host blog.ipfs.io;
proxy_set_header X-Ipfs-Gateway-Prefix /blog;
proxy_pass http://127.0.0.1:8080;
}
.ipfs/config:
"Gateway": {
"PathPrefixes": ["/blog"],
// ...
},
dnslink:
> dig TXT _dnslink.blog.ipfs.io
dnslink=/ipfs/QmWcBjXPAEdhXDATV4ghUpkAonNBbiyFx1VmmHcQe9HEGd
License: MIT
Signed-off-by: Lars Gierth <larsg@systemli.org>
It used to default to `application/json` but is now correctly set to
`application/protobuf`.
fixes#2469
License: MIT
Signed-off-by: Steven Allen <steven@stebalien.com>
Removes all of the hardcoded ports and defaults to non-standard API,
Gateway, and Swarm ports.
License: MIT
Signed-off-by: Stephen Whitmore <noffle@ipfs.io>
For the rest of the packages in util, move them to thirdparty
and update the references. util is gone!
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
The actual tests for the IPFS daemon within the image,
which were added in the previous commits, made Circle CI unhappy.
Circle CI runs an old version of Docker
which still uses LXC instead of libcontainer.
The existing lxc-attach hack is testament to that.
We might be able to get it work [1][2]
but it's really not worth it at the moment,
and Circle CI isn't a good option for Docker things, right now.
[1] https://jpetazzo.github.io/2014/03/23/lxc-attach-nsinit-nsenter-docker-0-9/
[2] https://github.com/jpetazzo/nsenter
License: MIT
Signed-off-by: Lars Gierth <larsg@systemli.org>
This:
- uses startup_cluster() from iptb-lib.sh,
- avoids running test_expect_success() inside test_expect_success()
as it makes the output confusing,
- makes the number of test nodes easily configurable.
License: MIT
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
- Have two Dockerfiles doing essentially the same,
but optimized for build time (for tests)
and image size (for Docker Hub)
- Fetch gx dependencies
- Expose port 4002 for utp
- Specify go version, currently 1.5.3-r0
- Create ephemeral fs-repo if none is mounted
- Have t0300-docker-image actually test IPFS, not just an echo
- Make everything a bit less hardcoded
- Remove dead shacheck
License: MIT
Signed-off-by: Lars Gierth <larsg@systemli.org>
This is done by skipping the step of resolving the final segment in the
path to a DAG node; instead preferring to look at the second-to-last
segmenet's links.
License: MIT
Signed-off-by: Stephen Whitmore <noffle@ipfs.io>
- Implements
https://github.com/ipfs/go-ipfs/pull/2232#issuecomment-173742385
- Separate test suite:
- we don't want to pollute other gateway tests with CORS headers
- (as of now) changing headers requires daemon restart anyway
License: MIT
Signed-off-by: Marcin Rataj <lidel@lidel.org>
Fixesipfs/go-ipfs#2155 by turning the hash path arguments into keys
and unpinning directly, rather than running a full core.Resolve on
them. This lets users fail fast when they try to remove pins that
they don't have locally.
Note that this will only work when the path is of the form <hash> or
/ipfs/<hash>. Given e.g. /ipfs/<hash>/foo, foo's key cannot be known
without first resolving <hash>, which may involve talking to the
network.
License: MIT
Signed-off-by: Stephen Whitmore <noffle@ipfs.io>
With GNU "grep" \b works but it's safer to use "egrep" which is also
POSIX rather than "grep" as not all the "grep"s out there might support \b.
License: MIT
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
dont GC blocks used by pinner
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
comment GC algo
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
add lock to blockstore to prevent GC from eating wanted blocks
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
improve FetchGraph
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
separate interfaces for blockstore and GCBlockstore
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
reintroduce indirect pinning, add enumerateChildren dag method
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
* ID service stream
* make the relay service use msmux
* fix nc tests
Note from jbenet: Maybe we should remove the old protocol/muxer
and see what breaks. It shouldn't be used by anything now.
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
Signed-off-by: Juan Batiz-Benet <juan@benet.ai>
WARNING: No migration performed! That needs to come in a separate
commit, perhaps amended into this one.
This is the minimal rewrite, only changing the storage from
JSON(+extra keys) in Datastore to IPFS objects. All of the pinning
state is still loaded in memory, and written from scratch on Flush. To
do more would require API changes, e.g. adding error returns.
Set/Multiset is not cleanly separated into a library, yet, as it's API
is expected to change radically.
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
sharness: Don't assume we know all things that can create garbage
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
OS X sed is documented as "-i SUFFIX", GNU sed as "-iSUFFIX". The one
consistent case seems to be "-iSUFFIX", where suffix cannot empty (or
OS X will parse the next argument as the suffix).
This used to leave around files named `refsout=` on Linux, and was
just confusing.
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
We have to do something special for CircleCI in docker_exec()
because "docker exec" doesn't work on CircleCi:
https://circleci.com/docs/docker#docker-exec
We indeed get "Unsupported: Exec is not supported by the lxc
driver" with CircleCi, when using "docker exec".
License: MIT
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
This is needed on OSX otherwise the trash directory for the
t0020-init.sh test fails to be removed due to a permissions
error.
This fixes https://github.com/ipfs/go-ipfs/issues/2026
License: MIT
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
pin add, pin rm, and pin ls will be recursive unless
specified with '=false' eg. 'ipfs pin add -r=false <file>'
tests for pinning have been updated/added
License: MIT
Signed-off-by: ForrestWeston <Forrest.Weston@gmail.com>
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
use go1.5 syntax to ensure builds on older versions fail
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
fix t0230
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
Let's first add a comment to explain why the wrapper printf()
is needed.
Then let's replace the last instructions by quotes inside the
wrapper printf() first argument, and let's also put there the
eventual space so that we can remove the printf on the above
line.
License: MIT
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
This commit adds a very basic process that will periodically go through
a list of given ids and republish the values for their ipns entries.
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
This function can be usefull in many places.
See for example:
https://github.com/ipfs/go-ipfs/pull/1742
Git has `git rev-parse --sq-quote` that does the same thing.
License: MIT
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
There was a stale assumption that streaming output from a channel would
always be json.
This commit removes that code, allowing Content-Type to appropriately be
set like other, non-channel-streaming commands.
License: MIT
Signed-off-by: Cayman Nava <caymannava@gmail.com>
It is simpler and less error prone to just pass to `ipfs daemon`
all the arguments that are passed to test_launch_ipfs_daemon().
Maybe the arguments should be shell quoted too, but that's
another issue.
License: MIT
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
use NewNode instead of NewIPFSNode in most of the codebase
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
make mocknet work with node constructor better
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
finish cleanup of old construction method
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
blockservice.New doesnt return an error anymore
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
break up node construction into separate function
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
add error case to default filling on node constructor
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
This new file comes from:
https://www.cl.cam.ac.uk/~mgk25/ucs/examples/UTF-8-test.txt
It is licensed under CC BY 4.0.
After discussing that with its author, I think it is ok to
add it as is to our repository. The only restriction might
be that we should indicate in the file any change we make
to it.
License: CC BY 4.0
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
This changes the pin behavior. It uses the filenames given through
the api, and allows files to be streamed faltly (not a hierarchy),
which is easier for other things (like vinyl in node-ipfs-api land).
Files can also be entirely out of order, and the garbage intermediate
directories will not be pinned (gc-ed later).
The changes also mean the output of add has changed slightly-- it
no longer shows the local path added, but rather the dag path
relative to the added roots. This is a small difference, but changes
tests.
The dagutils.Editor creates a lot of chaff (intermediate objects)
along the way. Wonder how we might minimize the writes to the
datastore...
This commit also removes the "NilRepo()" part of the --only-hash
mode. We need to store at least in an in-mem repo/datastore because
otherwise the dagutils.Editor breaks.
License: MIT
Signed-off-by: Juan Batiz-Benet <juan@benet.ai>
up until now there has been a very annoying bug with get, we would
get halting behavior. I'm not 100% sure this commit fixes it,
but it should. It certainly fixes others found in the process of
digging into the get / tar extractor code. (wish we could repro
the bug reliably enough to make a test case).
This is a much cleaner tar writer. the ad-hoc, error-prone synch
for the tar reader is gone (with i believe was incorrect). it is
replaced with a simple pipe and bufio. The tar logic is now in
tar.Writer, which writes unixfs dag nodes into a tar archive (no
need for synch here). And get's reader is constructed with DagArchive
which sets up the pipe + bufio.
NOTE: this commit also changes this behavior of `get`:
When retrieving a single file, if the file exists, get would fail.
this emulated the behavior of wget by default, which (without opts)
does not overwrite if the file is there. This change makes get
fail if the file is available locally. This seems more intuitive to
me as expected from a unix tool-- though perhaps it should be
discussed more before adopting.
Everything seems to work fine, and i have not been able to reproduce
the get halt bug.
License: MIT
Signed-off-by: Juan Batiz-Benet <juan@benet.ai>
this commit changes the behavior of ipfs add -w:
- it makes it able to work with ipfs add -r <dir>
- instead of hacking around the add, we simply just add a wrapper
directory around the whole result of the add. this means that
ipfs add -w calls will output _two_ lines, but this is actually
more correct than outputting one line, as two objects were added.
this _may_ break scripts out there which expect the output to
look a certain way. we should consider whether the old output is
more _useful_ (even if less in-line with the model.)
License: MIT
Signed-off-by: Juan Batiz-Benet <juan@benet.ai>
looks like the test was broken by GC-ing everything.
the pin expects $HASH to still be there.
License: MIT
Signed-off-by: Juan Batiz-Benet <juan@benet.ai>
License: MIT
Signed-off-by: Henry <cryptix@riseup.net>
t0080-repo.sh: added gateway assets to pinning tests
License: MIT
Signed-off-by: Henry <cryptix@riseup.net>
fix the nc wait. the issue was that stdin needs to remain _open_
but not receive any input for some time. If stdin receives (invalid)
input or closes, the other side terminates the connection before
writing out the muxer frames + identify handshake.
This commit also changes the use of `!` for `test_must_fail`
License: MIT
Signed-off-by: Juan Batiz-Benet <juan@benet.ai>
daemon output now includes initial swarm addresses. this is not a
full solution, as a change in network will not trigger re-printing.
We need a good way to do that.
This made me re-think how we're outputting these messages, perhaps
we should be throwing them as log.Events, and capturing some with
a special keyword to output to the user on stdout. Things like
network addresses being rebound, NATs being holepunched, external
network addresses being figured out, connections established, etc
may be valuable events to show the user. Of course, these should be
very few, as a noisy daemon is an annoying daemon.
License: MIT
Signed-off-by: Juan Batiz-Benet <juan@benet.ai>
IPFS_PATH should really be exported to make sure it is
available to the ipfs binary.
It looks like sharness tests fail otherwise on CircleCi.
License: MIT
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
We don't want to prefix these results with the argument. If there was
only one argument, the unprefixed results are still explicit.
License: MIT
Signed-off-by: W. Trevor King <wking@tremily.us>
Discussion with Juan on IRC ([1] through [2]) lead to this adjusted
JSON output. Benefits over the old output include:
* deduplication (we only check the children of a given Merkle node
once, even if multiple arguments resolve to that hash)
* alphabetized output (like POSIX's ls). As a side-effect of this
change, I'm also matching GNU Coreutils' ls output (maybe in POSIX?)
by printing an alphabetized list of non-directories (one per line)
first, with alphabetized directory lists afterwards.
[1]: https://botbot.me/freenode/ipfs/2015-06-12/?msg=41725570&page=5
[2]: https://botbot.me/freenode/ipfs/2015-06-12/?msg=41726547&page=5
License: MIT
Signed-off-by: W. Trevor King <wking@tremily.us>
This doesn't affect the text output, which was already using a
stringified name. The earlier stringification does change the JSON
output from an enumeration integer (e.g. 2) to the string form
(e.g. "File"). If/when we transition to Merkle-object types named by
their hash, we will probably want to revisit this and pass both the
type hash and human-readable-but-collision-prone name on to clients.
License: MIT
Signed-off-by: W. Trevor King <wking@tremily.us>
Change the approach to the directory-header control so we can set the
Argument value in the JSON response.
Stripping the trailing newline from the JSON output is annoying, but
looking over [1] I saw no easy way to add a newline to the JSON
output. And with the general framework that commands/ attempts to be,
it feels a bit funny to customize the JSON output for a command-line
program. Perhaps a workable solution is to have the command-line
client append newlines to any output that otherwise lacks them? But
that seems like a change best left to a separate series.
[1]: http://golang.org/pkg/encoding/json/
License: MIT
Signed-off-by: W. Trevor King <wking@tremily.us>
Instead of raising "keychains not yet implemented" whenever we have an
explicit node ID, only raise the error when the given node ID isn't
the local node. This allows folks to use the more-general
explicit-node-ID form in scripts and such now, as long as they use the
local node name when calling those scripts.
Also add a test for this case, and update the comment for the
one-argument case to match the current syntax for extracting a
multihash name string.
License: MIT
Signed-off-by: W. Trevor King <wking@tremily.us>
Folks operating at the Unix-filesystem level shouldn't care about that
level of Merkle-DAG detail. Before this commit we had:
$ ipfs unixfs ls /ipfs/QmSRCHG21Sbqm3EJG9aEBo4vS7Fqu86pAjqf99MyCdNxZ4/busybox
/ipfs/QmSRCHG21Sbqm3EJG9aEBo4vS7Fqu86pAjqf99MyCdNxZ4/busybox:
... several lines of empty-string names ...
And with this commit we have:
$ ipfs unixfs ls /ipfs/QmSRCHG21Sbqm3EJG9aEBo4vS7Fqu86pAjqf99MyCdNxZ4/busybox
/ipfs/QmSRCHG21Sbqm3EJG9aEBo4vS7Fqu86pAjqf99MyCdNxZ4/busybox
I also reworked the argument-prefixing (object.Argument) in the output
marshaller to avoid redundancies like:
$ ipfs unixfs ls /ipfs/QmSRCHG21Sbqm3EJG9aEBo4vS7Fqu86pAjqf99MyCdNxZ4/busybox
/ipfs/QmSRCHG21Sbqm3EJG9aEBo4vS7Fqu86pAjqf99MyCdNxZ4/busybox:
/ipfs/QmSRCHG21Sbqm3EJG9aEBo4vS7Fqu86pAjqf99MyCdNxZ4/busybox
As a side-effect of this rework, we no longer have the trailing blank
line that we used to have after the final directory listing.
The new ErrImplementation is like Python's NotImplementedError, and is
mostly a way to guard against external changes that would need
associated updates in this code. For example, once we see something
that's neither a file nor a directory, we'll have to update the switch
statement to handle those objects.
License: MIT
Signed-off-by: W. Trevor King <wking@tremily.us>
This is similar to 'ipfs ls ...', but it:
* Lists file sizes that match the content size:
$ ipfs --encoding=json unixfs ls /ipfs/QmSRCHG21Sbqm3EJG9aEBo4vS7Fqu86pAjqf99MyCdNxZ4
{
"Objects": [
{
"Argument": "/ipfs/QmSRCHG21Sbqm3EJG9aEBo4vS7Fqu86pAjqf99MyCdNxZ4",
"Links": [
{
"Name": "busybox",
"Hash": "QmPbjmmci73roXf9VijpyQGgRJZthiQfnEetaMRGoGYV5a",
"Size": 1947624,
"Type": 2
}
]
}
]
}
$ ipfs cat /ipfs/QmSRCHG21Sbqm3EJG9aEBo4vS7Fqu86pAjqf99MyCdNxZ4/busybox | wc -c
1947624
'ipfs ls ...', on the other hand, is using the Merkle-descendant
size, which also includes fanout links and the typing information
unixfs objects store in their Data:
$ ipfs --encoding=json ls /ipfs/QmSRCHG21Sbqm3EJG9aEBo4vS7Fqu86pAjqf99MyCdNxZ4
{
"Objects": [
{
"Hash": "/ipfs/QmSRCHG21Sbqm3EJG9aEBo4vS7Fqu86pAjqf99MyCdNxZ4",
"Links": [
{
"Name": "busybox",
"Hash": "QmPbjmmci73roXf9VijpyQGgRJZthiQfnEetaMRGoGYV5a",
"Size": 1948128,
"Type": 2
}
]
}
]
}
* Has a simpler text output corresponding to POSIX ls [1]:
$ ipfs unixfs ls /ipfs/QmV2FrBtvue5ve7vxbAzKz3mTdWq8wfMNPwYd8d9KHksCF/gentoo/stage3/amd64/2015-04-02
bin
dev
etc
proc
run
sys
$ ipfs ls /ipfs/QmV2FrBtvue5ve7vxbAzKz3mTdWq8wfMNPwYd8d9KHksCF/gentoo/stage3/amd64/2015-04-02
QmSRCHG21Sbqm3EJG9aEBo4vS7Fqu86pAjqf99MyCdNxZ4 1948183 bin/
QmUNLLsPACCz1vLxQVkXqqLX5R1X345qqfHbsf67hvA3Nn 4 dev/
QmUz1Z5jnQEjwr78fiMk5babwjJBDmhN5sx5HvPiTGGGjM 1207 etc/
QmUNLLsPACCz1vLxQVkXqqLX5R1X345qqfHbsf67hvA3Nn 4 proc/
QmUNLLsPACCz1vLxQVkXqqLX5R1X345qqfHbsf67hvA3Nn 4 run/
QmUNLLsPACCz1vLxQVkXqqLX5R1X345qqfHbsf67hvA3Nn 4 sys/
The minimal output allows us to start off with POSIX compliance and
then add options (which may or may not be POSIX compatible) to
adjust the output format as we get a better feel for what we need
([2] through [3]).
[1]: http://pubs.opengroup.org/onlinepubs/9699919799/utilities/ls.html
[2]: https://botbot.me/freenode/ipfs/2015-06-12/?msg=41724727&page=5
[3]: https://botbot.me/freenode/ipfs/2015-06-12/?msg=41725146&page=5
License: MIT
Signed-off-by: W. Trevor King <wking@tremily.us>
the iptb tests are failing all the time on travis. It's possible
this is still a port problem, and it may be something else.
Regardless, right now they're just adding noise.
This PR moves the addition of new blocks to our wantlist (and their
subsequent broadcast to the network) outside of the clientWorker loop.
This allows blocks to more quickly propogate to peers we are already
connected to, where before we had to wait for the previous findProviders
call in clientworker to complete before we could notify our partners of
the next blocks that we want. I then changed the naming of the
clientWorker and related variables to be a bit more appropriate to the
model. Although the clientWorker (now named providerConnector) feels a
bit awkward and should probably be changed.
fix test assumption
Previously we had a confusing situation, with:
* single-arg doc: published name <name> to <value>
* double-arg doc: published name <value> to <name>
* implementation: Published name <name> to <value>
Now we have the uniform:
Published to <name>: <value>
With the following goals:
1. It's clear that we're writing <value> to <name>'s IPNS slot in the
DHT.
2. We preserve the order of arguments from the command-line
invocation:
$ ipfs name publish <name> <value>
Published to <name>: <value>
commands/object: remove objectData() and objectLinks() helpers
resolver: added context parameters
sharness: $HASH carried the \r from the http protocol with
sharness: write curl output to individual files
http gw: break PUT handler until PR#1191
As test directories contain a space, we need to
properly quote paths, otherwise we get errors like:
```
umount: /home/christian/gocode/src/github.com/ipfs/go-ipfs/test/sharness/trash is not mounted (according to mtab)
umount: /home/christian/gocode/src/github.com/ipfs/go-ipfs/test/sharness/trash is not mounted (according to mtab)
```
instead of:
```
umount: /home/christian/gocode/src/github.com/ipfs/go-ipfs/test/sharness/trash directory.t0030-mount.sh/ipfs is not mounted (according to mtab)
umount: /home/christian/gocode/src/github.com/ipfs/go-ipfs/test/sharness/trash directory.t0030-mount.sh/ipns is not mounted (according to mtab)
```
License: MIT
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
As the ipfs mount call is now encapsulated in a
temporary function (see previous commit) its
output should not be tempered with by
test_must_fail.
License: MIT
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
As described in issue #1109 on OSX some output from
test_must_fail unfortunately goes into the "output"
file that we use to test the output from "ipfs mount".
This patch avoids the above by encapsulating the call
to "ipfs mount" into a temporary function.
License: MIT
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
sharness should only send the kill signal once, as that is
what a graceful shutdown should do. in the event that doesn't
happen, we should send it again, and then kill -9 to prevent it
lingering and messing with other tests.
This changes .go-ipfs to .ipfs everywhere.
And by the way this defines a DefaultPathName const
for this name.
License: MIT
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
This commit adds a new set of sharness tests for pinning, and addresses
bugs that were pointed out by said tests.
test/sharness: added more pinning tests
Pinning is currently broken. See issue #1051. This commit introduces
a few more pinning tests. These are by no means exhaustive, but
definitely surface the present problems going on. I believe these
tests are correct, but not sure. Pushing them as failing so that
pinning is fixed in this PR.
make pinning and merkledag.Get take contexts
improve 'add' commands usage of pinning
FIXUP: fix 'pin lists look good'
ipfs-pin-stat simple script to help check pinning
This is a simple shell script to help check pinning.
We ought to strive towards making adding commands this easy.
The http api is great and powerful, but our setup right now
gets in the way. Perhaps we can clean up that area.
updated t0081-repo-pinning
- fixed a couple bugs with the tests
- made it a bit clearer (still a lot going on)
- the remaining tests are correct and highlight a problem with
pinning. Namely, that recursive pinning is buggy. At least:
towards the end of the test, $HASH_DIR4 and $HASH_FILE4 should
be pinned indirectly, but they're not. And thus get gc-ed out.
There may be other problems too.
cc @whyrusleeping
fix grep params for context deadline check
fix bugs in pin and pin tests
check for block local before checking recursive pin
GNU Make's wildcard function does not recurse into subdirectories when
passed the '**' glob, which results in adding a dependency only to .go
files in the first level of subdirectories under the source root.
We shell out to 'find' instead, which catches all .go files in the
given directory.
Running make -jN would result in the tests starting to execute
before the tests binaries were built, resulting in the error:
"Cannot find the tests' local ipfs tool"
Each test now depends on the deps. They also depend on a new
target for cleaning the test results, so that the tests can
write new clean results.
The aggregate target also needs to depend on the same test
results clean target, as well as the tests themselves, so
that the aggregation happens when all tests have finished
running.
By introducing a separate target for cleaning test results we
also ensure that we don't end up removing and rebuilding
the binary on each test run.
The result is that the tests *can* be run with with -jN > 1,
but individual tests may still not supports this, so to get
stable test results it's still recommended to run them in
sequence.
The GOFLAGS variable makes it possible to run all sharness
tests with go binaries built with some special flags.
The "race" target makes it easy run the sharness tests
with go binaries built with the -race flag.
License: MIT
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
...and remove bin/* stuff from the .gitignore
as /test/bin is already in the root .gitignore.
License: MIT
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
This makes it possible to build binaries with
different flags.
The content of the GOFLAGS variable is stored
in a IPFS-BUILD-OPTIONS file, so that if GOFLAGS
changes a rebuild of the binaries with the new
flags is forced.
License: MIT
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
This script can be used in a Makefile to detect flag changes
and to save the new flags in a file.
License: MIT
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
