This changes the pin behavior. It uses the filenames given through
the api, and allows files to be streamed faltly (not a hierarchy),
which is easier for other things (like vinyl in node-ipfs-api land).
Files can also be entirely out of order, and the garbage intermediate
directories will not be pinned (gc-ed later).
The changes also mean the output of add has changed slightly-- it
no longer shows the local path added, but rather the dag path
relative to the added roots. This is a small difference, but changes
tests.
The dagutils.Editor creates a lot of chaff (intermediate objects)
along the way. Wonder how we might minimize the writes to the
datastore...
This commit also removes the "NilRepo()" part of the --only-hash
mode. We need to store at least in an in-mem repo/datastore because
otherwise the dagutils.Editor breaks.
License: MIT
Signed-off-by: Juan Batiz-Benet <juan@benet.ai>
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
implement rabin fingerprinting as a chunker for ipfs
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
vendor correctly
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
refactor chunking interface a little
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
work chunking interface changes up into importer
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
move chunker type parsing into its own file in chunk
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
up until now there has been a very annoying bug with get, we would
get halting behavior. I'm not 100% sure this commit fixes it,
but it should. It certainly fixes others found in the process of
digging into the get / tar extractor code. (wish we could repro
the bug reliably enough to make a test case).
This is a much cleaner tar writer. the ad-hoc, error-prone synch
for the tar reader is gone (with i believe was incorrect). it is
replaced with a simple pipe and bufio. The tar logic is now in
tar.Writer, which writes unixfs dag nodes into a tar archive (no
need for synch here). And get's reader is constructed with DagArchive
which sets up the pipe + bufio.
NOTE: this commit also changes this behavior of `get`:
When retrieving a single file, if the file exists, get would fail.
this emulated the behavior of wget by default, which (without opts)
does not overwrite if the file is there. This change makes get
fail if the file is available locally. This seems more intuitive to
me as expected from a unix tool-- though perhaps it should be
discussed more before adopting.
Everything seems to work fine, and i have not been able to reproduce
the get halt bug.
License: MIT
Signed-off-by: Juan Batiz-Benet <juan@benet.ai>
need to do it this way to avoid VERY confusing situations where
the user would change the API port (to another port, or maybe even
to :0). this way things dont break on the user, and by default,
users only need to change the API address and things should still
"just work"
License: MIT
Signed-off-by: Juan Batiz-Benet <juan@benet.ai>
ServeOptions take the node and muxer, they should get the listener
too as sometimes they need to operate on the listener address.
License: MIT
Signed-off-by: Juan Batiz-Benet <juan@benet.ai>
> ipfs add a b c
added Qmbvkmk9LFsGneteXk3G7YLqtLVME566ho6ibaQZZVHaC9 a
added QmR9pC5uCF3UExca8RSrCVL8eKv7nHMpATzbEQkAHpXmVM b
added QmetGxZTgo8tYAKQH1KLsY13MxqeVHbxYVmvzBzJAKU6Z7 c
added QmXg3WHLcjnz4ejeYF6FKVBkb4m1oKjQmF5fEWL9M1uQF3
> ipfs ls QmXg3WHLcjnz4ejeYF6FKVBkb4m1oKjQmF5fEWL9M1uQF3
Qmbvkmk9LFsGneteXk3G7YLqtLVME566ho6ibaQZZVHaC9 10 a
QmR9pC5uCF3UExca8RSrCVL8eKv7nHMpATzbEQkAHpXmVM 10 b
QmetGxZTgo8tYAKQH1KLsY13MxqeVHbxYVmvzBzJAKU6Z7 10 c
License: MIT
Signed-off-by: Juan Batiz-Benet <juan@benet.ai>
this commit changes the behavior of ipfs add -w:
- it makes it able to work with ipfs add -r <dir>
- instead of hacking around the add, we simply just add a wrapper
directory around the whole result of the add. this means that
ipfs add -w calls will output _two_ lines, but this is actually
more correct than outputting one line, as two objects were added.
this _may_ break scripts out there which expect the output to
look a certain way. we should consider whether the old output is
more _useful_ (even if less in-line with the model.)
License: MIT
Signed-off-by: Juan Batiz-Benet <juan@benet.ai>
unmounting wasn't happening, mostly because of a recent bug in
goprocess.SetTeardown. This commit bumps up some messages to
log.Warnings, as users may want to see them, and makes sure to
Unmount when a node shuts down.
License: MIT
Signed-off-by: Juan Batiz-Benet <juan@benet.ai>
this commit introduces more serious CORS tests that check
status response codes, and run real HTTP requests.
License: MIT
Signed-off-by: Juan Batiz-Benet <juan@benet.ai>
this commit makes the API handler short circuit the request if the
CORS headers say its not allowed. (the CORS handler only sets the
headers, but does not short-circuit)
It also makes the handler respect the referer again. See security
discussion at https://github.com/ipfs/go-ipfs/issues/1532
License: MIT
Signed-off-by: Juan Batiz-Benet <juan@benet.ai>
this commit adds the ability to specify arbitrary HTTP headers
for either the Gateway or the API. simply set the desired headers
on the config:
ipfs config --json API.HTTPHeaders.X-MyHdr '["meow :)"]'
ipfs config --json Gateway.HTTPHeaders.X-MyHdr '["meow :)"]'
License: MIT
Signed-off-by: Juan Batiz-Benet <juan@benet.ai>
Get had a random timeout of 60s. This commit fixes that, wiring
up our contexts correctly.
License: MIT
Signed-off-by: Juan Batiz-Benet <juan@benet.ai>
- First tab at integrating @krl's new index page
File icons are embedded in side icons.css (QmXB7PLRWH6bCiwrGh2MrBBjNkLv3mY3JdYXCikYZSwLED contains both the icons and bootstrap)
- Fix back links (..) (fixes#1365)
Thanks @JasonWoof for the insight. The back links now stop a t the root hash and work for links that do and dont end with a slash.
License: MIT
Signed-off-by: Henry <cryptix@riseup.net>
Add ErrNoComponents in ParsePath validation & remove redundant path
validation.
Any lines using core.Resolve & Resolver.ResolvePath will have their path
validated.
License: MIT
Signed-off-by: rht <rhtbot@gmail.com>
Currently `ipfs get -C <hash>` returns error even if <hash> is a file.
This PR is for the case when the compress flag is enabled, use the
dagreader directly and pipe to a gzip processor.
License: MIT
Signed-off-by: rht <rhtbot@gmail.com>
We don't want to prefix these results with the argument. If there was
only one argument, the unprefixed results are still explicit.
License: MIT
Signed-off-by: W. Trevor King <wking@tremily.us>
Discussion with Juan on IRC ([1] through [2]) lead to this adjusted
JSON output. Benefits over the old output include:
* deduplication (we only check the children of a given Merkle node
once, even if multiple arguments resolve to that hash)
* alphabetized output (like POSIX's ls). As a side-effect of this
change, I'm also matching GNU Coreutils' ls output (maybe in POSIX?)
by printing an alphabetized list of non-directories (one per line)
first, with alphabetized directory lists afterwards.
[1]: https://botbot.me/freenode/ipfs/2015-06-12/?msg=41725570&page=5
[2]: https://botbot.me/freenode/ipfs/2015-06-12/?msg=41726547&page=5
License: MIT
Signed-off-by: W. Trevor King <wking@tremily.us>
This doesn't affect the text output, which was already using a
stringified name. The earlier stringification does change the JSON
output from an enumeration integer (e.g. 2) to the string form
(e.g. "File"). If/when we transition to Merkle-object types named by
their hash, we will probably want to revisit this and pass both the
type hash and human-readable-but-collision-prone name on to clients.
License: MIT
Signed-off-by: W. Trevor King <wking@tremily.us>
Change the approach to the directory-header control so we can set the
Argument value in the JSON response.
Stripping the trailing newline from the JSON output is annoying, but
looking over [1] I saw no easy way to add a newline to the JSON
output. And with the general framework that commands/ attempts to be,
it feels a bit funny to customize the JSON output for a command-line
program. Perhaps a workable solution is to have the command-line
client append newlines to any output that otherwise lacks them? But
that seems like a change best left to a separate series.
[1]: http://golang.org/pkg/encoding/json/
License: MIT
Signed-off-by: W. Trevor King <wking@tremily.us>
Instead of raising "keychains not yet implemented" whenever we have an
explicit node ID, only raise the error when the given node ID isn't
the local node. This allows folks to use the more-general
explicit-node-ID form in scripts and such now, as long as they use the
local node name when calling those scripts.
Also add a test for this case, and update the comment for the
one-argument case to match the current syntax for extracting a
multihash name string.
License: MIT
Signed-off-by: W. Trevor King <wking@tremily.us>
ipfs-shell [1] accesses the Command objects directly to construct
requests for an external IPFS daemon API. This isn't a terribly
robust approach, because it doesn't handle version differences between
the version of go-ipfs used to build the daemon and the version used
to build the ipfs-shell-consuming application. But for cases where
you can get those APIs to match it works well. Making these two
commands public allows us to write ipfs-shell wrappers for them.
Until we figure out how to get ipfs-shell working without access to
core/commands, I think the best approach is to make future command
objects and their returned structures public, and to go back and
expose existing commands/structures on an as-needed basis.
In this case, I need the public PublishCmd for the Docker-registry
storage driver, and I made the IpnsCmd public at the same time to stay
consistent for both 'ipfs name ...' sub-commands.
[1]: https://github.com/whyrusleeping/ipfs-shell
License: MIT
Signed-off-by: W. Trevor King <wking@tremily.us>
There has been a regression such that ./t0030-mount.sh fails on
'ipfs mount' fails when there is no mount dir
The issue was a change in how fuse errors are reported to the client
process. We have introduced an optimistic categorization that hides
the obscure fusermount error and replaces it with something a bit
more helpful.
License: MIT
Signed-off-by: Juan Batiz-Benet <juan@benet.ai>
Except when there is an explicit os.Exit(1) after the Critical line,
then replace with Fatal{,f}.
golang's log and logrus already call os.Exit(1) by default with Fatal.
License: MIT
Signed-off-by: rht <rhtbot@gmail.com>
Folks operating at the Unix-filesystem level shouldn't care about that
level of Merkle-DAG detail. Before this commit we had:
$ ipfs unixfs ls /ipfs/QmSRCHG21Sbqm3EJG9aEBo4vS7Fqu86pAjqf99MyCdNxZ4/busybox
/ipfs/QmSRCHG21Sbqm3EJG9aEBo4vS7Fqu86pAjqf99MyCdNxZ4/busybox:
... several lines of empty-string names ...
And with this commit we have:
$ ipfs unixfs ls /ipfs/QmSRCHG21Sbqm3EJG9aEBo4vS7Fqu86pAjqf99MyCdNxZ4/busybox
/ipfs/QmSRCHG21Sbqm3EJG9aEBo4vS7Fqu86pAjqf99MyCdNxZ4/busybox
I also reworked the argument-prefixing (object.Argument) in the output
marshaller to avoid redundancies like:
$ ipfs unixfs ls /ipfs/QmSRCHG21Sbqm3EJG9aEBo4vS7Fqu86pAjqf99MyCdNxZ4/busybox
/ipfs/QmSRCHG21Sbqm3EJG9aEBo4vS7Fqu86pAjqf99MyCdNxZ4/busybox:
/ipfs/QmSRCHG21Sbqm3EJG9aEBo4vS7Fqu86pAjqf99MyCdNxZ4/busybox
As a side-effect of this rework, we no longer have the trailing blank
line that we used to have after the final directory listing.
The new ErrImplementation is like Python's NotImplementedError, and is
mostly a way to guard against external changes that would need
associated updates in this code. For example, once we see something
that's neither a file nor a directory, we'll have to update the switch
statement to handle those objects.
License: MIT
Signed-off-by: W. Trevor King <wking@tremily.us>
