Commit Graph

8806 Commits

Author SHA1 Message Date
Whyrusleeping
9bad5fe6fe
Merge pull request #4672 from ipfs/feat/coreapi-paths
Improve paths in CoreApi
2018-07-17 18:06:58 -05:00
Łukasz Magiera
4d5a96b0d6 coreapi: path rebase
License: MIT
Signed-off-by: Łukasz Magiera <magik6k@gmail.com>
2018-07-17 09:00:50 +02:00
Łukasz Magiera
7adf1cb40d coreapi: move path utils to interface
License: MIT
Signed-off-by: Łukasz Magiera <magik6k@gmail.com>
2018-07-17 08:44:54 +02:00
Łukasz Magiera
082498de6d coreapi: more docs for ResolvedPath
License: MIT
Signed-off-by: Łukasz Magiera <magik6k@gmail.com>
2018-07-17 08:43:48 +02:00
Łukasz Magiera
0adb69a773 coreapi: fix TestGatewayGet after rebase
License: MIT
Signed-off-by: Łukasz Magiera <magik6k@gmail.com>
2018-07-17 08:43:48 +02:00
Łukasz Magiera
0f6bd2d173 coreapi: fix resolved path root for some paths
License: MIT
Signed-off-by: Łukasz Magiera <magik6k@gmail.com>
2018-07-17 08:43:48 +02:00
Łukasz Magiera
e227fc3865 coreapi: fix ipns path resolving
License: MIT
Signed-off-by: Łukasz Magiera <magik6k@gmail.com>
2018-07-17 08:43:48 +02:00
Łukasz Magiera
604b187cca path: add tests for ipld paths
License: MIT
Signed-off-by: Łukasz Magiera <magik6k@gmail.com>
2018-07-17 08:43:48 +02:00
Łukasz Magiera
d5b5e0d7ac coreapi: add more docs for path
License: MIT
Signed-off-by: Łukasz Magiera <magik6k@gmail.com>
2018-07-17 08:43:48 +02:00
Łukasz Magiera
1d23bbeb0f coreapi: path review
License: MIT
Signed-off-by: Łukasz Magiera <magik6k@gmail.com>
2018-07-17 08:43:48 +02:00
Łukasz Magiera
9b288560a4 coreapi: path remainders
License: MIT
Signed-off-by: Łukasz Magiera <magik6k@gmail.com>
2018-07-17 08:43:48 +02:00
Łukasz Magiera
7ee6194352 coreapi: path.Mutable
License: MIT
Signed-off-by: Łukasz Magiera <magik6k@gmail.com>
2018-07-17 08:43:48 +02:00
Łukasz Magiera
f5f44ab246 coreapi: remove ctx from ParsePath, split ParseCid
License: MIT
Signed-off-by: Łukasz Magiera <magik6k@gmail.com>
2018-07-17 08:43:48 +02:00
Łukasz Magiera
15f8fc60c0 coreapi: separate path into two types
License: MIT
Signed-off-by: Łukasz Magiera <magik6k@gmail.com>
2018-07-17 08:43:48 +02:00
Łukasz Magiera
4f6c0666ea coreapi: move path impl to path.go
License: MIT
Signed-off-by: Łukasz Magiera <magik6k@gmail.com>
2018-07-17 08:42:06 +02:00
Łukasz Magiera
338e90e9c8 coreapi: expand public path api
License: MIT
Signed-off-by: Łukasz Magiera <magik6k@gmail.com>
2018-07-17 08:40:48 +02:00
Whyrusleeping
3218703f21
Merge pull request #5162 from ipfs/feat/improve-preload
always try to read ahead by at least 5 blocks in the PBDagReader
2018-07-16 23:43:14 -05:00
Whyrusleeping
2dcb7f0985
Merge pull request #5246 from ipfs/gx/update-go-cid
update go-cid
2018-07-16 22:29:35 -05:00
Steven Allen
9a9979b193 update go-cid
alternative to #5243 that updates go-cid and all packages that depend on it

License: MIT
Signed-off-by: Steven Allen <steven@stebalien.com>
2018-07-16 15:16:49 -07:00
Whyrusleeping
0ab36f00bc
Merge pull request #5118 from schomatis/feat/importer/remove-unixfsnode
importer: remove `UnixfsNode` from the balanced builder
2018-07-16 18:34:03 +02:00
Lucas Molas
474b77a2bd importer: remove UnixfsNode from the balanced builder
The `UnixfsNode` structure has multiple pointers to many (non-complementary)
mutually exclusive node types, only some of them are active (not-`nil`) at a
given time in the code path which made the code too convoluted. Specifically,
the most important distinction between node types was being hidden: leaf nodes
vs internal (non-leaf) nodes.

Remove entirely the use of `UnixfsNode` from the `balanced` package replacing it
in turn with the newly created `FSNodeOverDag` structure that represents the
UnixFS node encoded inside the DAG node, primarily used for internal node
representations. Leaf nodes are handled exclusively in the `NewLeafDataNode`
encapsulating its multiple representations (that we're previously exposed in
`UnixfsNode` as conflicting pointers).

The `builder.go` file has been completely rewritten, although the basic DAG
creation algorithm has been preserved (extending a full DAG by creating a new
root and linking the old one as its child), the most significant modification
has been in the loop of `Layout` that now only handles internal nodes (i.e.,
nodes with `depth` bigger than zero) to be able to adapt `fillNodeRec` to only
that scenario (avoiding the replace logic of the zero `depth` case with the
defective `Set` function, now removed). The `fillNodeRec` now explicitly returns
the `ipld.Node` and the size of the file data it's storing to propagate it
upwards into the DAG.

The `DagBuilderHelper` was heavily extended to incorporate `ipld.Node` functions
that would replace the `UnixfsNode` ones used by the balanced builder:
`NewLeafNode()`, `NewLeafDataNode()` and `AddNodeAndClose()`. Also, the
`ProcessFileStore` function was incorporated to encapsulate all the logic
related to the Filestore support which was scattered throughout the builder
logic, the `offset` that was being passed through most functions is now a part
of the `DagBuilderHelper`.

This has turned out to be a rather big commit, it should have been split into
more smaller and logically cohesive commits, but the `UnixfsNode` was too
entangled inside the logic and that would have required a progressive
modification of the `UnixfsNode` structure as well, which wasn't possible as it
is still being used by the balanced builder (the same reason why most of the
`UnixfsNode`-related functions cannot yet be removed, leaving the `helpers.go`
file mostly untouched).

License: MIT
Signed-off-by: Lucas Molas <schomatis@gmail.com>
2018-07-16 12:50:53 -03:00
Whyrusleeping
0349d9de43
Merge pull request #5239 from schomatis/fix/unixfs/directory/type-accessor
unixfs: fix `FSNode` accessor in `NewDirectoryFromNode`
2018-07-16 17:42:58 +02:00
Lucas Molas
26feaeb0cc unixfs: fix FSNode accessor in NewDirectoryFromNode
License: MIT
Signed-off-by: Lucas Molas <schomatis@gmail.com>
2018-07-16 12:27:43 -03:00
Whyrusleeping
419bfdc20f
Merge pull request #4924 from ipfs/docs/config
Add information about bloom filter to config.md
2018-07-16 17:12:10 +02:00
Whyrusleeping
b126601d80
Merge pull request #5160 from schomatis/feat/unixfs/dir-interface
unixfs: add a directory interface
2018-07-16 16:53:36 +02:00
Whyrusleeping
8fa1c881f1
Merge pull request #5205 from spartucus/patch-1
Refactor code
2018-07-16 16:52:58 +02:00
Whyrusleeping
bda81cd2d4
Merge pull request #5045 from schomatis/fix/cmd/files-ls-file-long
cmd: fix `files ls` to report hash and size for files
2018-07-16 16:52:13 +02:00
Jeromy
7927196fe4 version bump to 0.4.17-dev
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
2018-07-16 07:51:26 -07:00
Whyrusleeping
07feeec9c4
Merge pull request #5010 from ipfs/feat/diskusage2
Efficient "repo stat" (DiskUsage) and "--size-only" flag
2018-07-16 16:50:41 +02:00
Whyrusleeping
4f1391f477
Merge pull request #5139 from achingbrain/feat/specify-object-data-encoding
feat: Allow specifing how object data is encoded
2018-07-16 16:49:31 +02:00
Whyrusleeping
51607243c2
Merge pull request #5170 from schomatis/feat/mfs/root-val-as-dir
mfs: make `Root` value a `Directory`
2018-07-16 16:48:06 +02:00
Whyrusleeping
61d08eea4b
Merge pull request #5189 from schomatis/feat/pbdagreader/use-fsnode
pbdagreader: use FSNode instead of protobuf structure
2018-07-16 16:47:41 +02:00
Whyrusleeping
c9cda2c765
Merge pull request #5190 from schomatis/fix/dagreader/remove-offset
dagreader: remove `Offset()` method
2018-07-16 16:47:29 +02:00
Whyrusleeping
95f721c4df
Merge pull request #4896 from ipfs/feat/ai-mirror
URL store
2018-07-13 17:08:59 +02:00
Kevin Atkinson
1f29699d90 Address c.r. and additional tweaks.
License: MIT
Signed-off-by: Kevin Atkinson <k@kevina.org>
2018-07-13 09:07:52 -04:00
Kevin Atkinson
8dd970b73a filestore: Return consistent err msg. when file/urlstore is not enabled.
License: MIT
Signed-off-by: Kevin Atkinson <k@kevina.org>
2018-07-13 09:07:52 -04:00
Kevin Atkinson
6a4b1262a5 Make sure you can't add URL's unless the url store is enabled.
License: MIT
Signed-off-by: Kevin Atkinson <k@kevina.org>
2018-07-13 09:07:52 -04:00
Kevin Atkinson
0c2efb90d3 More test fixes.
License: MIT
Signed-off-by: Kevin Atkinson <k@kevina.org>
2018-07-13 09:07:52 -04:00
Kevin Atkinson
ed2bb81b8d Code cleanups to make code climate happy.
License: MIT
Signed-off-by: Kevin Atkinson <k@kevina.org>
2018-07-13 09:07:52 -04:00
Kevin Atkinson
0e24444280 Add some documentation to ipfs urlstore add command.
License: MIT
Signed-off-by: Kevin Atkinson <k@kevina.org>
2018-07-13 09:07:52 -04:00
Kevin Atkinson
b3457f240c Enhance tests.
License: MIT
Signed-off-by: Kevin Atkinson <k@kevina.org>
2018-07-13 09:07:52 -04:00
Kevin Atkinson
e5189f4230 Return better error code when an http request failed.
License: MIT
Signed-off-by: Kevin Atkinson <k@kevina.org>
2018-07-13 09:07:51 -04:00
Kevin Atkinson
90972095a9 Add test cases for urlstore.
License: MIT
Signed-off-by: Kevin Atkinson <k@kevina.org>
2018-07-13 09:07:51 -04:00
Kevin Atkinson
b53a1b3022 Add config option to enable urlstore.
License: MIT
Signed-off-by: Kevin Atkinson <k@kevina.org>
2018-07-13 09:07:51 -04:00
Kevin Atkinson
696a0f039e Simplify code: use prefix instead of flag to determine if a url
License: MIT
Signed-off-by: Kevin Atkinson <k@kevina.org>
2018-07-13 09:07:51 -04:00
Kevin Atkinson
d59a6e99a6 Fix "ipfs urlstore add" output.
License: MIT
Signed-off-by: Kevin Atkinson <k@kevina.org>
2018-07-13 09:04:22 -04:00
Jakub Sztandera
1a835202b2 filestore: add URLStore
License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protonmail.ch>
2018-07-13 09:04:22 -04:00
Whyrusleeping
92ac43afef
Merge pull request #5179 from ipfs/release-0.4.16
Ipfs 0.4.16 release
2018-07-13 13:06:39 +02:00
Jeromy
7bcc9caa84 Ipfs 0.4.16 release
License: MIT
Signed-off-by: Whyrusleeping <why@ipfs.com>
2018-07-13 13:00:41 +02:00
Steven Allen
265295c789 add a note about the fixed js interop
License: MIT
Signed-off-by: Steven Allen <steven@stebalien.com>
2018-07-13 13:00:23 +02:00