Commit Graph

176 Commits

Author SHA1 Message Date
Steven Allen
3eba14aa24 gx update
Updates:

* go-kad-dht: Query performance improvements, DHT client fixes, validates
  records on *local* put.
* go-libp2p-swarm/go-libp2p-transport: Timeout improvements.
* go-multiaddr-net: Exposes useful Conn methods (CloseWrite, CloseRead, etc.)
* go-log: fixes possible panic when enabling/disabling events.
* go-multiaddr: fixes possible panic when stringifying malformed multiaddrs,
  adds support for consuming /p2p/ multiaddrs.

fixes #5113
unblocks #4895

License: MIT
Signed-off-by: Steven Allen <steven@stebalien.com>
2018-06-26 17:11:33 -07:00
Whyrusleeping
9f9ddd561c
Merge pull request #5098 from schomatis/feat/unixfs/fsnode-include-pb
unixfs: integrate `pb.Data` into `FSNode` to avoid duplicating fields
2018-06-13 15:24:24 +08:00
Lucas Molas
934b8b0ddc unixfs: integrate pb.Data into FSNode
To avoid duplicating fields and making the code easier to follow.

Remove all of `FSNode` previous fields in favor on a single `pb.Data` structure
that is not exported. Accessor methods are added only for the necessary internal
fields. This takes up more memory, `pb.Data` is always created inside `FSNode`
and it stays there instead of just being created and destroyed during the
(un)marshal operations.

The removed fields `Data`, `blocksizes` and `Type` had a direct counterpart in
the embedded `pb.Data` structure, in contrast (only) the `subtotal` field
doesn't have one, it was used as a temporary accumulator to track the
`Filesize`, which is now being kept updated on every modification (to ensure the
entire `FSNode` is always at a valid state), so `subtotal` could just be removed
without the addition of any other field (this temporary accumulator  was
obscuring how `Filesize` was computed).

To keep `Filesize` up to date a method was added (`UpdateFilesize()`) to adjust
its value in the two places where the file size could be modified, when changing
its data (in `SetData()`, accessor method added) and when adding or removing
child nodes (in `AddBlockSize()` and `RemoveBlockSize()`).

A constructor method was added (`NewFSNode()`) to initialize the required
fields, like `Type` which is explicitly set, this deprecates the previous
methodology of just calling `new(FSNode)` and relying in the default value of
`pb.Data_DataType` (`Data_Raw`) to avoid an explicit assignment. Also,
`Filesize` is initialized to avoid being left with a `nil` value before
marshaling empty nodes, which would result in a different hash from previous
versions, to be backwards compatible. Previous versions of `GetBytes()` always
set the `Filesize` value, even though it is reflected as an `optional` field in
the `.proto` file (this may be an inaccurate field rule).

Without the duplicated fields the functions `GetBytes()` and `FSNodeFromBytes()`
are now reduced to simple `Marshal()` and `Unmarshal()` operations respectively.

License: MIT
Signed-off-by: Lucas Molas <schomatis@gmail.com>
2018-06-11 09:31:33 -03:00
Steven Allen
85acad9a01 gx update go-log, sys, go-crypto
* go-log
* sys
* go-crypto

License: MIT
Signed-off-by: Steven Allen <steven@stebalien.com>
2018-06-08 22:36:24 -07:00
Whyrusleeping
65b8d70eca
Merge pull request #4693 from ipfs/kevina/fix-zero-raw-leaf
Create a raw node instead of a file node when there is no content.
2018-06-08 16:41:24 +08:00
Steven Allen
1e9e2f453c update deps
License: MIT
Signed-off-by: Steven Allen <steven@stebalien.com>
2018-05-31 17:59:17 -07:00
Kevin Atkinson
6ab1e71473 Avoid changing hash for empty non-raw leaves.
License: MIT
Signed-off-by: Kevin Atkinson <k@kevina.org>
2018-05-31 04:27:34 -04:00
Kevin Atkinson
485a5e59e9 Create a raw node instead of a file node when there is no content.
This fixes things so when raw-leaves are enabled a zero size file creates
a zero size raw leaf.  When raw-leaves are not enabled the hash created
changes from QmbFMke1KXqnYyBBWxB74N4c5SBnJMVAiMNRcGu6x1AwQH to
Qmdsf68UUYTSSx3i4GtDJfxzpAEZt7Mp23m3qa36LYMSiW, since the type field
changed from TFile to TRaw.

License: MIT
Signed-off-by: Kevin Atkinson <k@kevina.org>
2018-05-31 04:05:30 -04:00
Whyrusleeping
c225c344e4
Merge pull request #4833 from ipfs/misc/remove-dead-code
misc: Remove some dead code
2018-04-20 23:39:55 +09:00
Whyrusleeping
076fd91e5f
Merge pull request #4835 from ipfs/fix/typos
misc: Fix a few typos
2018-04-20 23:35:58 +09:00
Lucas Molas
e047b58a33 dag: deduplicate AddNodeLinkClean into AddNodeLink
`AddNodeLink` used to cache the linked node whereas `AddNodeLinkClean`
did not, however, at some point the former was changed to do the same
thing as the latter (i.e., not cache the linked node). That is, they now
do the same thing so there's no reason to have both.

The name `AddNodeLink` is preserved, even though it used to imply the
cache functionality contrasting with the `Clean` suffix of
`AddNodeLinkClean`, with this function removed the cache connotation
doesn't hold anymore.

License: MIT
Signed-off-by: Lucas Molas <schomatis@gmail.com>
2018-04-16 22:25:24 -03:00
Łukasz Magiera
b290286dd7 misc: Fix a few typos
License: MIT
Signed-off-by: Łukasz Magiera <magik6k@gmail.com>
2018-03-30 09:47:22 +02:00
Łukasz Magiera
2ee5cfb90f fix error style
License: MIT
Signed-off-by: Łukasz Magiera <magik6k@gmail.com>
2018-03-30 09:43:27 +02:00
Łukasz Magiera
ae528640d9 misc: Remove some dead code
License: MIT
Signed-off-by: Łukasz Magiera <magik6k@gmail.com>
2018-03-21 01:34:04 +01:00
Lucas Molas
9eb864a1d3 dag: refactor trickle importer
License: MIT
Signed-off-by: Lucas Molas <schomatis@gmail.com>
2018-02-23 00:08:00 -03:00
Hector Sanjuan
381977476f Extract: chunker: rename "chunk" to "chunker" as it is more consistent
License: MIT
Signed-off-by: Hector Sanjuan <hector@protocol.ai>
2018-02-08 12:49:39 +01:00
Hector Sanjuan
76d228cf75 Extract chunker: Use last gx'ed go-ipfs-chunker
License: MIT
Signed-off-by: Hector Sanjuan <hector@protocol.ai>
2018-02-08 12:43:29 +01:00
Hector Sanjuan
1811425aab go-ipfs-chunker: Use the stable gx'ed release
License: MIT
Signed-off-by: Hector Sanjuan <hector@protocol.ai>
2018-02-08 12:34:51 +01:00
Hector Sanjuan
c613fbec38 WIP: Extract: importers/chunk module as go-ipfs-chunker
License: MIT
Signed-off-by: Hector Sanjuan <hector@protocol.ai>
2018-02-08 12:34:15 +01:00
Hector Sanjuan
bb061d615c Extract posinfo package to github.com/ipfs/go-ipfs-posinfo
License: MIT
Signed-off-by: Hector Sanjuan <hector@protocol.ai>
2018-02-07 13:39:53 +01:00
Whyrusleeping
d1a3fccd34
Merge pull request #4637 from ipfs/gx/update-go-log
gx: update go-log
2018-02-03 15:30:42 -08:00
Hector Sanjuan
b382191619 Fix missing rename in test
License: MIT
Signed-off-by: Hector Sanjuan <hector@protocol.ai>
2018-02-02 11:52:52 +01:00
Hector Sanjuan
3abc6aa063 Fix depedency
I changed it unnoticedly.

License: MIT
Signed-off-by: Hector Sanjuan <hector@protocol.ai>
2018-02-02 11:34:10 +01:00
Hector Sanjuan
5d90aa2a8f Docs: golint-ify "importers" module
This fixes all golint warnings in the importers module, adding
documentation and module descriptions.

License: MIT
Signed-off-by: Hector Sanjuan <hector@protocol.ai>
2018-02-01 23:47:38 +01:00
Steven Allen
b2cbfd299c gx: update go-log
License: MIT
Signed-off-by: Steven Allen <steven@stebalien.com>
2018-01-31 21:54:22 -08:00
Steven Allen
f9d935b984 rename import of go-ipld-format from node/format to ipld
License: MIT
Signed-off-by: Steven Allen <steven@stebalien.com>
2018-01-29 11:55:34 -08:00
Steven Allen
699b2c4bb8 make code-climate happier
License: MIT
Signed-off-by: Steven Allen <steven@stebalien.com>
2018-01-25 15:43:03 -08:00
Steven Allen
d154b4a990 merkledag: switch to new dag interface
Also:

* Update the blockstore/blockservice methods to match.
* Construct a new temporary offline dag instead of having a
  GetOfflineLinkService method.

License: MIT
Signed-off-by: Steven Allen <steven@stebalien.com>
2018-01-25 15:13:19 -08:00
Steven Allen
fe8846fcd7 gx: mass update
License: MIT
Signed-off-by: Steven Allen <steven@stebalien.com>
2018-01-24 15:58:44 -08:00
Jeromy
267c19ab9d some fixes for latest from go-ipfs-cmds
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
2018-01-21 11:11:47 -08:00
keks
fbc36b8962 update dependencies and update TODO
broken sharness test t0110 has been fixed in master

License: MIT
Signed-off-by: keks <keks@cryptoscope.co>
2018-01-21 11:11:47 -08:00
keks
e8ad094448 add TODO
License: MIT
Signed-off-by: keks <keks@cryptoscope.co>
2018-01-21 11:11:47 -08:00
keks
bd9576fa57 start adopting cmds3.0 - lots of errors!
- move go-ipfs-cmds/legacy to go-ipfs/commands/legacy
- update cmds,cmdkit; go test ./... ok

License: MIT
Signed-off-by: keks <keks@cryptoscope.co>
2018-01-21 11:11:47 -08:00
Steven Allen
c29f628907 add test for overallocation in chunker
License: MIT
Signed-off-by: Steven Allen <steven@stebalien.com>
2017-12-19 19:29:49 -08:00
Steven Allen
414b0ff1ba use DefaultSplitter function where appropriate
License: MIT
Signed-off-by: Steven Allen <steven@stebalien.com>
2017-12-19 19:08:55 -08:00
Steven Allen
aafbe65a13 Don't waste 256KiB buffers on small chunks.
License: MIT
Signed-off-by: Steven Allen <steven@stebalien.com>
2017-12-19 19:08:50 -08:00
Steven Allen
dfe7ef4fcf gx: update go-multihash
License: MIT
Signed-off-by: Steven Allen <steven@stebalien.com>
2017-12-14 13:09:51 -08:00
Steven Allen
76e1da02a8 gx: massive update
Note: This commit is technically broken. However, I need to make a bunch of
cmds changes to make this work and I'd rather not bundle both changes into a
single commit.

License: MIT
Signed-off-by: Steven Allen <steven@stebalien.com>
2017-11-20 22:20:27 -08:00
Jan Winkelmann
f287524949 cmd: use go-ipfs-cmds
License: MIT
Signed-off-by: keks <keks@cryptoscope.co>
2017-11-17 15:22:41 +01:00
Kevin Atkinson
2e15dcb647 Enable CidV1 (and other prefixes) in the Dag Modifier.
License: MIT
Signed-off-by: Kevin Atkinson <k@kevina.org>
2017-10-19 13:30:53 -04:00
Kevin Atkinson
7302c3ab66 Provide support for raw leaves in DAG modifier.
License: MIT
Signed-off-by: Kevin Atkinson <k@kevina.org>
2017-10-19 12:59:40 -04:00
Kevin Atkinson
189cf0cb03 Test raw leaves in trickle dag tests.
License: MIT
Signed-off-by: Kevin Atkinson <k@kevina.org>
2017-10-19 12:59:40 -04:00
Steven Allen
70d6629940 gx: update go-cid, go-multibase, base32
License: MIT
Signed-off-by: Steven Allen <steven@stebalien.com>
2017-09-01 17:46:49 -07:00
Jeromy
cccc6a94a3 update go-multihash and bubble up changes
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
2017-07-11 19:27:43 -07:00
Łukasz Magiera
8ab93aaeb2 Update go-datastore to 1.2.2, go-cid to 0.7.16
License: MIT
Signed-off-by: Łukasz Magiera <magik6k@gmail.com>
2017-07-04 20:18:57 +02:00
Steven Allen
13636bef50 blocks: gx import go-block-format
And updated related dependencies.

License: MIT
Signed-off-by: Steven Allen <steven@stebalien.com>
2017-06-29 22:12:14 +02:00
Steven Allen
bccd4d4e8f blocks: move block format to it's own repo
We need to reference it from outside of this repo.

License: MIT
Signed-off-by: Steven Allen <steven@stebalien.com>
2017-06-29 22:12:14 +02:00
zramsay
096b3232d5 address PR comments; remove commented/dead code
License: MIT
Signed-off-by: Zach Ramsay <zach.ramsay@gmail.com>
2017-06-01 12:13:04 -04:00
zramsay
c5df8f0796 apply the megacheck tool to improve code quality
License: MIT
Signed-off-by: Zach Ramsay <zach.ramsay@gmail.com>
2017-05-31 16:56:11 -04:00
Kevin Atkinson
e23dbd291b Add support for using an alternative hash function with raw nodes.
License: MIT
Signed-off-by: Kevin Atkinson <k@kevina.org>
2017-05-14 17:38:26 -04:00