instead of resolving all the pins first and then pinning, pin after resolving
each pin.
This:
1. Avoids storing all the nodes in memory.
2. Avoids not showing pin progress.
fixes#4122
License: MIT
Signed-off-by: Steven Allen <steven@stebalien.com>
This came from an old commit that used a TODO context. Now that we have a real
context, use that.
License: MIT
Signed-off-by: Steven Allen <steven@stebalien.com>
* Change ResolveToCid to take a Resolver and a NameSystem instead of an ipfs
Node.
* Make the pin/unpin methods use a Unixfs path resolver.
Closes: #3974
License: MIT
Signed-off-by: Steven Allen <steven@stebalien.com>
Errors from ColoredSet are now reported as encountered and errors
encountered when deleting blocks are no longer ignored.
License: MIT
Signed-off-by: Kevin Atkinson <k@kevina.org>
gc: report errors from ColoredSet as encountered
License: MIT
Signed-off-by: Kevin Atkinson <k@kevina.org>
GetStorageUsage() is super expensive as it involves a full repo scan,
and it's already bad enough that we have do it once.
The call that's being removed here is purely for cosmetical purposes:
printing the number of bytes freed by the GC run. Let's drop it.
License: MIT
Signed-off-by: Lars Gierth <larsg@systemli.org>
Also change existing 'Node' type to 'ProtoNode' and use that most
everywhere for now. As we move forward with the integration we will try
and use the Node interface in more places that we're currently using
ProtoNode.
License: MIT
Signed-off-by: Jeromy <why@ipfs.io>
This change adds the /ipfs/bitswap/1.1.0 protocol. The new protocol
adds a 'payload' field to the protobuf message and deprecates the
existing 'blocks' field. The 'payload' field is an array of pairs of cid
prefixes and block data. The cid prefixes are used to ensure the correct
codecs and hash functions are used to handle the block on the receiving
end.
License: MIT
Signed-off-by: Jeromy <why@ipfs.io>
Instead make LinkService a part of DAGService. The LinkService is now
simply an interface that DAGService implements. Also provide a
GetOfflineLinkService() method that the GC uses to get an offline
instance.
License: MIT
Signed-off-by: Kevin Atkinson <k@kevina.org>
This method will use the (also new) LinkService if it is available to
retrieving just the links for a MerkleDAG without necessary having to
retrieve the underlying block.
For now the main benefit is that the pinner will not break when a block
becomes invalid due to a change in the backing file. This is possible
because the metadata for a block (that includes the Links) is stored
separately and thus always available even if the backing file changes.
License: MIT
Signed-off-by: Kevin Atkinson <k@kevina.org>
* Update golog in go-ipfs
License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protonmail.ch>
* Update go-libp2p for go-log
License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protonmail.ch>
* Update go-libp2p-secio for go-log
License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protonmail.ch>
* Update go-libp2p-crypto for go-log
License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protonmail.ch>
* Update go-libp2p-peer for go-log
License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protonmail.ch>
* Import peersore, it wasn't imported
License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protonmail.ch>
* Update peerstore
License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protonmail.ch>
* Update peer
License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protonmail.ch>
* Update secio
License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protonmail.ch>
* Update go-libp2p
License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protonmail.ch>
We didn't previously initialize the Datastore config section.
The respective function exists, but was dead code up until now.
This lead to weird decisions like the GC code deciding on defaults,
and writing these to the config file. Don't want GC to touch the config.
License: MIT
Signed-off-by: Lars Gierth <larsg@systemli.org>
Fixesipfs/go-ipfs#2155 by turning the hash path arguments into keys
and unpinning directly, rather than running a full core.Resolve on
them. This lets users fail fast when they try to remove pins that
they don't have locally.
Note that this will only work when the path is of the form <hash> or
/ipfs/<hash>. Given e.g. /ipfs/<hash>/foo, foo's key cannot be known
without first resolving <hash>, which may involve talking to the
network.
License: MIT
Signed-off-by: Stephen Whitmore <noffle@ipfs.io>
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
dont GC blocks used by pinner
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
comment GC algo
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
add lock to blockstore to prevent GC from eating wanted blocks
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
improve FetchGraph
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
separate interfaces for blockstore and GCBlockstore
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
reintroduce indirect pinning, add enumerateChildren dag method
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
