commands/object: remove objectData() and objectLinks() helpers
resolver: added context parameters
sharness: $HASH carried the \r from the http protocol with
sharness: write curl output to individual files
http gw: break PUT handler until PR#1191
Currently garbage collection is triggered manually and there are no
age-restrictions on the removal. I expect we'll eventually follow Git
and auto-launch garbage collection when we hit some threshold of disk
consumption (gc.auto). I expect we'll also follow Git and keep
unpinned or unreachable objects (gc.pruneexpire, etc.). But we don't
seem to do either of those yet.
I'm not entirely clear on the role that this package is filling, but
this description seems like a reasonable guess based on a quick skim
through it's exported API.
The last references to CastToReaders were commented out in 6faeee83
(cmds2/add: temp fix for -r. horrible hack, 2014-11-11) and then
removed completely in 032e9c29 (core/commands2: Updated 'add' command
for new file API, 2014-11-16).
The last references to CastToStrings was removed in a0bd29d5
(core/commands2: Fixed swarm command for new arguments API,
2014-11-18).
The change to an array of readers comes from e096060b
(refactor(core/commands2/add) split loop, 2014-11-06), where it's used
to setup readers for each path in the argument list. However, since
6faeee83 (cmds2/add: temp fix for -r. horrible hack, 2014-11-11) the
argument looping moved outside of add() and into Run(), so we can drop
the multiple-reader support from add().
Adding a file can create multiple nodes (e.g. the splitter can chunk
the file into several blocks), but:
1. we were only appending a single node per reader to our returned
list, and
2. we are only using the final node in that returned list,
so this commit also adjusts add() to return a single node reference
instead on an array of nodes.
the random permutaton for bootstrap peers was not working as
intended, returning the first four bootstrap peers always.
this commit fixes it to be a random subset.
Once the server is asked to shut down, we stop accepting new
connections, but the 'manners' graceful shutdown will wait for
all existing connections closed to close before finishing.
For keep-alive connections this will never happen unless the
client detects that the server is shutting down through the
ipfs API itself, and closes the connection in response.
This is a problem e.g. with the webui's connections visualization,
which polls the swarm/peers endpoint once a second, and never
detects that the API server was shut down.
We can mitigate this by telling the server to disable keep-alive,
which will add a 'Connection: close' header to the next HTTP
response on the connection. A well behaving client should then
treat that correspondingly by closing the connection.
Unfortunately this doesn't happen immediately in all cases,
presumably depending on the keep-alive timeout of the browser
that set up the connection, but it's at least a step in the
right direction.
When closing a node, the node itself only takes care of tearing down
its own children. As corehttp sets up a server based on a node, it
needs to also ensure that the server is accounted for when determining
if the node has been fully closed.
The server may stay alive for quite a while due to waiting on
open connections to close before shutting down. We should
find ways to terminate these connections in a more controlled
manner, but in the meantime it's helpful to be able to see
why a shutdown of the ipfs daemon is taking so long.
This changes .go-ipfs to .ipfs everywhere.
And by the way this defines a DefaultPathName const
for this name.
License: MIT
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
We now consider debugerrors harmful: we've run into cases where
debugerror.Wrap() hid valuable error information (err == io.EOF?).
I've removed them from the main code, but left them in some tests.
Go errors are lacking, but unfortunately, this isn't the solution.
It is possible that debugerros.New or debugerrors.Errorf should
remain still (i.e. only remove debugerrors.Wrap) but we don't use
these errors often enough to keep.
This commit adds a new set of sharness tests for pinning, and addresses
bugs that were pointed out by said tests.
test/sharness: added more pinning tests
Pinning is currently broken. See issue #1051. This commit introduces
a few more pinning tests. These are by no means exhaustive, but
definitely surface the present problems going on. I believe these
tests are correct, but not sure. Pushing them as failing so that
pinning is fixed in this PR.
make pinning and merkledag.Get take contexts
improve 'add' commands usage of pinning
FIXUP: fix 'pin lists look good'
ipfs-pin-stat simple script to help check pinning
This is a simple shell script to help check pinning.
We ought to strive towards making adding commands this easy.
The http api is great and powerful, but our setup right now
gets in the way. Perhaps we can clean up that area.
updated t0081-repo-pinning
- fixed a couple bugs with the tests
- made it a bit clearer (still a lot going on)
- the remaining tests are correct and highlight a problem with
pinning. Namely, that recursive pinning is buggy. At least:
towards the end of the test, $HASH_DIR4 and $HASH_FILE4 should
be pinned indirectly, but they're not. And thus get gc-ed out.
There may be other problems too.
cc @whyrusleeping
fix grep params for context deadline check
fix bugs in pin and pin tests
check for block local before checking recursive pin
humanize bandwidth output
instrument conn.Conn for bandwidth metrics
add poll command for continuous bandwidth reporting
move bandwidth tracking onto multiaddr net connections
another mild refactor of recording locations
address concerns from PR
lower mock nodes in race test due to increased goroutines per connection
not exactly elegant, but fixes#871 in the short term. in the mid term, unless more `repo` commands show up, i suggest just replacing `repo gc` simply by `gc`.
- updated go-ctxgroup and goprocess
ctxgroup: AddChildGroup was changed to AddChild. Used in two files:
- p2p/net/mock/mock_net.go
- routing/dht/dht.go
- updated context from hg repo to git
prev. commit in hg was ad01a6fcc8a19d3a4478c836895ffe883bd2ceab. (context: make parentCancelCtx iterative)
represents commit 84f8955a887232b6308d79c68b8db44f64df455c in git repo
- updated context to master (b6fdb7d8a4ccefede406f8fe0f017fb58265054c)
Aaron Jacobs (2):
net/context: Don't accept a context in the DoSomethingSlow example.
context: Be clear that users must cancel the result of WithCancel.
Andrew Gerrand (1):
go.net: use golang.org/x/... import paths
Bryan C. Mills (1):
net/context: Don't leak goroutines in Done example.
Damien Neil (1):
context: fix removal of cancelled timer contexts from parent
David Symonds (2):
context: Fix WithValue example code.
net: add import comments.
Sameer Ajmani (1):
context: fix TestAllocs to account for ints in interfaces
Note: the dht-specific part of the bootstrap function
was only there to make sure to call `dht.Update(ctx, npeer)`.
This already happens on all new connections made by the
network, as the dht is signed up for notifications.
This allows someone to host a static site by pointing a TXT record at their
content in IPFS, and a CNAME record at an IPFS gateway.
Note that such a setup technically violates RFC1912 (section 2.4; "A CNAME
record is not allowed to coexist with any other data."), but tends to work in
practice.
We may want to consider changing the DNS->IPFS resolution scheme to allow this
scenario to be RFC-compliant (e.g. store the mapping on a well-known subdomain
to allow CNAME records on the domain itself).
License: MIT
Signed-off-by: Kevin Wallace <kevin@pentabarf.net>
Each option now additionally returns the mux to be used by future options. If
every options returns the mux it was passed, the current behavior is unchanged.
However, if the option returns an a new mux, it can mediate requests to handlers
provided by future options:
return func(n *core.IpfsNode, mux *http.ServeMux) (*http.ServeMux, error) {
childMux := http.NewServeMux()
mux.Handle("/", handlerThatDelegatesToChildMux)
return childMux, nil
}
License: MIT
Signed-off-by: Kevin Wallace <kevin@pentabarf.net>
ipns paths are mutable and should not be cached. this error is
a byproduct of the currently messy gateway route. We should split
the /ipfs and /ipns routes up.
⚠️ this commit makes your current configs unusable, as the
default bootstrap peers. You may need to edit your config.
Go from:
```js
Bootstrap: [
{
"Address": "/ip4/104.131.131.82/tcp/4001",
"PeerID": "QmaCpDMGvV2BGHeYERUEnRQAwe3N8SzbUtfsmvsqQLuvuJ"
}
]
```
To:
```js
Bootstrap: [
"/ip4/104.131.131.82/tcp/4001/ipfs/QmaCpDMGvV2BGHeYERUEnRQAwe3N8SzbUtfsmvsqQLuvuJ"
]
```
- core: daemon stdout print to cmd + daemon init checks
- core: fixed bug where the gateway was printed as "API"
- sharness/test-lib: daemon init checks
- sharness/test-lib: portable TCP port check
- sharness/init: fix test bits output
- sharness: use common hashes in one place.
- move t0100-http-gateway -> t0111-gateway-writable
- sharness: test-lib funcs for gateway config
- sharness/t0111-gateway-writable: use sh funcs
- sharness/t0111-gateway-writable: fixes
- escape all vars (always `cmd "$VAR"` never `cmd $VAR`)
- use $FILEPATH, not $path
- last test seems to fail
We had a problem: we were starting all the services with the network
live, and so would miss early messages. We were noticing bitswap
messages not handled (not in muxer). Many of the subsystems expect
the network to _exist_ when they start up, so we split construction
and starting to listen into two separate steps.
This commit removes the dependency on go-fuse-version, and thus the
fuse headers. It also introduces an elaborate troubleshooting process
that diagnoses whether fuse installed -- and which version -- with as
little requirements as possible (attept to use sysctl, fall-back on
the go-fuse-version binary, etc). It then nicely instructs the user
what to do next.
