Commit Graph

1625 Commits

Author SHA1 Message Date
Will Scott
94de605570
startup bootstrap 2020-04-15 08:47:17 -07:00
Will Scott
02cf54dec6
rebase on dual DHT 2020-04-15 08:47:16 -07:00
Will Scott
635a5c798e
integration test for the dual dht 2020-04-15 08:47:15 -07:00
Steven Allen
1f23fc000c feat: add autonat config options
1. Enable AutoNATService on _all_ nodes by default. If it's an issue, we can
disable it in RC3 but this will give us the best testing results.
2. Expose options to configure AutoNAT rate limiting.
2020-04-14 20:54:58 -07:00
Steven Allen
5772ac7a5f test(sharness): improve CAR tests to remove some potential races
* Instead of ensuring that we GC nothing, verify that the repo validates.
* Avoid a race between removing the fifos and trying to read from them.
* Avoid all 'bash -c' invocations. Instead, make liberal use of subshells and
  wait.

Hopefully fixes #7123.
2020-04-13 21:42:27 -07:00
Steven Allen
28e31e17a8
Merge pull request #7127 from ipfs/feat/dual-dht
feat: introduce the dual WAN/LAN DHT
2020-04-13 19:01:12 -07:00
Steven Allen
c12c184ca4 fix: invalidate cache on failed publish
If we fail to publish, _invalidate_ our cache. The publish may have partially
succeeded.
2020-04-13 18:22:04 -07:00
Steven Allen
d0d508b2e1 fix: fix 'dht query' command to actually return the closest peers
Then improve the tests such that they actually work.
2020-04-13 17:42:03 -07:00
Peter Rabbitson
cb5863c038 Temporarily disable gc-race test
I am going to rig a more robust version and run it locally to crash
Disabling in master in order to validate the pipe reading itself
works correctly and reliably
2020-04-13 19:28:14 +02:00
Peter Rabbitson
768c287f9b Extra potential deadlock debugging 2020-04-10 19:25:16 +02:00
Peter Rabbitson
edc27793fb Extra cross-iptb-network testcases 2020-04-10 19:25:16 +02:00
Peter Rabbitson
4319c12692 Fixup all original test issues 2020-04-10 19:25:14 +02:00
Steven Allen
bb08f7fe56 assign public IP addresses for tests that need them 2020-04-09 21:41:42 -07:00
Steven Allen
9a98158bed test(sharness): test address filtering
tests #6995
2020-04-08 20:26:42 -07:00
Peter Rabbitson
a903e23ab9 We are running under set -e :picard-facepalm: 2020-04-08 14:54:49 -07:00
Peter Rabbitson
f9ae330978 Harden test - no -z in busybox 2020-04-08 14:54:49 -07:00
Peter Rabbitson
fbe40883e6 Remove PresentInImport tracking as per conversation 2020-04-08 14:54:49 -07:00
Peter Rabbitson
90c58d6cac Dag import functionality only ( no progress )
This still works over "loosely defined" .car files
Please refer to the sharness tests for extra info

We can tighten this up if the sentiment is "Postel was wrong"
2020-04-08 14:54:49 -07:00
Peter Rabbitson
635e333d60 Dag export command, silent, no progress 2020-04-08 14:54:49 -07:00
Hector Sanjuan
1b490476e5 HTTP API: Disallow GET requests on API
This commit upgrades go-ipfs-cmds and configures the commands HTTP API Handler
to only allow POST/OPTIONS, disallowing GET and others in the handling of
command requests in the IPFS HTTP API (where before every type of request
method was handled, with GET/POST/PUT/PATCH being equivalent).

The Read-Only commands that the HTTP API attaches to the gateway endpoint will
additional handled GET as they did before (but stop handling PUT,DELETEs).

By limiting the request types we address the possibility that a website
accessed by a browser abuses the IPFS API by issuing GET requests to it which
have no Origin or Referrer set, and are thus bypass CORS and CSRF protections.

This is a breaking change for clients that relay on GET requests against the
HTTP endpoint (usually :5001). Applications integrating on top of the
gateway-read-only API should still work (including cross-domain access).

Co-Authored-By: Steven Allen <steven@stebalien.com>
Co-Authored-By: Marcin Rataj <lidel@lidel.org>
2020-04-05 09:57:57 +02:00
Steven Allen
06ba6d0fc2 fix: remove internal relay discovery
This logic collects a list of known relays by testing every new connection.

It exists so we can dial /p2p-circuit/p2p/QmFoobar addresses (circuit addresses
that don't specify the relay). However, this kind of address is useless outside
of basic demos as a random relay is practically guaranteed to not be connected
to the target peer. Picking a random relay to connect to some peer is almost
_never_ the desired behavior.
2020-03-31 12:24:55 -07:00
Steven Allen
aec498bbde test(sharness): make sure we can actually enable autonat
This isn't a real autonat test, but it's a regression test to make sure we don't blatantly break this.
2020-03-25 12:48:25 -07:00
Cornelius Toole
62de9ba9d3 feat(file-ignore): add ignore opts to add cmd
- add the `gitignore` or `ignore options to the add command
2020-03-20 09:22:04 -07:00
Steven Allen
05fe3086a5 fix(sharness): fix sharness tests to take removed bootstrap peers into account 2020-03-18 08:50:40 -07:00
Marcin Rataj
f9567a0a0f fix(gateway): curl without redirect on localhost
When request is sent to http://localhost:8080/ipfs/$cid response has
HTTP 301 status code and "Location" header with redirect destination at
$cid.ipfs.localhost:8080

Redirect is followed by browsersi, but not by commandline tools.
Status 301 is ignored by curl in default mode: it will print response
and won't follow redirect, user needs to add -L for that.

To fix curl, we return correct payload in body of HTTP 301 response,
but set Clear-Site-Data header to ensure Origin sandbox can't be abused.

This requires a surgical workaround:
If Location header is present in ResponseWriter's Header map,
we ensure http.ServeContent() returns HTTP 301

Context: https://github.com/ipfs/go-ipfs/pull/6982

License: MIT
Signed-off-by: Marcin Rataj <lidel@lidel.org>
2020-03-18 08:50:40 -07:00
Marcin Rataj
3ecccd6e1d feat(gateway): subdomain and proxy gateway
License: MIT
Signed-off-by: Marcin Rataj <lidel@lidel.org>
2020-03-18 08:50:38 -07:00
Steven Allen
97401b9e27
Merge pull request #6997 from ipfs/chore/sharness_fixes_2019-03-16
Chore/sharness fixes 2019 03 16
2020-03-17 14:33:22 -07:00
Peter Rabbitson
705bee7d3f Support pipes when named on the cli explicitly
Still throws an error when a pipe is encountered during directory recursion
2020-03-16 17:25:33 -07:00
Peter Rabbitson
51d4383d56 Abort individual sharness tests if a previous instance is present
Before starting each individual test check if a daemon is present with a
trash* directory as its workdir. Exit with failure if this is the case.
2020-03-15 19:57:29 +00:00
Steven Allen
e681a3aebe fix: revert changes to the user agent
Go-ipfs user agents usually look like go-ipfs/VERSION/COMMIT_or_EMPTY. However,
we changed this to /go-ipfs/VERSION(/COMMIT)? on master for a while. This change
reverts this to make parsing the user agent simpler.

Co-Authored-By: Oli Evans <oli@tableflip.io>
2020-03-13 13:50:11 -07:00
Peter Rabbitson
7b65c0703f Simplify grep in test to pass on MacOS 2020-03-12 04:42:53 +01:00
Steven Allen
a68c1af938 ci: update to go 1.14 2020-03-08 22:00:26 -07:00
Steven Allen
585731074e test(graphsync): test server-side graphsync 2020-02-26 09:11:17 -08:00
Steven Allen
6e8c25eaee test(sharness): test our tests
Make sure they:

1. Report that they're done. Otherwise, we'll silently succeed.
2. Have a description.
3. Make sure we cleanup IPFS.
2020-02-18 15:14:45 -05:00
Steven Allen
3385367bdc test(sharness): fix typo 2020-01-17 10:27:36 -08:00
Steven Allen
e58a32ab1e feat(commands/cid): add a flag for setting the codec
This will be useful when testing `refs local, `repo gc`, and `repo verify`
commands once we store blocks by multihash instead of by CID. At that point,
these commands will return raw v1 CIDs as the blockstore won't actually remember
the codec used to store the block.

Flags choice:

* Ideally, we'd use the `-f, --format` flags like every other command but we're
  already using `-f` (format) for the format string.
* Alternatively, I'd like to use `-c`. However, we're using _that_ for a global
  `--config` flag (bit of a waste given that it doesn't work...).

`--codec` will have to do for now.
2020-01-07 19:32:16 -06:00
Steven Allen
f8adc3ebdd fix(commands/cid): always use base58 for CIDv0 unless otherwise specified 2020-01-07 19:20:21 -06:00
Steven Allen
9a9ec021cf test(sharness): add gateway symlink test 2020-01-05 16:43:44 -08:00
flowed
07cbba7c40 Fix Typos 2019-12-25 03:12:41 +00:00
Peter Rabbitson
36dcf8eac5 Add two more missed cleanup clauses 2019-12-23 15:41:39 +01:00
Peter Rabbitson
4200bf453b At some point the directory name changed to testbeds
Without reflecting this in the tests we do not actually shut down
correctly during cleanup
2019-12-23 15:41:39 +01:00
Peter Rabbitson
52476c026c Remove couple linux-isms: these are not valid on macos
Some did actually prevent proper test shutdown
2019-12-23 15:41:39 +01:00
Peter Rabbitson
8c6d8d5bd1 Add GOIPFSTEST_SKIP_LOCAL_DEVTREE_DEPS_CHECK escape valve
Set this to '1' if you are doing local dev and have redirects in go.mod
2019-12-23 15:41:30 +01:00
Peter Rabbitson
153e52929a Make the docker check more specific
Docker on a mac might be available, but may not be runnng. The
way the original test was written is linux-specific anyway, so
just hoist the check into the DOCKER declaration
2019-12-23 15:41:30 +01:00
Peter Rabbitson
ba3a7d4a55 socat is not part of any default install, make tests conditional
54b1969c4b correctly switched from nc to socat, but did not account for it
not being as widely available. Adjust tests that depend on it with the
correct message.

There are also a number of comments pertaining to nc - they were not
adjusted.
2019-12-23 15:41:30 +01:00
Marcin Rataj
115b2ba6cd
fix: limit SW registration to content root
Introduces hardening proposed in:
https://github.com/ipfs/go-ipfs/issues/4025#issuecomment-342250616

License: MIT
Signed-off-by: Marcin Rataj <lidel@lidel.org>
2019-12-17 02:20:40 +01:00
Steven Allen
ff8e4fc89f test(sharness): fix tests for /ipfs -> /p2p change 2019-12-14 19:41:05 +01:00
Steven Allen
ab274664df test(sharness): use test profile
Otherwise, we have port conflicts and other issues.
2019-12-05 18:37:45 -05:00
Steven Allen
f1b28dab95 test(sharness): simplify init-config logic 2019-12-05 18:32:06 -05:00
Adin Schmahmann
ee42e34ee4 sharness(pubsub): turn off the DHT for pubsub tests 2019-12-05 13:26:46 -05:00