Improves the way we handle If-None-Match header:
- Support for more than one Etag passed in If-None-Match
- Match both strong and weak Etags to maximize caching across
various HTTP clients and libraries (some send weak Etags by default)
- Support for wildcard '*'
- Tests for If-None-Match behavior
* Extract functions from getOrHeadHandler to improve readability and prepare for later refactorings
* Address PR feedback on when to return errors or booleans
* Be explicit about use of *requestError vs error
* fix: report gateway http metrics only when response is successful
* fix(gw): 304 Not Modified as no-op
This fix ensures we don't do any additional work when Etag match
what user already has in their own cache.
Co-authored-by: Marcin Rataj <lidel@lidel.org>
* add deprecation warning when tracer plugins are loaded
* add response format attribute to span in gateway handler
* add note about tracing's experimental status in godoc
* add nil check for TTL when adding name span attrs
* add basic sharness test for integration with otel collector
* add nil check in UnixFSAPI.processLink
* test: sharness check all json objs for swarm span
* add env var docs to docs/environment-variables.md
* chore: pin the otel collector version
* add tracing spans per response type (#8841)
* docs: tracing with jaeger-ui
Co-authored-by: Marcin Rataj <lidel@lidel.org>
* fix(gw): validate requested CAR version
This adds validation of 'application/vnd.ipld.car;version=n' passed
in the Accept header by HTTP clients to align Gateway behavior with
the spec submitted to IANA.
* test: fix comment in test/sharness/t0118-gateway-car.sh
Co-authored-by: Gus Eggert <gus@gus.dev>
Co-authored-by: Gus Eggert <gus@gus.dev>
* feat(gw): response type histogram metrics
- response-type agnostic firstContentBlockGetMetric which counts the
latency til the first content block.
- car/block/file/gen-dir-index duration histogram metrics that show how
long each response type takes
* docs: improve metrics descriptions
* feat: more gw histogram buckets
0.05, 0.1, 0.25, 0.5, 1, 2, 5, 10, 30, 60 secs
as suggested in reviews at https://github.com/ipfs/go-ipfs/pull/8443
Co-authored-by: Marcin Rataj <lidel@lidel.org>
Co-authored-by: Gus Eggert <gus@gus.dev>
* feat: serveRawBlock implements ?format=block
* feat: serveCar implements ?format=car
* feat(gw): ?format= or Accept HTTP header
- extracted file-like content type responses to separate .go files
- Accept HTTP header with support for application/vnd.ipld.* types
* fix: use .bin for raw block content-disposition
.raw may be handled by something, depending on OS, and .bin
seems to be universally "binary file" across all systems:
https://en.wikipedia.org/wiki/List_of_filename_extensions_(A%E2%80%93E)
* refactor: gateway_handler_unixfs.go
- Moved UnixFS response handling to gateway_handler_unixfs*.go files.
- Removed support for X-Ipfs-Gateway-Prefix (Closes#7702)
* refactor: prefix cleanup and readable paths
- removed dead code after X-Ipfs-Gateway-Prefix is gone
(https://github.com/ipfs/go-ipfs/issues/7702)
- escaped special characters in content paths returned with http.Error
making them both safer and easier to reason about (e.g. when invisible
whitespace Unicode is used)
* chore(gateway): better logging for the http requests
* chore(gateway): removed defer and add more data to the final log
* chore(gateway): debug logging refactor
* chore(gateway): use debug w/o context when only msg
* doc: add cmd for log level
* chore: add more logs and address fedback
* chore(gateway): log subdomains and from=requestURI, refactor
* chore(gateway): fix debug redirect
I believe we figured that these were for "informational purposes", but
really, we _should_ always be able to resolve names to CIDs. If we
can't, there's probably something wrong with the directory.
This adds proper end-to-end tests for directory listing on Gateway port
that protects us against regressions oni each gw type:
- path gateway
- subdomain gateway
- dnslink website gateway
Tests cover:
- etag/unicode support
- breadcrumbs
- file name column
- hash column
This implements 'attachment' mode triggered then
?filename parameter is accompanied with &download=true
When Content-Disposition: attachment is detected by a modern browser
it will skip rendering and immediately open the "save as" dialog,
making this useful feature for using IPFS gateway as target of
"Download" links on various websites.
Parameter name was suggested in:
https://github.com/ipfs/go-ipfs/pull/4177#issuecomment-414870327
These changes are needed to prepare for the Directory page UI improvements
implemented in https://github.com/ipfs/dir-index-html/issues/37.
- update dir-index-html type structs
- emit gateway URL for root links
- emit CID of each directoryItem
- emit size of directory
- emit breadcrumbs
While the content of raw files retrieved via the gateway should never
change, the look and feel of the directory index can and will change
between versions of go-ipfs.
Incorporate the hash of assets/bindata.go into the ETag when appropriate
fixes https://github.com/ipfs/go-ipfs/pull/4233#issuecomment-631454543
Basically, there's a trade-off here:
1. We can support directory listings while supporting 404 pages (this PR).
2. If a 404 page is present, directory listings don't work.
Given that option 1 is more flexible and users shouldn't be _too_ confused if
they land on a directory with no index.html page, I've gone with that option.
Files already have an explicit Content-Type set. Be sure to do this for
directory listings as well to avoid a fallback to autodetection in
net/http. That fallback fails when a ResponseWriter is installed that
performs compression.
In the same way that an `index.html` file is rendered, if one is present, when the
requested path is a directory, now an `ipfs-404.html` file is rendered if
the requested file is not present within the specified IPFS object.
`ipfs-404.html` files are looked for in the directory of the requested path and each
parent until one is found, falling back on the well-known 404 error message.
License: MIT
Signed-off-by: JP Hastings-Spital <jphastings@gmail.com>
