feat: HTTP retrieval enabled by default (#10836)

config/http_retrieval.go

@@ -12,7 +12,7 @@ type HTTPRetrieval struct {
 }
 
 const (
-	DefaultHTTPRetrievalEnabled               = false // opt-in for now, until we figure out https://github.com/ipfs/specs/issues/496
+	DefaultHTTPRetrievalEnabled               = true
 	DefaultHTTPRetrievalNumWorkers            = 16
 	DefaultHTTPRetrievalTLSInsecureSkipVerify = false  // only for testing with self-signed HTTPS certs
 	DefaultHTTPRetrievalMaxBlockSize          = "2MiB" // matching bitswap: https://specs.ipfs.tech/bitswap-protocol/#block-sizes

docs/changelogs/v0.36.md

@@ -11,6 +11,7 @@ This release  was brought to you by the [Shipyard](http://ipshipyard.com/) team.
 
 - [Overview](#overview)
 - [🔦 Highlights](#-highlights)
+  - [HTTP Retrieval client enabled by default](#http-retrieval-client-enabled-by-default)
   - [Update go-log to v2](#update-go-log-to-v2)
   - [AutoNATv2 Client](#autonatv2-client)
     - [Smarter AutoTLS registration](#smarter-autotls-registration)
@@ -24,6 +25,12 @@ This release  was brought to you by the [Shipyard](http://ipshipyard.com/) team.
 
 ### 🔦 Highlights
 
+#### HTTP Retrieval client enabled by default
+
+This release promotes the HTTP Retrieval client from an experimental feature to a standard feature that is enabled by default. When possible, Kubo will be retrieving blocks over plain HTTPS (HTTP/2) without any extra user configuration.
+
+See [`HTTPRetrieval`](https://github.com/ipfs/kubo/blob/master/docs/config.md#httpretrieval) for more details.
+
 #### Update go-log to v2
 
 go-log v2 has been out for quite a while now and it is time to deprecate v1. 

docs/config.md

@@ -2686,25 +2686,26 @@ Type: `object`
 
 ### `HTTPRetrieval.Enabled`
 
-> [!CAUTION]
-> This feature is **EXPERIMENTAL** and may change in future release. Enable with caution, and provide feedback via GitHub issues.
-
 Controls whether HTTP-based block retrieval is enabled.
 
-When enabled, Kubo will be able to act on `/tls/http` (HTTP/2) providers ([Trustless HTTP Gateways](https://specs.ipfs.tech/http-gateways/trustless-gateway/)) returned by the [`Routing.DelegatedRouters`](#routingdelegatedrouters)
+When enabled, Kubo will act on `/tls/http` (HTTP/2) providers ([Trustless HTTP Gateways](https://specs.ipfs.tech/http-gateways/trustless-gateway/)) returned by the [`Routing.DelegatedRouters`](#routingdelegatedrouters)
 to perform pure HTTP [block retrievals](https://specs.ipfs.tech/http-gateways/trustless-gateway/#block-responses-application-vnd-ipld-raw)
-in addition to [Bitswap over Libp2p](#bitswap).
+(`/ipfs/cid?format=raw`, `Accept: application/vnd.ipld.raw`)
+alongside [Bitswap over Libp2p](#bitswap).
 
-HTTP requests for `application/vnd.ipld.raw` will be issued instead of Bitswap if a peer has a `/tls/http` multiaddr
+HTTP requests for `application/vnd.ipld.raw` will be made instead of Bitswap when a peer has a `/tls/http` multiaddr
 and the HTTPS server returns HTTP 200 for the [probe path](https://specs.ipfs.tech/http-gateways/trustless-gateway/#dedicated-probe-paths).
 
 > [!IMPORTANT]
-> - Requires TLS and HTTP/2.
+> This feature is relatively new. Please report any issues via [Github](https://github.com/ipfs/kubo/issues/new).
+>
+> Important notes:
+> - TLS and HTTP/2 are required. For privacy reasons, and to maintain feature-parity with browsers, unencrypted `http://` providers are ignored and not used.
 > - This feature works in the same way as Bitswap: connected HTTP-peers receive optimistic block requests even for content that they are not announcing.
-> - HTTP client does not follow redirects. Providers should keep announcements up to date.
+> - For performance reasons, and to avoid loops, the HTTP client does not follow redirects. Providers should keep announcements up to date.
 > - IPFS ecosystem is working towards [supporting HTTP providers on Amino DHT](https://github.com/ipfs/specs/issues/496). Currently, HTTP providers are mostly limited to results from [`Routing.DelegatedRouters`](#routingdelegatedrouters) endpoints and requires `Routing.Type=auto|autoclient`.
 
-Default: `false`
+Default: `true`
 
 Type: `flag`
 

test/sharness/t0119-prometheus-data/prometheus_metrics

@@ -7,6 +7,22 @@ exchange_bitswap_wantlists_seconds_bucket
 exchange_bitswap_wantlists_seconds_count
 exchange_bitswap_wantlists_seconds_sum
 exchange_bitswap_wantlists_total
+exchange_httpnet_request_duration_seconds_bucket
+exchange_httpnet_request_duration_seconds_count
+exchange_httpnet_request_duration_seconds_sum
+exchange_httpnet_request_sent_bytes
+exchange_httpnet_requests_body_failure
+exchange_httpnet_requests_failure
+exchange_httpnet_requests_in_flight
+exchange_httpnet_requests_total
+exchange_httpnet_response_bytes_bucket
+exchange_httpnet_response_bytes_count
+exchange_httpnet_response_bytes_sum
+exchange_httpnet_wantlists_items_total
+exchange_httpnet_wantlists_seconds_bucket
+exchange_httpnet_wantlists_seconds_count
+exchange_httpnet_wantlists_seconds_sum
+exchange_httpnet_wantlists_total
 go_gc_duration_seconds
 go_gc_duration_seconds_count
 go_gc_duration_seconds_sum