From 787b29b884d3f08f332702ddfdd36aaadfb8b8fe Mon Sep 17 00:00:00 2001 From: Marcin Rataj Date: Tue, 16 Dec 2025 20:47:50 +0100 Subject: [PATCH] docs: add p2p-tunnels.md with systemd examples - add dedicated docs/p2p-tunnels.md covering: - why p2p tunnels (NAT traversal, no public IP needed) - quick start with netcat - background and foreground modes - systemd integration with path-based activation - security considerations and troubleshooting - document Experimental.Libp2pStreamMounting in docs/config.md - simplify docs/experimental-features.md, link to new doc - add "Learn more" links to ipfs p2p listen/forward --help - update changelog entry with doc link - add cross-reference in misc/README.md --- core/commands/p2p.go | 4 + docs/changelogs/v0.40.md | 2 +- docs/config.md | 12 ++ docs/experimental-features.md | 95 ++------------- docs/p2p-tunnels.md | 220 ++++++++++++++++++++++++++++++++++ misc/README.md | 6 + 6 files changed, 250 insertions(+), 89 deletions(-) create mode 100644 docs/p2p-tunnels.md diff --git a/core/commands/p2p.go b/core/commands/p2p.go index cc30cd198..1de0bfca3 100644 --- a/core/commands/p2p.go +++ b/core/commands/p2p.go @@ -120,6 +120,8 @@ EXAMPLES: # Temporary forwarder (removed when command exits) ipfs p2p forward -f /x/myapp /ip4/127.0.0.1/tcp/3000 /p2p/PeerID + +Learn more: https://github.com/ipfs/kubo/blob/master/docs/p2p-tunnels.md `, }, Arguments: []cmds.Argument{ @@ -290,6 +292,8 @@ EXAMPLES: # Report connecting peer ID to the target application ipfs p2p listen -r /x/myapp /ip4/127.0.0.1/tcp/3000 + +Learn more: https://github.com/ipfs/kubo/blob/master/docs/p2p-tunnels.md `, }, Arguments: []cmds.Argument{ diff --git a/docs/changelogs/v0.40.md b/docs/changelogs/v0.40.md index fe335932c..7f39c9798 100644 --- a/docs/changelogs/v0.40.md +++ b/docs/changelogs/v0.40.md @@ -40,7 +40,7 @@ The `ipfs p2p listen` and `ipfs p2p forward` commands now support a `--foregroun Without `--foreground`, the commands return immediately and the listener persists in the daemon (existing behavior). -Run `ipfs p2p listen --help` or `ipfs p2p forward --help` for details and examples. +Run `ipfs p2p listen --help` or `ipfs p2p forward --help` for usage, or see [docs/p2p-tunnels.md](https://github.com/ipfs/kubo/blob/master/docs/p2p-tunnels.md) for examples. #### 📦️ Dependency updates diff --git a/docs/config.md b/docs/config.md index 23386f7e6..15debaa64 100644 --- a/docs/config.md +++ b/docs/config.md @@ -59,6 +59,7 @@ config file at runtime. - [`Discovery.MDNS.Enabled`](#discoverymdnsenabled) - [`Discovery.MDNS.Interval`](#discoverymdnsinterval) - [`Experimental`](#experimental) + - [`Experimental.Libp2pStreamMounting`](#experimentallibp2pstreammounting) - [`Gateway`](#gateway) - [`Gateway.NoFetch`](#gatewaynofetch) - [`Gateway.NoDNSLink`](#gatewaynodnslink) @@ -1069,6 +1070,17 @@ in the [new mDNS implementation](https://github.com/libp2p/zeroconf#readme). Toggle and configure experimental features of Kubo. Experimental features are listed [here](./experimental-features.md). +### `Experimental.Libp2pStreamMounting` + +Enables the `ipfs p2p` commands for tunneling TCP connections through libp2p +streams, similar to SSH port forwarding. + +See [docs/p2p-tunnels.md](p2p-tunnels.md) for usage examples. + +Default: `false` + +Type: `bool` + ## `Gateway` Options for the HTTP gateway. diff --git a/docs/experimental-features.md b/docs/experimental-features.md index 228cf5abf..e67225659 100644 --- a/docs/experimental-features.md +++ b/docs/experimental-features.md @@ -199,9 +199,8 @@ configured, the daemon will fail to start. ## ipfs p2p -Allows tunneling of TCP connections through Libp2p streams. If you've ever used -port forwarding with SSH (the `-L` option in OpenSSH), this feature is quite -similar. +Allows tunneling of TCP connections through libp2p streams, similar to SSH port +forwarding (`ssh -L`). ### State @@ -220,100 +219,20 @@ Experimental, will be stabilized in 0.6.0 > If you enable this and plan to expose CLI or HTTP RPC to other users or machines, > secure RPC API using [`API.Authorizations`](https://github.com/ipfs/kubo/blob/master/docs/config.md#apiauthorizations) or custom auth middleware. -The `p2p` command needs to be enabled in the config: - ```sh > ipfs config --json Experimental.Libp2pStreamMounting true ``` ### How to use -**Netcat example:** - -First, pick a protocol name for your application. Think of the protocol name as -a port number, just significantly more user-friendly. In this example, we're -going to use `/x/kickass/1.0`. - -***Setup:*** - -1. A "server" node with peer ID `$SERVER_ID` -2. A "client" node. - -***On the "server" node:*** - -First, start your application and have it listen for TCP connections on -port `$APP_PORT`. - -Then, configure the p2p listener by running: - -```sh -> ipfs p2p listen /x/kickass/1.0 /ip4/127.0.0.1/tcp/$APP_PORT -``` - -This will configure IPFS to forward all incoming `/x/kickass/1.0` streams to -`127.0.0.1:$APP_PORT` (opening a new connection to `127.0.0.1:$APP_PORT` per -incoming stream. - -***On the "client" node:*** - -First, configure the client p2p dialer, so that it forwards all inbound -connections on `127.0.0.1:SOME_PORT` to the server node listening -on `/x/kickass/1.0`. - -```sh -> ipfs p2p forward /x/kickass/1.0 /ip4/127.0.0.1/tcp/$SOME_PORT /p2p/$SERVER_ID -``` - -Next, have your application open a connection to `127.0.0.1:$SOME_PORT`. This -connection will be forwarded to the service running on `127.0.0.1:$APP_PORT` on -the remote machine. You can test it with netcat: - -***On "server" node:*** -```sh -> nc -v -l -p $APP_PORT -``` - -***On "client" node:*** -```sh -> nc -v 127.0.0.1 $SOME_PORT -``` - -You should now see that a connection has been established and be able to -exchange messages between netcat instances. - -(note that depending on your netcat version you may need to drop the `-v` flag) - -**SSH example** - -**Setup:** - -1. A "server" node with peer ID `$SERVER_ID` and running ssh server on the - default port. -2. A "client" node. - -_you can get `$SERVER_ID` by running `ipfs id -f "\n"`_ - -***First, on the "server" node:*** - -```sh -ipfs p2p listen /x/ssh /ip4/127.0.0.1/tcp/22 -``` - -***Then, on "client" node:*** - -```sh -ipfs p2p forward /x/ssh /ip4/127.0.0.1/tcp/2222 /p2p/$SERVER_ID -``` - -You should now be able to connect to your ssh server through a libp2p connection -with `ssh [user]@127.0.0.1 -p 2222`. - -> **Tip:** Both commands support `--foreground` (`-f`) flag for blocking behavior. -> Run `ipfs p2p listen --help` or `ipfs p2p forward --help` for details. +See [docs/p2p-tunnels.md](p2p-tunnels.md) for usage examples, foreground mode, +and systemd integration. ### Road to being a real feature -- [ ] More documentation +- [x] More documentation +- [x] `ipfs p2p forward` mode +- [ ] Ability to define tunnels via JSON config, similar to [`Peering.Peers`](https://github.com/ipfs/kubo/blob/master/docs/config.md#peeringpeers) ## p2p http proxy diff --git a/docs/p2p-tunnels.md b/docs/p2p-tunnels.md new file mode 100644 index 000000000..24a191ded --- /dev/null +++ b/docs/p2p-tunnels.md @@ -0,0 +1,220 @@ +# P2P Tunnels + +Kubo supports tunneling TCP connections through libp2p streams, similar to SSH +port forwarding (`ssh -L`). This allows exposing local services to remote peers +and forwarding remote services to local ports. + +- [Why P2P Tunnels?](#why-p2p-tunnels) +- [Quick Start](#quick-start) +- [Background Mode](#background-mode) +- [Foreground Mode](#foreground-mode) + - [systemd Integration](#systemd-integration) +- [Security Considerations](#security-considerations) +- [Troubleshooting](#troubleshooting) + +## Why P2P Tunnels? + +Unlike traditional SSH tunnels, libp2p-based tunnels do not require: + +- **No public IP or open ports**: The server does not need a static IP address + or port forwarding configured on the router. Connectivity to peers behind NAT + is facilitated by [Direct Connection Upgrade through Relay (DCUtR)](https://github.com/libp2p/specs/blob/master/relay/DCUtR.md), + which enables NAT hole-punching. + +- **No DNS or IP address management**: All you need is the server's PeerID and + an agreed-upon protocol name (e.g., `/x/ssh`). Kubo handles peer discovery + and routing via the [Amino DHT](https://specs.ipfs.tech/routing/kad-dht/). + +- **Simplified firewall rules**: Since connections are established through + libp2p's existing swarm connections, no additional firewall configuration is + needed beyond what Kubo already requires. + +This makes p2p tunnels useful for connecting to machines on home networks, +behind corporate firewalls, or in environments where traditional port forwarding +is not available. + +## Quick Start + +Enable the experimental feature: + +```console +$ ipfs config --json Experimental.Libp2pStreamMounting true +``` + +Test with netcat (`nc`) - no services required: + +**On the server:** + +```console +$ ipfs p2p listen /x/test /ip4/127.0.0.1/tcp/9999 +$ nc -l -p 9999 +``` + +**On the client:** + +Replace `$SERVER_ID` with the server's peer ID (get it with `ipfs id -f "\n"` +on the server). + +```console +$ ipfs p2p forward /x/test /ip4/127.0.0.1/tcp/9998 /p2p/$SERVER_ID +$ nc 127.0.0.1 9998 +``` + +Type in either terminal and the text appears in the other. Use Ctrl+C to exit. + +## Background Mode + +By default, `ipfs p2p listen` and `ipfs p2p forward` register the tunnel with +the daemon and return immediately. The tunnel persists until explicitly closed +with `ipfs p2p close` or the daemon shuts down. + +This example exposes a local SSH server (listening on `localhost:22`) to a +remote peer. The same pattern works for any TCP service. + +**On the server** (the machine running SSH): + +Register a p2p listener that forwards incoming connections to the local SSH +server. The protocol name `/x/ssh` is an arbitrary identifier that both peers +must agree on (the `/x/` prefix is required for custom protocols). + +```console +$ ipfs p2p listen /x/ssh /ip4/127.0.0.1/tcp/22 +``` + +**On the client:** + +Create a local port (`2222`) that tunnels through libp2p to the server's SSH +service. + +```console +$ ipfs p2p forward /x/ssh /ip4/127.0.0.1/tcp/2222 /p2p/$SERVER_ID +``` + +Now connect to SSH through the tunnel: + +```console +$ ssh user@127.0.0.1 -p 2222 +``` + +**Other services:** To tunnel a different service, change the port and protocol +name. For example, to expose a web server on port 8080, use `/x/mywebapp` and +`/ip4/127.0.0.1/tcp/8080`. + +## Foreground Mode + +Use `--foreground` (`-f`) to block until interrupted. The tunnel is +automatically removed when the command exits: + +```console +$ ipfs p2p listen /x/ssh /ip4/127.0.0.1/tcp/22 --foreground +Listening on /x/ssh, forwarding to /ip4/127.0.0.1/tcp/22, waiting for interrupt... +^C +Received interrupt, removing listener for /x/ssh +``` + +The listener/forwarder is automatically removed when: + +- The command receives Ctrl+C or SIGTERM +- `ipfs p2p close` is called +- The daemon shuts down + +This mode is useful for systemd services and scripts that need cleanup on exit. + +### systemd Integration + +The `--foreground` flag enables clean integration with systemd. The examples +below show how to run `ipfs p2p listen` as a user service that starts +automatically when the IPFS daemon is ready. + +Ensure IPFS daemon runs as a systemd user service. See +[misc/README.md](https://github.com/ipfs/kubo/blob/master/misc/README.md#systemd) +for setup instructions and where to place unit files. + +#### P2P listener with path-based activation + +Use a `.path` unit to wait for the daemon's RPC API to be ready before starting +the p2p listener. + +**`ipfs-p2p-tunnel.path`**: + +```systemd +[Unit] +Description=Monitor for IPFS daemon startup +After=ipfs.service +Requires=ipfs.service + +[Path] +PathExists=%h/.ipfs/api +Unit=ipfs-p2p-tunnel.service + +[Install] +WantedBy=default.target +``` + +The `%h` specifier expands to the user's home directory. If you use a custom +`IPFS_PATH`, adjust accordingly. + +**`ipfs-p2p-tunnel.service`**: + +```systemd +[Unit] +Description=IPFS p2p tunnel +Requires=ipfs.service + +[Service] +ExecStart=ipfs p2p listen /x/ssh /ip4/127.0.0.1/tcp/22 -f +Restart=on-failure +RestartSec=10 + +[Install] +WantedBy=default.target +``` + +#### Enabling the services + +```console +$ systemctl --user enable ipfs.service +$ systemctl --user enable ipfs-p2p-tunnel.path +$ systemctl --user start ipfs.service +``` + +The path unit monitors `~/.ipfs/api` and starts `ipfs-p2p-tunnel.service` +once the file exists. + +## Security Considerations + +> [!WARNING] +> This feature provides CLI and HTTP RPC users with the ability to set up port +> forwarding for localhost and LAN ports. If you enable this and plan to expose +> CLI or HTTP RPC to other users or machines, secure the RPC API using +> [`API.Authorizations`](https://github.com/ipfs/kubo/blob/master/docs/config.md#apiauthorizations) +> or custom auth middleware. + +## Troubleshooting + +### First launch fails but restarts work + +The `api` file may be created before the daemon is fully ready to handle p2p +commands. The service examples above use `Restart=on-failure` with +`RestartSec=10` to handle this transient condition. + +### Foreground listener stops when terminal closes + +When using `--foreground`, the listener stops if the terminal closes. For +persistent foreground listeners, use a systemd service, `nohup`, `tmux`, or +`screen`. Without `--foreground`, the listener persists in the daemon regardless +of terminal state. + +### Connection refused errors + +Verify: + +1. The experimental feature is enabled: `ipfs config Experimental.Libp2pStreamMounting` +2. The listener is active: `ipfs p2p ls` +3. Both peers can connect: `ipfs swarm connect /p2p/$PEER_ID` + +### Persistent tunnel configuration + +There is currently no way to define tunnels in the Kubo JSON config file. Use +`--foreground` mode with a systemd service for persistent tunnels. Support for +configuring tunnels via JSON config may be added in the future (PRs welcome!). diff --git a/misc/README.md b/misc/README.md index 28511d3fc..ea683519b 100644 --- a/misc/README.md +++ b/misc/README.md @@ -39,6 +39,12 @@ To run this in your user session, save it as `~/.config/systemd/user/ipfs.servic ``` Read more about `--user` services here: [wiki.archlinux.org:Systemd ](https://wiki.archlinux.org/index.php/Systemd/User#Automatic_start-up_of_systemd_user_instances) +#### P2P tunnel services + +For running `ipfs p2p listen` or `ipfs p2p forward` as systemd services, +see [docs/p2p-tunnels.md](../docs/p2p-tunnels.md) for examples using the +`--foreground` flag and path-based activation. + ### initd - Here is a full-featured sample service file: https://github.com/dylanPowers/ipfs-linux-service/blob/master/init.d/ipfs