* chore: disable AutoTLS in TCP-only transport tests
Tests were failing intermittently. Disabling AutoTLS when WebSocket
transport is disabled appears to resolve the issue.
* fix: make NewRandPort thread-safe
Track allocated ports globally to prevent conflicts
when tests run in parallel.
test was expecting immediate GC lock acquisition after pipe close,
but timing wasn't guaranteed. replaced blocking wait with 5-second
timeout to handle timing variations while still detecting deadlocks.
* Initial pass at Telemetry plugin
Currently, IP Shipyard, with the help of Probelab, monitor and extract
Amino/IPFS public network metrics with the use of DHT crawlers and
bootstrappers (via peerlog plugin). For example, we log all peer IDs seen and
their AgentVersion/Addresses obtained from the `identify` protocol, which
provides insights into protocol usage, total number of peers etc.
We would like to increase the ability to obtain more insights from the network
by collecting some more information in the future, but also to give users more
control over this collection (i.e. opt-out). The information collected will
not allow unique identification of anyone and is only used for aggregation.
Now, this PR explores a way of moving in this direction:
* A new "telemetry" fx plugin is in charge of dealing with telemetry
* The FX plugin allows to plug and make decisions / take actions during the setup phase:
* We can inspect whether we are using Private Networks before the libp2p.Host has been initialized.
* We can send telemetry after the libp2p Host is initialized.
* Everything is self-contained. Custom builds can remove the plugin altogether without needing to surgically edit the code.
As for behaviour:
* The user can opt-in/out via EnvVar, file in the repo path or plugin configuration.
* Users on private networks or with custom bootstrappers are detected, offered a wall of text explaining why we need telemetry and invited to opt-in. Opt-out happens otherwise on a timeout (with no input). Their preferences are stored.
* Users on standard settings are opted-in by default. This is the status quo in Kubo already, except they don't get a chance to opt out.
The telemetry libp2p protocol is yet to be defined, but expect something similar to identify, with a protobuf being pushed to bootstrappers or to a specific telemetry node that we define. In the case of pnets, this will be done with a temporary peer.
* checkpoint
* telemetry plugin: second pass
* On first run it generates a UUID and shows a message to the user.
* UUID is persistend to "telemetry_uuid"
* Sends telemetry 1 minute after boot and every 24h
* LogEvent is the thing containing all the telemetry that is sent
* Opt-out possible via env-var or plugin configuration
* Telemetry: add changelog and environment variable documentation
* docs: improved daemon message
making it more obvious nothing was sent yet
and that user had 15m to out-out
plus some debug logs that confirm opt-out
* refactor: rename IPFS_TELEMETRY_MODE to IPFS_TELEMETRY
* fix: add User-Agent header to telemetry requests
---------
Co-authored-by: Andrew Gillis <11790789+gammazero@users.noreply.github.com>
Co-authored-by: Marcin Rataj <lidel@lidel.org>
The test was flaky because `ipfs repo verify` doesn't detect corruption
of the empty directory block (served from memory even when corrupted on
disk). Exclude both empty file and empty directory blocks from random
corruption to make the test reliable.
---------
Co-authored-by: Andrew Gillis <11790789+gammazero@users.noreply.github.com>
* feat(gateway): concurrency and timeout limits
Depends on https://github.com/ipfs/boxo/pull/994
* chore: boxo master with final boxo#994
this includes race-condition fixes from ipfs/boxo#994
and increased `DefaultMaxConcurrentRequests = 4096`
* docs: concise config.md and changelog
Dependabot couldn't find the original pull request head commit, 0292612bd5e16ed34f2974225fe170b15d4eb821.
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* fix(ci): pass -buildvcs=false to go list in Makefile
Docker builds were failing because go list commands in mk/golang.mk
didn't inherit GOFLAGS from the environment, causing empty package
names when .git directory was incomplete.
This issue always existed but was masked until recent changes in
GitHub Actions runners or Docker golang image exposed it.
* fix(ci): use modern ENV key=value format in Dockerfile
Fixes Docker build deprecation warnings.
* feat(ipns): Add a parameter in name.publish to change the sequence number
* test: monotonic name publish --sequence
* docs: `name publish --sequence`
* chore: boxo main with PR 962
---------
Co-authored-by: Marcin Rataj <lidel@lidel.org>
* fix(relay): feed connected peers to AutoRelay discovery
Feed all connected swarm peers to AutoRelay as potential relay
candidates. This allows peers from HTTP routing and manual connections
to serve as relays, not just DHT-discovered peers.
Fixes#10899
* docs: changelog
this is rare, but if someone runs a lot of tests and had some unrelated
FUSE failures, sharness can get stuck on lsof step.
this ensures we never get stuck + there is more visibility so people
who care about FUSE have better time debugging
* feat: update log level command to show log levels
* test: add log level tests
* update TestCommands test
* docs: relation to GOLOG_LOG_LEVEL
* chore: update to latest go-log
* fix: do not output single subsystem name in CLI
* test: explicit subsystem request dont output subsystem
* LevelFromString renamed to Parse
* Modify `ipfs log level`
* Denote default level with sdubsystem name '(defult)'.
* make "*" an dalias for "all". Test to make sure both work the same.
Test was failing becuase path passed to `path.NewPath` (from boxo) did not include a required namespace. Prepending the namespace to the path with "/ipfs/" fixes this.
* Fix command stream result handling
Do not close the stream prematurely in a deferred function. Also, avoid possible shadowing errors.
Closes#9007
* test(cli): test/cli/cid_test.go
includes regression tests for https://github.com/ipfs/kubo/issues/9007
and better coverage than the old test/sharness/t0290-cid.sh
(identified bug in basemoji)
* fix: base256emoji in cid bases --prefix
changed the character display logic from ASCII range check to
unicode.IsPrint() to properly handle Unicode characters including
emojis.
* docs: changelog
---------
Co-authored-by: Marcin Rataj <lidel@lidel.org>
* fix: `ipfs cid format` without repo
these commands should work without daemon or repo
but were missing SetDoesNotUseRepo(true)
* test: test/cli/commands_without_repo_test.go
* Provide according to strategy
Updates boxo to a version with the changes from https://github.com/ipfs/boxo/pull/976, which decentralize the providing responsibilities (from a central providing.Exchange to blockstore, pinner, mfs).
The changes consist in initializing the Pinner, MFS and the blockstore with the provider.System, which is created first.
Since the provider.System is created first, the reproviding KeyChanFunc is set
later when we can create it once we have the Pinner, MFS and the blockstore.
Some additional work applies to the Add() workflow. Normally, blocks would get provided at the Blockstore or the Pinner, but when adding blocks AND a "pinned" strategy is used, the blockstore does not provide, and the
pinner does not traverse the DAG (and thus doesn't provide either), so we need to provide directly from the Adder. This is resolved by wrapping the DAGService in a "providingDAGService" which provides every added block, when using the "pinned" strategy.
`ipfs --offline add` when the ONLINE daemon is running will now announce blocks per the chosen strategy, where before it did not announce them. This is documented in the changelog. A couple of releases ago, adding with `ipfs --offline add` was faster, but this is no longer the case so we are not incurring in any penalties by sticking to the fact that the daemon is online and has a providing strategy that we follow.
Co-authored-by: gammazero <11790789+gammazero@users.noreply.github.com>
Co-authored-by: Marcin Rataj <lidel@lidel.org>
- Move `ipfs stat reprovide` to `ipfs provide stat`
- Mark `ipfs stat provide` as deprecated and replaces by `ipfs provide stat`
- Mark `ipfs stat reprovide` as deprecated and replaces by `ipfs provide stat`
- Remove redundant code from deprecated subcommands
Closes#10869
* feat(add): add support for naming pinned CID
Signed-off-by: kapil <kapilsareen584@gmail.com>
* fix(add): no double pinning and simplify pin-name
- modify PinRoot to accept name parameter, eliminating double pinning
- remove automatic filename fallback logic for cleaner behavior
- only create named pins when explicitly requested via --pin-name=value
- replace NoPinName constant with idiomatic empty string literals
- Update help text and tests to reflect explicit-only behavior
* docs: changelog
* chore: lint
* test: negative case for empty pin-name
* chore: gofmt
---------
Signed-off-by: kapil <kapilsareen584@gmail.com>
Co-authored-by: Marcin Rataj <lidel@lidel.org>
* refactor: remove goprocess
The `goprocess` package is no longer needed. It can be replaces by modern `context` and `context.AfterFunc`.
* mod tidy
* log unmount errors on shutdown
* Do not log non-mounted errors on shutdown
* Use WaitGroup associated with IPFS node to wait for services to whutdown
* Prefer explicit Close to context.ArterFunc
* Do not use node-level WaitGroup
* Unmount for non-supported platforms
* fix return values
* test: daemon shuts down gracefully
make sure ongoing operations dont block shutdown
* test(cli): add TestFUSE
* test: smarter RequiresFUSE
opportunistically run FUSE tests if env has fusermount
and TEST_FUSE was not explicitly set
* docs: changelog
---------
Co-authored-by: gammazero <gammazero@users.noreply.github.com>
Co-authored-by: Marcin Rataj <lidel@lidel.org>
No behaviour changes.
Currently we are using ProvideManyRouter for Bitswap, which is only meant to
use ContentDiscovery. This makes things more clear in that there is a
designated ContentDiscovery instance.
After boxo v0.33.1, this is a recommended step to fix http retrieval bugs.
Having a single ConnectEventManager prevents misdirected operations in the
network.Router to change the Connectedness state in a way that the counterpart
(httpnet or bsnet) can later correct.
* provider: clear reprovide queue when reprovide strategy changes
When the currently configured reprovide strategy does not match the previous strategy read from the datastore, then clear the reprovide queue and update the reprovide strategy that is stored in the datastore.
Depends on https://github.com/ipfs/boxo/pull/978Closes#10829
* Update docs/changelogs/v0.36.md
Co-authored-by: Guillaume Michel <guillaumemichel@users.noreply.github.com>
* update log message
* update boxo
* Move change log to v0.37.md
* Add `provide clear` command to clear provide queue
The `provide clear` command clears all items from the provide queue and prints out the number of items removed from the queue. The `quiet` option tells the command not to print output.
* refactor(cmds): ipfs provide clear
moving to new namespace to avoid conflicts, and also document other
commands
* docs: clarify Reprovider.Strategy
* chore: remove undesired md link
