Commit Graph

166 Commits

Author SHA1 Message Date
Marcin Rataj
b5078b005d
chore: remove deprecated go-ipfs Docker image publishing (#11081)
Some checks are pending
CodeQL / codeql (push) Waiting to run
Docker Check / lint (push) Waiting to run
Docker Check / build (push) Waiting to run
Gateway Conformance / gateway-conformance (push) Waiting to run
Gateway Conformance / gateway-conformance-libp2p-experiment (push) Waiting to run
Go Build / go-build (push) Waiting to run
Go Check / go-check (push) Waiting to run
Go Lint / go-lint (push) Waiting to run
Go Test / go-test (push) Waiting to run
Interop / interop-prep (push) Waiting to run
Interop / helia-interop (push) Blocked by required conditions
Interop / ipfs-webui (push) Blocked by required conditions
Sharness / sharness-test (push) Waiting to run
Spell Check / spellcheck (push) Waiting to run
stop publishing to ipfs/go-ipfs entirely - the deprecation stub
introduced in v0.39 is no longer needed. only ipfs/kubo is published.

- remove legacy-name job from docker-image.yml workflow
- remove .github/legacy/ (Dockerfile.goipfs-stub, goipfs_stub.sh)
- update bin scripts to use ipfs/kubo as default
2025-11-27 17:05:26 +01:00
Marcin Rataj
31ea50efbf
docs: mkreleaselog for 0.39 2025-11-27 02:25:39 +01:00
Marcin Rataj
e0e6cacc49 bin/mkreleaselog: add github handle resolution and deduplication
- convert from zsh to bash for portability and shellcheck support
- resolve GitHub handles via multiple methods:
  - noreply email pattern (user@users.noreply.github.com)
  - merge commit messages (Merge pull request #N from user/branch)
  - gh CLI API for PR authors (squash merge commits)
  - gh CLI API for commit authors (fallback for non-PR commits)
- deduplicate contributors by GitHub handle instead of author name
- cache resolved mappings in ~/.cache/mkreleaselog/github-handles.json
- output clickable GitHub profile links in contributor table
2025-11-27 02:06:51 +01:00
filipremb
78a2f2cf24 fix(docker): inlude symlinks in scanning for init scripts (#11077)
(cherry picked from commit 72f4c6f029)
2025-11-26 21:06:23 +01:00
Marcin Rataj
f4834e797d
fix: migrations for Windows (#11010)
Some checks failed
CodeQL / codeql (push) Waiting to run
Docker Check / lint (push) Waiting to run
Docker Check / build (push) Waiting to run
Gateway Conformance / gateway-conformance (push) Waiting to run
Gateway Conformance / gateway-conformance-libp2p-experiment (push) Waiting to run
Go Build / go-build (push) Waiting to run
Go Check / go-check (push) Waiting to run
Go Lint / go-lint (push) Waiting to run
Go Test / go-test (push) Waiting to run
Interop / interop-prep (push) Waiting to run
Interop / helia-interop (push) Blocked by required conditions
Interop / ipfs-webui (push) Blocked by required conditions
Sharness / sharness-test (push) Waiting to run
Spell Check / spellcheck (push) Waiting to run
Migrations / test (macos-latest) (push) Has been cancelled
Migrations / test (ubuntu-latest) (push) Has been cancelled
Migrations / test (windows-latest) (push) Has been cancelled
* test: add migration tests for Windows and macOS

- add dedicated CI workflow for migration tests on Windows/macOS
- workflow triggers on migration-related file changes only

* build: remove redundant go version checks

- remove GO_MIN_VERSION and check_go_version scripts
- go.mod already enforces minimum version (go 1.25)
- fixes make build on Windows

* fix: windows migration panic by reading config into memory

fixes migration panic on Windows when upgrading from v0.37 to v0.38
by reading the entire config file into memory before performing atomic
operations. this avoids file locking issues on Windows where open files
cannot be renamed.

also fixes:
- TestRepoDir to set USERPROFILE on Windows (not just HOME)
- CLI migration tests to sanitize directory names (remove colons)

minimal fix that solves the "panic: error can't be dealt with
transactionally: Access is denied" error without adding unnecessary
platform-specific complexity.

* fix: set PATH for CLI migration tests in CI

the CLI tests need the built ipfs binary to be in PATH

* fix: use ipfs shutdown for graceful daemon termination in tests

replaces platform-specific signal handling with ipfs shutdown command
which works consistently across all platforms including Windows

* fix: isolate PATH modifications in parallel migration tests

tests running in parallel with t.Parallel() were interfering with each
other through global PATH modifications via os.Setenv(). this caused
tests to download real migration binaries instead of using mocks,
leading to Windows failures due to path separator issues in external tools.

now each test builds its own custom PATH and passes it explicitly to
commands, preventing interference between parallel tests.

* chore: improve error messages in WithBackup

* fix: Windows CI migration test failures

- add .exe extension to mock migration binaries on Windows
- handle repo lock file properly in mock migration binary
- ensure lock is created and removed to prevent conflicts

* refactor: align atomicfile error handling with fs-repo-migrations

- check close error in Abort() before attempting removal
- leave temp file on rename failure for debugging (like fs-repo-15-to-16)
- improves consistency with external migration implementations

* fix: use req.Context in repo migrate to avoid double-lock

The repo migrate command was calling cctx.Context() which has a hidden
side effect: it lazily constructs the IPFS node by calling GetNode(),
which opens the repository and acquires repo.lock. When migrations then
tried to acquire the same lock, it failed with "lock is already held by us"
because go4.org/lock tracks locks per-process in a global map.

The fix uses req.Context instead, which is a plain context.Context with
no side effects. This provides what migrations need (cancellation handling)
without triggering node construction or repo opening.

Context types explained:
- req.Context: Standard Go context for request lifetime, cancellation,
  and timeouts. No side effects.
- cctx.Context(): Kubo-specific method that lazily constructs the full
  IPFS node (opens repo, acquires lock, initializes subsystems). Returns
  the node's internal context.

Why req.Context is correct here:
- Migrations work on raw filesystem (only need ConfigRoot path)
- Command has SetDoesNotUseRepo(true) - doesn't need running node
- Migrations handle their own locking via lockfile.Lock()
- Need cancellation support but not node lifecycle

The bug only appeared with embedded migrations (v16+) because they run
in-process. External migrations (pre-v16) were separate processes, so
each had isolated state. Sequential migrations (forward then backward)
in the same process exposed this latent double-lock issue.

Also adds repo.lock acquisition to RunEmbeddedMigrations to prevent
concurrent migration access, and removes the now-unnecessary daemon
lock check from the migrate command handler.

* fix: use req.Context for migrations and autoconf in daemon startup

daemon.go was incorrectly using cctx.Context() in two critical places:

1. Line 337: migrations call - cctx.Context() triggers GetNode() which
   opens the repo and acquires repo.lock BEFORE migrations run, causing
   "lock is already held by us" errors when migrations try to lock

2. Line 390: autoconf client.Start() - uses context for HTTP timeouts
   and background updater lifecycle, doesn't need node construction

Both now use req.Context (plain Go context) which provides:
- request lifetime and cancellation
- no side effects (doesn't construct node or open repo)
- correct lifecycle for HTTP requests and background goroutines
2025-10-08 18:02:04 +02:00
Marcin Rataj
35ba5091a5
feat: add docker stub for deprecated ipfs/go-ipfs name (#10998)
* feat: add docker stub for deprecated ipfs/go-ipfs name

implements docker part of #10941 by creating a stub image that redirects
users from ipfs/go-ipfs to ipfs/kubo

changes:
- add stub dockerfile and script in .github/legacy/
- modify docker-image.yml to push stub to ipfs/go-ipfs with same tags as ipfs/kubo
- remove ipfs/go-ipfs from get-docker-tags.sh to prevent docker-hub job from pushing to legacy name
- stub displays clear deprecation message directing users to ipfs/kubo:release

* docs: add v0.39 changelog highlight for go-ipfs deprecation
2025-10-02 20:08:24 +02:00
Marcin Rataj
9faefe316f
refactor(ci): optimize build workflows (#10973)
Some checks are pending
CodeQL / codeql (push) Waiting to run
Docker Check / lint (push) Waiting to run
Docker Check / build (push) Waiting to run
Gateway Conformance / gateway-conformance (push) Waiting to run
Gateway Conformance / gateway-conformance-libp2p-experiment (push) Waiting to run
Go Build / go-build (push) Waiting to run
Go Check / go-check (push) Waiting to run
Go Lint / go-lint (push) Waiting to run
Go Test / go-test (push) Waiting to run
Interop / interop-prep (push) Waiting to run
Interop / helia-interop (push) Blocked by required conditions
Interop / ipfs-webui (push) Blocked by required conditions
Sharness / sharness-test (push) Waiting to run
Spell Check / spellcheck (push) Waiting to run
* ci: optimize build workflows

- use go version from go.mod instead of hardcoding
- group platforms by OS for parallel builds
- remove legacy try-build targets

* fix: checkout before setup-go in all workflows

setup-go needs go.mod to be present, so checkout must happen first

* chore: remove deprecated // +build syntax

go 1.17+ uses //go:build, the old syntax is no longer needed

* simplify: remove nofuse tag from CI workflows

- workflows now rely on platform build constraints
- keep make nofuse target for manual builds
- remove unused appveyor.yml

* ci: remove legacy travis variable and fix gateway-conformance

- remove TRAVIS env variable from 4 workflows
- fix gateway-conformance checkout path to match working-directory
- replace deprecated cache-go-action with built-in setup-go caching
2025-09-19 14:44:38 +02:00
Marcin Rataj
67f7129b3c chore: update v0.37.0 changelog
- updated changelog and contributors with mkreleaselog output
- improved mkreleaselog to work with kubo from any directory
2025-08-27 19:15:05 +02:00
Marcin Rataj
550f464263
chore(ci): build docker images for staging branch 2024-10-21 15:58:34 +02:00
Henrique Dias
9047fed8d5
core/commands!: remove deprecated object APIs (#10375) 2024-03-22 09:32:30 +01:00
Piotr Galar
c5868a86be
refactor(ci): simplify Dockerfile and add docker image testing (#10021) 2023-07-27 19:53:24 +02:00
Piotr Galar
bf7d0fc99d
Merge pull request #9484 from ipfs/process-improvement-v0.18.0
Process Improvement: v0.18.0
2023-03-27 10:50:59 +02:00
galargh
ceed43ea0e chore: clean up after circleci removal 2023-03-21 14:57:14 +01:00
galargh
c1e1d5e040 chore: update after v0.18.1 2023-01-31 10:30:26 +01:00
galargh
fa21445d67 docs: update mkreleaselog output 2022-12-05 09:50:41 +01:00
Steven Allen
d2c49278f6 fix: pass the repo directory into the ignored_commit function
fixes #9012
2022-09-01 17:17:22 +02:00
Marcin Rataj
b9240fe432 refactor(docker): publish images under both names
This ensures we keep publishing the same image under both names:

    $ ./bin/get-docker-tags.sh $(date -u +%F) 88d88158c master                                                                                                                                 ...chore/rename-to-banana
    ipfs/kubo:master-2022-06-21-88d8815
    ipfs/go-ipfs:master-2022-06-21-88d8815
    ipfs/kubo:master-latest
    ipfs/go-ipfs:master-latest
2022-07-06 18:40:39 +02:00
Adin Schmahmann
00c4a0c120
Merge pull request #8941 from ipfs/docs/0.13.0-changelog
docs: v0.13.0 changelog
2022-05-04 13:10:39 -07:00
Caian Benedicto
63b0025664
feat(docker): /container-init.d for advanced initialization (#6577)
* Add initialization directory support to Docker image
* Add sharness test, fix bugs in init script
Fixed in init script:
- Added some missing quotes around expansions
- Fixed INIT_ARGS to not pass any args if IPFS_PROFILE isn't specified
- Use printf instead of "echo -e"
- Only run scripts in top-level of init dir
- Handle filenames correctly when finding init scripts (by using find + xargs)

* chore: docker cleanup
cleans up containers and images (useful when run on developer machine)

* remove container init documentation from README
There is already IPFS Docker documentation where this should live:
https://docs.ipfs.io/how-to/run-ipfs-inside-docker/

Co-authored-by: Caian <caian@ggaunicamp.com>
Co-authored-by: Marcin Rataj <lidel@lidel.org>
Co-authored-by: Gus Eggert <gus@gus.dev>
2022-04-12 19:44:03 +02:00
Gus Eggert
bb68a68525
feat: port collect-profiles.sh to 'ipfs diag profile' (#8786)
* feat: add block profiling to collect-profiles.sh

* feat: add more profiles to 'ipfs diag profile'

This adds mutex and block profiles, and brings the command up-to-par
with 'collect-profiles.sh', so that we can remove it.

Profiles are also now collected concurrently, which improves the
runtime from (profile_time * num_profiles) to just (profile_time).

Note that this has a backwards-incompatible change, removing
--cpu-profile-time in favor of the more general --profile-time, which
covers all sampling profiles.

* docs(cli): ipfs diag profile

* add CLI flag to select specific diag collectors

Co-authored-by: Marcin Rataj <lidel@lidel.org>
2022-04-12 11:58:03 -04:00
Omicron166
d13a09a076
remove unused import (#8787) 2022-03-25 08:48:16 -07:00
Gus Eggert
5e1b2248c5
feat: add full goroutine stack dump (#8790) 2022-03-16 09:42:22 -04:00
Nato Boram
ef63822ef0
fix(build): Recognize Go Beta versions in makefile (#8677)
* 🩹 Fix building with beta Go versions

* ✏️ Use `[:space:]` to be more canonical
2022-02-15 17:47:48 -05:00
odanado
0216bae307
docker: build for arm cpu (#8633)
ref: #4931
2022-02-15 16:54:47 -05:00
Marcin Rataj
c0f282f00f
ci: move Docker image build to Actions (#8467)
* ci: move docker image publishing to github

Closes #8330

* chore: remove dockerhub push from circleci
2021-09-28 22:30:36 +02:00
guseggert
3e49bb16b4 ci: publish Docker images for bifrost-* branches 2021-08-26 20:07:52 +00:00
Steven Allen
4e132af3ef fix(mkreleaselog): specify the parent commit when diffing
Specifically, if you specify a commit, you get only that commit. This
means you get _nothing_ if you specify a merge commit.

Now, we specify to diff between a commit and it's (merge) parent.
2021-08-23 17:04:51 -04:00
Gus Eggert
499b596efa Merge branch 'master' into ci-build-docker-with-plugins 2021-08-05 16:10:00 -04:00
Steven Allen
99ccde2f24 fix(mkreleaselog): match files anywhere in the path
I had _thought_ pathspecs needed to start with / to only match the root,
but apparently not.
2021-07-21 12:37:51 -07:00
Steven Allen
8fca584a3c fix(mkreleaselog): remove outdated comment 2021-07-21 11:33:09 -07:00
Steven Allen
d3604bb06a feat: improve mkreleaslog
1. Allow matching the entire module instead of just github
   orgs/usernames.
2. Allow excluding some modules.
3. Ignore files using a github pathspec and apply the same ignore
   patterns to the "contributors" section.
2021-07-21 10:54:26 -07:00
Steven Allen
0653862632
Merge pull request #8214 from ipfs/fix/mkreleaselog
fix(mkreleaselog): support multiple commit authors
2021-06-24 17:36:57 -07:00
Steven Allen
b4b10f5b98
Merge pull request #8121 from ipfs/fix/mkreleaselog-first-commit
fix(mkreleaselog): handle commit 0
2021-06-24 17:36:41 -07:00
Steven Allen
4be04bce4e fix(mkreleaselog): support multiple commit authors
This will output one entry per commit author in the "stat" log, instead
of just crashing.
2021-06-23 15:04:17 -07:00
Michael Burns
17ed9fbeb3
remove dht-stabalize specialcase for bifrost 2021-05-17 02:11:02 +00:00
Adin Schmahmann
502b68c111 build: ignore generated files in changelog 2021-05-14 10:15:49 -04:00
Steven Allen
573804aa4c fix(mkreleaselog): handle commit 0
At commit 0, there is no parent.
2021-05-10 16:45:14 -07:00
Steven Allen
c744981585 fix(mkreleaselog): partially handle v2 modules
We can now handle v2 modules that use tags, but not v2 modules that use
v2 sub-directories.
2021-04-13 12:45:15 -07:00
Oli Evans
3106e5f911
chore: dont docker tag rc as latest
- add guard for rc releases; publish them to docker hub with the matching git tag
- make the semver regex stricter and only publish as latest when a full semver tag with no pre-release suffix is the name of the git tag.
- add `release` tag as an alias of `latest` as per https://github.com/ipfs/go-ipfs/issues/3999#issuecomment-742228981

Tested manually as the push-docker-tag.sh script is set up for it:
```shell
./push-docker-tags.sh $(date -u +%F) testingsha release v0.9.0-test dry
Nothing to do. No docker tag defined for branch: release, tag: v0.9.0-test

./push-docker-tags.sh $(date -u +%F) testingsha release v0.9.0-rc1 dry
DRY RUN! I would have tagged and pushed the following...
docker tag ipfs/go-ipfs:wip ipfs/go-ipfs:v0.9.0-rc1
docker push ipfs/go-ipfs:v0.9.0-rc1

./push-docker-tags.sh $(date -u +%F) testingsha release v0.9.0 dry
DRY RUN! I would have tagged and pushed the following...
docker tag ipfs/go-ipfs:wip ipfs/go-ipfs:v0.9.0
docker push ipfs/go-ipfs:v0.9.0
DRY RUN! I would have tagged and pushed the following...
docker tag ipfs/go-ipfs:wip ipfs/go-ipfs:latest
docker push ipfs/go-ipfs:latest
DRY RUN! I would have tagged and pushed the following...
docker tag ipfs/go-ipfs:wip ipfs/go-ipfs:release
docker push ipfs/go-ipfs:release
```

fixes #3999

License: MIT
Signed-off-by: Oli Evans <oli@tableflip.io>
2021-04-06 16:39:22 +01:00
hannahhoward
66e0f8a1d9 build(Makefile): set supported platforms by go-version
Set supported platforms correctly by go version (no darwin-386 for >=1.15, no darwin-arm64 unless
>=1.16), make check-go-version work on mac

License: MIT
Signed-off-by: hannahhoward <hannah@hannahhoward.net>
2021-02-18 20:28:54 -08:00
Steven Allen
82e6674099 fix: go doesn't allow gnu short flags 2021-02-08 12:49:51 -08:00
Steven Allen
fed5f05134 build: simplify on gofmt 2021-02-08 12:44:15 -08:00
Steven Allen
30dcc26be8 feat: support the mailmap file in mkreleasenotes
This will use either the mailmap file in the current repo, or the mailmap file
in the target repo, when generating the contributor stats.
2020-05-26 21:46:45 -07:00
Steven Allen
048c1c37b3 fix(mkreleaselog): make robust against running in different working directories
1. Cd into the repo root before running `go list`.
2. Make sure the repo lives in the GOPATH.
2020-05-12 22:30:53 -07:00
Steven Allen
d2b1d5b7d4 fix(mkreleasenotes): include commits directly to master
This will include:

* Commits to master.
* Squashed merges.

In the release notes.
2020-05-08 19:37:56 -07:00
Steven Allen
2b3fafc243 feat(mkreleasenotes): include more repos
We should be including multiformats repos and repos from our core contributors.
2020-05-08 19:37:20 -07:00
Dimitris Apostolou
1e437c7e97
Fix typos and cleanup 2020-04-20 22:00:01 +03:00
Peter Rabbitson
5d3247f9b7 Exclude more concurrency-sensitive dirs from fmt test 2020-03-15 19:51:57 +00:00
Oli Evans
427557e903
wip: remove release tag. use more recent docker.
License: MIT
Signed-off-by: Oli Evans <oli@tableflip.io>
2020-03-13 14:31:45 +00:00
Oli Evans
9f6349b07b
wip: remove dry run
License: MIT
Signed-off-by: Oli Evans <oli@tableflip.io>
2020-03-12 19:04:37 +00:00