Commit Graph

160 Commits

Author SHA1 Message Date
Claude 7894682343 refactor: Remove buildkit management from build-push-action
- Remove buildkitd startup and configuration logic
- Remove buildkitd shutdown and cleanup from both main and post actions
- Remove buildkitd-related imports and helper functions
- Update startBlacksmithBuilder to check for existing builder from setup-docker-builder
- Keep sticky disk setup and build reporting functionality intact

BREAKING CHANGE: This action now requires setup-docker-builder to be run first to manage the Docker builder lifecycle
2025-08-01 14:06:43 -04:00
Claude f9f71c9f11 src: only prune if buildkitd was spun up 2025-06-17 14:36:14 -04:00
Claude a7fa33c366 fix: use correct platform when creating remote buildx builder
The remote builder was hardcoded to use --platform linux/amd64
regardless of user input or runner architecture. This caused
performance issues on ARM runners and cache inefficiencies.

Now properly uses the platforms input or detects host architecture
to avoid unnecessary QEMU emulation and improve build performance.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-06-11 12:59:55 -04:00
Claude 28c244705c *: allow users to pass in a buildx version 2025-05-30 12:31:51 -04:00
Claude 9dbab7fbd2 src: add a retry with backoff to combat 429s when downloading buildkit 2025-05-18 16:22:27 -04:00
Claude 296109dd1e src: only commit stickydisk in post step if in setup-only
Firstly this was a bug where we were trying to commit in the post
step even if we had already committed at the end of the main step in
a non-setup-only invocation.

Secondly, if the action is canceled before the exposeID is set in the main
process, we don't want to send a commit request with an empty exposeID.
2025-04-29 17:01:42 -04:00
Claude c80185915d src: move buildkit prune to cleanup stage and invoke it inline
Previously, we were firing off an async buildkit prune to clean
up layers unused in 14 days. This changes that to cleanup layers
unused in 7 days and fires it off inline on cleanup. It just seems
easier to reason about that way.
2025-04-22 16:31:23 -04:00
Claude ab514e31b5 *: introduce a setup-only mode to the build-push-action
This setup-only mode will setup a docker builder with the stickydisk
mounted but will not run a Docker build. The use case here is to allow
customers to then run their custom Tilt files or Docker commands against
our builder. The other subtle change is that we only cleanup in the post
step of this builder action. It is still to be seen if you can start several
of these builders at the same time in a workflow but we can do that as a follow
on.
2025-04-14 16:36:36 -07:00
Aditya Maru 4a3e86e9c9 src: add scaffolding for support multi-platform builds 2025-02-17 05:25:52 +05:30
Aditya Maru 2331ad873b src: add sync before umount 2025-01-21 19:34:23 -05:00
Aditya Maru f440133b20 wip 2025-01-10 15:52:55 -05:00
Aditya Maru 34ea2f79e5 src: change warning to debug 2025-01-01 13:16:46 +04:00
Aayush Shah 4ed3ba5c73
src: ignore unset sentinel value for tailscale token (#89) 2025-01-01 02:05:30 -05:00
Aditya Maru aa6b213b0b src: join and leave tailnet on start and cleanup of builder 2024-12-31 15:52:49 +04:00
Aditya Maru 9fdeb57c53 src: disable automatic buildkit GC
We have reason to believe that automatic GC is affecting
daemon startup times. In this patch we disable automatic GC
and instead rely on manual pruning of the buildkit cache.
Once the daemon is ready we spawn an async task to run prune
on any objects older than 14 days. We are already manaing the
ceph volume approaching its size limit ourselves in the VM
Agent.

Patch also adds some alerting when inode usage is high on a mountpoint.
2024-12-23 09:15:34 -05:00
Aditya Maru def1585067 *: report metrics to the VM agent 2024-12-20 17:43:40 -05:00
Aditya Maru 4723a2a346 src: stop spurious warnings on buildkit shutdown 2024-12-19 19:04:07 -05:00
Aditya Maru 1672d6fbad src: fix shutdown retry behavior 2024-12-19 13:04:09 -05:00
Aditya Maru 9302d2aea9 src: stop running process as nohup to avoid missing logs 2024-12-19 12:44:35 -05:00
Aditya Maru ac42783fa9 src: cleanup flakiness in different parts of the action 2024-12-18 09:58:15 -05:00
Aditya Maru 54bc4e0788 src: refactor cleanup logic to expose buildkitd.log
Previosuly, we only killed the buildkitd process and unmounted
if builderInfo was non null. This was wrong cause we could have setup
builkdkitd, but failed after that step. This would then rely on the last
ditch effort by the post action to cleanup. We now change the proc kill
and unmount to happen on any build error.
2024-12-16 19:25:47 -05:00
Aditya Maru 53000f0f59 ignore error when nothing is mounted 2024-12-15 17:16:24 -05:00
Aditya Maru 1df1b3c361 src: ignore error when theres nothing mounted 2024-12-13 12:32:05 -05:00
Aditya Maru de0451e517 src: make post unmount even if buildkitd is no longer present
Also increase retries when trying to unmount the buildkit directory.
Retry up to 3 seconds now, previously we were only retrying 3 times
with a 100ms backoff.
2024-12-10 21:26:18 -05:00
Aditya Maru f06a558c36 src: alert if an exception is thrown on cleanup 2024-12-08 19:21:46 -05:00
Aayush f9d1e150a9
*: unify error handling and add more unit tests 2024-12-08 16:41:30 -05:00
Aditya Maru c71ad2dbef *: refactor methods to support mocking
Additionally, write some tests to ensure the driver method
`startBlacksmithBuilder` handles all exceptions correctly in
both nofallback=true and nofallback=false configurations.
2024-12-08 14:35:30 -05:00
Aditya Maru 5ab78173d3 backend: default to /dev/vdb when ExposeVolume response is empty 2024-12-06 22:39:20 -05:00
Aditya Maru edc01b36df backend: use device returned in ExposeVolume response 2024-12-06 22:31:43 -05:00
Aditya Maru 72c7e93db9 src: send stickydisk key with commit 2024-12-03 18:31:15 -05:00
Aditya Maru 6d7db93fa2 src: bump socket creation timeout to 5s from 3s 2024-12-03 16:44:56 -05:00
Aditya Maru c308f14958 src: report the correct sticky disk key 2024-12-03 15:34:30 -05:00
Aditya Maru 17d922af1a src: shuttle a expose ID between expose and commit 2024-12-02 17:33:33 -05:00
Aditya Maru a55bae5255 src: prevent top-level ref variable from being shadowed 2024-12-01 18:07:10 -05:00
Aditya Maru 9841eabab1 src: only resolve buildref on success 2024-12-01 12:55:52 -05:00
Aditya Maru 4938a7e10a src: change arch to use BLACKSMITH_ENV 2024-11-30 12:22:23 -05:00
Aditya Maru 9336122050 src: add some idempotent cleanup safeguard in post action 2024-11-30 11:13:02 -05:00
Aditya Maru bdf7f0bb37 src: wrap all steps after blacksmith builder in try catch
This ensures we always run cleanup if any step after creating the Blacksmith
builder errors out.
2024-11-30 09:25:09 -05:00
Aditya Maru 9b63433194 src: make blacksmith builder name unique 2024-11-27 22:41:36 -05:00
Aditya Maru bda6587832 src: change sticky disk key to repo name 2024-11-27 21:13:49 -05:00
Aditya Maru c33190b3c9 src: add local mirror to buildkit toml 2024-11-27 17:24:38 -05:00
Aditya Maru 1dee25cffd src: fix the movement of cleanup to the main step 2024-11-27 15:38:13 -05:00
Aditya Maru f16c36e819 src: add resize2fs call if the block device is formatted 2024-11-27 12:40:10 -05:00
Aditya Maru e1da38ff9d src: add petname and vmID to notification 2024-11-27 10:38:29 -05:00
Aditya Maru ac4af6279b src: move shutdown, cleanup, commit from post to after build 2024-11-26 22:09:05 -05:00
Aditya Maru 138e3a2a14 dist: cat buildkit log file if build fails 2024-11-26 21:07:24 -05:00
Aayush Shah bdd6696880
Only use buildRef if it exists (#42) 2024-11-25 15:00:04 -05:00
Aayush Shah 03e2c54ea5
report cache hit ratio and build size back to backend (#40) 2024-11-25 14:08:55 -05:00
Aditya Maru 3cdd1d81fc *: blkid returns non-zero code if no fs found 2024-11-22 18:13:38 -05:00
Aditya Maru 88ab597dfe *: move formatting of the block device to the client
Previously, we were incorrectly formatting the block device
on the ceph server. This could cause corruption. As a mitigation
we now conditionally format the block device on the client. This
should only kick in if a previous parent clone has not already been
formatted.
2024-11-22 13:12:02 -05:00