Hacker News

points by eknkc 4 days ago | 0 comments

Our deploys are tied to GitHub actions and it is becoming a liability at this point. What alternatives do people use? I mean for FOSS there are some decent stuff but we need a reliable provider for private repos.

[–] stack_framer 4 days ago

We use GitLab where I work. The only problem is it calls pull requests "merge requests," and I don't think I'll ever get used to saying "MR" instead of "PR."

[–] jaapz 4 days ago

Why are all you github people pulling on your requests?

I am requesting to merge my changes into the main branch.

[–] dutchbart 4 days ago

Or you are requesting me to pull your changes into my branch.

[–] crote 3 days ago

Because in the good old days when git was still distributed, a merge often started with sending someone an email to obtain your newly-coded commits via `git pull`ing your "gpu-drivers-for-linus" branch from some URL pointing to your personal GitWeb install. In other words: it was a request to pull your code.

Similarly: Why are the Gitlab people so happy to merge? Where is my "Rebase Request", "Squash Request", or "Cherry-pick Request"?

[–] ajasmin 3 days ago

You can still do this on a GituHub PR. Pull the branch locally, read the code, try things out, rebase onto main…

Once all is good, push into remote HEAD and GithHub should auto-close.

Though, larger project tend to leave the final upstream merge to GitHub itself. Better access control and it’s easier to serialize merges through a single queue for very active projects.

[–] chrisweekly 3 days ago

good bit of lore.

related tidbit: in git, "pull" is basically shorthand for "fetch + merge".

[–] hhh 4 days ago

I have the opposite

[–] Roark66 4 days ago

This is exactly why I don't understand companies (like some of my clients) that move from self hosted Jenkins to GH Actions.

You're literally replacing a well designed resilient system with a single point of failure run by a different company...

[–] Chris2048 4 days ago

> a well designed resilient system

Wait, Jenkins?

[–] iamjake648 3 days ago

Like with all things, it really depends on having someone competent stand it up and run it. I think we've already reached the point that this is totally worth it for mid-sized companies as opposed to giving MS money for GHA.

[–] hiccuphippo 4 days ago

I get to tell my manager it's github's fault.

[–] saturn_vk 4 days ago

Self-hosted Forgejo with a bunch of runners might also be interesting for you. The workflow syntax is _almost_ identical to the github one.

[–] SkiFire13 4 days ago

Some of our deploys use CircleCI instead, but they were still broken because they couldn't checkout from Github.

[–] embedding-shape 4 days ago

You don't even have to migrate off GitHub if you don't want to.

Begin by refactoring the deployments so they're driven by checked in shellscripts, that accepts a git reference to deploy (prefer immutable commit SHAs over mutable branches/tags). Make it use the stuff it already has checked in, and make the actual GitHub Actions stuff as small and concise as possible, ideally each "run" should just call a shellscript with the git reference.

Then move so the deploys happen not on GitHub Actions itself, but on a "intermediate deploy host" that has the source code and access to the credentials you need, and make this host have one secure entrypoint, where someone (or something) can enter and run shellscripts, after pushing their code as a normal git push. Then switch so the deploys from GitHub Actions instead use this new host to do the deploys. Lock down this host hard.

When GitHub and/or GitHub Actions go down, the developer responsible for deploying stuff, should be able to get temporary git and ssh access to this host, can push any git data straight to the host, and their user on this host can run the deploy scripts. Keep in mind that nothing in your deployment process should depend on stuff that isn't also available on this "deploy host", if you have multiple synced repositories and so on.

Now things mostly work the same as people are used to, with the added benefit that when things go down again (like next week), you'll still be able to continue to deploy stuff.

I've done this refactor/migration for tens of companies in the last year, more or less following this exact template for all of them, with no issues reported from either of them, but lots of praise :)

[–] pelagicAustral 4 days ago

My deployments are tied to their container registry, so I'm equally screwed... Been thinking of going back to Dokku, I started working on a third wrapper around Dokku (yeah, I've literally done this twice already), and then self-host and self-provision... I honestly thinking seriously about this.

[–] misswaterfairy 4 days ago

Genuine question: if you're hosting private repos, is self-hosting an option?

Gitea Enterprise would be worth looking at.

https://about.gitea.com/products/gitea-enterprise/

[–] eknkc 4 days ago

Honestly, not really wanna deal with self hosting it. Would rather pay for a decent service at this point.

[–] import 4 days ago

There is already a blacksmith sh if you don't want to selfhost your runners. Or codeberg as a full solution if I am not mistaken.

[–] michaelbuckbee 4 days ago

+1 for Blacksmith. You need to have a Github org to use it as it won't work on personal accounts, but it feels like a magic want that just makes runners 2x as fast.

I have it for all my stuff (dozens of repos and hundreds of PRs/builds per week) and it's only once in the last year that I've exceeded the free resource levels. I don't know if it's just extra capacity or they're burning VC dollars in the hopes of inspiring posts like this, but either way I feel like I'm taking advantage of them at this point.

[–] booi 3 days ago

But doesn't Blacksmith piggy back off of the Github Actions control plane? Outages will just spread to Blacksmith

[–] crote 4 days ago

Codeberg only allows F/LOSS software, for most companies that's a dealbreaker.

[–] tcfhgj 4 days ago

Private projects usually are not owned by companies

[–] crote 3 days ago

Your proprietary personal hobby project doesn't need an entire publicly-accessible Forge, it can just live on your local drive or home server. It is also not critical enough that GitHub occasionally being down is a big deal.

If you're looking for a publicly-accessible Forge SaaS for your private repos and are willing to pay for it, you are almost certainly a company developing proprietary software.

[–] bonetruck 4 days ago

Forgejo backed by Woodpecker CI running on Kubernetes (Talos) has been amazing! Ditch Github Actions syntax for the simplicity of Woodpecker and containers. You'll thank me later.

[–] paaloeye 4 days ago

How do you run Docker containers inside your K8S? Let’s say you need a test database or test ClickHouse???

[–] NewJazz 4 days ago

1 web search later... Top result..

https://woodpecker-ci.org/docs/usage/services

[–] karolist 4 days ago

Self hosted Jenkins, been working wonderfully.

[–] maratc 4 days ago

I use Jenkins too, however any pipeline starts by checkout from github, so it failed this morning just the same.

[–] karolist 4 days ago

Same for me, ssh access was intermittent, but at least I have reliable local builds when GitHub actions is out but access works. With AI running Jenkins is not that hard. I build large cuda containers so cost alone ruled out hosted builds for my usecase. There's a bit of update churn but nothing broke so far. I ran around 2000 builds last year. I run it on epyc 7v12 128gb ddr4 host.

[–] delf 4 days ago

I self-host with GitSocial.

https://gitsocial.org/

[–] shimms 4 days ago

Buildkite with self hosted runners

[–] maccard 3 days ago

Which is 4x the price of github, _plus_ the cost of the runners on top of it.

It's a great service though

[–] myaccountonhn 4 days ago

Post receive hook on a bare git repo with gitolite for access control.

[–] rozularen 4 days ago

self hosted forgejo has been my saviour. im thinking in acquiring a new NUC server to increase compute and storage to my homelab to add some new runners

[–] icy 4 days ago

We're shipping private repos for Tangled very soon (tangled.org). Shoot me an email anirudh@!

[–] import 4 days ago

gitea is amazing.

[–] raddan 4 days ago

Seconded. Also very easy to set up. I’ve been running it for years, along with webhooks to kick off testing and website deployment jobs. I especially like the swagger browser that generate API calls for you to try in curl. I use the API pretty extensively.

[–] jelder 4 days ago

Git itself usually works, but we gave up on GHA and use https://www.rwx.com/ for tests, previews, and deployments. It’s 2x to 10x faster than GitHub Actions and testing uncommitted changes on real CI infrastructure is a total game changer.