Two Forgejo organizations split the repositories by role:
gitops holds what describes the cluster, and
k8s-services holds one source repository per deployed
service.
| Repo | Purpose |
|---|---|
gitops/apps |
ArgoCD Application and ApplicationSet
definitions |
gitops/charts |
One Helm chart directory per deployable, values per environment |
k8s-services/<app> |
Application source plus .forgejo/workflows/ci.yml |
ArgoCD reaches both gitops repos over Forgejo’s SSH
server:
ssh://[email protected]:30354/gitops/apps.git
ssh://[email protected]:30354/gitops/charts.git
ssh-git.yhkze.net is the unproxied DNS record — the
Cloudflare-proxied name cannot carry SSH.
Repository credentials — ArgoCD authenticates as a
dedicated restricted Forgejo machine account, not as the human admin
account. The account holds an SSH deploy key and read-only team access
to both repos, so a compromise of the cluster yields no write path into
git. Forgejo resolves the account from the key itself, which is why the
URL user is always the literal forgejo.
gitops/apps is applied by a single root Application that
recurses the whole repo and excludes only its own bootstrap file, so
ArgoCD self-manages every other Application and ApplicationSet after the
one manual apply.
flowchart LR
rootfile["root-app.yaml<br/>(applied once by hand)"] --> root[root-applications]
root --> std["Standalone Applications<br/>cert-manager · external-secrets · kargo<br/>ceph-csi-rbd · local-path-provisioner<br/>cloudnative-pg · monitoring · forgejo-runner"]
root --> appset["ApplicationSet<br/>development-services"]
appset --> svc["<service>-test<br/><service>-prod"]
std --> charts[(gitops/charts)]
svc --> charts
Platform components are standalone Applications because they are
singletons with no test/prod split, and each is explicitly excluded from
the ApplicationSet’s git generator so it is not picked up twice. All of
them sync automatically with prune and
selfHeal.
The development-services ApplicationSet is a matrix of a
git directory generator over gitops/charts and a list
generator of test and prod. A new chart
directory therefore yields <service>-test and
<service>-prod Applications, each deployed into its
own <service>-<env> namespace with
values/values-<env>.yaml layered over the chart’s
base values. Adding a service needs no edit to
gitops/apps.
Chart layout expected by the generator:
gitops/charts/
<service>/
Chart.yaml
values.yaml # base values: image.repository, service.port, …
values/
values-test.yaml # domain, replicas, image tag
values-prod.yaml
templates/
_helpers.tpl # must live here, not at chart root
deployment.yaml
service.yaml
ingress.yaml # IngressRoute CRD for the host Traefik
Helm only loads helper templates from templates/. A
_helpers.tpl at the chart root is silently ignored, and
every template referencing it fails to render.
Forgejo Actions runs the pipeline on runners inside the cluster. CI only builds and publishes an image; it never edits the charts repo and never talks to the Kubernetes API. Advancing an image through environments is Kargo’s job.
push → build-push (GHCR) → notify Kargo → Warehouse → Freight → Promotion → ArgoCD sync
Trunk-based, so the tag shape decides the environment rather than the branch name:
| Env | Trigger | Image tag | Domain |
|---|---|---|---|
| test | push to main |
mutable main |
<app>-test.yhkze.net |
| prod | push of a vX.Y.Z tag |
immutable vX.Y.Z |
<app>.yhkze.net |
Each environment has its own Warehouse subscribed to the matching tags, so a release tag is the only thing that can produce prod Freight.
GHCR cannot call into Forgejo, so nothing would otherwise tell Kargo
an image landed — the Warehouses would only notice when their own
five-minute interval elapsed. The final CI step POSTs to Kargo’s webhook
receiver to collapse that delay. The receiver URL contains a
hard-to-guess path derived from CI_WEBHOOK_SECRET, so treat
the whole URL as a credential:
kubectl get clusterconfig cluster -o jsonpath='{.status.webhookReceivers}'Configuration every service needs is set once on the
k8s-services organization and inherited by each repository
in it:
| Name | Kind | Purpose |
|---|---|---|
GHCR_PUSH_TOKEN |
secret | classic GitHub PAT, write:packages |
GHCR_PUSH_USERNAME |
variable | GitHub account owning that PAT |
GHCR_NAMESPACE |
variable | image prefix,
ghcr.io/<namespace>/<repo> |
KARGO_WEBHOOK_URL |
secret | Kargo receiver; the URL is itself a credential |
The username is a variable rather than a secret because it is not sensitive, and Forgejo masks secret values in logs — as a secret it would blank the image path out of every build log.
Because the image path is built from GHCR_NAMESPACE and
the repository name, ci.yml contains nothing
service-specific and copies between repositories unchanged.
Repository-level secrets hold only what that one service’s CI needs.
Registry credentials are split by direction. The
push token lives only in Forgejo, since only CI pushes.
The pull token lives in OpenBao at
registry/ghcr-pull, since its consumers — Kargo discovery
and kubelet image pulls — run inside the cluster. No git token is needed
either: Kargo pushes promotion commits with its own credential.
Reference OpenBao Secret Management.
The runner lives in the ci namespace, deployed by ArgoCD
from the forgejo-runner chart in
gitops/charts, with a Docker-in-Docker sidecar for image
builds and an HPA scaling 1-4 replicas on CPU. Concurrency is
runner.capacity jobs per replica, so the ceiling is
capacity times maxReplicas.
Runner 12.10.1 uses config-based connections; the
register command is gone. The config hands the Docker TLS
material to job containers so docker/build-push-action
works: the runner shares a /certs emptyDir with DinD, job
containers get -v /certs/client:/certs/client through
container.options and
DOCKER_HOST=tcp://localhost:2376 through
runner.envs, and container.network: host lets
them reach the daemon on localhost. The pattern follows
code.forgejo.org/forgejo/runner/examples/kubernetes/dind-docker.yaml.
The runner’s credential is held out of band and is absent from the
chart, so the chart is always safe to re-sync. Create a runner under
Site Admin → Actions → Runners → Create New Runner, then
store the token:
kubectl -n ci create secret generic runner-secret --from-literal=token='<TOKEN>'The UUID that registration returns is the runner’s identity rather
than a credential, so it lives in values.yaml as
runner.uuid. Set it there and let ArgoCD sync; a mismatch
makes the daemon register as a second, separate runner instead of
reclaiming the existing one.