Architecture

The three-org chain

fork-sync-all is the control plane for a three-organisation mirror chain on GitHub, with a fourth leg into GitLab:

Interested-Deving-1896  ──►  OpenOS-Project-OSP
        ▲                           │
        │                           ▼
        │              OpenOS-Project-Ecosystem-OOC
        │                           │
        │                           ▼
        │                  GitLab openos-project
        │             (14 subgroups, 225 repos mirrored)
        │
        └──── upstream-commits / upstream-prs (OSP + OOC → I-D-1896)
OrgRole
Interested-Deving-1896Primary — forks live here, all automation runs here
OpenOS-Project-OSPSecondary mirror — receives pushes from I-D-1896
OpenOS-Project-Ecosystem-OOCTertiary mirror — receives pushes from OSP
gitlab.com/openos-projectGitLab mirror — receives pushes from OSP via GitLab CI

All automation runs in Interested-Deving-1896/fork-sync-all. The other orgs are passive recipients — they do not run their own automation except for the GitLab CI mirror job that pushes back to GitLab.


Data flow

Inbound (upstream → I-D-1896)

Three paths bring upstream changes into Interested-Deving-1896:

  1. sync-forks.yml (daily) — syncs all GitHub forks with their upstream parents
  2. sync-registered-imports.yml (daily at 04:55 UTC) — re-syncs repos registered in registered-imports.json, including non-GitHub sources (GitLab, Bitbucket, Codeberg, etc.)
  3. upstream-commits.yml / upstream-prs.yml (daily) — detects direct commits and open PRs in OSP/OOC that haven't been reflected upstream, and opens PRs in I-D-1896

Outbound (I-D-1896 → OSP → OOC → GitLab)

The mirror chain runs in sequence, each leg triggered by the previous:

mirror-to-osp.yml  ──►  mirror-osp-to-ooc.yml  ──►  GitLab CI (sync-to-gitlab.yml)
   (every 6h at :13)     (every 6h at :45)           (on OSP push)

flush-lifecycle.yml is the top-level entry point for a full pipeline run. It sets FLUSH_ACTIVE=true, holds a sentinel runner slot, and coordinates the three-stage sequence with quota reservation and pause/resume at reset windows:

flush-lifecycle.yml
  └─ pre-flush-prep.yml    ← cancels stale runs, merges PRs, validates config
  └─ full-chain-flush.yml  ← all mirror + README + CI stages in order
  └─ post-flush-prep.yml   ← integrity checks and queue health verification

full-chain-flush.yml orchestrates the pipeline stages themselves — mirror chain, README updates, sync, and validation — but should be triggered via flush-lifecycle.yml rather than directly, so the FLUSH_ACTIVE mutex and quota reservation are active for the full run.

GitLab subgroup placement

config/gitlab-subgroups.yml is the single source of truth for which repos go into which GitLab subgroup. The 14 subgroups map to topic areas:

SubgroupReposTopic
incus_deving49Incus / container infrastructure
yaml-tooling_deving34YAML, CI, and tooling
ops30Operations and control plane
agnostic-api_deving29Unified Agnostic API — virtual filesystems, AI/LLM adapters, OS-compat layers
penguins-eggs_deving17penguins-eggs ecosystem
linux-kernel_filesystem_deving14Kernel and filesystem
cachyos_deving12CachyOS packages
ai-agents_deving10AI agent tooling
accessibility_deving9Screen readers, Braille, WCAG auditing, audio overviews
git-management_deving9Git tooling
neon-deving8KDE Neon ecosystem
rust-systems_deving2Rust system tools
taubyte_deving1Taubyte platform
immutable-filesystem_deving1Immutable Linux

Repos not listed in any subgroup fall into the ops default subgroup.


Quota management

Both GH_TOKEN and SYNC_TOKEN belong to the same GitHub user and share a single 5000 req/hr REST bucket. The system has three layers of protection:

quota-reserve.yml  ──►  queue-manager.yml
  (every 30 min)          (every 30 min)

quota-monitor.yml is a separate manual-dispatch-only tool for waiting out quota exhaustion. It is not part of the automatic quota management loop — see Operations for when and how to use it.

LayerThresholdAction
quota-reserve< 1000 remainingCancels tier-4 (LOW) queued runs
quota-reserve< 500 remainingCancels tier-3 (MEDIUM) queued runs
queue-managerRun queued > 25 minEvicts stale queued runs
queue-managerDuplicate workflowKeeps newest, cancels older

Workflow priority tiers are defined in config/workflow-priority-tiers.yml. Tier 1 (CRITICAL) runs are never cancelled. See Operations for the full quota reference.


Config files

FilePurpose
config/gitlab-subgroups.ymlGitLab subgroup placement for 225 repos (14 subgroups)
config/ona-projects.ymlOna project registry — maps repos to project IDs, environment classes, and tags
config/workflow-priority-tiers.ymlPriority tier for each workflow (used by queue-manager and quota-reserve)
config/workflow-quota-costs.ymlmin_quota + cost tiers per workflow — source of truth for quota-reserve and pre-flight checks
config/workflow-cost-profiles.ymlDetailed REST/GraphQL/GitLab/AI call estimates per workflow (used by rate-limit-profile.sh)
config/workflow-sync.ymlGitHub ↔ GitLab CI job mapping (used by validate-workflow-guards)
config/ota-registry.ymlRepos opted in to the OTA update system
config/ota-blocklist.ymlOrgs/namespaces excluded from OTA by default
config/template-manifest.ymlTemplate sync profiles and file ownership
config/template-consumers.ymlRepos consuming each template profile
registered-imports.jsonUpstream repos registered for ongoing sync

vendor/

vendor/ contains third-party components that fork-sync-all hosts or deploys. It is distinct from scripts/ (first-party automation) and config/ (config data).

Current components:

ComponentDescription
vendor/infra-dashboardMirror-health and package-search SPA + Rust API backend

All vendored components must be deployment-agnostic — no distro names, org-specific URLs, or hardcoded deployment values. See Contributing for the enforcement workflow.


Token architecture

Two GitHub PATs are in active use, both owned by the same user (ID 202036334) and sharing the same 5000 req/hr quota:

SecretUsed byScope
SYNC_TOKENMost workflowsrepo, workflow, admin:org
GH_TOKENValidation, README, config workflowsrepo, workflow

GitLab operations use GITLAB_SYNC_TOKEN (api, read/write_repository scope).

Token expiry is monitored weekly by token-health.yml. See Token Rotation for rotation procedures.