fork-sync-all

Sync and mirror infrastructure for the three-org chain:

Interested-Deving-1896  ──►  OpenOS-Project-OSP  ──►  OpenOS-Project-Ecosystem-OOC
        ▲                                                         │
        └─────────── upstream-commits / upstream-prs ────────────┘
                                    │
                                    ▼
                         GitLab openos-project
                    (11 subgroups, ~150 repos mirrored)

Documentation

• Full documentation — architecture, quota management, workflow reference, runbooks
• Workflow Triggers — every workflow, its schedule, and what else triggers it (plain text)

Workflows

Sync & Mirror

Import

Import workflow inputs:

• repo_url — source URL (GitHub, GitLab, Bitbucket, Codeberg, Sourcehut, Gitea, or any git host)
• repo_name — optional rename in Interested-Deving-1896 (defaults to source name)
• mirror_to_osp_ooc — push through the OSP → OOC chain immediately
• ongoing_sync — register in registered-imports.json for re-sync every 6h

Quota and queue management

Security and token management

Maintenance

Secrets

To add a missing secret, run in your terminal (value prompted securely, never logged):

gh secret set <SECRET_NAME> --repo Interested-Deving-1896/fork-sync-all

Registered Imports

registered-imports.json tracks repos imported via import-repo.yml with ongoing_sync enabled. The sync-registered-imports.yml workflow reads this file hourly and re-pulls each source.

Schema:

[
  {
    "source_url":  "https://gitlab.com/some-group/some-repo",
    "target_name": "some-repo",
    "platform":    "gitlab",
    "added":       "2026-05-02T18:00:00Z"
  }
]

To register a repo manually, run import-repo.yml with ongoing_sync: true, or edit the file directly and commit.

Rate limits

All workflows share a single SYNC_TOKEN. Understanding the limits prevents surprise failures and helps diagnose them when they do occur.

GitHub REST API

What a 403/429 means here: GitHub returns HTTP 403 for secondary rate limits and HTTP 429 for primary exhaustion. Both include X-RateLimit-Reset in the response headers. All scripts that call the GitHub API read this header and sleep until the reset window opens before retrying (up to 3 attempts).

Workflows most likely to hit limits: sync-forks.yml (scans all forks), reconcile-org-refs.yml (reads every file in every repo), and resolve-failures.yml (scans all repos across three orgs). These run sequentially within their own concurrency group so they don't compound each other's usage.

If a workflow fails with "API rate limit exceeded": the next scheduled run will succeed once the window resets. resolve-failures.yml will also catch and retry it automatically. No manual intervention is needed unless the token itself has been revoked.

GitHub Models API

Used by resolve-failures.yml and create-readmes.yml / update-readmes.yml for AI-assisted analysis and generation.

HTTP 429 from the Models API includes a Retry-After header. Scripts read this and sleep for the indicated duration before retrying (up to 3 attempts). If the quota is fully exhausted the script logs [models-rate-limit] GitHub Models quota exhausted and skips AI analysis for that run — the workflow still exits 0 so it doesn't generate a false failure notification.

GitLab API

Used by mirror-osp-to-gitlab.yml, sync-from-gitlab.yml, and sync-to-gitlab.yml.

HTTP 429 (and occasionally 403) from GitLab includes a RateLimit-Reset header. Scripts read this and sleep until the window resets before retrying.

git push limits

Mirror scripts that push via HTTPS (mirror-to-osp.yml, mirror-osp-to-ooc.yaml, sync-to-gitlab.yml, sync-registered-imports.yml, etc.) can hit transient push rejections under load — these are not HTTP API limits but git-level errors. All push steps retry up to 3 times with linear backoff (15 s, 30 s, 45 s) before failing.

The mirror-osp-to-ooc.yaml workflow additionally uses a concurrency group (mirror-to-ooc) so concurrent runs queue rather than race, which eliminates the cannot lock ref class of push failures.

Diagnosing a rate-limit failure

1. Open the failed run log and search for [rate-limit] or rate limit exceeded.
2. The log line includes the HTTP status, sleep duration, and attempt number.
3. If all 3 retries were exhausted the next scheduled run will succeed automatically — primary limits reset hourly, secondary limits within ~60 s.
4. If failures persist across multiple scheduled runs, check that SYNC_TOKEN is valid (gh auth status) and has the required scopes (repo, workflow, admin:org).

GitHub Actions runner minutes

Free tier: 2,000 min/month (Linux, resets 1st of each month). All jobs use ubuntu-latest (1× multiplier). At the current schedule density (~7 hourly workflows), this repo exceeds the free tier. A paid plan or self-hosted runner is required.

Symptoms of exhaustion: ubuntu-latest jobs queue indefinitely, 0 in-progress runs, no runners active. Check via Settings → Billing → Actions.

Recovery: Cancel all queued runs (they will never start), then wait for the monthly reset or add a self-hosted runner.

# Bulk cancel all queued runs (requires API quota)
gh api "repos/Interested-Deving-1896/fork-sync-all/actions/runs?per_page=100" \
  --jq '[.workflow_runs[] | select(.status=="queued") | .id] | .[]' | \
  xargs -I{} gh api -X POST \
    "repos/Interested-Deving-1896/fork-sync-all/actions/runs/{}/cancel"

Concurrency groups and stuck runs

All workflows use cancel-in-progress: true. A newer run always supersedes a queued one, preventing permanent queue buildup when runner minutes are exhausted mid-job.

Detecting stuck runs:

gh api "repos/Interested-Deving-1896/fork-sync-all/actions/runs?per_page=100" \
  --jq '[.workflow_runs[] | select(.status=="queued")] | length'

workflow_run trigger cost

workflow_run fires on every completed event regardless of conclusion. All listeners in this repo gate at the job level so they exit immediately (no runner cost) when the upstream conclusion doesn't match:

• Content processors (create-readmes, update-readmes, inject-badges, translate-readmes, lts-readmes, mirror-osp-to-gitlab): gate on conclusion == 'success'
• Watchdogs (mirror-orgs-watchdog): gate on conclusion == 'failure'

See DOCS/OPERATIONS.md for the full operational reference: quota tables, schedule summary, self-hosted runner setup, and quick-reference reset times.

GitLab subgroups

11 subgroups under gitlab.com/openos-project, ~150 repos mirrored:

Subgroup IDs and repo assignments are in config/gitlab-subgroups.yml.

Mirror chain timing

:00  mirror-to-osp.yml          Interested-Deving-1896 → OSP
:05  sync-pieroproietti          pieroproietti forks fast-path
:10  quota-reserve.yml           Cancel low-priority runs if quota < 1000
:15  queue-manager.yml           Deduplicate queued runs
:15  mirror-osp-to-ooc.yaml      OSP → OOC  (per-repo, injected by setup-osp-mirrors)
:23  upstream-prs.yml            OOC/OSP PRs → Interested-Deving-1896
:30  mirror-osp-to-gitlab.yml    OSP → GitLab openos-project
:45  upstream-commits.yml        Direct OSP/OOC commits → PRs in Interested-Deving-1896
:55  sync-registered-imports     External platform imports re-sync

Full chain (validate-config → full-chain-flush) runs on every push to main.