AGENTS.md

Conventions, patterns, and known pitfalls for AI agents working in this repo.

Repository overview

fork-sync-all is the control plane for the Interested-Deving-1896 GitHub org. It mirrors repos into OpenOS-Project-OSP (GitHub) and then to openos-project (GitLab), manages READMEs across ~49 OSP-bound repos, syncs upstream forks, and runs org-wide maintenance workflows.

Key config files:

• config/gitlab-subgroups.yml — single source of truth for GitLab subgroup placement
• registered-imports.json — upstream repos to keep in sync
• scripts/ — all automation scripts
• .github/workflows/ — GitHub Actions workflows

Key directories:

• vendor/ — third-party components hosted/deployed by fork-sync-all (e.g. infra-dashboard). Everything in scripts/ is first-party automation. Do not move scripts into vendor/.

GitHub API quota

Both GH_TOKEN and SYNC_TOKEN belong to the same user (ID 202036334) and share the same 5000 req/hr REST bucket. Treat them as one pool.

• raw.githubusercontent.com fetches do not count against the quota
• GraphQL counts as 1 call regardless of how many repos are queried
• The quota pre-flight in workflows uses MIN_QUOTA (typically 1000–1500) to skip runs when the bucket is too low; quota-monitor.sh retries after reset

When quota is at 0, avoid any gh api, curl .../api.github.com/..., or gh_get calls. Check reset time with:

curl -sf -H "Authorization: token $SYNC_TOKEN" \
  "https://api.github.com/rate_limit" | jq '{remaining, reset: (.resources.core.reset | todate)}'

Script conventions

All logging helpers must write to stderr

Every script defines some combination of info(), warn(), dry(), and log(). All must use >&2:

info() { echo "[script-name] $*" >&2; }
warn() { echo "[warn] $*" >&2; }
dry()  { echo "[dry-run] $*" >&2; }
log()  { echo "[$(date -u '+%H:%M:%S')] $*" >&2; }

Why this matters: Several functions are called inside $(...) subshell captures where their stdout becomes the captured value (e.g. README content, repo lists, API responses). Any logging call without >&2 inside such a function will corrupt the captured data.

This applies to includes/gh-api.sh too — merge_upstream() status messages must go to stderr since callers may capture its output via result=$(merge_upstream ...).

Known functions called inside $(...) captures — never emit to stdout inside these:

• rewrite_readme() in update-readmes.sh
• fill_missing_sections() in update-readmes.sh
• build_readme() in create-readmes.sh
• generate_*() functions in update-readmes.sh
• merge_upstream() in scripts/includes/gh-api.sh

YAML parsing

Always use yaml.safe_load — never hand-rolled regex/indent parsers:

import yaml
with open(config_path) as f:
    config = yaml.safe_load(f)
subgroups = config.get("subgroups", {}) or {}

This applies to gitlab-subgroups.yml parsing in all scripts.

includes/ scripts

scripts/includes/budget.sh, scripts/includes/gh-api.sh, and scripts/includes/quota-instrument.sh are sourced by many scripts and workflows. Changes there have broad impact.

• budget.sh — provides budget_init, budget_check, budget_report, osp_priority_repos, and workflow_min_quota. The latter reads per-workflow min_quota from config/workflow-quota-costs.yml.
• gh-api.sh — provides gh_api, gh_get, gh_api_graphql, merge_upstream, get_default_sha. All status messages use >&2. Guard against double-sourcing is in place (_GH_API_LOADED). gh_get URL is a convenience GET wrapper around gh_api with full retry and reset-aware backoff — the canonical implementation that individual scripts should migrate to (see consolidation note below).

gh_get / gh_api consolidation (complete)

All scripts now source includes/gh-api.sh for gh_get. The three tiers that existed during migration have been fully consolidated:

All new scripts should source includes/gh-api.sh and use gh_get directly. Do not define a local gh_get() in any new script.

• quota-instrument.sh — provides qi_begin / qi_end for measuring REST quota consumption per workflow run. Wire into the main job step of any workflow you want to instrument. Writes a structured HTML comment to GITHUB_STEP_SUMMARY that update-quota-costs.yml parses weekly to compute observed p50/p95 values.

REST → GraphQL conversion

Prefer GraphQL over paginated REST for any loop that fetches the same data for multiple repos. GraphQL counts as 1 REST call regardless of how many repos are queried.

Standard pattern for org repo lists:

result=$(curl -sf \
  -H "Authorization: token ${GH_TOKEN}" \
  -H "Content-Type: application/json" \
  "${GH_API}/graphql" \
  -d "{\"query\":\"{ organization(login: \\\"${ORG}\\\") { repositories(first: 100) { nodes { name } pageInfo { hasNextPage endCursor } } } }\"}" \
  2>/dev/null || echo "{}")
echo "$result" | python3 -c "
import json,sys
d=json.load(sys.stdin)
for n in d.get('data',{}).get('organization',{}).get('repositories',{}).get('nodes',[]):
    print(n['name'])
" 2>/dev/null

Prefetch pattern for per-repo metadata (existence, pushedAt, README): Batch all repos into a single GraphQL call using aliases, populate an associative array, then read from the cache in the loop — zero REST calls per repo:

declare -A _REPO_EXISTS=()
# ... build aliases, fire one GraphQL call, populate _REPO_EXISTS ...
# In the loop:
[[ -z "${_REPO_EXISTS[$repo]:-}" ]] && continue  # skip non-existent repos

See sync-registered-imports.sh (prefetch_repo_metadata), mirror-releases.sh (prefetch_upstream_existence), and inject-badges.sh (list_gh_repos + _README_CACHE) for reference implementations.

What cannot be converted to GraphQL:

• check-runs and statuses endpoints — not exposed in GraphQL
• actions/workflows and actions/secrets — not in GraphQL
• Write operations (create repo, push file, cancel run) — REST only

Tree fetches

Use ?recursive=1 on the git trees endpoint to get all file paths in one call, then check membership with grep -qxF before fetching individual files:

tree_json=$(gh_get "${GH_API}/repos/${owner}/${repo}/git/trees/HEAD?recursive=1")
tree_paths=$(echo "$tree_json" | jq -r '.tree[] | select(.type=="blob") | .path')
echo "$tree_paths" | grep -qxF "package.json" && # file exists, fetch it

Never probe file existence with per-file /contents/ calls in a loop.

YAML-safe shell in run: blocks

GitHub Actions run: blocks are YAML block scalars. The YAML parser processes the file before the shell runner sees it, so certain shell constructs break parsing even though they would be valid bash.

Patterns that break YAML — never use these inside run: blocks:

Safe alternatives:

# Multi-line python: collapse to one line
repos=$(python3 -c "import yaml; d=yaml.safe_load(open('config/x.yml')); print(' '.join(d.get('repos',[])))")

# Multi-line variable: use printf into a temp file
printf 'line1\nline2\n' > /tmp/body.txt

# Multi-line commit message: ANSI-C quoting
git commit -m $'subject\n\nbody line 1\nbody line 2'

# Or chained -m flags (each becomes a paragraph)
git commit -m "subject" -m "body paragraph"

# Heredoc end-marker: use a non-YAML-keyword name
cat > file.yml << 'CONFIG_EOF'
...
CONFIG_EOF

The validator catches these: python3 scripts/validate-workflow-guards.py runs a YAML parse check across all 75 workflow files. Run it after editing any workflow. The full-suite parse check is also embedded in validate-config.yml.

Workflow patterns

Queue and quota management

Three workflows protect the system from quota exhaustion cascades and runner starvation:

Priority tiers — single source of truth in config/workflow-priority-tiers.yml:

• Tier 1 CRITICAL — never cancelled (token rotation, queue/reserve management, config validation)
• Tier 2 HIGH — mirror chain, sync operations
• Tier 3 MEDIUM — READMEs, CI checks (default for unknown workflows)
• Tier 4 LOW — translation, dep graph, maintenance (cancelled first)

When adding a new workflow, add it to both:

1. config/workflow-priority-tiers.yml — by workflow name: field (not filename). Both queue-manager.sh and quota-reserve.sh load tiers from this file at runtime — no script edits needed.
2. config/workflow-sync.yml — under github_only (most workflows) or paired (if it has a GitLab CI counterpart). validate-workflow-guards.py warns on any workflow file not listed in either section.

Run python3 scripts/validate-workflow-guards.py after adding any workflow to confirm zero warnings.

dispatch-and-wait.sh exit codes:

• 0 — workflow completed successfully
• 1 — workflow failed or timed out
• 2 — workflow was cancelled (by queue-manager or manually) — retriable, not a real failure

full-chain-flush.yml and critical-deploy.sh both handle exit 2 with a warning rather than aborting.

Concurrency groups

All workflows triggered by schedule or workflow_run must have a concurrency group to prevent queue pile-ups:

concurrency:
  group: workflow-name
  cancel-in-progress: true

workflow_run triggers

Each workflow should have at most one workflow_run upstream trigger. Multiple triggers cause fan-out: N completions × M downstream workflows = queue explosion.

Every name in workflow_run.workflows: must exactly match the name: field of a workflow file that actually exists in .github/workflows/. A phantom name causes the trigger to fire on every push but the job fails immediately — GitHub cannot resolve the upstream workflow. validate-workflow-guards.py (Check 5) catches this automatically.

Quota pre-flight

All hourly/daily/frequent workflows include a quota pre-flight step before doing any API work. The step sets skip=true when remaining < MIN_QUOTA and subsequent steps check if: steps.quota.outputs.skip == 'false'.

Quota cost registry

config/workflow-quota-costs.yml is the single source of truth for per-workflow REST call cost estimates. It drives:

• quota-reserve.sh — cost-aware cancellation (min_quota per workflow)
• budget.sh workflow_min_quota() — pre-flight helper for self-skipping
• DOCS/quota-costs.md — rendered documentation in mdBook

Phase 1 values are code-audit estimates (basis: code-audit). Phase 2 (update-quota-costs.yml, weekly) replaces them with observed p50/p95 values (basis: observed) once ≥5 run samples exist per workflow.

When adding a new workflow that makes significant REST calls, add it to config/workflow-quota-costs.yml with estimated min_quota, cost_low, cost_mid, cost_high, and basis: code-audit. Wire qi_begin/qi_end from scripts/includes/quota-instrument.sh into its main job step so Phase 2 can measure it automatically.

Instrumented workflows (Phase 2 active):

• Sync All Forks
• Inject Built-with-Ona Badges
• Reconcile Org References
• Cleanup Stale Branches
• Check OSP-Bound CI Status
• Sync Registered Imports
• Mirror Interested-Deving-1896 → OSP
• Pre-Mirror CI Gate
• Verify Mirror Integrity
• Post-Flush Verification
• Pipeline Telemetry
• Translate Docs

Path filters + required status checks (gate job pattern)

When a workflow uses path filters to skip jobs on irrelevant changes, required status checks will block PRs indefinitely if the filtered jobs never run. Fix this with a gate job that always runs and reflects the filtered outcomes:

jobs:
  changes:
    name: Detect changes
    runs-on: ubuntu-latest
    outputs:
      shell: ${{ steps.filter.outputs.shell }}
    steps:
      - uses: actions/checkout@v4
      - uses: dorny/paths-filter@v3
        id: filter
        with:
          filters: |
            shell:
              - '**/*.sh'

  lint:
    name: ShellCheck
    needs: changes
    if: needs.changes.outputs.shell == 'true'
    runs-on: ubuntu-latest
    steps: [...]

  # Set THIS as the required status check — not the individual jobs above.
  ci-required:
    name: CI Required
    runs-on: ubuntu-latest
    needs: [lint]
    if: always()
    steps:
      - name: Check results
        run: |
          if echo "${{ join(needs.*.result, ' ') }}" | grep -qw "failure"; then
            exit 1
          fi

Branch protection must require CI Required (the gate job name), not the individual filtered job names. If the individual names are listed as required checks, PRs that skip those jobs will be permanently blocked.

Applied in: btrfs-dwarfs-framework/.github/workflows/ci.yaml

Template sync profiles

config/template-consumers.yml controls which repos receive automatic file updates from sync-template.yml. Each consumer has a profile that determines what gets injected.

Profile assignments

Critical rule

Never assign mirror profile to consumer repos. The mirror profile injects the full fork-sync-all mirror/sync suite (60+ workflow files, 100+ scripts) into repos that are targets of the mirror chain, not operators of it. This causes template pollution — files that have no purpose in the target repo and clutter its .github/workflows/ and scripts/ directories.

Template pollution cleanup

If a repo has been polluted by the mirror profile:

1. Check which files don't belong:

for f in .github/workflows/*.yml; do
  grep -q "SYNC_TOKEN\|openos-project\|mirror-to-osp\|registered-imports" "$f" \
    && echo "POLLUTION: $(basename $f)" \
    || echo "native:    $(basename $f)"
done

2. Remove them with git rm --cached and commit:

git rm --cached .github/workflows/add-mirror-repo.yml  # etc.
git commit -m "chore: remove fork-sync-all template pollution"

3. Delete the untracked files from disk:

git status --short | grep "^??" | awk '{print $2}' | xargs rm -f

4. Trigger cleanup-pollution.yml (workflow_dispatch) to clean remaining consumer repos automatically.

Repos cleaned of mirror pollution (2026-06-06)

• KPort — 74 files removed
• btrfs-dwarfs-framework — 133 files removed
• All other infra-core consumers — cleaned via cleanup-pollution.yml

Queue pile-up pattern

Workflows that trigger on .github/workflows/** (e.g. validate-config, update-workflow-triggers-doc) must have concurrency: cancel-in-progress: true to prevent stacking. Without it, rapid pushes create a queue of identical runs that consume quota on every reset, causing a deadlock where the queue can't drain because quota is always 0.

concurrency:
  group: workflow-name-${{ github.ref }}
  cancel-in-progress: true

OSP-bound repo list

The canonical list of ~49 repos that are mirrored to GitLab lives in config/gitlab-subgroups.yml. Parse it with yaml.safe_load — do not hardcode repo names anywhere else.

To get the list in bash:

python3 -c "
import yaml
data = yaml.safe_load(open('config/gitlab-subgroups.yml'))
for sg in data.get('subgroups', {}).values():
    for repo in (sg.get('repos') or []):
        print(repo)
"

GitLab subgroup IDs

Parent group: openos-project on GitLab (gitlab.com/openos-project)

All IDs are authoritative — sourced from config/gitlab-subgroups.yml. Do not hardcode them elsewhere.

README management

AI marker format

<!-- AI:start:section-name -->
content
<!-- AI:end:section-name -->

Eight AI-owned sections: what-it-does, architecture, ci, mirror-chain, contributors, origins, resources, license.

Human-owned sections (Install, Usage, Configuration, License) never get AI markers — they get placeholder HTML comments on first creation.

Three modes in update-readmes.sh

• rewrite — no AI markers present → build full template from scratch
• fill — some markers present but missing sections → inject missing ones
• update — all markers present → regenerate AI section content

check-readme-render.sh

Run this against any README before committing. It catches: leaked log lines, unclosed fences, unclosed AI markers, empty sections, missing H1, broken tables, bare [text] links, raw angle brackets.

bash scripts/check-readme-render.sh path/to/README.md

GitLab CI variables

These must be set as masked CI/CD variables in the openos-project/fork-sync-all GitLab project settings (not GitHub secrets):

Headroom proxy

A context compression proxy runs on port 8787 (started automatically via .ona/automations.yaml). To use it with Claude:

ANTHROPIC_BASE_URL=http://localhost:8787 claude
# or
headroom wrap claude

Check savings: headroom stats

Token rotation

Tracked tokens

The "PAT name" column is the display name shown at github.com/settings/tokens (classic).

| ACTIVITYSMITH_API_KEY | n/a (external service) | ActivitySmith API | ActivitySmith | unknown | full-chain-flush.yml (live activity tracking) — optional, skipped if unset | manual | | ACTIVITYSMITH_CHANNELS | n/a (external service) | ActivitySmith channel IDs | ActivitySmith | n/a | full-chain-flush.yml — optional, skipped if unset | manual | | ANTHROPIC_API_KEY | n/a (external service) | Anthropic API | Anthropic | n/a | validate-config.yml (AgentShield scan) — optional, skipped if unset | manual |

How to rotate a repo secret (SYNC_TOKEN, GH_SYNC_TOKEN, etc.)

1. Generate a new PAT at https://github.com/settings/tokens
2. Go to rotate-token.yml → Run workflow
3. Select the secret name from the dropdown
4. Paste the new token value into the token_value field
5. Leave validate checked — it confirms the token works before finishing
6. After the run completes, update the expiry date in this table

How to rotate an OSP org secret (ORG_MIRROR_OSP_TO_OOC, MIRROR_TOKEN)

OSP org secrets live in OpenOS-Project-OSP and require a token with admin:org on that org. SYNC_TOKEN only covers Interested-Deving-1896.

The rotate-token.yml workflow resolves the OSP token automatically in this priority order:

Option 1 — GitHub App (preferred, permanent)

A GitHub App installation token never expires and has fine-grained permissions.

One-time setup:

1. Create a GitHub App at https://github.com/settings/apps/new
   • Name: fork-sync-all-osp-rotator (or similar)
   • Permissions: Organization secrets → Read and write
   • Uncheck everything else
2. Install the App on OpenOS-Project-OSP org
3. Note the App ID (shown on the app settings page)
4. Generate a private key (PEM format) from the app settings page
5. Add two repo secrets to Interested-Deving-1896/fork-sync-all:
   • OSP_APP_ID — the numeric App ID
   • OSP_APP_PRIVATE_KEY — the full PEM contents (including header/footer)
6. Run rotate-token.yml — it will use the App automatically

Option 2 — Dedicated PAT (bridge until App is set up)

1. Generate a new PAT at https://github.com/settings/tokens with:
   • admin:org scope
   • Authorized for OpenOS-Project-OSP org (SSO authorize if required)
2. Add it as repo secret OSP_ADMIN_TOKEN in Interested-Deving-1896/fork-sync-all
3. Run rotate-token.yml — it will use OSP_ADMIN_TOKEN automatically

Option 3 — Manual fallback

If neither OSP_APP_* nor OSP_ADMIN_TOKEN is set, the workflow prints the exact error and the two options above. You can also update manually:

1. Generate a new PAT with admin:org on OpenOS-Project-OSP
2. Go to OSP org secrets and update the secret value directly
3. Update the expiry date in scripts/token-monitor.sh (OSP_ORG_SECRETS array) and in the table above

⚠️ Upcoming rotations (as of 2026-06-08):

• ADD_MIRROR_REPO_SYNC — expires 2026-08-13 (66 days). token-health.yml will open an issue around 2026-06-29.
• MIRROR_TOKEN / ORG_MIRROR_OSP_TO_OOC — expire 2026-09-01 (85 days). Alert ~2026-07-17.
• SYNC_TOKEN — expires 2026-09-02 (86 days). Alert ~2026-07-18.
• GH_SYNC_TOKEN / OSP_ADMIN_TOKEN — expire 2026-09-03 (87 days). Alert ~2026-07-19.

Automated monitoring

token-health.yml runs weekly (Monday 09:00 UTC) and warns at 45 days before expiry. When a token needs attention it opens a GitHub issue labelled token-monitor. Run it manually at any time to get a current status report.

vendor/ conventions

Agnostic-by-default rule

Everything imported into vendor/ must be deployment-agnostic. No distro names, org-specific URLs, org/repo slugs, or arch/repo paths may appear as hardcoded fallback values in shell ${VAR:-...}, YAML || '...', or TypeScript ?? '...' expressions. All deployment-identity values belong in CI variables or repo vars set per deployment.

Enforcement

scripts/check-vendor-agnostic.sh scans a vendor directory and exits 1 on violations:

bash scripts/check-vendor-agnostic.sh vendor/infra-dashboard   # specific component
bash scripts/check-vendor-agnostic.sh vendor                   # all of vendor/

enforce-agnostic-vendor.yml runs this automatically on every push/PR touching vendor/.

To suppress a specific line that is intentionally non-agnostic:

SOME_VAR="${SOME_VAR:-specific-value}"  # check-vendor-agnostic: ignore

What the checker flags vs. allows

Flagged (deployment-identity):

• Public URLs as fallbacks: ${VITE_ENDPOINT_URL:-https://api.myorg.com}
• Org/repo slugs: ${MIRRORLIST_REPO:-MyOrg/my-repo}
• Arch/repo paths: ${MIRROR_REPO_PATHS:-x86_64/core,x86_64/extra}
• Bare distro names: ${DISTRO:-cachyos}, ${DISTRO:-ubuntu}

Allowed (generic defaults):

• Localhost dev URLs: ${API_URL:-http://localhost:5862}
• Generic relative paths: ${MIRRORLIST_PATH:-mirrorlist/mirrorlist}
• Single-word tokens: ${LOG_LEVEL:-info}, ${ENV:-production}
• UI strings: ${APP_NAME:-Infra Dashboard}

Workflow integrations

import-repo → immediate sync

When ongoing_sync=true, import-repo.sh writes to registered-imports.json and then immediately dispatches sync-registered-imports.yml with repo_filter=<name> and force_sync=true. This avoids the up-to-6h wait for the scheduled run to pick up the new entry.

If the dispatch fails (quota, permissions), it falls back gracefully — the entry is still registered and will sync on the next scheduled run.

merge-to-monorepo → OSP mirror chain

merge-to-monorepo.yml has a mirror_monorepo boolean input (default: false). When set, it dispatches add-mirror-repo.yml for the newly created monorepo after a successful merge, entering it into the standard OSP mirror chain automatically.

Known pitfalls

• fill_missing_sections case statement — must handle all 8 AI sections. If you add a new section to ALL_AI_SECTIONS, add it to the case in fill_missing_sections, rewrite_readme, and the update mode loop.

• sync-registered-imports.sh does not create repos — ensure_gh_repo() handles creation now, but the target repo must be reachable via the GitHub API. New entries in registered-imports.json will auto-create the repo on first run.

• GitLab mirror chain — I-D-1896 → OpenOS-Project-OSP (GitHub) → openos-project (GitLab). Adding a repo to gitlab-subgroups.yml is required for GitLab mirroring. Adding to registered-imports.json is required for upstream sync. Both are independent — a repo can be in one without the other.

• _inter_repo_sleep in update-readmes.sh — quota-aware pacing. No delay when quota > 2000; scales to 30s when < 500. The cached _quota_remaining variable is decremented by 10 per repo to trigger re-checks before actually hitting the threshold.