Skip to main content

Operations

This page is the operational entrypoint for runtime checks, deployment-adjacent verification, and recovery-oriented fast paths.

Tool / Contract Summary​

  • Runtime/contract behavior is defined in compose.yaml, config/**, and verifier scripts under scripts/**.
  • Feature-to-test mapping is canonical in MODULE_FEATURES.md (docs/MODULE_FEATURES.md).
  • Backlog and issue evidence progression is tracked in AUTONOMOUS_BACKLOG.md (docs/AUTONOMOUS_BACKLOG.md).

Current Verified State​

  • Repo validation surfaces are implemented and runnable from this repository.
  • Live verification paths exist for guardrails, OIDC bridge behavior, and contract consumers.
  • External-owner runtime dependencies remain external and must not be replaced with local shadow logic.

Quick Verify Paths​

Repo-fast:

python scripts/validate-identity-claim-vocabulary.py --contract config/identity/claim-vocabulary.v2.yaml
python scripts/validate-stackwide-claim-session-core.py --contract config/identity/stackwide-claim-session-core.v1.yaml
python scripts/validate-admitted-surface-client-contracts.py --admitted-surfaces config/clients/admitted-surfaces.v1.yaml
python scripts/verify-runtime-materialization-drift.py
python -m unittest tests.test_validate_stackwide_claim_session_core tests.test_validate_admitted_surface_client_contracts

Runtime guardrails:

python scripts/verify-runtime-guardrails-v1.py
python scripts/verify-runtime-guardrails-v1.py --live-host <host> --ssh-user <user> --remote-repo-path <path>
python scripts/verify-runtime-materialization-drift.py --live-host <host> --ssh-user <user> --remote-repo-path <path>
python scripts/validate-plane-oidc-runtime.py --base-url http://<internal-runtime-redacted>:18091 --require-login-redirect
python scripts/host_materialize_issue281_heddle_owner_checkout_reconcile.py --host <live-host> --ssh-user <ssh-user> --remote-repo-path <remote-repo-path>
python scripts/verify_issue281_heddle_owner_checkout_reconcile.py --host <live-host> --ssh-user <ssh-user> --remote-repo-path <remote-repo-path>

Bridge/runtime checks:

python scripts/validate-plane-oidc-runtime.py --base-url <plane-url> --insecure
python scripts/validate-loom-oidc-runtime.py --base-url <loom-url> --insecure

Reed context gateway service client reconcile, Host172 production parameters:

python scripts/reconcile-reed-context-gateway-service-client.py \
  --fabric-readiness-url `api/v`1/identity/agent-federation/readiness \
  --fabric-read-token-env AGENT_RECONCILE_FABRIC_AUTH_TOKEN
  • --fabric-readiness-url must point at the agent-federation readiness endpoint above, which returns current_fabric_revision. The generic jhf-fabric-api /ready endpoint does not return this field and is the wrong target.
  • --fabric-read-token-env names an environment variable; on Host172 its value comes from the repo-owned .env file in this checkout.
  • The reconciler also talks to the internal Keycloak target http://<internal-runtime-redacted>:18090, whose admin password on Host172 lives at /var/lib/jhf-heddle/keycloak-admin-password.

Readiness / Drift / Monitoring​

  • Contract/readiness checks are fail-closed where upstream truth is required.
  • Drift and stale-revision posture is validated via dedicated scripts.
  • Runtime materialization must be compared across repo truth, active compose config, container env, and app readback instead of trusting only one layer.
  • Operational evidence should be bounded (timeouts, limited log windows, no unbounded followers).

Recovery and Fail-Closed Notes​

  • Use bounded diagnostics only; avoid indefinite streams and high-pressure loops.
  • Re-runs must be idempotent; verify a second execution path explicitly.
  • On missing or contradictory upstream truth, preserve fail-closed behavior instead of local fallback authoring.
  • runbooks/BOOTSTRAP.md (docs/runbooks/BOOTSTRAP.md)
  • runbooks/BACKUP_RESTORE.md (docs/runbooks/BACKUP_RESTORE.md)
  • runbooks/SECRET_ROTATION.md (docs/runbooks/SECRET_ROTATION.md)
  • HOST_DOCKER_LOG_GUARDRAILS.md (docs/HOST_DOCKER_LOG_GUARDRAILS.md)

License: AGPLv3 Project: https://helpifyr.com

Workspace Git/Scan Guardrails (Mandatory)​

  • Gitea is Source of Truth; local Windows workspaces are disposable working copies.
  • Never run Codex sessions on <local-workspace-redacted> root; always use a concrete repo path.
  • Limit active repo sessions to 2-3 in parallel.
  • Before each run in a repo: git fetch --prune, git checkout <branch>, git pull --ff-only.
  • No background git discovery loops (git status, git ls-files, worktree scans) without explicit scoped need.
  • Automation scripts must run repo-scoped only, never global over <local-workspace-redacted>.

scan_and_fix Standard​

  • scripts/scan_and_fix.sh must enforce runner timeout + single-run lock + .env fallback to <local-env-redacted> and <local-env-redacted>.
  • scripts/scan_open_issues_repo_only.sh must exist and query only current repo open issues via Gitea API.
  • On Windows workers the nested live-run dispatch must resolve a usable Git Bash binary and avoid the broken WindowsApps WSL shim; set SCAN_AND_FIX_BASH_EXE explicitly when auto-detection is insufficient.
  • Normal operational run:
    • bash scripts/scan_and_fix.sh --executor-cmd "<bounded executor command>"
  • Default queue scope is the live open Gitea issue set for JaddaHelpifyr/jhf-heddle, bounded to 20 issues unless overridden.
  • blocked_external issues are intentionally deprioritized behind repo-owned work in the same queue.
  • Repo-scoped dry-run:
    • bash scripts/scan_and_fix.sh --dry-run --max-issues 20
  • Repo-scoped bounded live dispatch:
    • bash scripts/scan_and_fix.sh --max-issues 20 --executor-cmd "<bounded executor command>"
  • No root-wide queue scans are allowed; the helper must only address JaddaHelpifyr/jhf-heddle.

Workspace Hygiene​

  • Daily cleanup: stale _worktrees/*, _tmp/*, test-results/*, large temporary artifacts.
  • Weekly cleanup: stale local branches/worktrees.
  • Never leave valuable artifacts as untracked files in workspace root.

Dirty-State Policy​

  • Dirty state is allowed while actively implementing.
  • Before new scan/automation runs: commit/stash, or use a dedicated worktree.
  • Never propagate dirty_unknown states.

Incident Playbook (git.exe storm)​

  1. Identify parent of git.exe (usually one Codex.exe).
  2. Stop only the offending process tree.
  3. Restart session on concrete repo path.
  4. Reduce parallel sessions.
  5. Verify git.exe count drops within 30-60s.