CI/CD Validation

Validating your .shellfirm.yaml in CI ensures that policy files are syntactically correct and that safety rules are not accidentally broken by code changes.

Validating policy syntax

The shellfirm policy validate command checks that a .shellfirm.yaml file is valid:

shellfirm policy validate

This verifies:

• YAML syntax is valid
• The version field is present and correct
• Pattern IDs in deny lists and overrides are properly formatted
• Custom check regex patterns compile successfully
• Challenge types are valid (Math, Enter, Yes)
• Severity levels are valid (Info, Low, Medium, High, Critical)

Showing the active policy

To display the policy that would be applied in the current directory:

shellfirm policy show

This outputs the merged policy including deny lists, overrides, and custom checks.

GitHub Actions

Validate policy on every PR

name: Validate shellfirm policy
on:
  pull_request:
    paths:
      - '.shellfirm.yaml'
      - '**/. shellfirm.yaml'

jobs:
  validate:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Install shellfirm
        run: |
          curl -L https://github.com/kaplanelad/shellfirm/releases/latest/download/shellfirm-x86_64-unknown-linux-gnu.tar.gz | tar xz
          sudo mv shellfirm /usr/local/bin/

      - name: Validate policy
        run: shellfirm policy validate

Enforce policy presence

Require that a .shellfirm.yaml exists in the repository:

name: Enforce shellfirm policy
on:
  pull_request:

jobs:
  check-policy:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Check policy file exists
        run: |
          if [ ! -f .shellfirm.yaml ]; then
            echo "::error::Missing .shellfirm.yaml policy file. See https://shellfirm.dev/docs/team-policies/overview"
            exit 1
          fi

      - name: Install shellfirm
        run: |
          curl -L https://github.com/kaplanelad/shellfirm/releases/latest/download/shellfirm-x86_64-unknown-linux-gnu.tar.gz | tar xz
          sudo mv shellfirm /usr/local/bin/

      - name: Validate policy
        run: shellfirm policy validate

Validate all policies in a monorepo

name: Validate all shellfirm policies
on:
  pull_request:
    paths:
      - '**/.shellfirm.yaml'

jobs:
  validate:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Install shellfirm
        run: |
          curl -L https://github.com/kaplanelad/shellfirm/releases/latest/download/shellfirm-x86_64-unknown-linux-gnu.tar.gz | tar xz
          sudo mv shellfirm /usr/local/bin/

      - name: Validate all policies
        run: |
          find . -name ".shellfirm.yaml" | while read policy; do
            echo "Validating: $policy"
            dir=$(dirname "$policy")
            (cd "$dir" && shellfirm policy validate)
          done

GitLab CI

validate-shellfirm-policy:
  stage: test
  before_script:
    - curl -L https://github.com/kaplanelad/shellfirm/releases/latest/download/shellfirm-x86_64-unknown-linux-gnu.tar.gz | tar xz
    - mv shellfirm /usr/local/bin/
  script:
    - shellfirm policy validate
  rules:
    - changes:
        - .shellfirm.yaml

Preventing policy weakening in PRs

While shellfirm's additive-only rule prevents runtime weakening, you may also want to prevent policy changes that remove deny-list entries or lower challenge types. This requires a code review process:

1. CODEOWNERS file -- require approval from security team for .shellfirm.yaml changes:

# CODEOWNERS
.shellfirm.yaml @security-team
**/.shellfirm.yaml @security-team

2. Branch protection rules -- require reviews for changes to policy files.

3. Custom CI check -- compare the policy in the PR against the base branch:

#!/bin/bash
# compare-policies.sh
# Check that the PR doesn't remove deny-list entries

BASE_DENY=$(git show origin/main:.shellfirm.yaml | yq '.deny[]' 2>/dev/null | sort)
PR_DENY=$(cat .shellfirm.yaml | yq '.deny[]' 2>/dev/null | sort)

REMOVED=$(comm -23 <(echo "$BASE_DENY") <(echo "$PR_DENY"))
if [ -n "$REMOVED" ]; then
  echo "::error::PR removes deny-list entries: $REMOVED"
  exit 1
fi