Future Feature: In-Pipeline Baseline Approval

Status: Proposal

Overview

This document describes a potential future feature for approving baseline updates directly in the Jenkins pipeline, eliminating the need for pull requests in certain workflows.

Current Workflow (Pull Request Based)

How it works now:

1. Visual differences detected → Pipeline creates git branch baseline-update/{BUILD_NUMBER}
2. Pushes updated baselines to branch
3. Creates pull request for review
4. Team reviews PR, approves, merges
5. Next build uses new baseline

Advantages:

• ✅ Full git history and attribution
• ✅ Code review process enforced
• ✅ Works for large teams
• ✅ Rollback capability via git

Disadvantages:

• ⚠️ Slower iteration (PR overhead)
• ⚠️ Requires git for baseline storage
• ⚠️ More complex for small teams

Future Alternative: In-Pipeline Approval

Concept

Instead of creating pull requests, pause the pipeline and ask for approval directly in Jenkins. If approved, commit and push to main branch immediately.

Use Cases

When this makes sense:

• Small, trusted teams
• Projects needing faster iteration
• Using non-git storage backends (S3, filesystem, database)
• Teams preferring streamlined workflow

When to stick with PR workflow:

• Large teams requiring code review
• Projects with strict change control
• Need for detailed review comments
• Regulatory/compliance requirements

Technical Implementation

Pattern: Baseline Update Approval Gate

stage('Approval') {
    when {
        expression { env.HAS_SCREENSHOT_UPDATES == 'true' }
    }

    options {
        timeout(time: 48, unit: 'HOURS')  // Auto-abort if not approved within 48h
    }

    input {
        message "Approve ${failedCount} baseline updates?"
        ok 'Approve and Merge'
        submitter 'qa-team,tech-leads'  // Restrict who can approve
        submitterParameter: 'APPROVER'  // Track who approved
    }

    steps {
        echo "Baseline approved by: ${env.APPROVER}"

        // Commit directly to main branch (or configured baseline branch)
        sh """
            git config user.email "jenkins@quatico.com"
            git config user.name "Jenkins (approved by ${env.APPROVER})"
            git add design-tests/baseline-images
            git commit -m "Update baseline (approved by ${env.APPROVER})"
            git push origin develop
        """
    }
}

Key Components

1. Milestone Pattern (Auto-Cancel Old Approvals)

milestone 1  // Cancel older builds waiting for approval

input message: 'Approve baseline?'

milestone 2  // Once approved, cancel any older builds

How it works:

• When build #10 reaches milestone 1, builds #1-9 waiting at milestone 1 are aborted
• Prevents accumulation of pending approvals
• Ensures only latest build needs approval

2. Submitter Tracking

input {
    message 'Approve?'
    submitterParameter: 'APPROVER'
}

// Later in commit message or Slack notification
"Baseline updated by ${env.APPROVER}"

3. Timeout Protection

options {
    timeout(time: 48, unit: 'HOURS')
}

input {
    message 'Approve within 48 hours or build will abort'
}

4. Conditional Approval (Only When Needed)

when {
    expression { env.HAS_SCREENSHOT_UPDATES == 'true' }
}

Future Enhancement: HTML Report Approval Button

Concept

Add an "Approve Baseline" button directly in the reg-cli HTML report that triggers Jenkins approval via API.

Implementation

1. Modify Report HTML Add approval button to report (custom reg-cli template or post-process HTML):

<button onclick="approveBaseline()">✅ Approve Baseline Updates</button>

<script>
async function approveBaseline() {
    const buildUrl = 'https://jenkins.../job/.../123/';
    const inputId = 'baseline-approval-gate';

    await fetch(`${buildUrl}input/${inputId}/submit`, {
        method: 'POST',
        credentials: 'include'
    });

    alert('Baseline approved!');
}
</script>

2. Jenkins Input with ID

input(
    id: 'baseline-approval-gate',  // Stable ID for API access
    message: 'Approve baseline updates'
)

3. API Endpoint

POST /job/{jobName}/{buildNumber}/input/{inputId}/submit

Benefits:

• ✅ One-click approval from report itself
• ✅ Review and approve in same UI
• ✅ No context switching to Jenkins
• ✅ Streamlined UX

Non-Git Storage Backends

Why In-Pipeline Approval Enables This

Current PR workflow requires:

• Git repository for baselines
• Git hosting with PR support (GitHub, Bitbucket, GitLab)
• Team access to git repo

In-pipeline approval enables:

• S3/Cloud Storage: Upload baselines to S3, approve in pipeline, update without git
• Filesystem: Store baselines on shared filesystem, no version control overhead
• Database: Store baseline metadata + images in database for easier querying
• Hybrid: Images in S3, metadata in database, no git needed

Example: S3 Storage Backend

stage('Update Baseline') {
    when {
        expression { env.HAS_SCREENSHOT_UPDATES == 'true' }
    }

    input {
        message "Upload ${failedCount} baseline updates to S3?"
        ok 'Upload'
        submitterParameter: 'APPROVER'
    }

    steps {
        // Upload to S3 instead of git commit
        sh """
            aws s3 sync design-tests/actual-images/ \
                s3://bfh-baselines/\${GIT_COMMIT}/baseline-images/ \
                --metadata approver=${env.APPROVER}
        """

        // Update config to point to new baseline
        sh """
            echo '${GIT_COMMIT}' > design-tests/.baseline-version
            git add design-tests/.baseline-version
            git commit -m "Update baseline pointer (approved by ${env.APPROVER})"
            git push origin develop
        """
    }
}

Migration Path

Phase 1: Current (MVP)

• Use pull request workflow
• Mature, proven approach
• Good for initial rollout

Phase 2: Optional In-Pipeline Approval

• Make approval method configurable
• Projects choose PR or in-pipeline
• Both workflows supported

Phase 3: HTML Report Button

• Add approval button to report
• Streamlined UX
• Optional enhancement

Phase 4: Alternative Storage

• Support S3, filesystem, database backends
• Git becomes optional
• More flexibility for different use cases

Configuration Design

// Future .designTests.js config
export default {
  baseline: {
    storage: 'git' | 's3' | 'filesystem' | 'custom',
    approval: {
      method: 'pull-request' | 'pipeline-input' | 'auto',
      timeout: { time: 48, unit: 'HOURS' },
      submitters: ['qa-team', 'tech-leads'],
      trackApprover: true
    }
  }
}

Security Considerations

Pull Request Workflow:

• ✅ Code review enforced
• ✅ Approval tracked in git
• ✅ Easy rollback
• ✅ Audit trail in git log

In-Pipeline Approval:

• ⚠️ No code review step
• ⚠️ Requires careful submitter restrictions
• ⚠️ Build log is audit trail (less permanent than git)
• ⚠️ Rollback requires manual git revert or restore from backup

Recommendation:

• Use PR workflow for production systems
• Use in-pipeline for dev/test environments
• Require stricter submitter lists for in-pipeline approval

Implementation Checklist

When implementing this feature:

• Add approval.method config option
• Implement input step with milestone pattern
• Add submitterParameter tracking
• Add timeout configuration
• Support alternative storage backends (S3, filesystem)
• Add report HTML button (optional enhancement)
• Document security implications
• Provide migration guide from PR workflow
• Add integration tests
• Update CLAUDE.md and README.md

References

• Jenkins Pipeline Syntax - Input
• Pipeline: Input Step
• Jenkins Pipeline Milestone
• Jenkins Remote API for Input