Why Code Auditing Matters

Modern challenges:

• Rapid delivery cycles → shorter review time

• Complex codebases → harder to maintain

• Security threats → constant evolution

• Technical debt → accumulates silently

Solution: Systematic, AI-assisted auditing

Course Objectives

By the end of this lecture:

1. Understand technical debt types and measurement

2. Apply audit frameworks (ISO/IEC 25010, CWE/OWASP)

3. Use Claude Code for comprehensive audits

4. Automate audits in CI/CD pipelines

5. Implement responsible AI auditing practices

Key Concept

"Every line of code is a liability until proven safe."

Auditing philosophy:

• Prevention over reaction

• Continuous over sporadic

• Automated over manual

• Contextual over rule-based

Definition (Ward Cunningham, 1992)

"Not refactoring is like taking a loan—you gain speed now, pay interest later."

The debt metaphor:

• Principal: Suboptimal code written initially

• Interest: Extra effort required for changes

• Compound interest: Debt makes future debt worse

Types of Technical Debt

mindmap
  root((Technical<br/>Debt))
    Code Debt
      Duplication
      Complexity
      Style inconsistency
    Design Debt
      Poor modularity
      Wrong abstractions
      Tight coupling
    Test Debt
      Missing tests
      Flaky tests
      Low coverage
    Documentation Debt
      Missing docs
      Outdated docs
      Knowledge silos
    Security Debt
      Known vulnerabilities
      Outdated dependencies
      Weak authentication

Code Debt

Examples:

1. Duplication

   def calc_discount_electronics(price):
       if price > 1000: return price * 0.15
       # ... repeated logic
   
   def calc_discount_books(price):
       if price > 1000: return price * 0.15
       # ... repeated logic

2. High complexity

   • Cyclomatic complexity > 10

   • Deeply nested conditionals

Design Debt

Poor modularity:

class GodClass:
    def handle_user_auth(self): ...
    def process_payment(self): ...
    def send_email(self): ...
    def generate_report(self): ...
    def update_database(self): ...
    # 50+ methods in one class!

Better: Separate concerns into focused classes

Test Debt

Missing test coverage:

# Production code
def transfer_funds(from_account, to_account, amount):
    # Critical financial logic
    # NO TESTS! 😱
    pass

Impact:

• Fear of refactoring

• Regression bugs

• Slow development

Measuring Technical Debt

Static metrics:

• Cyclomatic complexity (McCabe)

• Maintainability index

• Lines of code (LOC)

• Code duplication percentage

Tools:

• SonarQube: Debt ratio, code smells

• Radon: Python complexity

• CodeClimate: Maintainability scores

AI-Enhanced Debt Measurement

Claude Code prompt:

Analyze this repository for technical debt.

Rank top 5 debt items by:
  impact × remediation_effort

Consider:
- Complexity, duplication, test coverage
- Dependencies, documentation
- Security vulnerabilities

Output: Prioritized list with estimated hours

Benefit: Context-aware prioritization

Audit ≠ Review

Audit Dimensions

The Four Pillars:

1. Security: Vulnerabilities, exploits

2. Quality: Maintainability, complexity

3. Standards: Style guides, conventions

4. Reliability: Error handling, edge cases

Goal: Holistic assessment, not just "does it work?"

Audit Strategies

1. Static Analysis

• AST inspection

• Pattern matching

• Type checking

2. Dynamic Analysis

• Runtime instrumentation

• Test coverage measurement

• Performance profiling

3. Semantic Analysis

• LLM reasoning (Claude Code)

• Logic anomaly detection

• Context-aware validation

Hybrid Approach: Best of All Worlds

graph TB
    Code[Source Code] --> Static[Static Tools]
    Code --> Dynamic[Dynamic Analysis]

    Static --> Results[Raw Findings]
    Dynamic --> Results

    Results --> AI[Claude Code]
    AI --> Context[Context Analysis]
    Context --> Report[Prioritized Report]

    style AI fill:#FF6B35
    style Report fill:#4CAF50

Advantage: Reduced false positives, better prioritization

Architecture Recap

Capabilities:

• 200k token context window

• Multi-file reasoning

• MCP tool integration

• Structured output (JSON/Markdown)

Perfect for:

• Cross-file analysis

• Dependency tracking

• Logic flow verification

Typical Audit Prompts

1. Code Smells

Analyze repo for code smells.
Rate severity 1-5.
Focus: duplication, complexity, poor naming.

2. Security

Scan for:
- SQL injection
- XSS vulnerabilities
- Hardcoded secrets
Output: security_audit.json

Typical Audit Prompts (cont.)

3. Standards Compliance

Audit for PEP 8 and PEP 484 compliance.
Suggest auto-fixable changes.
Highlight breaking changes separately.

4. Documentation Debt

List all public functions lacking docstrings.
Categorize by module.
Estimate effort to document.

Integration Scenarios

Local development:

claude code audit src/ --rules security,quality

MCP integration:

# Via MCP tool
result = mcp_call("code-audit", {
    "target": "src/",
    "tools": ["ruff", "bandit"],
    "severity_threshold": "important"
})

CI/CD:

# GitHub Actions
- name: Claude Audit
  run: python tools/scripts/run_audit.py

Advanced Use: Agent Chains

Multi-agent workflow:

graph LR
    A[Analysis Agent] -->|issues.json| B[Fix Agent]
    B -->|patches/| C[Validation Agent]
    C -->|report.md| D[Human Review]

    style A fill:#FF6B35
    style C fill:#4CAF50

1. Analysis: Detect issues

2. Fix: Propose remediation

3. Validation: Test fixes

4. Review: Human approval

Audit Memory: Trend Detection

Track debt over time:

audits/
├── 2025-01-01_audit.json
├── 2025-02-01_audit.json
├── 2025-03-01_audit.json
└── trends.json

Benefits:

• Identify accumulating debt

• Measure remediation effectiveness

• Predict future debt

ISO/IEC 25010 Quality Model

8 Quality Characteristics:

1. Functional Suitability

2. Performance Efficiency

3. Compatibility

4. Usability

5. Reliability

6. Security ← Audit focus

7. Maintainability ← Audit focus

8. Portability

ISO 25010: Security

Sub-characteristics:

• Confidentiality: Data protection

• Integrity: Data accuracy

• Non-repudiation: Action traceability

• Accountability: User identification

• Authenticity: Identity verification

Audit mapping:

• SQL injection → Integrity

• Hardcoded secrets → Confidentiality

ISO 25010: Maintainability

Sub-characteristics:

• Modularity: Isolated changes

• Reusability: Component reuse

• Analyzability: Easy diagnosis

• Modifiability: Easy changes

• Testability: Easy testing

Audit mapping:

• High complexity → Analyzability

• Code duplication → Reusability

CWE / OWASP Mapping

Common Weakness Enumeration (CWE):

Use in audits: Standardized vulnerability references

Style Guides & Standards

Language-specific:

• Python: PEP 8, PEP 484 (type hints)

• JavaScript: ESLint, Airbnb Style Guide

• Java: Google Java Style

• C++: Google C++ Style

• PHP: PSR-12

Benefit: Team consistency, onboarding ease

Audit Deliverable Format

Human-readable: audit.md

# Code Audit Report

## Executive Summary
- Critical: 3
- Important: 12
- Minor: 45

## Critical Issues
### 1. SQL Injection (CWE-89)
File: src/auth.py:42
...

Machine-readable: audit.json

{
  "issues": [
    {"severity": "critical", "cwe": "CWE-89", ...}
  ]
}

Context

Project: Python FastAPI backend Size: 15k LOC, 87 files Goal: Pre-release audit

Team concerns:

• Outdated dependencies?

• Security vulnerabilities?

• Technical debt level?

Audit Workflow

Step 1: Static Analysis

ruff check src/ > out/ruff.txt
bandit -r src/ -o out/bandit.json
pip-audit > out/pip-audit.txt

Step 2: Claude Analysis

Analyze src/ using context from out/*.

Focus:
1. Dependency vulnerabilities
2. Code complexity
3. Error handling

Output: audit.md + audit.json

Findings Summary

Priority: Fix 🔴 before release

Remediation Plan

Issue 1: FastAPI upgrade

# Test upgrade path
pip install fastapi==0.110.0
pytest tests/
# If tests pass → commit

Issue 2: SQL injection

# Before
query = f"SELECT * FROM users WHERE id={user_id}"

# After
query = "SELECT * FROM users WHERE id=?"
cursor.execute(query, (user_id,))

Validation

After fixes:

# Re-run static tools
bandit -r src/ -q

# Re-run Claude audit
claude code audit src/ --compare previous_audit.json

# Verify improvements
python tools/scripts/audit_diff.py

Result:

• 🔴 Critical: 3 → 0

• 🔶 Important: 12 → 8

• ⚪ Minor: 45 → 40

Pre-Commit Hook

File: .git/hooks/pre-commit

#!/usr/bin/env bash
set -e

# Run static analysis
ruff check src/ || exit 1
bandit -r src/ --exit-zero -o out/bandit.json

# Run AI audit
python tools/scripts/audit_gate.py

# Block if critical issues
if grep -q '"severity": "critical"' audit.json; then
    echo "❌ Critical issues found"
    exit 1
fi

echo "✅ Audit passed"

GitHub Actions Workflow

name: Code Audit

on: [push, pull_request]

jobs:
  audit:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3

      - name: Install tools
        run: pip install ruff bandit

      - name: Run audit
        run: python tools/scripts/run_audit.py

      - name: Upload report
        uses: actions/upload-artifact@v3
        with:
          name: audit-report
          path: audit.md

Periodic Debt Reports

Scheduled audit:

# Cron: Daily at 2 AM
0 2 * * * cd /opt/myapp && \
  python tools/scripts/scheduled_audit.py

Benefits:

• Track debt trends

• Early warning system

• Historical data for analysis

Audit Dashboard

Visualization:

graph LR
    Audits[Daily Audits] --> DB[(PostgreSQL)]
    DB --> API[REST API]
    API --> Dashboard[Web Dashboard]
    Dashboard --> Metrics[KPIs]
    Dashboard --> Trends[Trend Charts]
    Dashboard --> Alerts[Alert System]

    style Dashboard fill:#4CAF50

Metrics:

• Debt accumulation rate

• Time to fix critical issues

• Hot spots (files with recurring issues)

Transparency

Include in reports:

# Audit Metadata

- **Tool**: Claude Code (claude-sonnet-4-5-20250929)
- **Date**: 2025-11-01T14:30:00Z
- **Prompt**: [Full prompt text]
- **Static tools**: ruff 0.1.9, bandit 1.7.5
- **Auditor**: Olivier Vitrac

Why: Reproducibility, accountability

Human-in-the-Loop

Never blindly apply AI suggestions:

1. Review: Human examines AI findings

2. Validate: Verify with tests

3. Approve: Human makes final decision

4. Apply: Automated or manual

Critical areas:

• Security fixes

• Architecture changes

• Public API modifications

IP & Data Governance

Claude Pro Max:

• No training on your data

• Enterprise privacy controls

• Audit logs

Best practices:

• Don't include secrets in prompts

• Use environment variables

• Redact sensitive data

Avoiding False Positives

Strategies:

1. Start conservative: High severity threshold

2. Iterate prompts: Refine based on results

3. Use static tools as baseline: Cross-validate

4. Build false positive database: Learn over time

Typical false positive rate:

• Static tools alone: 30-50%

• AI-assisted: 10-20%

Prompt Engineering for Audits

Be specific:

• ❌ "Check code quality"

• ✅ "Audit for PEP 8 violations, cyclomatic complexity >10, missing type hints"

Define output format:

Output format:
- audit.md (human-readable)
- audit.json (machine-readable)

JSON schema: {
  "issues": [{
    "file": str,
    "line": int,
    "severity": "critical"|"important"|"minor",
    ...
  }]
}

Incremental Adoption

Phase 1: Observation

• Run audits, generate reports

• Don't block builds

• Learn typical issues

Phase 2: Soft Gates

• Block on critical (🔴) only

• Warn on important (🔶)

• Ignore minor (⚪)

Phase 3: Strict Enforcement

• Block on 🔴 and 🔶

• Track trends for ⚪

Team Integration

Onboarding:

1. Train team on audit reading

2. Establish triage process

3. Define severity criteria

4. Create remediation SLAs

SLA example:

• 🔴 Critical: Fix within 24 hours

• 🔶 Important: Fix within 1 week

• ⚪ Minor: Fix within sprint

What We Covered

✅ Technical debt types and measurement ✅ Audit dimensions and strategies ✅ Claude Code for comprehensive audits ✅ ISO/IEC 25010 and CWE/OWASP frameworks ✅ Case study: Real-world audit ✅ CI/CD integration ✅ Responsible AI auditing

Three Pillars of Auditing

graph TD
    A[Detect] --> B[Understand]
    B --> C[Refactor]
    C --> D[Verify]
    D -->|Continuous| A

    style A fill:#FF6B35
    style B fill:#F7931E
    style C fill:#FDC830
    style D fill:#4CAF50

1. Detect: Find issues (static + AI)

2. Understand: Contextualize and prioritize

3. Refactor: Fix systematically

4. Verify: Test and re-audit

Key Takeaways

1. Technical debt is inevitable; manage it proactively

2. Hybrid auditing (static + AI) beats either alone

3. Standards mapping (ISO 25010, CWE) provides structure

4. Automation enables continuous auditing

5. Human oversight remains essential

Hands-On Exercises

Try these on your codebase:

1. Run static analysis + Claude audit

2. Map findings to ISO/IEC 25010

3. Calculate debt remediation estimate

4. Track trends over 1 month

5. Integrate audit gate in CI/CD

Resources

Documentation:

• ISO/IEC 25010: Quality model

• CWE: https://cwe.mitre.org/

• OWASP Top 10: https://owasp.org/

• Claude Code: https://docs.anthropic.com/

Course materials:

• lectures/lecture2_audit/pages/ — Detailed guides

• lectures/lecture2_audit/content.md — Summary

Next Steps

Immediate:

• Run audit on internal project

• Establish baseline metrics

• Create audit dashboard

Long-term:

• Integrate into CI/CD

• Train team on triage

• Build trend database

Questions?

Contact: Olivier Vitrac, PhD, HDR olivier.vitrac@adservio.fr

Course repository: lectures/lecture2_audit/