CMMC Scout: AI-powered CMMC Level 2 compliance assessment agent for defense contractors
Created by: Mike Morris
Defense contractors are facing a compliance cliff. Starting in 2025, you can't bid on DoD contracts without CMMC Level 2 certification—but most small manufacturers have no idea if they'll pass. A formal C3PAO assessment costs $15-50K, and if you fail, you're out that money plus months of remediation time before you can try again.
I've spent years as a CMMC consultant watching companies gamble on assessments they weren't ready for. The 110 controls in NIST SP 800-171 are dense, technical, and interconnected. Business owners don't know what "limit unsuccessful logon attempts" actually requires in practice, or that their email-based access approvals won't cut it.
I wanted to build an AI agent that could conduct the pre-assessment conversation I have with clients—asking the right follow-up questions, explaining what compliance actually looks like, and generating a clear remediation roadmap. Not to replace human consultants, but to make that first "where do we stand?" conversation accessible to contractors who can't afford us yet.
What It Does
CMMC Scout is an AI agent that conducts interactive compliance assessments for CMMC Level 2 certification.
The user experience:
Log in via Auth0 with role-based access (assessor vs. client)
Select a security domain to assess (e.g., Access Control)
The agent asks contextual questions about each control
User responds with their current practices
Agent scores each control (Met / Partially Met / Not Met) with plain-English explanations
Generate a gap report with prioritized remediation steps, estimated costs, and timelines
What makes it production-ready:
Every assessment decision streams to Redpanda for audit logging—because compliance work needs receipts
Akka actors manage assessment state, so sessions can recover from failures
Comet tracks prompt performance across controls for continuous improvement
Auth0 ensures sensitive assessment data stays protected
How It Works
Architecture decisions:
I chose an actor-based model with Akka because assessments are inherently stateful, long-running conversations. Each assessment session is an actor that maintains state across multiple control evaluations. If the system crashes mid-assessment, the actor can recover. This maps naturally to how real assessments work—you don't lose your progress.
Redpanda handles event streaming for every significant action: assessment started, control evaluated, gap identified, report generated. This gives me a complete audit trail (essential for compliance work) and enables real-time dashboards for consultants managing multiple client assessments.
I used Auth0 for authentication because security-by-design matters when you're handling sensitive compliance data. Role-based access control means clients only see their own assessments while consultants can manage multiple clients.
Comet tracks my prompt engineering experiments. Different controls need different assessment approaches—some are yes/no technical checks, others require nuanced conversation about policies and procedures. Comet lets me version prompts and measure which approaches yield the most accurate assessments.
Tech stack:
Python / FastAPI for the backend
Claude API for the assessment agent
Akka for workflow orchestration
Redpanda for event streaming
Auth0 for authentication
Comet for ML observability
PostgreSQL for persistence
Data: I loaded the 22 Access Control domain controls from NIST SP 800-171 Rev 2, including assessment objectives and discussion points. The agent uses this as context when formulating questions and scoring responses.