SYSTEM CASE STUDY
Drift: Detecting Requirement Drift with Local AI
A tool to make scope changes visible and approvable
Introduction
Drift is a local-first SaaS workspace that detects requirement drift from client messages and turns scope changes into reviewable project decisions. It addresses the problem of silent scope creep by making small, incremental changes visible and approvable. The system uses a fine-tuned local AI model to analyze client input against approved requirements, identifying drift and generating change requests. The architecture separates the AI runtime from the main backend, allowing independent evolution of the product workflow and model runtime.
Problem
Most projects experience silent scope creep through small, incremental requests that appear harmless individually but accumulate into significant scope changes. These changes are often not tracked or approved, leading to extra work without proper compensation. Existing tools do not provide a way to systematically detect and approve these changes, making it difficult for teams to manage scope effectively.
Solution
Drift detects requirement drift by comparing new client messages or meeting notes against approved requirements. The system uses a fine-tuned local AI model to analyze these inputs and identify drift, returning labels, scores, and reasoning. The architecture separates the AI runtime from the main backend, allowing independent evolution of the product workflow and model runtime. The system includes features like workspaces, requirements capture, baseline snapshots, and a change request workflow to manage scope changes systematically.
Architecture
Browser UI with React + TypeScript for user interaction Go backend API handling REST calls and business logic MongoDB for persistence of projects, requirements, analyses, and approvals Local AI runtime using GGUF model files and llama.cpp for inference
Key features
- Workspaces and projects for organizing client work
- Requirement capture with effort, tags, source text, and acceptance criteria
- Baseline snapshots for approved scope
- Local model-backed drift analysis
- Saved analysis history
- Change request drafting from detected drift
- Approval queue with approve, reject, and revision decisions
- Document upload workflow
- Billing page placeholder for SaaS presentation
- Evaluation dashboard for local model quality checks
Engineering challenges
- Balancing the need for accurate drift detection with the risk of false positives
- Ensuring the local AI model remains up-to-date without requiring constant retraining
- Maintaining separation between the AI runtime and the main backend while ensuring seamless integration
- Managing the size and complexity of the GGUF model files while keeping them out of version control
Technical highlights
- The architecture is well-documented, but the actual performance of the local AI model is not yet fully validated
- The system's ability to handle real-world use cases is still being tested
- The separation of AI runtime from the main backend is a good design choice, but its effectiveness in practice is not yet fully understood
Impact summary
Drift provides a clear, systematic way to manage scope changes by making them visible, explainable, and approvable. This reduces the risk of silent scope creep and ensures that teams can track and manage changes effectively.
SYSTEM COMPONENTS