rulecheck

LLM-powered rule verification for codebases.

rulecheck uses LLM agents to verify that code changes follow rules defined in markdown files. Each rule gets its own agent that reviews diffs, explores the codebase as needed, and reports pass/fail.

Installation

# Clone the repository
git clone ssh://git@git.trainraider.win/trainraider/rulechecker.git
cd rulechecker

# Install with uv
uv sync

# Or install with pip
pip install .

Configuration

rulecheck requires a config.json file with LLM provider settings.

Config locations

The first config file found is used:

1. ./config.json (current directory)
2. ~/.config/rulecheck/config.json

Config format

{
  "providers": {
    "local": {
      "alias": "Local",
      "api_key": "your-api-key-here",
      "base_url": "http://localhost:8080/v1",
      "models": {
        "Qwen3.6-35B-A3B": {
          "alias": "Qwen 3.6 35B"
        }
      }
    }
  },
  "selected_model": {
    "provider": "local",
    "model": "Qwen3.6-35B-A3B"
  },
  "workers": 8
}

Config fields

Field	Description
providers	Object containing LLM provider configurations
providers.<name>.base_url	API endpoint URL (required)
providers.<name>.api_key	API key for authentication
providers.<name>.alias	Friendly name for the provider
providers.<name>.models	Object of available models
providers.<name>.models.<name>.alias	Friendly name for the model
selected_model.provider	Provider key to use
selected_model.model	Model key to use
workers	Number of parallel agents (default: 8)

Rule files

INRULES.md

Rules for your own code. Place INRULES.md files in directories to define rules for that scope. Rules apply to the directory and all subdirectories.

# Code Standards

## Type Hints
All functions must have type annotations for parameters and return values.

## No Print Statements
Do not use print() in production code. Use logging instead.

## Docstrings
Public functions must have docstrings.

OUTRULES.md

Rules that libraries define for their consumers. If you maintain a library, place OUTRULES.md at its root to specify how other projects should use it. When those projects run rulecheck, it finds your OUTRULES.md and verifies their code follows your rules.

# Usage Rules for MyLib

## API Version
Only use stable API endpoints. Do not use /experimental/* routes.

## Error Handling
Always catch MyLibError, not bare Exception.

## Initialization
Always call init_client() before making API calls.

Usage

Basic usage

# Check open changes (default)
rulecheck

# Check all files in the repo
rulecheck -F

# Verbose output (show passed rules too)
rulecheck -v

# Check specific directory
rulecheck /path/to/project

Command-line options

Option	Description
-F, --full	Check all files, not just open changes
-m, --model	Override model from config
-w, --workers	Override worker count from config
-v, --verbose	Show all rule results, not just failures
--api-key	Override API key from config
--api-base	Override API base URL from config
directory	Directory to check (default: current)

Examples

# Use a different model
rulecheck -m openai/gpt-4o

# Run with more workers
rulecheck -w 16

# Check specific project with verbose output
rulecheck -v /path/to/project

# Override API endpoint
rulecheck --api-base https://api.example.com/v1

How it works

1. Discovery: Finds INRULES.md in your project tree and OUTRULES.md in your dependencies
2. Extraction: Parses markdown headings into individual rules
3. Scoping: Each rule's agent is confined to the directory where its rule file lives
4. Verification: Spawns parallel LLM agents, each reviewing diffs against one rule
5. Reporting: Aggregates results and reports failures

Agent behavior

Each agent:

• Receives the git diff for files in scope
• Can read any file within its scope directory for context
• Cannot access files outside its scope
• Reports pass/fail with a brief reason

Example output

Loaded configuration from ~/.config/rulecheck/config.json

1 rule(s) failed:

  ✗ [src/myapp] No Print Statements
    Found print() call at src/myapp/utils.py:42

1/1 rules failed.