# OCR Business Card Testing Framework

**Last Updated:** 2026-01-20

## Overview

The OCR Business Card Testing Framework provides comprehensive testing infrastructure for validating business card OCR extraction accuracy. The framework includes 100+ test cases covering German and English cards, various layouts, edge cases, and quality variations.

## Directory Structure

```
v2/scripts/ocr-tests/
├── test-data-schema.json          # JSON schema for test cases
├── TestCaseLoader.php             # Loads and validates test cases
├── MetricsCalculator.php          # Calculates precision, recall, F1, CER, WER
├── ReportGenerator.php            # Generates HTML, JSON, CSV, console reports
├── run-all-tests.php              # Main test runner
├── regression-tests.json          # Known regression test cases
├── test-cases-german-basic.json   # German basic field tests
├── test-cases-english-basic.json  # English basic field tests
├── test-cases-mixed-language.json # Mixed language tests
├── test-cases-urls.json           # URL edge cases
├── test-cases-company.json        # Company name edge cases
├── test-cases-jobtitle.json       # Job title edge cases
└── test-data/                      # Test data directory
    ├── german/                     # German-only cards
    ├── english/                    # English-only cards
    ├── mixed/                      # Mixed language cards
    ├── edge-cases/                 # Edge cases and failure modes
    ├── layouts/                    # Different layout variations
    └── quality/                    # Quality variations (blur, skew, etc.)
```

## Running Tests

### Basic Usage

Run all tests with console output:

```bash
php v2/scripts/ocr-tests/run-all-tests.php
```

### Filter Tests

Filter by language:

```bash
php v2/scripts/ocr-tests/run-all-tests.php --filter=language:german
```

Filter by edge case tag:

```bash
php v2/scripts/ocr-tests/run-all-tests.php --filter=edge_case_tags:url-contamination
```

### Generate Reports

Generate HTML report:

```bash
php v2/scripts/ocr-tests/run-all-tests.php --format=html --output=report.html
```

Generate JSON report:

```bash
php v2/scripts/ocr-tests/run-all-tests.php --format=json --output=report.json
```

Generate CSV report:

```bash
php v2/scripts/ocr-tests/run-all-tests.php --format=csv --output=report.csv
```

### Verbose Output

Show detailed output for each test:

```bash
php v2/scripts/ocr-tests/run-all-tests.php --verbose
```

### Quiet Mode

Suppress progress output:

```bash
php v2/scripts/ocr-tests/run-all-tests.php --quiet
```

## Test Case Format

Test cases are defined in JSON format following the schema in `test-data-schema.json`:

```json
{
  "id": "test-001",
  "name": "Test Case Name",
  "description": "Description of what this test validates",
  "language": "german|english|mixed",
  "layout_type": "single-column|two-column|centered|...",
  "ocr_text": "Simulated OCR text output\nLine 2\nLine 3",
  "ground_truth": {
    "firstname": "Max",
    "lastname": "Mustermann",
    "email": "max@example.com",
    "phone": "+493012345678",
    "company": "Muster GmbH",
    "jobtitle": "Geschäftsführer",
    "salutation": "Herr"
  },
  "edge_case_tags": ["umlauts", "legal-form"],
  "expected_results": {
    "should_extract": ["firstname", "lastname", "email"],
    "should_not_extract": ["United States"]
  }
}
```

## Adding New Test Cases

### 1. Create Test Case File

Create a new JSON file in the appropriate directory:

- `test-data/german/` - German-only cards
- `test-data/english/` - English-only cards
- `test-data/mixed/` - Mixed language cards
- `test-data/edge-cases/` - Edge cases

### 2. Define Test Case

Add test case following the schema:

```json
{
  "test_cases": [
    {
      "id": "unique-id",
      "name": "Test Name",
      "description": "What this test validates",
      "language": "german",
      "ocr_text": "...",
      "ground_truth": { ... }
    }
  ]
}
```

### 3. Validate Test Case

Test cases are automatically validated when loaded. Check for errors:

```bash
php v2/scripts/ocr-tests/run-all-tests.php 2>&1 | grep -i error
```

## Metrics

The framework calculates the following metrics:

### Field-Level Metrics

- **Precision**: Correctness of extracted value (1.0 = perfect match)
- **Recall**: Completeness of extraction (1.0 = field extracted)
- **F1 Score**: Harmonic mean of precision and recall

### Overall Metrics

- **Overall Accuracy**: Percentage of fields correctly extracted
- **Character Error Rate (CER)**: Edit distance normalized by length
- **Word Error Rate (WER)**: Word-level edit distance

### Error Types

- **Missing**: Field expected but not extracted
- **Wrong**: Field extracted but incorrect value
- **Contaminated**: Field contains correct value but also extra content
- **Correct**: Field correctly extracted

## Interpreting Results

### Console Output

```
SUMMARY
--------------------------------------------------------------------------------
Total Tests:  100
Passed:       85
Failed:       15
Accuracy:     87.50%
Timestamp:    2026-01-20 12:00:00

FIELD-LEVEL ACCURACY
--------------------------------------------------------------------------------
Firstname      95/100 ( 95.0%) ████████████████████████████████████████████████
Lastname       92/100 ( 92.0%) ██████████████████████████████████████████████
Email          98/100 ( 98.0%) ████████████████████████████████████████████████
Phone          94/100 ( 94.0%) ███████████████████████████████████████████████
Company        88/100 ( 88.0%) █████████████████████████████████████████████
Jobtitle       82/100 ( 82.0%) ██████████████████████████████████████████
Salutation     90/100 ( 90.0%) ██████████████████████████████████████████████
```

### HTML Report

The HTML report provides:
- Summary statistics
- Detailed test results with field-by-field comparison
- Visual indicators for passed/failed tests
- Accuracy metrics per field

### JSON Report

The JSON report includes:
- Summary statistics
- Complete test results with extracted values
- Field-level metrics
- Error type breakdown

### CSV Report

The CSV report provides:
- One row per test case
- Extracted values for all fields
- Status and accuracy per test

## Best Practices

### Test Case Creation

1. **Use realistic OCR text**: Simulate what Google Vision API would return
2. **Include edge cases**: Test boundary conditions and failure modes
3. **Cover both languages**: Include German and English test cases
4. **Tag appropriately**: Use edge_case_tags to categorize tests
5. **Validate ground truth**: Ensure ground truth values are correct

### Test Execution

1. **Run before changes**: Establish baseline before making modifications
2. **Run after changes**: Verify improvements/regressions
3. **Filter strategically**: Use filters to focus on specific areas
4. **Review failures**: Analyze failed tests to identify patterns
5. **Track metrics**: Monitor accuracy trends over time

### Regression Testing

Known regression cases are tracked in `regression-tests.json`:

- **Lukas Klein card**: Company should be "ordio", job title should be "Inside Sales Consultant"
- **Emily Bates card**: Company should be "Emily Bates Agency", job title should be "Designer"

Always run regression tests before deploying changes:

```bash
php v2/scripts/ocr-tests/run-all-tests.php --filter=edge_case_tags:regression
```

## Troubleshooting

### Test Cases Not Loading

Check for JSON syntax errors:

```bash
php -l v2/scripts/ocr-tests/test-cases-*.json
```

Validate against schema:

```bash
# Use jsonlint or similar tool
jsonlint v2/scripts/ocr-tests/test-cases-*.json
```

### Extraction Function Not Found

Ensure OCR API file is included:

```php
require_once __DIR__ . '/../api/ocr-business-card.php';
```

### Low Accuracy

1. Review failed test cases
2. Check OCR text quality
3. Verify ground truth values
4. Analyze error patterns
5. Consider adding more test cases for problematic areas

## Related Documentation

- [Field Extraction Guide](FIELD_EXTRACTION_GUIDE.md) - OCR field extraction patterns
- [Troubleshooting Guide](TROUBLESHOOTING.md) - Common extraction failures
- [OCR API Documentation](../api/ocr-business-card.php) - API reference

## Future Enhancements

- [ ] Image-based test cases (with actual business card images)
- [ ] Automated test case generation from real cards
- [ ] Performance benchmarking
- [ ] Integration with CI/CD pipeline
- [ ] Visual diff tool for test results
- [ ] Machine learning model training data export