# Otterly.ai Prompt Research Process

**Last Updated:** 2025-12-10

## Overview

This document outlines the complete process for researching, generating, and maintaining prompts for Otterly.ai AI search monitoring. This process should be executed quarterly to keep prompts current with search trends.

## Process Steps

### 1. Data Collection Phase

#### 1.1 Google Search Console (GSC)

```bash
php v2/scripts/otterly/pull-gsc-data.php
```

**Requirements:**

- GSC API credentials configured in `v2/config/google-api-credentials.php`
- Service account has access to ordio.com property
- No additional configuration needed

**Output:** `v2/data/otterly/gsc-queries.json`

**Frequency:** Quarterly

#### 1.2 Google Analytics 4 (GA4)

```bash
php v2/scripts/otterly/pull-ga4-data.php
```

**Requirements:**

- GA4 Property ID must be updated in script
- Service account has access to GA4 property
- Update `$propertyId` variable in script

**Output:** `v2/data/otterly/ga4-queries.json`

**Frequency:** Quarterly

#### 1.3 SISTRIX API

```bash
php v2/scripts/otterly/pull-sistrix-data.php
```

**Requirements:**

- SISTRIX API key must be added to script
- Update `$sistrixApiKey` variable
- Credit limits are enforced automatically (default: 100 credits/day)

**Output:**

- `v2/data/otterly/sistrix-keywords.json`
- `v2/data/otterly/sistrix-credits-log.json` (credit usage tracking)

**Frequency:** Quarterly (cached for 24 hours)

### 2. Prompt Generation Phase

```bash
php v2/scripts/otterly/generate-otterly-prompts.php
```

**What it does:**

1. Loads prompt categories and competitor data
2. Generates prompts from:
   - Industry-specific use cases
   - Product-specific keywords
   - Competitor comparisons
   - Data sources (GSC, GA4, SISTRIX)
3. Scores all prompts using multi-criteria algorithm
4. Selects top 50 prompts
5. Exports to CSV and JSON

**Output:**

- `v2/data/otterly/otterly-prompts.csv` (for Otterly.ai import)
- `v2/data/otterly/otterly-prompts.json` (full metadata)

### 3. Validation Phase

```bash
php v2/scripts/otterly/validate-prompts.php
```

**Checks:**

- German language accuracy
- Competitive tracking capability (not too specific)
- Intent diversity
- Geographic targeting (Germany)
- **Conversational quality** (10-25 words, question format, natural structure)
- **Context inclusion** (industry, use case)
- **Average word count** (target: 10-25 words)
- **Conversational score** (target: 30+/50)

**Output:** Validation report with errors and warnings

### 4. Import to Otterly.ai

1. Log in to Otterly.ai
2. Navigate to Prompts section
3. Import CSV file: `v2/data/otterly/otterly-prompts.csv`
4. Verify all 50 prompts imported correctly
5. Set monitoring frequency (recommended: weekly)

## Best Practices

### Prompt Selection Criteria

1. **High Intent**: Prioritize commercial/transactional intent (comparative, solution-seeking)
2. **Product Focus**: Prioritize product-related queries (70%+ of prompts should be product-focused)
3. **Competitive Tracking**: Prompts should not be too specific to allow tracking against all competitors
4. **Geographic Relevance**: All prompts should target Germany (de-DE)
5. **Language Accuracy**: All prompts must be in German
6. **Exclude Supporting Content**: Filter out calculator, template, webinar queries (max 2 calculator queries)
7. **Direct Competitors Only**: Use only direct competitors (Schichtplanung as primary focus) + HR suites with shift focus
8. **Diversity**: Ensure representation across:
   - Product categories (min 35 product-related prompts)
   - Competitor comparisons (min 10 competitor prompts)
   - Intent types (min 12 comparative, min 8 solution-seeking, min 5 task-based)

### Competitor Classification

**Direct Competitors** (Primary Focus on Schichtplanung):

- Planday, Papershift, Staffomatic, Shiftbase, 7shifts, Crewmeister, Deputy, Homebase, When I Work, Gastromatic, Aplano, Planerio, Planovo, Quinyx, Workforce.com

**HR Suites with Shift Focus** (Secondary):

- Connecteam (has Schichtplanung as core feature)

**Indirect Competitors** (Excluded):

- Personio, BambooHR, Lattice, Leapsome, Kenjo, Rippling, Workday (focus on HR/Recruiting, not shift planning)

See `v2/data/otterly/competitors-classified.json` for full classification.

### Product Relevance Filtering

**Product-Related Keywords** (Boost):

- schichtplan, dienstplan, zeiterfassung, arbeitszeiterfassung, personalverwaltung, lohnabrechnung, payroll, personalakte, abwesenheit, urlaub, gastronomie, einzelhandel, pflege

**Supporting Content Keywords** (Exclude/Penalize):

- rechner, vorlage, webinar, download, whitepaper, excel, pdf, template, kalkulator

**Scoring Impact**:

- Product-related queries: +30 to +45 points
- Calculator queries: -50 points (strong penalty)
- GSC queries with product relevance: +15 additional points

### Conversational Prompt Transformation

**System Overview:**

All prompts are automatically transformed from keywords/queries to conversational format using:

- **Templates**: `v2/data/otterly/prompt-templates.json` - Template library for each intent type
- **Context Rules**: `v2/data/otterly/context-rules.json` - Industry, use case, company size indicators
- **Transformation Functions**:
  - `analyzeQuery()` - Extracts intent, keywords, industry, use case
  - `transformKeywordToPrompt()` - Converts keywords to conversational prompts
  - `transformGSCQueryToPrompt()` - Converts GSC queries to conversational prompts
  - `transformCompetitorPrompt()` - Converts competitor names to conversational comparisons
  - `transformIndustryPrompt()` - Creates industry-specific conversational prompts

**Quality Scoring:**

- **Conversational Quality Score**: 0-50 points based on:
  - Length (10-25 words = +20 points)
  - Question format (+15 points)
  - Natural language structure (+10 points)
  - Context inclusion (+15 points)
  - Keyword stuffing penalty (-5 points)

**Validation:**

- `isConversationalPrompt()` - Validates prompts meet conversational standards (8+ words, question words, natural structure)
- Non-conversational prompts are filtered unless they have high data source value (impressions > 500)

### Scoring Algorithm Tuning

The scoring algorithm can be adjusted in `generate-otterly-prompts.php`:

- **Intent Weights**: Adjust `$intentScores` array
- **Product Relevance**: Adjust product keyword boost (+30 to +45) and calculator penalty (-50)
- **Conversational Quality**: Adjust `scoreConversationalQuality()` weights (length, question format, context)
- **Data Source Weights**: Adjust `$sourceScores` array
- **Category Diversity**: Adjust diversity bonus calculation
- **Geographic Bonus**: Adjust Germany-specific scoring
- **Quality Thresholds**: Adjust min score (40), max calculator queries (2), min product-related (35), min competitor prompts (10), min conversational (40)

### Credit Management (SISTRIX)

- **Daily Limit**: Default 100 credits/day (adjustable)
- **Caching**: Results cached for 24 hours
- **Fallback**: Uses cached data if credit limit reached
- **Monitoring**: Check `sistrix-credits-log.json` regularly

## Troubleshooting

### GSC API Errors

**Error:** "Failed to initialize GSC service"

**Solution:**

1. Verify credentials file exists: `v2/config/google-api-credentials.json`
2. Check service account has GSC access
3. Verify site URL is correct in script

### GA4 API Errors

**Error:** "No queries found"

**Solution:**

1. Verify Property ID is correct
2. Check date range (may need to adjust if no recent data)
3. Verify service account has GA4 access
4. Check if organic search traffic exists in date range

### SISTRIX Credit Issues

**Error:** "Credit limit reached"

**Solution:**

1. Check `sistrix-credits-log.json` for usage
2. Wait for daily reset (midnight)
3. Increase daily limit if needed (adjust `$maxCredits`)
4. Use cached data as fallback

### Low Intent Diversity

**Issue:** All prompts have same intent type

**Solution:**

1. Adjust scoring algorithm to favor diverse intents
2. Manually add competitor comparison prompts
3. Include more informational and task-based prompts
4. Re-run generator with adjusted weights

### Prompts Too Specific

**Issue:** Prompts mention Ordio specifically (not good for competitive tracking)

**Solution:**

1. Review scoring algorithm penalty for specific prompts
2. Filter out prompts containing "Ordio" (except in "vs" comparisons)
3. Manually review and adjust prompts before import

## Maintenance Schedule

### Quarterly Updates

1. **Month 1**: Pull fresh data from all APIs
2. **Month 1**: Regenerate prompts
3. **Month 1**: Validate and import to Otterly.ai
4. **Months 2-3**: Monitor performance in Otterly.ai
5. **Month 3**: Review and adjust based on results

### Annual Review

1. Review competitor list (add/remove competitors)
2. Update industry categories if needed
3. Review and update product categories
4. Adjust scoring algorithm based on performance data
5. Document lessons learned

## File Structure

```
v2/
├── data/
│   └── otterly/
│       ├── prompt-categories.json      # Intent and category definitions
│       ├── competitors.json            # Competitor list
│       ├── gsc-queries.json           # GSC data (when pulled)
│       ├── ga4-queries.json           # GA4 data (when pulled)
│       ├── sistrix-keywords.json      # SISTRIX data (when pulled)
│       ├── sistrix-credits-log.json   # Credit usage tracking
│       ├── sistrix-cache.json         # SISTRIX cache
│       ├── otterly-prompts.csv        # Final prompts (Otterly.ai import)
│       └── otterly-prompts.json       # Full metadata
└── scripts/
    └── otterly/
        ├── pull-gsc-data.php          # GSC data puller
        ├── pull-ga4-data.php          # GA4 data puller
        ├── pull-sistrix-data.php      # SISTRIX data puller
        ├── generate-otterly-prompts.php # Main generator
        └── validate-prompts.php       # Validator
```

## Success Metrics

- **Prompt Quality**: 100% pass validation
- **Intent Diversity**: At least 3 different intent types
- **Category Coverage**: All product and industry categories represented
- **Competitor Coverage**: Top 20 competitors included
- **Data Integration**: At least 2 of 3 data sources successfully pulled
- **Otterly.ai Import**: All 50 prompts imported successfully

## References

- [Otterly.ai Help Center](https://help.otterly.ai/)
- [Otterly.ai Prompt Research Guide](https://help.otterly.ai/relevant-prompts)
- [GEO Optimization Best Practices](https://otterly.ai/blog/ai-search-optimization-how-to-get-found-in-a-bot-first-world/)
- Main research document: `docs/otterly-prompt-research.md`