themblem/doc/batch-model-config-analysis.md

Analysis: Admin Configuration for Model Selection per Batch

Executive Summary

This document analyzes how to add a feature that allows admins to configure which AI model to use for each batch during QR verification. Currently, the system uses a default model for all batches when feature_comparison_threshold > 0.01.

Current State

1. Batch Model (CodeBatch)

• Location: api/products/models.py:162
• Key field: feature_comparison_threshold (FloatField, default=0.0)
  • When > 0.01, triggers v5 QR verification
  • When <= 0.01, skips v5 QR verification

2. QR Verification Flow

• Location: api/products/views.py:665-680
• Method: QrVerifyView.do_v5_qr_verify()
• Current behavior:
  • Makes POST request to https://themblem.com/api/v5/qr_verify
  • Sends image files only
  • Does NOT specify a model parameter
  • Server uses default model: best_model_ep82_pos98.94_neg96.13_20250720_222102.pt

3. V5 QR Verify Server

• Location: emblem5/ai/server.py:83-114
• Endpoint: /api/v5/qr_verify
• Model selection:
  • Accepts model parameter from form data: fd.get('model', default_model)
  • Default model defined in emblem5/ai/common.py:42
  • Models are downloaded from OSS bucket emblem-models if not cached locally

4. Admin UI

• Location: web/src/views/code-batch.vue
• Current editable fields include feature_comparison_threshold
• Uses GenericManager component for CRUD operations

Requirements

1. Database: Add a field to CodeBatch to store the model identifier
2. API: Pass the model parameter to v5 QR verify endpoint when specified
3. UI: Allow admins to configure the model per batch
4. Backward Compatibility: Existing batches should continue using default model if not configured

Design Approach

Option 1: Simple Model Name Field (Recommended)

• Add verification_model field to CodeBatch (CharField, nullable)
• Store model filename (e.g., best_model_ep82_pos98.94_neg96.13_20250720_222102.pt)
• Pass to v5 endpoint when feature_comparison_threshold > 0.01 and model is set
• Pros: Simple, minimal changes, flexible
• Cons: No validation of model existence, manual entry required

Option 2: Model Reference with Validation

• Add verification_model field + create VerificationModel table
• Store available models in database
• Validate model exists before allowing selection
• Pros: Better validation, can track model metadata
• Cons: More complex, requires model management UI

Option 3: Use GlobalConfig for Model List

• Store available models in GlobalConfig
• Add verification_model field to CodeBatch
• Validate against GlobalConfig list
• Pros: Reuses existing config system, simple validation
• Cons: Still requires manual model management

Recommendation: Option 1 for minimal changes, with future enhancement to Option 3 if needed.

Implementation Plan

Phase 1: Database Changes

1. Create Migration

   • Add verification_model field to CodeBatch model
   • Type: CharField(max_length=255, null=True, blank=True)
   • Verbose name: "验证模型名称（可选，留空使用默认模型）"
   • Default: None (uses server default)

2. Update Model

   # api/products/models.py
   class CodeBatch(models.Model):
       # ... existing fields ...
       verification_model = models.CharField(
           max_length=255, 
           null=True, 
           blank=True,
           verbose_name="验证模型名称（可选，留空使用默认模型）"
       )
   

Phase 2: API Changes

1. Update do_v5_qr_verify() method

   • Location: api/products/views.py:668
   • Accept model_name parameter
   • Pass model in form data if provided
   • Fallback to default (current behavior) if not provided

2. Update QrVerifyView.post() method

   • Location: api/products/views.py:722
   • Get model from sc.batch.verification_model
   • Pass to do_v5_qr_verify() when calling

3. Code Changes

   def do_v5_qr_verify(self, imgs, messages, model_name=None):
       files = {}
       for i, img in enumerate(imgs):
           files[f'frame_{i}_{img[1]}'] = img[0]
   
       data = {}
       if model_name:
           data['model'] = model_name
   
       resp = requests.post(
           "https://themblem.com/api/v5/qr_verify",
           files=files,
           data=data,  # Add model parameter if specified
       )
       # ... rest of method ...
   
   # In post() method:
   if sc.batch.feature_comparison_threshold > 0.01:
       model_name = sc.batch.verification_model if hasattr(sc.batch, 'verification_model') else None
       self.do_v5_qr_verify(imgs, messages, model_name=model_name)
   

Phase 3: UI Changes

1. Update Vue Component

   • Location: web/src/views/code-batch.vue
   • Add verification_model to editable_fields array
   • Add to visible_fields array (optional, for display)

2. Optional: Model Selection Helper

   • Could add a dropdown/autocomplete if model list is available
   • For now, text input is sufficient (admins know model names)

Phase 4: Testing

1. Unit Tests

   • Test do_v5_qr_verify() with and without model parameter
   • Test backward compatibility (null model uses default)

2. Integration Tests

   • Test QR verification with custom model
   • Test QR verification without model (default behavior)
   • Test with feature_comparison_threshold = 0 (should skip)

3. Manual Testing

   • Create batch with custom model
   • Verify model is passed to v5 endpoint
   • Verify default model used when field is empty

Database Schema Change

# Migration file: api/products/migrations/XXXX_add_verification_model_to_codebatch.py

from django.db import migrations, models

class Migration(migrations.Migration):
    dependencies = [
        ('products', 'XXXX_previous_migration'),
    ]

    operations = [
        migrations.AddField(
            model_name='codebatch',
            name='verification_model',
            field=models.CharField(
                max_length=255,
                blank=True,
                null=True,
                verbose_name='验证模型名称（可选，留空使用默认模型）'
            ),
        ),
    ]

API Changes Summary

Modified Endpoints

• None (internal method change only)

New Behavior

• QrVerifyView.do_v5_qr_verify() now accepts optional model_name parameter
• When CodeBatch.verification_model is set, it's passed to v5 endpoint
• When CodeBatch.verification_model is null/empty, server uses default model

UI Changes Summary

Modified Views

• web/src/views/code-batch.vue
  • Add verification_model to editable fields
  • Optional: Add to visible fields for display

User Experience

• Admin can edit batch and set verification_model field
• Field is optional - leaving empty uses default model
• Text input allows entering model filename directly

Backward Compatibility

✅ Fully Compatible

• Existing batches have verification_model = None
• When None, v5 endpoint uses default model (current behavior)
• No breaking changes to existing functionality

Future Enhancements

1. Model Management UI

   • List available models from OSS bucket
   • Show model metadata (accuracy, date, etc.)
   • Auto-complete model selection

2. Model Validation

   • Check if model exists before saving
   • Show warning if model not found

3. Model Versioning

   • Track model versions per batch
   • Allow rollback to previous models

4. A/B Testing

   • Support multiple models per batch
   • Compare model performance

Files to Modify

1. api/products/models.py - Add verification_model field
2. api/products/views.py - Update do_v5_qr_verify() and QrVerifyView.post()
3. web/src/views/code-batch.vue - Add field to editable fields
4. Create migration file in api/products/migrations/

Estimated Effort

• Database Migration: 15 minutes
• Backend Changes: 30 minutes
• Frontend Changes: 15 minutes
• Testing: 1 hour
• Total: ~2 hours

Risks and Considerations

1. Model Availability: No validation that model exists in OSS bucket

   • Mitigation: Server will return error if model not found, which is acceptable

2. Model Naming: Admins must know exact model filename

   • Mitigation: Document model names, or add helper UI later

3. Default Model Changes: If default model changes in server, batches without explicit model will use new default

   • Mitigation: This is expected behavior, but could be documented

4. Performance: No performance impact expected

   • Model download is cached on server side

Conclusion

This feature can be implemented with minimal changes following Option 1 (Simple Model Name Field). The implementation is backward compatible and allows admins to configure models per batch while maintaining default behavior for existing batches.