# Visual Content Detection Patterns

Complete reference for detecting visual content in user inputs.

## File Extension Patterns

### Image Files (High Priority)
```
.png, .jpg, .jpeg, .gif, .bmp, .webp, .svg, .ico, .tiff, .tif, .heic, .heif, .raw, .psd, .ai, .eps
```

**Detection Rule**: Case-insensitive match anywhere in input

### Document Files with Visual Content (Medium Priority)
```
.pdf (may contain diagrams), .ppt, .pptx (slides with visuals), .vsdx (Visio), .drawio
```

**Detection Rule**: File extension + visual keywords

## Keyword Patterns

### Chinese Visual Keywords
```
一级关键词（高优先级）:
图片，图像，照片，截图，图表，图示，图形，影像，画面

二级关键词（中优先级）:
流程图，架构图，时序图，ER 图，思维导图，柱状图，饼图，折线图
设计图，原型图，线框图，界面，UI，UX
表格，表单，清单，列表

三级关键词（低优先级）:
显示，展示，呈现，可视化，看图，读图
```

### English Visual Keywords
```
High Priority:
image, photo, picture, screenshot, snapshot, capture, diagram, chart, graph, plot, figure

Medium Priority:
flowchart, architecture, sequence diagram, ER diagram, mind map, bar chart, pie chart, line graph
design, mockup, wireframe, interface, UI, UX, layout
table, form, list, grid

Low Priority:
show, display, visualize, view, look at, see
```

### Technical Visual Keywords
```
Schema, model, blueprint, spec, technical drawing
Dashboard, widget, panel, visualization
Map, heatmap, scatter plot, histogram
Infographic, poster, banner, thumbnail
```

## Pattern Matching Rules

### Rule 1: File Path + Extension
```regex
[\w\-\.\/]+?\.(png|jpg|jpeg|gif|bmp|webp|svg|ico|tiff|heic)
```

**Action**: Immediate delegation to multimodal-looker

### Rule 2: Markdown Image Syntax
```regex
!\[([^\]]*)\]\(([^\)]+)\)
```

**Action**: Extract alt text and URL, delegate to multimodal-looker

### Rule 3: Base64 Image Data
```regex
data:image\/(png|jpeg|gif|webp);base64,[A-Za-z0-9+/=]+
```

**Action**: Extract base64 data, save to temp file, delegate

### Rule 4: Keyword + File Reference
```
(图片 | 图像|diagram|chart|screenshot).*?[\w\-\.\/]+\.(png|jpg|jpeg|gif|bmp|webp)
```

**Action**: Confirm intent, then delegate

### Rule 5: Keyword Only (Ambiguous)
```
(帮我看看这个图 | 分析这张图片 | 这个图表显示)
```

**Action**: Ask for clarification: "请问是哪张图片？"

## Context-Aware Detection

### Code Development Context
When user is working on code:
- `architecture.png` → Architecture diagram
- `screenshot.png` → Error or UI screenshot
- `mockup.jpg` → Design reference

**Action**: Assume technical visual, delegate with context

### Data Analysis Context
When user mentions data:
- `chart`, `graph`, `plot`, `visualization`
- `sales_chart.png`, `trend_graph.jpg`

**Action**: Assume data visualization, request data extraction

### Design Context
When user discusses design:
- `mockup`, `wireframe`, `prototype`, `design`
- `ui_design.png`, `wireframe.jpg`

**Action**: Assume design visual, request UI/UX analysis

## Detection Confidence Levels

| Level | Confidence | Triggers | Action |
|-------|------------|----------|--------|
| HIGH | 90-100% | Image file + visual keyword | Auto-delegate |
| MEDIUM | 60-89% | Image file OR strong keyword | Confirm then delegate |
| LOW | 30-59% | Weak keyword only | Ask for clarification |
| NONE | 0-29% | No visual signals | Process as text |

## Edge Cases

### Ambiguous References
```
"看这个" (without specifying what)
"这个文件" (could be text or image)
```
**Handling**: Ask "请问是哪个文件？是图片吗？"

### Multiple Images
```
"比较这两张图：img1.png 和 img2.png"
```
**Handling**: Delegate both, request comparison

### Image in Code Block
````
```
![image](path.png)
```
````
**Handling**: Still detect as visual content (user may be documenting)

### URL Images
```
https://example.com/image.png
http://cdn.site.com/chart.jpg
```
**Handling**: Detect as visual, may need download first

## Implementation Checklist

- [ ] Scan input for file extensions
- [ ] Check for markdown image syntax
- [ ] Search for visual keywords
- [ ] Evaluate context (code, data, design)
- [ ] Assign confidence level
- [ ] Execute appropriate action (delegate/confirm/ask)

## Testing Examples

### Should Trigger (High Confidence)
```
分析这个截图：error.png
看这张架构图 design/architecture.png
![流程图](flow.png) 显示什么？
帮我看看 data:image/png;base64,...
```

### Should Trigger (Medium Confidence)
```
这个图片怎么优化？screenshot.png
diagram.jpg 有什么改进建议
```

### Should Ask (Low Confidence)
```
帮我看看这个图 (no file specified)
这个设计怎么样？(unclear if visual attached)
```

### Should Not Trigger
```
帮我写代码
这个文本怎么格式化
纯文字内容
```