初始化项目：词曲 & 音眼资产导入工具

- import_hk_songs.py: 主数据导入脚本
- lyric_dedup/: 歌词去重核心模块
- test_dedup.py: 单元测试 & L2 benchmark
- serve_l2_dashboard.py + l2_review_dashboard.html: 复核前端
- .gitignore: 排除 .env、数据文件、缓存、输出目录
- README.md: 项目说明文档

+# 源库 - 音眼正式
+SOURCE_DB_HOST=
+SOURCE_DB_PORT=3306
+SOURCE_DB_USER=
+SOURCE_DB_PASSWORD=
+SOURCE_DB_NAME=
+
+# 目标库 - 词曲库
+TARGET_DB_HOST=
+TARGET_DB_PORT=3306
+TARGET_DB_USER=
+TARGET_DB_PASSWORD=
+TARGET_DB_NAME=
+TARGET_TABLE_NAME=hk_songs
+
+# OSS 配置
+OSS_ACCESS_KEY_ID=
+OSS_ACCESS_KEY_SECRET=
+OSS_ENDPOINT=
+OSS_BUCKET_NAME=
+OSS_FILE_BASE_NAME=

+# Python
+__pycache__/
+*.pyc
+*.pyo
+.venv/
+.pytest_cache/
+
+# Environment
+.env
+
+# macOS
+.DS_Store
+
+# Generated output
+output/
+
+# 数据文件（不纳入版本控制）
+*.csv
+*.sql
+*.xlsx

+# 词曲 & 音眼资产导入工具
+
+本项目用于将音眼（Hikoon）数据库中的歌词和歌曲数据去重、比对后，导入至词曲库的 `hk_songs` 表，并提供 L2 歌词去重测试和人工复核前端。
+
+## 项目结构
+
+```
+.
+├── import_hk_songs.py          # 主脚本：聚合查询、去重、导入
+├── lyric_dedup/                # 歌词去重核心模块
+│   ├── normalization.py        # 歌词文本规范化
+│   ├── checker.py              # 去重匹配逻辑
+│   ├── eval_dataset.py         # 评测数据集构建
+│   ├── file_import.py          # 文件导入工具
+│   └── cli.py                  # CLI 入口
+├── test_dedup.py               # 单元测试 & L2 benchmark
+├── serve_l2_dashboard.py       # L2 测试复核前端服务
+├── l2_review_dashboard.html    # 前端页面
+├── requirements.txt            # Python 依赖
+├── 测试流程指南.md              # L2 测试流程说明
+└── .env                        # 数据库配置（不纳入版本控制）
+```
+
+## 快速开始
+
+### 1. 安装依赖
+
+```bash
+pip install -r requirements.txt
+```
+
+### 2. 配置数据库
+
+复制 `.env.example` 为 `.env` 并填写数据库连接信息：
+
+```env
+YINYAN_DB_HOST=...
+YINYAN_DB_PORT=3306
+YINYAN_DB_USER=...
+YINYAN_DB_PASSWORD=...
+YINYAN_DB_NAME=...
+
+CIKU_DB_HOST=...
+CIKU_DB_PORT=3306
+CIKU_DB_USER=...
+CIKU_DB_PASSWORD=...
+CIKU_DB_NAME=...
+```
+
+### 3. 运行数据导入
+
+```bash
+python import_hk_songs.py --help
+```
+
+### 4. 运行 L2 歌词去重测试
+
+小样本验证：
+
+```bash
+RUN_L2_BENCHMARK=1 \
+L2_BENCHMARK_LIMIT=20 \
+L2_BENCHMARK_EXISTING_LIMIT=50 \
+L2_BENCHMARK_TOPKS=20,100 \
+python -m pytest test_dedup.py::TestL2Benchmark::test_l2_recall_topk_efficiency_and_review_artifacts -q -s
+```
+
+详见 [测试流程指南.md](./测试流程指南.md)。
+
+### 5. 启动复核前端
+
+```bash
+python serve_l2_dashboard.py --host 127.0.0.1 --port 8765
+```
+
+然后访问 http://127.0.0.1:8765 查看测试报告、比对歌词、人工标注。
+
+## 普通单元测试
+
+```bash
+python -m pytest test_dedup.py -q
+```
+
+## 注意事项
+
+- `.env` 文件包含数据库凭据，**不要**提交到仓库。
+- `output/` 目录存放测试生成的报告和歌词文件，不纳入版本控制。
+- CSV / SQL / Excel 等数据文件不纳入版本控制。

+#!/usr/bin/env python3
+"""
+hk_music_record 聚合数据导入脚本
+从音眼测试库聚合查询数据，导入至词曲库 hk_songs 表
+"""
+
+import argparse
+import csv
+import logging
+import os
+import re
+import sys
+import time
+import hashlib
+import io
+import socket
+import threading
+import unicodedata
+from collections import defaultdict
+from concurrent.futures import ThreadPoolExecutor, as_completed
+from datetime import datetime
+from pathlib import Path
+
+import opencc
+import pymysql
+import requests
+from dotenv import load_dotenv
+from tqdm import tqdm
+
+# lyric_dedup 模块
+sys.path.insert(0, str(Path(__file__).resolve().parent))
+from lyric_dedup import DuplicateChecker, DuplicateDecision, LyricRecord
+
+# 加载 .env 配置
+load_dotenv()
+
+# 输出目录
+OUTPUT_DIR = Path(__file__).resolve().parent / 'output'
+LOG_DIR = OUTPUT_DIR / 'logs'
+REPORT_DIR = OUTPUT_DIR / 'reports'
+LOG_DIR.mkdir(parents=True, exist_ok=True)
+REPORT_DIR.mkdir(parents=True, exist_ok=True)
+
+# 日志配置
+logging.basicConfig(
+    level=logging.INFO,
+    format='%(asctime)s [%(levelname)s] %(message)s',
+    handlers=[
+        logging.StreamHandler(sys.stdout),
+        logging.FileHandler(LOG_DIR / f'import_hk_songs_{datetime.now().strftime("%Y%m%d_%H%M%S")}.log', encoding='utf-8')
+    ]
+)
+logger = logging.getLogger(__name__)
+
+# ==================== 数据库配置 ====================
+
+SOURCE_DB_CONFIG = {
+    'host': os.getenv('SOURCE_DB_HOST'),
+    'port': int(os.getenv('SOURCE_DB_PORT', 3306)),
+    'user': os.getenv('SOURCE_DB_USER'),
+    'password': os.getenv('SOURCE_DB_PASSWORD'),
+    'database': os.getenv('SOURCE_DB_NAME'),
+    'charset': 'utf8mb4',
+    'cursorclass': pymysql.cursors.DictCursor,
+}
+
+TARGET_DB_CONFIG = {
+    'host': os.getenv('TARGET_DB_HOST'),
+    'port': int(os.getenv('TARGET_DB_PORT', 3306)),
+    'user': os.getenv('TARGET_DB_USER'),
+    'password': os.getenv('TARGET_DB_PASSWORD'),
+    'database': os.getenv('TARGET_DB_NAME'),
+    'charset': 'utf8mb4',
+}
+
+# ==================== OSS 配置 ====================
+
+OSS_CONFIG = {
+    'access_key_id': os.getenv('OSS_ACCESS_KEY_ID'),
+    'access_key_secret': os.getenv('OSS_ACCESS_KEY_SECRET'),
+    'endpoint': os.getenv('OSS_ENDPOINT'),
+    'bucket_name': os.getenv('OSS_BUCKET_NAME'),
+    'base_url': os.getenv('OSS_FILE_BASE_NAME'),
+}
+
+# ==================== 聚合查询 SQL ====================
+
+AGGREGATE_SQL = """
+SELECT
+  sp.id AS source_id,
+  COALESCE(NULLIF(TRIM(sp.song_name), ''),
+           NULLIF(TRIM(r.record_name), '')) AS name,
+  COALESCE(NULLIF(TRIM(sp.lyricist_name), ''),
+           NULLIF(TRIM(r.lyricist_name), '')) AS lyricist,
+  COALESCE(NULLIF(TRIM(sp.composer_name), ''),
+           NULLIF(TRIM(r.composer_name), '')) AS composer,
+  CASE
+    WHEN COALESCE(sp.release_time, r.pub_time) IS NOT NULL THEN 2
+    ELSE 1
+  END AS issue_status,
+  NULL AS intro,
+  COALESCE(NULLIF(TRIM(sp.singer_name), ''),
+           NULLIF(TRIM(r.singer_name), '')) AS singer,
+  COALESCE(NULLIF(TRIM(r.storage_url), ''),
+           NULLIF(TRIM(rs.diy_music_material_file), ''),
+           NULLIF(TRIM(rs.audition_url), ''),
+           NULLIF(TRIM(r.platform_play_url), '')) AS audio_url_source,
+  NULLIF(TRIM(rs.accompaniment), '') AS accompany_url_source,
+  rl.lyric AS lyrics_txt_content,
+  r.duration AS song_time,
+  FLOOR(rs.start_position / 1000) AS song_start,
+  FLOOR(rs.end_position / 1000) AS song_end,
+  NULLIF(TRIM(rs.music_material_file), '') AS creation_url_source,
+  NULLIF(TRIM(rs.voice_midi), '') AS opern_url_source,
+  r.version_name AS cover_version,
+  COALESCE(sp.release_time, r.pub_time) AS issue_time,
+  COALESCE(NULLIF(TRIM(r.front_cover), ''),
+           NULLIF(TRIM(rs.front_cover), ''),
+           NULLIF(TRIM(ss.front_cover_new), '')) AS cover_url_source,
+  0 AS animation_type,
+  CASE
+    WHEN rs.bpm IS NULL OR rs.bpm = 0 THEN NULL
+    WHEN rs.bpm < 80 THEN 1
+    WHEN rs.bpm <= 120 THEN 2
+    ELSE 3
+  END AS bpm_class,
+  CASE ss.review_status
+    WHEN '0' THEN 1
+    WHEN '1' THEN 3
+    WHEN '2' THEN 2
+    WHEN '3' THEN 4
+    WHEN '4' THEN 4
+    WHEN '5' THEN 0
+    ELSE 0
+  END AS review_status,
+  CASE WHEN rs.is_imputation = 1 THEN 2 ELSE 1 END AS in_status,
+  CASE
+    WHEN ss.is_grounding = 1 THEN 1
+    WHEN ss.is_grounding = 0 THEN 2
+    ELSE NULL
+  END AS song_status,
+  COALESCE(sp.create_time, r.create_time) AS commit_time,
+  NULL AS review_time,
+  rs.ground_date AS shelf_time,
+  rs.remark AS review_remark,
+  sp.create_time AS create_time,
+  sp.creator AS creator,
+  sp.update_time AS modify_time,
+  sp.updater AS modifier,
+  0 AS deleted,
+  NULL AS cooperate_type,
+  NULL AS off_shelf_remark,
+  NULL AS musician_id,
+  NULL AS commit_id,
+  NULLIF(TRIM(rs.voice_midi), '') AS sheet_music_source,
+  0 AS commit_desc,
+  NULL AS price,
+  'hk_song_platform' AS source_table_name,
+  CAST(sp.id AS CHAR) AS source_song_id,
+  NULL AS lyric_archive_element_id,
+  NULL AS melody_archive_element_id,
+  NULL AS audio_fingerprint
+FROM hk_song_platform sp
+LEFT JOIN (
+  SELECT song_id, MIN(record_id) AS record_id
+  FROM hk_song_and_record
+  WHERE is_main_version = 1
+  GROUP BY song_id
+) sr
+  ON sr.song_id = sp.id
+LEFT JOIN hk_music_record r
+  ON r.id = sr.record_id
+ AND r.deleted = b'0'
+LEFT JOIN hk_music_record_state rs
+  ON rs.record_id = r.id
+ AND rs.deleted = b'0'
+LEFT JOIN hk_music_record_lyric rl
+  ON rl.raw_record_id = r.raw_record_id
+ AND rl.deleted = b'0'
+LEFT JOIN hk_song_state ss
+  ON ss.song_id = sp.id
+ AND ss.deleted = b'0'
+WHERE sp.deleted = b'0'
+  AND COALESCE(NULLIF(TRIM(sp.copyright_id), ''), '') <> '1871475560046465025'
+  AND COALESCE(NULLIF(TRIM(sp.copyright_name), ''), '') <> '连城小睿音乐工作室'
+"""
+
+# ==================== 目标表 INSERT 语句 ====================
+
+TARGET_TABLE_NAME = os.getenv('TARGET_TABLE_NAME', 'hk_songs')
+
+INSERT_SQL_TEMPLATE = """
+INSERT INTO {table} (
+  id, name, lyricist, composer, issue_status, intro,
+  audio_url, accompany_url, lyrics_url, lrc_url,
+  song_time, song_start, song_end,
+  creation_url, opern_url, cover_version, issue_time,
+  cover_url, animation_type, bpm_class, review_status,
+  in_status, song_status, commit_time, review_time,
+  shelf_time, review_remark, create_time, creator,
+  modify_time, modifier, deleted, cooperate_type, singer,
+  off_shelf_remark, musician_id, commit_id, sheet_music,
+  commit_desc, price, source_table_name, source_song_id,
+  lyric_archive_element_id, melody_archive_element_id, audio_fingerprint
+) VALUES (
+  %s,%s,%s,%s,%s,%s,
+  %s,%s,%s,%s,
+  %s,%s,%s,
+  %s,%s,%s,%s,
+  %s,%s,%s,%s,
+  %s,%s,%s,%s,
+  %s,%s,%s,%s,
+  %s,%s,%s,%s,%s,
+  %s,%s,%s,%s,
+  %s,%s,%s,%s,
+  %s,%s,%s
+)
+"""
+
+# URL 字段索引映射 (在 row tuple 中的位置)
+# 0:id, 1:name, 2:lyricist, 3:composer, 4:issue_status, 5:intro,
+# 6:audio_url, 7:accompany_url, 8:lyrics_url, 9:lrc_url,
+# 10:song_time, 11:song_start, 12:song_end,
+# 13:creation_url, 14:opern_url, 15:cover_version, 16:issue_time,
+# 17:cover_url, 18:animation_type, 19:bpm_class, 20:review_status,
+# 21:in_status, 22:song_status, 23:commit_time, 24:review_time,
+# 25:shelf_time, 26:review_remark, 27:create_time, 28:creator,
+# 29:modify_time, 30:modifier, 31:deleted, 32:cooperate_type, 33:singer,
+# 34:off_shelf_remark, 35:musician_id, 36:commit_id, 37:sheet_music,
+# 38:commit_desc, 39:price, 40:source_table_name, 41:source_song_id,
+# 42:lyric_archive_element_id, 43:melody_archive_element_id, 44:audio_fingerprint
+
+# 需要 OSS 处理的 URL 字段索引
+OSS_URL_INDICES = {
+    6: 'audio',       # audio_url
+    7: 'accompany',   # accompany_url
+    13: 'creation',   # creation_url
+    14: 'opern',      # opern_url
+    17: 'cover',      # cover_url
+    37: 'sheet_music', # sheet_music
+}
+OSS_LYRIC_INDEX = 8   # lyrics_url (从文本生成)
+
+
+class SnowflakeIdGenerator:
+    """生成业务表使用的 bigint 主键。"""
+
+    # Twitter Snowflake epoch，生成值与现有 19 位业务 ID 量级一致。
+    EPOCH_MS = 1288834974657
+
+    def __init__(self):
+        node_seed = f"{socket.gethostname()}:{os.getpid()}:{time.time_ns()}".encode('utf-8')
+        self.worker_id = int(hashlib.sha1(node_seed).hexdigest(), 16) & 0x3FF
+        self.sequence = 0
+        self.last_ms = -1
+        self.lock = threading.Lock()
+
+    def next_id(self) -> int:
+        with self.lock:
+            now_ms = int(time.time() * 1000)
+            if now_ms < self.last_ms:
+                now_ms = self.last_ms
+
+            if now_ms == self.last_ms:
+                self.sequence = (self.sequence + 1) & 0xFFF
+                if self.sequence == 0:
+                    while now_ms <= self.last_ms:
+                        now_ms = int(time.time() * 1000)
+            else:
+                self.sequence = 0
+
+            self.last_ms = now_ms
+            return ((now_ms - self.EPOCH_MS) << 22) | (self.worker_id << 12) | self.sequence
+
+
+ID_GENERATOR = SnowflakeIdGenerator()
+
+
+def get_oss_bucket():
+    """初始化 OSS Bucket"""
+    import oss2
+    auth = oss2.Auth(OSS_CONFIG['access_key_id'], OSS_CONFIG['access_key_secret'])
+    bucket = oss2.Bucket(auth, OSS_CONFIG['endpoint'], OSS_CONFIG['bucket_name'])
+    return bucket
+
+
+def download_file(url, timeout=30):
+    """下载文件，返回 (content_bytes, content_type) 或 (None, None)"""
+    if not url:
+        return None, None
+    # 如果不是 http/https 开头，尝试补前缀（源库有些是相对路径）
+    if not url.startswith(('http://', 'https://')):
+        return None, None
+    try:
+        resp = requests.get(url, timeout=timeout, stream=True)
+        resp.raise_for_status()
+        content_type = resp.headers.get('Content-Type', '')
+        return resp.content, content_type
+    except Exception as e:
+        logger.warning(f"下载失败: {url}, 错误: {e}")
+        return None, None
+
+
+def upload_to_oss(bucket, content, oss_key, content_type=None):
+    """上传内容到 OSS，返回 OSS URL 或 None"""
+    if not content:
+        return None
+    try:
+        headers = {}
+        if content_type:
+            headers['Content-Type'] = content_type
+        bucket.put_object(oss_key, content, headers=headers if headers else None)
+        return f"{OSS_CONFIG['base_url']}/{oss_key}"
+    except Exception as e:
+        logger.warning(f"OSS 上传失败: {oss_key}, 错误: {e}")
+        return None
+
+
+def process_url_field(bucket, source_url, field_type, record_id, skip_oss=False):
+    """处理 URL 字段：下载并上传 OSS，或透传"""
+    if not source_url:
+        return None
+
+    if skip_oss:
+        return source_url
+
+    content, content_type = download_file(source_url)
+    if not content:
+        # 下载失败，透传源值
+        return source_url
+
+    # 生成 OSS key
+    ext = _guess_ext(source_url, content_type, field_type)
+    oss_key = f"music_library/{field_type}/{record_id}{ext}"
+    oss_url = upload_to_oss(bucket, content, oss_key, content_type)
+    return oss_url or source_url
+
+
+def _is_lrc_format(text):
+    """检测歌词文本是否为 LRC 格式（包含 [mm:ss.xx] 时间标签）"""
+    if not text:
+        return False
+    return bool(re.search(r'\[\d{1,2}:\d{2}[:.\d]*\]', text[:500]))
+
+
+def process_lyrics(bucket, lyric_text, record_id, skip_oss=False):
+    """处理歌词：始终上传 .txt 到 lyrics_url；如果是 LRC 格式，额外上传 .lrc 到 lrc_url
+    返回 (lyrics_url, lrc_url)
+    """
+    if not lyric_text:
+        return None, None
+
+    if skip_oss:
+        if _is_lrc_format(lyric_text):
+            return lyric_text, lyric_text
+        return lyric_text, None
+
+    content = lyric_text.encode('utf-8')
+    # 始终上传 .txt 到 lyrics_url
+    oss_key_txt = f"music_library/lyric/{record_id}.txt"
+    lyrics_url = upload_to_oss(bucket, content, oss_key_txt, 'text/plain; charset=utf-8') or lyric_text
+
+    # 如果是 LRC 格式，额外上传 .lrc 到 lrc_url
+    lrc_url = None
+    if _is_lrc_format(lyric_text):
+        oss_key_lrc = f"music_library/lyric/{record_id}.lrc"
+        lrc_url = upload_to_oss(bucket, content, oss_key_lrc, 'text/plain; charset=utf-8') or lyric_text
+
+    return lyrics_url, lrc_url
+
+
+def _guess_ext(url, content_type, field_type):
+    """根据 URL 或 Content-Type 猜测文件扩展名"""
+    # 先从 URL 中提取
+    if '.' in url.split('/')[-1]:
+        ext = '.' + url.split('/')[-1].split('.')[-1].split('?')[0]
+        if len(ext) <= 6:
+            return ext
+    # 根据 content_type
+    type_map = {
+        'audio/mpeg': '.mp3', 'audio/wav': '.wav', 'audio/x-wav': '.wav',
+        'audio/ogg': '.ogg', 'audio/flac': '.flac', 'audio/aac': '.aac',
+        'image/jpeg': '.jpg', 'image/png': '.png', 'image/webp': '.webp',
+        'image/gif': '.gif',
+        'application/pdf': '.pdf', 'text/plain': '.txt',
+        'audio/midi': '.mid', 'application/x-midi': '.mid',
+    }
+    for ct, ext in type_map.items():
+        if ct in (content_type or ''):
+            return ext
+    # 按字段类型默认
+    defaults = {'audio': '.mp3', 'accompany': '.mp3', 'cover': '.jpg',
+                'creation': '.mp3', 'opern': '.mid', 'sheet_music': '.mid'}
+    return defaults.get(field_type, '')
+
+
+def build_row_tuple(row, bucket, skip_oss=False):
+    """将源查询结果转为目标表行 tuple，处理 OSS 字段"""
+    record_id = row['source_id']
+
+    # 处理 URL 字段
+    audio_url = process_url_field(bucket, row.get('audio_url_source'), 'audio', record_id, skip_oss)
+    accompany_url = process_url_field(bucket, row.get('accompany_url_source'), 'accompany', record_id, skip_oss)
+    creation_url = process_url_field(bucket, row.get('creation_url_source'), 'creation', record_id, skip_oss)
+    opern_url = process_url_field(bucket, row.get('opern_url_source'), 'opern', record_id, skip_oss)
+    cover_url = process_url_field(bucket, row.get('cover_url_source'), 'cover', record_id, skip_oss)
+    sheet_music = process_url_field(bucket, row.get('sheet_music_source'), 'sheet_music', record_id, skip_oss)
+    lyrics_url, lrc_url = process_lyrics(bucket, row.get('lyrics_txt_content'), record_id, skip_oss)
+
+    return (
+        ID_GENERATOR.next_id(),              # id
+        row.get('name'),                     # name
+        row.get('lyricist'),                 # lyricist
+        row.get('composer'),                 # composer
+        row.get('issue_status'),             # issue_status
+        row.get('intro'),                    # intro
+        audio_url,                           # audio_url
+        accompany_url,                       # accompany_url
+        lyrics_url,                          # lyrics_url
+        lrc_url,                             # lrc_url
+        row.get('song_time'),                # song_time
+        row.get('song_start'),               # song_start
+        row.get('song_end'),                 # song_end
+        creation_url,                        # creation_url
+        opern_url,                           # opern_url
+        row.get('cover_version'),            # cover_version
+        row.get('issue_time'),               # issue_time
+        cover_url,                           # cover_url
+        row.get('animation_type', 0),        # animation_type
+        row.get('bpm_class'),                # bpm_class
+        row.get('review_status', 0),         # review_status
+        row.get('in_status', 1),             # in_status
+        row.get('song_status'),              # song_status
+        row.get('commit_time'),              # commit_time
+        row.get('review_time'),              # review_time
+        row.get('shelf_time'),               # shelf_time
+        row.get('review_remark'),            # review_remark
+        row.get('create_time'),              # create_time
+        row.get('creator'),                  # creator
+        row.get('modify_time'),              # modify_time
+        row.get('modifier'),                 # modifier
+        0,                                 # deleted
+        row.get('cooperate_type'),           # cooperate_type
+        row.get('singer'),                   # singer
+        row.get('off_shelf_remark'),         # off_shelf_remark
+        row.get('musician_id'),              # musician_id
+        row.get('commit_id'),                # commit_id
+        sheet_music,                         # sheet_music
+        row.get('commit_desc', 0),           # commit_desc
+        row.get('price'),                    # price
+        row.get('source_table_name'),        # source_table_name
+        row.get('source_song_id'),           # source_song_id
+        row.get('lyric_archive_element_id'), # lyric_archive_element_id
+        row.get('melody_archive_element_id'),# melody_archive_element_id
+        row.get('audio_fingerprint'),        # audio_fingerprint
+    )
+
+
+def process_row_with_oss(args_tuple):
+    """线程工作函数：处理单行的 OSS 上传/下载"""
+    idx, row, bucket, skip_oss = args_tuple
+    try:
+        tuple_row = build_row_tuple(row, bucket, skip_oss=skip_oss)
+        return idx, tuple_row, None
+    except Exception as e:
+        return idx, None, e
+
+
+# ==================== 去重逻辑 ====================
+
+_T2S = opencc.OpenCC('t2s')
+_METADATA_PUNCT = set(' \t\n\r，。！？；：、"“”‘’·…—～！￥（）【】《》〈〉「」『』﹏,.:;!?()[]{}<>|/\\_-')
+_INSTRUMENTAL_LYRIC_RE = re.compile(r'(?:纯音乐|純音樂|无歌词|無歌詞|没有歌词|沒有歌詞|instrumental)', re.IGNORECASE)
+
+
+def _normalize_meta(text: str | None) -> str:
+    """元数据规范化：全角转半角(NFKC)、繁转简、去空格标点、小写"""
+    if not text:
+        return ''
+    text = unicodedata.normalize('NFKC', text)  # 全角数字/字母/标点 → 半角
+    text = _T2S.convert(text.strip().lower())
+    return ''.join(c for c in text if c not in _METADATA_PUNCT)
+
+
+def is_instrumental_lyrics(text: str | None) -> bool:
+    """识别纯音乐/无歌词提示文本，这类内容不参与 L2 歌词去重。"""
+    if not text:
+        return False
+    normalized = _T2S.convert(unicodedata.normalize('NFKC', str(text)).lower())
+    return bool(_INSTRUMENTAL_LYRIC_RE.search(normalized))
+
+
+def build_l1_index(target_conn, table_name: str) -> dict[tuple[str, str, str], int]:
+    """从目标库加载已有歌曲的元数据索引，返回 {(name, lyricist, composer): id}"""
+    sql = f"SELECT id, name, lyricist, composer FROM {table_name} WHERE deleted = '0'"
+    index: dict[tuple[str, str, str], int] = {}
+    try:
+        with target_conn.cursor(pymysql.cursors.DictCursor) as cursor:
+            cursor.execute(sql)
+            for row in cursor.fetchall():
+                key = (
+                    _normalize_meta(row.get('name')),
+                    _normalize_meta(row.get('lyricist')),
+                    _normalize_meta(row.get('composer')),
+                )
+                if key[0]:  # name 非空才入索引
+                    index[key] = row['id']
+        logger.info(f"L1 元数据索引构建完成: {len(index)} 条")
+    except Exception as e:
+        logger.error(f"L1 索引构建失败: {e}")
+    return index
+
+
+def check_l1(row: dict, l1_index: dict) -> tuple[bool, int | None]:
+    """L1 元数据去重，返回 (is_duplicate, matched_id)"""
+    key = (
+        _normalize_meta(row.get('name')),
+        _normalize_meta(row.get('lyricist')),
+        _normalize_meta(row.get('composer')),
+    )
+    if key[0] and key in l1_index:
+        return True, l1_index[key]
+    return False, None
+
+
+class L2CandidateIndex:
+    """L2 歌词候选召回索引，最终判定仍交给 DuplicateChecker。"""
+
+    def __init__(self, checker: DuplicateChecker, mode: str = 'topk', top_k: int = 200):
+        if mode not in {'full', 'topk'}:
+            raise ValueError("mode must be 'full' or 'topk'")
+        if top_k < 1:
+            raise ValueError('top_k must be >= 1')
+        self.checker = checker
+        self.mode = mode
+        self.top_k = top_k
+        self.records: dict[str, LyricRecord] = {}
+        self.order: dict[str, int] = {}
+        self.exact_index: dict[str, set[str]] = defaultdict(set)
+        self.token_index: dict[str, set[str]] = defaultdict(set)
+        self.line_index: dict[str, set[str]] = defaultdict(set)
+
+    def __len__(self) -> int:
+        return len(self.records)
+
+    def add(self, record: LyricRecord) -> None:
+        if record.record_id in self.records:
+            return
+
+        indexed = self.checker._index(record)
+        self.records[record.record_id] = record
+        self.order[record.record_id] = len(self.order)
+        self.exact_index[indexed.exact_hash].add(record.record_id)
+
+        for token in self._tokens_for_recall(indexed):
+            self.token_index[token].add(record.record_id)
+        for line in self._lines_for_recall(indexed):
+            self.line_index[line].add(record.record_id)
+
+    def recall(self, record: LyricRecord) -> list[LyricRecord]:
+        if self.mode == 'full':
+            return list(self.records.values())
+
+        query = self.checker._index(record)
+        exact_ids = self.exact_index.get(query.exact_hash)
+        if exact_ids:
+            return [self.records[record_id] for record_id in sorted(exact_ids, key=self.order.get)]
+
+        scores: dict[str, int] = defaultdict(int)
+        for token in self._tokens_for_recall(query):
+            for record_id in self.token_index.get(token, ()):
+                scores[record_id] += 1
+        for line in self._lines_for_recall(query):
+            for record_id in self.line_index.get(line, ()):
+                scores[record_id] += 4
+
+        ranked_ids = sorted(
+            scores,
+            key=lambda record_id: (-scores[record_id], self.order[record_id]),
+        )[:self.top_k]
+        return [self.records[record_id] for record_id in ranked_ids]
+
+    @staticmethod
+    def _tokens_for_recall(indexed) -> set[str]:
+        return set(indexed.tokens) | set(indexed.primary_tokens) | set(indexed.translation_tokens) | set(indexed.fallback_tokens)
+
+    @staticmethod
+    def _lines_for_recall(indexed) -> set[str]:
+        return set(indexed.normalized.primary_lines or indexed.normalized.unique_lines or indexed.fallback_lines)
+
+
+def check_l2(
+    row: dict,
+    checker: DuplicateChecker,
+    candidates: list[LyricRecord] | L2CandidateIndex,
+) -> tuple[str, float, str | None, str]:
+    """L2 歌词内容去重，返回 (decision, confidence, matched_id, reason)"""
+    lyrics_text = row.get('lyrics_txt_content')
+    if not lyrics_text or not lyrics_text.strip():
+        return 'new', 1.0, None, '无歌词内容，跳过 L2'
+    if is_instrumental_lyrics(lyrics_text):
+        return 'new', 1.0, None, '歌词内容包含纯音乐/无歌词提示，跳过 L2'
+
+    record_id = row.get('source_id', row.get('id'))
+    record = LyricRecord(
+        record_id=str(record_id),
+        lyrics=lyrics_text,
+        title=row.get('name'),
+        artist=row.get('singer'),
+        lyricist=row.get('lyricist'),
+        composer=row.get('composer'),
+    )
+    if isinstance(candidates, L2CandidateIndex):
+        candidates = candidates.recall(record)
+    if not candidates:
+        return 'new', 1.0, None, '无候选集'
+
+    result = checker.check_record_against_candidates(record, candidates, max_candidates=5)
+    matched_id = None
+    if result.candidates:
+        best = result.candidates[0]
+        matched_id = best.record_id if result.decision != DuplicateDecision.NEW else None
+
+    return result.decision.value, result.confidence, matched_id, result.reason
+
+
+def _candidate_by_id(
+    candidates: list[LyricRecord] | L2CandidateIndex,
+    record_id: str | None,
+) -> LyricRecord | None:
+    if record_id is None:
+        return None
+    record_id = str(record_id)
+    if isinstance(candidates, L2CandidateIndex):
+        return candidates.records.get(record_id)
+    return next((candidate for candidate in candidates if candidate.record_id == record_id), None)
+
+
+def _writers_differ(row: dict, candidate: LyricRecord | None) -> bool:
+    if candidate is None:
+        return False
+    row_lyricist = _normalize_meta(row.get('lyricist'))
+    row_composer = _normalize_meta(row.get('composer'))
+    candidate_lyricist = _normalize_meta(candidate.lyricist)
+    candidate_composer = _normalize_meta(candidate.composer)
+    return (
+        bool(row_lyricist or row_composer or candidate_lyricist or candidate_composer)
+        and (row_lyricist, row_composer) != (candidate_lyricist, candidate_composer)
+    )
+
+
+def classify_dedup_action(
+    row: dict,
+    l1_index: dict,
+    checker: DuplicateChecker,
+    candidates: list[LyricRecord] | L2CandidateIndex,
+) -> dict:
+    """Return the import-level dedup action.
+
+    L1 is only a recall/risk signal. L2 content decides merge/review/new.
+    """
+    l1_matched, l1_matched_id = check_l1(row, l1_index)
+    l2_candidates_for_check: list[LyricRecord] | L2CandidateIndex = candidates
+    if isinstance(candidates, L2CandidateIndex):
+        lyrics_text = row.get('lyrics_txt_content') or ''
+        record = LyricRecord(
+            record_id=str(row.get('source_id', row.get('id'))),
+            lyrics=lyrics_text,
+            title=row.get('name'),
+            artist=row.get('singer'),
+            lyricist=row.get('lyricist'),
+            composer=row.get('composer'),
+        )
+        recalled = candidates.recall(record)
+        if l1_matched_id is not None:
+            l1_candidate = candidates.records.get(str(l1_matched_id))
+            if l1_candidate and all(candidate.record_id != l1_candidate.record_id for candidate in recalled):
+                recalled.append(l1_candidate)
+        l2_candidates_for_check = recalled
+
+    decision, confidence, matched_id, reason = check_l2(row, checker, l2_candidates_for_check)
+
+    if decision == 'duplicate':
+        matched_candidate = _candidate_by_id(l2_candidates_for_check, matched_id)
+        return {
+            'action': 'merge',
+            'decision': decision,
+            'confidence': confidence,
+            'matched_id': matched_id,
+            'reason': reason,
+            'l1_matched': l1_matched,
+            'l1_matched_id': l1_matched_id,
+            'merge_authors': _writers_differ(row, matched_candidate),
+            'matched_candidate': matched_candidate,
+        }
+
+    if decision == 'review' or (l1_matched and reason == '无候选集'):
+        review_matched_id = matched_id or (str(l1_matched_id) if l1_matched_id is not None else None)
+        matched_candidate = _candidate_by_id(l2_candidates_for_check, review_matched_id)
+        return {
+            'action': 'review',
+            'decision': 'review',
+            'confidence': confidence,
+            'matched_id': review_matched_id,
+            'reason': reason if decision == 'review' else 'L1 元数据命中但缺少可比对歌词，需要人工复核',
+            'l1_matched': l1_matched,
+            'l1_matched_id': l1_matched_id,
+            'merge_authors': False,
+            'matched_candidate': matched_candidate,
+        }
+
+    return {
+        'action': 'new',
+        'decision': decision,
+        'confidence': confidence,
+        'matched_id': matched_id,
+        'reason': reason,
+        'l1_matched': l1_matched,
+        'l1_matched_id': l1_matched_id,
+        'merge_authors': False,
+        'matched_candidate': None,
+    }
+
+
+class DedupReport:
+    """去重结果 CSV 日志"""
+
+    def __init__(self, filepath: str):
+        self.filepath = filepath
+        self.file = open(filepath, 'w', encoding='utf-8', newline='')
+        self.writer = csv.writer(self.file)
+        self.writer.writerow(['id', 'name', 'stage', 'decision', 'confidence', 'matched_id', 'reason'])
+        self.lock = threading.Lock()
+
+    def write(self, record_id, name, stage, decision, confidence='', matched_id='', reason=''):
+        with self.lock:
+            self.writer.writerow([record_id, name, stage, decision, confidence, matched_id or '', reason])
+            self.file.flush()
+
+    def close(self):
+        self.file.close()
+
+
+class ReviewDecisionReport:
+    """人工审核前的去重决策清单。"""
+
+    FIELDNAMES = [
+        'source_id', 'name', 'lyricist', 'composer', 'singer',
+        'query_lyrics_path', 'action', 'decision', 'confidence', 'matched_id',
+        'matched_name', 'matched_lyricist', 'matched_composer', 'matched_lyrics_path',
+        'l1_matched_id',
+        'merge_authors', 'reason', 'review_decision', 'review_note',
+    ]
+
+    def __init__(self, filepath: str):
+        self.filepath = filepath
+        self.asset_dir = Path(filepath).with_suffix('') / 'lyrics'
+        self.asset_dir.mkdir(parents=True, exist_ok=True)
+        self.file = open(filepath, 'w', encoding='utf-8', newline='')
+        self.writer = csv.DictWriter(self.file, fieldnames=self.FIELDNAMES)
+        self.writer.writeheader()
+
+    def _write_lyrics_asset(self, prefix: str, record_id, title, lyrics: str | None) -> str:
+        if not lyrics:
+            return ''
+        safe_title = re.sub(r'[^\w\u4e00-\u9fff.-]+', '_', str(title or 'untitled'))[:80] or 'untitled'
+        path = self.asset_dir / f"{prefix}_{record_id}_{safe_title}.txt"
+        path.write_text(str(lyrics), encoding='utf-8')
+        return str(path)
+
+    def write(self, row: dict, action: dict) -> None:
+        source_id = row.get('source_id', row.get('id'))
+        matched_candidate = action.get('matched_candidate')
+        query_lyrics_path = self._write_lyrics_asset(
+            'query',
+            source_id,
+            row.get('name'),
+            row.get('lyrics_txt_content'),
+        )
+        matched_lyrics_path = ''
+        matched_name = ''
+        matched_lyricist = ''
+        matched_composer = ''
+        if matched_candidate:
+            matched_name = matched_candidate.title or ''
+            matched_lyricist = matched_candidate.lyricist or ''
+            matched_composer = matched_candidate.composer or ''
+            matched_lyrics_path = self._write_lyrics_asset(
+                'matched',
+                matched_candidate.record_id,
+                matched_candidate.title,
+                matched_candidate.lyrics,
+            )
+        self.writer.writerow({
+            'source_id': source_id,
+            'name': row.get('name', ''),
+            'lyricist': row.get('lyricist', ''),
+            'composer': row.get('composer', ''),
+            'singer': row.get('singer', ''),
+            'query_lyrics_path': query_lyrics_path,
+            'action': action.get('action', ''),
+            'decision': action.get('decision', ''),
+            'confidence': f"{float(action.get('confidence', 0.0)):.4f}",
+            'matched_id': action.get('matched_id') or '',
+            'matched_name': matched_name,
+            'matched_lyricist': matched_lyricist,
+            'matched_composer': matched_composer,
+            'matched_lyrics_path': matched_lyrics_path,
+            'l1_matched_id': action.get('l1_matched_id') or '',
+            'merge_authors': '1' if action.get('merge_authors') else '0',
+            'reason': action.get('reason', ''),
+            'review_decision': '',
+            'review_note': '',
+        })
+
+    def close(self):
+        self.file.close()
+
+
+_APPROVED_REVIEW_DECISIONS = {'import', '导入', 'new', '新增', 'approve', 'approved', 'yes', 'y', '1', 'true'}
+
+
+def load_approved_import_ids(review_csv_path: str) -> set[str]:
+    """读取人工审核后的 CSV，返回确认要导入目标库的 source_id 集合。"""
+    approved: set[str] = set()
+    with open(review_csv_path, 'r', encoding='utf-8-sig', newline='') as f:
+        reader = csv.DictReader(f)
+        if not reader.fieldnames or 'source_id' not in reader.fieldnames:
+            raise ValueError('审核结果 CSV 必须包含 source_id 列')
+        decision_field = next(
+            (field for field in ('review_decision', 'import_decision', 'approved', 'action') if field in reader.fieldnames),
+            None,
+        )
+        if decision_field is None:
+            raise ValueError('审核结果 CSV 必须包含 review_decision/import_decision/approved/action 之一')
+        for row in reader:
+            decision = str(row.get(decision_field, '')).strip().lower()
+            if decision in _APPROVED_REVIEW_DECISIONS:
+                source_id = str(row.get('source_id', '')).strip()
+                if source_id:
+                    approved.add(source_id)
+    return approved
+
+
+def main():
+    parser = argparse.ArgumentParser(description='hk_music_record 聚合数据导入至 hk_songs')
+    parser.add_argument('--limit', type=int, default=None, help='导入数据数量（默认全量）')
+    parser.add_argument('--offset', type=int, default=0, help='起始偏移量（默认 0）')
+    parser.add_argument('--skip-oss', action='store_true', help='跳过 OSS 上传，直接透传源 URL')
+    parser.add_argument('--batch-size', type=int, default=500, help='批量提交大小（默认 500）')
+    parser.add_argument('--workers', type=int, default=8, help='OSS 并发下载/上传线程数（默认 8）')
+    parser.add_argument('--skip-dedup', action='store_true', help='跳过去重，全部写入')
+    parser.add_argument('--load-existing-lyrics', action='store_true',
+                        help='启动时从目标库下载已有歌词文本构建 L2 候选集（默认关闭）')
+    parser.add_argument('--dedup-threshold', type=float, default=0.78,
+                        help='L2 去重 Jaccard 阈值（默认 0.78）')
+    parser.add_argument('--l2-recall', choices=['topk', 'full'], default='topk',
+                        help='L2 候选召回模式：topk=倒排索引召回，full=全量候选精排（默认 topk）')
+    parser.add_argument('--l2-recall-top-k', type=int, default=200,
+                        help='L2 topk 召回候选数（默认 200）')
+    parser.add_argument('--dedup-only', action='store_true',
+                        help='纯预检模式：只生成去重/人工审核清单，不上传 OSS、不写目标库')
+    parser.add_argument('--review-result-csv',
+                        help='人工审核后的 CSV；只导入 review_decision/import_decision 标记为导入的 source_id')
+    parser.add_argument('--dry-run', action='store_true', help='仅查询不写入，预览数据')
+    args = parser.parse_args()
+
+    logger.info("=" * 60)
+    logger.info("hk_music_record 聚合导入开始")
+    logger.info(f"参数: limit={args.limit}, offset={args.offset}, skip_oss={args.skip_oss}, "
+                f"batch_size={args.batch_size}, workers={args.workers}, "
+                f"skip_dedup={args.skip_dedup}, dedup_threshold={args.dedup_threshold}, "
+                f"l2_recall={args.l2_recall}, l2_recall_top_k={args.l2_recall_top_k}, "
+                f"dedup_only={args.dedup_only}, review_result_csv={args.review_result_csv}, "
+                f"dry_run={args.dry_run}")
+    logger.info(f"目标表: {TARGET_TABLE_NAME}")
+    logger.info("=" * 60)
+
+    # 初始化 OSS
+    bucket = None
+    if not args.skip_oss and not args.dedup_only:
+        try:
+            bucket = get_oss_bucket()
+            logger.info("OSS Bucket 初始化成功")
+        except Exception as e:
+            logger.error(f"OSS 初始化失败: {e}")
+            logger.info("提示: 可使用 --skip-oss 跳过 OSS 上传")
+            sys.exit(1)
+
+    # 连接目标库（提前连接，用于增量去重过滤）
+    logger.info(f"连接目标库: {TARGET_DB_CONFIG['host']}:{TARGET_DB_CONFIG['port']}/{TARGET_DB_CONFIG['database']}")
+    target_conn = pymysql.connect(**TARGET_DB_CONFIG)
+    insert_sql = INSERT_SQL_TEMPLATE.format(table=TARGET_TABLE_NAME)
+
+    # ===== 加载已导入的 source_song_id 集合（增量去重）=====
+    imported_ids: set[str] = set()
+    try:
+        id_sql = f"SELECT source_song_id FROM {TARGET_TABLE_NAME} WHERE source_table_name = 'hk_song_platform'"
+        with target_conn.cursor() as cursor:
+            cursor.execute(id_sql)
+            for r in cursor.fetchall():
+                sid = r[0] if isinstance(r, tuple) else r.get('source_song_id')
+                if sid is not None:
+                    imported_ids.add(str(sid))
+        logger.info(f"已导入 source_song_id 集合加载完成: {len(imported_ids)} 条")
+    except Exception as e:
+        logger.warning(f"加载已导入 ID 失败（首次导入可忽略）: {e}")
+
+    # 连接源库
+    logger.info(f"连接源库: {SOURCE_DB_CONFIG['host']}:{SOURCE_DB_CONFIG['port']}/{SOURCE_DB_CONFIG['database']}")
+    source_conn = pymysql.connect(**SOURCE_DB_CONFIG)
+
+    try:
+        with source_conn.cursor() as cursor:
+            sql = AGGREGATE_SQL
+            if args.limit is not None:
+                sql += f"\nLIMIT {args.limit}"
+            if args.offset:
+                sql += f"\nOFFSET {args.offset}"
+
+            logger.info("执行聚合查询...")
+            cursor.execute(sql)
+            all_rows = cursor.fetchall()
+            logger.info(f"源库查询到 {len(all_rows)} 条数据")
+
+            # 增量过滤：排除已导入的 source_song_id
+            if imported_ids:
+                rows = [r for r in all_rows if str(r['source_id']) not in imported_ids]
+                skipped = len(all_rows) - len(rows)
+                if skipped:
+                    logger.info(f"增量过滤跳过 {skipped} 条（已导入），剩余 {len(rows)} 条待处理")
+            else:
+                rows = all_rows
+
+            if args.review_result_csv:
+                approved_ids = load_approved_import_ids(args.review_result_csv)
+                before_review_filter = len(rows)
+                rows = [r for r in rows if str(r['source_id']) in approved_ids]
+                logger.info(
+                    f"人工审核结果过滤: approved={len(approved_ids)} | "
+                    f"待处理 {before_review_filter} -> {len(rows)}"
+                )
+
+            total = len(rows)
+            logger.info(f"待导入 {total} 条数据")
+
+            if total == 0:
+                logger.info("无数据需要导入")
+                return
+
+            if args.dry_run:
+                logger.info("=== DRY RUN 模式，预览前 3 条 ===")
+                for i, row in enumerate(rows[:3]):
+                    logger.info(f"Row {i+1}: source_id={row['source_id']}, name={row.get('name')}, "
+                                f"lyricist={row.get('lyricist')}, composer={row.get('composer')}, "
+                                f"audio_url_source={row.get('audio_url_source', '')[:80]}")
+                return
+
+    finally:
+        source_conn.close()
+        logger.info("源库连接已关闭")
+
+    # ===== 初始化去重 =====
+    l1_index: dict[tuple[str, str, str], int] = {}
+    l2_checker = None
+    l2_candidates: L2CandidateIndex | None = None
+    dedup_report = None
+    review_report = None
+
+    if not args.skip_dedup:
+        logger.info("正在构建 L1 元数据索引...")
+        l1_index = build_l1_index(target_conn, TARGET_TABLE_NAME)
+
+        logger.info(f"初始化 L2 歌词去重检查器 (阈值={args.dedup_threshold})")
+        l2_checker = DuplicateChecker(
+            duplicate_jaccard_threshold=args.dedup_threshold,
+        )
+        l2_candidates = L2CandidateIndex(
+            l2_checker,
+            mode=args.l2_recall,
+            top_k=args.l2_recall_top_k,
+        )
+        logger.info(f"L2 候选召回模式: {args.l2_recall} (top_k={args.l2_recall_top_k})")
+
+        # 可选加载目标库已有歌词
+        if args.load_existing_lyrics:
+            logger.info("加载目标库已有歌词文本（可能需要较长时间）...")
+            try:
+                lyric_sql = f"SELECT id, name, singer, lyricist, composer, lyrics_url FROM {TARGET_TABLE_NAME} WHERE deleted = '0' AND lyrics_url IS NOT NULL"
+                with target_conn.cursor(pymysql.cursors.DictCursor) as cursor:
+                    cursor.execute(lyric_sql)
+                    for r in cursor.fetchall():
+                        url = r.get('lyrics_url')
+                        if url and url.startswith(('http://', 'https://')):
+                            content, _ = download_file(url, timeout=10)
+                            if content:
+                                try:
+                                    text = content.decode('utf-8')
+                                except UnicodeDecodeError:
+                                    text = content.decode('utf-8', errors='replace')
+                                l2_candidates.add(LyricRecord(
+                                    record_id=str(r['id']),
+                                    lyrics=text,
+                                    title=r.get('name'),
+                                    artist=r.get('singer'),
+                                    lyricist=r.get('lyricist'),
+                                    composer=r.get('composer'),
+                                ))
+                logger.info(f"L2 候选集加载完成: {len(l2_candidates)} 条")
+            except Exception as e:
+                logger.error(f"加载已有歌词失败: {e}")
+
+        # 初始化去重报告 CSV
+        report_path = str(REPORT_DIR / f'dedup_report_{datetime.now().strftime("%Y%m%d_%H%M%S")}.csv')
+        dedup_report = DedupReport(report_path)
+        logger.info(f"去重报告将写入: {report_path}")
+        if not args.review_result_csv:
+            review_path = str(REPORT_DIR / f'review_decisions_{datetime.now().strftime("%Y%m%d_%H%M%S")}.csv')
+            review_report = ReviewDecisionReport(review_path)
+            logger.info(f"人工审核清单将写入: {review_path}")
+
+    if args.dedup_only:
+        if args.skip_dedup:
+            logger.error("--dedup-only 不能与 --skip-dedup 同时使用")
+            return
+        decision_counts = defaultdict(int)
+        try:
+            for row in tqdm(rows, desc="去重审核清单", unit="条"):
+                record_id = row['source_id']
+                record_name = row.get('name', '')
+                action = classify_dedup_action(row, l1_index, l2_checker, l2_candidates)
+                decision_counts[action['action']] += 1
+                review_report.write(row, action)
+                dedup_report.write(
+                    record_id,
+                    record_name,
+                    'DEDUP',
+                    action['action'],
+                    confidence=f"{float(action.get('confidence', 0.0)):.4f}",
+                    matched_id=action.get('matched_id') or '',
+                    reason=action.get('reason', ''),
+                )
+
+                if action['action'] == 'new':
+                    lyrics_text = row.get('lyrics_txt_content')
+                    if lyrics_text and lyrics_text.strip():
+                        l2_candidates.add(LyricRecord(
+                            record_id=str(record_id),
+                            lyrics=lyrics_text,
+                            title=record_name,
+                            artist=row.get('singer'),
+                            lyricist=row.get('lyricist'),
+                            composer=row.get('composer'),
+                        ))
+                    key = (
+                        _normalize_meta(row.get('name')),
+                        _normalize_meta(row.get('lyricist')),
+                        _normalize_meta(row.get('composer')),
+                    )
+                    if key[0] and key not in l1_index:
+                        l1_index[key] = record_id
+        finally:
+            review_report.close()
+            if dedup_report:
+                dedup_report.close()
+        logger.info(
+            f"去重审核清单完成: {review_report.filepath} | "
+            f"new={decision_counts['new']} review={decision_counts['review']} merge={decision_counts['merge']}"
+        )
+        return
+
+    # 统计计数器（线程安全）
+    stats_lock = threading.Lock()
+    stats = {
+        'inserted': 0, 'errors': 0,
+        'l1_dup': 0, 'l2_dup': 0, 'l2_review': 0, 'l2_new': 0,
+    }
+    start_time = time.time()
+
+    try:
+        # 创建进度条
+        pbar_oss = tqdm(total=total, desc="OSS 处理", unit="条", position=0, leave=True,
+                        bar_format='{l_bar}{bar}| {n_fmt}/{total_fmt} [{elapsed}<{remaining}, {rate_fmt}]')
+        pbar_db = tqdm(total=total, desc="DB 写入", unit="条", position=1, leave=True,
+                       bar_format='{l_bar}{bar}| {n_fmt}/{total_fmt} [{elapsed}<{remaining}, {rate_fmt}]')
+
+        for batch_start in range(0, total, args.batch_size):
+            batch_rows = rows[batch_start:batch_start + args.batch_size]
+            batch_size_actual = len(batch_rows)
+
+            # ===== 并发处理 OSS =====
+            # 用 dict 按索引保序
+            results_map = {}
+            tasks = [(batch_start + i, row, bucket, args.skip_oss) for i, row in enumerate(batch_rows)]
+
+            with ThreadPoolExecutor(max_workers=args.workers) as executor:
+                futures = {executor.submit(process_row_with_oss, t): t[0] for t in tasks}
+                for future in as_completed(futures):
+                    idx, tuple_row, err = future.result()
+                    if err:
+                        with stats_lock:
+                            stats['errors'] += 1
+                        logger.error(f"OSS 处理失败 source_id={batch_rows[idx - batch_start].get('source_id')}: {err}")
+                    else:
+                        results_map[idx] = tuple_row
+                    pbar_oss.update(1)
+
+            # 按原始顺序组装 batch_data
+            batch_data = [results_map[i] for i in range(batch_start, batch_start + batch_size_actual) if i in results_map]
+
+            # ===== 去重过滤 =====
+            if not args.skip_dedup and not args.review_result_csv and batch_data:
+                filtered_data = []
+                for i, tuple_row in enumerate(batch_data):
+                    orig_idx = batch_start + i
+                    row = batch_rows[i] if i < len(batch_rows) else None
+                    if row is None:
+                        filtered_data.append(tuple_row)
+                        continue
+
+                    record_id = row['source_id']
+                    record_name = row.get('name', '')
+
+                    action = classify_dedup_action(row, l1_index, l2_checker, l2_candidates)
+                    decision = action['decision']
+                    confidence = action['confidence']
+                    matched_action_id = action['matched_id']
+                    reason = action['reason']
+
+                    if action['l1_matched']:
+                        with stats_lock:
+                            stats['l1_dup'] += 1
+
+                    if action['action'] == 'merge':
+                        with stats_lock:
+                            stats['l2_dup'] += 1
+                        merge_note = '；作者字段需增量合并' if action.get('merge_authors') else ''
+                        dedup_report.write(record_id, record_name, 'L2', 'merge',
+                                           confidence=f'{confidence:.4f}',
+                                           matched_id=matched_action_id, reason=f'{reason}{merge_note}')
+                        if review_report:
+                            review_report.write(row, action)
+                        continue
+
+                    if action['action'] == 'review':
+                        with stats_lock:
+                            stats['l2_review'] += 1
+                        dedup_report.write(record_id, record_name, 'L2', 'review',
+                                           confidence=f'{confidence:.4f}',
+                                           matched_id=matched_action_id, reason=reason)
+                        if review_report:
+                            review_report.write(row, action)
+                        continue
+
+                    with stats_lock:
+                        stats['l2_new'] += 1
+                    dedup_report.write(record_id, record_name, 'L2', 'new',
+                                       confidence=f'{confidence:.4f}',
+                                       matched_id=matched_action_id or '',
+                                       reason=reason)
+                    if review_report:
+                        review_report.write(row, action)
+                    filtered_data.append(tuple_row)
+
+                    # 加入候选集供后续比对
+                    lyrics_text = row.get('lyrics_txt_content')
+                    if lyrics_text and lyrics_text.strip():
+                        l2_candidates.add(LyricRecord(
+                            record_id=str(record_id),
+                            lyrics=lyrics_text,
+                            title=record_name,
+                            artist=row.get('singer'),
+                            lyricist=row.get('lyricist'),
+                            composer=row.get('composer'),
+                        ))
+
+                batch_data = filtered_data
+
+            if not batch_data:
+                pbar_db.update(batch_size_actual)
+                continue
+
+            # ===== 顺序写入数据库（单连接，保证不出错）=====
+            try:
+                with target_conn.cursor() as cursor:
+                    cursor.executemany(insert_sql, batch_data)
+                    target_conn.commit()
+                    batch_inserted = len(batch_data)
+                    with stats_lock:
+                        stats['inserted'] += batch_inserted
+
+                    # 更新 L1 索引（防止批次内重复）
+                    if not args.skip_dedup:
+                        for i, row in enumerate(batch_rows):
+                            key = (
+                                _normalize_meta(row.get('name')),
+                                _normalize_meta(row.get('lyricist')),
+                                _normalize_meta(row.get('composer')),
+                            )
+                            if key[0] and key not in l1_index:
+                                l1_index[key] = row['source_id']
+            except Exception as e:
+                logger.error(f"批量写入失败 (offset {batch_start}): {e}")
+                target_conn.rollback()
+                # 降级逐条写入，保证其他行不受影响
+                for row_i, tuple_row in enumerate(batch_data):
+                    try:
+                        with target_conn.cursor() as cursor:
+                            cursor.execute(insert_sql, tuple_row)
+                            target_conn.commit()
+                            with stats_lock:
+                                stats['inserted'] += 1
+                    except Exception as e2:
+                        with stats_lock:
+                            stats['errors'] += 1
+                        logger.error(f"单条写入失败 (batch offset {batch_start + row_i}): {e2}")
+
+            pbar_db.update(batch_size_actual)
+
+            # 更新进度条描述（附加统计）
+            with stats_lock:
+                pbar_db.set_postfix(
+                    ins=stats['inserted'],
+                    err=stats['errors'],
+                    l1=stats['l1_dup'],
+                    l2=stats['l2_dup'],
+                    rev=stats['l2_review'],
+                    refresh=False
+                )
+
+        pbar_oss.close()
+        pbar_db.close()
+
+        elapsed_total = time.time() - start_time
+        with stats_lock:
+            logger.info("")
+            logger.info("=" * 60)
+            logger.info("导入完成!")
+            logger.info(f"总计: {total} | 插入: {stats['inserted']} | 错误: {stats['errors']}")
+            if not args.skip_dedup:
+                logger.info(f"去重统计: L1跳过={stats['l1_dup']} | "
+                            f"L2合并命中={stats['l2_dup']} | L2待审={stats['l2_review']} | "
+                            f"L2新词={stats['l2_new']}")
+                if dedup_report:
+                    logger.info(f"去重报告: {dedup_report.filepath}")
+                if review_report:
+                    logger.info(f"人工审核清单: {review_report.filepath}")
+            logger.info(f"耗时: {elapsed_total:.1f}s | 平均速度: {total/elapsed_total:.1f} 条/秒")
+            logger.info("=" * 60)
+
+    finally:
+        target_conn.close()
+        logger.info("目标库连接已关闭")
+        if dedup_report:
+            dedup_report.close()
+        if review_report:
+            review_report.close()
+
+
+if __name__ == '__main__':
+    main()

+<!doctype html>
+<html lang="zh-CN">
+<head>
+  <meta charset="utf-8">
+  <meta name="viewport" content="width=device-width, initial-scale=1">
+  <title>L2 歌词召回复核</title>
+  <style>
+    :root {
+      --bg: #f6f7f9;
+      --panel: #ffffff;
+      --line: #d7dce2;
+      --text: #17202a;
+      --muted: #64717f;
+      --accent: #1264a3;
+      --accent-soft: #e8f2fb;
+      --hit: #a13d15;
+      --hit-soft: #fff0e8;
+      --review: #7b5b00;
+      --review-soft: #fff7d7;
+      --ok: #1f7a4d;
+      --ok-soft: #e7f6ee;
+      --conflict: #b42318;
+      --conflict-soft: #fff1f0;
+      --shadow: 0 1px 2px rgba(20, 30, 40, 0.08);
+    }
+
+    * { box-sizing: border-box; }
+
+    body {
+      margin: 0;
+      font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", "PingFang SC", "Microsoft YaHei", sans-serif;
+      background: var(--bg);
+      color: var(--text);
+      letter-spacing: 0;
+    }
+
+    header {
+      height: 56px;
+      display: flex;
+      align-items: center;
+      gap: 16px;
+      padding: 0 20px;
+      background: var(--panel);
+      border-bottom: 1px solid var(--line);
+      position: sticky;
+      top: 0;
+      z-index: 5;
+    }
+
+    h1 {
+      margin: 0;
+      font-size: 18px;
+      line-height: 1.2;
+      font-weight: 650;
+      white-space: nowrap;
+    }
+
+    select, input, button {
+      height: 34px;
+      border: 1px solid var(--line);
+      background: #fff;
+      color: var(--text);
+      border-radius: 6px;
+      padding: 0 10px;
+      font-size: 14px;
+    }
+
+    button {
+      cursor: pointer;
+      background: var(--accent);
+      color: #fff;
+      border-color: var(--accent);
+      font-weight: 600;
+    }
+
+    button.secondary {
+      background: #fff;
+      color: var(--text);
+      border-color: var(--line);
+    }
+
+    button.danger {
+      background: var(--conflict);
+      border-color: var(--conflict);
+      color: #fff;
+    }
+
+    .toolbar {
+      display: flex;
+      align-items: center;
+      gap: 10px;
+      min-width: 0;
+      flex: 1;
+    }
+
+    .toolbar label {
+      color: var(--muted);
+      font-size: 13px;
+      white-space: nowrap;
+    }
+
+    main {
+      display: grid;
+      grid-template-columns: 360px 1fr;
+      height: calc(100vh - 56px);
+      min-height: 640px;
+    }
+
+    aside {
+      border-right: 1px solid var(--line);
+      background: var(--panel);
+      overflow: auto;
+      padding: 14px;
+    }
+
+    .content {
+      overflow: auto;
+      padding: 14px 18px 24px;
+    }
+
+    .section {
+      margin-bottom: 14px;
+      background: var(--panel);
+      border: 1px solid var(--line);
+      border-radius: 8px;
+      box-shadow: var(--shadow);
+    }
+
+    .section-head {
+      min-height: 42px;
+      display: flex;
+      align-items: center;
+      justify-content: space-between;
+      gap: 10px;
+      padding: 10px 12px;
+      border-bottom: 1px solid var(--line);
+    }
+
+    .section-title {
+      margin: 0;
+      font-size: 14px;
+      font-weight: 650;
+    }
+
+    .section-body {
+      padding: 12px;
+    }
+
+    .metrics {
+      display: grid;
+      grid-template-columns: repeat(2, minmax(0, 1fr));
+      gap: 8px;
+    }
+
+    .metric {
+      border: 1px solid var(--line);
+      border-radius: 6px;
+      padding: 8px;
+      background: #fbfcfd;
+    }
+
+    .metric .label {
+      color: var(--muted);
+      font-size: 12px;
+    }
+
+    .metric .value {
+      margin-top: 3px;
+      font-size: 18px;
+      font-weight: 700;
+    }
+
+    .tabs {
+      display: grid;
+      grid-template-columns: 1fr 1fr;
+      gap: 6px;
+      margin-bottom: 12px;
+    }
+
+    .tabs button {
+      background: #fff;
+      color: var(--text);
+      border-color: var(--line);
+    }
+
+    .tabs button.active {
+      background: var(--accent-soft);
+      color: var(--accent);
+      border-color: #91c1e8;
+    }
+
+    .query-list {
+      display: flex;
+      flex-direction: column;
+      gap: 8px;
+    }
+
+    .query-item {
+      width: 100%;
+      text-align: left;
+      display: flex;
+      flex-direction: column;
+      align-items: stretch;
+      background: #fff;
+      color: var(--text);
+      border: 1px solid var(--line);
+      border-radius: 7px;
+      padding: 10px;
+      min-height: 0;
+      height: auto;
+      overflow: visible;
+    }
+
+    .query-item.active {
+      border-color: var(--accent);
+      background: var(--accent-soft);
+    }
+
+    .query-item.hit {
+      border-left: 4px solid var(--hit);
+    }
+
+    .query-item.conflict {
+      border-color: var(--conflict);
+      background: var(--conflict-soft);
+    }
+
+    .query-title {
+      font-weight: 650;
+      font-size: 14px;
+      line-height: 1.25;
+      margin-bottom: 5px;
+    }
+
+    .query-meta {
+      color: var(--muted);
+      font-size: 12px;
+      line-height: 1.35;
+    }
+
+    .query-badges {
+      display: flex;
+      gap: 6px;
+      flex-wrap: wrap;
+      margin-top: 8px;
+      min-height: 22px;
+    }
+
+    .pill {
+      display: inline-flex;
+      align-items: center;
+      height: 22px;
+      border-radius: 999px;
+      padding: 0 8px;
+      font-size: 12px;
+      font-weight: 650;
+      border: 1px solid var(--line);
+      background: #fff;
+      color: var(--muted);
+      white-space: nowrap;
+    }
+
+    .pill.duplicate { background: var(--hit-soft); color: var(--hit); border-color: #efb394; }
+    .pill.review { background: var(--review-soft); color: var(--review); border-color: #e2cc72; }
+    .pill.new { background: var(--ok-soft); color: var(--ok); border-color: #93d0b0; }
+    .pill.conflict { background: var(--conflict-soft); color: var(--conflict); border-color: #f1aaa5; }
+    .pill.l1 { background: #eef1f4; color: #384452; border-color: #c8d0d9; }
+
+    .detail-grid {
+      display: grid;
+      grid-template-columns: minmax(320px, 0.9fr) minmax(420px, 1.1fr);
+      gap: 14px;
+      align-items: start;
+    }
+
+    .kv {
+      display: grid;
+      grid-template-columns: 82px 1fr;
+      gap: 6px 10px;
+      font-size: 13px;
+      line-height: 1.4;
+    }
+
+    .kv div:nth-child(odd) {
+      color: var(--muted);
+    }
+
+    .candidates {
+      display: grid;
+      grid-template-columns: repeat(auto-fill, minmax(280px, 1fr));
+      gap: 10px;
+    }
+
+    .candidate {
+      border: 1px solid var(--line);
+      border-radius: 8px;
+      background: var(--panel);
+      padding: 10px;
+      cursor: pointer;
+      min-height: 132px;
+    }
+
+    .candidate.active {
+      border-color: var(--accent);
+      box-shadow: inset 0 0 0 1px var(--accent);
+    }
+
+    .candidate.duplicate {
+      background: var(--hit-soft);
+      border-color: #efb394;
+    }
+
+    .candidate.review {
+      background: var(--review-soft);
+      border-color: #e2cc72;
+    }
+
+    .candidate.conflict {
+      border-color: var(--conflict);
+      box-shadow: inset 4px 0 0 var(--conflict);
+    }
+
+    .candidate.l1hint {
+      box-shadow: inset 4px 0 0 var(--accent);
+    }
+
+    .candidate-head {
+      display: flex;
+      align-items: flex-start;
+      justify-content: space-between;
+      gap: 8px;
+      margin-bottom: 8px;
+    }
+
+    .rank {
+      width: 26px;
+      height: 26px;
+      border-radius: 50%;
+      background: #eef1f4;
+      display: inline-flex;
+      align-items: center;
+      justify-content: center;
+      font-size: 12px;
+      font-weight: 700;
+      color: var(--muted);
+      flex: none;
+    }
+
+    .candidate-name {
+      font-weight: 700;
+      font-size: 14px;
+      line-height: 1.25;
+      word-break: break-word;
+    }
+
+    .score-row {
+      display: grid;
+      grid-template-columns: repeat(3, minmax(0, 1fr));
+      gap: 6px;
+      margin-top: 8px;
+      color: var(--muted);
+      font-size: 12px;
+    }
+
+    .lyrics-grid {
+      display: grid;
+      grid-template-columns: 1fr 1fr;
+      gap: 12px;
+    }
+
+    pre {
+      margin: 0;
+      min-height: 420px;
+      max-height: 68vh;
+      overflow: auto;
+      white-space: pre-wrap;
+      word-break: break-word;
+      font-family: "SFMono-Regular", Consolas, "Liberation Mono", monospace;
+      font-size: 13px;
+      line-height: 1.55;
+      background: #fbfcfd;
+      border: 1px solid var(--line);
+      border-radius: 8px;
+      padding: 12px;
+    }
+
+    table {
+      width: 100%;
+      border-collapse: collapse;
+      font-size: 13px;
+      background: var(--panel);
+    }
+
+    th, td {
+      border-bottom: 1px solid var(--line);
+      padding: 8px 10px;
+      text-align: left;
+      vertical-align: top;
+    }
+
+    th {
+      color: var(--muted);
+      font-size: 12px;
+      font-weight: 650;
+      background: #fbfcfd;
+      position: sticky;
+      top: 0;
+      z-index: 1;
+    }
+
+    .empty {
+      padding: 36px;
+      text-align: center;
+      color: var(--muted);
+      border: 1px dashed var(--line);
+      border-radius: 8px;
+      background: #fff;
+    }
+
+    .path {
+      color: var(--muted);
+      font-size: 12px;
+      word-break: break-all;
+    }
+
+    .review-bar {
+      display: flex;
+      gap: 8px;
+      flex-wrap: wrap;
+      align-items: center;
+    }
+
+    .review-bar button {
+      background: #fff;
+      color: var(--text);
+      border-color: var(--line);
+    }
+
+    .review-bar button.active {
+      background: var(--accent-soft);
+      color: var(--accent);
+      border-color: #91c1e8;
+    }
+
+    .review-bar input {
+      flex: 1;
+      min-width: 220px;
+    }
+
+    @media (max-width: 1100px) {
+      main { grid-template-columns: 1fr; height: auto; }
+      aside { border-right: 0; border-bottom: 1px solid var(--line); max-height: 420px; }
+      .detail-grid, .lyrics-grid { grid-template-columns: 1fr; }
+      header { height: auto; min-height: 56px; flex-wrap: wrap; padding: 10px; }
+      .toolbar { flex-wrap: wrap; }
+    }
+  </style>
+</head>
+<body>
+  <header>
+    <h1>L2 歌词召回复核</h1>
+    <div class="toolbar">
+      <label for="runSelect">结果</label>
+      <select id="runSelect"></select>
+      <label for="topKSelect">topK</label>
+      <select id="topKSelect"></select>
+      <input id="searchInput" type="search" placeholder="搜索歌名 / ID">
+      <label for="pageSizeSelect">每页</label>
+      <select id="pageSizeSelect">
+        <option value="20">20</option>
+        <option value="50" selected>50</option>
+        <option value="100">100</option>
+        <option value="200">200</option>
+      </select>
+      <button id="exportBtn">导出标注</button>
+      <button id="importReviewedBtn">入库审核通过</button>
+      <button id="reloadBtn" class="secondary">刷新</button>
+    </div>
+  </header>
+
+  <main>
+    <aside>
+      <div class="tabs">
+        <button id="tabRetrieval" class="active" data-mode="retrieval">召回样本</button>
+        <button id="tabHits" data-mode="hits">命中样本</button>
+      </div>
+      <div class="section">
+        <div class="section-head">
+          <h2 class="section-title">效率指标</h2>
+        </div>
+        <div class="section-body">
+          <div id="metrics" class="metrics"></div>
+        </div>
+      </div>
+      <div class="section">
+        <div class="section-head">
+          <h2 class="section-title">样本</h2>
+          <span id="sampleCount" class="pill"></span>
+        </div>
+        <div class="section-body">
+          <div id="queryList" class="query-list"></div>
+          <div class="review-bar" style="margin-top:10px">
+            <button id="prevPageBtn" class="secondary">上一页</button>
+            <button id="nextPageBtn" class="secondary">下一页</button>
+          </div>
+        </div>
+      </div>
+    </aside>
+
+    <section class="content">
+      <div id="detail"></div>
+    </section>
+  </main>
+
+  <script>
+    const state = {
+      runs: [],
+      run: null,
+      summary: [],
+      groups: [],
+      queryRows: [],
+      totalGroups: 0,
+      hasMore: false,
+      page: 1,
+      pageSize: 50,
+      mode: 'retrieval',
+      topK: '',
+      queryId: '',
+      candidateId: '',
+      lyricsCache: new Map()
+    };
+
+    const els = {
+      runSelect: document.getElementById('runSelect'),
+      topKSelect: document.getElementById('topKSelect'),
+      searchInput: document.getElementById('searchInput'),
+      pageSizeSelect: document.getElementById('pageSizeSelect'),
+      exportBtn: document.getElementById('exportBtn'),
+      importReviewedBtn: document.getElementById('importReviewedBtn'),
+      reloadBtn: document.getElementById('reloadBtn'),
+      prevPageBtn: document.getElementById('prevPageBtn'),
+      nextPageBtn: document.getElementById('nextPageBtn'),
+      tabRetrieval: document.getElementById('tabRetrieval'),
+      tabHits: document.getElementById('tabHits'),
+      metrics: document.getElementById('metrics'),
+      sampleCount: document.getElementById('sampleCount'),
+      queryList: document.getElementById('queryList'),
+      detail: document.getElementById('detail')
+    };
+
+    function esc(value) {
+      return String(value ?? '').replace(/[&<>"']/g, ch => ({
+        '&': '&amp;', '<': '&lt;', '>': '&gt;', '"': '&quot;', "'": '&#39;'
+      })[ch]);
+    }
+
+    function normalizeMeta(value) {
+      const punctuation = new Set(' \t\n\r，。！？；：、"“”‘’·…—～！￥（）【】《》〈〉「」『』﹏,.:;!?()[]{}<>|/\\\\_-'.split(''));
+      return String(value || '')
+        .normalize('NFKC')
+        .trim()
+        .toLowerCase()
+        .split('')
+        .filter(ch => !punctuation.has(ch))
+        .join('');
+    }
+
+    function l1Match(row) {
+      if (row.l1_metadata_match === '1') return true;
+      if (row.l1_metadata_match === '0') return false;
+      return Boolean(row.query_name) &&
+        normalizeMeta(row.query_name) === normalizeMeta(row.candidate_name || row.matched_name) &&
+        normalizeMeta(row.query_lyricist) === normalizeMeta(row.candidate_lyricist || row.matched_lyricist) &&
+        normalizeMeta(row.query_composer) === normalizeMeta(row.candidate_composer || row.matched_composer);
+    }
+
+    function rowDecision(row) {
+      return row.decision || row.candidate_decision || '';
+    }
+
+    function isConflict(row) {
+      if (row.l1_l2_conflict === '1') return true;
+      const decision = rowDecision(row);
+      const l1 = l1Match(row);
+      return (decision === 'new' && l1) || (['duplicate', 'review'].includes(decision) && !l1);
+    }
+
+    function hasL1Hint(rows) {
+      return rows.some(row => rowDecision(row) === 'new' && l1Match(row));
+    }
+
+    function hasConflict(rows) {
+      return rows.some(isConflict);
+    }
+
+    function reviewKey(row) {
+      const candidateId = row.candidate_id || row.matched_id || '';
+      return [state.run?.id || '', state.topK || '', row.query_source_id || '', candidateId].join('::');
+    }
+
+    function loadReviews() {
+      try {
+        return JSON.parse(localStorage.getItem('l2ReviewAnnotations') || '{}');
+      } catch {
+        return {};
+      }
+    }
+
+    function saveReviews(reviews) {
+      localStorage.setItem('l2ReviewAnnotations', JSON.stringify(reviews));
+    }
+
+    function getReview(row) {
+      return loadReviews()[reviewKey(row)] || {};
+    }
+
+    function setReview(row, patch) {
+      const reviews = loadReviews();
+      const key = reviewKey(row);
+      reviews[key] = { ...(reviews[key] || {}), ...patch, updated_at: new Date().toISOString() };
+      saveReviews(reviews);
+    }
+
+    function getJSON(url) {
+      return fetch(url).then(res => {
+        if (!res.ok) throw new Error(`${res.status} ${res.statusText}`);
+        return res.json();
+      });
+    }
+
+    function dataPath() {
+      if (!state.run) return '';
+      return state.mode === 'hits' ? state.run.hits : state.run.retrieval;
+    }
+
+    function metric(label, value) {
+      return `<div class="metric"><div class="label">${esc(label)}</div><div class="value">${esc(value)}</div></div>`;
+    }
+
+    function renderMetrics() {
+      const row = state.summary.find(item => String(item.top_k) === String(state.topK)) || {};
+      els.metrics.innerHTML = [
+        metric('总数', row.total_count || '-'),
+        metric('已入库 new', row.new_count || '-'),
+        metric('耗时(s)', row.elapsed_seconds || '-'),
+        metric('吞吐(条/s)', row.throughput_per_second || '-'),
+        metric('平均召回', row.avg_recalled_candidates || '-'),
+        metric('命中数', row.hit_count || '0'),
+        metric('merge', row.duplicate_count || '0'),
+        metric('review', row.review_count || '0')
+      ].join('');
+    }
+
+    function renderList() {
+      const groups = state.groups;
+      const totalPages = Math.max(1, Math.ceil(state.totalGroups / state.pageSize));
+      els.sampleCount.textContent = `${state.totalGroups} · ${state.page}/${totalPages}`;
+      if (!groups.length) {
+        els.queryList.innerHTML = '<div class="empty">没有样本</div>';
+        state.queryId = '';
+        state.queryRows = [];
+        renderDetail();
+        return;
+      }
+      els.queryList.innerHTML = groups.map(item => {
+        const hitClass = item.has_hit ? ' hit' : '';
+        const conflict = item.has_conflict ? ' conflict' : '';
+        const active = item.id === state.queryId ? ' active' : '';
+        const badges = [
+          item.has_conflict ? '<span class="pill conflict">L1/L2 冲突</span>' : '',
+          item.has_l1_hint ? '<span class="pill l1">L1 命中候选</span>' : ''
+        ].join(' ');
+        return `<button class="query-item${active}${hitClass}${conflict}" data-query-id="${esc(item.id)}">
+          <div class="query-title">${esc(item.query_name || '(无歌名)')}</div>
+          <div class="query-meta">ID ${esc(item.query_source_id)}</div>
+          <div class="query-meta">词 ${esc(item.query_lyricist || '-')} · 曲 ${esc(item.query_composer || '-')} · ${esc(item.count)} 条</div>
+          <div class="query-badges">${badges}</div>
+        </button>`;
+      }).join('');
+      for (const btn of els.queryList.querySelectorAll('.query-item')) {
+        btn.addEventListener('click', async () => {
+          state.queryId = btn.dataset.queryId;
+          state.candidateId = '';
+          await loadQueryRows();
+          renderAll();
+        });
+      }
+      els.prevPageBtn.disabled = state.page <= 1;
+      els.nextPageBtn.disabled = !state.hasMore;
+    }
+
+    function selectedQueryRows() {
+      return state.queryRows;
+    }
+
+    function selectedCandidate(rows) {
+      if (!rows.length) return null;
+      if (state.mode === 'hits') return rows[0];
+      if (!state.candidateId || !rows.some(row => row.candidate_id === state.candidateId)) {
+        state.candidateId = rows[0].candidate_id || '';
+      }
+      return rows.find(row => row.candidate_id === state.candidateId) || rows[0];
+    }
+
+    async function readText(path) {
+      if (!path) return '';
+      if (state.lyricsCache.has(path)) return state.lyricsCache.get(path);
+      const data = await getJSON(`/api/text?path=${encodeURIComponent(path)}`);
+      state.lyricsCache.set(path, data.text || '');
+      return data.text || '';
+    }
+
+    function candidateCard(row) {
+      const decision = row.candidate_decision || '';
+      const active = row.candidate_id === state.candidateId ? ' active' : '';
+      const conflict = isConflict(row);
+      const l1 = l1Match(row);
+      const extraClass = `${conflict ? ' conflict' : ''}${l1 ? ' l1hint' : ''}`;
+      return `<div class="candidate ${esc(decision)}${active}${extraClass}" data-candidate-id="${esc(row.candidate_id)}">
+        <div class="candidate-head">
+          <span class="rank">${esc(row.rank || '')}</span>
+          <div style="flex:1;min-width:0">
+            <div class="candidate-name">${esc(row.candidate_name || '(无歌名)')}</div>
+          </div>
+          <span class="pill ${esc(decision)}">${esc(decision || '-')}</span>
+        </div>
+        <div class="query-meta">词 ${esc(row.candidate_lyricist || '-')} · 曲 ${esc(row.candidate_composer || '-')}</div>
+        <div style="margin-top:7px;display:flex;gap:5px;flex-wrap:wrap">
+          ${l1 ? '<span class="pill l1">L1 元数据命中</span>' : '<span class="pill">L1 未命中</span>'}
+          ${conflict ? '<span class="pill conflict">冲突</span>' : ''}
+        </div>
+        <div class="score-row">
+          <div>conf<br><b>${esc(row.candidate_confidence || '-')}</b></div>
+          <div>jac<br><b>${esc(row.candidate_jaccard || '-')}</b></div>
+          <div>line<br><b>${esc(row.candidate_line_coverage || '-')}</b></div>
+        </div>
+      </div>`;
+    }
+
+    async function renderDetail() {
+      const rows = selectedQueryRows();
+      if (!rows.length) {
+        els.detail.innerHTML = '<div class="empty">选择一个样本查看详情</div>';
+        return;
+      }
+      const query = rows[0];
+      const candidate = selectedCandidate(rows);
+      const queryText = await readText(query.query_lyrics_path);
+      const candidatePath = state.mode === 'hits' ? candidate?.matched_lyrics_path : candidate?.candidate_lyrics_path;
+      const candidateText = await readText(candidatePath);
+      const hitDecision = state.mode === 'hits' ? candidate.decision : candidate?.candidate_decision;
+      const candidateName = state.mode === 'hits' ? candidate?.matched_name : candidate?.candidate_name;
+      const candidateLyricist = state.mode === 'hits' ? candidate?.matched_lyricist : candidate?.candidate_lyricist;
+      const candidateComposer = state.mode === 'hits' ? candidate?.matched_composer : candidate?.candidate_composer;
+      const reason = state.mode === 'hits' ? candidate?.reason : candidate?.candidate_reason;
+      const selectedReview = candidate ? getReview(candidate) : {};
+      const conflict = candidate ? isConflict(candidate) : false;
+      const l1 = candidate ? l1Match(candidate) : false;
+
+      els.detail.innerHTML = `
+        <div class="detail-grid">
+          <div class="section">
+            <div class="section-head">
+              <h2 class="section-title">新入库歌词</h2>
+              <span class="pill">ID ${esc(query.query_source_id)}</span>
+            </div>
+            <div class="section-body">
+              <div class="kv">
+                <div>歌名</div><div>${esc(query.query_name || '-')}</div>
+                <div>作词</div><div>${esc(query.query_lyricist || '-')}</div>
+                <div>作曲</div><div>${esc(query.query_composer || '-')}</div>
+                <div>歌词</div><div class="path">${esc(query.query_lyrics_path || '-')}</div>
+              </div>
+            </div>
+          </div>
+          <div class="section">
+            <div class="section-head">
+              <h2 class="section-title">${state.mode === 'hits' ? '命中候选' : '召回候选'}</h2>
+              <span class="pill ${esc(hitDecision || '')}">${esc(hitDecision || '-')}</span>
+            </div>
+            <div class="section-body">
+              <div class="kv">
+                <div>歌名</div><div>${esc(candidateName || '-')}</div>
+                <div>作词</div><div>${esc(candidateLyricist || '-')}</div>
+                <div>作曲</div><div>${esc(candidateComposer || '-')}</div>
+                <div>原因</div><div>${esc(reason || '-')}</div>
+                <div>L1</div><div>${l1 ? '<span class="pill l1">元数据命中</span>' : '<span class="pill">未命中</span>'} ${conflict ? '<span class="pill conflict">L1/L2 冲突</span>' : ''}</div>
+                <div>歌词</div><div class="path">${esc(candidatePath || '-')}</div>
+              </div>
+            </div>
+          </div>
+        </div>
+
+        <div class="section">
+          <div class="section-head"><h2 class="section-title">人工标注</h2></div>
+          <div class="section-body">
+            <div class="review-bar">
+              ${['duplicate', 'not_duplicate', 'unsure'].map(value => `
+                <button class="review-choice ${selectedReview.final_decision === value ? 'active' : ''}" data-review="${value}">
+                  ${value === 'duplicate' ? '确认重复' : value === 'not_duplicate' ? '确认不重复' : '待确认'}
+                </button>`).join('')}
+              <input id="reviewNote" value="${esc(selectedReview.note || '')}" placeholder="人工备注">
+              <button id="saveReviewBtn" class="secondary">保存</button>
+            </div>
+          </div>
+        </div>
+
+        ${state.mode === 'retrieval' ? `
+          <div class="section">
+            <div class="section-head"><h2 class="section-title">Top10 候选</h2></div>
+            <div class="section-body"><div class="candidates">${rows.map(candidateCard).join('') || '<div class="empty">无召回候选</div>'}</div></div>
+          </div>` : ''}
+
+        <div class="section">
+          <div class="section-head"><h2 class="section-title">歌词对比</h2></div>
+          <div class="section-body">
+            <div class="lyrics-grid">
+              <pre>${esc(queryText || '未读取到歌词')}</pre>
+              <pre>${esc(candidateText || '未读取到歌词')}</pre>
+            </div>
+          </div>
+        </div>`;
+
+      for (const card of els.detail.querySelectorAll('.candidate')) {
+        card.addEventListener('click', () => {
+          state.candidateId = card.dataset.candidateId;
+          renderDetail();
+        });
+      }
+      for (const btn of els.detail.querySelectorAll('.review-choice')) {
+        btn.addEventListener('click', () => {
+          if (!candidate) return;
+          setReview(candidate, { final_decision: btn.dataset.review, note: document.getElementById('reviewNote')?.value || '' });
+          renderDetail();
+        });
+      }
+      const saveBtn = document.getElementById('saveReviewBtn');
+      if (saveBtn) {
+        saveBtn.addEventListener('click', () => {
+          if (!candidate) return;
+          setReview(candidate, { note: document.getElementById('reviewNote')?.value || '' });
+          renderDetail();
+        });
+      }
+    }
+
+    function csvValue(value) {
+      const text = String(value ?? '');
+      if (/[",\n\r]/.test(text)) return `"${text.replace(/"/g, '""')}"`;
+      return text;
+    }
+
+    function exportAnnotations() {
+      const reviews = loadReviews();
+      const rows = [];
+      for (const row of state.queryRows) {
+        const review = reviews[reviewKey(row)] || {};
+        rows.push({
+          run_id: state.run?.id || '',
+          top_k: state.topK,
+          query_source_id: row.query_source_id || '',
+          query_name: row.query_name || '',
+          query_lyricist: row.query_lyricist || '',
+          query_composer: row.query_composer || '',
+          candidate_id: row.candidate_id || row.matched_id || '',
+          candidate_name: row.candidate_name || row.matched_name || '',
+          candidate_lyricist: row.candidate_lyricist || row.matched_lyricist || '',
+          candidate_composer: row.candidate_composer || row.matched_composer || '',
+          l2_decision: rowDecision(row),
+          l1_metadata_match: l1Match(row) ? '1' : '0',
+          l1_l2_conflict: isConflict(row) ? '1' : '0',
+          final_decision: review.final_decision || '',
+          note: review.note || '',
+          updated_at: review.updated_at || ''
+        });
+      }
+      const fields = Object.keys(rows[0] || {
+        run_id: '', top_k: '', query_source_id: '', query_name: '', query_lyricist: '',
+        query_composer: '', candidate_id: '', candidate_name: '', candidate_lyricist: '',
+        candidate_composer: '', l2_decision: '', l1_metadata_match: '', l1_l2_conflict: '',
+        final_decision: '', note: '', updated_at: ''
+      });
+      const csv = [fields.join(','), ...rows.map(row => fields.map(field => csvValue(row[field])).join(','))].join('\n');
+      const blob = new Blob([csv], { type: 'text/csv;charset=utf-8' });
+      const url = URL.createObjectURL(blob);
+      const a = document.createElement('a');
+      a.href = url;
+      a.download = `l2_review_annotations_${state.run?.id || 'run'}_topk_${state.topK || 'all'}_${state.mode}.csv`;
+      document.body.appendChild(a);
+      a.click();
+      a.remove();
+      URL.revokeObjectURL(url);
+    }
+
+    async function importReviewed() {
+      if (!state.run) return;
+      const reviews = loadReviews();
+      const prefix = `${state.run.id}::${state.topK || ''}::`;
+      const rows = [];
+      for (const [key, review] of Object.entries(reviews)) {
+        if (!key.startsWith(prefix)) continue;
+        if (review.final_decision !== 'not_duplicate') continue;
+        const parts = key.split('::');
+        const sourceId = parts[2] || '';
+        if (!sourceId) continue;
+        rows.push({
+          source_id: sourceId,
+          review_decision: 'import',
+          review_note: review.note || ''
+        });
+      }
+      if (!rows.length) {
+        alert('当前结果中没有标注为“确认不重复”的 review 样本。');
+        return;
+      }
+      if (!confirm(`确认入库 ${rows.length} 条审核通过样本？`)) return;
+      els.importReviewedBtn.disabled = true;
+      els.importReviewedBtn.textContent = '入库中...';
+      try {
+        const res = await fetch('/api/import-reviewed', {
+          method: 'POST',
+          headers: { 'Content-Type': 'application/json' },
+          body: JSON.stringify({ run_id: state.run.id, rows })
+        });
+        const payload = await res.json();
+        if (!res.ok) throw new Error(payload.error || payload.output || `${res.status} ${res.statusText}`);
+        alert(`入库完成。结果文件：${payload.review_csv}`);
+      } catch (err) {
+        alert(`入库失败：${err.message}`);
+      } finally {
+        els.importReviewedBtn.disabled = false;
+        els.importReviewedBtn.textContent = '入库审核通过';
+      }
+    }
+
+    function renderAll() {
+      renderMetrics();
+      renderList();
+      renderDetail();
+      els.tabRetrieval.classList.toggle('active', state.mode === 'retrieval');
+      els.tabHits.classList.toggle('active', state.mode === 'hits');
+    }
+
+    async function loadGroups() {
+      if (!state.run || !state.topK) return;
+      els.queryList.innerHTML = '<div class="empty">加载样本...</div>';
+      const params = new URLSearchParams({
+        path: dataPath(),
+        top_k: state.topK,
+        q: els.searchInput.value.trim(),
+        page: String(state.page),
+        page_size: String(state.pageSize)
+      });
+      const data = await getJSON(`/api/groups?${params.toString()}`);
+      state.groups = data.groups || [];
+      state.totalGroups = data.total || 0;
+      state.hasMore = Boolean(data.has_more);
+      if (!state.groups.some(group => group.id === state.queryId)) {
+        state.queryId = state.groups[0]?.id || '';
+        state.candidateId = '';
+      }
+      await loadQueryRows();
+    }
+
+    async function loadQueryRows() {
+      state.queryRows = [];
+      if (!state.queryId || !state.run || !state.topK) return;
+      const params = new URLSearchParams({
+        path: dataPath(),
+        top_k: state.topK,
+        query_id: state.queryId
+      });
+      const data = await getJSON(`/api/query?${params.toString()}`);
+      state.queryRows = data.rows || [];
+    }
+
+    async function loadRun(runId) {
+      state.run = state.runs.find(run => run.id === runId) || state.runs[0];
+      if (!state.run) return;
+      const summary = await getJSON(`/api/csv?path=${encodeURIComponent(state.run.summary)}`);
+      state.summary = summary.rows || [];
+      const topKs = Array.from(new Set(state.summary.map(row => row.top_k))).filter(Boolean);
+      els.topKSelect.innerHTML = topKs.map(k => `<option value="${esc(k)}">${esc(k)}</option>`).join('');
+      state.topK = topKs[0] || '';
+      els.topKSelect.value = state.topK;
+      els.pageSizeSelect.value = String(state.pageSize);
+      state.page = 1;
+      state.queryId = '';
+      state.candidateId = '';
+      await loadGroups();
+      renderAll();
+    }
+
+    async function loadRuns() {
+      const data = await getJSON('/api/reports');
+      state.runs = data.runs || [];
+      els.runSelect.innerHTML = state.runs.map(run =>
+        `<option value="${esc(run.id)}">${esc(run.id)}</option>`
+      ).join('');
+      if (!state.runs.length) {
+        els.detail.innerHTML = '<div class="empty">没有找到 l2_topk_retrieval_top10 / duplicate_hits / summary 结果文件</div>';
+        return;
+      }
+      await loadRun(state.runs[0].id);
+    }
+
+    els.runSelect.addEventListener('change', () => loadRun(els.runSelect.value));
+    els.topKSelect.addEventListener('change', async () => {
+      state.topK = els.topKSelect.value;
+      state.page = 1;
+      state.queryId = '';
+      state.candidateId = '';
+      await loadGroups();
+      renderAll();
+    });
+    els.searchInput.addEventListener('input', async () => {
+      state.page = 1;
+      state.queryId = '';
+      state.candidateId = '';
+      await loadGroups();
+      renderAll();
+    });
+    els.reloadBtn.addEventListener('click', loadRuns);
+    els.exportBtn.addEventListener('click', exportAnnotations);
+    els.importReviewedBtn.addEventListener('click', importReviewed);
+    els.pageSizeSelect.addEventListener('change', async () => {
+      state.pageSize = Number(els.pageSizeSelect.value) || 50;
+      state.page = 1;
+      state.queryId = '';
+      state.candidateId = '';
+      await loadGroups();
+      renderAll();
+    });
+    els.prevPageBtn.addEventListener('click', async () => {
+      if (state.page <= 1) return;
+      state.page -= 1;
+      state.queryId = '';
+      state.candidateId = '';
+      await loadGroups();
+      renderAll();
+    });
+    els.nextPageBtn.addEventListener('click', async () => {
+      if (!state.hasMore) return;
+      state.page += 1;
+      state.queryId = '';
+      state.candidateId = '';
+      await loadGroups();
+      renderAll();
+    });
+    els.tabRetrieval.addEventListener('click', async () => {
+      state.mode = 'retrieval';
+      state.page = 1;
+      state.queryId = '';
+      state.candidateId = '';
+      await loadGroups();
+      renderAll();
+    });
+    els.tabHits.addEventListener('click', async () => {
+      state.mode = 'hits';
+      state.page = 1;
+      state.queryId = '';
+      state.candidateId = '';
+      await loadGroups();
+      renderAll();
+    });
+
+    loadRuns().catch(err => {
+      els.detail.innerHTML = `<div class="empty">${esc(err.message)}</div>`;
+    });
+  </script>
+</body>
+</html>

+"""Lyric duplicate detection utilities."""
+
+from lyric_dedup.checker import DuplicateCheckResult
+from lyric_dedup.checker import DuplicateChecker
+from lyric_dedup.checker import DuplicateDecision
+from lyric_dedup.checker import LyricRecord
+
+__all__ = [
+    "DuplicateCheckResult",
+    "DuplicateChecker",
+    "DuplicateDecision",
+    "LyricRecord",
+]

+"""Lyric candidate ranking and duplicate decision rules."""
+
+from __future__ import annotations
+
+import hashlib
+import re
+import unicodedata
+from dataclasses import dataclass
+from enum import Enum
+
+from lyric_dedup.normalization import NormalizedLyrics
+from lyric_dedup.normalization import fingerprint_text
+from lyric_dedup.normalization import lyric_tokens
+from lyric_dedup.normalization import normalize_lyrics
+
+
+class DuplicateDecision(str, Enum):
+    DUPLICATE = "duplicate"
+    REVIEW = "review"
+    NEW = "new"
+
+
+@dataclass(frozen=True)
+class LyricRecord:
+    record_id: str
+    lyrics: str
+    title: str | None = None
+    artist: str | None = None
+    lyricist: str | None = None
+    composer: str | None = None
+
+
+@dataclass(frozen=True)
+class CandidateMatch:
+    record_id: str
+    decision: DuplicateDecision
+    confidence: float
+    jaccard: float
+    line_coverage: float
+    primary_jaccard: float
+    primary_line_coverage: float
+    translation_jaccard: float
+    translation_line_coverage: float
+    matched_unique_lines: tuple[str, ...]
+    reason: str
+
+
+@dataclass(frozen=True)
+class DuplicateCheckResult:
+    decision: DuplicateDecision
+    confidence: float
+    candidates: tuple[CandidateMatch, ...]
+    normalized_full_text: str
+    reason: str
+
+
+@dataclass(frozen=True)
+class _IndexedRecord:
+    record: LyricRecord
+    normalized: NormalizedLyrics
+    exact_hash: str
+    tokens: set[str]
+    primary_tokens: set[str]
+    translation_tokens: set[str]
+    fallback_lines: tuple[str, ...]
+    fallback_tokens: set[str]
+
+
+class DuplicateChecker:
+    """Rank PostgreSQL-recalled candidates and produce the final decision."""
+
+    def __init__(
+        self,
+        *,
+        duplicate_jaccard_threshold: float = 0.78,
+        duplicate_line_coverage_threshold: float = 0.72,
+        duplicate_high_coverage_jaccard_threshold: float = 0.78,
+        duplicate_high_coverage_line_coverage_threshold: float = 0.90,
+        review_jaccard_threshold: float = 0.45,
+        review_line_coverage_threshold: float = 0.35,
+        review_query_coverage_threshold: float = 0.40,
+        fragment_query_coverage_threshold: float = 0.80,
+        fragment_max_line_ratio: float = 0.75,
+        fragment_min_matched_lines: int = 3,
+        chorus_short_line_count_threshold: int = 6,
+        chorus_material_overlap_threshold: float = 0.20,
+        chorus_material_query_coverage_threshold: float = 0.40,
+        confidence_jaccard_weight: float = 0.58,
+        confidence_line_coverage_weight: float = 0.42,
+        metadata_duplicate_jaccard_threshold: float = 0.88,
+        metadata_duplicate_line_coverage_threshold: float = 0.65,
+        metadata_duplicate_min_primary_lines: int = 8,
+        auto_duplicate_min_primary_lines: int = 4,
+    ) -> None:
+        self.duplicate_jaccard_threshold = duplicate_jaccard_threshold
+        self.duplicate_line_coverage_threshold = duplicate_line_coverage_threshold
+        self.duplicate_high_coverage_jaccard_threshold = duplicate_high_coverage_jaccard_threshold
+        self.duplicate_high_coverage_line_coverage_threshold = duplicate_high_coverage_line_coverage_threshold
+        self.review_jaccard_threshold = review_jaccard_threshold
+        self.review_line_coverage_threshold = review_line_coverage_threshold
+        self.review_query_coverage_threshold = review_query_coverage_threshold
+        self.fragment_query_coverage_threshold = fragment_query_coverage_threshold
+        self.fragment_max_line_ratio = fragment_max_line_ratio
+        self.fragment_min_matched_lines = fragment_min_matched_lines
+        self.chorus_short_line_count_threshold = chorus_short_line_count_threshold
+        self.chorus_material_overlap_threshold = chorus_material_overlap_threshold
+        self.chorus_material_query_coverage_threshold = chorus_material_query_coverage_threshold
+        self.confidence_jaccard_weight = confidence_jaccard_weight
+        self.confidence_line_coverage_weight = confidence_line_coverage_weight
+        self.metadata_duplicate_jaccard_threshold = metadata_duplicate_jaccard_threshold
+        self.metadata_duplicate_line_coverage_threshold = metadata_duplicate_line_coverage_threshold
+        self.metadata_duplicate_min_primary_lines = metadata_duplicate_min_primary_lines
+        self.auto_duplicate_min_primary_lines = auto_duplicate_min_primary_lines
+
+    def check_record_against_candidates(
+        self,
+        record: LyricRecord,
+        candidates: list[LyricRecord],
+        *,
+        max_candidates: int = 10,
+    ) -> DuplicateCheckResult:
+        """Rank explicitly supplied candidates without doing in-memory recall.
+
+        PostgreSQL-backed callers should use this method after database recall so
+        there is only one retrieval path: PG returns candidates, Python ranks and
+        decides.
+        """
+        query = self._index(record)
+        ranked = sorted(
+            (
+                self._rank_exact_candidate(query, indexed)
+                if indexed.exact_hash == query.exact_hash
+                else self._rank_candidate(query, indexed)
+                for indexed in (self._index(candidate) for candidate in candidates)
+            ),
+            key=lambda item: (item.decision == DuplicateDecision.DUPLICATE, item.confidence, item.jaccard),
+            reverse=True,
+        )[:max_candidates]
+
+        duplicate = next((candidate for candidate in ranked if candidate.decision == DuplicateDecision.DUPLICATE), None)
+        if duplicate is not None:
+            return DuplicateCheckResult(
+                decision=DuplicateDecision.DUPLICATE,
+                confidence=duplicate.confidence,
+                candidates=tuple(ranked),
+                normalized_full_text=query.normalized.normalized_full_text,
+                reason=duplicate.reason,
+            )
+
+        review = next((candidate for candidate in ranked if candidate.decision == DuplicateDecision.REVIEW), None)
+        if review is not None:
+            return DuplicateCheckResult(
+                decision=DuplicateDecision.REVIEW,
+                confidence=review.confidence,
+                candidates=tuple(ranked),
+                normalized_full_text=query.normalized.normalized_full_text,
+                reason=review.reason,
+            )
+
+        return DuplicateCheckResult(
+            decision=DuplicateDecision.NEW,
+            confidence=1.0 - (ranked[0].confidence if ranked else 0.0),
+            candidates=tuple(ranked),
+            normalized_full_text=query.normalized.normalized_full_text,
+            reason=(
+                ranked[0].reason
+                if ranked and ranked[0].reason.startswith("无有效歌词")
+                else "精确匹配、近重复召回和字面重合信号都较低"
+            ),
+        )
+
+    def _index(self, record: LyricRecord) -> _IndexedRecord:
+        normalized = normalize_lyrics(record.lyrics)
+        return self._index_normalized(record, normalized)
+
+    def _index_normalized(self, record: LyricRecord, normalized: NormalizedLyrics) -> _IndexedRecord:
+        tokens = lyric_tokens(normalized)
+        primary_tokens = lyric_tokens(normalized, lines=normalized.primary_lines)
+        translation_tokens = lyric_tokens(normalized, lines=normalized.translation_lines)
+        fallback_lines = tuple(_fallback_no_lyrics_lines(record.lyrics))
+        fallback_tokens = set(fallback_lines)
+        exact_hash = hashlib.sha256(_exact_fingerprint(normalized, fallback_lines).encode("utf-8")).hexdigest()
+        return _IndexedRecord(
+            record=record,
+            normalized=normalized,
+            exact_hash=exact_hash,
+            tokens=tokens,
+            primary_tokens=primary_tokens,
+            translation_tokens=translation_tokens,
+            fallback_lines=fallback_lines,
+            fallback_tokens=fallback_tokens,
+        )
+
+    def _rank_exact_candidate(self, query: _IndexedRecord, candidate: _IndexedRecord) -> CandidateMatch:
+        low_confidence_split = (
+            query.normalized.split_confidence == "low" or candidate.normalized.split_confidence == "low"
+        )
+        translation_jaccard = _jaccard(query.translation_tokens, candidate.translation_tokens)
+        translation_coverage, _ = _line_coverage_lines(
+            query.normalized.translation_lines,
+            candidate.normalized.translation_lines,
+        )
+        no_effective_lyrics = not query.normalized.primary_lines and not candidate.normalized.primary_lines
+        if no_effective_lyrics:
+            decision = DuplicateDecision.NEW
+            confidence = 0.0
+            reason = "无有效歌词，不使用空内容或元数据兜底指纹自动判重"
+        elif low_confidence_split:
+            decision = DuplicateDecision.REVIEW
+            confidence = 0.95
+            reason = "原文哈希一致，但疑似整段翻译结构拆分置信度较低，需要人工复核"
+        elif _is_generic_primary_lyrics(query.normalized) or _is_generic_primary_lyrics(candidate.normalized):
+            decision = DuplicateDecision.REVIEW
+            confidence = 0.95
+            reason = "规范化后的原文哈希一致，但有效歌词过短或为通用提示，需要人工复核"
+        elif query.normalized.translation_lines or candidate.normalized.translation_lines:
+            decision = DuplicateDecision.DUPLICATE
+            confidence = 1.0
+            reason = "规范化后的原文歌词哈希完全一致，翻译行未参与自动判重"
+        else:
+            decision = DuplicateDecision.DUPLICATE
+            confidence = 1.0
+            reason = "规范化后的原文歌词哈希完全一致"
+        return CandidateMatch(
+            record_id=candidate.record.record_id,
+            decision=decision,
+            confidence=confidence,
+            jaccard=1.0,
+            line_coverage=1.0,
+            primary_jaccard=1.0,
+            primary_line_coverage=1.0,
+            translation_jaccard=round(translation_jaccard, 4),
+            translation_line_coverage=round(translation_coverage, 4),
+            matched_unique_lines=query.normalized.primary_lines,
+            reason=reason,
+        )
+
+    def _rank_candidate(self, query: _IndexedRecord, candidate: _IndexedRecord) -> CandidateMatch:
+        if not query.normalized.primary_lines or not candidate.normalized.primary_lines:
+            return _rank_no_effective_lyrics_candidate(query, candidate)
+
+        jaccard = _jaccard(query.tokens, candidate.tokens)
+        coverage, matched_lines = _line_coverage(query.normalized, candidate.normalized)
+        primary_jaccard = _jaccard(query.primary_tokens, candidate.primary_tokens)
+        primary_coverage, primary_matched_lines = _line_coverage_lines(
+            query.normalized.primary_lines,
+            candidate.normalized.primary_lines,
+        )
+        query_primary_coverage = _matched_query_line_ratio(query.normalized.primary_lines, primary_matched_lines)
+        translation_jaccard = _jaccard(query.translation_tokens, candidate.translation_tokens)
+        translation_coverage, translation_matched_lines = _line_coverage_lines(
+            query.normalized.translation_lines,
+            candidate.normalized.translation_lines,
+        )
+        chorus_only = _is_chorus_only_match(query.normalized, candidate.normalized, primary_matched_lines)
+        translation_only = (
+            bool(translation_matched_lines)
+            and primary_jaccard < self.review_jaccard_threshold
+            and primary_coverage < self.review_line_coverage_threshold
+            and (translation_jaccard >= self.review_jaccard_threshold or translation_coverage >= self.review_line_coverage_threshold)
+        )
+        low_confidence_split = (
+            query.normalized.split_confidence == "low" or candidate.normalized.split_confidence == "low"
+        )
+        query_coverage = _matched_query_line_ratio(query.normalized.unique_lines, matched_lines)
+        is_plain_fragment = _is_plain_fragment(
+            query.normalized.primary_lines,
+            candidate.normalized.primary_lines,
+            primary_matched_lines,
+            min_query_coverage=self.fragment_query_coverage_threshold,
+            max_line_ratio=self.fragment_max_line_ratio,
+            min_matched_lines=self.fragment_min_matched_lines,
+        )
+        has_review_level_overlap = (
+            primary_jaccard >= self.review_jaccard_threshold
+            or jaccard >= self.review_jaccard_threshold
+            or (
+                primary_coverage >= self.review_line_coverage_threshold
+                and query_primary_coverage >= self.review_query_coverage_threshold
+            )
+            or (
+                coverage >= self.review_line_coverage_threshold
+                and query_coverage >= self.review_query_coverage_threshold
+            )
+        )
+        has_material_chorus_overlap = chorus_only and (
+            query.normalized.content_line_count <= self.chorus_short_line_count_threshold
+            or (
+                primary_jaccard >= self.chorus_material_overlap_threshold
+                and query_primary_coverage >= self.chorus_material_query_coverage_threshold
+            )
+            or (
+                jaccard >= self.chorus_material_overlap_threshold
+                and query_coverage >= self.chorus_material_query_coverage_threshold
+            )
+            or (
+                primary_coverage >= self.chorus_material_overlap_threshold
+                and query_primary_coverage >= self.chorus_material_query_coverage_threshold
+            )
+            or (
+                coverage >= self.chorus_material_overlap_threshold
+                and query_coverage >= self.chorus_material_query_coverage_threshold
+            )
+        )
+        has_low_confidence_split_overlap = low_confidence_split and has_review_level_overlap
+        has_cover_signal = _has_cover_signal(query.record) or _has_cover_signal(candidate.record)
+        has_metadata_supported_duplicate = self._has_metadata_supported_duplicate(
+            query,
+            candidate,
+            primary_jaccard=primary_jaccard,
+            primary_coverage=primary_coverage,
+        )
+        has_enough_auto_duplicate_lyrics = _has_enough_primary_lyrics(
+            query.normalized,
+            candidate.normalized,
+            min_lines=self.auto_duplicate_min_primary_lines,
+        )
+        has_generic_primary_lyrics = (
+            _is_generic_primary_lyrics(query.normalized)
+            or _is_generic_primary_lyrics(candidate.normalized)
+        )
+
+        confidence = round(
+            (self.confidence_jaccard_weight * primary_jaccard)
+            + (self.confidence_line_coverage_weight * primary_coverage),
+            4,
+        )
+        if is_plain_fragment:
+            decision = DuplicateDecision.REVIEW
+            reason = "歌词片段只覆盖候选完整歌词的一部分，需要人工复核"
+        elif (
+            (
+                primary_jaccard >= self.duplicate_jaccard_threshold
+                or (
+                    primary_jaccard >= self.duplicate_high_coverage_jaccard_threshold
+                    and primary_coverage >= self.duplicate_high_coverage_line_coverage_threshold
+                )
+            )
+            and primary_coverage >= self.duplicate_line_coverage_threshold
+            and not chorus_only
+            and not translation_only
+            and not low_confidence_split
+            and not has_cover_signal
+            and has_enough_auto_duplicate_lyrics
+            and not has_generic_primary_lyrics
+        ):
+            decision = DuplicateDecision.DUPLICATE
+            if query.normalized.translation_lines or candidate.normalized.translation_lines:
+                reason = "原文歌词高度一致，翻译行未参与自动判重"
+            else:
+                reason = "原文 n-gram 字面相似度高，且行级覆盖范围广"
+        elif has_metadata_supported_duplicate:
+            decision = DuplicateDecision.REVIEW
+            reason = "同词曲/同标题且原文歌词主体高度一致，需要人工复核"
+        elif has_material_chorus_overlap:
+            decision = DuplicateDecision.REVIEW
+            reason = "重合内容主要集中在重复副歌行，需要人工复核"
+        elif (
+            translation_only
+            or has_low_confidence_split_overlap
+            or has_review_level_overlap
+            or (has_cover_signal and (primary_jaccard >= self.review_jaccard_threshold or primary_coverage >= self.review_line_coverage_threshold))
+        ):
+            decision = DuplicateDecision.REVIEW
+            reason = "候选相似度达到复核阈值，需要人工确认"
+            if translation_only:
+                reason = "仅翻译行相似，原文字面重合不足，不自动判重"
+            elif has_low_confidence_split_overlap:
+                reason = "疑似整段翻译结构但拆分置信度较低，需要人工复核"
+            elif has_cover_signal:
+                reason = "疑似翻唱/版本歌词相似，需要人工复核"
+        else:
+            decision = DuplicateDecision.NEW
+            reason = "候选重合度低于复核阈值"
+
+        return CandidateMatch(
+            record_id=candidate.record.record_id,
+            decision=decision,
+            confidence=confidence,
+            jaccard=round(jaccard, 4),
+            line_coverage=round(coverage, 4),
+            primary_jaccard=round(primary_jaccard, 4),
+            primary_line_coverage=round(primary_coverage, 4),
+            translation_jaccard=round(translation_jaccard, 4),
+            translation_line_coverage=round(translation_coverage, 4),
+            matched_unique_lines=tuple(matched_lines),
+            reason=reason,
+        )
+
+    def _has_metadata_supported_duplicate(
+        self,
+        query: _IndexedRecord,
+        candidate: _IndexedRecord,
+        *,
+        primary_jaccard: float,
+        primary_coverage: float,
+    ) -> bool:
+        if primary_jaccard < self.metadata_duplicate_jaccard_threshold:
+            return False
+        if primary_coverage < self.metadata_duplicate_line_coverage_threshold:
+            return False
+        if not _has_enough_primary_lyrics(
+            query.normalized,
+            candidate.normalized,
+            min_lines=self.metadata_duplicate_min_primary_lines,
+        ):
+            return False
+        if query.normalized.translation_lines or candidate.normalized.translation_lines:
+            return False
+        return _same_title(query.record, candidate.record) or _same_writers(query.record, candidate.record)
+
+
+def _rank_no_effective_lyrics_candidate(query: _IndexedRecord, candidate: _IndexedRecord) -> CandidateMatch:
+    fallback_jaccard = _jaccard(query.fallback_tokens, candidate.fallback_tokens)
+    fallback_coverage, matched_lines = _line_coverage_lines(query.fallback_lines, candidate.fallback_lines)
+    if fallback_jaccard >= 0.35 and fallback_coverage >= 0.35 and len(matched_lines) >= 2:
+        return CandidateMatch(
+            record_id=candidate.record.record_id,
+            decision=DuplicateDecision.REVIEW,
+            confidence=round((0.58 * fallback_jaccard) + (0.42 * fallback_coverage), 4),
+            jaccard=round(fallback_jaccard, 4),
+            line_coverage=round(fallback_coverage, 4),
+            primary_jaccard=0.0,
+            primary_line_coverage=0.0,
+            translation_jaccard=0.0,
+            translation_line_coverage=0.0,
+            matched_unique_lines=tuple(matched_lines),
+            reason="无有效歌词，文件内容兜底特征高度相似，需要人工复核",
+        )
+    if fallback_jaccard >= 0.2 or fallback_coverage >= 0.2:
+        return CandidateMatch(
+            record_id=candidate.record.record_id,
+            decision=DuplicateDecision.REVIEW,
+            confidence=round((0.58 * fallback_jaccard) + (0.42 * fallback_coverage), 4),
+            jaccard=round(fallback_jaccard, 4),
+            line_coverage=round(fallback_coverage, 4),
+            primary_jaccard=0.0,
+            primary_line_coverage=0.0,
+            translation_jaccard=0.0,
+            translation_line_coverage=0.0,
+            matched_unique_lines=tuple(matched_lines),
+            reason="无有效歌词，文件内容兜底特征部分相似，需要人工复核",
+        )
+    return CandidateMatch(
+        record_id=candidate.record.record_id,
+        decision=DuplicateDecision.NEW,
+        confidence=0.0,
+        jaccard=round(fallback_jaccard, 4),
+        line_coverage=round(fallback_coverage, 4),
+        primary_jaccard=0.0,
+        primary_line_coverage=0.0,
+        translation_jaccard=0.0,
+        translation_line_coverage=0.0,
+        matched_unique_lines=(),
+        reason="无有效歌词，且文件内容兜底特征未命中",
+    )
+
+
+def _jaccard(left: set[str], right: set[str]) -> float:
+    if not left and not right:
+        return 1.0
+    if not left or not right:
+        return 0.0
+    return len(left & right) / len(left | right)
+
+
+def _normalize_meta(text: str | None) -> str:
+    if not text:
+        return ""
+    punctuation = " \t\n\r，。！？；：、\"“”‘’·…—～！￥（）【】《》〈〉「」『』﹏,.:;!?()[]{}<>|/\\_-"
+    text = unicodedata.normalize("NFKC", str(text)).strip().lower()
+    return "".join(char for char in text if char not in punctuation)
+
+
+def _same_title(left: LyricRecord, right: LyricRecord) -> bool:
+    left_title = _normalize_meta(left.title)
+    return bool(left_title) and left_title == _normalize_meta(right.title)
+
+
+def _same_writers(left: LyricRecord, right: LyricRecord) -> bool:
+    left_lyricist = _normalize_meta(left.lyricist)
+    left_composer = _normalize_meta(left.composer)
+    return bool(left_lyricist or left_composer) and (
+        left_lyricist == _normalize_meta(right.lyricist)
+        and left_composer == _normalize_meta(right.composer)
+    )
+
+
+def _has_cover_signal(record: LyricRecord) -> bool:
+    haystack = " ".join(
+        part for part in (record.title, record.artist, record.lyrics[:500]) if part
+    )
+    normalized = unicodedata.normalize("NFKC", haystack).lower()
+    if any(marker in normalized for marker in ("翻唱", "原唱", "片段", "串烧", "dj版")):
+        return True
+    return bool(re.search(r"\b(?:cover|covered by|remix|live|mashup|medley|dj)\b", normalized))
+
+
+def _has_enough_primary_lyrics(
+    left: NormalizedLyrics,
+    right: NormalizedLyrics,
+    *,
+    min_lines: int,
+) -> bool:
+    lyric_lines = set(left.primary_lines) | set(right.primary_lines)
+    meaningful = [
+        line for line in lyric_lines
+        if not _is_metadata_like_line(line) and len(line) >= 4
+    ]
+    return len(meaningful) >= min_lines
+
+
+def _is_metadata_like_line(line: str) -> bool:
+    normalized = _normalize_meta(line)
+    metadata_prefixes = (
+        "词",
+        "曲",
+        "作词",
+        "作曲",
+        "编曲",
+        "原唱",
+        "和声",
+        "制作",
+        "监制",
+        "录音",
+        "混音",
+        "母带",
+        "统筹",
+        "企划",
+        "营销",
+        "项目总监",
+        "出品",
+        "发行",
+        "op",
+        "sp",
+        "lyrics",
+        "composer",
+        "writer",
+        "producer",
+        "arranger",
+    )
+    return normalized.startswith(metadata_prefixes)
+
+
+def _is_generic_primary_lyrics(normalized: NormalizedLyrics) -> bool:
+    """Reject short non-song prompts from automatic duplicate decisions."""
+    primary_lines = normalized.primary_lines
+    if not primary_lines:
+        return True
+    meaningful = [
+        line for line in primary_lines
+        if not _is_metadata_like_line(line) and len(_normalize_meta(line)) > 3
+    ]
+    if len(meaningful) > 2:
+        return False
+
+    compact = _normalize_meta(" ".join(primary_lines))
+    if len(compact) <= 3:
+        return True
+    generic_markers = (
+        "dj音乐",
+        "请欣赏",
+        "纯音乐",
+        "无歌词",
+        "伴奏",
+        "instrumental",
+    )
+    return any(marker in compact for marker in generic_markers)
+
+
+def _exact_fingerprint(normalized: NormalizedLyrics, fallback_lines: tuple[str, ...]) -> str:
+    primary_text = fingerprint_text(normalized)
+    if primary_text:
+        return f"lyrics|{primary_text}"
+    return "no_effective_lyrics_content|" + "\n".join(fallback_lines)
+
+
+def _fallback_no_lyrics_lines(text: str) -> list[str]:
+    import re
+    import unicodedata
+
+    lines: list[str] = []
+    for raw_line in unicodedata.normalize("NFKC", text).splitlines():
+        line = raw_line.strip().lower()
+        line = re.sub(r"\[(?:\d{1,2}:)?\d{1,2}:\d{2}(?:[.:]\d{1,3})?\]", "", line)
+        line = re.sub(r"[【\[].{0,80}?[】\]]", "", line)
+        if "歌词来自" in line or "qq音乐" in line or "网易云" in line or "酷狗" in line:
+            continue
+        if "未经" in line or "不得翻唱" in line or "不得翻录" in line or "著作权" in line:
+            continue
+        punctuation = "，。！？；：、“”‘’·…—～！￥（）【】《》〈〉「」『』﹏,.;:!?()[]{}<>|/\\_-"
+        line = "".join(" " if char in punctuation else char for char in line)
+        line = re.sub(r"\s+", " ", line).strip()
+        if line:
+            lines.append(line)
+    return list(dict.fromkeys(lines))
+
+
+def _line_coverage(left: NormalizedLyrics, right: NormalizedLyrics) -> tuple[float, list[str]]:
+    return _line_coverage_lines(left.unique_lines, right.unique_lines)
+
+
+def _line_coverage_lines(left: tuple[str, ...], right: tuple[str, ...]) -> tuple[float, list[str]]:
+    left_lines = set(left)
+    right_lines = set(right)
+    if not left_lines and not right_lines:
+        return 1.0, []
+    if not left_lines or not right_lines:
+        return 0.0, []
+    matched = sorted(left_lines & right_lines)
+    return len(matched) / max(len(left_lines), len(right_lines)), matched
+
+
+def _matched_query_line_ratio(query_lines: tuple[str, ...], matched_lines: list[str]) -> float:
+    query_unique_lines = set(query_lines)
+    if not query_unique_lines:
+        return 0.0
+    return len(set(matched_lines)) / len(query_unique_lines)
+
+
+def _is_plain_fragment(
+    query_lines: tuple[str, ...],
+    candidate_lines: tuple[str, ...],
+    matched_lines: list[str],
+    *,
+    min_query_coverage: float,
+    max_line_ratio: float,
+    min_matched_lines: int,
+) -> bool:
+    query_unique_lines = set(query_lines)
+    candidate_unique_lines = set(candidate_lines)
+    matched_unique_lines = set(matched_lines)
+    if not query_unique_lines or not candidate_unique_lines:
+        return False
+    if len(matched_unique_lines) < min_matched_lines:
+        return False
+    line_ratio = len(query_unique_lines) / len(candidate_unique_lines)
+    query_coverage = len(matched_unique_lines) / len(query_unique_lines)
+    return line_ratio <= max_line_ratio and query_coverage >= min_query_coverage
+
+
+def _is_chorus_only_match(left: NormalizedLyrics, right: NormalizedLyrics, matched_lines: list[str]) -> bool:
+    if not matched_lines:
+        return False
+    matched = set(matched_lines)
+    repeated_matches = [
+        line
+        for line in matched
+        if left.line_counts.get(line, 0) >= 2 or right.line_counts.get(line, 0) >= 2
+    ]
+    if len(matched) <= 2 and repeated_matches:
+        return True
+    if repeated_matches and len(repeated_matches) / len(matched) >= 0.8:
+        matched_ratio_left = sum(left.line_counts.get(line, 0) for line in matched) / max(left.content_line_count, 1)
+        matched_ratio_right = sum(right.line_counts.get(line, 0) for line in matched) / max(right.content_line_count, 1)
+        return min(matched_ratio_left, matched_ratio_right) < 0.7
+    return False

+"""PostgreSQL-backed command line tools for lyric duplicate checking."""
+
+from __future__ import annotations
+
+import argparse
+import json
+from pathlib import Path
+
+from lyric_dedup.eval_dataset import generate_eval_set
+from lyric_dedup.file_import import record_from_file
+
+
+def main() -> None:
+    parser = argparse.ArgumentParser(prog="lyric-dedup")
+    subparsers = parser.add_subparsers(dest="command", required=True)
+
+    check = subparsers.add_parser("check-file", help="check one .lrc/.txt file using PostgreSQL recall")
+    check.add_argument("--dsn", default="postgresql:///lyric_dedup")
+    check.add_argument("--file", required=True)
+    check.add_argument("--max-candidates", type=int, default=5)
+    check.add_argument("--recall-limit", type=int, default=100)
+    check.add_argument("--enable-trgm", action="store_true")
+    check.add_argument("--trgm-threshold", type=float, default=0.3)
+    check.add_argument("--statement-timeout-ms", type=int, default=5000)
+
+    generate = subparsers.add_parser("generate-eval-set", help="generate labeled eval samples from a lyric library")
+    generate.add_argument("--library-dir", required=True)
+    generate.add_argument("--lyrics-dir", required=True)
+    generate.add_argument("--csv", required=True)
+    generate.add_argument("--size", type=int, default=100)
+    generate.add_argument("--positive-ratio", type=float, default=0.3)
+    generate.add_argument("--seed", type=int, default=20260602)
+    generate.add_argument(
+        "--profile",
+        choices=("standard", "hard"),
+        default="standard",
+        help="evaluation sample profile: standard production mix or harder business-realistic edge mix",
+    )
+
+    args = parser.parse_args()
+    if args.command == "check-file":
+        check_file_pg(args)
+    elif args.command == "generate-eval-set":
+        summary = generate_eval_set(
+            library_dir=Path(args.library_dir),
+            output_dir=Path(args.lyrics_dir),
+            csv_path=Path(args.csv),
+            size=args.size,
+            positive_ratio=args.positive_ratio,
+            seed=args.seed,
+            profile=args.profile,
+        )
+        print(json.dumps(summary, ensure_ascii=False))
+
+
+def check_file_pg(args: argparse.Namespace) -> None:
+    from dedup_server.config import ServerConfig
+    from dedup_server.service import DedupService
+
+    record = record_from_file(Path(args.file))
+    config = ServerConfig(
+        dsn=args.dsn,
+        max_candidates=args.max_candidates,
+        recall_limit=args.recall_limit,
+        enable_trgm=args.enable_trgm,
+        trgm_threshold=args.trgm_threshold,
+        statement_timeout_ms=args.statement_timeout_ms,
+    )
+    service = DedupService(config=config)
+    result = service.check(record.lyrics, title=record.title, artist=record.artist)
+    print(
+        json.dumps(
+            {
+                "source": args.file,
+                "decision": result.decision,
+                "duplicate": result.duplicate,
+                "confidence": result.confidence,
+                "reason": result.reason,
+                "candidate_count": result.candidate_count,
+            },
+            ensure_ascii=False,
+            indent=2,
+        )
+    )
+
+
+if __name__ == "__main__":
+    main()

+"""Generate production-style labeled evaluation samples from a lyric library."""
+
+from __future__ import annotations
+
+import csv
+import hashlib
+import json
+import random
+import re
+import sys
+from collections import Counter
+from dataclasses import dataclass
+from pathlib import Path
+
+from lyric_dedup.checker import LyricRecord
+from lyric_dedup.file_import import iter_lyric_files
+from lyric_dedup.file_import import record_from_file
+from lyric_dedup.normalization import NormalizedLyrics
+from lyric_dedup.normalization import fingerprint_text
+from lyric_dedup.normalization import normalize_lyrics
+
+
+STANDARD_SAMPLE_MIX = {
+    "positive_full_duplicate": 0.30,
+    "negative_real_holdout_full_song": 0.40,
+    "negative_fragment": 0.10,
+    "negative_shared_chorus": 0.05,
+    "negative_translation_only": 0.05,
+    "negative_same_theme_synthetic": 0.05,
+    "edge_short_or_placeholder": 0.05,
+}
+DEFAULT_SAMPLE_MIX = STANDARD_SAMPLE_MIX
+
+HARD_SAMPLE_MIX = {
+    "positive_realistic_variant": 0.30,
+    "negative_real_holdout_full_song": 0.20,
+    "negative_near_neighbor_holdout_full_song": 0.20,
+    "negative_long_fragment": 0.15,
+    "negative_shared_chorus": 0.05,
+    "negative_translation_only": 0.04,
+    "negative_catalog_mashup": 0.04,
+    "edge_short_or_placeholder": 0.02,
+}
+
+
+def _progress(message: str) -> None:
+    print(f"[eval-gen] {message}", file=sys.stderr, flush=True)
+
+
+def _progress_count(label: str, current: int, total: int, *, step: int = 1000) -> None:
+    if total <= 0:
+        return
+    if current == 1 or current == total or current % step == 0:
+        _progress(f"{label}: {current}/{total}")
+
+
+@dataclass(frozen=True)
+class LyricProfile:
+    path: Path
+    record_id: str
+    raw_text: str
+    title: str
+    artist: str
+    normalized: NormalizedLyrics
+    line_count: int
+    char_count: int
+    line_count_bucket: str
+    language_bucket: str
+    source_bucket: str
+    normalized_hash: str
+    has_translation: bool
+
+
+@dataclass(frozen=True)
+class GeneratedSample:
+    sample_id: str
+    file: str
+    expected: str
+    sample_type: str
+    source: str
+    source_record_id: str = ""
+    candidate_record_id: str = ""
+    line_count_bucket: str = ""
+    language_bucket: str = ""
+    source_bucket: str = ""
+    title: str = ""
+    artist: str = ""
+    notes: str = ""
+
+
+def generate_eval_set(
+    *,
+    library_dir: Path,
+    output_dir: Path,
+    csv_path: Path,
+    size: int = 100,
+    positive_ratio: float = 0.30,
+    seed: int = 20260602,
+    index_path: Path | None = None,
+    eval_index_path: Path | None = None,
+    profile: str = "standard",
+) -> dict[str, object]:
+    """Generate a stratified production evaluation set.
+
+    ``positive_ratio`` is kept for CLI compatibility. It overrides the default
+    positive quota while keeping the remaining negative categories proportional.
+    """
+    if size <= 0:
+        raise ValueError("size must be positive")
+
+    if profile not in {"standard", "hard"}:
+        raise ValueError("profile must be 'standard' or 'hard'")
+
+    _progress(f"start generation: profile={profile}, size={size}, positive_ratio={positive_ratio}, seed={seed}")
+    rng = random.Random(seed)
+    profiles = profile_library(library_dir)
+    if not profiles:
+        raise ValueError(f"{library_dir} 下没有 .lrc/.txt 歌词文件")
+
+    output_dir.mkdir(parents=True, exist_ok=True)
+    csv_path.parent.mkdir(parents=True, exist_ok=True)
+    _progress(f"clean output dir: {output_dir}")
+    _clean_generated_output_dir(output_dir)
+
+    plan = _sample_plan(size, positive_ratio=positive_ratio, profile=profile)
+    _progress(f"sample plan: {plan}")
+    holdout_count = min(_holdout_plan_count(plan), max(1, len(profiles) // 2))
+    holdout_profiles = _stratified_unique_sample(
+        profiles,
+        holdout_count,
+        rng,
+    )
+    holdout_ids = {profile.record_id for profile in holdout_profiles}
+    indexed_profiles = [profile for profile in profiles if profile.record_id not in holdout_ids] or profiles
+    groups = _profile_groups(indexed_profiles)
+    samples: list[GeneratedSample] = []
+
+    if profile == "hard":
+        samples.extend(
+            _build_hard_samples(
+                plan,
+                groups=groups,
+                holdout_profiles=holdout_profiles,
+                indexed_profiles=indexed_profiles,
+                output_dir=output_dir,
+                csv_base=csv_path.parent,
+                rng=rng,
+                start_index=len(samples) + 1,
+            )
+        )
+    else:
+        _progress("build positive_full_duplicate samples")
+        samples.extend(
+            _build_positive_samples(
+                _stratified_sample(groups["normal"], plan["positive_full_duplicate"], rng),
+                output_dir,
+                csv_path.parent,
+                rng,
+                start_index=len(samples) + 1,
+            )
+        )
+        _progress(f"built samples: {len(samples)}/{size}")
+        _progress("build negative_real_holdout_full_song samples")
+        samples.extend(
+            _build_holdout_full_song_samples(
+                holdout_profiles[: plan["negative_real_holdout_full_song"]],
+                output_dir,
+                csv_path.parent,
+                start_index=len(samples) + 1,
+            )
+        )
+        _progress(f"built samples: {len(samples)}/{size}")
+        _progress("build negative_fragment samples")
+        samples.extend(
+            _build_fragment_samples(
+                _stratified_sample(groups["fragmentable"], plan["negative_fragment"], rng),
+                output_dir,
+                csv_path.parent,
+                rng,
+                start_index=len(samples) + 1,
+            )
+        )
+        _progress(f"built samples: {len(samples)}/{size}")
+        _progress("build negative_shared_chorus samples")
+        samples.extend(
+            _build_shared_chorus_samples(
+                _stratified_sample(groups["normal"], plan["negative_shared_chorus"], rng),
+                output_dir,
+                csv_path.parent,
+                rng,
+                start_index=len(samples) + 1,
+            )
+        )
+        _progress(f"built samples: {len(samples)}/{size}")
+        _progress("build negative_translation_only samples")
+        samples.extend(
+            _build_translation_only_samples(
+                _stratified_sample(groups["foreign"], plan["negative_translation_only"], rng),
+                output_dir,
+                csv_path.parent,
+                rng,
+                start_index=len(samples) + 1,
+            )
+        )
+        _progress(f"built samples: {len(samples)}/{size}")
+        _progress("build negative_same_theme_synthetic samples")
+        samples.extend(
+            _build_same_theme_synthetic_samples(
+                plan["negative_same_theme_synthetic"],
+                output_dir,
+                csv_path.parent,
+                rng,
+                start_index=len(samples) + 1,
+            )
+        )
+        _progress(f"built samples: {len(samples)}/{size}")
+        _progress("build edge_short_or_placeholder samples")
+        samples.extend(
+            _build_edge_samples(
+                _stratified_sample(groups["edge"], plan["edge_short_or_placeholder"], rng),
+                output_dir,
+                csv_path.parent,
+                rng,
+                start_index=len(samples) + 1,
+            )
+        )
+    _progress(f"built samples: {len(samples)}/{size}")
+
+    if len(samples) < size:
+        _progress(f"top up with negative_same_theme_synthetic samples: {size - len(samples)}")
+        samples.extend(
+            _build_same_theme_synthetic_samples(
+                size - len(samples),
+                output_dir,
+                csv_path.parent,
+                rng,
+                start_index=len(samples) + 1,
+            )
+        )
+    samples = samples[:size]
+    rng.shuffle(samples)
+
+    _progress(f"write csv: {csv_path}")
+    _write_csv(samples, csv_path, seed=seed)
+    _progress("write manifest")
+    manifest = _write_manifest(
+        profiles=profiles,
+        samples=samples,
+        csv_path=csv_path,
+        output_dir=output_dir,
+        seed=seed,
+        plan=plan,
+        index_path=index_path,
+        eval_index_path=eval_index_path,
+        holdout_count=len(holdout_profiles),
+        profile=profile,
+    )
+    _progress("generation complete")
+    return manifest
+
+
+def profile_library(library_dir: Path) -> list[LyricProfile]:
+    profiles: list[LyricProfile] = []
+    paths = iter_lyric_files(library_dir)
+    _progress(f"profile library: 0/{len(paths)}")
+    for index, path in enumerate(paths, start=1):
+        record = record_from_file(path, base_dir=library_dir)
+        raw_text = record.lyrics
+        normalized = normalize_lyrics(raw_text)
+        lines = normalized.primary_lines or normalized.unique_lines
+        line_count = len(lines)
+        normalized_text = fingerprint_text(normalized) or normalized.normalized_full_text
+        source_bucket = _source_bucket(path)
+        profiles.append(
+            LyricProfile(
+                path=path,
+                record_id=record.record_id,
+                raw_text=raw_text,
+                title=record.title or "",
+                artist=record.artist or "",
+                normalized=normalized,
+                line_count=line_count,
+                char_count=len(normalized_text),
+                line_count_bucket=_line_count_bucket(line_count),
+                language_bucket=_language_bucket(lines),
+                source_bucket=source_bucket,
+                normalized_hash=hashlib.sha256(normalized_text.encode("utf-8")).hexdigest(),
+                has_translation=bool(normalized.translation_lines),
+            )
+        )
+        _progress_count("profile library", index, len(paths), step=5000)
+    return profiles
+
+
+def _sample_plan(size: int, *, positive_ratio: float, profile: str) -> dict[str, int]:
+    positive_ratio = max(0.0, min(1.0, positive_ratio))
+    mix = dict(HARD_SAMPLE_MIX if profile == "hard" else STANDARD_SAMPLE_MIX)
+    positive_key = "positive_realistic_variant" if profile == "hard" else "positive_full_duplicate"
+    negative_total = sum(value for key, value in mix.items() if key != positive_key)
+    mix[positive_key] = positive_ratio
+    for key in list(mix):
+        if key != positive_key:
+            base_mix = HARD_SAMPLE_MIX if profile == "hard" else STANDARD_SAMPLE_MIX
+            mix[key] = (1.0 - positive_ratio) * (base_mix[key] / negative_total)
+
+    plan = {key: int(size * value) for key, value in mix.items()}
+    remainder = size - sum(plan.values())
+    for key in sorted(mix, key=mix.get, reverse=True):
+        if remainder <= 0:
+            break
+        plan[key] += 1
+        remainder -= 1
+    return plan
+
+
+def _holdout_plan_count(plan: dict[str, int]) -> int:
+    return plan.get("negative_real_holdout_full_song", 0) + plan.get("negative_near_neighbor_holdout_full_song", 0)
+
+
+def _profile_groups(profiles: list[LyricProfile]) -> dict[str, list[LyricProfile]]:
+    normal = [profile for profile in profiles if profile.line_count >= 6]
+    edge = [profile for profile in profiles if profile.line_count <= 5]
+    return {
+        "normal": normal or profiles,
+        "fragmentable": [profile for profile in profiles if profile.line_count >= 12] or normal or profiles,
+        "foreign": [
+            profile
+            for profile in profiles
+            if profile.language_bucket in {"latin", "mixed", "jp_kr"} and profile.line_count >= 4
+        ]
+        or normal
+        or profiles,
+        "edge": edge or normal or profiles,
+    }
+
+
+def _stratified_sample(profiles: list[LyricProfile], count: int, rng: random.Random) -> list[LyricProfile]:
+    if count <= 0 or not profiles:
+        return []
+    buckets: dict[tuple[str, str, str], list[LyricProfile]] = {}
+    for profile in profiles:
+        key = (profile.line_count_bucket, profile.language_bucket, profile.source_bucket)
+        buckets.setdefault(key, []).append(profile)
+
+    selected: list[LyricProfile] = []
+    bucket_keys = list(buckets)
+    rng.shuffle(bucket_keys)
+    cursors = {key: rng.sample(items, len(items)) for key, items in buckets.items()}
+    while len(selected) < count and bucket_keys:
+        progressed = False
+        for key in list(bucket_keys):
+            if len(selected) >= count:
+                break
+            items = cursors[key]
+            if not items:
+                bucket_keys.remove(key)
+                continue
+            selected.append(items.pop())
+            progressed = True
+        if not progressed:
+            break
+    while len(selected) < count:
+        selected.append(rng.choice(profiles))
+    return selected
+
+
+def _stratified_unique_sample(profiles: list[LyricProfile], count: int, rng: random.Random) -> list[LyricProfile]:
+    if count <= 0 or not profiles:
+        return []
+    return _stratified_sample(profiles, min(count, len(profiles)), rng)
+
+
+def _build_positive_samples(
+    profiles: list[LyricProfile],
+    output_dir: Path,
+    csv_base: Path,
+    rng: random.Random,
+    *,
+    start_index: int,
+) -> list[GeneratedSample]:
+    samples: list[GeneratedSample] = []
+    for offset, profile in enumerate(profiles):
+        raw = profile.raw_text
+        lines = _content_lines(raw)
+        variants = [
+            ("positive_exact_copy", raw),
+            ("positive_timestamped", _add_timestamps(lines)),
+            ("positive_punctuation_noise", _add_punctuation_noise(lines, rng)),
+            ("positive_platform_noise", _with_platform_noise(lines)),
+            ("positive_blank_line_noise", _add_blank_line_noise(lines)),
+            ("positive_chorus_count_changed", _change_repeated_line_counts(lines)),
+            ("positive_translation_added", _translation_added(lines)),
+            ("positive_typo_noise", _add_typo_noise(lines, rng)),
+        ]
+        sample_type, text = variants[offset % len(variants)]
+        index = start_index + offset
+        path = _write_sample_file(output_dir, f"pos_{index:05d}_{sample_type}.txt", text)
+        samples.append(_sample_from_profile(index, path, csv_base, "应去重", sample_type, profile))
+        _progress_count("positive_full_duplicate", len(samples), len(profiles))
+    return samples
+
+
+def _build_hard_samples(
+    plan: dict[str, int],
+    *,
+    groups: dict[str, list[LyricProfile]],
+    holdout_profiles: list[LyricProfile],
+    indexed_profiles: list[LyricProfile],
+    output_dir: Path,
+    csv_base: Path,
+    rng: random.Random,
+    start_index: int,
+) -> list[GeneratedSample]:
+    samples: list[GeneratedSample] = []
+
+    _progress("build positive_realistic_variant samples")
+    samples.extend(
+        _build_realistic_positive_samples(
+            _stratified_sample(groups["normal"], plan["positive_realistic_variant"], rng),
+            output_dir,
+            csv_base,
+            rng,
+            start_index=start_index + len(samples),
+        )
+    )
+    _progress(f"built samples: {len(samples)}")
+
+    real_holdout_count = plan.get("negative_real_holdout_full_song", 0)
+    _progress("build negative_real_holdout_full_song samples")
+    samples.extend(
+        _build_holdout_full_song_samples(
+            holdout_profiles[:real_holdout_count],
+            output_dir,
+            csv_base,
+            start_index=start_index + len(samples),
+        )
+    )
+    _progress(f"built samples: {len(samples)}")
+
+    near_count = plan.get("negative_near_neighbor_holdout_full_song", 0)
+    _progress("build negative_near_neighbor_holdout_full_song samples")
+    near_holdouts = _near_neighbor_holdouts(
+        holdout_profiles[real_holdout_count:],
+        indexed_profiles,
+        near_count,
+    )
+    samples.extend(
+        _build_holdout_full_song_samples(
+            near_holdouts,
+            output_dir,
+            csv_base,
+            start_index=start_index + len(samples),
+            sample_type="negative_near_neighbor_holdout_full_song",
+            notes="full real holdout lyric selected for catalog line overlap with indexed songs",
+        )
+    )
+    _progress(f"built samples: {len(samples)}")
+
+    _progress("build negative_long_fragment samples")
+    samples.extend(
+        _build_fragment_samples(
+            _stratified_sample(groups["fragmentable"], plan.get("negative_long_fragment", 0), rng),
+            output_dir,
+            csv_base,
+            rng,
+            start_index=start_index + len(samples),
+            sample_type="negative_long_fragment",
+            long_fragment=True,
+            notes="realistic long partial lyric upload, not a full-song duplicate",
+        )
+    )
+    _progress(f"built samples: {len(samples)}")
+
+    _progress("build negative_shared_chorus samples")
+    samples.extend(
+        _build_shared_chorus_samples(
+            _stratified_sample(groups["normal"], plan.get("negative_shared_chorus", 0), rng),
+            output_dir,
+            csv_base,
+            rng,
+            start_index=start_index + len(samples),
+        )
+    )
+    _progress(f"built samples: {len(samples)}")
+
+    _progress("build negative_translation_only samples")
+    samples.extend(
+        _build_translation_only_samples(
+            _stratified_sample(groups["foreign"], plan.get("negative_translation_only", 0), rng),
+            output_dir,
+            csv_base,
+            rng,
+            start_index=start_index + len(samples),
+        )
+    )
+    _progress(f"built samples: {len(samples)}")
+
+    _progress("build negative_catalog_mashup samples")
+    samples.extend(
+        _build_catalog_mashup_samples(
+            _stratified_sample(groups["normal"], plan.get("negative_catalog_mashup", 0) * 3, rng),
+            plan.get("negative_catalog_mashup", 0),
+            output_dir,
+            csv_base,
+            rng,
+            start_index=start_index + len(samples),
+        )
+    )
+    _progress(f"built samples: {len(samples)}")
+
+    _progress("build edge_short_or_placeholder samples")
+    samples.extend(
+        _build_edge_samples(
+            _stratified_sample(groups["edge"], plan.get("edge_short_or_placeholder", 0), rng),
+            output_dir,
+            csv_base,
+            rng,
+            start_index=start_index + len(samples),
+        )
+    )
+    return samples
+
+
+def _build_realistic_positive_samples(
+    profiles: list[LyricProfile],
+    output_dir: Path,
+    csv_base: Path,
+    rng: random.Random,
+    *,
+    start_index: int,
+) -> list[GeneratedSample]:
+    samples: list[GeneratedSample] = []
+    for offset, profile in enumerate(profiles):
+        content_lines = _content_lines(profile.raw_text)
+        primary_lines = list(profile.normalized.primary_lines or profile.normalized.unique_lines) or content_lines
+        variants = [
+            ("positive_platform_mixed_noise", _platform_mixed_noise(content_lines, rng)),
+            ("positive_near_full_missing_section", _near_full_missing_section(primary_lines, rng)),
+            ("positive_block_translation_added", _block_translation_added(primary_lines)),
+            ("positive_typo_and_punctuation_noise", _stronger_typo_and_punctuation_noise(content_lines, rng)),
+            ("positive_timestamped_platform_variant", _timestamped_platform_variant(content_lines)),
+            ("positive_chorus_count_changed", _change_repeated_line_counts(content_lines)),
+        ]
+        sample_type, text = variants[offset % len(variants)]
+        index = start_index + offset
+        path = _write_sample_file(output_dir, f"pos_{index:05d}_{sample_type}.txt", text)
+        samples.append(_sample_from_profile(index, path, csv_base, "应去重", sample_type, profile))
+        _progress_count("positive_realistic_variant", len(samples), len(profiles))
+    return samples
+
+
+def _build_holdout_full_song_samples(
+    profiles: list[LyricProfile],
+    output_dir: Path,
+    csv_base: Path,
+    *,
+    start_index: int,
+    sample_type: str = "negative_real_holdout_full_song",
+    notes: str = "full real lyric held out from the generated eval index",
+) -> list[GeneratedSample]:
+    samples: list[GeneratedSample] = []
+    for offset, profile in enumerate(profiles):
+        index = start_index + offset
+        text = profile.raw_text
+        path = _write_sample_file(output_dir, f"neg_{index:05d}_{sample_type}.txt", text)
+        samples.append(
+            _sample_from_profile(
+                index,
+                path,
+                csv_base,
+                "不应去重",
+                sample_type,
+                profile,
+                notes=notes,
+            )
+        )
+        _progress_count(sample_type, len(samples), len(profiles))
+    return samples
+
+
+def _build_same_theme_synthetic_samples(
+    count: int,
+    output_dir: Path,
+    csv_base: Path,
+    rng: random.Random,
+    *,
+    start_index: int,
+) -> list[GeneratedSample]:
+    samples: list[GeneratedSample] = []
+    for offset in range(count):
+        index = start_index + offset
+        text = _same_theme_synthetic(index, rng)
+        path = _write_sample_file(output_dir, f"neg_{index:05d}_negative_same_theme_synthetic.txt", text)
+        samples.append(
+            GeneratedSample(
+                sample_id=f"sample-{index:05d}",
+                file=str(path.relative_to(csv_base)),
+                expected="不应去重",
+                sample_type="negative_same_theme_synthetic",
+                source="synthetic",
+                notes="same-theme synthetic full lyric not copied from library",
+            )
+        )
+        _progress_count("negative_same_theme_synthetic", len(samples), count)
+    return samples
+
+
+def _build_fragment_samples(
+    profiles: list[LyricProfile],
+    output_dir: Path,
+    csv_base: Path,
+    rng: random.Random,
+    *,
+    start_index: int,
+    sample_type: str = "negative_fragment",
+    long_fragment: bool = False,
+    notes: str = "partial lyric fragment only",
+) -> list[GeneratedSample]:
+    samples: list[GeneratedSample] = []
+    for offset, profile in enumerate(profiles):
+        lines = list(profile.normalized.primary_lines or profile.normalized.unique_lines)
+        text = _long_song_fragment(lines, rng) if long_fragment else _single_song_fragment(lines, rng)
+        index = start_index + offset
+        path = _write_sample_file(output_dir, f"neg_{index:05d}_{sample_type}.txt", text)
+        samples.append(
+            _sample_from_profile(
+                index,
+                path,
+                csv_base,
+                "不应去重",
+                sample_type,
+                profile,
+                notes=notes,
+            )
+        )
+        _progress_count(sample_type, len(samples), len(profiles))
+    return samples
+
+
+def _build_catalog_mashup_samples(
+    profiles: list[LyricProfile],
+    count: int,
+    output_dir: Path,
+    csv_base: Path,
+    rng: random.Random,
+    *,
+    start_index: int,
+) -> list[GeneratedSample]:
+    samples: list[GeneratedSample] = []
+    if count <= 0 or not profiles:
+        return samples
+    for offset in range(count):
+        index = start_index + offset
+        picked = rng.sample(profiles, k=min(3, len(profiles)))
+        text = _catalog_mashup_text(picked, rng)
+        path = _write_sample_file(output_dir, f"neg_{index:05d}_negative_catalog_mashup.txt", text)
+        samples.append(
+            GeneratedSample(
+                sample_id=f"sample-{index:05d}",
+                file=str(path.relative_to(csv_base)),
+                expected="不应去重",
+                sample_type="negative_catalog_mashup",
+                source=" | ".join(str(profile.path) for profile in picked),
+                notes="medley-style partial lyric assembled from multiple catalog songs",
+            )
+        )
+        _progress_count("negative_catalog_mashup", len(samples), count)
+    return samples
+
+
+def _build_shared_chorus_samples(
+    profiles: list[LyricProfile],
+    output_dir: Path,
+    csv_base: Path,
+    rng: random.Random,
+    *,
+    start_index: int,
+) -> list[GeneratedSample]:
+    samples: list[GeneratedSample] = []
+    for offset, profile in enumerate(profiles):
+        lines = list(profile.normalized.primary_lines or profile.normalized.unique_lines)
+        repeated = _repeated_or_sampled_lines(profile.normalized, rng)
+        text = "\n".join(
+            [
+                "清晨的光落在新的街口",
+                "我把故事重新写给以后",
+                *repeated,
+                *repeated,
+                "所有答案都从这里开始",
+            ]
+        )
+        index = start_index + offset
+        path = _write_sample_file(output_dir, f"neg_{index:05d}_negative_shared_chorus.txt", text)
+        samples.append(
+            _sample_from_profile(
+                index,
+                path,
+                csv_base,
+                "不应去重",
+                "negative_shared_chorus",
+                profile,
+                notes="shared repeated lines with new surrounding content",
+            )
+        )
+        _progress_count("negative_shared_chorus", len(samples), len(profiles))
+    return samples
+
+
+def _build_translation_only_samples(
+    profiles: list[LyricProfile],
+    output_dir: Path,
+    csv_base: Path,
+    rng: random.Random,
+    *,
+    start_index: int,
+) -> list[GeneratedSample]:
+    samples: list[GeneratedSample] = []
+    for offset, profile in enumerate(profiles):
+        lines = list(profile.normalized.translation_lines) or [
+            _pseudo_translation(idx) for idx in range(1, min(8, max(profile.line_count, 4)) + 1)
+        ]
+        rng.shuffle(lines)
+        text = "\n".join(lines[:8])
+        index = start_index + offset
+        path = _write_sample_file(output_dir, f"neg_{index:05d}_negative_translation_only.txt", text)
+        samples.append(
+            _sample_from_profile(
+                index,
+                path,
+                csv_base,
+                "不应去重",
+                "negative_translation_only",
+                profile,
+                notes="translation-like text without matching original lyric",
+            )
+        )
+        _progress_count("negative_translation_only", len(samples), len(profiles))
+    return samples
+
+
+def _build_edge_samples(
+    profiles: list[LyricProfile],
+    output_dir: Path,
+    csv_base: Path,
+    rng: random.Random,
+    *,
+    start_index: int,
+) -> list[GeneratedSample]:
+    samples: list[GeneratedSample] = []
+    for offset, profile in enumerate(profiles):
+        lines = list(profile.normalized.primary_lines or profile.normalized.unique_lines)
+        if profile.line_count <= 1:
+            text = _same_theme_synthetic(start_index + offset, rng)
+            notes = "zero or one effective line; use synthetic edge negative"
+        else:
+            text = _short_shared_snippet(lines, rng)
+            notes = "short lyric edge case with limited overlap"
+        index = start_index + offset
+        path = _write_sample_file(output_dir, f"neg_{index:05d}_edge_short_or_placeholder.txt", text)
+        samples.append(
+            _sample_from_profile(
+                index,
+                path,
+                csv_base,
+                "不应去重",
+                "edge_short_or_placeholder",
+                profile,
+                notes=notes,
+            )
+        )
+        _progress_count("edge_short_or_placeholder", len(samples), len(profiles))
+    return samples
+
+
+def _sample_from_profile(
+    index: int,
+    path: Path,
+    csv_base: Path,
+    expected: str,
+    sample_type: str,
+    profile: LyricProfile,
+    *,
+    candidate_record_id: str = "",
+    notes: str = "",
+) -> GeneratedSample:
+    return GeneratedSample(
+        sample_id=f"sample-{index:05d}",
+        file=str(path.relative_to(csv_base)),
+        expected=expected,
+        sample_type=sample_type,
+        source=str(profile.path),
+        source_record_id=profile.record_id,
+        candidate_record_id=candidate_record_id,
+        line_count_bucket=profile.line_count_bucket,
+        language_bucket=profile.language_bucket,
+        source_bucket=profile.source_bucket,
+        title=profile.title,
+        artist=profile.artist,
+        notes=notes,
+    )
+
+
+def _write_sample_file(output_dir: Path, name: str, text: str) -> Path:
+    path = output_dir / name
+    path.write_text(text.strip() + "\n", encoding="utf-8")
+    return path
+
+
+def _write_csv(samples: list[GeneratedSample], csv_path: Path, *, seed: int) -> None:
+    fieldnames = [
+        "id",
+        "file",
+        "expected",
+        "sample_type",
+        "source",
+        "source_record_id",
+        "candidate_record_id",
+        "line_count_bucket",
+        "language_bucket",
+        "source_bucket",
+        "title",
+        "artist",
+        "seed",
+        "notes",
+    ]
+    with csv_path.open("w", encoding="utf-8", newline="") as file:
+        writer = csv.DictWriter(file, fieldnames=fieldnames)
+        writer.writeheader()
+        for sample in samples:
+            writer.writerow(
+                {
+                    "id": sample.sample_id,
+                    "file": sample.file,
+                    "expected": sample.expected,
+                    "sample_type": sample.sample_type,
+                    "source": sample.source,
+                    "source_record_id": sample.source_record_id,
+                    "candidate_record_id": sample.candidate_record_id,
+                    "line_count_bucket": sample.line_count_bucket,
+                    "language_bucket": sample.language_bucket,
+                    "source_bucket": sample.source_bucket,
+                    "title": sample.title,
+                    "artist": sample.artist,
+                    "seed": seed,
+                    "notes": sample.notes,
+                }
+            )
+
+
+def _write_manifest(
+    *,
+    profiles: list[LyricProfile],
+    samples: list[GeneratedSample],
+    csv_path: Path,
+    output_dir: Path,
+    seed: int,
+    plan: dict[str, int],
+    index_path: Path | None,
+    eval_index_path: Path,
+    holdout_count: int,
+    profile: str,
+) -> dict[str, object]:
+    manifest = {
+        "profile": profile,
+        "seed": seed,
+        "library_files": len(profiles),
+        "sample_size": len(samples),
+        "plan": plan,
+        "source_index": str(index_path) if index_path else "",
+        "eval_index": str(eval_index_path) if eval_index_path else "",
+        "holdout_records": holdout_count,
+        "lyrics_dir": str(output_dir),
+        "csv": str(csv_path),
+        "manifest": str(csv_path.with_suffix(csv_path.suffix + ".manifest.json")),
+        "sample_type_counts": dict(Counter(sample.sample_type for sample in samples)),
+        "expected_counts": dict(Counter(sample.expected for sample in samples)),
+        "line_count_bucket_counts": dict(Counter(profile.line_count_bucket for profile in profiles)),
+        "language_bucket_counts": dict(Counter(profile.language_bucket for profile in profiles)),
+        "source_bucket_counts": dict(Counter(profile.source_bucket for profile in profiles).most_common(50)),
+        "unique_source_records": len({sample.source_record_id for sample in samples if sample.source_record_id}),
+    }
+    csv_path.with_suffix(csv_path.suffix + ".manifest.json").write_text(
+        json.dumps(manifest, ensure_ascii=False, indent=2),
+        encoding="utf-8",
+    )
+    return manifest
+
+
+def _near_neighbor_holdouts(
+    holdout_profiles: list[LyricProfile],
+    indexed_profiles: list[LyricProfile],
+    count: int,
+) -> list[LyricProfile]:
+    if count <= 0 or not holdout_profiles:
+        return []
+    if not indexed_profiles:
+        return holdout_profiles[:count]
+
+    line_to_indexed_count: Counter[str] = Counter()
+    for profile in indexed_profiles:
+        for line in set(profile.normalized.primary_lines or profile.normalized.unique_lines):
+            if len(line) >= 4:
+                line_to_indexed_count[line] += 1
+
+    scored: list[tuple[float, LyricProfile]] = []
+    for profile in holdout_profiles:
+        lines = set(profile.normalized.primary_lines or profile.normalized.unique_lines)
+        useful_lines = {line for line in lines if len(line) >= 4}
+        if not useful_lines:
+            score = 0.0
+        else:
+            shared = sum(1 for line in useful_lines if line_to_indexed_count[line] > 0)
+            common_weight = sum(min(line_to_indexed_count[line], 5) for line in useful_lines)
+            score = (shared / len(useful_lines)) + (common_weight / (len(useful_lines) * 20))
+        scored.append((score, profile))
+
+    scored.sort(key=lambda item: item[0], reverse=True)
+    return [profile for _, profile in scored[:count]]
+
+
+def _content_lines(text: str) -> list[str]:
+    lines = [line.strip() for line in text.splitlines() if line.strip()]
+    return lines or [text.strip()]
+
+
+def _clean_generated_output_dir(output_dir: Path) -> None:
+    for path in output_dir.iterdir():
+        if path.is_file() and path.suffix.lower() in {".txt", ".lrc"}:
+            path.unlink()
+
+
+def _line_count_bucket(line_count: int) -> str:
+    if line_count == 0:
+        return "zero"
+    if line_count <= 5:
+        return "short"
+    if line_count <= 40:
+        return "normal"
+    return "long"
+
+
+def _language_bucket(lines: tuple[str, ...]) -> str:
+    text = "\n".join(lines)
+    cjk = len(re.findall(r"[\u4e00-\u9fff]", text))
+    latin = len(re.findall(r"[A-Za-z]", text))
+    kana = len(re.findall(r"[\u3040-\u30ff]", text))
+    hangul = len(re.findall(r"[\uac00-\ud7af]", text))
+    if kana or hangul:
+        return "jp_kr"
+    if cjk and latin:
+        return "mixed"
+    if cjk:
+        return "zh"
+    if latin:
+        return "latin"
+    return "other"
+
+
+def _source_bucket(path: Path) -> str:
+    stem = path.stem
+    parts = stem.split("_")
+    if len(parts) >= 2:
+        code = re.sub(r"\d+$", "", parts[-1])
+        return code or "unknown"
+    return "unknown"
+
+
+def _add_timestamps(lines: list[str]) -> str:
+    return "\n".join(f"[00:{idx % 60:02d}.00]{line}" for idx, line in enumerate(lines, start=1))
+
+
+def _platform_mixed_noise(lines: list[str], rng: random.Random) -> str:
+    noisy = _add_blank_line_noise(lines).splitlines()
+    if noisy:
+        noisy = _add_punctuation_noise(noisy, rng).splitlines()
+    return "\n".join(["作词：未知", "歌词来自平台同步", *noisy, "未经著作权人许可 不得商业使用"])
+
+
+def _timestamped_platform_variant(lines: list[str]) -> str:
+    timestamped = _add_timestamps(lines).splitlines()
+    return "\n".join(["[00:00.00]歌词贡献者：用户上传", *timestamped])
+
+
+def _add_punctuation_noise(lines: list[str], rng: random.Random) -> str:
+    marks = ["！", "?", "...", "，", "。"]
+    return "\n".join(f"{line}{rng.choice(marks)}" for line in lines)
+
+
+def _with_platform_noise(lines: list[str]) -> str:
+    return "\n".join(["歌词来自QQ音乐", "作词：测试", *lines, "未经著作权人许可 不得翻唱"])
+
+
+def _add_blank_line_noise(lines: list[str]) -> str:
+    result: list[str] = []
+    for idx, line in enumerate(lines, start=1):
+        result.append(line)
+        if idx % 4 == 0:
+            result.append("")
+    return "\n".join(result)
+
+
+def _change_repeated_line_counts(lines: list[str]) -> str:
+    seen: set[str] = set()
+    result: list[str] = []
+    for line in lines:
+        if line in seen:
+            continue
+        seen.add(line)
+        result.append(line)
+    return "\n".join(result or lines)
+
+
+def _translation_added(lines: list[str]) -> str:
+    result: list[str] = []
+    for idx, line in enumerate(lines, start=1):
+        result.append(line)
+        if _looks_foreign(line) and idx <= 24:
+            result.append(_pseudo_translation(idx))
+    return "\n".join(result)
+
+
+def _block_translation_added(lines: list[str]) -> str:
+    body = "\n".join(lines)
+    translation_count = min(8, max(4, len(lines) // 4))
+    translations = [_pseudo_translation(index) for index in range(1, translation_count + 1)]
+    return "\n".join([body, "", *translations])
+
+
+def _near_full_missing_section(lines: list[str], rng: random.Random) -> str:
+    if len(lines) <= 8:
+        return "\n".join(lines)
+    drop_count = max(1, min(max(1, len(lines) // 5), 8))
+    start = rng.randrange(0, max(1, len(lines) - drop_count + 1))
+    kept = lines[:start] + lines[start + drop_count :]
+    return "\n".join(kept or lines)
+
+
+def _add_typo_noise(lines: list[str], rng: random.Random) -> str:
+    if not lines:
+        return ""
+    result = list(lines)
+    editable_indexes = [index for index, line in enumerate(result) if _can_typo_line(line)]
+    if not editable_indexes:
+        return "\n".join(result)
+    typo_count = max(1, min(4, len(editable_indexes) // 8 or 1))
+    for index in rng.sample(editable_indexes, k=min(typo_count, len(editable_indexes))):
+        result[index] = _typo_line(result[index], rng)
+    return "\n".join(result)
+
+
+def _stronger_typo_and_punctuation_noise(lines: list[str], rng: random.Random) -> str:
+    if not lines:
+        return ""
+    result = _add_punctuation_noise(lines, rng).splitlines()
+    editable_indexes = [index for index, line in enumerate(result) if _can_typo_line(line)]
+    typo_count = max(1, min(8, len(editable_indexes) // 6 or 1))
+    for index in rng.sample(editable_indexes, k=min(typo_count, len(editable_indexes))):
+        result[index] = _typo_line(result[index], rng)
+    return "\n".join(result)
+
+
+def _can_typo_line(line: str) -> bool:
+    return bool(re.search(r"[A-Za-z]{4,}|[\u4e00-\u9fff]{4,}", line))
+
+
+def _typo_line(line: str, rng: random.Random) -> str:
+    words = list(re.finditer(r"[A-Za-z]{4,}", line))
+    if words and rng.random() < 0.65:
+        match = rng.choice(words)
+        typo = _typo_english_word(match.group(0), rng)
+        return line[: match.start()] + typo + line[match.end() :]
+    cjk_positions = [index for index, char in enumerate(line) if "\u4e00" <= char <= "\u9fff"]
+    if cjk_positions:
+        index = rng.choice(cjk_positions)
+        return line[:index] + _typo_cjk_char(line[index]) + line[index + 1 :]
+    return line
+
+
+def _typo_english_word(word: str, rng: random.Random) -> str:
+    if len(word) <= 4 or rng.random() < 0.55:
+        remove_at = rng.randrange(1, max(2, len(word) - 1))
+        return word[:remove_at] + word[remove_at + 1 :]
+    swap_at = rng.randrange(1, max(2, len(word) - 2))
+    chars = list(word)
+    chars[swap_at], chars[swap_at + 1] = chars[swap_at + 1], chars[swap_at]
+    return "".join(chars)
+
+
+def _typo_cjk_char(char: str) -> str:
+    replacements = {
+        "你": "妳",
+        "爱": "爰",
+        "夜": "液",
+        "里": "裏",
+        "风": "凤",
+        "雨": "兩",
+        "听": "昕",
+        "说": "説",
+        "想": "相",
+        "梦": "夣",
+        "心": "芯",
+        "光": "先",
+        "城": "诚",
+        "远": "迩",
+        "回": "囬",
+        "走": "赱",
+        "海": "毎",
+        "天": "夭",
+    }
+    return replacements.get(char, char)
+
+
+def _single_song_fragment(lines: list[str], rng: random.Random) -> str:
+    if len(lines) <= 4:
+        return "\n".join(lines[: max(1, len(lines) // 2)])
+    fragment_len = max(2, min(8, len(lines) // rng.choice([3, 4, 5])))
+    start = rng.randrange(0, max(1, len(lines) - fragment_len + 1))
+    return "\n".join(lines[start : start + fragment_len])
+
+
+def _long_song_fragment(lines: list[str], rng: random.Random) -> str:
+    if len(lines) <= 8:
+        return _single_song_fragment(lines, rng)
+    fragment_len = max(6, min(len(lines) - 1, int(len(lines) * rng.uniform(0.35, 0.60))))
+    start = rng.randrange(0, max(1, len(lines) - fragment_len + 1))
+    return "\n".join(lines[start : start + fragment_len])
+
+
+def _catalog_mashup_text(profiles: list[LyricProfile], rng: random.Random) -> str:
+    sections: list[str] = []
+    for profile in profiles:
+        lines = list(profile.normalized.primary_lines or profile.normalized.unique_lines)
+        if not lines:
+            continue
+        section_len = min(max(2, len(lines) // 8), 5)
+        start = rng.randrange(0, max(1, len(lines) - section_len + 1))
+        sections.extend(lines[start : start + section_len])
+    if not sections:
+        return _same_theme_synthetic(0, rng)
+    return "\n".join(sections)
+
+
+def _short_shared_snippet(lines: list[str], rng: random.Random) -> str:
+    snippet = rng.sample(lines, k=min(2, len(lines))) if lines else []
+    synthetic = [
+        "清晨的风吹过新的街口",
+        "我把昨天放进安静的口袋",
+        *snippet,
+        "故事从这里重新开始",
+        "灯光落下我继续往前走",
+    ]
+    return "\n".join(synthetic)
+
+
+def _repeated_or_sampled_lines(normalized: NormalizedLyrics, rng: random.Random) -> list[str]:
+    repeated = [line for line, count in normalized.line_counts.items() if count >= 2]
+    if repeated:
+        return rng.sample(repeated, k=min(2, len(repeated)))
+    lines = list(normalized.primary_lines or normalized.unique_lines)
+    return rng.sample(lines, k=min(2, len(lines))) if lines else []
+
+
+def _same_theme_synthetic(index: int, rng: random.Random) -> str:
+    starts = ["我在夜里想起远方的你", "城市灯火陪我走过雨季", "风把旧名字吹向清晨"]
+    middles = ["那些没说完的话留在风里", "新的路口慢慢亮起", "时间把答案交给下一站"]
+    ends = ["明天醒来我们各自继续", "我会把今天写成新的旋律", "故事从这里重新开始"]
+    return "\n".join(
+        [
+            rng.choice(starts),
+            rng.choice(middles),
+            rng.choice(ends),
+            f"这是第 {index} 个全新测试样本",
+        ]
+    )
+
+
+def _pseudo_translation(index: int) -> str:
+    translations = [
+        "今晚我仍然想念你",
+        "风会带走所有疲惫",
+        "黑暗里也会有光",
+        "别让昨天困住自己",
+        "我们终会继续向前",
+        "雨停以后天空会亮",
+        "把遗憾留在旧时光",
+        "你已经足够好了",
+    ]
+    return translations[(index - 1) % len(translations)]
+
+
+def _looks_foreign(line: str) -> bool:
+    latin = len(re.findall(r"[A-Za-z]", line))
+    cjk = len(re.findall(r"[\u4e00-\u9fff]", line))
+    return latin > 0 and cjk == 0

+"""Import LRC/TXT lyric files into records."""
+
+from __future__ import annotations
+
+import hashlib
+from pathlib import Path
+
+from lyric_dedup.checker import LyricRecord
+
+
+SUPPORTED_SUFFIXES = {".lrc", ".txt"}
+
+
+def iter_lyric_files(root: str | Path) -> list[Path]:
+    base = Path(root)
+    return sorted(
+        path
+        for path in base.rglob("*")
+        if path.is_file() and path.suffix.lower() in SUPPORTED_SUFFIXES
+    )
+
+
+def read_lyric_file(path: str | Path) -> str:
+    file_path = Path(path)
+    data = file_path.read_bytes()
+    for encoding in ("utf-8-sig", "utf-8", "gb18030", "big5"):
+        try:
+            return data.decode(encoding)
+        except UnicodeDecodeError:
+            continue
+    return data.decode("utf-8", errors="replace")
+
+
+def record_from_file(path: str | Path, *, base_dir: str | Path | None = None) -> LyricRecord:
+    file_path = Path(path)
+    lyrics = read_lyric_file(file_path)
+    title, artist = _metadata_from_name(file_path.stem)
+    record_id = _record_id(file_path, base_dir)
+    return LyricRecord(record_id=record_id, lyrics=lyrics, title=title, artist=artist)
+
+
+def records_from_dir(root: str | Path) -> list[LyricRecord]:
+    return [record_from_file(path, base_dir=root) for path in iter_lyric_files(root)]
+
+
+def _record_id(path: Path, base_dir: str | Path | None) -> str:
+    if base_dir is None:
+        source = str(path.resolve())
+    else:
+        source = str(path.resolve().relative_to(Path(base_dir).resolve()))
+    digest = hashlib.sha1(source.encode("utf-8")).hexdigest()[:12]
+    return f"{digest}:{source}"
+
+
+def _metadata_from_name(stem: str) -> tuple[str | None, str | None]:
+    cleaned = stem.removesuffix("-歌词").removesuffix("_歌词").removesuffix(" 歌词").strip()
+    if " - " in cleaned:
+        artist, title = cleaned.split(" - ", 1)
+        return title.strip() or None, artist.strip() or None
+    for sep in ("-", "_"):
+        if sep in cleaned:
+            title, artist = cleaned.rsplit(sep, 1)
+            return title.strip() or None, artist.strip() or None
+    return stem.strip() or None, None

+"""Lyric-specific normalization and feature extraction."""
+
+from __future__ import annotations
+
+import re
+import string
+import unicodedata
+from collections import Counter
+from dataclasses import dataclass
+
+import opencc
+
+
+_T2S_CONVERTER = opencc.OpenCC("t2s")
+
+_TIMESTAMP_RE = re.compile(r"\[((?:\d{1,2}:)?\d{1,2}:\d{2}(?:[.:]\d{1,3})?)\]")
+_BRACKET_RE = re.compile(r"[\[(（【<《].{0,40}?[\])）】>》]")
+_ROLE_PREFIX_RE = re.compile(r"^\s*(?:男|女|合|主歌|副歌|verse|chorus|bridge|rap)\s*[:：]\s*", re.IGNORECASE)
+_CREDIT_PREFIX_RE = re.compile(
+    r"^\s*(?:作词|作詞|作曲|编曲|編曲|制作|製作|监制|監製|录音|錄音|混音|母带|"
+    r"出品|发行|發行|歌词|歌詞|lyric(?:s)?|composer|writer|producer|arranger|"
+    r"copyright|未经|未經|qq音乐|酷狗|网易云|網易雲|lrc)",
+    re.IGNORECASE,
+)
+_WATERMARK_RE = re.compile(
+    r"(?:qq音乐|酷狗音乐|网易云音乐|網易雲音樂|虾米音乐|歌词网|歌詞網|"
+    r"music\.163\.com|www\.|http[s]?://|\blrc\b)",
+    re.IGNORECASE,
+)
+_INSTRUMENTAL_RE = re.compile(
+    r"(?:纯音乐|純音樂|无歌词|無歌詞|没有歌词|沒有歌詞|instrumental)",
+    re.IGNORECASE,
+)
+_CJK_RE = re.compile(r"[\u4e00-\u9fff]")
+_LATIN_RE = re.compile(r"[a-zA-Z]")
+_KANA_RE = re.compile(r"[\u3040-\u30ff]")
+_HANGUL_RE = re.compile(r"[\uac00-\ud7af]")
+_WORD_RE = re.compile(r"[a-z0-9]+|[\u4e00-\u9fff]", re.IGNORECASE)
+_INLINE_SPLIT_RE = re.compile(r"\s+(?:/|\||｜)\s+|(?<=[A-Za-z])\s*[-—]\s*(?=[\u4e00-\u9fff])")
+
+
+@dataclass(frozen=True)
+class _LineEntry:
+    text: str
+    timestamp: str | None
+    language: str
+    source_index: int
+
+
+@dataclass(frozen=True)
+class NormalizedLyrics:
+    raw_text: str
+    normalized_full_text: str
+    normalized_lines: tuple[str, ...]
+    unique_lines: tuple[str, ...]
+    line_counts: dict[str, int]
+    content_line_count: int
+    primary_lines: tuple[str, ...]
+    translation_lines: tuple[str, ...]
+    unknown_lines: tuple[str, ...]
+    line_roles: tuple[str, ...]
+    split_confidence: str
+    split_reason: str
+
+
+def normalize_lyrics(text: str) -> NormalizedLyrics:
+    """Normalize lyrics while preserving line-level structure for ranking."""
+    entries: list[_LineEntry] = []
+    for index, raw_line in enumerate(unicodedata.normalize("NFKC", text).splitlines()):
+        entries.extend(_clean_line_entries(raw_line, index))
+
+    cleaned_lines = [entry.text for entry in entries]
+    roles, confidence, reason = _assign_line_roles(entries)
+    primary_lines = tuple(entry.text for entry, role in zip(entries, roles, strict=False) if role == "primary")
+    translation_lines = tuple(entry.text for entry, role in zip(entries, roles, strict=False) if role == "translation")
+    unknown_lines = tuple(entry.text for entry, role in zip(entries, roles, strict=False) if role == "unknown")
+    if not primary_lines:
+        primary_lines = tuple(cleaned_lines)
+        roles = tuple("primary" for _ in cleaned_lines)
+        if cleaned_lines and confidence == "none":
+            reason = "未检测到可分离的翻译结构，全部有效行按原文处理"
+
+    counts = Counter(cleaned_lines)
+    unique_lines = tuple(dict.fromkeys(cleaned_lines))
+    return NormalizedLyrics(
+        raw_text=text,
+        normalized_full_text="\n".join(cleaned_lines),
+        normalized_lines=tuple(cleaned_lines),
+        unique_lines=unique_lines,
+        line_counts=dict(counts),
+        content_line_count=len(cleaned_lines),
+        primary_lines=tuple(dict.fromkeys(primary_lines)),
+        translation_lines=tuple(dict.fromkeys(translation_lines)),
+        unknown_lines=tuple(dict.fromkeys(unknown_lines)),
+        line_roles=tuple(roles),
+        split_confidence=confidence,
+        split_reason=reason,
+    )
+
+
+def fingerprint_text(normalized: NormalizedLyrics) -> str:
+    """Return a text form suitable for exact hashing.
+
+    Repeated adjacent or non-adjacent lyric lines are collapsed so different chorus
+    repeat counts do not prevent exact duplicate detection.
+    """
+    return "\n".join(normalized.primary_lines or normalized.unique_lines)
+
+
+def lyric_tokens(
+    normalized: NormalizedLyrics,
+    ngram_size: int = 3,
+    *,
+    lines: tuple[str, ...] | None = None,
+) -> set[str]:
+    """Build mixed CJK/Latin n-grams with repeated lines down-weighted."""
+    tokens: set[str] = set()
+    selected_lines = lines if lines is not None else normalized.unique_lines
+    for line in selected_lines:
+        units = _token_units(line)
+        if len(units) < ngram_size:
+            if units:
+                tokens.add(" ".join(units))
+            continue
+        for start in range(len(units) - ngram_size + 1):
+            tokens.add(" ".join(units[start : start + ngram_size]))
+    return tokens
+
+
+def _clean_line_entries(raw_line: str, source_index: int) -> list[_LineEntry]:
+    timestamp_match = _TIMESTAMP_RE.search(raw_line)
+    timestamp = timestamp_match.group(1) if timestamp_match else None
+    line = _TIMESTAMP_RE.sub("", raw_line)
+    line = _ROLE_PREFIX_RE.sub("", line).strip()
+    inline_entries = _split_inline_translation(line, timestamp, source_index)
+    if inline_entries:
+        return inline_entries
+    return _entry_from_text(line, timestamp, source_index)
+
+
+def _split_inline_translation(line: str, timestamp: str | None, source_index: int) -> list[_LineEntry]:
+    parts = [part.strip() for part in _INLINE_SPLIT_RE.split(line, maxsplit=1)]
+    if len(parts) != 2:
+        return []
+    left_entries = _entry_from_text(parts[0], timestamp, source_index)
+    right_entries = _entry_from_text(parts[1], timestamp, source_index)
+    if not left_entries or not right_entries:
+        return []
+    left_lang = left_entries[0].language
+    right_lang = right_entries[0].language
+    if _is_foreign_language(left_lang) and right_lang == "zh":
+        return [left_entries[0], right_entries[0]]
+    if left_lang == "zh" and _is_foreign_language(right_lang):
+        return [right_entries[0], left_entries[0]]
+    return []
+
+
+def _entry_from_text(text: str, timestamp: str | None, source_index: int) -> list[_LineEntry]:
+    line = _BRACKET_RE.sub("", text)
+    line = _T2S_CONVERTER.convert(line.strip().lower())
+    if not line or _is_noise_line(line):
+        return []
+    line = _strip_symbols(line)
+    if not line:
+        return []
+    return [_LineEntry(text=line, timestamp=timestamp, language=_detect_language(line), source_index=source_index)]
+
+
+def _assign_line_roles(entries: list[_LineEntry]) -> tuple[tuple[str, ...], str, str]:
+    if not entries:
+        return (), "none", "没有有效歌词行"
+
+    timestamp_roles = _roles_by_same_timestamp(entries)
+    if timestamp_roles is not None:
+        return timestamp_roles, "high", "同时间戳下检测到外文行和中文行配对"
+
+    inline_roles = _roles_by_inline_translation(entries)
+    if inline_roles is not None:
+        return inline_roles, "medium", "同一原始行内检测到明显的外文和中文翻译"
+
+    alternating_roles = _roles_by_alternating_translation(entries)
+    if alternating_roles is not None:
+        return alternating_roles, "high", "检测到稳定的外文行和中文翻译行交替结构"
+
+    block_roles = _roles_by_translation_block(entries)
+    if block_roles is not None:
+        return block_roles, "low", "检测到疑似原文段落加中文翻译段落，置信度较低"
+
+    return tuple("primary" for _ in entries), "none", "未检测到可分离的翻译结构，全部有效行按原文处理"
+
+
+def _roles_by_same_timestamp(entries: list[_LineEntry]) -> tuple[str, ...] | None:
+    roles = ["unknown"] * len(entries)
+    groups: dict[str, list[int]] = {}
+    for idx, entry in enumerate(entries):
+        if entry.timestamp:
+            groups.setdefault(entry.timestamp, []).append(idx)
+
+    paired = 0
+    for indexes in groups.values():
+        if len(indexes) < 2:
+            continue
+        foreign = [idx for idx in indexes if _is_foreign_language(entries[idx].language)]
+        chinese = [idx for idx in indexes if entries[idx].language == "zh"]
+        if not foreign or not chinese:
+            continue
+        for idx in foreign:
+            roles[idx] = "primary"
+        for idx in chinese:
+            roles[idx] = "translation"
+        paired += 1
+
+    if paired == 0:
+        return None
+    for idx, role in enumerate(roles):
+        if role == "unknown":
+            roles[idx] = "primary"
+    return tuple(roles)
+
+
+def _roles_by_alternating_translation(entries: list[_LineEntry]) -> tuple[str, ...] | None:
+    roles = ["unknown"] * len(entries)
+    pairs = 0
+    idx = 0
+    while idx < len(entries) - 1:
+        current = entries[idx]
+        nxt = entries[idx + 1]
+        if _is_foreign_language(current.language) and nxt.language == "zh":
+            roles[idx] = "primary"
+            roles[idx + 1] = "translation"
+            pairs += 1
+            idx += 2
+            continue
+        idx += 1
+
+    if pairs < 2:
+        return None
+    assigned = sum(1 for role in roles if role != "unknown")
+    if assigned / len(entries) < 0.65:
+        return None
+    for idx, role in enumerate(roles):
+        if role == "unknown":
+            roles[idx] = "primary"
+    return tuple(roles)
+
+
+def _roles_by_inline_translation(entries: list[_LineEntry]) -> tuple[str, ...] | None:
+    roles = ["primary"] * len(entries)
+    pairs = 0
+    by_source: dict[int, list[int]] = {}
+    for idx, entry in enumerate(entries):
+        by_source.setdefault(entry.source_index, []).append(idx)
+    for indexes in by_source.values():
+        if len(indexes) != 2:
+            continue
+        first, second = indexes
+        if _is_foreign_language(entries[first].language) and entries[second].language == "zh":
+            roles[first] = "primary"
+            roles[second] = "translation"
+            pairs += 1
+        elif entries[first].language == "zh" and _is_foreign_language(entries[second].language):
+            roles[first] = "translation"
+            roles[second] = "primary"
+            pairs += 1
+    return tuple(roles) if pairs else None
+
+
+def _roles_by_translation_block(entries: list[_LineEntry]) -> tuple[str, ...] | None:
+    if len(entries) < 4:
+        return None
+    midpoint = len(entries) // 2
+    first = entries[:midpoint]
+    second = entries[midpoint:]
+    first_foreign = sum(1 for entry in first if _is_foreign_language(entry.language))
+    second_zh = sum(1 for entry in second if entry.language == "zh")
+    if first_foreign / len(first) >= 0.75 and second_zh / len(second) >= 0.75:
+        return tuple("primary" if idx < midpoint else "translation" for idx in range(len(entries)))
+    return None
+
+
+def _detect_language(line: str) -> str:
+    cjk = len(_CJK_RE.findall(line))
+    latin = len(_LATIN_RE.findall(line))
+    kana = len(_KANA_RE.findall(line))
+    hangul = len(_HANGUL_RE.findall(line))
+    if hangul:
+        return "kr"
+    if kana:
+        return "jp"
+    if cjk and latin:
+        return "mixed"
+    if cjk:
+        return "zh"
+    if latin:
+        return "latin"
+    return "other"
+
+
+def _is_foreign_language(language: str) -> bool:
+    return language in {"latin", "jp", "kr", "other"}
+
+
+def _is_noise_line(line: str) -> bool:
+    if _CREDIT_PREFIX_RE.search(line) or _WATERMARK_RE.search(line):
+        return True
+    if _INSTRUMENTAL_RE.search(line):
+        return True
+    has_cjk_or_latin = bool(_CJK_RE.search(line) or _LATIN_RE.search(line))
+    if not has_cjk_or_latin:
+        return True
+    compact = _strip_symbols(line)
+    return len(compact) <= 1
+
+
+def _strip_symbols(line: str) -> str:
+    punctuation = string.punctuation + "，。！？；：、“”‘’·…—～！￥（）【】《》〈〉「」『』﹏"
+    line = "".join(" " if char in punctuation else char for char in line)
+    line = re.sub(r"\s+", " ", line)
+    line = re.sub(r"(?<=[\u4e00-\u9fff])\s+(?=[\u4e00-\u9fff])", "", line)
+    return line.strip()
+
+
+def _token_units(line: str) -> list[str]:
+    units: list[str] = []
+    for match in _WORD_RE.finditer(line):
+        token = match.group(0).lower()
+        if _CJK_RE.fullmatch(token):
+            units.append(token)
+        else:
+            units.append(token)
+    return units

+pymysql
+python-dotenv
+oss2
+requests
+tqdm
+opencc-python-reimplemented

+#!/usr/bin/env python3
+"""Local server for the L2 lyric review dashboard."""
+
+from __future__ import annotations
+
+import argparse
+import csv
+import json
+import mimetypes
+import subprocess
+import sys
+import time
+from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
+from pathlib import Path
+from urllib.parse import parse_qs, unquote, urlparse
+
+
+ROOT = Path(__file__).resolve().parent
+REPORT_DIR = ROOT / "output" / "reports"
+DASHBOARD = ROOT / "l2_review_dashboard.html"
+REPORT_DIR.mkdir(parents=True, exist_ok=True)
+GROUP_INDEX_CACHE: dict[tuple[str, int, int, str], list[dict[str, object]]] = {}
+
+
+def _json_response(handler: BaseHTTPRequestHandler, payload: object, status: int = 200) -> None:
+    body = json.dumps(payload, ensure_ascii=False).encode("utf-8")
+    try:
+        handler.send_response(status)
+        handler.send_header("Content-Type", "application/json; charset=utf-8")
+        handler.send_header("Content-Length", str(len(body)))
+        handler.end_headers()
+        handler.wfile.write(body)
+    except (BrokenPipeError, ConnectionResetError):
+        return
+
+
+def _error(handler: BaseHTTPRequestHandler, message: str, status: int = 400) -> None:
+    _json_response(handler, {"error": message}, status=status)
+
+
+def _safe_path(raw_path: str) -> Path:
+    path = Path(unquote(raw_path)).expanduser()
+    if not path.is_absolute():
+        path = ROOT / path
+    resolved = path.resolve()
+    allowed_roots = [ROOT.resolve(), REPORT_DIR.resolve()]
+    if not any(resolved == base or base in resolved.parents for base in allowed_roots):
+        raise ValueError(f"path is outside workspace: {raw_path}")
+    if not resolved.exists() or not resolved.is_file():
+        raise FileNotFoundError(str(resolved))
+    return resolved
+
+
+def _read_csv(path: Path) -> list[dict[str, str]]:
+    with open(path, "r", encoding="utf-8-sig", newline="") as f:
+        return list(csv.DictReader(f))
+
+
+def _iter_csv(path: Path):
+    with open(path, "r", encoding="utf-8-sig", newline="") as f:
+        yield from csv.DictReader(f)
+
+
+def _report_runs() -> list[dict[str, str]]:
+    summaries = {
+        path.name.replace("l2_topk_benchmark_summary_", "").removesuffix(".csv"): path
+        for path in REPORT_DIR.glob("l2_topk_benchmark_summary_*.csv")
+    }
+    retrievals = {
+        path.name.replace("l2_topk_retrieval_top10_", "").removesuffix(".csv"): path
+        for path in REPORT_DIR.glob("l2_topk_retrieval_top10_*.csv")
+    }
+    hits = {
+        path.name.replace("l2_topk_duplicate_hits_", "").removesuffix(".csv"): path
+        for path in REPORT_DIR.glob("l2_topk_duplicate_hits_*.csv")
+    }
+    run_ids = sorted(set(summaries) & set(retrievals) & set(hits), reverse=True)
+    benchmark_runs = [
+        {
+            "id": run_id,
+            "type": "benchmark",
+            "summary": str(summaries[run_id]),
+            "retrieval": str(retrievals[run_id]),
+            "hits": str(hits[run_id]),
+        }
+        for run_id in run_ids
+    ]
+    review_runs = [
+        {
+            "id": path.name.replace("review_decisions_", "").removesuffix(".csv"),
+            "type": "import",
+            "summary": str(path),
+            "retrieval": str(path),
+            "hits": str(path),
+            "review": str(path),
+        }
+        for path in sorted(REPORT_DIR.glob("review_decisions_*.csv"), reverse=True)
+    ]
+    return review_runs + benchmark_runs
+
+
+def _row_decision(row: dict[str, str]) -> str:
+    return row.get("action") or row.get("decision") or row.get("candidate_decision") or ""
+
+
+def _is_import_review_row(row: dict[str, str]) -> bool:
+    return "source_id" in row and "action" in row and "query_source_id" not in row
+
+
+def _dashboard_row(row: dict[str, str]) -> dict[str, str]:
+    if not _is_import_review_row(row):
+        return row
+    source_id = row.get("source_id", "")
+    action = row.get("action", "")
+    return {
+        **row,
+        "top_k": "import",
+        "rank": "1",
+        "query_source_id": source_id,
+        "query_name": row.get("name", ""),
+        "query_lyricist": row.get("lyricist", ""),
+        "query_composer": row.get("composer", ""),
+        "query_lyrics_path": row.get("query_lyrics_path", ""),
+        "candidate_id": row.get("matched_id", ""),
+        "candidate_name": row.get("matched_name", ""),
+        "candidate_lyricist": row.get("matched_lyricist", ""),
+        "candidate_composer": row.get("matched_composer", ""),
+        "candidate_lyrics_path": row.get("matched_lyrics_path", ""),
+        "candidate_decision": action,
+        "candidate_confidence": row.get("confidence", ""),
+        "candidate_reason": row.get("reason", ""),
+        "decision": action,
+        "l1_metadata_match": "1" if row.get("l1_matched_id") else "0",
+        "l1_l2_conflict": "0",
+    }
+
+
+def _review_summary(path: Path) -> list[dict[str, str]]:
+    counts = {"new": 0, "merge": 0, "review": 0}
+    total = 0
+    for row in _iter_csv(path):
+        total += 1
+        action = row.get("action", "")
+        if action in counts:
+            counts[action] += 1
+    return [{
+        "top_k": "import",
+        "elapsed_seconds": "-",
+        "throughput_per_second": "-",
+        "avg_recalled_candidates": "-",
+        "hit_count": str(counts["merge"] + counts["review"]),
+        "duplicate_count": str(counts["merge"]),
+        "review_count": str(counts["review"]),
+        "new_count": str(counts["new"]),
+        "total_count": str(total),
+    }]
+
+
+def _norm(value: str | None) -> str:
+    return (value or "").strip().lower()
+
+
+def _matches_term(row: dict[str, object], term: str) -> bool:
+    if not term:
+        return True
+    haystack = " ".join(
+        [
+            row.get("query_source_id", ""),
+            row.get("query_name", ""),
+            row.get("query_lyricist", ""),
+            row.get("query_composer", ""),
+        ]
+    ).lower()
+    return term in haystack
+
+
+def _group_index(path: Path, top_k: str) -> list[dict[str, object]]:
+    stat = path.stat()
+    cache_key = (str(path), stat.st_mtime_ns, stat.st_size, top_k)
+    cached = GROUP_INDEX_CACHE.get(cache_key)
+    if cached is not None:
+        return cached
+
+    groups_by_id: dict[str, dict[str, object]] = {}
+    order = 0
+    for raw_row in _iter_csv(path):
+        row = _dashboard_row(raw_row)
+        if top_k and str(row.get("top_k", "")) != top_k:
+            continue
+        query_id = row.get("query_source_id", "")
+        if not query_id:
+            continue
+
+        group = groups_by_id.get(query_id)
+        if group is None:
+            group = {
+                "id": query_id,
+                "_order": order,
+                "query_source_id": query_id,
+                "query_name": row.get("query_name", ""),
+                "query_lyricist": row.get("query_lyricist", ""),
+                "query_composer": row.get("query_composer", ""),
+                "count": 0,
+                "has_hit": False,
+                "has_conflict": False,
+                "has_l1_hint": False,
+            }
+            groups_by_id[query_id] = group
+            order += 1
+
+        group["count"] = int(group["count"]) + 1
+        decision = _row_decision(row)
+        l1_match = row.get("l1_metadata_match") == "1"
+        conflict = row.get("l1_l2_conflict") == "1"
+        group["has_hit"] = bool(group["has_hit"]) or decision in {"duplicate", "review"}
+        group["has_conflict"] = bool(group["has_conflict"]) or conflict
+        group["has_l1_hint"] = bool(group["has_l1_hint"]) or (decision == "new" and l1_match)
+
+    groups = sorted(
+        groups_by_id.values(),
+        key=lambda group: (
+            0 if group["has_conflict"] else 1,
+            0 if group["has_l1_hint"] else 1,
+            0 if group["has_hit"] else 1,
+            int(group["_order"]),
+        ),
+    )
+    GROUP_INDEX_CACHE.clear()
+    GROUP_INDEX_CACHE[cache_key] = groups
+    return groups
+
+
+def _groups_response(path: Path, top_k: str, term: str, page: int, page_size: int) -> dict[str, object]:
+    filtered = [group for group in _group_index(path, top_k) if _matches_term(group, term)]
+    start = max(0, (page - 1) * page_size)
+    end = start + page_size
+    page_groups = filtered[start:end]
+    return {
+        "total": len(filtered),
+        "page": page,
+        "page_size": page_size,
+        "has_more": end < len(filtered),
+        "groups": [{key: value for key, value in group.items() if not key.startswith("_")} for group in page_groups],
+    }
+
+
+def _query_rows_response(path: Path, top_k: str, query_id: str) -> dict[str, object]:
+    rows = []
+    found = False
+    for raw_row in _iter_csv(path):
+        row = _dashboard_row(raw_row)
+        if str(row.get("top_k", "")) != top_k:
+            continue
+        current_query_id = row.get("query_source_id", "")
+        if current_query_id == query_id:
+            found = True
+            rows.append(row)
+            continue
+        if found:
+            break
+    return {"rows": rows}
+
+
+def _write_review_import_csv(rows: list[dict[str, str]]) -> Path:
+    path = REPORT_DIR / f"review_import_approved_{time.strftime('%Y%m%d_%H%M%S')}.csv"
+    with open(path, "w", encoding="utf-8", newline="") as f:
+        writer = csv.DictWriter(f, fieldnames=["source_id", "review_decision", "review_note"])
+        writer.writeheader()
+        for row in rows:
+            writer.writerow({
+                "source_id": row.get("source_id", ""),
+                "review_decision": row.get("review_decision", ""),
+                "review_note": row.get("review_note", ""),
+            })
+    return path
+
+
+def _run_import(review_csv: Path) -> dict[str, object]:
+    cmd = [
+        "uv",
+        "run",
+        "python",
+        "import_hk_songs.py",
+        "--review-result-csv",
+        str(review_csv),
+        "--load-existing-lyrics",
+    ]
+    result = subprocess.run(
+        cmd,
+        cwd=ROOT,
+        text=True,
+        stdout=subprocess.PIPE,
+        stderr=subprocess.STDOUT,
+        check=False,
+    )
+    return {
+        "command": " ".join(cmd),
+        "returncode": result.returncode,
+        "output": result.stdout[-12000:],
+    }
+
+
+class Handler(BaseHTTPRequestHandler):
+    def log_message(self, fmt: str, *args: object) -> None:
+        print(f"{self.address_string()} - {fmt % args}")
+
+    def do_GET(self) -> None:
+        parsed = urlparse(self.path)
+        if parsed.path == "/":
+            self._serve_file(DASHBOARD)
+            return
+        if parsed.path == "/api/reports":
+            _json_response(self, {"runs": _report_runs()})
+            return
+        if parsed.path == "/api/csv":
+            params = parse_qs(parsed.query)
+            raw_path = params.get("path", [""])[0]
+            try:
+                path = _safe_path(raw_path)
+                if path.name.startswith("review_decisions_"):
+                    _json_response(self, {"path": str(path), "rows": _review_summary(path)})
+                else:
+                    _json_response(self, {"path": str(path), "rows": _read_csv(path)})
+            except Exception as exc:  # noqa: BLE001 - local diagnostic API
+                _error(self, str(exc), status=404)
+            return
+        if parsed.path == "/api/groups":
+            params = parse_qs(parsed.query)
+            try:
+                path = _safe_path(params.get("path", [""])[0])
+                top_k = params.get("top_k", [""])[0]
+                term = _norm(params.get("q", [""])[0])
+                page = max(1, int(params.get("page", ["1"])[0]))
+                page_size = min(200, max(10, int(params.get("page_size", ["50"])[0])))
+                _json_response(self, _groups_response(path, top_k, term, page, page_size))
+            except Exception as exc:  # noqa: BLE001 - local diagnostic API
+                _error(self, str(exc), status=404)
+            return
+        if parsed.path == "/api/query":
+            params = parse_qs(parsed.query)
+            try:
+                path = _safe_path(params.get("path", [""])[0])
+                top_k = params.get("top_k", [""])[0]
+                query_id = params.get("query_id", [""])[0]
+                _json_response(self, _query_rows_response(path, top_k, query_id))
+            except Exception as exc:  # noqa: BLE001 - local diagnostic API
+                _error(self, str(exc), status=404)
+            return
+        if parsed.path == "/api/text":
+            params = parse_qs(parsed.query)
+            raw_path = params.get("path", [""])[0]
+            try:
+                path = _safe_path(raw_path)
+                text = path.read_text(encoding="utf-8", errors="replace")
+                _json_response(self, {"path": str(path), "text": text})
+            except Exception as exc:  # noqa: BLE001 - local diagnostic API
+                _error(self, str(exc), status=404)
+            return
+        if parsed.path.startswith("/static/"):
+            self._serve_file(ROOT / parsed.path.lstrip("/"))
+            return
+        _error(self, "not found", status=404)
+
+    def do_POST(self) -> None:
+        parsed = urlparse(self.path)
+        if parsed.path != "/api/import-reviewed":
+            _error(self, "not found", status=404)
+            return
+        try:
+            length = int(self.headers.get("Content-Length", "0"))
+            payload = json.loads(self.rfile.read(length).decode("utf-8") or "{}")
+            rows = payload.get("rows") or []
+            if not isinstance(rows, list):
+                raise ValueError("rows must be a list")
+            approved = [
+                {
+                    "source_id": str(row.get("source_id", "")).strip(),
+                    "review_decision": "import",
+                    "review_note": str(row.get("review_note", "")).strip(),
+                }
+                for row in rows
+                if str(row.get("source_id", "")).strip()
+                and str(row.get("review_decision", "")).strip().lower() in {"import", "导入", "not_duplicate"}
+            ]
+            if not approved:
+                raise ValueError("没有可入库的审核通过样本")
+            review_csv = _write_review_import_csv(approved)
+            result = _run_import(review_csv)
+            status = 200 if result["returncode"] == 0 else 500
+            _json_response(self, {"review_csv": str(review_csv), **result}, status=status)
+        except Exception as exc:  # noqa: BLE001 - local diagnostic API
+            _error(self, str(exc), status=400)
+
+    def _serve_file(self, path: Path) -> None:
+        if not path.exists() or not path.is_file():
+            _error(self, "not found", status=404)
+            return
+        content_type = mimetypes.guess_type(path.name)[0] or "application/octet-stream"
+        body = path.read_bytes()
+        self.send_response(200)
+        self.send_header("Content-Type", content_type)
+        self.send_header("Content-Length", str(len(body)))
+        self.end_headers()
+        self.wfile.write(body)
+
+
+def main() -> None:
+    parser = argparse.ArgumentParser(description="Serve L2 lyric review dashboard")
+    parser.add_argument("--host", default="127.0.0.1")
+    parser.add_argument("--port", type=int, default=8765)
+    args = parser.parse_args()
+
+    server = ThreadingHTTPServer((args.host, args.port), Handler)
+    print(f"L2 dashboard: http://{args.host}:{args.port}")
+    server.serve_forever()
+
+
+if __name__ == "__main__":
+    try:
+        main()
+    except KeyboardInterrupt:
+        sys.exit(0)

+#!/usr/bin/env python3
+"""
+歌词去重入库功能完整测试用例
+覆盖 L1（元数据匹配）和 L2（歌词内容匹配）两层去重逻辑
+"""
+
+import csv
+import os
+import sys
+import tempfile
+import time
+from concurrent.futures import ThreadPoolExecutor, as_completed
+from pathlib import Path
+from typing import Any
+
+import pymysql
+import pytest
+from tqdm import tqdm
+
+# 确保项目根目录在 sys.path
+sys.path.insert(0, str(Path(__file__).resolve().parent))
+
+from import_hk_songs import (
+    AGGREGATE_SQL,
+    REPORT_DIR,
+    SOURCE_DB_CONFIG,
+    TARGET_DB_CONFIG,
+    TARGET_TABLE_NAME,
+    OSS_CONFIG,
+    _is_lrc_format,
+    _normalize_meta,
+    build_row_tuple,
+    classify_dedup_action,
+    check_l1,
+    check_l2,
+    download_file,
+    is_instrumental_lyrics,
+    process_lyrics,
+    DedupReport,
+    L2CandidateIndex,
+    load_approved_import_ids,
+)
+from lyric_dedup import DuplicateChecker, DuplicateDecision, LyricRecord
+from lyric_dedup.checker import CandidateMatch
+
+
+# ====================================================================
+# 聚合 SQL 导入规则
+# ====================================================================
+
+class TestAggregateSqlRules:
+    """测试源库聚合查询中的业务过滤规则"""
+
+    def test_excludes_liancheng_xiaorui_compositions_but_not_yinyan_self_made_compositions(self):
+        assert '1871475560046465025' in AGGREGATE_SQL
+        assert '连城小睿音乐工作室' in AGGREGATE_SQL
+
+        where_clause = AGGREGATE_SQL.split('WHERE', 1)[1]
+        assert 'sp.copyright_id' in where_clause
+        assert 'sp.copyright_name' in where_clause
+        assert '音眼自制' not in where_clause
+
+
+# ====================================================================
+# L1 元数据规范化 _normalize_meta
+# ====================================================================
+
+class TestNormalizeMeta:
+    """测试元数据规范化函数"""
+
+    def test_empty_none(self):
+        assert _normalize_meta(None) == ''
+        assert _normalize_meta('') == ''
+        assert _normalize_meta('   ') == ''
+
+    def test_basic_chinese(self):
+        assert _normalize_meta('遗失的心跳') == '遗失的心跳'
+
+    def test_lowercase(self):
+        assert _normalize_meta('Hello World') == 'helloworld'
+
+    def test_strip_spaces(self):
+        assert _normalize_meta('  周杰伦  ') == '周杰伦'
+
+    def test_remove_punctuation(self):
+        assert _normalize_meta('你，好吗？') == '你好吗'
+        assert _normalize_meta('test song') == 'testsong'
+
+    def test_traditional_to_simplified(self):
+        """繁体转简体"""
+        assert _normalize_meta('遺失的心跳') == '遗失的心跳'
+        assert _normalize_meta('愛') == '爱'
+        assert _normalize_meta('國語') == '国语'
+
+    def test_mixed_content(self):
+        """混合中英文+标点+空格"""
+        result = _normalize_meta('  Hello 世界 ，Test！ ')
+        assert result == 'hello世界test'
+
+    def test_punctuation_only(self):
+        assert _normalize_meta('，。！？') == ''
+
+    def test_fullwidth_digits(self):
+        """全角数字转半角（NFKC）"""
+        assert _normalize_meta('⑦裏香') == '7里香'  # ⑦→7，裏→里（繁转简）
+        assert _normalize_meta('第①章') == '第1章'
+        assert _normalize_meta('１２３') == '123'
+
+    def test_fullwidth_letters(self):
+        """全角字母转半角（NFKC）"""
+        assert _normalize_meta('ＡＢＣ') == 'abc'
+        assert _normalize_meta('Ｈｅｌｌｏ') == 'hello'
+
+    def test_fullwidth_punctuation(self):
+        """全角标点经 NFKC 后能被正常删除"""
+        # NFKC 将全角标点转为半角，然后被标点删除步骤清除
+        assert _normalize_meta('你好！？世界') == '你好世界'
+
+
+# ====================================================================
+# L1 元数据去重 check_l1
+# ====================================================================
+
+class TestCheckL1:
+    """测试 L1 元数据去重"""
+
+    def setup_method(self):
+        """构建测试索引"""
+        # 索引 key 必须用 _normalize_meta 规范化后的值
+        self.l1_index = {
+            (_normalize_meta('遗失的心跳'), _normalize_meta('萧亚轩'), _normalize_meta('Per Eklund')): 1001,
+            (_normalize_meta('妥协'), _normalize_meta('Wonderful'), _normalize_meta('阿沁')): 1002,
+            (_normalize_meta('晴天'), _normalize_meta('周杰伦'), _normalize_meta('周杰伦')): 1003,
+        }
+
+    def test_exact_match(self):
+        row = {'name': '遗失的心跳', 'lyricist': '萧亚轩', 'composer': 'Per Eklund'}
+        is_dup, matched_id = check_l1(row, self.l1_index)
+        assert is_dup is True
+        assert matched_id == 1001
+
+    def test_no_match(self):
+        row = {'name': '全新的歌', 'lyricist': '未知', 'composer': '未知'}
+        is_dup, matched_id = check_l1(row, self.l1_index)
+        assert is_dup is False
+        assert matched_id is None
+
+    def test_traditional_chinese_match(self):
+        """繁体名应匹配简体索引"""
+        row = {'name': '遺失的心跳', 'lyricist': '蕭亞軒', 'composer': 'Per Eklund'}
+        is_dup, matched_id = check_l1(row, self.l1_index)
+        assert is_dup is True
+        assert matched_id == 1001
+
+    def test_punctuation_ignored(self):
+        """标点差异应被忽略"""
+        row = {'name': '晴天', 'lyricist': '周杰伦', 'composer': '周杰伦！'}
+        is_dup, matched_id = check_l1(row, self.l1_index)
+        assert is_dup is True
+        assert matched_id == 1003
+
+    def test_empty_name_not_matched(self):
+        """空歌名不应匹配任何记录"""
+        row = {'name': '', 'lyricist': '萧亚轩', 'composer': 'Per Eklund'}
+        is_dup, matched_id = check_l1(row, self.l1_index)
+        assert is_dup is False
+
+    def test_none_fields(self):
+        """None 字段不应崩溃"""
+        row = {'name': '晴天', 'lyricist': None, 'composer': None}
+        is_dup, matched_id = check_l1(row, self.l1_index)
+        assert is_dup is False
+
+    def test_partial_match_not_duplicate(self):
+        """只有部分字段匹配不算重复"""
+        row = {'name': '晴天', 'lyricist': '方文山', 'composer': '周杰伦'}
+        is_dup, matched_id = check_l1(row, self.l1_index)
+        # lyricist 不同，不算重复
+        assert is_dup is False
+
+
+# ====================================================================
+# L2 歌词内容去重 check_l2
+# ====================================================================
+
+class TestCheckL2:
+    """测试 L2 歌词内容去重"""
+
+    def setup_method(self):
+        self.checker = DuplicateChecker()
+
+    def test_empty_lyrics(self):
+        """空歌词应返回 new"""
+        row = {'id': 1, 'lyrics_txt_content': '', 'name': 'test', 'singer': 'test'}
+        decision, confidence, matched_id, reason = check_l2(row, self.checker, [])
+        assert decision == 'new'
+        assert '无歌词内容' in reason
+
+    def test_none_lyrics(self):
+        row = {'id': 1, 'lyrics_txt_content': None, 'name': 'test', 'singer': 'test'}
+        decision, _, _, _ = check_l2(row, self.checker, [])
+        assert decision == 'new'
+
+    def test_no_candidates(self):
+        """无候选集应返回 new"""
+        row = {
+            'id': 1,
+            'lyrics_txt_content': '你是我的眼\n带我领略四季的变换\n你是我的眼\n带我穿越拥挤的人潮',
+            'name': '你是我的眼',
+            'singer': '萧煌奇',
+        }
+        decision, confidence, matched_id, reason = check_l2(row, self.checker, [])
+        assert decision == 'new'
+        assert '无候选集' in reason
+
+    def test_metadata_only_lyrics_are_not_duplicate(self):
+        """双方都只有元数据行时，L2 不应自动判重。"""
+        candidate = LyricRecord(
+            record_id='existing_metadata_only',
+            lyrics='作词：张三\n作曲：李四',
+            title='旧歌',
+        )
+        row = {
+            'id': 2,
+            'lyrics_txt_content': '作词：王五\n作曲：赵六',
+            'name': '新歌',
+            'singer': '测试歌手',
+        }
+
+        decision, confidence, matched_id, reason = check_l2(row, self.checker, [candidate])
+
+        assert decision == 'new'
+        assert matched_id is None
+        assert '无有效歌词' in reason
+
+    def test_instrumental_only_lyrics_are_not_duplicate(self):
+        """双方都是纯音乐提示时，L2 不应自动判重。"""
+        candidate = LyricRecord(
+            record_id='existing_instrumental',
+            lyrics='纯音乐，请欣赏',
+            title='纯音乐 A',
+        )
+        row = {
+            'id': 3,
+            'lyrics_txt_content': '纯音乐，请欣赏',
+            'name': '纯音乐 B',
+            'singer': '测试歌手',
+        }
+
+        decision, confidence, matched_id, reason = check_l2(row, self.checker, [candidate])
+
+        assert decision == 'new'
+        assert matched_id is None
+        assert '纯音乐' in reason
+
+    def test_lyrics_containing_instrumental_hint_skip_l2(self):
+        """歌词内容包含纯音乐提示时，L2 应直接跳过。"""
+        candidate = LyricRecord(
+            record_id='existing_lyrics',
+            lyrics='纯音乐\n这是一段伴奏提示',
+            title='旧纯音乐',
+        )
+        row = {
+            'id': 4,
+            'lyrics_txt_content': '本曲为纯音乐，请欣赏',
+            'name': '新纯音乐',
+            'singer': '测试歌手',
+        }
+
+        decision, confidence, matched_id, reason = check_l2(row, self.checker, [candidate])
+
+        assert decision == 'new'
+        assert matched_id is None
+        assert '纯音乐' in reason
+
+    def test_exact_duplicate(self):
+        """完全相同的歌词应判为 duplicate"""
+        lyrics = '\n'.join([
+            '你是我的眼 带我领略四季的变换',
+            '你是我的眼 带我穿越拥挤的人潮',
+            '你是我的眼 带我阅读浩瀚的书海',
+            '因为你是我的眼 让我看见这世界',
+            '就在我眼前 一切都没改变',
+            '眼前的世界 如此的清晰',
+            '我知道 你就是我的眼',
+            '让我看见 这美丽的世界',
+        ])
+        candidate = LyricRecord(record_id='existing_1', lyrics=lyrics, title='你是我的眼', artist='萧煌奇')
+        row = {
+            'id': 2,
+            'lyrics_txt_content': lyrics,
+            'name': '你是我的眼',
+            'singer': '萧煌奇',
+        }
+        decision, confidence, matched_id, reason = check_l2(
+            row, self.checker, [candidate]
+        )
+        assert decision == 'duplicate'
+        assert confidence >= 0.9
+        assert matched_id == 'existing_1'
+
+    def test_similar_lyrics_high_overlap(self):
+        """高度相似的歌词应判为 duplicate 或 review"""
+        base_lyrics = '\n'.join([
+            '如果有一天 我变得更复杂',
+            '是否还记得 当初的模样',
+            '那些年少轻狂的日子',
+            '如今已变成了回忆',
+            '我站在街头 看着人来人往',
+            '寻找曾经熟悉的面孔',
+            '时间带走了许多',
+            '却带不走我对你的思念',
+            '如果有一天 我们再相遇',
+            '请告诉我 你还记得我吗',
+        ])
+
+        # 修改几行制造高相似度变体
+        similar_lyrics = '\n'.join([
+            '如果有一天 我变得更复杂',
+            '是否还记得 当初的样子',       # 微小改动
+            '那些年少轻狂的日子',
+            '如今已变成了美好回忆',       # 加了"美好"
+            '我站在街头 看着人来人往',
+            '寻找曾经熟悉的那些面孔',     # 加了"那些"
+            '时间带走了许多东西',           # 加了"东西"
+            '却带不走我对你的思念',
+            '如果有一天 我们再相遇',
+            '请告诉我 你还记得我吗',
+        ])
+
+        candidate = LyricRecord(record_id='c1', lyrics=base_lyrics, title='如果有一天')
+        row = {
+            'id': 2,
+            'lyrics_txt_content': similar_lyrics,
+            'name': '如果有一天(另一版)',
+            'singer': '未知',
+        }
+        decision, confidence, matched_id, reason = check_l2(
+            row, self.checker, [candidate]
+        )
+        # 高相似度 -> 应该不是 new（duplicate 或 review）
+        assert decision in ('duplicate', 'review'), f"Expected duplicate/review but got {decision}"
+
+    def test_completely_different_lyrics(self):
+        """完全不同的歌词应判为 new"""
+        lyrics_a = '\n'.join([
+            '清晨的阳光洒在窗台',
+            '我端起咖啡望向窗外',
+            '新的一天又将开始',
+            '把昨天的烦恼留在梦里',
+        ])
+        lyrics_b = '\n'.join([
+            '夜晚的星空如此美丽',
+            '我独自走在寂静的路上',
+            '风吹过脸颊带着凉意',
+            '远方的灯火闪烁不停',
+        ])
+        candidate = LyricRecord(record_id='c1', lyrics=lyrics_a, title='清晨')
+        row = {
+            'id': 2,
+            'lyrics_txt_content': lyrics_b,
+            'name': '夜晚',
+            'singer': '未知',
+        }
+        decision, confidence, matched_id, reason = check_l2(
+            row, self.checker, [candidate]
+        )
+        assert decision == 'new'
+
+    def test_fragment_goes_to_review(self):
+        """歌词片段不应自动判重，也不应自动放行，应留给人工审核。"""
+        full_lyrics = '\n'.join([
+            '第一行歌词内容在这里',
+            '第二行歌词继续写下去',
+            '第三行歌词还是不同',
+            '第四行歌词依然在变化',
+            '第五行歌词新的表达',
+            '第六行歌词更加精彩',
+            '第七行歌词快要结束',
+            '第八行歌词最后收尾',
+        ])
+        # 取其中 3 行作为可识别的片段
+        fragment = '\n'.join([
+            '第三行歌词还是不同',
+            '第四行歌词依然在变化',
+            '第五行歌词新的表达',
+        ])
+        candidate = LyricRecord(record_id='c1', lyrics=full_lyrics, title='完整歌曲')
+        row = {
+            'id': 2,
+            'lyrics_txt_content': fragment,
+            'name': '片段歌曲',
+            'singer': '未知',
+        }
+        decision, _, _, _ = check_l2(row, self.checker, [candidate])
+        assert decision == 'review'
+
+    def test_lrc_with_timestamps_still_detects(self):
+        """带 LRC 时间戳的歌词仍能正确去重"""
+        plain_lyrics = '\n'.join([
+            '你是我的小呀小苹果',
+            '怎么爱你都不嫌多',
+            '红红的小脸温暖我的心窝',
+            '点亮我生命的火火火火火',
+            '你是我的小呀小苹果',
+            '就像天边最美的云朵',
+            '春天又来到了花开满山坡',
+            '种下希望就会收获',
+        ])
+        lrc_lyrics = '\n'.join([
+            '[00:12.00]你是我的小呀小苹果',
+            '[00:16.00]怎么爱你都不嫌多',
+            '[00:20.00]红红的小脸温暖我的心窝',
+            '[00:24.00]点亮我生命的火火火火火',
+            '[00:28.00]你是我的小呀小苹果',
+            '[00:32.00]就像天边最美的云朵',
+            '[00:36.00]春天又来到了花开满山坡',
+            '[00:40.00]种下希望就会收获',
+        ])
+        candidate = LyricRecord(record_id='c1', lyrics=plain_lyrics, title='小苹果')
+        row = {
+            'id': 2,
+            'lyrics_txt_content': lrc_lyrics,
+            'name': '小苹果(LRC版)',
+            'singer': '筷子兄弟',
+        }
+        decision, confidence, matched_id, reason = check_l2(
+            row, self.checker, [candidate]
+        )
+        assert decision == 'duplicate', f"Expected duplicate, got {decision}: {reason}"
+
+    def test_multiple_candidates_best_match(self):
+        """多个候选时选最佳匹配"""
+        lyrics_query = '\n'.join([
+            '我是一只小小鸟',
+            '想要飞却怎么也飞不高',
+            '我寻寻觅觅寻寻觅觅',
+            '一个温暖的怀抱',
+            '我是一只小小鸟',
+            '想要飞却怎么也飞不高',
+            '这样的要求算不算太高',
+            '我是一只小小鸟',
+        ])
+        candidate_exact = LyricRecord(record_id='exact', lyrics=lyrics_query, title='小小鸟')
+        candidate_different = LyricRecord(
+            record_id='diff',
+            lyrics='完全不同的歌词内容\n这里是第二行\n第三行也不一样\n第四行更是如此',
+            title='不同的歌',
+        )
+        row = {
+            'id': 3,
+            'lyrics_txt_content': lyrics_query,
+            'name': '小小鸟',
+            'singer': '赵传',
+        }
+        decision, confidence, matched_id, _ = check_l2(
+            row, self.checker, [candidate_different, candidate_exact]
+        )
+        assert decision == 'duplicate'
+        assert matched_id == 'exact'
+
+    def test_threshold_override(self):
+        """自定义阈值影响判定"""
+        lyrics = '\n'.join([
+            '风吹过脸庞带来你的消息',
+            '雨落在窗前想起那个夏季',
+            '我站在路口等一个奇迹',
+            '你是否还记得我们的约定',
+        ])
+        candidate = LyricRecord(record_id='c1', lyrics=lyrics, title='约定')
+        row = {
+            'id': 2,
+            'lyrics_txt_content': lyrics,
+            'name': '约定',
+            'singer': '光良',
+        }
+
+        # 极低阈值 -> 更容易判为 duplicate
+        strict_checker = DuplicateChecker(duplicate_jaccard_threshold=0.3)
+        decision_strict, _, _, _ = check_l2(row, strict_checker, [candidate])
+        assert decision_strict == 'duplicate'
+
+    def test_strong_review_same_writers_promotes_to_duplicate(self):
+        """同词曲且主体歌词强重合的 review 边界样本应升级为 duplicate。"""
+        candidate_lyrics = '\n'.join([
+            '词：王琪',
+            '曲：王琪',
+            '那夜的雨也没能留住你',
+            '山谷的风它陪着我哭泣',
+            '你的驼铃声仿佛还在我耳边响起',
+            '告诉我你曾来过这里',
+            '我酿的酒喝不醉我自己',
+            '你唱的歌却让我一醉不起',
+            '我愿意陪你翻过雪山穿越戈壁',
+            '可你不辞而别还断绝了所有的消息',
+            '心上人我在可可托海等你',
+            '他们说你嫁到了伊犁',
+        ])
+        query_lyrics = '\n'.join([
+            '歌曲：可可托海的牧羊人',
+            '词：王琪',
+            '曲：王琪',
+            '那夜的雨也没能留住你',
+            '山谷的风它陪着我哭泣',
+            '你的驼铃声仿佛还在我耳边响起',
+            '告诉我你曾来过这里',
+            '我酿的酒喝不醉我自己',
+            '你唱的歌却让我一醉不起',
+            '我愿意陪你翻过雪山穿越戈壁',
+            '可你不辞而别还断绝了',
+            '所有的消息',
+            '心上人我在可可托海等你',
+        ])
+        candidate = LyricRecord(
+            record_id='c1',
+            lyrics=candidate_lyrics,
+            title='可可托海的牧羊人',
+            lyricist='王琪',
+            composer='王琪',
+        )
+        row = {
+            'id': 2,
+            'lyrics_txt_content': query_lyrics,
+            'name': '歌曲：可可托海的牧羊人',
+            'singer': '测试歌手',
+            'lyricist': '王琪',
+            'composer': '王琪',
+        }
+
+        decision, _, matched_id, reason = check_l2(row, self.checker, [candidate])
+
+        assert decision == 'duplicate', reason
+        assert matched_id == 'c1'
+
+    def test_strong_review_cover_version_stays_review(self):
+        """翻唱/cover 版本即使歌词强重合，也应留给人工审核。"""
+        candidate_lyrics = '\n'.join([
+            '词：陈粒',
+            '曲：陈粒',
+            '窗外雨都停了',
+            '屋里灯还黑着',
+            '数着你的冷漠',
+            '把玩着寂寞',
+            '电话还没拨已经口渴',
+            '为你熬的夜都冷了',
+            '数的羊都跑了',
+            '一个两个',
+            '嘲笑我笑我耳朵失灵的',
+            '笑我放你走了走了走了',
+        ])
+        cover_lyrics = '\n'.join([
+            '走马 - 摩登兄弟翻唱版',
+            '原唱：陈粒',
+            '词：陈粒',
+            '曲：陈粒',
+            '窗外雨都停了',
+            '屋里灯还黑着',
+            '数着你的冷漠',
+            '把玩着寂寞',
+            '电话还没拨已经口渴',
+            '为你熬的夜都冷了',
+            '数的羊都跑了',
+            '一个两个',
+            '嘲笑我笑我耳朵失灵的',
+        ])
+        candidate = LyricRecord(
+            record_id='c1',
+            lyrics=candidate_lyrics,
+            title='走马',
+            artist='陈粒',
+            lyricist='陈粒',
+            composer='陈粒',
+        )
+        row = {
+            'id': 2,
+            'lyrics_txt_content': cover_lyrics,
+            'name': '走马 翻唱版',
+            'singer': '摩登兄弟',
+            'lyricist': '陈粒',
+            'composer': '陈粒',
+        }
+
+        decision, _, matched_id, reason = check_l2(row, self.checker, [candidate])
+
+        assert decision == 'review', reason
+        assert matched_id == 'c1'
+
+    def test_credit_only_high_review_stays_review(self):
+        """高分只来自词曲/制作信息时，不能升级为 duplicate。"""
+        candidate = LyricRecord(
+            record_id='c1',
+            lyrics='词：安智英\n曲：安智英/Vanilla Man\n사랑할 수밖에 - 脸红的思春期',
+            title='사랑할 수밖에',
+            lyricist='安智英',
+            composer='安智英/Vanilla Man',
+        )
+        row = {
+            'id': 2,
+            'lyrics_txt_content': '词：安智英\n曲：安智英/Vanilla Man\n스노우볼 - 脸红的思春期',
+            'name': '스노우볼',
+            'singer': '脸红的思春期',
+            'lyricist': '安智英',
+            'composer': '安智英/Vanilla Man',
+        }
+
+        decision, _, _, reason = check_l2(row, self.checker, [candidate])
+
+        assert decision != 'duplicate', reason
+
+    def test_same_writers_different_lyrics_never_duplicates(self):
+        """词曲作者相同但歌词内容不同，不应因元数据相同被合并。"""
+        candidate = LyricRecord(
+            record_id='c1',
+            lyrics='\n'.join([
+                '第一首歌的第一句',
+                '第一首歌的第二句',
+                '第一首歌的第三句',
+                '第一首歌的第四句',
+                '第一首歌的第五句',
+                '第一首歌的第六句',
+                '第一首歌的第七句',
+                '第一首歌的第八句',
+            ]),
+            title='同词曲旧歌',
+            lyricist='共同作者',
+            composer='共同作者',
+        )
+        row = {
+            'id': 2,
+            'lyrics_txt_content': '\n'.join([
+                '另一首歌的第一句',
+                '另一首歌的第二句',
+                '另一首歌的第三句',
+                '另一首歌的第四句',
+                '另一首歌的第五句',
+                '另一首歌的第六句',
+                '另一首歌的第七句',
+                '另一首歌的第八句',
+            ]),
+            'name': '同词曲新歌',
+            'singer': '测试歌手',
+            'lyricist': '共同作者',
+            'composer': '共同作者',
+        }
+
+        decision, _, matched_id, reason = check_l2(row, self.checker, [candidate])
+
+        assert decision != 'duplicate', reason
+
+    def test_no_effective_lyrics_fallback_never_duplicates(self):
+        """无有效歌词兜底特征相似也只能复核，不能自动合并。"""
+        candidate = LyricRecord(
+            record_id='c1',
+            lyrics='作词：张三\n作曲：李四\n编曲：王五',
+            title='旧元数据歌',
+        )
+        row = {
+            'id': 2,
+            'lyrics_txt_content': '作词：张三\n作曲：李四\n编曲：王五',
+            'name': '新元数据歌',
+            'singer': '测试歌手',
+        }
+
+        decision, _, matched_id, reason = check_l2(row, self.checker, [candidate])
+
+        assert decision != 'duplicate', reason
+
+    def test_same_lyrics_different_writers_duplicates_for_author_merge(self):
+        """歌词内容相同，即使词曲作者不同，也应命中合并，作者差异交给导入层增量处理。"""
+        lyrics = '\n'.join([
+            '同一份歌词第一行',
+            '同一份歌词第二行',
+            '同一份歌词第三行',
+            '同一份歌词第四行',
+            '同一份歌词第五行',
+            '同一份歌词第六行',
+            '同一份歌词第七行',
+            '同一份歌词第八行',
+        ])
+        candidate = LyricRecord(
+            record_id='c1',
+            lyrics=lyrics,
+            title='旧歌名',
+            lyricist='旧词作者',
+            composer='旧曲作者',
+        )
+        row = {
+            'id': 2,
+            'lyrics_txt_content': lyrics,
+            'name': '新歌名',
+            'singer': '测试歌手',
+            'lyricist': '新词作者',
+            'composer': '新曲作者',
+        }
+
+        decision, _, matched_id, reason = check_l2(row, self.checker, [candidate])
+
+        assert decision == 'duplicate', reason
+        assert matched_id == 'c1'
+
+
+# ====================================================================
+# LRC 格式检测 _is_lrc_format
+# ====================================================================
+
+class TestIsLrcFormat:
+    """测试 LRC 格式检测"""
+
+    def test_none_empty(self):
+        assert _is_lrc_format(None) is False
+        assert _is_lrc_format('') is False
+
+    def test_plain_text(self):
+        assert _is_lrc_format('你是我的小苹果\n怎么爱你都不嫌多') is False
+
+    def test_lrc_with_timestamps(self):
+        assert _is_lrc_format('[00:12.00]你是我的小苹果\n[00:16.00]怎么爱你都不嫌多') is True
+
+    def test_lrc_with_mm_ss(self):
+        assert _is_lrc_format('[01:30]歌词内容在这里') is True
+
+    def test_lrc_with_hours(self):
+        assert _is_lrc_format('[1:30:00]歌词内容') is True
+
+    def test_bracket_without_time(self):
+        """非时间标签的方括号不算 LRC"""
+        assert _is_lrc_format('[前奏]歌曲开始\n[副歌]高潮部分') is False
+
+
+# ====================================================================
+# process_lyrics 歌词处理
+# ====================================================================
+
+class TestProcessLyrics:
+    """测试歌词处理函数（skip-oss 模式）"""
+
+    def test_empty_lyrics(self):
+        lyrics_url, lrc_url = process_lyrics(None, '', 1, skip_oss=True)
+        assert lyrics_url is None
+        assert lrc_url is None
+
+    def test_none_lyrics(self):
+        lyrics_url, lrc_url = process_lyrics(None, None, 1, skip_oss=True)
+        assert lyrics_url is None
+        assert lrc_url is None
+
+    def test_plain_text_skip_oss(self):
+        """纯文本 + skip_oss -> lyrics_url 有值，lrc_url 为空"""
+        text = '你是我的小苹果\n怎么爱你都不嫌多'
+        lyrics_url, lrc_url = process_lyrics(None, text, 1, skip_oss=True)
+        assert lyrics_url == text
+        assert lrc_url is None
+
+    def test_lrc_format_skip_oss(self):
+        """LRC 格式 + skip_oss -> 两个字段都有值"""
+        text = '[00:12.00]你是我的小苹果\n[00:16.00]怎么爱你都不嫌多'
+        lyrics_url, lrc_url = process_lyrics(None, text, 1, skip_oss=True)
+        assert lyrics_url == text
+        assert lrc_url == text
+
+
+# ====================================================================
+# 入库行构建 build_row_tuple
+# ====================================================================
+
+class TestBuildRowTuple:
+    """测试目标表写入 tuple 构建"""
+
+    def test_build_row_tuple_includes_generated_primary_key(self):
+        row = {
+            'source_id': 123,
+            'name': '测试歌曲',
+            'lyricist': '词作者',
+            'composer': '曲作者',
+            'issue_status': 1,
+            'source_table_name': 'hk_song_platform',
+            'source_song_id': '123',
+        }
+
+        tuple_row = build_row_tuple(row, None, skip_oss=True)
+
+        assert len(tuple_row) == 45
+        assert isinstance(tuple_row[0], int)
+        assert tuple_row[0] > 0
+        assert tuple_row[1] == '测试歌曲'
+        assert tuple_row[40] == 'hk_song_platform'
+        assert tuple_row[41] == '123'
+
+
+# ====================================================================
+# L2 候选召回索引 L2CandidateIndex
+# ====================================================================
+
+class TestL2CandidateIndex:
+    """测试 L2 topK 召回索引"""
+
+    def test_topk_recall_keeps_exact_hash_match(self):
+        checker = DuplicateChecker()
+        index = L2CandidateIndex(checker, mode='topk', top_k=1)
+        exact = LyricRecord(
+            record_id='exact',
+            lyrics='你是我的眼\n带我领略四季的变换\n你是我的眼\n带我穿越拥挤的人潮',
+            title='你是我的眼',
+        )
+        unrelated = LyricRecord(record_id='unrelated', lyrics='完全不同的歌词\n没有任何关系', title='不同')
+        index.add(unrelated)
+        index.add(exact)
+
+        recalled = index.recall(exact)
+
+        assert [record.record_id for record in recalled] == ['exact']
+
+    def test_topk_recall_limits_candidates_before_ranking(self):
+        checker = DuplicateChecker()
+        index = L2CandidateIndex(checker, mode='topk', top_k=2)
+        query = LyricRecord(
+            record_id='query',
+            lyrics='春天的风吹过山岗\n花开的声音落在心上\n我沿着河流寻找月光',
+            title='春风',
+        )
+        related_a = LyricRecord(record_id='a', lyrics='春天的风吹过山岗\n花开的声音留在心上', title='a')
+        related_b = LyricRecord(record_id='b', lyrics='我沿着河流寻找月光\n春天的风吹过远方', title='b')
+        unrelated = LyricRecord(record_id='c', lyrics='城市霓虹闪烁不停\n陌生人穿过雨夜', title='c')
+        for record in (unrelated, related_a, related_b):
+            index.add(record)
+
+        recalled = index.recall(query)
+
+        recalled_ids = {record.record_id for record in recalled}
+        assert len(recalled) == 2
+        assert recalled_ids == {'a', 'b'}
+
+    def test_check_l2_uses_index_recall_then_existing_ranker(self):
+        checker = DuplicateChecker()
+        index = L2CandidateIndex(checker, mode='topk', top_k=10)
+        lyrics = '\n'.join([
+            '你是我的眼 带我领略四季的变换',
+            '你是我的眼 带我穿越拥挤的人潮',
+            '你是我的眼 带我阅读浩瀚的书海',
+            '因为你是我的眼 让我看见这世界',
+        ])
+        index.add(LyricRecord(record_id='existing', lyrics=lyrics, title='你是我的眼'))
+
+        decision, confidence, matched_id, reason = check_l2(
+            {
+                'source_id': 999,
+                'lyrics_txt_content': lyrics,
+                'name': '你是我的眼',
+                'singer': '萧煌奇',
+            },
+            checker,
+            index,
+        )
+
+        assert decision == 'duplicate'
+        assert confidence == 1.0
+        assert matched_id == 'existing'
+        assert '哈希完全一致' in reason
+
+
+# ====================================================================
+# DedupReport 去重报告
+# ====================================================================
+
+class TestDedupReport:
+    """测试去重报告 CSV 输出"""
+
+    def test_write_and_read(self):
+        with tempfile.NamedTemporaryFile(mode='w', suffix='.csv', delete=False) as f:
+            filepath = f.name
+
+        try:
+            report = DedupReport(filepath)
+            report.write(1001, '遗失的心跳', 'L1', 'duplicate', matched_id=500, reason='元数据完全匹配')
+            report.write(1002, '妥协', 'L2', 'new', confidence='0.9500', reason='无相似候选')
+            report.write(1003, '晴天', 'L2', 'review', confidence='0.6200', matched_id=800,
+                         reason='候选相似度达到复核阈值')
+            report.close()
+
+            with open(filepath, 'r', encoding='utf-8') as f:
+                reader = csv.reader(f)
+                rows = list(reader)
+
+            assert len(rows) == 4  # header + 3 data rows
+            assert rows[0] == ['id', 'name', 'stage', 'decision', 'confidence', 'matched_id', 'reason']
+            assert rows[1][0] == '1001'
+            assert rows[1][2] == 'L1'
+            assert rows[1][3] == 'duplicate'
+            assert rows[2][3] == 'new'
+            assert rows[3][3] == 'review'
+            assert rows[3][5] == '800'
+        finally:
+            os.unlink(filepath)
+
+
+class TestReviewResultCsv:
+    """测试人工审核结果过滤。"""
+
+    def test_load_approved_import_ids_only_returns_explicit_imports(self, tmp_path):
+        review_csv = tmp_path / 'review.csv'
+        review_csv.write_text(
+            '\n'.join([
+                'source_id,review_decision,review_note',
+                '100,import,确认导入',
+                '101,review,继续复核',
+                '102,skip,合并',
+                '103,导入,中文确认',
+                '104,,空白不导入',
+            ]),
+            encoding='utf-8',
+        )
+
+        assert load_approved_import_ids(str(review_csv)) == {'100', '103'}
+
+
+# ====================================================================
+# 集成测试：L1 + L2 联合去重流程
+# ====================================================================
+
+class TestDedupIntegration:
+    """测试 L1+L2 联合去重的完整流程"""
+
+    def setup_method(self):
+        self.checker = DuplicateChecker()
+        # 模拟已有的 L1 索引
+        self.l1_index = {
+            ('晴天', '周杰伦', '周杰伦'): 100,
+            ('七里香', '方文山', '周杰伦'): 101,
+        }
+        # 模拟已有的 L2 候选集
+        self.l2_candidates = [
+            LyricRecord(
+                record_id='100',
+                lyrics='\n'.join([
+                    '故事的小黄花 从出生那年就飘着',
+                    '童年的荡秋千 随记忆一直晃到现在',
+                    'Re So So Si Do Si La',
+                    'So La Si Si Si Si La Si La So',
+                    '吹着前奏望着天空 我想起花瓣试着掉落',
+                    '为你翘课的那一天 花落的那一天',
+                    '教室的那一间 我怎么看不见',
+                    '消失的下雨天 我好想再淋一遍',
+                ]),
+                title='晴天',
+                artist='周杰伦',
+            ),
+        ]
+
+    def test_l1_catches_metadata_duplicate(self):
+        """L1 应拦住元数据重复的记录"""
+        row = {'name': '晴天', 'lyricist': '周杰伦', 'composer': '周杰伦',
+               'id': 200, 'lyrics_txt_content': '不同的歌词内容', 'singer': '周杰伦'}
+        is_dup, matched_id = check_l1(row, self.l1_index)
+        assert is_dup is True
+        assert matched_id == 100
+
+    def test_l2_catches_lyric_duplicate(self):
+        """L2 应拦住歌词重复但元数据不同的记录"""
+        # 歌名不同但歌词一样
+        row = {
+            'id': 201,
+            'name': '晴天(翻唱版)',
+            'lyricist': '翻唱歌手',
+            'composer': '翻唱歌手',
+            'lyrics_txt_content': '\n'.join([
+                '故事的小黄花 从出生那年就飘着',
+                '童年的荡秋千 随记忆一直晃到现在',
+                'Re So So Si Do Si La',
+                'So La Si Si Si Si La Si La So',
+                '吹着前奏望着天空 我想起花瓣试着掉落',
+                '为你翘课的那一天 花落的那一天',
+                '教室的那一间 我怎么看不见',
+                '消失的下雨天 我好想再淋一遍',
+            ]),
+            'singer': '翻唱歌手',
+        }
+        # L1 不命中
+        is_dup, _ = check_l1(row, self.l1_index)
+        assert is_dup is False
+
+        # L2 应命中
+        decision, confidence, matched_id, reason = check_l2(
+            row, self.checker, self.l2_candidates
+        )
+        assert decision == 'duplicate'
+        assert matched_id == '100'
+
+    def test_l2_passes_different_song(self):
+        """L2 应放过真正不同的歌"""
+        row = {
+            'id': 202,
+            'name': '稻香',
+            'lyricist': '周杰伦',
+            'composer': '周杰伦',
+            'lyrics_txt_content': '\n'.join([
+                '对这个世界如果你有太多的抱怨',
+                '跌倒了就不敢继续往前走',
+                '为什么人要这么的脆弱 堕落',
+                '请你打开电视看看',
+                '多少人为生命在努力勇敢的走下去',
+                '我们是不是该知足',
+                '珍惜一切 就算没有拥有',
+                '还记得你说家是唯一的城堡',
+            ]),
+            'singer': '周杰伦',
+        }
+        # L1 不命中（歌名不同）
+        is_dup, _ = check_l1(row, self.l1_index)
+        assert is_dup is False
+
+        # L2 也不命中
+        decision, _, _, _ = check_l2(row, self.checker, self.l2_candidates)
+        assert decision == 'new'
+
+    def test_l1_match_with_different_lyrics_is_not_skipped(self):
+        """L1 命中只作为召回信号；歌词不同不能跳过或合并。"""
+        row = {
+            'id': 203,
+            'name': '晴天',
+            'lyricist': '周杰伦',
+            'composer': '周杰伦',
+            'lyrics_txt_content': '\n'.join([
+                '这是一首完全不同的歌第一句',
+                '这是一首完全不同的歌第二句',
+                '这是一首完全不同的歌第三句',
+                '这是一首完全不同的歌第四句',
+                '这是一首完全不同的歌第五句',
+                '这是一首完全不同的歌第六句',
+                '这是一首完全不同的歌第七句',
+                '这是一首完全不同的歌第八句',
+            ]),
+            'singer': '周杰伦',
+        }
+
+        action = classify_dedup_action(row, self.l1_index, self.checker, self.l2_candidates)
+
+        assert action['action'] == 'new'
+        assert action['l1_matched_id'] == 100
+        assert action['matched_id'] is None
+
+    def test_same_lyrics_different_writers_returns_merge_author_action(self):
+        """歌词相同但作者不同，导入决策应合并并标记作者增量。"""
+        lyrics = self.l2_candidates[0].lyrics
+        row = {
+            'id': 204,
+            'name': '另一个歌名',
+            'lyricist': '新词作者',
+            'composer': '新曲作者',
+            'lyrics_txt_content': lyrics,
+            'singer': '新歌手',
+        }
+
+        action = classify_dedup_action(row, self.l1_index, self.checker, self.l2_candidates)
+
+        assert action['action'] == 'merge'
+        assert action['merge_authors'] is True
+        assert action['matched_id'] == '100'
+
+    def test_full_pipeline_simulation(self):
+        """模拟完整 L1 -> L2 -> 写入 的管线流程"""
+        batch = [
+            # 1. L1 重复（元数据匹配）
+            {'id': 300, 'name': '晴天', 'lyricist': '周杰伦', 'composer': '周杰伦',
+             'lyrics_txt_content': '随便什么歌词', 'singer': '周杰伦'},
+            # 2. L2 重复（歌词相同但元数据不同）
+            {'id': 301, 'name': '晴天 Remix', 'lyricist': 'DJ', 'composer': 'DJ',
+             'lyrics_txt_content': '\n'.join([
+                 '故事的小黄花 从出生那年就飘着',
+                 '童年的荡秋千 随记忆一直晃到现在',
+                 'Re So So Si Do Si La',
+                 'So La Si Si Si Si La Si La So',
+                 '吹着前奏望着天空 我想起花瓣试着掉落',
+                 '为你翘课的那一天 花落的那一天',
+                 '教室的那一间 我怎么看不见',
+                 '消失的下雨天 我好想再淋一遍',
+             ]), 'singer': 'DJ'},
+            # 3. 新歌（L1+L2 都不命中）
+            {'id': 302, 'name': '夜曲', 'lyricist': '方文山', 'composer': '周杰伦',
+             'lyrics_txt_content': '\n'.join([
+                 '一群嗜血的蚂蚁 被腐肉所吸引',
+                 '我面无表情 看孤独的风景',
+                 '失去你 爱恨开始分明',
+                 '失去你 还有什么事好关心',
+                 '当太阳升起的时候',
+                 '我开始学会怎么去忘记',
+                 '你的笑容勉强不来',
+                 '爱深埋在尘埃 没有人能明白',
+             ]), 'singer': '周杰伦'},
+        ]
+
+        accepted = []
+        skipped_l1 = []
+        skipped_l2 = []
+
+        for row in batch:
+            # L1
+            is_dup, matched_id = check_l1(row, self.l1_index)
+            if is_dup:
+                skipped_l1.append((row['id'], matched_id))
+                continue
+
+            # L2
+            decision, confidence, l2_matched_id, reason = check_l2(
+                row, self.checker, self.l2_candidates
+            )
+            if decision == 'duplicate':
+                skipped_l2.append((row['id'], l2_matched_id))
+                continue
+
+            accepted.append(row['id'])
+            # 加入候选集
+            if row.get('lyrics_txt_content'):
+                self.l2_candidates.append(LyricRecord(
+                    record_id=str(row['id']),
+                    lyrics=row['lyrics_txt_content'],
+                    title=row.get('name'),
+                    artist=row.get('singer'),
+                ))
+
+        # 验证结果
+        assert len(skipped_l1) == 1  # id=300 被 L1 拦住
+        assert skipped_l1[0] == (300, 100)
+        assert len(skipped_l2) == 1  # id=301 被 L2 拦住
+        assert skipped_l2[0] == (301, '100')
+        assert accepted == [302]      # id=302 通过两层去重
+        assert len(self.l2_candidates) == 2  # 原1条 + 新增1条
+
+
+# ====================================================================
+# 边界情况
+# ====================================================================
+
+class TestEdgeCases:
+    """边界情况和异常场景"""
+
+    def test_very_short_lyrics(self):
+        """极短歌词不应崩溃"""
+        checker = DuplicateChecker()
+        candidate = LyricRecord(record_id='c1', lyrics='爱', title='短歌')
+        row = {'id': 1, 'lyrics_txt_content': '爱', 'name': '短歌', 'singer': 'test'}
+        decision, _, _, _ = check_l2(row, checker, [candidate])
+        # 极短歌词可能是 duplicate（哈希匹配）或 new，但不应崩溃
+        assert decision in ('duplicate', 'review', 'new')
+
+    def test_whitespace_only_lyrics(self):
+        """纯空白歌词"""
+        checker = DuplicateChecker()
+        row = {'id': 1, 'lyrics_txt_content': '   \n\n  ', 'name': 'test', 'singer': 'test'}
+        decision, _, _, reason = check_l2(row, checker, [])
+        assert decision == 'new'
+
+    def test_l1_index_with_duplicate_names(self):
+        """L1 索引中同名歌曲但不同作者"""
+        l1_index = {
+            ('晴天', '周杰伦', '周杰伦'): 100,
+            ('晴天', '张三', '李四'): 101,
+        }
+        row1 = {'name': '晴天', 'lyricist': '周杰伦', 'composer': '周杰伦'}
+        row2 = {'name': '晴天', 'lyricist': '张三', 'composer': '李四'}
+        row3 = {'name': '晴天', 'lyricist': '王五', 'composer': '赵六'}
+
+        assert check_l1(row1, l1_index) == (True, 100)
+        assert check_l1(row2, l1_index) == (True, 101)
+        assert check_l1(row3, l1_index) == (False, None)
+
+    def test_l2_with_lrc_timestamps_normalization(self):
+        """L2 应正确处理带时间戳的歌词"""
+        checker = DuplicateChecker()
+        plain = '我爱你中国\n心爱的母亲\n我为你流泪\n也为你自豪'
+        with_ts = '[00:01.00]我爱你中国\n[00:05.00]心爱的母亲\n[00:09.00]我为你流泪\n[00:13.00]也为你自豪'
+
+        candidate = LyricRecord(record_id='c1', lyrics=plain, title='我爱你中国')
+        row = {'id': 2, 'lyrics_txt_content': with_ts, 'name': '我爱你中国', 'singer': '汪峰'}
+        decision, _, _, _ = check_l2(row, checker, [candidate])
+        # normalization 应该去掉时间戳后识别为相同歌词
+        assert decision == 'duplicate'
+
+
+class TestTargetTableBaseline:
+    """测试把目标表已有数据下载到本地后再作为去重测试基线。"""
+
+    def test_download_target_table_baseline_writes_manifest_and_local_lyrics(self, monkeypatch, tmp_path):
+        rows = [
+            {
+                'id': 100,
+                'name': '晴天',
+                'singer': '周杰伦',
+                'lyricist': '周杰伦',
+                'composer': '周杰伦',
+                'lyrics_url': 'https://example.test/qingtian.txt',
+            },
+            {
+                'id': 101,
+                'name': '本地歌词',
+                'singer': '测试歌手',
+                'lyricist': '测试词',
+                'composer': '测试曲',
+                'lyrics_url': '已经存在的歌词文本',
+            },
+        ]
+
+        class FakeCursor:
+            def __enter__(self):
+                return self
+
+            def __exit__(self, exc_type, exc, tb):
+                return False
+
+            def execute(self, sql):
+                self.sql = sql
+
+            def fetchall(self):
+                return rows
+
+        class FakeConnection:
+            def cursor(self):
+                return FakeCursor()
+
+            def close(self):
+                pass
+
+        monkeypatch.setattr(pymysql, 'connect', lambda **kwargs: FakeConnection())
+        monkeypatch.setattr(
+            sys.modules[__name__],
+            'download_file',
+            lambda url, timeout=30: ('故事的小黄花\n童年的荡秋千'.encode('utf-8'), 'text/plain'),
+        )
+
+        items, manifest_path = _download_target_table_baseline(
+            tmp_path,
+            limit=0,
+            download_timeout=3,
+            show_progress=False,
+        )
+
+        assert manifest_path.exists()
+        assert len(items) == 2
+        assert [item['record'].record_id for item in items] == ['100', '101']
+        assert items[0]['record'].lyrics == '故事的小黄花\n童年的荡秋千'
+        assert items[1]['record'].lyrics == '已经存在的歌词文本'
+        assert Path(items[0]['local_lyrics_path']).exists()
+        assert Path(items[1]['local_lyrics_path']).exists()
+
+        with open(manifest_path, 'r', encoding='utf-8') as f:
+            manifest_rows = list(csv.DictReader(f))
+        assert manifest_rows[0]['id'] == '100'
+        assert manifest_rows[0]['local_lyrics_path'] == items[0]['local_lyrics_path']
+
+    def test_download_target_table_baseline_resolves_relative_oss_path(self, monkeypatch, tmp_path):
+        rows = [
+            {
+                'id': 100,
+                'name': '相对路径歌词',
+                'singer': '测试歌手',
+                'lyricist': '测试词',
+                'composer': '测试曲',
+                'lyrics_url': '/music_library/music_lyric/20250626/100.txt',
+            },
+        ]
+        requested_urls = []
+
+        class FakeCursor:
+            def __enter__(self):
+                return self
+
+            def __exit__(self, exc_type, exc, tb):
+                return False
+
+            def execute(self, sql):
+                self.sql = sql
+
+            def fetchall(self):
+                return rows
+
+        class FakeConnection:
+            def cursor(self):
+                return FakeCursor()
+
+            def close(self):
+                pass
+
+        monkeypatch.setattr(pymysql, 'connect', lambda **kwargs: FakeConnection())
+        monkeypatch.setitem(OSS_CONFIG, 'base_url', 'https://oss.example.test')
+
+        def fake_download(url, timeout=30):
+            requested_urls.append(url)
+            return '真正的歌词内容\n第二行'.encode('utf-8'), 'text/plain'
+
+        monkeypatch.setattr(sys.modules[__name__], 'download_file', fake_download)
+
+        items, _ = _download_target_table_baseline(
+            tmp_path,
+            limit=0,
+            download_timeout=3,
+            show_progress=False,
+        )
+
+        assert requested_urls == ['https://oss.example.test/music_library/music_lyric/20250626/100.txt']
+        assert len(items) == 1
+        assert items[0]['record'].lyrics == '真正的歌词内容\n第二行'
+        assert Path(items[0]['local_lyrics_path']).read_text(encoding='utf-8') == '真正的歌词内容\n第二行'
+
+    def test_download_target_table_baseline_skips_relative_oss_path_when_download_fails(self, monkeypatch, tmp_path):
+        rows = [
+            {
+                'id': 100,
+                'name': '下载失败',
+                'singer': '测试歌手',
+                'lyricist': '测试词',
+                'composer': '测试曲',
+                'lyrics_url': '/music_library/music_lyric/20250626/missing.txt',
+            },
+        ]
+
+        class FakeCursor:
+            def __enter__(self):
+                return self
+
+            def __exit__(self, exc_type, exc, tb):
+                return False
+
+            def execute(self, sql):
+                self.sql = sql
+
+            def fetchall(self):
+                return rows
+
+        class FakeConnection:
+            def cursor(self):
+                return FakeCursor()
+
+            def close(self):
+                pass
+
+        monkeypatch.setattr(pymysql, 'connect', lambda **kwargs: FakeConnection())
+        monkeypatch.setitem(OSS_CONFIG, 'base_url', 'https://oss.example.test')
+        monkeypatch.setattr(sys.modules[__name__], 'download_file', lambda url, timeout=30: (None, None))
+
+        items, manifest_path = _download_target_table_baseline(
+            tmp_path,
+            limit=0,
+            download_timeout=3,
+            show_progress=False,
+        )
+
+        assert items == []
+        with open(manifest_path, 'r', encoding='utf-8') as f:
+            manifest_rows = list(csv.DictReader(f))
+        assert manifest_rows == []
+
+    def test_download_target_table_baseline_accepts_download_workers(self, monkeypatch, tmp_path):
+        rows = [
+            {
+                'id': idx,
+                'name': f'并发歌词{idx}',
+                'singer': '测试歌手',
+                'lyricist': '测试词',
+                'composer': '测试曲',
+                'lyrics_url': f'https://example.test/{idx}.txt',
+            }
+            for idx in range(1, 4)
+        ]
+
+        class FakeCursor:
+            def __enter__(self):
+                return self
+
+            def __exit__(self, exc_type, exc, tb):
+                return False
+
+            def execute(self, sql):
+                self.sql = sql
+
+            def fetchall(self):
+                return rows
+
+        class FakeConnection:
+            def cursor(self):
+                return FakeCursor()
+
+            def close(self):
+                pass
+
+        monkeypatch.setattr(pymysql, 'connect', lambda **kwargs: FakeConnection())
+        monkeypatch.setattr(
+            sys.modules[__name__],
+            'download_file',
+            lambda url, timeout=30: (f'歌词内容 {url}'.encode('utf-8'), 'text/plain'),
+        )
+
+        items, _ = _download_target_table_baseline(
+            tmp_path,
+            limit=0,
+            download_timeout=3,
+            download_workers=2,
+            show_progress=False,
+        )
+
+        assert [item['source_id'] for item in items] == ['1', '2', '3']
+
+    def test_download_target_table_baseline_skips_instrumental_lyrics(self, monkeypatch, tmp_path):
+        rows = [
+            {
+                'id': 100,
+                'name': '纯音乐',
+                'singer': '测试歌手',
+                'lyricist': '',
+                'composer': '',
+                'lyrics_url': 'https://example.test/instrumental.txt',
+            },
+        ]
+
+        class FakeCursor:
+            def __enter__(self):
+                return self
+
+            def __exit__(self, exc_type, exc, tb):
+                return False
+
+            def execute(self, sql):
+                self.sql = sql
+
+            def fetchall(self):
+                return rows
+
+        class FakeConnection:
+            def cursor(self):
+                return FakeCursor()
+
+            def close(self):
+                pass
+
+        monkeypatch.setattr(pymysql, 'connect', lambda **kwargs: FakeConnection())
+        monkeypatch.setattr(
+            sys.modules[__name__],
+            'download_file',
+            lambda url, timeout=30: ('本曲为纯音乐，请欣赏'.encode('utf-8'), 'text/plain'),
+        )
+
+        items, manifest_path = _download_target_table_baseline(
+            tmp_path,
+            limit=0,
+            download_timeout=3,
+            show_progress=False,
+        )
+
+        assert items == []
+        with open(manifest_path, 'r', encoding='utf-8') as f:
+            assert list(csv.DictReader(f)) == []
+
+
+def _env_bool(name: str, default: bool = False) -> bool:
+    value = os.getenv(name)
+    if value is None:
+        return default
+    return value.strip().lower() in {'1', 'true', 'yes', 'y', 'on'}
+
+
+def _env_int(name: str, default: int) -> int:
+    value = os.getenv(name)
+    if not value:
+        return default
+    return int(value)
+
+
+def _env_float(name: str, default: float | None = None) -> float | None:
+    value = os.getenv(name)
+    if not value:
+        return default
+    return float(value)
+
+
+def _parse_topks(value: str | None) -> list[int]:
+    if not value:
+        return [20, 50, 100, 200, 500]
+    return [int(item.strip()) for item in value.split(',') if item.strip()]
+
+
+def _safe_filename(value: str | None, fallback: str) -> str:
+    text = value or fallback
+    text = ''.join(char if char.isalnum() or char in {'-', '_'} else '_' for char in text)
+    return text[:80] or fallback
+
+
+def _write_lyrics_file(base_dir: Path, prefix: str, record: LyricRecord) -> Path:
+    base_dir.mkdir(parents=True, exist_ok=True)
+    filename = f"{prefix}_{_safe_filename(record.record_id, 'record')}_{_safe_filename(record.title, 'untitled')}.txt"
+    path = base_dir / filename
+    path.write_text(record.lyrics or '', encoding='utf-8')
+    return path
+
+
+def _progress(iterable, *, desc: str, unit: str, leave: bool = True, total: int | None = None):
+    return tqdm(
+        iterable,
+        desc=desc,
+        unit=unit,
+        leave=leave,
+        total=total,
+        dynamic_ncols=True,
+        mininterval=1.0,
+        smoothing=0.1,
+    )
+
+
+def _lyrics_text_from_target_row(row: dict[str, Any], download_timeout: int) -> str:
+    lyrics_url = str(row.get('lyrics_url') or '').strip()
+    if not lyrics_url:
+        return ''
+    if lyrics_url.startswith(('http://', 'https://')):
+        content, _ = download_file(lyrics_url, timeout=download_timeout)
+        if not content:
+            return ''
+        return content.decode('utf-8', errors='replace')
+    if lyrics_url.startswith('/'):
+        base_url = (OSS_CONFIG.get('base_url') or '').rstrip('/')
+        if not base_url:
+            return ''
+        content, _ = download_file(f"{base_url}{lyrics_url}", timeout=download_timeout)
+        if not content:
+            return ''
+        return content.decode('utf-8', errors='replace')
+    return lyrics_url
+
+
+def _target_baseline_item_from_row(
+    row: dict[str, Any],
+    lyric_dir: Path,
+    download_timeout: int,
+) -> tuple[dict[str, Any], dict[str, Any]] | None:
+    lyrics = _lyrics_text_from_target_row(row, download_timeout)
+    if not lyrics.strip():
+        return None
+    if is_instrumental_lyrics(lyrics):
+        return None
+
+    record = LyricRecord(
+        record_id=str(row['id']),
+        lyrics=lyrics,
+        title=row.get('name'),
+        artist=row.get('singer'),
+        lyricist=row.get('lyricist'),
+        composer=row.get('composer'),
+    )
+    local_lyrics_path = _write_lyrics_file(lyric_dir, 'target', record)
+    item = {
+        'record': record,
+        'source_id': str(row['id']),
+        'name': row.get('name') or '',
+        'singer': row.get('singer') or '',
+        'lyricist': row.get('lyricist') or '',
+        'composer': row.get('composer') or '',
+        'lyrics_url': row.get('lyrics_url') or '',
+        'local_lyrics_path': str(local_lyrics_path),
+    }
+    manifest_row = {
+        'id': item['source_id'],
+        'name': item['name'],
+        'singer': item['singer'],
+        'lyricist': item['lyricist'],
+        'composer': item['composer'],
+        'lyrics_url': item['lyrics_url'],
+        'local_lyrics_path': item['local_lyrics_path'],
+    }
+    return item, manifest_row
+
+
+def _download_target_table_baseline(
+    output_dir: Path,
+    limit: int,
+    download_timeout: int,
+    download_workers: int = 8,
+    show_progress: bool = True,
+) -> tuple[list[dict[str, Any]], Path]:
+    """下载 TARGET_TABLE 已有数据到本地，返回可作为去重基线的记录。"""
+    baseline_dir = output_dir / 'target_table_baseline'
+    lyric_dir = baseline_dir / 'lyrics'
+    baseline_dir.mkdir(parents=True, exist_ok=True)
+    lyric_dir.mkdir(parents=True, exist_ok=True)
+
+    sql = (
+        f"SELECT id, name, singer, lyricist, composer, lyrics_url FROM {TARGET_TABLE_NAME} "
+        "WHERE deleted = '0' AND lyrics_url IS NOT NULL "
+        "ORDER BY id"
+    )
+    if limit > 0:
+        sql += f" LIMIT {limit}"
+
+    if show_progress:
+        tqdm.write(f"查询 TARGET_TABLE 基线: table={TARGET_TABLE_NAME}, limit={limit or 'ALL'}")
+    conn = pymysql.connect(**TARGET_DB_CONFIG, cursorclass=pymysql.cursors.DictCursor)
+    try:
+        with conn.cursor() as cursor:
+            cursor.execute(sql)
+            rows = cursor.fetchall()
+    finally:
+        conn.close()
+    if show_progress:
+        tqdm.write(f"TARGET_TABLE 基线查询完成: {len(rows)} 条")
+
+    download_workers = max(1, download_workers)
+    if show_progress:
+        tqdm.write(f"下载 TARGET_TABLE 歌词: workers={download_workers}, timeout={download_timeout}s")
+
+    results: list[tuple[int, dict[str, Any], dict[str, Any]]] = []
+    if download_workers == 1:
+        iterable = _progress(list(enumerate(rows)), desc='下载基线', unit='条') if show_progress else enumerate(rows)
+        for index, row in iterable:
+            result = _target_baseline_item_from_row(row, lyric_dir, download_timeout)
+            if result is not None:
+                item, manifest_row = result
+                results.append((index, item, manifest_row))
+    else:
+        with ThreadPoolExecutor(max_workers=download_workers) as executor:
+            futures = {
+                executor.submit(_target_baseline_item_from_row, row, lyric_dir, download_timeout): index
+                for index, row in enumerate(rows)
+            }
+            iterable = (
+                _progress(as_completed(futures), desc='下载基线', unit='条', total=len(futures))
+                if show_progress else as_completed(futures)
+            )
+            for future in iterable:
+                index = futures[future]
+                try:
+                    result = future.result()
+                except Exception as exc:
+                    if show_progress:
+                        tqdm.write(f"TARGET_TABLE 歌词处理失败: index={index}, error={exc}")
+                    continue
+                if result is not None:
+                    item, manifest_row = result
+                    results.append((index, item, manifest_row))
+
+    results.sort(key=lambda result: result[0])
+    items = [item for _, item, _ in results]
+    manifest_rows = [manifest_row for _, _, manifest_row in results]
+
+    manifest_path = baseline_dir / 'target_table_baseline.csv'
+    _write_csv(
+        manifest_path,
+        manifest_rows,
+        ['id', 'name', 'singer', 'lyricist', 'composer', 'lyrics_url', 'local_lyrics_path'],
+        show_progress=show_progress,
+        desc='写基线清单',
+    )
+    if show_progress:
+        tqdm.write(
+            f"TARGET_TABLE 基线已保存: 有效={len(items)}/{len(rows)}, "
+            f"manifest={manifest_path}, lyrics_dir={lyric_dir}"
+        )
+    return items, manifest_path
+
+
+def _fetch_source_lyric_records(limit: int, offset: int, show_progress: bool = True) -> list[dict[str, Any]]:
+    sql = AGGREGATE_SQL
+    sql += f"\nLIMIT {limit}"
+    if offset:
+        sql += f"\nOFFSET {offset}"
+
+    if show_progress:
+        tqdm.write(f"查询源库歌词: limit={limit}, offset={offset}")
+    conn = pymysql.connect(**SOURCE_DB_CONFIG)
+    try:
+        with conn.cursor() as cursor:
+            cursor.execute(sql)
+            rows = cursor.fetchall()
+    finally:
+        conn.close()
+    if show_progress:
+        tqdm.write(f"源库查询完成: {len(rows)} 条")
+
+    records: list[dict[str, Any]] = []
+    iterable = _progress(rows, desc='整理源库', unit='条') if show_progress else rows
+    for row in iterable:
+        lyrics = row.get('lyrics_txt_content')
+        if not lyrics or not lyrics.strip():
+            continue
+        if is_instrumental_lyrics(lyrics):
+            continue
+        record = LyricRecord(
+            record_id=str(row['source_id']),
+            lyrics=lyrics,
+            title=row.get('name'),
+            artist=row.get('singer'),
+            lyricist=row.get('lyricist'),
+            composer=row.get('composer'),
+        )
+        records.append({
+            'record': record,
+            'source_id': str(row['source_id']),
+            'name': row.get('name') or '',
+            'singer': row.get('singer') or '',
+            'lyricist': row.get('lyricist') or '',
+            'composer': row.get('composer') or '',
+        })
+    return records
+
+
+def _load_existing_lyric_records(
+    limit: int,
+    download_timeout: int,
+    show_progress: bool = True,
+    download_workers: int = 8,
+) -> list[dict[str, Any]]:
+    output_dir = REPORT_DIR / f"target_table_baseline_{time.strftime('%Y%m%d_%H%M%S')}"
+    records, manifest_path = _download_target_table_baseline(
+        output_dir,
+        limit,
+        download_timeout,
+        download_workers=download_workers,
+        show_progress=show_progress,
+    )
+    print(f"TARGET_TABLE baseline manifest: {manifest_path}")
+    return records
+
+
+def _build_record_meta(items: list[dict[str, Any]]) -> dict[str, dict[str, Any]]:
+    return {item['record'].record_id: item for item in items}
+
+
+def _candidate_match_to_row(match: CandidateMatch) -> dict[str, Any]:
+    return {
+        'candidate_decision': match.decision.value,
+        'candidate_confidence': f"{match.confidence:.4f}",
+        'candidate_jaccard': f"{match.jaccard:.4f}",
+        'candidate_line_coverage': f"{match.line_coverage:.4f}",
+        'candidate_primary_jaccard': f"{match.primary_jaccard:.4f}",
+        'candidate_primary_line_coverage': f"{match.primary_line_coverage:.4f}",
+        'candidate_reason': match.reason,
+    }
+
+
+def _l1_metadata_match(left: dict[str, Any], right: dict[str, Any]) -> bool:
+    return (
+        bool(left.get('name'))
+        and _normalize_meta(left.get('name')) == _normalize_meta(right.get('name'))
+        and _normalize_meta(left.get('lyricist')) == _normalize_meta(right.get('lyricist'))
+        and _normalize_meta(left.get('composer')) == _normalize_meta(right.get('composer'))
+    )
+
+
+def _run_l2_retrieval_benchmark(
+    source_items: list[dict[str, Any]],
+    seed_items: list[dict[str, Any]],
+    top_k: int,
+    output_dir: Path,
+    top_n: int,
+    show_progress: bool = True,
+) -> dict[str, Any]:
+    checker = DuplicateChecker()
+    index = L2CandidateIndex(checker, mode='topk', top_k=top_k)
+    record_meta = _build_record_meta(seed_items)
+    lyric_dir = output_dir / 'lyrics'
+    lyric_paths: dict[str, str] = {}
+
+    seed_iterable = _progress(seed_items, desc=f'建索引 k={top_k}', unit='条', leave=False) if show_progress else seed_items
+    for seed in seed_iterable:
+        record = seed['record']
+        index.add(record)
+        lyric_paths[record.record_id] = seed.get('local_lyrics_path') or str(
+            _write_lyrics_file(lyric_dir, 'candidate', record)
+        )
+
+    retrieval_rows: list[dict[str, Any]] = []
+    hit_rows: list[dict[str, Any]] = []
+    decision_counts = {'duplicate': 0, 'review': 0, 'new': 0}
+    total_recalled = 0
+
+    started = time.perf_counter()
+    source_iterable = _progress(source_items, desc=f'L2 k={top_k}', unit='条') if show_progress else source_items
+    for source in source_iterable:
+        record = source['record']
+        query_lyric_path = str(_write_lyrics_file(lyric_dir, 'query', record))
+        recalled = index.recall(record)
+        total_recalled += len(recalled)
+        row = {
+            'source_id': record.record_id,
+            'lyrics_txt_content': record.lyrics,
+            'name': record.title,
+            'singer': record.artist,
+        }
+        decision, confidence, matched_id, reason = check_l2(row, checker, index)
+        decision_counts[decision] += 1
+        if show_progress:
+            source_iterable.set_postfix(
+                dup=decision_counts['duplicate'],
+                rev=decision_counts['review'],
+                new=decision_counts['new'],
+                avg=f"{(total_recalled / max(sum(decision_counts.values()), 1)):.1f}",
+                refresh=False,
+            )
+
+        if decision in {'duplicate', 'review'}:
+            hit_rows.append({
+                'top_k': top_k,
+                'query_source_id': source['source_id'],
+                'query_name': source['name'],
+                'query_lyricist': source['lyricist'],
+                'query_composer': source['composer'],
+                'query_lyrics_path': query_lyric_path,
+                'decision': decision,
+                'confidence': f"{confidence:.4f}",
+                'matched_id': matched_id or '',
+                'matched_name': record_meta.get(matched_id or '', {}).get('name', ''),
+                'matched_lyricist': record_meta.get(matched_id or '', {}).get('lyricist', ''),
+                'matched_composer': record_meta.get(matched_id or '', {}).get('composer', ''),
+                'matched_lyrics_path': lyric_paths.get(matched_id or '', ''),
+                'reason': reason,
+            })
+
+        query_rank_result = checker.check_record_against_candidates(record, recalled, max_candidates=top_n)
+        by_candidate_id = {match.record_id: match for match in query_rank_result.candidates}
+        for rank, candidate in enumerate(recalled[:top_n], start=1):
+            candidate_meta = record_meta.get(candidate.record_id, {})
+            l1_match = _l1_metadata_match(source, candidate_meta)
+            if candidate.record_id not in lyric_paths:
+                lyric_paths[candidate.record_id] = str(_write_lyrics_file(lyric_dir, 'candidate', candidate))
+            match = by_candidate_id.get(candidate.record_id)
+            match_row = _candidate_match_to_row(match) if match else {
+                'candidate_decision': '',
+                'candidate_confidence': '',
+                'candidate_jaccard': '',
+                'candidate_line_coverage': '',
+                'candidate_primary_jaccard': '',
+                'candidate_primary_line_coverage': '',
+                'candidate_reason': '未进入精排 topN',
+            }
+            retrieval_rows.append({
+                'top_k': top_k,
+                'rank': rank,
+                'query_source_id': source['source_id'],
+                'query_name': source['name'],
+                'query_lyricist': source['lyricist'],
+                'query_composer': source['composer'],
+                'query_lyrics_path': query_lyric_path,
+                'candidate_id': candidate.record_id,
+                'candidate_name': candidate_meta.get('name', candidate.title or ''),
+                'candidate_lyricist': candidate_meta.get('lyricist', ''),
+                'candidate_composer': candidate_meta.get('composer', ''),
+                'candidate_lyrics_path': lyric_paths[candidate.record_id],
+                'l1_metadata_match': '1' if l1_match else '0',
+                'l1_l2_conflict': '1' if (
+                    (decision == 'new' and l1_match)
+                    or (decision in {'duplicate', 'review'} and candidate.record_id == (matched_id or '') and not l1_match)
+                ) else '0',
+                **match_row,
+            })
+
+        if decision == 'new':
+            index.add(record)
+            record_meta[record.record_id] = source
+            lyric_paths[record.record_id] = query_lyric_path
+
+    elapsed = time.perf_counter() - started
+    return {
+        'top_k': top_k,
+        'elapsed_seconds': elapsed,
+        'throughput_per_second': len(source_items) / elapsed if elapsed > 0 else 0.0,
+        'avg_recalled': total_recalled / len(source_items) if source_items else 0.0,
+        'decision_counts': decision_counts,
+        'retrieval_rows': retrieval_rows,
+        'hit_rows': hit_rows,
+    }
+
+
+def _write_csv(
+    path: Path,
+    rows: list[dict[str, Any]],
+    fieldnames: list[str],
+    show_progress: bool = False,
+    desc: str | None = None,
+) -> None:
+    with open(path, 'w', encoding='utf-8', newline='') as f:
+        writer = csv.DictWriter(f, fieldnames=fieldnames)
+        writer.writeheader()
+        iterable = _progress(rows, desc=desc or f'写 {path.name}', unit='行') if show_progress else rows
+        for row in iterable:
+            writer.writerow(row)
+
+
+def _summarize_l1_l2_conflicts(rows: list[dict[str, Any]]) -> dict[str, int]:
+    conflict_rows = [row for row in rows if row.get('l1_l2_conflict') == '1']
+    l1_hit_l2_new_rows = [
+        row for row in conflict_rows
+        if (row.get('candidate_decision') or row.get('decision') or '') == 'new'
+        and row.get('l1_metadata_match') == '1'
+    ]
+    l2_hit_l1_miss_rows = [
+        row for row in conflict_rows
+        if (row.get('candidate_decision') or row.get('decision') or '') in {'duplicate', 'review'}
+        and row.get('l1_metadata_match') != '1'
+    ]
+    return {
+        'l1_l2_conflict_row_count': len(conflict_rows),
+        'l1_l2_conflict_query_count': len({row.get('query_source_id', '') for row in conflict_rows}),
+        'l1_hit_l2_new_row_count': len(l1_hit_l2_new_rows),
+        'l1_hit_l2_new_query_count': len({row.get('query_source_id', '') for row in l1_hit_l2_new_rows}),
+        'l2_hit_l1_miss_row_count': len(l2_hit_l1_miss_rows),
+        'l2_hit_l1_miss_query_count': len({row.get('query_source_id', '') for row in l2_hit_l1_miss_rows}),
+    }
+
+
+def _write_l2_benchmark_reports(
+    summary_rows: list[dict[str, Any]],
+    retrieval_rows: list[dict[str, Any]],
+    hit_rows: list[dict[str, Any]],
+    show_progress: bool = True,
+) -> tuple[Path, Path, Path]:
+    timestamp = time.strftime('%Y%m%d_%H%M%S')
+    summary_path = REPORT_DIR / f'l2_topk_benchmark_summary_{timestamp}.csv'
+    retrieval_path = REPORT_DIR / f'l2_topk_retrieval_top10_{timestamp}.csv'
+    hits_path = REPORT_DIR / f'l2_topk_duplicate_hits_{timestamp}.csv'
+
+    _write_csv(summary_path, summary_rows, list(summary_rows[0].keys()), show_progress, '写入 L2 汇总报告')
+    retrieval_fields = [
+        'top_k', 'rank',
+        'query_source_id', 'query_name', 'query_lyricist', 'query_composer', 'query_lyrics_path',
+        'candidate_id', 'candidate_name', 'candidate_lyricist', 'candidate_composer', 'candidate_lyrics_path',
+        'l1_metadata_match', 'l1_l2_conflict',
+        'candidate_decision', 'candidate_confidence', 'candidate_jaccard', 'candidate_line_coverage',
+        'candidate_primary_jaccard', 'candidate_primary_line_coverage', 'candidate_reason',
+    ]
+    hit_fields = [
+        'top_k',
+        'query_source_id', 'query_name', 'query_lyricist', 'query_composer', 'query_lyrics_path',
+        'decision', 'confidence', 'matched_id', 'matched_name', 'matched_lyricist',
+        'matched_composer', 'matched_lyrics_path', 'reason',
+    ]
+    _write_csv(retrieval_path, retrieval_rows, retrieval_fields, show_progress, '写入 L2 召回明细')
+    _write_csv(hits_path, hit_rows, hit_fields, show_progress, '写入 L2 命中明细')
+
+    return summary_path, retrieval_path, hits_path
+
+
+# ====================================================================
+# 手动 L2 检索效率/人工复核材料评测
+# ====================================================================
+
+@pytest.mark.skipif(
+    not _env_bool('RUN_L2_BENCHMARK'),
+    reason='设置 RUN_L2_BENCHMARK=1 才会连接数据库并运行 L2 topK 评测',
+)
+class TestL2Benchmark:
+    """只测试歌词下载和 L2 去重，不上传/下载音频、封面等资源，不写库。"""
+
+    def test_l2_recall_topk_efficiency_and_review_artifacts(self):
+        limit = _env_int('L2_BENCHMARK_LIMIT', 200)
+        offset = _env_int('L2_BENCHMARK_OFFSET', 0)
+        topks = _parse_topks(os.getenv('L2_BENCHMARK_TOPKS'))
+        top_n = _env_int('L2_BENCHMARK_RETRIEVAL_TOP_N', 10)
+        load_existing = _env_bool('L2_BENCHMARK_LOAD_EXISTING', True)
+        existing_limit = _env_int('L2_BENCHMARK_EXISTING_LIMIT', 500)
+        download_timeout = _env_int('L2_BENCHMARK_DOWNLOAD_TIMEOUT', 10)
+        download_workers = _env_int('L2_BENCHMARK_DOWNLOAD_WORKERS', 8)
+        show_progress = _env_bool('L2_BENCHMARK_PROGRESS', True)
+        output_dir = REPORT_DIR / f"l2_topk_benchmark_assets_{time.strftime('%Y%m%d_%H%M%S')}"
+
+        baseline_manifest_path = None
+        if load_existing:
+            seed_records, baseline_manifest_path = _download_target_table_baseline(
+                output_dir,
+                existing_limit,
+                download_timeout,
+                download_workers=download_workers,
+                show_progress=show_progress,
+            )
+        else:
+            seed_records = []
+        print(f"L2 benchmark source query: limit={limit}, offset={offset}")
+        records = _fetch_source_lyric_records(limit, offset, show_progress=show_progress)
+        assert records, '源库查询结果中没有可评测的歌词内容'
+        print(f"L2 benchmark loaded: source_records={len(records)}, seed_records={len(seed_records)}, topks={topks}")
+        if baseline_manifest_path:
+            print(f"TARGET_TABLE baseline manifest: {baseline_manifest_path}")
+
+        summary_rows = []
+        retrieval_rows = []
+        hit_rows = []
+
+        topk_iterable = _progress(topks, desc='topK配置', unit='组') if show_progress else topks
+        for top_k in topk_iterable:
+            result = _run_l2_retrieval_benchmark(
+                records,
+                seed_records,
+                top_k,
+                output_dir,
+                top_n,
+                show_progress=show_progress,
+            )
+            decision_counts = result['decision_counts']
+            conflict_counts = _summarize_l1_l2_conflicts(result['retrieval_rows'])
+            summary_rows.append({
+                'mode': 'topk',
+                'top_k': top_k,
+                'source_records': len(records),
+                'seed_records': len(seed_records),
+                'retrieval_top_n': top_n,
+                'download_workers': download_workers,
+                'elapsed_seconds': f"{result['elapsed_seconds']:.4f}",
+                'throughput_per_second': f"{result['throughput_per_second']:.2f}",
+                'avg_recalled_candidates': f"{result['avg_recalled']:.2f}",
+                'duplicate_count': decision_counts['duplicate'],
+                'review_count': decision_counts['review'],
+                'new_count': decision_counts['new'],
+                'hit_count': decision_counts['duplicate'] + decision_counts['review'],
+                **conflict_counts,
+                'assets_dir': str(output_dir),
+                'target_table_baseline_manifest': str(baseline_manifest_path or ''),
+            })
+            retrieval_rows.extend(result['retrieval_rows'])
+            hit_rows.extend(result['hit_rows'])
+            if show_progress:
+                topk_iterable.set_postfix(
+                    top_k=top_k,
+                    dup=decision_counts['duplicate'],
+                    rev=decision_counts['review'],
+                    new=decision_counts['new'],
+                    refresh=False,
+                )
+
+        summary_path, retrieval_path, hits_path = _write_l2_benchmark_reports(
+            summary_rows,
+            retrieval_rows,
+            hit_rows,
+            show_progress=show_progress,
+        )
+        print(f"\nL2 benchmark summary: {summary_path}")
+        print(f"L2 benchmark retrieval top{top_n}: {retrieval_path}")
+        print(f"L2 benchmark duplicate/review hits: {hits_path}")
+        print(f"L2 benchmark lyric assets: {output_dir}")
+        for row in summary_rows:
+            print(row)
+
+
+if __name__ == '__main__':
+    pytest.main([__file__, '-v', '--tb=short'])

+# L2 歌词去重测试流程指南
+
+本文档说明如何运行 L2 歌词召回测试、查看测试产物，并启动前端页面做人工复核。
+
+## 1. 运行 L2 测试脚本
+
+在项目根目录执行：
+
+```bash
+RUN_L2_BENCHMARK=1 \
+L2_BENCHMARK_LIMIT=20000000 \
+L2_BENCHMARK_OFFSET=0 \
+L2_BENCHMARK_LOAD_EXISTING=1 \
+L2_BENCHMARK_EXISTING_LIMIT=50000 \
+L2_BENCHMARK_DOWNLOAD_WORKERS=32 \
+L2_BENCHMARK_TOPKS=20 \
+L2_BENCHMARK_RETRIEVAL_TOP_N=10 \
+python -m pytest test_dedup.py::TestL2Benchmark::test_l2_recall_topk_efficiency_and_review_artifacts -q -s
+```
+
+这个测试只处理歌词：
+
+- 从源库读取待导入歌词。
+- 可选从目标测试库读取已有 `lyrics_url` 并下载歌词文件。
+- 不下载或上传音频、封面、伴奏、曲谱等资源。
+- 不写入数据库。
+- 对不同 `top_k` 召回配置输出效率指标和人工复核材料。
+
+## 2. 参数说明
+
+| 参数                              | 默认值                | 说明                                                                            |
+| --------------------------------- | --------------------- | ------------------------------------------------------------------------------- |
+| `RUN_L2_BENCHMARK`              | 未开启                | 必须设为 `1` 才会运行联网 benchmark；否则 pytest 会跳过该测试。               |
+| `L2_BENCHMARK_LIMIT`            | `200`               | 从源库查询多少条待评测歌曲。                                                    |
+| `L2_BENCHMARK_OFFSET`           | `0`                 | 源库查询偏移量，用于分段抽样。                                                  |
+| `L2_BENCHMARK_LOAD_EXISTING`    | `1`                 | 是否加载目标测试库已有歌词作为历史候选。设为 `0` 时只测试本批新歌之间的召回。 |
+| `L2_BENCHMARK_EXISTING_LIMIT`   | `500`               | 从目标测试库加载多少条已有歌词候选；设为 `0` 表示不加 SQL `LIMIT`。         |
+| `L2_BENCHMARK_TOPKS`            | `20,50,100,200,500` | 要对比的 topK 召回候选规模，逗号分隔。                                          |
+| `L2_BENCHMARK_RETRIEVAL_TOP_N`  | `10`                | 每条新歌在结果表中保留前多少个候选。前端当前按 top10 展示。                     |
+| `L2_BENCHMARK_DOWNLOAD_TIMEOUT` | `10`                | 下载已有歌词文件的超时时间，单位秒。                                            |
+
+## 3. 输出结果
+
+测试完成后会在 `output/reports/` 下生成：
+
+```text
+l2_topk_benchmark_summary_YYYYMMDD_HHMMSS.csv
+l2_topk_retrieval_top10_YYYYMMDD_HHMMSS.csv
+l2_topk_duplicate_hits_YYYYMMDD_HHMMSS.csv
+l2_topk_benchmark_assets_YYYYMMDD_HHMMSS/lyrics/
+```
+
+各文件含义：
+
+| 文件                                   | 说明                                                                                                     |
+| -------------------------------------- | -------------------------------------------------------------------------------------------------------- |
+| `l2_topk_benchmark_summary_*.csv`    | 每个 topK 的效率汇总，包括耗时、吞吐、平均召回候选数、duplicate/review/new/hit 数量。                    |
+| `l2_topk_retrieval_top10_*.csv`      | 每条新歌的 topN 召回候选明细，包含新歌和候选的歌名、歌手、词曲作者、歌词文件路径、相似度指标、判定原因。 |
+| `l2_topk_duplicate_hits_*.csv`       | 只保留命中 `duplicate` 或 `review` 的样本，适合人工重点复核。                                        |
+| `l2_topk_benchmark_assets_*/lyrics/` | 新歌和候选歌词正文文件。CSV 中只保留路径，不直接塞歌词全文。                                             |
+
+`l2_topk_retrieval_top10_*.csv` 还包含：
+
+- `l1_metadata_match`：新歌与候选按 L1 元数据规则（歌名、作词人、作曲人）是否命中。
+- `l1_l2_conflict`：L1 与 L2 结论是否冲突。比如 L2 判新歌但召回候选中有 L1 命中，或 L2 判重复但命中候选 L1 未命中。
+
+## 4. 启动前端页面
+
+运行：
+
+```bash
+python serve_l2_dashboard.py --host 127.0.0.1 --port 8765
+```
+
+然后打开：
+
+```text
+http://127.0.0.1:8765
+```
+
+前端会自动扫描 `output/reports/` 下成套的 benchmark 结果文件。
+
+## 5. 前端使用方式
+
+页面主要功能：
+
+- 选择不同测试 run。
+- 切换不同 `top_k`。
+- 查看效率指标：耗时、吞吐、平均召回候选数、duplicate/review/new/hit 数量。
+- 在“召回样本”中查看每条新歌的 topN 候选。
+- 在“命中样本”中只查看 `duplicate` / `review` 样本。
+- 左右并排查看新歌歌词和候选歌词。
+- 高亮 L1/L2 冲突样本。
+- 对样本做人工标注：确认重复、确认不重复、待确认。
+- 点击“导出标注”导出当前 run、topK 和当前视图下的人工标注 CSV。
+- 按歌名、歌手、ID 搜索样本。
+
+如果刚跑完新测试但页面没有更新，点击页面右上角“刷新”。
+
+## 6. 推荐测试方式
+
+先用小样本确认流程：
+
+```bash
+RUN_L2_BENCHMARK=1 \
+L2_BENCHMARK_LIMIT=20 \
+L2_BENCHMARK_EXISTING_LIMIT=50 \
+L2_BENCHMARK_TOPKS=20,100 \
+python -m pytest test_dedup.py::TestL2Benchmark::test_l2_recall_topk_efficiency_and_review_artifacts -q -s
+```
+
+再扩大样本：
+
+```bash
+RUN_L2_BENCHMARK=1 \
+L2_BENCHMARK_LIMIT=1000 \
+L2_BENCHMARK_EXISTING_LIMIT=5000 \
+L2_BENCHMARK_TOPKS=20,50,100,200,500 \
+python -m pytest test_dedup.py::TestL2Benchmark::test_l2_recall_topk_efficiency_and_review_artifacts -q -s
+```
+
+如果目标库已有歌词很多，`L2_BENCHMARK_EXISTING_LIMIT=0` 会加载全部候选，运行时间和歌词下载时间会明显增加。
+
+## 7. 普通单元测试
+
+运行全部普通测试：
+
+```bash
+python -m pytest test_dedup.py -q
+```
+
+默认情况下，L2 benchmark 会被跳过，不会联网。

+音眼数据库配置：
+
+测试：rm-bp18h64ad9ak4d7h5do.mysql.rds.aliyuncs.com  3306   root  Hikoon123!  hikoon-data-test
+
+正式：username: yinyan360
+          password: yinyan_hikoon!@#6699   内网： rm-bp10xcwu0930i0h00.mysql.rds.aliyuncs.com 外网：rm-bp10xcwu0930i0h00ro.mysql.rds.aliyuncs.com   端口：3306
\ No newline at end of file