Skip to content

wanghai-tech / hikoon-ACR

Go to a project
Toggle navigation
Toggle navigation pinning
Commit 96c9ce7d 96c9ce7d40ae89231e6f9d1a37d73e2c5bb380ff by cnb.bofCdSsphPA
Options

Validate the PostgreSQL ACR storage path with live evidence 

Constraint: The new data model had to be proven against the user-provided PostgreSQL instance and stay aligned with Phase-1 encoder-only decisions
Rejected: Document-only schema guidance without a live database run | It would leave retrieval correctness and table intent unproven
Confidence: high
Scope-risk: narrow
Directive: Keep future retrieval experiments writing through model/feature/reference registries instead of adding fixed per-model columns
Tested: /usr/local/miniconda3/bin/python scripts/live_pgvector_music20_eval.py --dsn 'postgres://d2:d2pass@127.0.0.1:5432/d2' --schema acr_test --reset-schema --output data/pgvector_eval/music20/live_pgvector_report.json; /usr/local/miniconda3/bin/python scripts/evaluate_songid_pgvector_path.py --reference-embeddings-jsonl data/pgvector_eval/music20/reference_embeddings.jsonl --query-embeddings-jsonl data/pgvector_eval/music20/query_embeddings.jsonl --output data/pgvector_eval/music20/songid_eval_report_fresh.json; /usr/local/miniconda3/bin/python -m py_compile scripts/live_pgvector_music20_eval.py scripts/evaluate_songid_pgvector_path.py; git diff --check -- docs/README.md docs/CHANGELOG.md docs/postgres_db_schema_samples.md acr-engine/scripts/live_pgvector_music20_eval.py acr-engine/data/pgvector_eval/music20/live_pgvector_report.json acr-engine/data/pgvector_eval/music20/songid_eval_report_fresh.json
Not-tested: MERT/MuQ live embeddings, type_8/type_16 live JSONL coverage, multi-recording/cover-lane decision flow
1 parent b220751b
Inline Side-by-side
Showing 6 changed files with 1428 additions and 0 deletions
1 {
2 "schema": "acr_test",
3 "dsn_redacted": "postgres://d2:***@127.0.0.1:5432/d2",
4 "input": {
5 "reference_embeddings_jsonl": "/workspace/acr-engine/data/pgvector_eval/music20/reference_embeddings.jsonl",
6 "query_embeddings_jsonl": "/workspace/acr-engine/data/pgvector_eval/music20/query_embeddings.jsonl",
7 "reference_count": 20,
8 "query_count": 22
9 },
10 "registry": {
11 "model_id": 1,
12 "feature_set_id": 1,
13 "reference_set_id": 1,
14 "retrieval_index_id": 1
15 },
16 "table_counts": {
17 "canonical_song": 20,
18 "work": 20,
19 "recording": 20,
20 "recording_asset": 20,
21 "audio_window": 20,
22 "audio_embedding": 20,
23 "retrieval_candidate": 220,
24 "match_decision": 22
25 },
26 "lineage_negative_test": {
27 "passed": true,
28 "error_type": "RaiseException",
29 "message": "Invalid asset_id=1 or recording_id=1000000 for audio_window"
30 },
31 "evaluation": {
32 "backend": "postgresql+pgvector-live",
33 "note": "Reference embeddings are stored in schema v2; 24-d logical embeddings are zero-padded to vector(192) for physical storage.",
34 "overall": {
35 "count": 22,
36 "top1": 0.909091,
37 "top3": 0.954545,
38 "top10": 0.954545,
39 "mrr": 0.934343,
40 "mean_rank": 1.8182,
41 "median_rank": 1.0
42 },
43 "by_query_type": {
44 "1": {
45 "count": 20,
46 "top1": 1.0,
47 "top3": 1.0,
48 "top10": 1.0,
49 "mrr": 1.0,
50 "mean_rank": 1.0,
51 "median_rank": 1.0
52 },
53 "7": {
54 "count": 2,
55 "top1": 0.0,
56 "top3": 0.5,
57 "top10": 0.5,
58 "mrr": 0.277778,
59 "mean_rank": 10.0,
60 "median_rank": 10.0
61 }
62 },
63 "confusion_focus": {
64 "7": {
65 "query_type": 7,
66 "metrics": {
67 "count": 2,
68 "top1": 0.0,
69 "top3": 0.5,
70 "top10": 0.5,
71 "mrr": 0.277778,
72 "mean_rank": 10.0,
73 "median_rank": 10.0
74 },
75 "interpretation": "light confusion / transformed query"
76 },
77 "8": {
78 "query_type": 8,
79 "metrics": {
80 "count": 0
81 },
82 "interpretation": "harder confusion bucket"
83 },
84 "16": {
85 "query_type": 16,
86 "metrics": {
87 "count": 0
88 },
89 "interpretation": "strong confusion or far-domain bucket"
90 }
91 },
92 "examples": {
93 "1": [
94 {
95 "query_id": "music20-q0000-t1-song100",
96 "song_id": "100",
97 "rank": 1,
98 "top3": [
99 {
100 "song_id": "100",
101 "canonical_song_id": 1,
102 "evidence_window_id": 1,
103 "combined_score": 0.9099869376417087,
104 "max_sim": 0.9999854862685651,
105 "top3_avg": 0.9999854862685651,
106 "vote": 1
107 },
108 {
109 "song_id": "116",
110 "canonical_song_id": 17,
111 "evidence_window_id": 17,
112 "combined_score": 0.8674688834706314,
113 "max_sim": 0.9527432038562573,
114 "top3_avg": 0.9527432038562573,
115 "vote": 1
116 },
117 {
118 "song_id": "103",
119 "canonical_song_id": 4,
120 "evidence_window_id": 4,
121 "combined_score": 0.8665370278518509,
122 "max_sim": 0.9517078087242788,
123 "top3_avg": 0.9517078087242788,
124 "vote": 1
125 }
126 ]
127 },
128 {
129 "query_id": "music20-q0001-t1-song101",
130 "song_id": "101",
131 "rank": 1,
132 "top3": [
133 {
134 "song_id": "101",
135 "canonical_song_id": 2,
136 "evidence_window_id": 2,
137 "combined_score": 0.9099997586011674,
138 "max_sim": 0.999999731779075,
139 "top3_avg": 0.999999731779075,
140 "vote": 1
141 },
142 {
143 "song_id": "118",
144 "canonical_song_id": 19,
145 "evidence_window_id": 19,
146 "combined_score": 0.8930541242989376,
147 "max_sim": 0.9811712492210417,
148 "top3_avg": 0.9811712492210417,
149 "vote": 1
150 },
151 {
152 "song_id": "116",
153 "canonical_song_id": 17,
154 "evidence_window_id": 17,
155 "combined_score": 0.892017854392,
156 "max_sim": 0.9800198382133333,
157 "top3_avg": 0.9800198382133333,
158 "vote": 1
159 }
160 ]
161 },
162 {
163 "query_id": "music20-q0002-t1-song102",
164 "song_id": "102",
165 "rank": 1,
166 "top3": [
167 {
168 "song_id": "102",
169 "canonical_song_id": 3,
170 "evidence_window_id": 3,
171 "combined_score": 0.9099973714353238,
172 "max_sim": 0.9999970793725819,
173 "top3_avg": 0.9999970793725819,
174 "vote": 1
175 },
176 {
177 "song_id": "113",
178 "canonical_song_id": 14,
179 "evidence_window_id": 14,
180 "combined_score": 0.878619819365752,
181 "max_sim": 0.9651331326286134,
182 "top3_avg": 0.9651331326286134,
183 "vote": 1
184 },
185 {
186 "song_id": "118",
187 "canonical_song_id": 19,
188 "evidence_window_id": 19,
189 "combined_score": 0.8727551417721799,
190 "max_sim": 0.9586168241913111,
191 "top3_avg": 0.9586168241913111,
192 "vote": 1
193 }
194 ]
195 },
196 {
197 "query_id": "music20-q0003-t1-song103",
198 "song_id": "103",
199 "rank": 1,
200 "top3": [
201 {
202 "song_id": "103",
203 "canonical_song_id": 4,
204 "evidence_window_id": 4,
205 "combined_score": 0.9078967457382905,
206 "max_sim": 0.9976630508203228,
207 "top3_avg": 0.9976630508203228,
208 "vote": 1
209 },
210 {
211 "song_id": "116",
212 "canonical_song_id": 17,
213 "evidence_window_id": 17,
214 "combined_score": 0.8892688048103843,
215 "max_sim": 0.9769653386782048,
216 "top3_avg": 0.9769653386782048,
217 "vote": 1
218 },
219 {
220 "song_id": "109",
221 "canonical_song_id": 10,
222 "evidence_window_id": 10,
223 "combined_score": 0.8786497490793317,
224 "max_sim": 0.9651663878659241,
225 "top3_avg": 0.9651663878659241,
226 "vote": 1
227 }
228 ]
229 },
230 {
231 "query_id": "music20-q0004-t1-song104",
232 "song_id": "104",
233 "rank": 1,
234 "top3": [
235 {
236 "song_id": "104",
237 "canonical_song_id": 5,
238 "evidence_window_id": 5,
239 "combined_score": 0.9099890834089845,
240 "max_sim": 0.9999878704544272,
241 "top3_avg": 0.9999878704544272,
242 "vote": 1
243 },
244 {
245 "song_id": "109",
246 "canonical_song_id": 10,
247 "evidence_window_id": 10,
248 "combined_score": 0.8646899513807881,
249 "max_sim": 0.9496555015342091,
250 "top3_avg": 0.9496555015342091,
251 "vote": 1
252 },
253 {
254 "song_id": "116",
255 "canonical_song_id": 17,
256 "evidence_window_id": 17,
257 "combined_score": 0.8414633946738618,
258 "max_sim": 0.9238482163042909,
259 "top3_avg": 0.9238482163042909,
260 "vote": 1
261 }
262 ]
263 }
264 ],
265 "7": [
266 {
267 "query_id": "music20-q0020-t7-song111",
268 "song_id": "111",
269 "rank": 18,
270 "top3": [
271 {
272 "song_id": "109",
273 "canonical_song_id": 10,
274 "evidence_window_id": 10,
275 "combined_score": 0.8765411333280498,
276 "max_sim": 0.9628234814756109,
277 "top3_avg": 0.9628234814756109,
278 "vote": 1
279 },
280 {
281 "song_id": "116",
282 "canonical_song_id": 17,
283 "evidence_window_id": 17,
284 "combined_score": 0.8749381679370203,
285 "max_sim": 0.9610424088189115,
286 "top3_avg": 0.9610424088189115,
287 "vote": 1
288 },
289 {
290 "song_id": "118",
291 "canonical_song_id": 19,
292 "evidence_window_id": 19,
293 "combined_score": 0.8641276021561776,
294 "max_sim": 0.9490306690624195,
295 "top3_avg": 0.9490306690624195,
296 "vote": 1
297 }
298 ]
299 },
300 {
301 "query_id": "music20-q0021-t7-song116",
302 "song_id": "116",
303 "rank": 2,
304 "top3": [
305 {
306 "song_id": "109",
307 "canonical_song_id": 10,
308 "evidence_window_id": 10,
309 "combined_score": 0.8701787704282636,
310 "max_sim": 0.9557541893647373,
311 "top3_avg": 0.9557541893647373,
312 "vote": 1
313 },
314 {
315 "song_id": "116",
316 "canonical_song_id": 17,
317 "evidence_window_id": 17,
318 "combined_score": 0.8674951972070233,
319 "max_sim": 0.9527724413411371,
320 "top3_avg": 0.9527724413411371,
321 "vote": 1
322 },
323 {
324 "song_id": "103",
325 "canonical_song_id": 4,
326 "evidence_window_id": 4,
327 "combined_score": 0.8659579133987426,
328 "max_sim": 0.9510643482208252,
329 "top3_avg": 0.9510643482208252,
330 "vote": 1
331 }
332 ]
333 }
334 ]
335 }
336 }
337 }
...\ No newline at end of file ...\ No newline at end of file
1 {
2 "backend": "faiss-as-pgvector-standin",
3 "note": "Uses song-level aggregation compatible with a future pgvector online path.",
4 "overall": {
5 "count": 22,
6 "top1": 0.909091,
7 "top3": 0.954545,
8 "top10": 0.954545,
9 "mrr": 0.934343,
10 "mean_rank": 1.8182,
11 "median_rank": 1.0
12 },
13 "by_query_type": {
14 "1": {
15 "count": 20,
16 "top1": 1.0,
17 "top3": 1.0,
18 "top10": 1.0,
19 "mrr": 1.0,
20 "mean_rank": 1.0,
21 "median_rank": 1.0
22 },
23 "7": {
24 "count": 2,
25 "top1": 0.0,
26 "top3": 0.5,
27 "top10": 0.5,
28 "mrr": 0.277778,
29 "mean_rank": 10.0,
30 "median_rank": 10.0
31 }
32 },
33 "examples": {
34 "1": [
35 {
36 "song_id": "100",
37 "rank": 1,
38 "top3": [
39 [
40 "100",
41 0.9099869644641876,
42 0.9999855160713196,
43 0.9999855160713196,
44 1
45 ],
46 [
47 "116",
48 0.8674689626693726,
49 0.9527432918548584,
50 0.9527432918548584,
51 1
52 ],
53 [
54 "103",
55 0.8665370559692382,
56 0.9517078399658203,
57 0.9517078399658203,
58 1
59 ]
60 ]
61 },
62 {
63 "song_id": "101",
64 "rank": 1,
65 "top3": [
66 [
67 "101",
68 0.9099996781349182,
69 0.9999996423721313,
70 0.9999996423721313,
71 1
72 ],
73 [
74 "118",
75 0.8930539643764497,
76 0.9811710715293884,
77 0.9811710715293884,
78 1
79 ],
80 [
81 "116",
82 0.8920178270339967,
83 0.9800198078155518,
84 0.9800198078155518,
85 1
86 ]
87 ]
88 },
89 {
90 "song_id": "102",
91 "rank": 1,
92 "top3": [
93 [
94 "102",
95 0.9099974250793457,
96 0.9999971389770508,
97 0.9999971389770508,
98 1
99 ],
100 [
101 "113",
102 0.878619978427887,
103 0.9651333093643188,
104 0.9651333093643188,
105 1
106 ],
107 [
108 "118",
109 0.8727551674842834,
110 0.9586168527603149,
111 0.9586168527603149,
112 1
113 ]
114 ]
115 },
116 {
117 "song_id": "103",
118 "rank": 1,
119 "top3": [
120 [
121 "103",
122 0.9078967189788818,
123 0.9976630210876465,
124 0.9976630210876465,
125 1
126 ],
127 [
128 "116",
129 0.8892688846588135,
130 0.9769654273986816,
131 0.9769654273986816,
132 1
133 ],
134 [
135 "109",
136 0.8786498045921325,
137 0.965166449546814,
138 0.965166449546814,
139 1
140 ]
141 ]
142 },
143 {
144 "song_id": "104",
145 "rank": 1,
146 "top3": [
147 [
148 "104",
149 0.9099890029430389,
150 0.999987781047821,
151 0.999987781047821,
152 1
153 ],
154 [
155 "109",
156 0.8646899795532226,
157 0.9496555328369141,
158 0.9496555328369141,
159 1
160 ],
161 [
162 "116",
163 0.8414634442329406,
164 0.9238482713699341,
165 0.9238482713699341,
166 1
167 ]
168 ]
169 }
170 ],
171 "7": [
172 {
173 "song_id": "111",
174 "rank": 18,
175 "top3": [
176 [
177 "109",
178 0.8765411591529846,
179 0.9628235101699829,
180 0.9628235101699829,
181 1
182 ],
183 [
184 "116",
185 0.8749382710456848,
186 0.9610425233840942,
187 0.9610425233840942,
188 1
189 ],
190 [
191 "118",
192 0.8641276276111602,
193 0.9490306973457336,
194 0.9490306973457336,
195 1
196 ]
197 ]
198 },
199 {
200 "song_id": "116",
201 "rank": 2,
202 "top3": [
203 [
204 "109",
205 0.8701787447929383,
206 0.9557541608810425,
207 0.9557541608810425,
208 1
209 ],
210 [
211 "116",
212 0.8674952483177185,
213 0.9527724981307983,
214 0.9527724981307983,
215 1
216 ],
217 [
218 "103",
219 0.8659579670429229,
220 0.95106440782547,
221 0.95106440782547,
222 1
223 ]
224 ]
225 }
226 ]
227 }
228 }
...\ No newline at end of file ...\ No newline at end of file
1 #!/usr/bin/env /usr/local/miniconda3/bin/python
2 from __future__ import annotations
3
4 import argparse
5 import json
6 from collections import defaultdict
7 from dataclasses import dataclass
8 from pathlib import Path
9 from statistics import median
10 from typing import Any
11
12 import psycopg
13
14 ROOT = Path(__file__).resolve().parents[1]
15 DEFAULT_SCHEMA_SQL = ROOT / 'sql' / 'acr_pg_schema_v2.sql'
16 DEFAULT_REFERENCE = ROOT / 'data' / 'pgvector_eval' / 'music20' / 'reference_embeddings.jsonl'
17 DEFAULT_QUERY = ROOT / 'data' / 'pgvector_eval' / 'music20' / 'query_embeddings.jsonl'
18 DEFAULT_OUTPUT = ROOT / 'data' / 'pgvector_eval' / 'music20' / 'live_pgvector_report.json'
19
20
21 @dataclass
22 class EntityIds:
23 canonical_song_id: int
24 work_id: int
25 recording_id: int
26 asset_id: int
27 window_id: int
28 embedding_id: int
29
30
31 def load_jsonl(path: Path) -> list[dict[str, Any]]:
32 return [json.loads(line) for line in path.read_text(encoding='utf-8').splitlines() if line.strip()]
33
34
35 def pad_embedding(vec: list[float], target_dim: int = 192) -> list[float]:
36 if len(vec) > target_dim:
37 raise ValueError(f'embedding dim {len(vec)} > target {target_dim}')
38 if len(vec) == target_dim:
39 return vec
40 return vec + [0.0] * (target_dim - len(vec))
41
42
43 def vec_literal(vec: list[float]) -> str:
44 return '[' + ','.join(f'{x:.10f}' for x in vec) + ']'
45
46
47 def compute_metrics(ranks: list[int], topk: int) -> dict[str, Any]:
48 if not ranks:
49 return {'count': 0}
50 return {
51 'count': len(ranks),
52 'top1': round(sum(1 for r in ranks if r == 1) / len(ranks), 6),
53 'top3': round(sum(1 for r in ranks if r <= 3) / len(ranks), 6),
54 f'top{topk}': round(sum(1 for r in ranks if r <= topk) / len(ranks), 6),
55 'mrr': round(sum(1.0 / r for r in ranks) / len(ranks), 6),
56 'mean_rank': round(sum(ranks) / len(ranks), 4),
57 'median_rank': median(ranks),
58 }
59
60
61 def aggregate_song_scores(rows: list[dict[str, Any]]) -> list[dict[str, Any]]:
62 grouped: dict[str, list[dict[str, Any]]] = defaultdict(list)
63 for row in rows:
64 grouped[row['song_id']].append(row)
65 ranked = []
66 for song_id, vals in grouped.items():
67 vals.sort(key=lambda x: x['score'], reverse=True)
68 scores = [v['score'] for v in vals]
69 max_sim = scores[0]
70 top3_avg = sum(scores[:3]) / min(3, len(scores))
71 vote = len(scores)
72 combined = 0.6 * max_sim + 0.3 * top3_avg + 0.1 * min(vote / 10.0, 1.0)
73 ranked.append({
74 'song_id': song_id,
75 'canonical_song_id': vals[0]['canonical_song_id'],
76 'evidence_window_id': vals[0]['window_id'],
77 'combined_score': combined,
78 'max_sim': max_sim,
79 'top3_avg': top3_avg,
80 'vote': vote,
81 })
82 ranked.sort(key=lambda x: x['combined_score'], reverse=True)
83 return ranked
84
85
86 def reset_schema(conn: psycopg.Connection, schema: str) -> None:
87 conn.execute(f'DROP SCHEMA IF EXISTS {schema} CASCADE;')
88 conn.execute(f'CREATE SCHEMA {schema};')
89 conn.execute(f'SET search_path TO {schema}, public;')
90
91
92 def apply_schema(conn: psycopg.Connection, schema_sql: Path) -> None:
93 sql_text = schema_sql.read_text(encoding='utf-8')
94 conn.execute(sql_text)
95
96
97 def seed_registry(conn: psycopg.Connection) -> tuple[int, int, int, int]:
98 model_id = conn.execute(
99 """
100 INSERT INTO model_registry (
101 model_name, model_family, model_version, model_source, model_uri,
102 license_name, input_sample_rate, default_window_sec, default_hop_sec,
103 output_embedding_dim, pooling_supported, metadata_json
104 ) VALUES (
105 'local_chroma24', 'chroma_baseline', 'v1', 'repo-local-eval',
106 'acr-engine/scripts/live_pgvector_music20_eval.py', 'internal-eval',
107 22050, 8.0, 8.0, 24, ARRAY['mean_std'],
108 '{"storage_padding":"zero-pad to vector(192) for pgvector compatibility"}'::jsonb
109 )
110 ON CONFLICT (model_name, model_version) DO UPDATE
111 SET updated_at = NOW()
112 RETURNING model_id;
113 """
114 ).fetchone()[0]
115
116 feature_set_id = conn.execute(
117 """
118 INSERT INTO feature_set_registry (
119 model_id, feature_name, feature_level, extraction_granularity,
120 window_sec, hop_sec, embedding_dim, pooling_strategy, layer_selection,
121 normalize_l2, distance_metric, quantization_type, feature_schema_version,
122 config_json, status
123 ) VALUES (
124 %s, 'chroma24_songid_eval', 'window', 'window',
125 8.0, 8.0, 24, 'mean_std', 'na', TRUE, 'cosine', NULL, 'v1',
126 '{"physical_storage":"audio_embedding_vector_192","padding":"zero"}'::jsonb,
127 'active'
128 )
129 RETURNING feature_set_id;
130 """,
131 (model_id,),
132 ).fetchone()[0]
133
134 reference_set_id = conn.execute(
135 """
136 INSERT INTO reference_set_registry (set_name, description, encoder_scope, status, metadata_json)
137 VALUES (
138 'music20_live_reference',
139 '20-song local live pgvector evaluation reference set',
140 'local_chroma24',
141 'active',
142 '{"purpose":"live_pgvector_music20_eval"}'::jsonb
143 )
144 ON CONFLICT (set_name) DO UPDATE SET updated_at = NOW()
145 RETURNING reference_set_id;
146 """
147 ).fetchone()[0]
148
149 retrieval_index_id = conn.execute(
150 """
151 INSERT INTO retrieval_index_registry (
152 feature_set_id, index_name, index_backend, index_type, storage_uri,
153 shard_no, row_count, index_status, config_json, built_at
154 ) VALUES (
155 %s, 'music20_live_pgvector_hnsw', 'pgvector', 'hnsw_cosine',
156 'postgres://d2@127.0.0.1/d2#acr_test.audio_embedding_vector_192',
157 0, 0, 'active', '{"physical_dim":192,"logical_dim":24}'::jsonb, NOW()
158 )
159 RETURNING retrieval_index_id;
160 """,
161 (feature_set_id,),
162 ).fetchone()[0]
163
164 return model_id, feature_set_id, reference_set_id, retrieval_index_id
165
166
167 def ingest_references(conn: psycopg.Connection, refs: list[dict[str, Any]], feature_set_id: int, reference_set_id: int) -> dict[str, EntityIds]:
168 entities: dict[str, EntityIds] = {}
169 for idx, row in enumerate(refs):
170 song_id = str(row['song_id'])
171 canonical_song_id = conn.execute(
172 """
173 INSERT INTO canonical_song (biz_song_code, title, title_norm, primary_artist, primary_artist_norm, rights_status, metadata_json)
174 VALUES (%s, %s, %s, %s, %s, %s, %s::jsonb)
175 RETURNING canonical_song_id;
176 """,
177 (song_id, f'Song {song_id}', f'song {song_id}', f'Artist {song_id}', f'artist {song_id}', 'protected', json.dumps({'source': 'music20_live_eval'})),
178 ).fetchone()[0]
179 work_id = conn.execute(
180 """
181 INSERT INTO work (canonical_song_id, work_code, work_title, work_title_norm, composer, publisher, metadata_json)
182 VALUES (%s, %s, %s, %s, %s, %s, %s::jsonb)
183 RETURNING work_id;
184 """,
185 (canonical_song_id, f'work-{song_id}', f'Song {song_id}', f'song {song_id}', f'Composer {song_id}', 'Unknown', json.dumps({'note': '1:1 work for eval'})),
186 ).fetchone()[0]
187 recording_id = conn.execute(
188 """
189 INSERT INTO recording (
190 work_id, canonical_song_id, recording_code, recording_title, artist_name,
191 album_name, version_type, is_reference, reference_priority, duration_sec, metadata_json
192 ) VALUES (%s, %s, %s, %s, %s, %s, %s, TRUE, %s, %s, %s::jsonb)
193 RETURNING recording_id;
194 """,
195 (work_id, canonical_song_id, f'rec-{song_id}', f'Song {song_id} Reference', f'Artist {song_id}', 'music20', 'master_reference', 100 + idx, 8.0, json.dumps({'source_audio_path': row['audio_path']})),
196 ).fetchone()[0]
197 asset_id = conn.execute(
198 """
199 INSERT INTO recording_asset (
200 recording_id, asset_role, storage_uri, storage_scheme, file_ext, mime_type,
201 sample_rate, channels, codec_name, duration_sec, normalized_storage_uri,
202 ingest_status, metadata_json
203 ) VALUES (%s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s::jsonb)
204 RETURNING asset_id;
205 """,
206 (recording_id, 'reference_audio', row['audio_path'], 'file', Path(row['audio_path']).suffix.lstrip('.'), 'audio/wav', 22050, 1, 'pcm_s16le', 8.0, row['audio_path'], 'ready', json.dumps({'type': 'reference'})),
207 ).fetchone()[0]
208 window_id = conn.execute(
209 """
210 INSERT INTO audio_window (
211 asset_id, recording_id, work_id, canonical_song_id,
212 window_index, start_sec, end_sec, duration_sec,
213 segment_role, segment_type, quality_score, active_for_index, metadata_json
214 ) VALUES (%s, %s, %s, %s, 0, 0.0, 8.0, 8.0, 'reference', 'full_clip', 1.0, TRUE, %s::jsonb)
215 RETURNING window_id;
216 """,
217 (asset_id, recording_id, work_id, canonical_song_id, json.dumps({'source_audio_path': row['audio_path']})),
218 ).fetchone()[0]
219 embedding_id = conn.execute(
220 """
221 INSERT INTO audio_embedding (
222 feature_set_id, extraction_job_id, asset_id, window_id, recording_id, work_id,
223 canonical_song_id, embedding_storage_mode, embedding_uri, vector_norm, checksum,
224 is_indexed, metadata_json
225 ) VALUES (%s, NULL, %s, %s, %s, %s, %s, %s, NULL, %s, NULL, TRUE, %s::jsonb)
226 RETURNING embedding_id;
227 """,
228 (feature_set_id, asset_id, window_id, recording_id, work_id, canonical_song_id, 'pgvector_inline_192_padded', 1.0, json.dumps({'logical_embedding_dim': len(row['embedding'])})),
229 ).fetchone()[0]
230 conn.execute(
231 'INSERT INTO audio_embedding_vector_192 (embedding_id, embedding) VALUES (%s, %s::vector);',
232 (embedding_id, vec_literal(pad_embedding(row['embedding']))),
233 )
234 conn.execute(
235 'INSERT INTO reference_set_member (reference_set_id, recording_id, member_role) VALUES (%s, %s, %s);',
236 (reference_set_id, recording_id, 'hot_reference'),
237 )
238 entities[song_id] = EntityIds(canonical_song_id, work_id, recording_id, asset_id, window_id, embedding_id)
239 return entities
240
241
242 def run_lineage_negative_test(conn: psycopg.Connection, entity: EntityIds) -> dict[str, Any]:
243 try:
244 with conn.transaction():
245 conn.execute(
246 """
247 INSERT INTO audio_window (
248 asset_id, recording_id, work_id, canonical_song_id, window_index,
249 start_sec, end_sec, duration_sec, segment_role, segment_type, quality_score, active_for_index
250 ) VALUES (%s, %s, %s, %s, 999, 0.0, 8.0, 8.0, 'reference', 'bad_lineage', 0.0, TRUE);
251 """,
252 (entity.asset_id, entity.recording_id + 999999, entity.work_id, entity.canonical_song_id),
253 )
254 return {'passed': False, 'note': 'bad lineage insert unexpectedly succeeded'}
255 except Exception as exc:
256 return {'passed': True, 'error_type': type(exc).__name__, 'message': str(exc).splitlines()[0]}
257
258
259 def fetch_raw_candidates(conn: psycopg.Connection, feature_set_id: int, query_vec: list[float], topn: int) -> list[dict[str, Any]]:
260 rows = conn.execute(
261 """
262 SELECT
263 cs.biz_song_code AS song_id,
264 ae.canonical_song_id,
265 aw.window_id,
266 1 - (aev.embedding <=> %s::vector) AS score
267 FROM audio_embedding_vector_192 aev
268 JOIN audio_embedding ae ON ae.embedding_id = aev.embedding_id
269 JOIN canonical_song cs ON cs.canonical_song_id = ae.canonical_song_id
270 JOIN audio_window aw ON aw.window_id = ae.window_id
271 WHERE ae.feature_set_id = %s
272 ORDER BY aev.embedding <=> %s::vector
273 LIMIT %s;
274 """,
275 (vec_literal(pad_embedding(query_vec)), feature_set_id, vec_literal(pad_embedding(query_vec)), topn),
276 ).fetchall()
277 return [
278 {
279 'song_id': r[0],
280 'canonical_song_id': r[1],
281 'window_id': r[2],
282 'score': float(r[3]),
283 }
284 for r in rows
285 ]
286
287
288 def persist_candidates(conn: psycopg.Connection, query_id: str, retrieval_index_id: int, feature_set_id: int, ranked: list[dict[str, Any]], topk: int) -> None:
289 for i, item in enumerate(ranked[:topk], start=1):
290 conn.execute(
291 """
292 INSERT INTO retrieval_candidate (
293 query_id, retrieval_index_id, feature_set_id, source_lane,
294 candidate_level, candidate_id, evidence_window_id, raw_score,
295 normalized_score, rank_no, metadata_json
296 ) VALUES (%s, %s, %s, 'semantic', 'canonical_song', %s, %s, %s, %s, %s, %s::jsonb);
297 """,
298 (query_id, retrieval_index_id, feature_set_id, item['canonical_song_id'], item['evidence_window_id'], item['max_sim'], item['combined_score'], i, json.dumps({'vote': item['vote'], 'song_id': item['song_id']})),
299 )
300
301
302 def persist_decision(conn: psycopg.Connection, query_id: str, ranked: list[dict[str, Any]]) -> None:
303 top = ranked[0] if ranked else None
304 conn.execute(
305 """
306 INSERT INTO match_decision (
307 query_id, canonical_song_id, work_id, recording_id,
308 decision_status, decision_score, decision_reason, metadata_json
309 ) VALUES (%s, %s, NULL, NULL, %s, %s, %s, %s::jsonb);
310 """,
311 (
312 query_id,
313 top['canonical_song_id'] if top else None,
314 'matched' if top else 'no_match',
315 top['combined_score'] if top else None,
316 'top1 semantic candidate from live pgvector eval' if top else 'no candidate',
317 json.dumps({'top_song_id': top['song_id']} if top else {}),
318 ),
319 )
320
321
322 def evaluate_live(conn: psycopg.Connection, feature_set_id: int, retrieval_index_id: int, queries: list[dict[str, Any]], topn: int, topk: int) -> dict[str, Any]:
323 by_type: dict[str, list[int]] = defaultdict(list)
324 examples: dict[str, list[dict[str, Any]]] = defaultdict(list)
325 confusion_focus: dict[str, dict[str, Any]] = {}
326
327 for idx, q in enumerate(queries):
328 qtype = str(q['query_type'])
329 query_id = f'music20-q{idx:04d}-t{qtype}-song{q["song_id"]}'
330 raw_rows = fetch_raw_candidates(conn, feature_set_id, q['embedding'], topn)
331 ranked = aggregate_song_scores(raw_rows)
332 gold = str(q['song_id'])
333 rank = next((i + 1 for i, item in enumerate(ranked) if item['song_id'] == gold), len(ranked) + 1)
334 by_type[qtype].append(rank)
335 persist_candidates(conn, query_id, retrieval_index_id, feature_set_id, ranked, topk)
336 persist_decision(conn, query_id, ranked)
337 if len(examples[qtype]) < 5:
338 examples[qtype].append({
339 'query_id': query_id,
340 'song_id': gold,
341 'rank': rank,
342 'top3': ranked[:3],
343 })
344
345 for qtype in ('7', '8', '16'):
346 ranks = by_type.get(qtype, [])
347 confusion_focus[qtype] = {
348 'query_type': int(qtype),
349 'metrics': compute_metrics(ranks, topk),
350 'interpretation': {
351 '7': 'light confusion / transformed query',
352 '8': 'harder confusion bucket',
353 '16': 'strong confusion or far-domain bucket',
354 }[qtype],
355 }
356
357 all_ranks = [r for ranks in by_type.values() for r in ranks]
358 return {
359 'backend': 'postgresql+pgvector-live',
360 'note': 'Reference embeddings are stored in schema v2; 24-d logical embeddings are zero-padded to vector(192) for physical storage.',
361 'overall': compute_metrics(all_ranks, topk),
362 'by_query_type': {qtype: compute_metrics(ranks, topk) for qtype, ranks in by_type.items()},
363 'confusion_focus': confusion_focus,
364 'examples': examples,
365 }
366
367
368 def main() -> None:
369 ap = argparse.ArgumentParser()
370 ap.add_argument('--dsn', required=True)
371 ap.add_argument('--schema', default='acr_test')
372 ap.add_argument('--schema-sql', default=str(DEFAULT_SCHEMA_SQL))
373 ap.add_argument('--reference-embeddings-jsonl', default=str(DEFAULT_REFERENCE))
374 ap.add_argument('--query-embeddings-jsonl', default=str(DEFAULT_QUERY))
375 ap.add_argument('--output', default=str(DEFAULT_OUTPUT))
376 ap.add_argument('--topn', type=int, default=20)
377 ap.add_argument('--topk', type=int, default=10)
378 ap.add_argument('--reset-schema', action='store_true')
379 args = ap.parse_args()
380
381 refs = load_jsonl(Path(args.reference_embeddings_jsonl))
382 queries = load_jsonl(Path(args.query_embeddings_jsonl))
383
384 with psycopg.connect(args.dsn, autocommit=True) as conn:
385 if args.reset_schema:
386 reset_schema(conn, args.schema)
387 else:
388 conn.execute(f'CREATE SCHEMA IF NOT EXISTS {args.schema};')
389 conn.execute(f'SET search_path TO {args.schema}, public;')
390 apply_schema(conn, Path(args.schema_sql))
391 model_id, feature_set_id, reference_set_id, retrieval_index_id = seed_registry(conn)
392 entities = ingest_references(conn, refs, feature_set_id, reference_set_id)
393 lineage_check = run_lineage_negative_test(conn, next(iter(entities.values())))
394 report = evaluate_live(conn, feature_set_id, retrieval_index_id, queries, args.topn, args.topk)
395 conn.execute('UPDATE retrieval_index_registry SET row_count = %s WHERE retrieval_index_id = %s;', (len(refs), retrieval_index_id))
396 counts = {
397 'canonical_song': conn.execute('SELECT count(*) FROM canonical_song;').fetchone()[0],
398 'work': conn.execute('SELECT count(*) FROM work;').fetchone()[0],
399 'recording': conn.execute('SELECT count(*) FROM recording;').fetchone()[0],
400 'recording_asset': conn.execute('SELECT count(*) FROM recording_asset;').fetchone()[0],
401 'audio_window': conn.execute('SELECT count(*) FROM audio_window;').fetchone()[0],
402 'audio_embedding': conn.execute('SELECT count(*) FROM audio_embedding;').fetchone()[0],
403 'retrieval_candidate': conn.execute('SELECT count(*) FROM retrieval_candidate;').fetchone()[0],
404 'match_decision': conn.execute('SELECT count(*) FROM match_decision;').fetchone()[0],
405 }
406
407 payload = {
408 'schema': args.schema,
409 'dsn_redacted': 'postgres://d2:***@127.0.0.1:5432/d2',
410 'input': {
411 'reference_embeddings_jsonl': args.reference_embeddings_jsonl,
412 'query_embeddings_jsonl': args.query_embeddings_jsonl,
413 'reference_count': len(refs),
414 'query_count': len(queries),
415 },
416 'registry': {
417 'model_id': model_id,
418 'feature_set_id': feature_set_id,
419 'reference_set_id': reference_set_id,
420 'retrieval_index_id': retrieval_index_id,
421 },
422 'table_counts': counts,
423 'lineage_negative_test': lineage_check,
424 'evaluation': report,
425 }
426
427 out = Path(args.output)
428 out.parent.mkdir(parents=True, exist_ok=True)
429 out.write_text(json.dumps(payload, ensure_ascii=False, indent=2), encoding='utf-8')
430 print(json.dumps(payload, ensure_ascii=False, indent=2))
431
432
433 if __name__ == '__main__':
434 main()
1 ## 2026-06-04 1 ## 2026-06-04
2 2
3 - 新增 [PostgreSQL 落库样例与 live 测试链路](./postgres_db_schema_samples.md)，补齐 `acr_pg_schema_v2.sql` 的真实落库样例、`pgvector` live 检索验证、lineage trigger 负例测试，以及当前召回/混淆结果解读。
4 - 新增 `acr-engine/scripts/live_pgvector_music20_eval.py`，支持对用户提供的 PostgreSQL 执行隔离 schema 建表、样例数据导入、`pgvector` live 检索、`retrieval_candidate` / `match_decision` 落表与评测报告生成。
5 - 新增 `acr-engine/data/pgvector_eval/music20/live_pgvector_report.json` 与 `songid_eval_report_fresh.json`，记录本轮 live PostgreSQL + pgvector 与 FAISS stand-in 的对齐结果：overall `top1=0.9091` / `top3=0.9545`，但 `type_7` 仍明显偏弱。
3 - 重写 [session-handoff 交接文档](./session-handoff.md)，将其从历史流水账收敛为“下次启动即用”的启动手册，明确当前稳定结论、推荐阅读顺序、已验证/未验证边界，以及下一步应从 PostgreSQL v2 schema 与 Phase-1 encoder-only 执行链开始推进。 6 - 重写 [session-handoff 交接文档](./session-handoff.md)，将其从历史流水账收敛为“下次启动即用”的启动手册，明确当前稳定结论、推荐阅读顺序、已验证/未验证边界，以及下一步应从 PostgreSQL v2 schema 与 Phase-1 encoder-only 执行链开始推进。
4 - 新增 [Phase-1 实施清单](./phase1-implementation-checklist.md)，把 encoder-only 路线拆成主数据、reference set、feature set、索引、评测的可执行阶段。 7 - 新增 [Phase-1 实施清单](./phase1-implementation-checklist.md)，把 encoder-only 路线拆成主数据、reference set、feature set、索引、评测的可执行阶段。
5 - 新增 [模型与 Feature Set 初始化手册](./model-feature-registry-bootstrap.md)，补齐 model_registry / feature_set_registry / reference_set_registry 的初始化约定与示例 SQL。 8 - 新增 [模型与 Feature Set 初始化手册](./model-feature-registry-bootstrap.md)，补齐 model_registry / feature_set_registry / reference_set_registry 的初始化约定与示例 SQL。
......
...@@ -54,6 +54,7 @@ ...@@ -54,6 +54,7 @@
54 | [acr-architecture.md](./acr-architecture.md) | 当前系统蓝图、角色分工、在线/离线链路 | 架构、开发、运维 | 54 | [acr-architecture.md](./acr-architecture.md) | 当前系统蓝图、角色分工、在线/离线链路 | 架构、开发、运维 |
55 | [sota-evolution-guide.md](./sota-evolution-guide.md) | SOTA 演进路径、Phase-1 encoder-only 方案、后续升级路线 | 架构、模型、检索 | 55 | [sota-evolution-guide.md](./sota-evolution-guide.md) | SOTA 演进路径、Phase-1 encoder-only 方案、后续升级路线 | 架构、模型、检索 |
56 | [postgresql-data-model.md](./postgresql-data-model.md) | PostgreSQL 数据字典、DDL 设计意图、流程图、查询路径 | 数据、后端、检索、平台 | 56 | [postgresql-data-model.md](./postgresql-data-model.md) | PostgreSQL 数据字典、DDL 设计意图、流程图、查询路径 | 数据、后端、检索、平台 |
57 | [postgres_db_schema_samples.md](./postgres_db_schema_samples.md) | PostgreSQL 实际落库样例、live pgvector 测试链路、召回/混淆结果 | 数据、后端、检索、平台 |
57 | [phase1-implementation-checklist.md](./phase1-implementation-checklist.md) | Phase-1 落地 checklist，按阶段拆执行项 | 架构、开发、平台 | 58 | [phase1-implementation-checklist.md](./phase1-implementation-checklist.md) | Phase-1 落地 checklist，按阶段拆执行项 | 架构、开发、平台 |
58 | [model-feature-registry-bootstrap.md](./model-feature-registry-bootstrap.md) | 模型、feature set、reference set 初始化手册 | 模型、检索、数据 | 59 | [model-feature-registry-bootstrap.md](./model-feature-registry-bootstrap.md) | 模型、feature set、reference set 初始化手册 | 模型、检索、数据 |
59 | [training-data-and-pgvector-guide.md](./training-data-and-pgvector-guide.md) | 当前训练/manifest/pgvector 原型链说明 | 开发、数据 | 60 | [training-data-and-pgvector-guide.md](./training-data-and-pgvector-guide.md) | 当前训练/manifest/pgvector 原型链说明 | 开发、数据 |
......
1 # PostgreSQL DB Schema Samples / 落库样例与 live 测试链路
2
3 > 更新：2026-06-04
4 > 目标：给后续开发一个**可直接照着做**的 PostgreSQL 落库样例，同时保留一次真实 `pgvector` live 测试的证据。
5
6 ---
7
8 ## 一页结论
9
10 这次已经在用户提供的 PostgreSQL 上完成了下面几件事：
11
12 1. **真实连接 PostgreSQL 成功**
13 - DSN：`postgres://d2:***@127.0.0.1:5432/d2`
14 - PostgreSQL：`17.5`
15 - 已确认扩展 `vector` 存在
16
17 2. **真实应用 schema v2 成功**
18 - 使用隔离 schema：`acr_test`
19 - DDL 来源：`acr-engine/sql/acr_pg_schema_v2.sql`
20 - 已成功创建主数据、registry、embedding、candidate、decision 等表
21
22 3. **真实插入了一套完整的样例数据链**
23 - `canonical_song -> work -> recording -> recording_asset -> audio_window`
24 - `model_registry -> feature_set_registry -> audio_embedding -> retrieval_index_registry`
25 - `reference_set_registry -> reference_set_member`
26
27 4. **真实跑通了一轮 PostgreSQL + pgvector 检索评测**
28 - 输入：`acr-engine/data/pgvector_eval/music20/*.jsonl`
29 - 输出：`acr-engine/data/pgvector_eval/music20/live_pgvector_report.json`
30 - live pgvector 指标和现有 FAISS stand-in 指标**一致**：
31 - overall `top1=0.9091`
32 - overall `top3=0.9545`
33 - `query_type=1`: `top1=1.0`
34 - `query_type=7`: `top1=0.0`, `top3=0.5`
35
36 5. **lineage trigger 已被验证有效**
37 - 脚本主动构造了一次错误 lineage 的 `audio_window`
38 - PostgreSQL 正确拒绝插入
39
40 ---
41
42 ## 本次使用的 live 测试资产
43
44 ### 数据库
45
46 | 项目 | 值 |
47 |---|---|
48 | Host | `127.0.0.1` |
49 | Port | `5432` |
50 | DB | `d2` |
51 | User | `d2` |
52 | PostgreSQL | `17.5` |
53 | 扩展 | `vector`, `pg_trgm`, `ltree`, `hstore` 等 |
54 | 本次测试 schema | `acr_test` |
55
56 ### 代码与产物
57
58 | 类型 | 路径 |
59 |---|---|
60 | 推荐 DDL | `acr-engine/sql/acr_pg_schema_v2.sql` |
61 | live 测试脚本 | `acr-engine/scripts/live_pgvector_music20_eval.py` |
62 | live 报告 | `acr-engine/data/pgvector_eval/music20/live_pgvector_report.json` |
63 | FAISS 对照报告 | `acr-engine/data/pgvector_eval/music20/songid_eval_report_fresh.json` |
64 | 历史对照报告 | `acr-engine/data/pgvector_eval/music20/songid_eval_report.json` |
65
66 ---
67
68 ## 这次实际落进去的数据链
69
70 ```mermaid
71 flowchart LR
72 A[reference_embeddings.jsonl] --> B[canonical_song]
73 B --> C[work]
74 C --> D[recording]
75 D --> E[recording_asset]
76 E --> F[audio_window]
77 F --> G[audio_embedding]
78 G --> H[audio_embedding_vector_192]
79
80 I[model_registry] --> J[feature_set_registry]
81 J --> G
82
83 K[reference_set_registry] --> L[reference_set_member]
84 D --> L
85
86 M[query_embeddings.jsonl] --> N[SQL pgvector search]
87 H --> N
88 N --> O[retrieval_candidate]
89 O --> P[match_decision]
90 ```
91
92 ---
93
94 ## 为什么这次 live 测试要把 24 维 embedding pad 到 192 维
95
96 当前 `schema v2` 里提供了：
97 - `audio_embedding_vector_192`
98 - `audio_embedding_vector_768`
99
100 而这次本地 `music20` 样例 embedding 是 **24 维 chroma 特征**。
101
102 所以本次 live 测试采用的策略是：
103
104 - **逻辑维度**：`24`
105 - **物理落盘维度**：`192`
106 - **做法**：后面补 `0`，写入 `vector(192)`
107
108 这样做的原因：
109 - 不需要临时改 schema
110 - 仍然可以验证 schema v2 + pgvector + retrieval 链路
111 - 对这批样例的余弦相似度排序不会产生方向性错误（所有向量都以同样方式补零）
112
113 这只是**验证链路**用法。
114
115 生产里应按真实 encoder 维度选择：
116 - `MERT` / `MuQ` 之类高维 embedding：直接落合适物理表
117 - 如果后续维度更多，建议继续扩成 `audio_embedding_vector_<dim>` 分桶策略
118
119 ---
120
121 ## 本次实际落盘样例
122
123 以下内容来自 `acr_test` schema 的真实查询结果。
124
125 ### 1. canonical_song
126
127 ```json
128 {"canonical_song_id":1,"biz_song_code":"100","title":"Song 100","primary_artist":"Artist 100","rights_status":"protected"}
129 {"canonical_song_id":2,"biz_song_code":"101","title":"Song 101","primary_artist":"Artist 101","rights_status":"protected"}
130 ```
131
132 ### 2. work
133
134 ```json
135 {"work_id":1,"canonical_song_id":1,"work_code":"work-100","work_title":"Song 100","composer":"Composer 100"}
136 {"work_id":2,"canonical_song_id":2,"work_code":"work-101","work_title":"Song 101","composer":"Composer 101"}
137 ```
138
139 ### 3. recording
140
141 ```json
142 {"recording_id":1,"work_id":1,"canonical_song_id":1,"recording_code":"rec-100","version_type":"master_reference","is_reference":true,"reference_priority":100}
143 {"recording_id":2,"work_id":2,"canonical_song_id":2,"recording_code":"rec-101","version_type":"master_reference","is_reference":true,"reference_priority":101}
144 ```
145
146 ### 4. recording_asset
147
148 ```json
149 {"asset_id":1,"recording_id":1,"asset_role":"reference_audio","storage_uri":"/workspace/downloads/100/type_11/93dfdeb0-7da5-42a8-9c71-cf12af57dd191650256918.wav","storage_scheme":"file","duration_sec":8.0,"ingest_status":"ready"}
150 {"asset_id":2,"recording_id":2,"asset_role":"reference_audio","storage_uri":"/workspace/downloads/101/type_11/83c0c07f-4f96-4ff4-998c-58db910f3cfa1650256915.wav","storage_scheme":"file","duration_sec":8.0,"ingest_status":"ready"}
151 ```
152
153 ### 5. audio_window
154
155 ```json
156 {"window_id":1,"asset_id":1,"recording_id":1,"work_id":1,"canonical_song_id":1,"window_index":0,"start_sec":0.0,"end_sec":8.0,"segment_role":"reference","segment_type":"full_clip"}
157 {"window_id":2,"asset_id":2,"recording_id":2,"work_id":2,"canonical_song_id":2,"window_index":0,"start_sec":0.0,"end_sec":8.0,"segment_role":"reference","segment_type":"full_clip"}
158 ```
159
160 ### 6. model_registry / feature_set_registry
161
162 ```json
163 {"model_id":1,"model_name":"local_chroma24","model_family":"chroma_baseline","model_version":"v1","output_embedding_dim":24,"default_window_sec":8.0}
164 {"feature_set_id":1,"model_id":1,"feature_name":"chroma24_songid_eval","embedding_dim":24,"distance_metric":"cosine","feature_schema_version":"v1"}
165 ```
166
167 ### 7. audio_embedding
168
169 ```json
170 {"embedding_id":1,"feature_set_id":1,"asset_id":1,"window_id":1,"recording_id":1,"canonical_song_id":1,"embedding_storage_mode":"pgvector_inline_192_padded","is_indexed":true}
171 {"embedding_id":2,"feature_set_id":1,"asset_id":2,"window_id":2,"recording_id":2,"canonical_song_id":2,"embedding_storage_mode":"pgvector_inline_192_padded","is_indexed":true}
172 ```
173
174 ### 8. reference_set_registry / retrieval_index_registry
175
176 ```json
177 {"reference_set_id":1,"set_name":"music20_live_reference","encoder_scope":"local_chroma24","status":"active"}
178 {"retrieval_index_id":1,"feature_set_id":1,"index_name":"music20_live_pgvector_hnsw","index_backend":"pgvector","index_type":"hnsw_cosine","row_count":20,"index_status":"active"}
179 ```
180
181 ### 9. retrieval_candidate / match_decision
182
183 ```json
184 {"retrieval_candidate_id":1,"query_id":"music20-q0000-t1-song100","source_lane":"semantic","candidate_level":"canonical_song","candidate_id":1,"raw_score":0.99998549,"normalized_score":0.90998694,"rank_no":1}
185 {"retrieval_candidate_id":2,"query_id":"music20-q0000-t1-song100","source_lane":"semantic","candidate_level":"canonical_song","candidate_id":17,"raw_score":0.9527432,"normalized_score":0.86746888,"rank_no":2}
186 {"match_decision_id":1,"query_id":"music20-q0000-t1-song100","canonical_song_id":1,"decision_status":"matched","decision_score":0.90998694}
187 ```
188
189 ---
190
191 ## 本次 live 测试的表规模
192
193 | 表 | 行数 |
194 |---|---:|
195 | `canonical_song` | 20 |
196 | `work` | 20 |
197 | `recording` | 20 |
198 | `recording_asset` | 20 |
199 | `audio_window` | 20 |
200 | `audio_embedding` | 20 |
201 | `retrieval_candidate` | 220 |
202 | `match_decision` | 22 |
203
204 说明：
205 - 20 条 reference song
206 - 22 条 query
207 - 每条 query 写入 top10 candidate，因此 `22 * 10 = 220`
208
209 ---
210
211 ## 本次测试链路与逻辑
212
213 ### A. schema / 数据完整性测试
214
215 1. 连接 PostgreSQL
216 2. 创建隔离 schema：`acr_test`
217 3. 执行 `acr_pg_schema_v2.sql`
218 4. 初始化：
219 - `model_registry`
220 - `feature_set_registry`
221 - `reference_set_registry`
222 - `retrieval_index_registry`
223 5. 导入 20 条 reference 样例
224 6. 验证表计数是否正确
225 7. 主动插入一条错误 lineage 的 `audio_window`
226 8. 预期 PostgreSQL trigger 拒绝该写入
227
228 ### B. live 检索评测测试
229
230 1. 从 `reference_embeddings.jsonl` 读 20 条 reference embedding
231 2. 写入 `audio_embedding` + `audio_embedding_vector_192`
232 3. 从 `query_embeddings.jsonl` 读 22 条 query embedding
233 4. 每条 query 用 SQL 执行 `pgvector cosine` 检索
234 5. 在应用层做 song-level aggregation：
235 - `max_sim`
236 - `top3_avg`
237 - `vote`
238 - `combined = 0.6 * max_sim + 0.3 * top3_avg + 0.1 * vote_factor`
239 6. 将 top10 候选落表到 `retrieval_candidate`
240 7. 将 top1 决策落表到 `match_decision`
241 8. 计算：
242 - overall `top1/top3/top10/mrr`
243 - `by_query_type`
244 - `confusion_focus`
245
246 ### C. confusion test 口径
247
248 当前这次 live 样例里只实际包含：
249 - `type_1`
250 - `type_7`
251
252 因此：
253 - `type_7` 可以作为 **当前 live confusion check**
254 - `type_8 / type_16` 这次 live JSONL 没覆盖到，只能结合历史业务样本结果一起看
255
256 ---
257
258 ## live pgvector 结果
259
260 ### 1. overall
261
262 | 指标 | 值 |
263 |---|---:|
264 | query 数 | 22 |
265 | top1 | `0.9091` |
266 | top3 | `0.9545` |
267 | top10 | `0.9545` |
268 | MRR | `0.9343` |
269 | mean rank | `1.8182` |
270
271 ### 2. by query type
272
273 | query_type | count | top1 | top3 | top10 | 解释 |
274 |---|---:|---:|---:|---:|---|
275 | `1` | 20 | `1.0` | `1.0` | `1.0` | clean / near-clean |
276 | `7` | 2 | `0.0` | `0.5` | `0.5` | 当前 live confusion 样例 |
277 | `8` | 0 | N/A | N/A | N/A | 本次 live JSONL 未覆盖 |
278 | `16` | 0 | N/A | N/A | N/A | 本次 live JSONL 未覆盖 |
279
280 ### 3. 和现有 FAISS stand-in 的一致性
281
282 | 路径 | overall top1 | overall top3 | type_1 top1 | type_7 top1 | type_7 top3 |
283 |---|---:|---:|---:|---:|---:|
284 | live PostgreSQL + pgvector | `0.9091` | `0.9545` | `1.0` | `0.0` | `0.5` |
285 | FAISS stand-in | `0.9091` | `0.9545` | `1.0` | `0.0` | `0.5` |
286
287 结论：
288
289 > 当前 `acr_test` 上的 live pgvector 路径，已经和现有 stand-in 检索逻辑对齐。
290 > 问题不在“PostgreSQL 落盘导致召回变坏”，而在当前样例 embedding 对混淆类 query 本身就不够强。
291
292 ---
293
294 ## 混淆测试补充视图
295
296 ### 1. 当前 live 样例视图
297
298 | query_type | 数据来源 | top1 | top3 | 结论 |
299 |---|---|---:|---:|---|
300 | `7` | `live_pgvector_report.json` | `0.0` | `0.5` | 已明显偏弱 |
301
302 ### 2. 历史本地 20-song 小样本视图
303
304 来自：`acr-engine/data/local_eval/music20_summary.json`
305
306 | query_type | top1 | top3 |
307 |---|---:|---:|
308 | `1` | `1.0` | `1.0` |
309 | `7` | `0.45` | `0.65` |
310 | `8` | `0.4667` | `0.7333` |
311 | `16` | `0.4167` | `0.4167` |
312
313 说明：
314 - 这是**本地小样本 chroma/FAISS sanity flow** 的结果
315 - 它比当前 live JSONL 的 type_7 好，是因为样本构成不同
316 - 不能把这个结果直接当作生产效果，但可以当作“当前特征在小样本内并非完全不可用”的旁证
317
318 ### 3. 历史业务语料 voice correctness 视图
319
320 | query_type | 文件 | top1 | top3 | 结论 |
321 |---|---|---:|---:|---|
322 | `7` | `voice_workspace20_type7_eval.json` | `0.0` | `0.05` | 极弱 |
323 | `8` | `voice_workspace20_type8_eval.json` | `0.0` | `0.0` | 极弱 |
324 | `16` | `voice_workspace20_type16_eval.json` | `0.0` | `0.0` | 极弱 |
325
326 结论：
327
328 > 只要 query 进入更真实、更混淆的业务样本，当前这条 baseline 仍然远远不够。
329 > PostgreSQL 落库没问题，真正的问题还是 **embedding lane 对 hard case 的判别力不足**。
330
331 ---
332
333 ## 这次验证了什么，没验证什么
334
335 ### 已验证
336
337 - PostgreSQL 真实连通可用
338 - `vector` 扩展可用
339 - schema v2 可以真实 apply
340 - main lineage trigger 可以真实拦截坏数据
341 - 样例数据链可以按 `song -> work -> recording -> asset -> window -> embedding` 落盘
342 - live pgvector 检索和现有 stand-in 逻辑一致
343 - `retrieval_candidate` / `match_decision` 可以真实承载在线结果
344
345 ### 未验证
346
347 - 还没把 `MERT` / `MuQ` 真正接进这套 live 路径
348 - 这次 live 样例没有覆盖 `type_8 / type_16` 的 JSONL embedding
349 - 这次只验证了 20-song 级别，不代表 30w song 的索引性能
350 - 还没做多 recording / 多 version / cover lane 的聚合测试
351
352 ---
353
354 ## 推荐的下一步
355
356 ### 路线 1：继续做 PostgreSQL 工程化
357
358 1. 把 `live_pgvector_music20_eval.py` 泛化成：
359 - 可导入任意 manifest/reference set
360 - 可选择 encoder / feature set
361 - 可直接生成 `retrieval_candidate` / `match_decision` 报告
362 2. 增加：
363 - `audio_embedding_vector_1024` / 其他常见维度表
364 - bulk COPY / batched insert
365 - HNSW 参数管理
366
367 ### 路线 2：继续做混淆类效果验证
368
369 1. 构造真正覆盖 `type_8 / type_16` 的 query embedding JSONL
370 2. 用同一条 live script 重跑 PostgreSQL 评测
371 3. 对比：
372 - `Chromaprint only`
373 - `semantic only`
374 - `fusion`
375 4. 输出 confusion bucket 报告
376
377 ### 路线 3：切到 Phase-1 encoder-only 主线
378
379 1. 保留当前 PostgreSQL 结构不变
380 2. 将 `local_chroma24` 替换成：
381 - `MERT-v1-95M`
382 - `MuQ`
383 3. 继续复用：
384 - `model_registry`
385 - `feature_set_registry`
386 - `reference_set_registry`
387 - `retrieval_index_registry`
388 4. 重新测：
389 - clean
390 - type_7
391 - type_8
392 - type_16
393 - 业务 voice bucket
394
395 ---
396
397 ## 复现命令
398
399 ### 1. live PostgreSQL + pgvector 测试
400
401 ```bash
402 cd /workspace/acr-engine
403 /usr/local/miniconda3/bin/python scripts/live_pgvector_music20_eval.py \
404 --dsn 'postgres://d2:d2pass@127.0.0.1:5432/d2' \
405 --schema acr_test \
406 --reset-schema \
407 --output data/pgvector_eval/music20/live_pgvector_report.json
408 ```
409
410 ### 2. FAISS stand-in 对照测试
411
412 ```bash
413 cd /workspace/acr-engine
414 /usr/local/miniconda3/bin/python scripts/evaluate_songid_pgvector_path.py \
415 --reference-embeddings-jsonl data/pgvector_eval/music20/reference_embeddings.jsonl \
416 --query-embeddings-jsonl data/pgvector_eval/music20/query_embeddings.jsonl \
417 --output data/pgvector_eval/music20/songid_eval_report_fresh.json
418 ```
419
420 ---
421
422 ## 一句话结论
423
424 > PostgreSQL 这条路已经可以真实落 schema、落样例、落 candidate、落 decision，也能真实跑 pgvector 检索。
425 > 当前最大的短板不再是“怎么存”，而是 **当前 baseline embedding 对混淆 query 的召回仍然明显不够**。