Skip to content

wanghai-tech / hikoon-ACR

Go to a project
Toggle navigation
Toggle navigation pinning
Commit 87959076 879590762a309cd1430042677d51e90bdf53c68b by cnb.bofCdSsphPA
Options

Generate release artifacts for the open-dataset smoke path 

Constraint: Open-dataset workflow needed the same reporting/release outputs as the synthetic baseline to be operationally useful
Rejected: Treat open-data smoke as a one-off test only | Leaves no reusable benchmark or release documentation trail
Confidence: high
Scope-risk: narrow
Directive: Every future real-dataset smoke run should emit eval JSON plus artifact bundle in the same directory
Tested: /usr/local/miniconda3/bin/python scripts/generate_artifacts.py --eval-json reports/open-smoke-fixed/fma/eval.json --config-json reports/open-smoke-fixed/fma/config.json --output-dir reports/open-smoke-fixed/fma --model-version open-smoke-fixed --data-version synthetic_as_open_fixed_fma
Not-tested: Artifact generation on a larger real downloaded corpus with multiple hard-case buckets
1 parent dc9ef1b8
Inline Side-by-side
Showing 7 changed files with 166 additions and 0 deletions
{
"generated_at": "2026-06-02T05:01:12Z",
"model_version": "open-smoke-fixed",
"data_version": "synthetic_as_open_fixed_fma",
"files": {
"benchmark_report": "reports/open-smoke-fixed/fma/benchmark-report.md",
"model_card": "reports/open-smoke-fixed/fma/model-card.md",
"release_checklist": "reports/open-smoke-fixed/fma/release-checklist.md"
}
}
\ No newline at end of file
# Benchmark Report
## 一页结论
- 模型版本：open-smoke-fixed
- 数据版本：synthetic_as_open_fixed_fma
- 核心结论：top1=1.0 top5=1.0
- 是否通过上线门禁：TBD
## 1. 评测范围图
```mermaid
flowchart LR
A[open-smoke-fixed] --> B[synthetic_as_open_fixed_fma]
A --> C[Scenario Buckets]
A --> D[Latency / Ops]
```
## 2. 指标表
| Bucket | top1 | top5 | MRR | FAR | Notes |
|---|---:|---:|---:|---:|---|
| clean | 1.0 | 1.0 | | | |
## 3. 文字分析
- 最强项：clean/augmented buckets if present
- 最弱项：see hard-case summary
- 与上一版本对比：TBD
## 4. 细节附录
- 原始 JSON 报告：embedded source
## Sources
- docs/industrial-benchmark-spec.md
{
"model": {
"embed_dim": 192,
"channels": 512,
"n_mels": 128,
"use_band_split": true
},
"data": {
"source_dataset": "fma",
"manifests_dir": "data/external_ingested/synthetic_as_open_fixed/fma/manifests",
"query_duration": 5.0
},
"run": {
"train_command": "/usr/local/miniconda3/bin/python train.py --data data/external_ingested/synthetic_as_open_fixed/fma/manifests --output data/models_open_smoke_fixed --device cpu --epochs 1 --batch-size 2",
"index_command": "/usr/local/miniconda3/bin/python run_demo.py build-index --data data/external_ingested/synthetic_as_open_fixed/fma/manifests --model data/models_open_smoke_fixed/best_model.pt --output data/index_open_smoke_fixed --device cpu",
"eval_command": "/usr/local/miniconda3/bin/python evaluate.py --data data/external_ingested/synthetic_as_open_fixed/fma/manifests --model data/models_open_smoke_fixed/best_model.pt --index-prefix data/index_open_smoke_fixed/reference --split test --device cpu --fast-eval --output-json reports/open-smoke-fixed/fma/eval.json"
}
}
# Model Card
## 一页结论
- 模型名称：ACR Hybrid Encoder
- 版本：open-smoke-fixed
- 适用场景：music ACR prototype / retrieval
- 不适用场景：未经白名单数据验证的生产商用全量上线
## 1. 模型结构图
```mermaid
flowchart LR
A[Input Audio] --> B[128 Mel + BandSplit]
B --> C[Encoder]
C --> D[Embedding]
D --> E[Hybrid Retrieval]
```
## 2. 关键信息表
| 项 | 内容 |
|---|---|
| embed_dim | 192 |
| channels | 512 |
| n_mels | 128 |
| use_band_split | True |
| benchmark report | reports/open-smoke-fixed/fma/benchmark-report.md |
## 3. 文字说明
- 训练方式：retrieval-oriented pair training
- 模型限制：hard-case accuracy still evolving
- 风险提示：requires whitelist-reviewed datasets for commercial deployment
## 4. 细节附录
- config embedded from source JSON
## Sources
- docs/dataset-spec.md
- docs/benchmark-report-template.md
# Release Checklist
## 一页结论
发布前必须同时满足：质量通过、合规通过、服务通过、文档齐全。
## 1. 发布门禁图
```mermaid
flowchart TD
A[open-smoke-fixed] --> B[Benchmark Pass]
A --> C[License Review Pass]
A --> D[Service Smoke Pass]
A --> E[Docs Complete]
```
## 2. Checklist 表
| 项目 | 状态 |
|---|---|
| benchmark report 已生成 | yes |
| model card 已生成 | yes |
| license registry 已更新 | pending |
| service smoke test 通过 | yes |
| dataset whitelist 已确认 | pending |
| changelog 已更新 | pending |
## 3. 文字说明
- 当前用于工程治理与预发布检查，不代表已满足商用法律门槛。
## 4. 细节附录
- benchmark 报告路径：reports/open-smoke-fixed/fma/benchmark-report.md
- model card 路径：reports/open-smoke-fixed/fma/model-card.md
## Sources
- docs/dataset-sources-and-licensing.md
- docs/industrial-benchmark-spec.md
......@@ -93,6 +93,28 @@
- 开放数据接入链路现在已经完整闭环
- 真实 FMA / MTG-Jamendo 本地目录接入时，可直接复用同一流程
### Stage: 开放数据 smoke 发布制品
完成项：
- 为 `reports/open-smoke-fixed/fma/` 补充 `config.json`
- 用 `scripts/generate_artifacts.py` 生成开放数据 smoke 的：
- benchmark report
- model card
- release checklist
- artifact manifest
验证结果：
- `/usr/local/miniconda3/bin/python scripts/generate_artifacts.py --eval-json reports/open-smoke-fixed/fma/eval.json --config-json reports/open-smoke-fixed/fma/config.json --output-dir reports/open-smoke-fixed/fma --model-version open-smoke-fixed --data-version synthetic_as_open_fixed_fma` 成功
- 产物存在：
- `reports/open-smoke-fixed/fma/benchmark-report.md`
- `reports/open-smoke-fixed/fma/model-card.md`
- `reports/open-smoke-fixed/fma/release-checklist.md`
- `reports/open-smoke-fixed/fma/artifact-manifest.json`
结论：
- 现在开放数据链路已经不只是“能跑”，还具备基础发布/汇报产物
- 下一步替换成真实 FMA / MTG-Jamendo 本地目录后，可直接复用同一 release 流程
### Stage: confused 定向优化 v6（sample-level weighting）
完成项：
......
......@@ -10,6 +10,7 @@
2. **prepare-local**
3. **validate-local**
4. 再进入训练与评估
5. 生成 benchmark / model card / release artifacts
---
......@@ -22,6 +23,7 @@ flowchart LR
C --> D[validate-local]
D --> E[train.py]
D --> F[evaluate.py]
F --> G[artifact bundle]
```
---
......@@ -35,6 +37,7 @@ flowchart LR
| 生成清单 | [`src/data/external_adapters.py`](../acr-engine/src/data/external_adapters.py) `prepare-local ...` | 产出 train/test/catalog |
| 训练前校验 | [`src/data/external_adapters.py`](../acr-engine/src/data/external_adapters.py) `validate-local ...` | 确认结构正确 |
| 训练 smoke | [`train.py`](../acr-engine/train.py) `--data ... --dry-run` | 验证 manifests 可直接进入训练 |
| 发布制品 | [`scripts/generate_artifacts.py`](../acr-engine/scripts/generate_artifacts.py) | 生成 benchmark/model-card/release-checklist |
---
......@@ -49,6 +52,7 @@ flowchart LR
/usr/local/miniconda3/bin/python train.py --data data/external_ingested/fma/manifests --output data/models_fma_smoke --device cpu --epochs 1 --batch-size 2 --dry-run
/usr/local/miniconda3/bin/python run_demo.py build-index --data data/external_ingested/fma/manifests --model data/models_fma_smoke/best_model.pt --output data/index_fma_smoke --device cpu
/usr/local/miniconda3/bin/python evaluate.py --data data/external_ingested/fma/manifests --model data/models_fma_smoke/best_model.pt --index-prefix data/index_fma_smoke/reference --split test --device cpu --fast-eval --output-json reports/fma-smoke/eval.json
/usr/local/miniconda3/bin/python scripts/generate_artifacts.py --eval-json reports/fma-smoke/eval.json --config-json reports/fma-smoke/config.json --output-dir reports/fma-smoke --model-version fma-smoke --data-version fma_local
```
### 3.2 多目录比较
......@@ -87,6 +91,10 @@ flowchart LR
- `build-index + evaluate`：
- `top1=1.0`
- `topk=1.0`
- `generate_artifacts`：
- `benchmark-report.md`
- `model-card.md`
- `release-checklist.md`
---
......