Preserve restart-safe handoff for the capped FMA benchmark
Record the latest delivered benchmark evidence, active work-root, partial results, and exact resume commands so a new session can continue without rediscovering context. Constraint: User requested immediate delivery artifacts before the long benchmark fully finishes Rejected: Wait for the entire cap16 benchmark to finish before handing off | Would delay delivery and risk losing resumable context Confidence: high Scope-risk: narrow Directive: Update the handoff again once high_energy and repeated_section_aware finish on cap16 Tested: Verified partial eval files for hybrid and beat_aware; verified active cap16 benchmark processes; verified session-handoff contains resume commands and partial scores Not-tested: Final multi-strategy cap16 ranking because high_energy and repeated_section_aware are still running
Showing
2 changed files
with
141 additions
and
0 deletions
| ... | @@ -2,6 +2,32 @@ | ... | @@ -2,6 +2,32 @@ |
| 2 | 2 | ||
| 3 | ## 2026-06-02 | 3 | ## 2026-06-02 |
| 4 | 4 | ||
| 5 | ### Stage: 交付当前切片 benchmark 续跑 handoff | ||
| 6 | |||
| 7 | 完成项: | ||
| 8 | - 更新 [session-handoff.md](./session-handoff.md) | ||
| 9 | - 记录最新关键提交: | ||
| 10 | - `6232787` | ||
| 11 | - `f04a314` | ||
| 12 | - `d7a0894` | ||
| 13 | - `b6cdf66` | ||
| 14 | - 记录中规模真实 FMA capped benchmark 的续跑入口 | ||
| 15 | - 写入当前已拿到的 partial result: | ||
| 16 | - `hybrid`: `num_queries=12`, `top1=1.0`, `topk=1.0` | ||
| 17 | - `beat_aware`: `num_queries=12`, `top1=0.9167`, `topk=1.0` | ||
| 18 | |||
| 19 | 验证结果: | ||
| 20 | - 当前进程确认: | ||
| 21 | - `scripts/ab_smoke_segmentation.py ... --work-root /tmp/ab_smoke_seg_cap16` | ||
| 22 | - `high_energy` 策略仍在进行中 | ||
| 23 | - 已落盘评测文件确认: | ||
| 24 | - `/tmp/ab_smoke_seg_cap16/hybrid/fma_reports_smoke/eval.json` | ||
| 25 | - `/tmp/ab_smoke_seg_cap16/beat_aware/fma_reports_smoke/eval.json` | ||
| 26 | |||
| 27 | 结论: | ||
| 28 | - 当前 session 即使立即中断,也已经具备可恢复的续跑交接材料 | ||
| 29 | - 新 session 可以直接从 `/tmp/ab_smoke_seg_cap16` 和 `docs/session-handoff.md` 接手,而不需要重新梳理上下文 | ||
| 30 | |||
| 5 | ### Stage: 为切片策略评测补齐公平 query cap,并澄清 librosa 分段现状 | 31 | ### Stage: 为切片策略评测补齐公平 query cap,并澄清 librosa 分段现状 |
| 6 | 32 | ||
| 7 | 完成项: | 33 | 完成项: | ... | ... |
| ... | @@ -254,10 +254,125 @@ | ... | @@ -254,10 +254,125 @@ |
| 254 | 254 | ||
| 255 | 近几次关键提交建议优先看: | 255 | 近几次关键提交建议优先看: |
| 256 | 256 | ||
| 257 | - `6232787` Make segmentation strategy benchmarks comparable under fixed query budgets | ||
| 258 | - `f04a314` Benchmark real FMA segmentation strategies on a capped smoke subset | ||
| 259 | - `d7a0894` Favor beat-aligned candidate segments for music ACR training/query generation | ||
| 260 | - `b6cdf66` Add high-energy and onset-aware music segment selection | ||
| 257 | - `d221852` Add explicit drop zones for real open-music corpora | 261 | - `d221852` Add explicit drop zones for real open-music corpora |
| 258 | - `eee15ac` Automate the full open-dataset smoke workflow behind one command | 262 | - `eee15ac` Automate the full open-dataset smoke workflow behind one command |
| 259 | - `8795907` Generate release artifacts for the open-dataset smoke path | 263 | - `8795907` Generate release artifacts for the open-dataset smoke path |
| 260 | - `dc9ef1b` Close the open-dataset smoke loop through evaluation | 264 | - `dc9ef1b` Close the open-dataset smoke loop through evaluation |
| 265 | |||
| 266 | --- | ||
| 267 | |||
| 268 | ## 9. 最新真实数据切片 benchmark 状态(重启后优先续跑这里) | ||
| 269 | |||
| 270 | ### 已完成的最新事实 | ||
| 271 | |||
| 272 | 当前项目已经不是“只有 random 切片”: | ||
| 273 | |||
| 274 | - 训练/外部 query 生成支持: | ||
| 275 | - `random` | ||
| 276 | - `silence_aware` | ||
| 277 | - `high_energy` | ||
| 278 | - `onset_aware` | ||
| 279 | - `beat_aware` | ||
| 280 | - `repeated_section_aware` | ||
| 281 | - `hybrid` | ||
| 282 | - 已接入的 `librosa` 逻辑: | ||
| 283 | - `effects.split` | ||
| 284 | - `onset.onset_detect` | ||
| 285 | - `beat.beat_track` | ||
| 286 | - `feature.chroma_cqt` | ||
| 287 | |||
| 288 | ### 已完成的小规模 capped 验证 | ||
| 289 | |||
| 290 | 命令: | ||
| 291 | |||
| 292 | ```bash | ||
| 293 | cd /workspace/acr-engine | ||
| 294 | /usr/local/miniconda3/bin/python scripts/ab_smoke_segmentation.py \ | ||
| 295 | --dataset fma \ | ||
| 296 | --input-dir data/raw/fma_small_audio \ | ||
| 297 | --work-root /tmp/ab_smoke_cap \ | ||
| 298 | --subset-size 6 \ | ||
| 299 | --query-duration 8 \ | ||
| 300 | --train-epochs 1 \ | ||
| 301 | --batch-size 2 \ | ||
| 302 | --device cpu \ | ||
| 303 | --strategies hybrid \ | ||
| 304 | --max-test-queries 5 \ | ||
| 305 | --output-json /tmp/ab_smoke_cap/report.json | ||
| 306 | ``` | ||
| 307 | |||
| 308 | 结果: | ||
| 309 | - `max_test_queries = 5` | ||
| 310 | - `num_queries = 5` | ||
| 311 | - `top1 = 1.0` | ||
| 312 | - `topk = 1.0` | ||
| 313 | |||
| 314 | ### 正在进行的中规模 capped FMA benchmark | ||
| 315 | |||
| 316 | 命令: | ||
| 317 | |||
| 318 | ```bash | ||
| 319 | cd /workspace/acr-engine | ||
| 320 | /usr/local/miniconda3/bin/python scripts/ab_smoke_segmentation.py \ | ||
| 321 | --dataset fma \ | ||
| 322 | --input-dir data/raw/fma_small_audio \ | ||
| 323 | --work-root /tmp/ab_smoke_seg_cap16 \ | ||
| 324 | --subset-size 16 \ | ||
| 325 | --query-duration 8 \ | ||
| 326 | --train-epochs 1 \ | ||
| 327 | --batch-size 2 \ | ||
| 328 | --device cpu \ | ||
| 329 | --strategies hybrid beat_aware high_energy repeated_section_aware \ | ||
| 330 | --max-test-queries 12 \ | ||
| 331 | --output-json /tmp/ab_smoke_seg_cap16/report.json | ||
| 332 | ``` | ||
| 333 | |||
| 334 | 在本次交接时,已拿到的 partial result: | ||
| 335 | |||
| 336 | | 策略 | num_queries | top1 | topk | 状态 | | ||
| 337 | |---|---:|---:|---:|---| | ||
| 338 | | `hybrid` | 12 | 1.0 | 1.0 | 已完成 | | ||
| 339 | | `beat_aware` | 12 | 0.9167 | 1.0 | 已完成 | | ||
| 340 | | `high_energy` | - | - | - | 进行中 | | ||
| 341 | | `repeated_section_aware` | - | - | - | 未开始/未完成 | | ||
| 342 | |||
| 343 | ### 重启后第一优先动作 | ||
| 344 | |||
| 345 | 1. 先检查: | ||
| 346 | |||
| 347 | ```bash | ||
| 348 | pgrep -af 'ab_smoke_seg_cap16|external_adapters.py smoke-local fma /tmp/ab_smoke_seg_cap16|evaluate.py --data /tmp/ab_smoke_seg_cap16|run_demo.py build-index --data /tmp/ab_smoke_seg_cap16' | ||
| 349 | ``` | ||
| 350 | |||
| 351 | 2. 如果还在跑,等待 `/tmp/ab_smoke_seg_cap16/report.json` | ||
| 352 | 3. 如果中断: | ||
| 353 | - 保留已有 `/tmp/ab_smoke_seg_cap16/hybrid`、`/tmp/ab_smoke_seg_cap16/beat_aware` 结果作人工记录 | ||
| 354 | - 重新跑剩余策略,或单独跑: | ||
| 355 | |||
| 356 | ```bash | ||
| 357 | cd /workspace/acr-engine | ||
| 358 | /usr/local/miniconda3/bin/python src/data/external_adapters.py smoke-local \ | ||
| 359 | fma /tmp/ab_smoke_seg_cap16/subset_audio \ | ||
| 360 | --output-root /tmp/ab_smoke_seg_cap16/high_energy \ | ||
| 361 | --eval-ratio 0.2 \ | ||
| 362 | --query-duration 8.0 \ | ||
| 363 | --query-strategy high_energy \ | ||
| 364 | --segment-strategy high_energy \ | ||
| 365 | --train-epochs 1 \ | ||
| 366 | --batch-size 2 \ | ||
| 367 | --device cpu \ | ||
| 368 | --max-test-queries 12 \ | ||
| 369 | --seed 42 | ||
| 370 | ``` | ||
| 371 | |||
| 372 | 4. 完整结果出来后: | ||
| 373 | - 更新 [open-dataset-workflow.md](./open-dataset-workflow.md) | ||
| 374 | - 更新 [CHANGELOG.md](./CHANGELOG.md) | ||
| 375 | - commit + push | ||
| 261 | - `b766c74` Make open-dataset manifests trainable end to end | 376 | - `b766c74` Make open-dataset manifests trainable end to end |
| 262 | - `fa23144` Add a single-page open dataset workflow for training prep | 377 | - `fa23144` Add a single-page open dataset workflow for training prep |
| 263 | - `af33be3` Condense docs and add manifest validation before training | 378 | - `af33be3` Condense docs and add manifest validation before training | ... | ... |
-
Please register or sign in to post a comment