Let future sessions wait on archive completion without manual polling
Constraint: The real FMA archive still needs time, but once it finishes the workflow should transition into extraction and readiness with minimal operator attention Rejected: Keep completion detection as an entirely manual loop | Wastes attention and slows handoff at the exact moment the archive becomes useful Confidence: high Scope-risk: narrow Directive: Use wait_for_fma_and_prepare.py as the passive bridge from long-running download to active dataset onboarding whenever unattended waiting is acceptable Tested: /usr/local/miniconda3/bin/python -m py_compile acr-engine/scripts/wait_for_fma_and_prepare.py; /usr/local/miniconda3/bin/python acr-engine/scripts/wait_for_fma_and_prepare.py --interval 2 --max-cycles 2 Not-tested: The completed-path handoff into extraction remains pending full archive completion
Showing
4 changed files
with
80 additions
and
0 deletions
| 1 | #!/usr/bin/env python3 | ||
| 2 | """Wait for the FMA archive to finish, then run post-download readiness.""" | ||
| 3 | |||
| 4 | from __future__ import annotations | ||
| 5 | |||
| 6 | import argparse | ||
| 7 | import json | ||
| 8 | import subprocess | ||
| 9 | import time | ||
| 10 | |||
| 11 | PYTHON = "/usr/local/miniconda3/bin/python" | ||
| 12 | INSPECT = [PYTHON, "scripts/prepare_fma_archive.py", "inspect"] | ||
| 13 | POST = [PYTHON, "scripts/fma_postdownload_ready.py"] | ||
| 14 | |||
| 15 | |||
| 16 | def inspect() -> dict: | ||
| 17 | return json.loads(subprocess.check_output(INSPECT, text=True)) | ||
| 18 | |||
| 19 | |||
| 20 | def main(): | ||
| 21 | parser = argparse.ArgumentParser() | ||
| 22 | parser.add_argument("--interval", type=float, default=30.0) | ||
| 23 | parser.add_argument("--max-cycles", type=int, default=3) | ||
| 24 | args = parser.parse_args() | ||
| 25 | |||
| 26 | snapshots = [] | ||
| 27 | for _ in range(args.max_cycles): | ||
| 28 | snap = inspect() | ||
| 29 | snapshots.append(snap) | ||
| 30 | if snap.get("archive_size", 0) >= snap.get("archive_bytes_expected", 0): | ||
| 31 | result = json.loads(subprocess.check_output(POST, text=True)) | ||
| 32 | print(json.dumps({"status": "completed", "snapshots": snapshots, "postdownload": result}, indent=2, ensure_ascii=False)) | ||
| 33 | return | ||
| 34 | time.sleep(args.interval) | ||
| 35 | |||
| 36 | print(json.dumps({"status": "waiting", "snapshots": snapshots}, indent=2, ensure_ascii=False)) | ||
| 37 | |||
| 38 | |||
| 39 | if __name__ == "__main__": | ||
| 40 | main() |
| ... | @@ -237,6 +237,30 @@ | ... | @@ -237,6 +237,30 @@ |
| 237 | 237 | ||
| 238 | 238 | ||
| 239 | 239 | ||
| 240 | |||
| 241 | ### Stage: FMA 完成前等待并自动切换 | ||
| 242 | |||
| 243 | 完成项: | ||
| 244 | - 新增 [acr-engine/scripts/wait_for_fma_and_prepare.py](../acr-engine/scripts/wait_for_fma_and_prepare.py) | ||
| 245 | - 支持: | ||
| 246 | - 周期性检查 FMA archive 是否已完整 | ||
| 247 | - 完整后自动调用 `fma_postdownload_ready.py` | ||
| 248 | - 未完成时返回结构化 `waiting` 快照 | ||
| 249 | - 将脚本接入 [docs/open-dataset-workflow.md](./open-dataset-workflow.md) 与 [docs/session-handoff.md](./session-handoff.md) | ||
| 250 | |||
| 251 | 验证结果: | ||
| 252 | - `/usr/local/miniconda3/bin/python -m py_compile scripts/wait_for_fma_and_prepare.py` 成功 | ||
| 253 | - `/usr/local/miniconda3/bin/python scripts/wait_for_fma_and_prepare.py --interval 2 --max-cycles 2` 成功 | ||
| 254 | - 当前结果: | ||
| 255 | - 第 1 次快照 `archive_size=2110291968` | ||
| 256 | - 第 2 次快照 `archive_size=2115256320` | ||
| 257 | - `status=waiting` | ||
| 258 | - 最新进度 `27.5439%` | ||
| 259 | |||
| 260 | 结论: | ||
| 261 | - 现在仓库已经具备“等待完成 -> 自动切入解压/就绪检查”的衔接能力 | ||
| 262 | - 后续 session 可以用单命令挂起等待,而不是反复手工轮询 | ||
| 263 | |||
| 240 | ### Stage: FMA 下载完成后自动就绪 | 264 | ### Stage: FMA 下载完成后自动就绪 |
| 241 | 265 | ||
| 242 | 完成项: | 266 | 完成项: | ... | ... |
| ... | @@ -147,6 +147,20 @@ cd acr-engine | ... | @@ -147,6 +147,20 @@ cd acr-engine |
| 147 | 147 | ||
| 148 | 如果归档还没下完,会返回结构化 `archive_not_complete`。 | 148 | 如果归档还没下完,会返回结构化 `archive_not_complete`。 |
| 149 | 149 | ||
| 150 | |||
| 151 | ### FMA 完成前等待并自动切换 | ||
| 152 | |||
| 153 | ```bash | ||
| 154 | cd acr-engine | ||
| 155 | /usr/local/miniconda3/bin/python scripts/wait_for_fma_and_prepare.py --interval 30 --max-cycles 120 | ||
| 156 | ``` | ||
| 157 | |||
| 158 | 作用: | ||
| 159 | |||
| 160 | - 周期性检查 `fma_small.zip` 是否完成 | ||
| 161 | - 一旦完成,自动进入 [scripts/fma_postdownload_ready.py](../acr-engine/scripts/fma_postdownload_ready.py) | ||
| 162 | - 如果还没完成,则返回 `waiting` 和最近的进度快照 | ||
| 163 | |||
| 150 | ## Sources | 164 | ## Sources |
| 151 | - See [dataset-spec.md](./dataset-spec.md) | 165 | - See [dataset-spec.md](./dataset-spec.md) |
| 152 | - See [dataset-sources-and-licensing.md](./dataset-sources-and-licensing.md) | 166 | - See [dataset-sources-and-licensing.md](./dataset-sources-and-licensing.md) | ... | ... |
| ... | @@ -319,3 +319,5 @@ | ... | @@ -319,3 +319,5 @@ |
| 319 | - [CHANGELOG.md](./CHANGELOG.md) | 319 | - [CHANGELOG.md](./CHANGELOG.md) |
| 320 | 320 | ||
| 321 | - FMA 下载完成后可直接执行:[acr-engine/scripts/fma_postdownload_ready.py](../acr-engine/scripts/fma_postdownload_ready.py) | 321 | - FMA 下载完成后可直接执行:[acr-engine/scripts/fma_postdownload_ready.py](../acr-engine/scripts/fma_postdownload_ready.py) |
| 322 | |||
| 323 | - 若需要等待下载完成并自动切到解压/就绪检查,可直接执行:[acr-engine/scripts/wait_for_fma_and_prepare.py](../acr-engine/scripts/wait_for_fma_and_prepare.py) | ... | ... |
-
Please register or sign in to post a comment