Compare commits

...
10 Commits
Author SHA1 Message Date
snowy-yang 22b263f547 refactor: SMTP 发信改 aiosmtplib,最后一个阻塞网络调用从 to_thread 里取出来
为什么:smtplib 是同步栈,原先靠 asyncio.to_thread 卸载——线程池里躺着的仍是一次
完整的 TCP + TLS 握手加多轮 SMTP 交互,属于「同步网络 IO 藏在 to_thread 里」:既
占着线程,也不产生并发收益。这是本系列最后一条此类遗留(aria2p 那条已在 C2 清零)。

改动:
- auth_system/mail.py:_smtp_connect / _smtp_send 两个同步函数合成一个原生异步
  _smtp_send,底层换 aiosmtplib.SMTP。TLS 口径与原实现一一对应:465 用 use_tls
  (隐式 TLS),其余端口用 start_tls(显式 STARTTLS,先握手再 LOGIN);两条路都传
  显式构造的 ssl.create_default_context(),不依赖运行时默认校验口径。finally 里
  quit 失败只吞连接关闭错误,sendmail 的真实异常原样上抛——send_verify_code 依赖
  它决定「不写库、不烧 24h 额度」。
- auth_system/mail.py:send_email 改直接 await;移除 import asyncio / import smtplib。
- auth_system/__init__.py:撤掉 _smtp_connect 的再导出——它是同步时代的辅助函数,
  全仓无消费方,新实现里也不存在对应物。
- web/routers/register.py:更新过时的模块说明(原文仍写「smtplib 阻塞发送已卸载」)。
- pyproject.toml / uv.lock:新增 aiosmtplib>=5.1.3,<6.0。
- tests/test_mail_smtp.py(新增 5 条):SMTP 路径此前零覆盖,而「端口 → TLS 方式」
  的映射属「授权码会不会明文过网」级别的差异,一旦被改反不会有任何地方报警。用假
  SMTP 替身记录构造参数与调用序、全程不触网:587 必须 start_tls 且 LOGIN 在 connect
  之后、465 必须 use_tls 且不得再叠 STARTTLS、端口缺省回落 587、quit 失败不掩盖结果、
  sendmail 失败原样上抛且仍关连接。

验证:
- uv run ruff check .  → All checks passed!
- uv run pyright       → 0 errors, 0 warnings, 0 informations
  (aiosmtplib 自带 py.typed,故对它的调用是有效类型校验;另用 inspect 逐一核对了
   5.1.3 的 SMTP / connect / login / sendmail / quit 参数名)
- uv run pytest        → 397 项,0 失败 / 0 error / 0 skipped(C3 后 392,新增 5)
- data/site.db sha256 前后一致(a19f5dbf…6487),无 -wal/-shm 残留
2026-09-29 15:28:13 +08:00
snowy-yang e8cc7cb34a refactor: 本地文件 IO 归位——重活留线程、微秒级直接同步,移除零使用依赖 aiofiles
为什么:文件 IO 这条线上 to_thread 用得过宽。凡是「单文件 exists / unlink / stat」
这类微秒级元数据操作,线程调度开销(几十微秒起)比操作本身还大,套 to_thread 只是
给同步调用加了层异步外壳,不产生任何并发收益;真正需要卸载的是整库复制、ZIP 整段
拷贝、SQLite 探测这类活。本批按这个口径把两端都摆正,并顺手修掉一处真实缺陷。

改动:
- web/routers/admin_site.py 备份上传:原 `content = await f.read()` 会把整库一次性
  读进内存(端点对外无大小上限,等于给了上传方一个内存放大器),落盘后还要把文件
  重新打开读一次头。改为 _save_upload_stream 按 1 MiB 逐块搬运——内存占用恒为一块,
  文件头在写第一块时就地截获(省掉那趟回读)。仍属阻塞 IO,故保留在 to_thread 里。
- web/routers/admin_site.py backup_list:撤掉仅为一次 stat 而拆出的嵌套函数 +
  to_thread,直接在事件循环里做完。
- web/routers/admin.py:save_cookie / logout / _get_cookie_info 四处 to_thread 改
  直接调用——它们读写 1KB 级的 cookie.txt,全程不碰网络。
- music_api/qr_client.py:logout() 的 print 改 logger。该方法既被 CLI 也被 admin
  端点调用,在 web 语境里 print 会把内容直接喷到服务端 stdout,绕过 logging 配置、
  无法被收集,也污染容器日志。
- auth_system/backup.py:三处单文件 unlink(占位文件回滚、删备份、裁剪 pre_restore_)
  改直接同步;把只为配合 to_thread 而拆出的 _backup_path_sync 并回 backup_path。
  保留在线程里的:整库 copy2 / os.replace、glob + N 次 stat、SQLite 探测、以及持
  跨线程锁的 _reserve_backup_path(内联会让事件循环等一把别的线程可能持有的锁)。
- web/routers/delivery.py:两处 os.path.exists 改直接同步(带 ASYNC240 行内豁免,
  与 core/db.py、auth_system/db.py 同一写法),顺带删掉随之未使用的 asyncio 导入。
- pyproject.toml / uv.lock:移除 aiofiles。它全仓零引用,且底层同样是 to_thread——
  对 1KB 的 cookie 文件不会带来任何真异步收益。

验证:
- uv run ruff check .  → All checks passed!
- uv run pyright       → 0 errors, 0 warnings, 0 informations
- uv run pytest        → 392 项,0 失败 / 0 error / 0 skipped
  (含 test_admin_backup_upload 四条:接受真实 SQLite / 缺字段拒 / 后缀拒 /
   后缀对但内容非库必须拒且不留半成品)
- data/site.db sha256 前后一致(a19f5dbf…6487),无 -wal/-shm 残留
2026-09-29 15:20:51 +08:00
snowy-yang 046225e38d refactor: aria2 改手写原生异步 JSON-RPC,aria2p 与同步 HTTP 栈从依赖树清零
为什么:aria2p 内部是 requests + websocket-client 的同步实现,每个 RPC 都是一次
「同步网络 IO 藏在 to_thread 里」——线程池跑得再满也不产生真正的并发,每次轮询
还要额外付一次线程跳转。这是本项目最后一处此类残留,删掉后整个进程的网络 IO
都走同一个 aiohttp 连接池。

改动:
- music_downloader/_aria2.py 整体重写:新增 _Aria2Rpc,直接向 aria2 的
  JSON-RPC 2.0 端点发 getVersion / addUri / tellStatus / forceRemove /
  removeDownloadResult,一律走 core.http_client.request_once(不重试,与 aria2p
  口径一致);新增 Aria2RpcError 承载 HTTP 200 + error 对象的协议错误。
- _abort 逐条复刻 aria2p.remove(force=True, files=?, clean=True) 语义:终态走
  removeDownloadResult;在途先 forceRemove 再清结果表;控制文件 .aria2 无条件
  删除(clean=True 默认);仅当 files 且主移除成功才删数据文件。
- get_backend / _try_start 改原生异步:asyncio.Lock 单飞 + await rpc.get_version()
  探活,不再用 to_thread 包 subprocess.Popen;超时路径改 to_thread(_reap) 回收。
- 新增 Protocol _Rpc / _ProcessHandle(补 returncode 声明)——只声明本模块真正
  调用的方法,测试据此用轻量替身驱动状态机,不必依赖真实 aria2 进程。
- pyproject.toml 删 aria2p 依赖并同步更新过时注释;uv lock 连带移除 8 个传递
  依赖:requests / urllib3 / websocket-client / charset-normalizer / loguru /
  platformdirs / win32-setctime。至此全仓无任何 requests 栈。
- tests/test_aria2.py 重写为 21 项用例(_FakeRpc / _FakeProc / _LiveProc /
  _FakeHTTP 替身),新增终态体积守卫、_try_start 三分支(就绪重试 / 超时不就绪
  杀进程 / 进程提前退出)与 JSON-RPC 信封四例。
- tests/test_downloader.py 的 get_backend 替身由同步 lambda 改为协程——调用点
  已直接 await,同步替身会被 await 成 None 而报 TypeError。

验证(在 aria2p 已物理移除的环境上复核,故能真正证明无残留依赖):
- uv run ruff check .  → All checks passed!
- uv run pyright       → 0 errors, 0 warnings, 0 informations
- uv run pytest        → 392 项,0 失败 / 0 error / 0 skipped(基线 382,净增 10)
- data/site.db sha256 前后一致(a19f5dbf…6487),无 -wal/-shm 残留
2026-09-29 15:14:56 +08:00
snowy-yang 6c7f8f589e refactor: VIP 单飞锁改 asyncio.Lock,去掉每请求一次线程跳转
web/routers/account.py 的音质探测单飞锁原本是 core.state 里的 threading.Lock,
在协程里用 `await asyncio.to_thread(lock.acquire, True, 0.5)` 做有界等待,每次
VIP 请求都白付一次线程池调度。

为什么可以直接换:这把锁**只在协程里用**。原注释写「与 core.state 中同一把锁
共享」,但 grep 全仓只有 _detect_vip 一个消费者(sync 时代另有一个线程侧使用者,
早已不存在)。改为 asyncio.Lock + asyncio.wait_for(acquire(), 0.5),超时分支用
`except TimeoutError` 走「不等 8 级探测、直接用账号接口降级」的原有语义。

顺带把锁从 core.state 移出:core.state 自己声明「依赖仅限标准库
threading/collections」,而这把锁是 asyncio 的。现在它跟着唯一消费者走。

锁按事件循环隔离(以 id(loop) 为键):asyncio.Lock 绑定创建它的 loop,应用主 loop
与测试各自的 loop 并存,跨 loop 复用会抛 "bound to a different event loop"
(B2 迁移 auth_system 时踩过同一个坑)。

门禁:ruff 全过 / pyright 0-0-0 / pytest 382 项全绿(0 失败)。
2026-09-29 14:57:24 +08:00
snowy-yang 133beab492 chore: 删除 HTTP 线三个垫片,同步/异步桥接层彻底清零
删 core/sync_http.py(requests 同形同步门面)、core/async_bridge.py(run_sync
同步↔异步桥)、core/http_errors.py(两垫片共用的 requests 同形异常层次),
以及 tests/test_async_bridge.py。

为什么必须一次删完:三者是**自闭环三角**——sync_http 是 async_bridge 与
http_errors 的唯一消费者,删掉任何一个其余两个立刻悬空。前序 B1 把 worker 变成
协程、B2 把 auth_system 迁到 aiosqlite、更早一轮把 music_api 改走
core.http_client 之后,这条链上的生产消费者已归零;http_errors 更是在 B2 之后
只剩 sync_http 一个使用者(此前 sync_http 的 docstring 还提到
models._RetryFromScratch 继承它的 RequestException——B1 已改为继承 Exception)。

删除前的核查(避免误删活代码):
- run_sync / get_loop / SyncResponse / Session( 全仓零残留引用(只剩
  aiohttp.ClientSession,那是真正的客户端,不受影响)。
- 所有 HTTPError 命中都是 core.http_client.UpstreamHTTPError——与
  core.http_errors 的 HTTPError **名字像但不是一个东西**,删除时别被 grep 骗。
- requests 早已不是声明依赖、全仓零导入;删掉 http_errors 后连 requests 同形
  异常的最后残影也没了。
- core/__init__.py 只有 docstring,无需改动;web/app.py 的 async_bridge.shutdown()
  已在 B1 移除。

顺带修正两处过时措辞:tests/test_search_music.py 的错误映射注释
RequestException → UpstreamError(异常层次已换);pyproject.toml 中 pyright
include 的理由注释 requests→sync_http → requests→aiohttp(去掉已删模块名、
保留原意)。auth_system/geo.py 的历史注释本就是过去时,保持不动。

门禁:ruff 全过 / pyright 0-0-0 / pytest 382 项全绿(0 失败 0 error 0 skipped)。
项数 384→382 是 test_async_bridge.py 两条用例**随被测对象一起删除**的预期减项,
不是跳过或漏跑。data/site.db 读写前后 sha256 一致(a19f5db…6487)。

至此「同步调异步 / 异步桥同步」两类垫片全部拆净:DB 线(B1 worker + B2
auth_system)、HTTP 线(本批)。生产代码里 to_thread 仅剩正当的阻塞卸载
(PBKDF2 / smtplib / 备份文件 IO / ZIP 组装 / rmtree),threading.Event 仅剩
取消与暂停的跨线程置位载体。
2026-09-28 20:18:08 +08:00
snowy-yang 96af56cc27 refactor: auth_system 全面异步化(aiosqlite),拆掉 web/services 层的 to_thread 桥接
拆掉两类垫片的最后一处:auth_system 原为同步 sqlite3 栈,web 层与 services 的
worker 只能靠 asyncio.to_thread 逐个桥接才不阻塞事件循环——每次 DB 访问都白付一次
线程池调度,且「同步调异步」的边界散落在 100+ 个调用点。B1 已把 worker 变成协程,
本批把访问层本身也迁到原生异步,两条线在此合拢。

改动(按子系统):
- auth_system:db.py 照 core/db.py 现有范式重建为原生异步访问层(aiosqlite + WAL)
  ——读路径 _q 每次独立连接、无锁、可并发;写路径 _x / write_transaction 用
  asyncio.Lock 串行化,写者不阻塞读者。users / cards / credits / ledger / logs /
  settings / mail / geo / backup 九模块随之全部 async/await;security.py 是纯计算
  (pbkdf2/normalize)不动。__init__ 去掉 _DB_LOCK 与 get_conn 导出。
- web:app / deps / billing / tasks_registry 与 routers/{admin_site,auth,music,
  register,user} 直接 await,拆掉包裹的 to_thread。deps.secret_key 改 async,
  连带 _parse_cookie 也 async(会话解析要读密钥)。
- services:_taskcore / tasks / downloads / podcast 的账本调用与查询直接 await。
  4 处 start_task 调用点(download.py)确认已 await。

设计取舍(为什么这么做,而不是直接包一层):
- 锁按**事件循环**隔离(_LOOP_LOCKS 以 id(loop) 为键):asyncio.Lock 绑定创建它的
  loop,应用主 loop 与测试各自的 loop 并存,跨 loop 复用会抛 "bound to a different
  event loop"。
- WAL 切换用**独立锁** _wal_lock 而非 _write_lock:execute() 已持写锁,再在锁内
  connect→_configure 会自锁死(asyncio.Lock 非重入)。切换 + 标记在专用锁内串行并
  二次复查,否则并发首连会同时执行 PRAGMA journal_mode=WAL 争排他锁报
  database is locked(core/db.py 同款教训,实测复现)。
- force_wal 保留:journal_mode 持久化在库文件头,库文件被删重建时新文件不继承,
  必须重设。exists() 判定须在 aiosqlite.connect **之前**(connect 会创建空文件)。
- connect() 用 DB_PATH.parent 兜底建目录,优于 DATA_DIR:生产两者相同,但测试
  monkeypatch 只换 DB_PATH 时兜底仍能命中正确目录。

留在 to_thread 上的仅剩**正当的阻塞卸载**,不是同步门面:PBKDF2 的 10 万次迭代
(登录链路每次都要跑)、smtplib 发信、备份文件的 copy2 / os.replace、secret.key
的文件 IO。原有 aria2 / downloader / zip 侧的 to_thread 未动。

不变量逐条保持(迁移只改执行模型,不改语义):
- ledger_settle 幂等:账本非 open 直接 return 0,重启后重放不重复退款;
- 终态先到先得:_mark_task_terminal 对已 done/error/canceled 的任务直接 return,
  堵住 canceled→error 的非法迁移边;
- settle 必须在任务锁**外**调用(ledger_settle 要抢 SQLite 写锁,busy 最长 15s,
  持任务锁会让全站进度/提交/取消随库停摆);
- reserve_and_open 的预扣与开账同事务,消除原两步之间的崩溃丢账窗口。

路径真源红线:DATA_DIR / DB_PATH / BACKUP_DIR / SECRET_KEY_PATH 仍是 auth_system.db
的**模块全局**(用与 core.db 相同的表达式,但**不是**它的别名)。测试只 patch
auth_system.db.*,若改成 core.db 的别名会静默把测试写到生产库。已核 data/site.db
读写前后 sha256 完全一致。
顺带清掉 get_conn:迁移后它只剩「转调 connect()」一层空壳、全仓零消费者(测试
直接开裸连接、备份用 db.connect()),保留等于留一条「拿裸连接自己写」的口子。

测试:19 个文件同步迁移——10 个文件的 web.deps.secret_key 替身改 async;
test_db_conn.py 的多线程首连用例按异步语义拆成 2 条(单 loop gather(16) 覆盖批量
并发首连、WAL 就绪后 8 线程各自 loop 建连覆盖跨 loop);test_billing_scope.py 去掉
多余的 @pytest.mark.anyio;被 await 且**正常返回**的 monkeypatch 替身一律改 async
(仅 raise 的同步替身仍可用)。

门禁:ruff 全过 / pyright 0-0-0(standard 模式含 tests 与顶层模块)/ pytest
384 项全绿(0 失败 0 error 0 skipped;383→384 因 test_db_conn 拆用例)。
data/site.db 读写前后 sha256 一致(a19f5db…6487),无 -wal/-shm 边车残留。

下一步 B3:删 core/sync_http.py、core/async_bridge.py、core/http_errors.py 及
 tests/test_async_bridge.py——三者在生产侧已零消费者(http_errors 仅 sync_http
 自用,requests 已非声明依赖)。
2026-09-28 20:13:19 +08:00
snowy-yang 69a50c2c31 refactor: worker 子系统全面异步化,拆掉同步转异步的桥接层
拆掉「同步调异步 / 异步桥同步」两类垫片的第二半(HTTP 线 B2 见前序提交)。
worker 侧此前是「起线程跑同步函数、函数体内部再用 asyncio.to_thread 把活塞给
另一条循环」——线程与协程互等,取消/暂停只能靠轮询。本轮把 worker 自身变成
协程,由 _start_task 直接挂到应用事件循环上,桥接层随之失去存在理由。

改动(按子系统):
- music_downloader:downloader / _aria2 / _concurrency / cover / _http /
  __main__ 全链路协程化。per-file 锁改 asyncio.Lock;计数闸改
  「active 计数 + await asyncio.sleep(0.5)」的可取消等待(原阻塞式 with 会让
  已取消任务仍要等全局名额腾出,卡死并拖后腿退款);aria2p 的同步 RPC 仍经
  to_thread,轮询改 asyncio.sleep;新增 core.http_client.iter_chunks,把**流式
  读取期**的 aiohttp.ClientError 收敛成 UpstreamNetworkError(请求建立阶段早已
  有这条路径,读取阶段此前没有,断流会绕过调用方的 UpstreamError 分类)。
  删除 get_http_session / _SESSION_LOCAL。
  models._RetryFromScratch 不再继承 requests.RequestException 而继承 Exception:
  它是「续传起点失效、残片已清、从头重下」,既不是确定性失败
  (DownloadException),也不是残片已保留的网络失败(UpstreamError),必须独立
  成类且在 except 链中先于二者捕获。
- music_api:podcast.py 全部函数协程化(直链解析内部改
  await NeteaseAPI.get_song_url);compat.py 删除——16 个死掉的同步门面早已
  清空,文件只剩空壳。
- services:_taskcore / _zipcore / batch / downloads / podcast / tasks 协程化。
  threading.Thread → loop.create_task(任务句柄字段 worker_thread →
  worker_task),ThreadPoolExecutor → asyncio.gather + asyncio.Semaphore,
  终态函数与结算回调全部改 await;监控线程里「worker 已死」判定改 task.done()。
  删除 services/streaming.py(同步流垫片,唯一消费者已迁到
  core.http_client.open)。zipfile 实例仍非线程安全,故「下载」并行、
  「入包」严格串行。
- web:路由层 130+ 处 to_thread 拆除——被包住的目标自己已是协程,再包一层只是
  把协程丢进线程里跑一圈。tasks_registry.start_task 直接 create_task。
  web/app.py 关闭段去掉 async_bridge.shutdown()(不再有第二条约循环)。
  web/routers/download.py 的播客端点补上 3 处漏掉的 await
  (extract_podcast_ref / get_radio_detail)——漏掉时不报错、只静默返回协程对象,
  靠 pyright 抓出来后修正。

留在 to_thread 上的都是**正当的阻塞卸载**:同步 SQLite 落库与账本结算、
mutagen 全量重写标签、zipfile 整段拷贝、shutil.rmtree、aria2p 的同步 RPC、
PBKDF2 的 10 万次迭代。ruff 对残留的本地元数据操作(exists/stat/os.replace/
往 .part 顺序写分块)按「微秒级、卸载的调度开销大于操作本身」放行,pyproject
逐条注明理由。

取消/暂停仍由 threading.Event 承载:置位方是请求线程/监控线程,读取方是 worker
协程,只有「置位」与「轮询读取」两种操作,既不需要跨循环唤醒也不会卡住事件
循环——比 asyncio.Event 更省心(ASYNC110 同理按此豁免)。

测试:10 个文件的替身边界同步迁移——网络替身从 core.sync_http 的
get_http_session 换成 downloader/服务模块的 _http.open + **真实** iter_chunks
(顺带钉住读取期异常收敛);`_run` 改为在独立线程里 asyncio.run(worker(...));
aria2 的假时钟从 time.sleep 换成 _POLL_INTERVAL=0;monitor 的线程句柄替身换成
done() 可控的轻量替身(asyncio.Task 的构造需要运行中的事件循环,而监控循环跑在
常驻线程里);podcast / 封面 / 结算包装等替身一律改 async。

门禁:ruff 全过 / pyright 0-0 / pytest 383 项全绿(0 失败)。

下一步 B2:auth_system 12 个模块改 aiosqlite 背板、web 层剩余 to_thread 收尾;
B3:删 core/sync_http.py、core/async_bridge.py、core/http_errors.py(现仅
sync_http 自用)。
2026-09-28 19:36:56 +08:00
snowy-yang 65adfc9089 refactor: 删除 aiohttp_compat 垫片,music_api 改走原生 core.http_client
core/aiohttp_compat 是「同步转异步」迁移期的 requests 同形异步垫片:它存在的
唯一理由是让既有 `requests.post/get + raise_for_status()` 风格的代码以最小
改动迁到异步。其两个调用方(music_api/api.py、music_api/client.py)**本就已是
async def**——垫片只剩「保留 requests 语义」这一层包装,属于迁移残留。

改动:
- api.py:23 处 requests.post/get → await core.http_client.request()。
  该文件每处都在请求后立刻 raise_for_status() 才取 .json(),从不把 4xx 当数据读,
  故「_once 对非 2xx 直接抛 UpstreamHTTPError」与旧语义等价(原 raise_for_status
  抛的 HTTPError 也一律落入 RequestException 分支);
  except requests.RequestException → except UpstreamError。
- client.py:两个自管重试的方法改用新公开的 core.http_client.request_once()
  (单次、不重试),保留原有「5xx 退避重试一次 / 4xx 立即失败」的分流——
  把 status_code>=500 的判断换成 UpstreamHTTPError.status,行为逐条对齐
  (4xx=风控不重试,网络/5xx 重试一次)。响应体上限交由客户端 max_bytes 统一把关。
  post_request_full 返回类型改为 HTTPResult(qr.py 用到的 .text/.cookies/.headers
  三者在 HTTPResult 上齐备)。
- core/http_client:_once 更名并公开为 request_once(自管重试的调用方需要它)。
  注:这是为「删垫片」而补的唯一一个小口子,不是新增抽象。

注意一处**有意的行为变化**:api.py 原经垫片请求零重试,现在 request() 自带
「网络异常/5xx 退避重试一次、4xx 不重试」(core.http_client 既定口径)。
429/460 属 4xx 不重试,不加剧风控。

- 删除 core/aiohttp_compat.py;web/app.py 去掉其 close();pyproject 去掉其
  per-file-ignore;core/http_errors.py 注释更正为「仅供 core.sync_http」。
- 测试:test_aiohttp_compat.py 拆解为 test_async_bridge.py(只留桥的两条契约,
  垫片自身的 4 条随垫片消亡);test_search_music / test_playlist_detail 的替身
  边界从 api.requests.post 换成 api._http.request,异常类型换成 UpstreamNetworkError。

门禁:ruff 全过 / pyright 0-0 / pytest 383 项全绿(0 失败;383=387-6+2,账面吻合)。
2026-09-28 19:17:03 +08:00
snowy-yang b089dadac2 refactor: geo 走原生异步 HTTP,清掉短链展开死链
「同步转异步」的残留清理:

1) auth_system/geo.query_ip_province 原经 core.sync_http 同步门面(在后台
   事件循环桥接)查询归属地,但其唯一调用方 record_login_location 的入口
   本来就是 async 的登录链路(web/routers/auth 里还套了一层 to_thread)。
   改为直接 await core.http_client——桥接纯属多余,顺带把 record_login_location
   转 async,落库仍走同步 sqlite 但经 asyncio.to_thread 卸载,避免 sqlite busy
   等待(最长 15s)冻结事件循环。
   调用方两处(普通用户/管理员登录)去掉 to_thread 包装,直接 await。

2) MusicAPIService._extract_music_id 是死代码:ID 归一化早已迁到
   web.request_utils(注释里明说是它的迁移口径),全仓零调用。它也是
   core.utils.expand_shortlink 的唯一调用方——一并删除,core.utils 由此
   彻底不再 import core.sync_http。

门禁:ruff 全过 / pyright 0-0 / pytest 387 项全绿(0 失败)。
2026-09-28 19:10:45 +08:00
snowy-yang 064a89d726 refactor: 删除旧前端单体,React 产物成为唯一首页来源
为什么:React 迁移已验证完毕(浏览器冒烟全过、golden 刷新),
legacy 模板兜底分支指向已删除的文件只会制造幻觉路径。删除
templates/index.html(466KB)与 static/site_auth.js(80KB)及
/static 挂载;jinja2 随兜底分支失去唯一使用者,连同 markupsafe
一并移出依赖(63 包 -2)。_render_index 简化为直接读构建产物——
dist 已入库恒存在,无需分支。
2026-09-27 18:41:30 +08:00
88 changed files with 2767 additions and 12013 deletions

No files matched your search

-6
View File
@@ -53,7 +53,6 @@ from .credits import (
reserve_credits,
)
from .db import (
_DB_LOCK,
BACKUP_DIR,
DATA_DIR,
DB_PATH,
@@ -63,7 +62,6 @@ from .db import (
SESSION_KEY,
_q,
_x,
get_conn,
get_secret_key,
init_db,
)
@@ -99,7 +97,6 @@ from .mail import (
_EMAIL_SENT_24H,
_SMTP_ENC_PREFIX,
_code_fails_trim,
_smtp_connect,
_smtp_password_decrypt,
_smtp_password_encrypt,
check_verify_code,
@@ -161,7 +158,6 @@ __all__ = [
'_CODE_FAILS',
'_CODE_FAILS_MAX',
'_CODE_PURPOSES',
'_DB_LOCK',
'_EMAIL_SENT_24H',
'_LOG_WRITE_COUNTER',
'_PBKDF2_ITERS',
@@ -179,7 +175,6 @@ __all__ = [
'_last_province',
'_normalize_email',
'_q',
'_smtp_connect',
'_smtp_password_decrypt',
'_smtp_password_encrypt',
'_user_public',
@@ -201,7 +196,6 @@ __all__ = [
'format_credits',
'generate_cards',
'get_announcement',
'get_conn',
'get_secret_key',
'get_setting',
'get_smtp_config',
+113 -56
View File
@@ -1,9 +1,13 @@
# -*- coding: utf-8 -*-
"""数据库备份、恢复与自动备份线程。
(自 auth_system.py 拆出,行为与原实现一致。)
(自 auth_system.py 拆出,行为与原实现一致;DB 备份改用 aiosqlite 在线备份
API。文件系统操作的归位口径:整库复制/替换、glob + N 次 stat、SQLite 探测这类
真活经 asyncio.to_thread 卸载;单文件 exists/unlink 是微秒级元数据操作,线程
调度开销比操作本身还贵,直接在事件循环里做完。)
"""
import asyncio
import math
import os
import shutil
@@ -14,6 +18,8 @@ import time
from pathlib import Path
from typing import Any
import aiosqlite
import auth_system.db as db # 绝对导入子模块:不经过包属性,避免与包 __init__ 互指成环
from .settings import (
@@ -24,17 +30,16 @@ from .settings import (
# ============ 数据库备份 ============
# 备份命名专用锁:create_backup 不再持有全局 _DB_LOCK(不阻塞写请求),
# 但同秒并发的两次备份仍需保证文件名唯一
# 备份命名专用锁:占位命名不阻塞写请求,但同秒并发的两次备份仍需保证文件名唯一
_BACKUP_NAME_LOCK = threading.Lock()
def create_backup() -> str:
"""创建备份(唯一文件名——同秒冲突自动追加序号,不再互相覆盖)。
def _reserve_backup_path() -> Path:
"""锁内挑选唯一文件名并创建占位文件(同步微秒级本地 IO,仅建一个空文件)。
WAL 模式下用独立只读连接走 SQLite 在线备份 API:拿到的是一致性快照,
且不持有 _DB_LOCK——原实现备份期间阻塞全站写请求,大库时会卡住所有用户。"""
db.BACKUP_DIR.mkdir(parents=True, exist_ok=True)
先占位(而不是备份完再命名)保证同秒并发下命名唯一:备份前的空文件与
sqlite3.connect 建出的空库等价,随后在线备份写入其内容。
"""
with _BACKUP_NAME_LOCK:
base = 'backup_' + time.strftime('%Y%m%d_%H%M%S')
name = base + '.db'
@@ -42,32 +47,46 @@ def create_backup() -> str:
while (db.BACKUP_DIR / name).exists():
name = f'{base}-{k}.db'
k += 1
# 占位创建目标文件(锁内),保证并发下同秒命名仍唯一
dest = sqlite3.connect(str(db.BACKUP_DIR / name))
conn = db.get_conn()
dest = db.BACKUP_DIR / name
dest.touch()
return dest
async def create_backup() -> str:
"""创建备份(唯一文件名——同秒冲突自动追加序号,不再互相覆盖)。
WAL 模式下用独立只读连接走 SQLite 在线备份 API(aiosqlite Connection.backup):
拿到的是一致性快照,且不持有写锁——原实现备份期间阻塞全站写请求,
大库时会卡住所有用户。"""
db.BACKUP_DIR.mkdir(parents=True, exist_ok=True)
dest_path = await asyncio.to_thread(_reserve_backup_path)
name = dest_path.name
try:
conn = await db.connect()
try:
conn.backup(dest)
dest = await aiosqlite.connect(str(dest_path))
try:
await conn.backup(dest)
finally:
dest.close()
await dest.close()
finally:
await conn.close()
# 任何备份产生后统一裁剪(手动/自动/上传同配额)
try:
prune_backups(20, 5)
await prune_backups(20, 5)
except Exception:
pass
return name
except Exception:
# 备份失败:清掉占位文件,不留半个坏备份
# 备份失败:清掉占位文件,不留半个坏备份(单文件 unlink 是微秒级操作,直接做)
try:
(db.BACKUP_DIR / name).unlink(missing_ok=True)
dest_path.unlink(missing_ok=True)
except OSError:
pass
raise
finally:
conn.close()
def list_backups() -> list[dict[str, Any]]:
def _list_backups_sync() -> list[dict[str, Any]]:
"""备份列表(按文件修改时间倒序——按名字排序会让 upload_ 永远排前,误删新自动备份)"""
if not db.BACKUP_DIR.exists():
return []
@@ -83,13 +102,24 @@ def list_backups() -> list[dict[str, Any]]:
return out
def backup_path(name: str) -> Path | None:
async def list_backups() -> list[dict[str, Any]]:
# glob + 每份备份一次 stat,份数可达数十:够不上「单次微秒级元数据操作」,
# 仍留在线程里(BACKUP_DIR 可能是绑定挂载/网络盘,逐个 stat 会累积)
return await asyncio.to_thread(_list_backups_sync)
async def backup_path(name: str) -> Path | None:
"""备份文件路径解析(防目录穿越)。
单文件 exists + 后缀判定,微秒级元数据操作——直接同步做,线程调度开销比
操作本身还大。(原先拆出 _backup_path_sync 只为配合 to_thread。)
"""
name = Path(name).name # 防目录穿越
p = db.BACKUP_DIR / name
return p if p.exists() and p.suffix == '.db' else None
def _backup_is_valid_db(src: Path) -> bool:
def _backup_is_valid_db_sync(src: Path) -> bool:
"""深度校验备份文件:SQLite 头 + 能真正打开 + 含 users 表,三者缺一不可"""
try:
with open(src, 'rb') as f:
@@ -105,22 +135,33 @@ def _backup_is_valid_db(src: Path) -> bool:
return False
async def _backup_is_valid_db(src: Path) -> bool:
"""同步 sqlite3 探测(打开只读库)经 to_thread 卸载,不阻塞事件循环。"""
return await asyncio.to_thread(_backup_is_valid_db_sync, src)
# 恢复前静止钩子——由应用层注册(取消全部在途下载任务),避免 worker 按旧库账目结算
_before_restore_hook = None
def register_before_restore_hook(fn) -> None:
"""注册恢复前钩子(纯内存登记,无 IO,保持同步)。"""
global _before_restore_hook
_before_restore_hook = fn
def restore_backup(name: str) -> None:
async def restore_backup(name: str) -> None:
"""恢复备份(先验证、留回滚、后替换、失败自动回滚,源备份文件保留不移动)
恢复前先通过钩子取消全部在途任务(worker 不再引用旧库用户结算)"""
src = backup_path(name)
恢复前先通过钩子取消全部在途任务(worker 不再引用旧库用户结算)
整个替换过程持**当前事件循环的写锁**(经 db._write_lock())串行化;新库
结构初始化调用 db._init_db_locked(不自行取锁)以避免 asyncio.Lock 非重入
造成的自锁死。
"""
src = await backup_path(name)
if not src:
raise ValueError('备份文件不存在')
if not _backup_is_valid_db(src):
if not await _backup_is_valid_db(src):
raise ValueError('不是有效的站点数据库备份(校验失败,未做任何修改)')
if _before_restore_hook is not None:
try:
@@ -128,44 +169,53 @@ def restore_backup(name: str) -> None:
except Exception:
pass
safety = db.BACKUP_DIR / ('pre_restore_' + time.strftime('%Y%m%d_%H%M%S') + '.db')
with db._DB_LOCK:
async with db._write_lock():
# 0) 先尽力清 WAL 并合并进主库(被占用导致 checkpoint 忙时明确失败,不假装成功);
# 必须在留安全副本之前做——否则副本缺 WAL 里未合并的最近提交,回滚点不完整
conn = db.get_conn()
conn = await db.connect()
try:
row = conn.execute('PRAGMA wal_checkpoint(TRUNCATE)').fetchone()
row = await (await conn.execute('PRAGMA wal_checkpoint(TRUNCATE)')).fetchone()
busy = bool(row and row[0] == 1) # 1 = busy
finally:
conn.close()
if busy:
raise RuntimeError('数据库当前有其他读写在途(checkpoint 忙),请稍后重试恢复')
# 1) checkpoint 后留安全副本(包含全部已提交数据,可回滚)
shutil.copy2(str(db.DB_PATH), str(safety))
await asyncio.to_thread(shutil.copy2, str(db.DB_PATH), str(safety))
# 2) 源备份复制到临时文件(不动源备份本身)
tmp = db.DB_PATH.with_suffix('.db.restore_tmp')
shutil.copy2(str(src), str(tmp))
await asyncio.to_thread(shutil.copy2, str(src), str(tmp))
finally:
# 先关闭旧库连接:Windows 下替换被打开的库文件会失败
await conn.close()
# 3) 原子替换 + 立即校验
os.replace(str(tmp), str(db.DB_PATH))
await asyncio.to_thread(os.replace, str(tmp), str(db.DB_PATH))
newconn = await db.connect()
try:
db.init_db()
await db._init_db_locked(newconn)
except Exception:
await newconn.close()
# 替换后初始化失败:回滚安全副本,恢复原库
shutil.copy2(str(safety), str(db.DB_PATH))
db.init_db()
raise RuntimeError('恢复后数据库初始化失败,已自动回滚到恢复前状态') from None
# 裁剪移出 _DB_LOCK——glob/unlink 的 I/O 不阻塞全站写
await asyncio.to_thread(shutil.copy2, str(safety), str(db.DB_PATH))
rc = await db.connect()
try:
prune_backups(20, 5)
await db._init_db_locked(rc)
finally:
await rc.close()
raise RuntimeError('恢复后数据库初始化失败,已自动回滚到恢复前状态') from None
finally:
await newconn.close()
# 裁剪移出写锁——glob/unlink 的 I/O 不阻塞全站写
try:
await prune_backups(20, 5)
except Exception:
pass
def delete_backup(name: str) -> bool:
p = backup_path(name)
async def delete_backup(name: str) -> bool:
p = await backup_path(name)
# 允许删除 pre_restore_ 安全副本(配额裁剪已保证至少保留
# 最近 5 份;原实现一刀切拒删导致恢复频繁时无界累积且管理员手动也删不掉)
if p:
p.unlink()
p.unlink() # 单文件 unlink 是微秒级元数据操作,直接同步做(省一次线程跳转)
return True
return False
@@ -176,43 +226,49 @@ _backup_thread_started = False
_BACKUP_ERR_TS = {'ts': 0.0, 'n': 0} # 备份失败限频记录(失败不能零痕迹)
def prune_backups(keep_normal: int = 20, keep_pre_restore: int = 5) -> None:
async def prune_backups(keep_normal: int = 20, keep_pre_restore: int = 5) -> None:
"""备份配额裁剪(从自动备份循环抽出为独立函数)——
非pre_restore 保留最近 N 份、pre_restore_ 安全副本保留最近 M 份。
在 create_backup/上传备份/恢复备份后统一调用(原实现只在自动备份循环里
裁剪,手动备份/上传/恢复会让 data/backups 无界累积)。"""
baks = [b for b in list_backups() if not b['name'].startswith('pre_restore_')]
baks = [b for b in await list_backups() if not b['name'].startswith('pre_restore_')]
for b in baks[keep_normal:]:
try:
delete_backup(b['name'])
await delete_backup(b['name'])
except Exception:
pass
pre = [b for b in list_backups() if b['name'].startswith('pre_restore_')]
pre = [b for b in await list_backups() if b['name'].startswith('pre_restore_')]
for b in pre[keep_pre_restore:]:
try:
(db.BACKUP_DIR / b['name']).unlink()
(db.BACKUP_DIR / b['name']).unlink() # 同上:单文件 unlink,直接同步
except OSError:
pass
def _auto_backup_loop() -> None:
while True:
try:
interval = float(get_setting('backup_interval_hours', '0') or 0)
async def _auto_backup_tick() -> None:
"""自动备份单次检查(在独立事件循环里运行,见 _auto_backup_loop)。"""
interval = float(await get_setting('backup_interval_hours', '0') or 0)
# 非有限值(NaN/inf)或负数一律视为关闭
if not math.isfinite(interval):
interval = 0
set_setting('backup_interval_hours', '0')
await set_setting('backup_interval_hours', '0')
if interval > 0:
last = float(get_setting('last_backup_ts', '0') or 0)
last = float(await get_setting('last_backup_ts', '0') or 0)
if not math.isfinite(last):
last = 0
if time.time() - last >= interval * 3600:
create_backup()
set_setting('last_backup_ts', str(time.time()))
await create_backup()
await set_setting('last_backup_ts', str(time.time()))
# 仅保留最近 20 份(按修改时间,不按文件名)
# pre_restore_* 安全副本不计入 20 份配额,仅各自保留最近 5 份
prune_backups(20, 5)
await prune_backups(20, 5)
def _auto_backup_loop() -> None:
"""自动备份后台**线程**主循环(线程无法 await,每次 tick 用 asyncio.run 起独立 loop)。"""
while True:
try:
asyncio.run(_auto_backup_tick())
except Exception:
# 定时备份失败绝不能静默(磁盘满/权限问题时 owner 以为有备份、
# 出事时没有)——限频告警,每 10 分钟最多一条(对齐监控线程口径)
@@ -228,6 +284,7 @@ def _auto_backup_loop() -> None:
def start_auto_backup() -> None:
"""启动自动备份后台线程(纯线程启动,无 IO,保持同步)。"""
global _backup_thread_started
if _backup_thread_started:
return
+23 -53
View File
@@ -1,7 +1,7 @@
# -*- coding: utf-8 -*-
"""次数卡密生成 / 兑换 / 管理。
(自 auth_system.py 拆出,行为与原实现一致。)
(自 auth_system.py 拆出,行为与原实现一致;DB 访问已迁移为原生异步。)
"""
import secrets
@@ -9,15 +9,14 @@ import time
from typing import Any
from .db import (
_DB_LOCK,
_q,
get_conn,
write_transaction,
)
# ============ 卡密 ============
def generate_cards(count: int, credits: int, created_by: str = '',
async def generate_cards(count: int, credits: int, created_by: str = '',
prefix: str = '', note: str = '', valid_days: int = 0) -> list[str]:
"""生成次数卡密(credits = 每张卡可下载次数)
prefix: 自定义卡密前缀(默认 VIP-)
@@ -28,29 +27,24 @@ def generate_cards(count: int, credits: int, created_by: str = '',
prefix = ''.join(ch for ch in (prefix or 'VIP') if ch.isalnum()).upper()[:10] or 'VIP'
expire_at = int(time.time()) + int(valid_days) * 86400 if valid_days and int(valid_days) > 0 else 0
codes = []
with _DB_LOCK:
conn = get_conn()
try:
async with write_transaction() as conn:
for _ in range(count):
code = prefix + '-' + secrets.token_hex(5).upper()
conn.execute(
await conn.execute(
'INSERT INTO cards (code, minutes, created_by, created_at, note, expire_at) '
'VALUES (?, ?, ?, ?, ?, ?)',
(code, int(credits), created_by, int(time.time()), note or '', expire_at))
codes.append(code)
conn.commit()
finally:
conn.close()
return codes
def list_cards(limit: int = 100, offset: int = 0) -> tuple[list[dict[str, Any]], int]:
async def list_cards(limit: int = 100, offset: int = 0) -> tuple[list[dict[str, Any]], int]:
"""卡密列表(支持分页 limit/offset,管理路由透传)
返回 (当页卡密列表, 卡密总数)——总数供前端分页控件用,调用方按元组解包"""
limit = max(1, min(int(limit), 500))
offset = max(0, int(offset))
rows = _q('SELECT * FROM cards ORDER BY id DESC LIMIT ? OFFSET ?', (limit, offset))
total_row = _q('SELECT COUNT(*) AS c FROM cards')
rows = await _q('SELECT * FROM cards ORDER BY id DESC LIMIT ? OFFSET ?', (limit, offset))
total_row = await _q('SELECT COUNT(*) AS c FROM cards')
total = int(total_row[0]['c']) if total_row else 0
now = time.time()
cards = []
@@ -62,56 +56,38 @@ def list_cards(limit: int = 100, offset: int = 0) -> tuple[list[dict[str, Any]],
return cards, total
def set_card_disabled(code: str, disabled: bool) -> None:
async def set_card_disabled(code: str, disabled: bool) -> None:
"""禁用/恢复未使用卡密(已使用卡密无禁用意义,拒绝操作)"""
code = (code or '').strip().upper()
with _DB_LOCK:
conn = get_conn()
try:
row = conn.execute('SELECT used_by, disabled FROM cards WHERE code = ?', (code,)).fetchone()
async with write_transaction() as conn:
row = await (await conn.execute('SELECT used_by, disabled FROM cards WHERE code = ?', (code,))).fetchone()
if not row:
raise ValueError('卡密不存在')
if row['used_by']:
raise ValueError('卡密已被使用,无需禁用')
if bool(row['disabled']) == bool(disabled):
raise ValueError('卡密已处于该状态')
conn.execute('UPDATE cards SET disabled = ? WHERE code = ?',
await conn.execute('UPDATE cards SET disabled = ? WHERE code = ?',
(1 if disabled else 0, code))
conn.commit()
except Exception:
conn.rollback()
raise
finally:
conn.close()
def delete_card(code: str) -> None:
async def delete_card(code: str) -> None:
"""删除未使用卡密(已使用卡密含兑换凭证:使用者/时间,按 owner 规则不可删)"""
code = (code or '').strip().upper()
with _DB_LOCK:
conn = get_conn()
try:
row = conn.execute('SELECT used_by FROM cards WHERE code = ?', (code,)).fetchone()
async with write_transaction() as conn:
row = await (await conn.execute('SELECT used_by FROM cards WHERE code = ?', (code,))).fetchone()
if not row:
raise ValueError('卡密不存在')
if row['used_by']:
raise ValueError('已使用的卡密不可删除(保留兑换凭证)')
conn.execute('DELETE FROM cards WHERE code = ?', (code,))
conn.commit()
except Exception:
conn.rollback()
raise
finally:
conn.close()
await conn.execute('DELETE FROM cards WHERE code = ?', (code,))
def redeem_card(code: str, username: str) -> tuple[int, int]:
async def redeem_card(code: str, username: str) -> tuple[int, int]:
"""兑换卡密,返回 (卡密次数, 新剩余次数)(标记已用与加余额同事务,失败整体回滚不吞卡)"""
code = (code or '').strip().upper()
with _DB_LOCK:
conn = get_conn()
try:
row = conn.execute('SELECT * FROM cards WHERE code = ?', (code,)).fetchone()
async with write_transaction() as conn:
row = await (await conn.execute('SELECT * FROM cards WHERE code = ?', (code,))).fetchone()
if not row:
raise ValueError('卡密不存在')
if row['used_by']:
@@ -123,14 +99,14 @@ def redeem_card(code: str, username: str) -> tuple[int, int]:
# 卡密标记 + 余额增加同事务:任一失败整体回滚,卡密可重试兑换。
# WHERE 带 used_by=""/disabled=0 防重标记 + rowcount 校验:命中 0 行
# (跨进程并发抢卡)立即回滚报错,绝不出现"未标记已用却加了余额"
cur = conn.execute(
cur = await conn.execute(
'UPDATE cards SET used_by = ?, used_at = ? '
'WHERE code = ? AND used_by = "" AND disabled = 0',
(username, int(time.time()), code))
if cur.rowcount != 1:
raise ValueError('卡密已被使用')
urow = conn.execute('SELECT role, duration_seconds FROM users WHERE username = ?',
(username,)).fetchone()
urow = await (await conn.execute('SELECT role, duration_seconds FROM users WHERE username = ?',
(username,))).fetchone()
if not urow:
raise ValueError('用户不存在')
minutes = int(row['minutes'])
@@ -139,12 +115,6 @@ def redeem_card(code: str, username: str) -> tuple[int, int]:
remaining = -1
else:
remaining = remaining + minutes
conn.execute('UPDATE users SET duration_seconds = ? WHERE username = ?',
await conn.execute('UPDATE users SET duration_seconds = ? WHERE username = ?',
(remaining, username))
conn.commit()
return minutes, remaining
except Exception:
conn.rollback()
raise
finally:
conn.close()
+24 -38
View File
@@ -1,15 +1,14 @@
# -*- coding: utf-8 -*-
"""下载次数增减与格式化。
(自 auth_system.py 拆出,行为与原实现一致。)
(自 auth_system.py 拆出,行为与原实现一致;DB 访问已迁移为原生异步。)
"""
import time
from .db import (
_DB_LOCK,
_x,
get_conn,
write_transaction,
)
from .users import (
get_user_by_id,
@@ -19,92 +18,79 @@ from .users import (
# ============ 使用次数(按次计费:一首歌 = 1 次) ============
def refund_credits(uid: int, count: int) -> int:
async def refund_credits(uid: int, count: int) -> int:
"""退回下载次数(批量打包失败部分自动退费)。返回剩余次数(-1 表示不限)"""
user = get_user_by_id(uid)
user = await get_user_by_id(uid)
if user and (user['duration_seconds'] or 0) == -1:
return -1 # 不限次数用户无需退费
count = max(0, int(count))
if count <= 0:
return (user['duration_seconds'] or 0) if user else 0
_x('UPDATE users SET duration_seconds = COALESCE(duration_seconds, 0) + ? WHERE id = ?',
await _x('UPDATE users SET duration_seconds = COALESCE(duration_seconds, 0) + ? WHERE id = ?',
(count, uid))
user = get_user_by_id(uid)
user = await get_user_by_id(uid)
return (user['duration_seconds'] or 0) if user else 0
def charge_credits(uid: int, count: int = 1) -> int:
async def charge_credits(uid: int, count: int = 1) -> int:
"""按次扣减使用次数(管理员与不限次数(-1)用户不扣减,返回 -1)。返回剩余次数
扣减在 _DB_LOCK 内"读取-判断-写入"一次完成,防止并发双击导致超扣/双花"""
user = get_user_by_id(uid)
扣减在写事务内"读取-判断-写入"一次完成,防止并发双击导致超扣/双花"""
user = await get_user_by_id(uid)
if not user:
return 0
if user['role'] == 'admin' or (user['duration_seconds'] or 0) == -1:
_x('UPDATE users SET last_active = ? WHERE id = ?', (int(time.time()), uid))
await _x('UPDATE users SET last_active = ? WHERE id = ?', (int(time.time()), uid))
return -1
count = max(1, int(count))
with _DB_LOCK:
conn = get_conn()
try:
row = conn.execute('SELECT duration_seconds FROM users WHERE id = ?', (uid,)).fetchone()
async with write_transaction() as conn:
row = await (await conn.execute('SELECT duration_seconds FROM users WHERE id = ?', (uid,))).fetchone()
cur = int(row['duration_seconds'] or 0) if row else 0
if cur == -1:
conn.execute('UPDATE users SET last_active = ? WHERE id = ?', (int(time.time()), uid))
conn.commit()
await conn.execute('UPDATE users SET last_active = ? WHERE id = ?', (int(time.time()), uid))
return -1
newval = max(0, cur - count)
conn.execute('UPDATE users SET duration_seconds = ?, last_active = ? WHERE id = ?',
await conn.execute('UPDATE users SET duration_seconds = ?, last_active = ? WHERE id = ?',
(newval, int(time.time()), uid))
conn.commit()
return newval
finally:
conn.close()
def reserve_credits(uid: int, count: int = 1):
async def reserve_credits(uid: int, count: int = 1):
"""原子预扣下载次数(下载类接口在路由执行前调用,防止并发双击双花)。
足额:扣减并返回剩余次数;不限次数(-1)/管理员:不扣减返回 -1;
不足:返回 None(调用方应拒绝请求)"""
count = max(1, int(count))
with _DB_LOCK:
conn = get_conn()
try:
row = conn.execute('SELECT role, duration_seconds FROM users WHERE id = ?', (uid,)).fetchone()
async with write_transaction() as conn:
row = await (await conn.execute('SELECT role, duration_seconds FROM users WHERE id = ?', (uid,))).fetchone()
if not row:
return None
if row['role'] == 'admin':
conn.execute('UPDATE users SET last_active = ? WHERE id = ?', (int(time.time()), uid))
conn.commit()
await conn.execute('UPDATE users SET last_active = ? WHERE id = ?', (int(time.time()), uid))
return -1
cur = int(row['duration_seconds'] or 0)
if cur == -1:
conn.execute('UPDATE users SET last_active = ? WHERE id = ?', (int(time.time()), uid))
conn.commit()
await conn.execute('UPDATE users SET last_active = ? WHERE id = ?', (int(time.time()), uid))
return -1
if cur < count:
return None
conn.execute('UPDATE users SET duration_seconds = ?, last_active = ? WHERE id = ?',
await conn.execute('UPDATE users SET duration_seconds = ?, last_active = ? WHERE id = ?',
(cur - count, int(time.time()), uid))
conn.commit()
return cur - count
finally:
conn.close()
def add_credits(username: str, count: int) -> int:
async def add_credits(username: str, count: int) -> int:
"""增加次数(卡密充值/试用领取),返回新剩余次数;不限次数(-1)用户保持不限
累加用 SQL 原生自增(原子),避免并发充值互相覆盖"""
user = get_user_by_name(username)
user = await get_user_by_name(username)
if not user:
return 0
if user['role'] == 'admin' or (user['duration_seconds'] or 0) == -1:
return -1
_x('UPDATE users SET duration_seconds = COALESCE(duration_seconds, 0) + ? WHERE id = ?',
await _x('UPDATE users SET duration_seconds = COALESCE(duration_seconds, 0) + ? WHERE id = ?',
(max(0, int(count)), user['id']))
user = get_user_by_name(username)
user = await get_user_by_name(username)
return (user['duration_seconds'] or 0) if user else 0
+173 -74
View File
@@ -1,17 +1,31 @@
# -*- coding: utf-8 -*-
"""路径常量、连接与建表。
"""路径常量、连接与建表(原生异步 SQLite 访问层:aiosqlite + WAL)。
(自 auth_system.py 拆出,行为与原实现一致。)
(自 auth_system.py 拆出;本文件由同步 sqlite3 栈迁移为原生异步,设计照
`core/db.py`——同一套「读写同时进行」口径:)
- WAL 模式:读事务与写事务互不阻塞(读快照隔离);
- 读路径(`_q`):每次操作独立连接、无锁、可并发;
- 写路径(`_x` / `write_transaction`):SQLite 全库同一时刻只允许一个写者,
用 `asyncio.Lock` 串行化写事务,但写者不阻塞读者;
- `busy_timeout`:多进程/多连接下遇短暂写锁自旋等待而非立刻 SQLITE_BUSY。
路径常量仍是本模块的**唯一真源**:DB 访问函数在**调用时**读取本模块的
`DB_PATH` 等全局(测试 monkeypatch `auth_system.db.DB_PATH` 依旧生效)。
"""
import asyncio
import os
import secrets
import sqlite3
import sys
import threading
import time
from collections.abc import AsyncGenerator, Sequence
from contextlib import asynccontextmanager
from pathlib import Path
import aiosqlite
from .security import (
_hash_new,
_normalize_email,
@@ -37,68 +51,136 @@ SESSION_KEY = 'site_uid'
DEFAULT_REGISTER_CREDITS = 3 # 注册直接赠送 3 次下载次数(无需领取;TRIAL_CREDITS 死常量已删)
_DB_LOCK = threading.RLock()
# 锁按事件循环隔离:asyncio.Lock 绑定创建它的 loop,跨 loop 复用会抛
# "bound to a different event loop"(应用主 loop + 测试各自的 loop 并存,必须分开持有)。
_LOOP_LOCKS: dict[tuple[int, str], asyncio.Lock] = {}
def _loop_lock(kind: str) -> asyncio.Lock:
key = (id(asyncio.get_running_loop()), kind)
lock = _LOOP_LOCKS.get(key)
if lock is None:
lock = asyncio.Lock()
_LOOP_LOCKS[key] = lock
return lock
def _write_lock() -> asyncio.Lock:
"""写事务串行锁(每事件循环一个)。"""
return _loop_lock('write')
def _wal_lock() -> asyncio.Lock:
"""WAL 切换专用锁(不能用 _write_lock:execute() 已持它再调
connect→_configure 会自锁死)。"""
return _loop_lock('wal')
# journal_mode=WAL 持久化在库文件头里,按路径设一次即可。每连接重复执行
# 实测多付 ~1ms(新连接首条语句要建 WAL 索引读锁),且并发首连会争排他锁
# 报 database is locked(core/db.py 同款教训)。专用锁 + 二次复查防护。
_WAL_READY: set[str] = set()
_WAL_SETUP_LOCK = threading.Lock()
_WAL_READY_FOR: str | None = None
# ============ 数据库 ============
def get_conn() -> sqlite3.Connection:
key = str(DB_PATH)
# exists() 检查(~10us)必须在 connect 之前:connect 会创建空文件,
# 之后 exists 恒为 True,删库重建的场景就检测不到了
async def _configure(conn: aiosqlite.Connection, force_wal: bool = False) -> None:
global _WAL_READY_FOR
conn.row_factory = aiosqlite.Row
# busy_timeout 先设:让后续语句遇锁自旋等待,而不是立刻报 database is locked
await conn.execute('PRAGMA busy_timeout=15000')
key = str(DB_PATH) # 调用时读取模块全局——测试 monkeypatch 后仍生效
if force_wal or _WAL_READY_FOR != key:
# 切换日志模式需要排他锁,且 `_WAL_READY_FOR` 的检查不是原子的——
# 并发连接须在专用锁内串行完成「切换 + 标记」,并二次复查,否则批量并发
# 连接会同时执行 PRAGMA journal_mode=WAL 争锁(实测 database is locked)
async with _wal_lock():
if force_wal or _WAL_READY_FOR != key:
await conn.execute('PRAGMA journal_mode=WAL')
_WAL_READY_FOR = key
await conn.execute('PRAGMA synchronous=NORMAL') # WAL 下兼顾安全与吞吐
await conn.execute('PRAGMA foreign_keys=ON')
async def connect() -> aiosqlite.Connection:
"""新建一个已配置好的连接(调用方负责 close)。"""
# exists() 必须在 aiosqlite.connect 之前判断:connect 会创建空文件,之后
# exists 恒为 True,库文件被删重建的场景就检测不到了(journal_mode 不持久
# 于新文件,必须重设)。
force_wal = not DB_PATH.exists()
# mkdir 移出 happy path(实测 190us/次):connect 直接失败(目录缺失)
# 才补建重试。库文件被删的场景同样由此兜底重建。
# mkdir 移出 happy path(实测 190us/次):connect 直接失败(目录缺失)才补建
# 重试。库文件被删的场景同样由此兜底重建。用 DB_PATH.parent 而非 DATA_DIR:
# 生产两者相同,测试 monkeypatch 只换 DB_PATH 时兜底仍能命中正确目录
# (core/db.py 用 DATA_DIR 的前提是两处都换)。
try:
conn = sqlite3.connect(str(DB_PATH), timeout=15)
conn = await aiosqlite.connect(str(DB_PATH), timeout=15)
except sqlite3.OperationalError:
# DB_PATH.parent 而非 DATA_DIR:生产两者相同,测试 monkeypatch 只换
# DB_PATH 时兜底仍能命中正确目录
DB_PATH.parent.mkdir(parents=True, exist_ok=True)
conn = sqlite3.connect(str(DB_PATH), timeout=15)
conn.row_factory = sqlite3.Row
if key not in _WAL_READY or force_wal:
# 读路径不持 _DB_LOCK,多线程可能同时首连:锁内串行执行 journal_mode
# 切换(防并发争排他锁报 database is locked)。锁内不复查 DB_PATH.exists()——
# connect 已把文件建出来,exists 恒真;force_wal 是本线程的观测结论,
# 重复执行 PRAGMA 在已是 WAL 的库上幂等且廉价(~2us)
with _WAL_SETUP_LOCK:
conn.execute('PRAGMA journal_mode=WAL')
_WAL_READY.add(key)
conn = await aiosqlite.connect(str(DB_PATH), timeout=15)
await _configure(conn, force_wal)
return conn
def _q(sql: str, params: tuple = ()) -> list[sqlite3.Row]:
# 注:同步时代的公开连接门面 get_conn() 已删除——它在迁移后只剩「转调 connect()」
# 一层空壳且全仓零消费者(测试直接开裸连接,备份用 db.connect())。保留它等于留一条
# 「拿裸连接自己写」的口子,与本次「拆掉同步门面、写路径统一走 _x/write_transaction」
# 的目标相悖。
# ============ 读路径(并发,无锁) ============
async def _q(sql: str, params: Sequence = ()) -> list[aiosqlite.Row]:
"""单语句读:**不持锁**。
WAL 模式下每个连接读到一致性快照,读与写互不阻塞;旧实现对读也加 _DB_LOCK,
WAL 模式下每个连接读到一致性快照,读与写互不阻塞;旧实现对读也加锁,
等于把全部读串行化并被写挡住。此处仅放开单语句读——写事务(_x 与各
`with _DB_LOCK` 块)仍全程串行,故不引入写-写竞争。
`write_transaction` 块)仍全程串行,故不引入写-写竞争。
"""
conn = get_conn()
conn = await connect()
try:
return conn.execute(sql, params).fetchall()
cur = await conn.execute(sql, params)
return list(await cur.fetchall())
finally:
conn.close()
await conn.close()
def _x(sql: str, params: tuple = ()) -> int:
"""执行写操作,返回 lastrowid / rowcount"""
with _DB_LOCK:
conn = get_conn()
# ============ 写路径(串行,不阻塞读) ============
async def _x(sql: str, params: Sequence = ()) -> int:
"""执行单条写语句,返回 lastrowid(INSERT)或 rowcount(UPDATE/DELETE)"""
async with _write_lock():
conn = await connect()
try:
cur = conn.execute(sql, params)
conn.commit()
cur = await conn.execute(sql, params)
await conn.commit()
return cur.lastrowid if cur.lastrowid else cur.rowcount
finally:
conn.close()
await conn.close()
@asynccontextmanager
async def write_transaction() -> AsyncGenerator[aiosqlite.Connection, None]:
"""写事务上下文:多条写语句原子提交(串行锁内)。
用法:
async with write_transaction() as conn:
await conn.execute(...)
await conn.execute(...)
# 退出即 commit;异常则 rollback
"""
async with _write_lock():
conn = await connect()
try:
yield conn
await conn.commit()
except Exception:
await conn.rollback()
raise
finally:
await conn.close()
_SCHEMA_SQL = '''
@@ -148,30 +230,30 @@ _SCHEMA_SQL = '''
'''
def _create_schema(conn: sqlite3.Connection) -> None:
async def _create_schema(conn: aiosqlite.Connection) -> None:
"""建表 + 单行配置种子行(幂等:全部 IF NOT EXISTS / INSERT OR IGNORE)。"""
conn.executescript(_SCHEMA_SQL)
await conn.executescript(_SCHEMA_SQL)
# 单行配置表
conn.execute('''CREATE TABLE IF NOT EXISTS smtp_config (
await conn.execute('''CREATE TABLE IF NOT EXISTS smtp_config (
id INTEGER PRIMARY KEY CHECK (id = 1),
host TEXT DEFAULT '', port INTEGER DEFAULT 587,
sender TEXT DEFAULT '', password TEXT DEFAULT '',
updated_at INTEGER DEFAULT 0
)''')
conn.execute('INSERT OR IGNORE INTO smtp_config (id, port) VALUES (1, 587)')
await conn.execute('INSERT OR IGNORE INTO smtp_config (id, port) VALUES (1, 587)')
# 键值设置表
conn.execute('''CREATE TABLE IF NOT EXISTS settings (
await conn.execute('''CREATE TABLE IF NOT EXISTS settings (
key TEXT PRIMARY KEY, value TEXT DEFAULT ''
)''')
conn.execute('''CREATE TABLE IF NOT EXISTS announcements (
await conn.execute('''CREATE TABLE IF NOT EXISTS announcements (
id INTEGER PRIMARY KEY CHECK (id = 1),
title TEXT DEFAULT '', content TEXT DEFAULT '',
popup_mode TEXT DEFAULT 'off',
updated_at INTEGER DEFAULT 0
)''')
conn.execute('INSERT OR IGNORE INTO announcements (id) VALUES (1)')
await conn.execute('INSERT OR IGNORE INTO announcements (id) VALUES (1)')
# 计费账本(任务实扣/交付/退款/终态的持久化唯一记录,结算幂等)
conn.execute('''CREATE TABLE IF NOT EXISTS billing_ledger (
await conn.execute('''CREATE TABLE IF NOT EXISTS billing_ledger (
task_id TEXT PRIMARY KEY,
user_id INTEGER NOT NULL,
reserved INTEGER DEFAULT 0,
@@ -183,11 +265,13 @@ def _create_schema(conn: sqlite3.Connection) -> None:
)''')
def _apply_column_migrations(conn: sqlite3.Connection) -> None:
async def _apply_column_migrations(conn: aiosqlite.Connection) -> None:
"""轻量迁移:新增列。仅"列已存在"可忽略(其余迁移失败必须暴露,不能吞掉)"""
_already: set[str] = set()
for trow in conn.execute("SELECT name FROM sqlite_master WHERE type='table'").fetchall():
for crow in conn.execute(f"PRAGMA table_info({trow['name']})").fetchall():
trows = await (await conn.execute("SELECT name FROM sqlite_master WHERE type='table'")).fetchall()
for trow in trows:
crows = await (await conn.execute(f"PRAGMA table_info({trow['name']})")).fetchall()
for crow in crows:
_already.add(f"{trow['name']}.{crow['name']}")
for alter_sql, tbl_col in (
('ALTER TABLE users ADD COLUMN status INTEGER DEFAULT 1', 'users.status'),
@@ -202,82 +286,87 @@ def _apply_column_migrations(conn: sqlite3.Connection) -> None:
if tbl_col in _already:
continue # 幂等:列已存在
try:
conn.execute(alter_sql)
await conn.execute(alter_sql)
except sqlite3.OperationalError as e:
if 'duplicate column' not in str(e).lower():
# 磁盘/锁/结构等真实故障:抛出,禁止静默带病初始化
raise
def _ensure_email_unique_index(conn: sqlite3.Connection) -> None:
async def _ensure_email_unique_index(conn: aiosqlite.Connection) -> None:
"""非空邮箱唯一(部分索引,不约束历史空串)。
存量库若已有重复非空邮箱,索引创建失败仅告警不阻断启动
(查重逻辑在注册/建号路由另有兜底)。
"""
try:
conn.execute('CREATE UNIQUE INDEX IF NOT EXISTS idx_users_email_nonempty '
await conn.execute('CREATE UNIQUE INDEX IF NOT EXISTS idx_users_email_nonempty '
'ON users(email) WHERE email != \'\'')
except sqlite3.IntegrityError:
print('[警告] users.email 存在重复的非空邮箱,唯一索引未创建(请人工清理后重启)')
def _migrate_user_emails(conn: sqlite3.Connection) -> None:
async def _migrate_user_emails(conn: aiosqlite.Connection) -> None:
"""users.email 口径统一为归一化(lower + 剥 Gmail +tag)——与
get_user_by_email/找回/邮箱登录的查询口径对齐。逐行迁移,归一化后与他
人撞唯一索引的行保留原值跳过(与"存量重复仅告警"同口径)。幂等:已
归一化的行零写入。
"""
for urow in conn.execute("SELECT id, email FROM users WHERE email != ''").fetchall():
urows = await (await conn.execute("SELECT id, email FROM users WHERE email != ''")).fetchall()
for urow in urows:
fixed = _normalize_email(urow['email'])
if not fixed or fixed == urow['email']:
continue
try:
conn.execute('UPDATE users SET email = ? WHERE id = ?', (fixed, urow['id']))
await conn.execute('UPDATE users SET email = ? WHERE id = ?', (fixed, urow['id']))
except sqlite3.IntegrityError:
print(f"[警告] 用户 {urow['id']} 的邮箱归一化后与他人重复,保留原值")
def _ensure_initial_admin(conn: sqlite3.Connection) -> None:
async def _ensure_initial_admin(conn: aiosqlite.Connection) -> None:
"""初始管理员(新装不再使用公开默认口令,随机生成并写入日志一次)。
引导条件改为"不存在任何管理员角色"——按用户名查会在
管理员改名(如 admin→qfbsh)后误判缺失,重启即重建出一个新的 admin 账号。
"""
row = conn.execute("SELECT id FROM users WHERE role = 'admin' LIMIT 1").fetchone()
row = await (await conn.execute("SELECT id FROM users WHERE role = 'admin' LIMIT 1")).fetchone()
if row:
return
initial_pass = secrets.token_urlsafe(12)
conn.execute(
await conn.execute(
'INSERT INTO users (username, password_hash, salt, role, duration_seconds, created_at) '
'VALUES (?, ?, ?, ?, -1, ?)',
(DEFAULT_ADMIN_USER, _hash_new(initial_pass), '',
'admin', int(time.time())))
# ——非 ASCII 输出须防 GBK 重定向下 UnicodeEncodeError
# ——非 ASCII 输出须防 GBK 重定向下的 UnicodeEncodeError
try:
print(f'[初始化] 已创建管理员账号 admin,初始密码:{initial_pass}(请立即登录并修改,本提示不再显示)')
except UnicodeEncodeError:
print('[init] admin account created, initial password printed above failed to encode')
def init_db() -> None:
async def _init_db_locked(conn: aiosqlite.Connection) -> None:
"""在**已持有写锁**的连接上执行全部结构演进(不自行取锁)。
restore_backup 需要在自己的写锁临界区里对新库跑一遍初始化,不能调用会
再次取写锁的 init_db(asyncio.Lock 非重入,会自锁死),故拆出本函数。
"""
await _create_schema(conn)
await _apply_column_migrations(conn)
await _ensure_email_unique_index(conn)
await _migrate_user_emails(conn)
await _ensure_initial_admin(conn)
async def init_db() -> None:
"""初始化表结构与默认数据"""
DATA_DIR.mkdir(parents=True, exist_ok=True)
with _DB_LOCK:
conn = get_conn()
try:
_create_schema(conn)
_apply_column_migrations(conn)
_ensure_email_unique_index(conn)
_migrate_user_emails(conn)
_ensure_initial_admin(conn)
conn.commit()
finally:
conn.close()
DATA_DIR.mkdir(parents=True, exist_ok=True) # noqa: ASYNC240 - 启动期一次的目录创建,无需线程池
async with write_transaction() as conn:
await _init_db_locked(conn)
def get_secret_key() -> str:
"""获取(或生成)持久化的会话签名密钥(原子写入 + 失败显式告警)"""
def _get_secret_key_sync() -> str:
"""同步实现(见 get_secret_key 的异步包装):原子写入 + 失败显式告警"""
key = ''
try:
if SECRET_KEY_PATH.exists():
@@ -303,3 +392,13 @@ def get_secret_key() -> str:
except UnicodeEncodeError:
print('[warn] secret key persistence failed; using temporary key for this process', file=sys.stderr)
return new_key
async def get_secret_key() -> str:
"""获取(或生成)持久化的会话签名密钥(原子写入 + 失败显式告警)。
内部是**同步阻塞文件 IO**(exists/read/write/os.replace),整段经
`asyncio.to_thread` 卸载到线程——这是正当的阻塞 IO 卸载(登录链路每次
签会话都要读一次密钥),不是把异步包装成同步的门面。
"""
return await asyncio.to_thread(_get_secret_key_sync)
+21 -11
View File
@@ -1,13 +1,15 @@
# -*- coding: utf-8 -*-
"""登录 IP 归属地探测与异地自动封禁。
(自 auth_system.py 拆出,行为与原实现一致。)
(自 auth_system.py 拆出,行为与原实现一致;DB 访问已迁移为原生异步。)
"""
import json
import sqlite3
import time
from core.http_client import client as _http
from .db import (
_x,
)
@@ -31,9 +33,13 @@ def _is_private_ip(ip: str) -> bool:
return False
def query_ip_province(ip: str) -> str:
async def query_ip_province(ip: str) -> str:
"""查询 IP 归属省份(内网/失败返回空串),带进程内缓存
改用官方路径 /json/{query} 与 zh-CN 语言参数"""
改用官方路径 /json/{query} 与 zh-CN 语言参数。
走 core.http_client 的原生异步通道(原实现经 core.sync_http 同步门面
在后台事件循环桥接——调用方本就是 async 的登录链路,无需再桥一次)。
"""
ip = (ip or '').split(',')[0].strip()
if not ip or ip in ('-', '127.0.0.1', '0.0.0.0') or _is_private_ip(ip):
return ''
@@ -43,8 +49,8 @@ def query_ip_province(ip: str) -> str:
try:
import urllib.parse as _ul
from core import sync_http as _rq
r = _rq.get('http://ip-api.com/json/' + _ul.quote(ip),
r = await _http.request(
'GET', 'http://ip-api.com/json/' + _ul.quote(ip),
params={'lang': 'zh-CN', 'fields': 'status,regionName'},
timeout=5)
d = r.json()
@@ -67,12 +73,16 @@ PROVINCE_WINDOW = 24 * 3600 # 省份变化检测窗口:24 小时
PROVINCE_BAN_LIMIT = 3 # 窗口内出现 >=3 个不同省份即封禁
def record_login_location(user: sqlite3.Row, ip: str) -> tuple:
async def record_login_location(user: sqlite3.Row, ip: str) -> tuple:
"""记录本次登录省份并检测异常:24h 内出现 >=3 个不同省份自动封禁。
返回 (是否被封禁, 说明)"""
返回 (是否被封禁, 说明)
落库走原生异步通道(auth_system 已迁移为异步 DB 层),直接 await,
不再经 asyncio.to_thread 桥接。
"""
if user['role'] == 'admin':
return False, ''
province = query_ip_province(ip)
province = await query_ip_province(ip)
if not province:
return False, ''
now = int(time.time())
@@ -86,11 +96,11 @@ def record_login_location(user: sqlite3.Row, ip: str) -> tuple:
for h in history:
if h.get('p') and h['p'] not in provinces:
provinces.append(h['p'])
_x('UPDATE users SET recent_provinces = ? WHERE id = ?',
await _x('UPDATE users SET recent_provinces = ? WHERE id = ?',
(json.dumps(history, ensure_ascii=False), user['id']))
if len(provinces) >= PROVINCE_BAN_LIMIT:
reason = f'24小时内检测到 {len(provinces)} 个不同省份登录({"/".join(provinces[-3:])}),疑似账号倒卖,已自动封禁'
ban_user(user['id'], reason)
add_log('fail', user['username'], ip, '自动封禁', reason)
await ban_user(user['id'], reason)
await add_log('fail', user['username'], ip, '自动封禁', reason)
return True, reason
return False, province
+39 -68
View File
@@ -1,17 +1,16 @@
# -*- coding: utf-8 -*-
"""预扣账本(开账 / 改名 / 补扣 / 结算 / 对账)。
(自 auth_system.py 拆出,行为与原实现一致。)
(自 auth_system.py 拆出,行为与原实现一致;DB 访问已迁移为原生异步。)
"""
import time
from typing import Any
from .db import (
_DB_LOCK,
_q,
_x,
get_conn,
write_transaction,
)
from .logs import (
add_log,
@@ -20,80 +19,62 @@ from .logs import (
# ============ 计费账本(实扣、交付、退款、终态的持久化唯一记录) ============
def ledger_open(task_id: str, user_id: int, reserved: int) -> None:
async def ledger_open(task_id: str, user_id: int, reserved: int) -> None:
"""任务计费开账:记录本任务实际预扣次数(admin/-1 用户为 0)。
task_id 唯一;重复 task_id 由任务系统在提交侧拒绝,这里幂等保护。"""
with _DB_LOCK:
conn = get_conn()
try:
conn.execute(
await _x(
'INSERT OR IGNORE INTO billing_ledger (task_id, user_id, reserved, status, created_at) '
'VALUES (?, ?, ?, \'open\', ?)',
(str(task_id), int(user_id), max(0, int(reserved)), int(time.time())))
conn.commit()
finally:
conn.close()
def reserve_and_open(uid: int, count: int, ledger_key: str) -> int | None:
async def reserve_and_open(uid: int, count: int, ledger_key: str) -> int | None:
"""预扣与开账同一事务(原两步之间存在崩溃丢账窗口)。
足额:扣减+开账并返回剩余;不限/管理员:开账(reserved=0)返回 -1;不足:返回 None(不开账)。"""
count = max(1, int(count))
key = str(ledger_key)
with _DB_LOCK:
conn = get_conn()
try:
row = conn.execute('SELECT role, duration_seconds FROM users WHERE id = ?', (uid,)).fetchone()
async with write_transaction() as conn:
row = await (await conn.execute('SELECT role, duration_seconds FROM users WHERE id = ?', (uid,))).fetchone()
if not row:
return None
now = int(time.time())
if row['role'] == 'admin':
conn.execute('UPDATE users SET last_active = ? WHERE id = ?', (now, uid))
conn.execute(
await conn.execute('UPDATE users SET last_active = ? WHERE id = ?', (now, uid))
await conn.execute(
'INSERT OR IGNORE INTO billing_ledger (task_id, user_id, reserved, status, created_at) '
'VALUES (?, ?, 0, \'open\', ?)', (key, int(uid), now))
conn.commit()
return -1
cur = int(row['duration_seconds'] or 0)
if cur == -1:
conn.execute('UPDATE users SET last_active = ? WHERE id = ?', (now, uid))
conn.execute(
await conn.execute('UPDATE users SET last_active = ? WHERE id = ?', (now, uid))
await conn.execute(
'INSERT OR IGNORE INTO billing_ledger (task_id, user_id, reserved, status, created_at) '
'VALUES (?, ?, 0, \'open\', ?)', (key, int(uid), now))
conn.commit()
return -1
if cur < count:
return None
conn.execute('UPDATE users SET duration_seconds = ?, last_active = ? WHERE id = ?',
await conn.execute('UPDATE users SET duration_seconds = ?, last_active = ? WHERE id = ?',
(cur - count, now, uid))
conn.execute(
await conn.execute(
'INSERT OR IGNORE INTO billing_ledger (task_id, user_id, reserved, status, created_at) '
'VALUES (?, ?, ?, \'open\', ?)', (key, int(uid), count, now))
conn.commit()
return cur - count
finally:
conn.close()
def ledger_rename(old_key: str, new_key: str) -> bool:
async def ledger_rename(old_key: str, new_key: str) -> bool:
"""把请求级临时账本行改名为正式 task_id(任务接手账目)。
old 不存在/已关闭/new 已存在 → False。"""
with _DB_LOCK:
conn = get_conn()
try:
dup = conn.execute('SELECT 1 FROM billing_ledger WHERE task_id = ?', (str(new_key),)).fetchone()
async with write_transaction() as conn:
dup = await (await conn.execute('SELECT 1 FROM billing_ledger WHERE task_id = ?', (str(new_key),))).fetchone()
if dup:
return False
cur = conn.execute(
cur = await conn.execute(
'UPDATE billing_ledger SET task_id = ? WHERE task_id = ? AND status = \'open\'',
(str(new_key), str(old_key)))
conn.commit()
return cur.rowcount == 1
finally:
conn.close()
def ledger_supplement(task_id: str, extra: int) -> bool:
async def ledger_supplement(task_id: str, extra: int) -> bool:
"""整台播客等"先扣 1 后按实际数量补扣"场景:
原子补扣用户余额并同步累加账本 reserved。任一步失败返回 False(调用方必须让任务失败,
不允许无账本继续交付)。余额不足同样返回 False。"""
@@ -101,39 +82,34 @@ def ledger_supplement(task_id: str, extra: int) -> bool:
extra = int(extra)
if extra <= 0:
return True
with _DB_LOCK:
conn = get_conn()
try:
row = conn.execute('SELECT user_id, status FROM billing_ledger WHERE task_id = ?',
(task_id,)).fetchone()
async with write_transaction() as conn:
row = await (await conn.execute('SELECT user_id, status FROM billing_ledger WHERE task_id = ?',
(task_id,))).fetchone()
if not row or row['status'] != 'open':
return False
urow = conn.execute('SELECT role, duration_seconds FROM users WHERE id = ?',
(row['user_id'],)).fetchone()
urow = await (await conn.execute('SELECT role, duration_seconds FROM users WHERE id = ?',
(row['user_id'],))).fetchone()
if not urow or urow['role'] == 'admin' or int(urow['duration_seconds'] or 0) == -1:
return True # 不限用户:无需补扣,账本也不加
cur_credits = int(urow['duration_seconds'] or 0)
if cur_credits < extra:
return False # 余额不足:任务必须失败(不允许扣到 0 硬做)
conn.execute('UPDATE users SET duration_seconds = duration_seconds - ? WHERE id = ?',
await conn.execute('UPDATE users SET duration_seconds = duration_seconds - ? WHERE id = ?',
(extra, row['user_id']))
conn.execute('UPDATE billing_ledger SET reserved = reserved + ? WHERE task_id = ?',
await conn.execute('UPDATE billing_ledger SET reserved = reserved + ? WHERE task_id = ?',
(extra, task_id))
conn.commit()
return True
except Exception:
conn.rollback()
return False
finally:
conn.close()
def ledger_get(task_id: str) -> dict[str, Any] | None:
rows = _q('SELECT * FROM billing_ledger WHERE task_id = ?', (str(task_id),))
async def ledger_get(task_id: str) -> dict[str, Any] | None:
rows = await _q('SELECT * FROM billing_ledger WHERE task_id = ?', (str(task_id),))
return dict(rows[0]) if rows else None
def ledger_settle(task_id: str, delivered: int, final_status: str) -> int:
async def ledger_settle(task_id: str, delivered: int, final_status: str) -> int:
"""任务计费结算(幂等):按账本实扣与实际交付计算应退次数,原子退款并落终态。
- delivered:本次任务实际交付给用户的数量(单曲/单集 0 或 1;批量为成功入包数)
- 重复调用/重启后再次调用:账本已非 open 直接返回 0,不重复退款
@@ -143,10 +119,8 @@ def ledger_settle(task_id: str, delivered: int, final_status: str) -> int:
返回实际退款次数(落余额的部分;-1/admin 场景为 0)。"""
task_id = str(task_id)
delivered = max(0, int(delivered))
with _DB_LOCK:
conn = get_conn()
try:
row = conn.execute('SELECT * FROM billing_ledger WHERE task_id = ?', (task_id,)).fetchone()
async with write_transaction() as conn:
row = await (await conn.execute('SELECT * FROM billing_ledger WHERE task_id = ?', (task_id,))).fetchone()
if not row or row['status'] != 'open':
return 0
refund = max(0, int(row['reserved']) - delivered)
@@ -156,45 +130,42 @@ def ledger_settle(task_id: str, delivered: int, final_status: str) -> int:
# 账本 refunded 记录的是"按账本应付"口径(该用户已有无限次数,作废可接受);
# 原注释"照样退"与实现相反,已更正为此口径。
if refund > 0:
conn.execute('UPDATE users SET duration_seconds = duration_seconds + ? '
await conn.execute('UPDATE users SET duration_seconds = duration_seconds + ? '
'WHERE id = ? AND role != \'admin\' AND duration_seconds != -1',
(refund, row['user_id']))
conn.execute('UPDATE billing_ledger SET delivered = ?, refunded = ?, status = ?, '
await conn.execute('UPDATE billing_ledger SET delivered = ?, refunded = ?, status = ?, '
'settled_at = ? WHERE task_id = ?',
(delivered, refund, str(final_status)[:20], int(time.time()), task_id))
conn.commit()
return refund
finally:
conn.close()
def ledger_recover_orphans(max_age_seconds: int = 2 * 3600) -> int:
async def ledger_recover_orphans(max_age_seconds: int = 2 * 3600) -> int:
"""启动对账:任务表在内存,进程重启后账本中仍 open 的行代表
"已扣款但任务已丢失"——退回全部实扣。仅处理超过宽限期的行,
避免误伤同库其他存活进程的进行中任务。返回退款笔数。"""
now = int(time.time())
recovered = 0
rows = _q('SELECT task_id FROM billing_ledger WHERE status = \'open\' AND created_at <= ?',
rows = await _q('SELECT task_id FROM billing_ledger WHERE status = \'open\' AND created_at <= ?',
(now - int(max_age_seconds),))
for r in rows:
try:
refunded = ledger_settle(r['task_id'], 0, 'lost_restart')
refunded = await ledger_settle(r['task_id'], 0, 'lost_restart')
if refunded:
add_log('success', 'system', 'system', '重启对账退款',
await add_log('success', 'system', 'system', '重启对账退款',
f'任务 {r["task_id"]} 因进程重启丢失,退回 {refunded} 次')
else:
ledger_settle(r['task_id'], 0, 'lost_restart')
await ledger_settle(r['task_id'], 0, 'lost_restart')
recovered += 1
except Exception:
pass
return recovered
def ledger_cleanup_settled(max_age_seconds: int = 90 * 86400) -> int:
async def ledger_cleanup_settled(max_age_seconds: int = 90 * 86400) -> int:
"""清理已结算的历史账本行(每任务一行、只增不删会让表无限膨胀;
settled 状态行无对账价值,保留 90 天足够追溯)"""
try:
return _x('DELETE FROM billing_ledger WHERE status != \'open\' AND settled_at <= ?',
return await _x('DELETE FROM billing_ledger WHERE status != \'open\' AND settled_at <= ?',
(int(time.time()) - int(max_age_seconds),))
except Exception:
return 0
+11 -17
View File
@@ -1,16 +1,15 @@
# -*- coding: utf-8 -*-
"""后台审计日志。
(自 auth_system.py 拆出,行为与原实现一致。)
(自 auth_system.py 拆出,行为与原实现一致;DB 访问已迁移为原生异步。)
"""
import time
from .db import (
_DB_LOCK,
_q,
_x,
get_conn,
write_transaction,
)
# ============ 日志 ============
@@ -20,37 +19,32 @@ from .db import (
_LOG_WRITE_COUNTER = {'n': 0}
def add_log(log_type: str, username: str, ip: str, action: str, detail: str = '') -> None:
async def add_log(log_type: str, username: str, ip: str, action: str, detail: str = '') -> None:
try:
_x('INSERT INTO logs (type, username, ip, action, detail, time) VALUES (?, ?, ?, ?, ?, ?)',
await _x('INSERT INTO logs (type, username, ip, action, detail, time) VALUES (?, ?, ?, ?, ?, ?)',
(log_type, username or '', ip or '-', action or '', detail or '',
time.strftime('%Y-%m-%d %H:%M:%S')))
# 控制日志表体积(每 50 次写入触发一次裁剪)
_LOG_WRITE_COUNTER['n'] += 1
if _LOG_WRITE_COUNTER['n'] % 50 == 0:
_x('DELETE FROM logs WHERE id NOT IN (SELECT id FROM logs ORDER BY id DESC LIMIT 5000)')
await _x('DELETE FROM logs WHERE id NOT IN (SELECT id FROM logs ORDER BY id DESC LIMIT 5000)')
except Exception:
pass
def query_logs(log_type: str, limit: int = 100, offset: int = 0):
async def query_logs(log_type: str, limit: int = 100, offset: int = 0):
"""分页查询日志,返回 (日志列表, 总条数)"""
limit = max(1, min(int(limit), 500))
offset = max(0, int(offset))
rows = _q('SELECT * FROM logs WHERE type = ? ORDER BY id DESC LIMIT ? OFFSET ?',
rows = await _q('SELECT * FROM logs WHERE type = ? ORDER BY id DESC LIMIT ? OFFSET ?',
(log_type, limit, offset))
total_row = _q('SELECT COUNT(*) AS c FROM logs WHERE type = ?', (log_type,))
total_row = await _q('SELECT COUNT(*) AS c FROM logs WHERE type = ?', (log_type,))
total = int(total_row[0]['c']) if total_row else 0
return [dict(r) for r in rows], total
def clear_logs(log_type: str) -> int:
async def clear_logs(log_type: str) -> int:
"""清除指定类型的全部日志,返回删除条数"""
with _DB_LOCK:
conn = get_conn()
try:
cur = conn.execute('DELETE FROM logs WHERE type = ?', (log_type,))
conn.commit()
async with write_transaction() as conn:
cur = await conn.execute('DELETE FROM logs WHERE type = ?', (log_type,))
return cur.rowcount
finally:
conn.close()
+67 -65
View File
@@ -1,13 +1,14 @@
# -*- coding: utf-8 -*-
"""SMTP 配置、发信与邮箱验证码。
(自 auth_system.py 拆出,行为与原实现一致。)
(自 auth_system.py 拆出,行为与原实现一致;DB 访问与 SMTP 发信均已迁移为
原生异步(aiosqlite / aiosmtplib),不再有阻塞调用藏在 to_thread 里。)
"""
import base64
import contextlib
import re
import secrets
import smtplib
import sqlite3
import ssl
import time
@@ -16,12 +17,13 @@ from email.mime.text import MIMEText
from email.utils import formataddr
from typing import Any
import aiosmtplib
from .db import (
_DB_LOCK,
_q,
_x,
get_conn,
get_secret_key,
write_transaction,
)
from .security import (
_normalize_email,
@@ -46,81 +48,96 @@ def _code_fails_trim() -> None:
_SMTP_ENC_PREFIX = 'enc1:' # 信封加密标记前缀(存量明文读取时透明升级)
def _smtp_password_encrypt(plaintext: str) -> str:
async def _smtp_password_encrypt(plaintext: str) -> str:
"""用 data/secret.key 派生 Fernet 密钥加密 SMTP 授权码。
原实现明文落库、并随数据库备份一起导出,等于扩大泄露半径。"""
import hashlib as _hl
from cryptography.fernet import Fernet
key = _hl.sha256(get_secret_key().encode('utf-8')).digest()
key = _hl.sha256((await get_secret_key()).encode('utf-8')).digest()
f = Fernet(base64.urlsafe_b64encode(key))
return _SMTP_ENC_PREFIX + f.encrypt(plaintext.encode('utf-8')).decode('utf-8')
def _smtp_password_decrypt(ciphertext: str) -> str:
async def _smtp_password_decrypt(ciphertext: str) -> str:
if not ciphertext.startswith(_SMTP_ENC_PREFIX):
return ciphertext # 存量明文:透明兼容,下次保存时升级为密文
import hashlib as _hl
from cryptography.fernet import Fernet
key = _hl.sha256(get_secret_key().encode('utf-8')).digest()
key = _hl.sha256((await get_secret_key()).encode('utf-8')).digest()
f = Fernet(base64.urlsafe_b64encode(key))
return f.decrypt(ciphertext[len(_SMTP_ENC_PREFIX):].encode('utf-8')).decode('utf-8')
def get_smtp_config() -> dict[str, Any]:
rows = _q('SELECT * FROM smtp_config WHERE id = 1')
async def get_smtp_config() -> dict[str, Any]:
rows = await _q('SELECT * FROM smtp_config WHERE id = 1')
cfg = dict(rows[0]) if rows else {'host': '', 'port': 587, 'sender': '', 'password': ''}
# 读取时解密密码——调用方(SMTP 连接/后台展示)拿到的始终是明文
if cfg.get('password'):
try:
cfg['password'] = _smtp_password_decrypt(cfg['password'])
cfg['password'] = await _smtp_password_decrypt(cfg['password'])
except Exception:
cfg['password'] = ''
return cfg
def save_smtp_config(host: str, port: int, sender: str, password: str) -> None:
enc_pwd = _smtp_password_encrypt(password) if password else ''
_x('UPDATE smtp_config SET host = ?, port = ?, sender = ?, password = ?, updated_at = ? WHERE id = 1',
async def save_smtp_config(host: str, port: int, sender: str, password: str) -> None:
enc_pwd = await _smtp_password_encrypt(password) if password else ''
await _x('UPDATE smtp_config SET host = ?, port = ?, sender = ?, password = ?, updated_at = ? WHERE id = 1',
(host, int(port or 587), sender, enc_pwd, int(time.time())))
def smtp_configured() -> bool:
cfg = get_smtp_config()
async def smtp_configured() -> bool:
cfg = await get_smtp_config()
return bool(cfg['host'] and cfg['sender'] and cfg['password'])
def _smtp_connect() -> smtplib.SMTP:
cfg = get_smtp_config()
async def _smtp_send(cfg: dict[str, Any], sender: str, to: str, raw: str) -> None:
"""原生异步发信(aiosmtplib)。
465 走隐式 TLS(use_tls),其余端口走显式 STARTTLS(start_tls)——与原先
smtplib 的 SMTP_SSL / starttls 分支一一对应;两条路都用显式构造的 TLS 上下文
校验证书与主机名,不依赖各运行时默认值。原实现是 smtplib 同步栈再经
asyncio.to_thread 卸载,换成 aiosmtplib 后不再占用线程池。
"""
port = int(cfg['port'] or 587)
# 显式校验证书与主机名的 TLS 上下文(不依赖各运行时默认值)
ctx = ssl.create_default_context()
if port == 465:
server = smtplib.SMTP_SSL(cfg['host'], port, timeout=15, context=ctx)
else:
server = smtplib.SMTP(cfg['host'], port, timeout=15)
server.starttls(context=ctx)
server.login(cfg['sender'], cfg['password'])
return server
implicit_tls = port == 465
smtp = aiosmtplib.SMTP(
hostname=cfg['host'],
port=port,
timeout=15,
use_tls=implicit_tls,
start_tls=not implicit_tls,
tls_context=ctx,
)
await smtp.connect()
try:
await smtp.login(cfg['sender'], cfg['password'])
await smtp.sendmail(sender, [to], raw)
finally:
# 关连接失败不影响「信已发出」的事实,也绝不能盖掉上面的真实异常
with contextlib.suppress(Exception):
await smtp.quit()
def send_email(to: str, subject: str, body: str) -> None:
async def send_email(to: str, subject: str, body: str) -> None:
"""收件人格式校验下沉到本入口——未来任何新调用方漏校验也不再成为
CRLF/信封注入点(现有调用点的校验仅作 UX 提示)。"""
to = _normalize_email(to)
if not re.match(r'^[\w.%+-]+@[\w.-]+\.[A-Za-z]{2,}$', to, re.ASCII):
raise ValueError('收件邮箱格式不正确')
cfg = get_smtp_config()
cfg = await get_smtp_config()
msg = MIMEText(body, 'plain', 'utf-8')
# email 库推荐写法:Header 对象由 Generator._fold 的 Header 分支做 RFC 2047 编码
# (typeshed 把 Message.__setitem__ 的值参数收窄成 str,属桩过窄;运行时已实测正确)
msg['Subject'] = Header(subject, 'utf-8') # type: ignore[assignment]
msg['From'] = formataddr(('云村小助手', cfg['sender']))
msg['To'] = to
with _smtp_connect() as server:
server.sendmail(cfg['sender'], [to], msg.as_string())
# SMTP 已是原生异步(aiosmtplib),直接 await——不再有同步网络 IO 需要卸载
await _smtp_send(cfg, cfg['sender'], to, msg.as_string())
_EMAIL_SENT_24H: dict = {} # 同邮箱 24h 发送计数(内存态,重启清零可接受)
@@ -152,7 +169,7 @@ _CODE_EMAIL_TEMPLATES = {
}
def send_verify_code(email: str, purpose: str = 'register') -> None:
async def send_verify_code(email: str, purpose: str = 'register') -> None:
"""生成并发送邮箱验证码(10 分钟有效)(先发信成功再落库;同用途旧码即时作废)。
① 同邮箱 24h 硬上限 5 封(防邮件轰炸代理/发信配额消耗);
② email 先归一化(Gmail +tag 别名剥离)——别名刷号被邮箱查重堵死。
@@ -166,7 +183,7 @@ def send_verify_code(email: str, purpose: str = 'register') -> None:
# 原实现 5000 字符邮箱可入库)
if len(email) > 254 or not re.match(r'^[\w.%+-]+@[\w.-]+\.[A-Za-z]{2,}$', email, re.ASCII):
raise ValueError('邮箱格式不正确')
if not smtp_configured():
if not await smtp_configured():
raise RuntimeError('邮箱服务未配置,请联系管理员')
# 同邮箱 24h ≤5 封(内存计数,不再依赖会被消费 DELETE 清零的 DB 行数);
# admin2fa 豁免 24h 上限与 60s 冷却(见 _CODE_PURPOSES 处注释)
@@ -176,14 +193,14 @@ def send_verify_code(email: str, purpose: str = 'register') -> None:
raise RuntimeError('该邮箱验证码发送已达上限(24 小时 5 封),请明日再试')
# 60 秒内限制重发(按同邮箱同用途的最近一封计)
if purpose != 'admin2fa':
rows = _q('SELECT created_at FROM email_codes WHERE email = ? AND purpose = ? ORDER BY id DESC LIMIT 1',
rows = await _q('SELECT created_at FROM email_codes WHERE email = ? AND purpose = ? ORDER BY id DESC LIMIT 1',
(email, purpose))
if rows and time.time() - rows[0]['created_at'] < 60:
raise RuntimeError('发送过于频繁,请 1 分钟后再试')
code = f'{secrets.randbelow(1000000):06d}'
subject, body_tpl = _CODE_EMAIL_TEMPLATES[purpose]
# 先发送:SMTP 失败不留任何有效验证码与冷却记录;成功后才计入 24h 额度(admin2fa 不计数)
send_email(email, subject, body_tpl.format(code=code))
await send_email(email, subject, body_tpl.format(code=code))
if purpose != 'admin2fa':
sent.append(now_ts)
_EMAIL_SENT_24H[email] = sent
@@ -195,20 +212,15 @@ def send_verify_code(email: str, purpose: str = 'register') -> None:
if len(_EMAIL_SENT_24H) > 5000:
for k in list(_EMAIL_SENT_24H.keys())[:len(_EMAIL_SENT_24H) // 2]:
_EMAIL_SENT_24H.pop(k, None)
with _DB_LOCK:
conn = get_conn()
try:
async with write_transaction() as conn:
# 新码生效同时作废同邮箱同用途旧码;顺手清理全表过期码
conn.execute('DELETE FROM email_codes WHERE (email = ? AND purpose = ?) OR expires_at <= ?',
await conn.execute('DELETE FROM email_codes WHERE (email = ? AND purpose = ?) OR expires_at <= ?',
(email, purpose, int(time.time())))
conn.execute('INSERT INTO email_codes (email, code, expires_at, created_at, purpose) VALUES (?, ?, ?, ?, ?)',
await conn.execute('INSERT INTO email_codes (email, code, expires_at, created_at, purpose) VALUES (?, ?, ?, ?, ?)',
(email, code, int(time.time() + 600), int(time.time()), purpose))
conn.commit()
finally:
conn.close()
def check_verify_code(email: str, code: str, purpose: str = 'register') -> int:
async def check_verify_code(email: str, code: str, purpose: str = 'register') -> int:
"""校验验证码(单条 DELETE 原子消费,rowcount=1 才算通过,杜绝并发双读)。
成功返回剩余有效秒数(真值),失败返回 0。
同一邮箱累计错 5 次→该用途验证码作废(防 10 分钟窗口内挂机猜码)。
@@ -217,33 +229,29 @@ def check_verify_code(email: str, code: str, purpose: str = 'register') -> int:
return 0
email = _normalize_email(email or '')
fail_key = purpose + ':' + email # 失败计数按"用途:邮箱"隔离,互不连带
with _DB_LOCK:
conn = get_conn()
row = None
try:
cur = conn.execute('DELETE FROM email_codes WHERE email = ? AND code = ? AND purpose = ? AND expires_at > ? '
async with write_transaction() as conn:
cur = await conn.execute('DELETE FROM email_codes WHERE email = ? AND code = ? AND purpose = ? AND expires_at > ? '
'RETURNING expires_at',
(email, (code or '').strip(), purpose, int(time.time())))
row = cur.fetchone()
conn.commit()
if row:
_CODE_FAILS.pop(fail_key, None)
return max(0, int(row[0]) - int(time.time()))
row = await cur.fetchone()
except sqlite3.OperationalError as _oe:
# 老版本 SQLite 无 RETURNING:回退为删除后判 rowcount。
# 仅"不支持 RETURNING"走回退——库忙(database is locked)原样
# 上抛,否则第一次等满 busy 15s、回退再等 15s,全程持锁冻结全站写路径
if 'RETURNING' not in str(_oe) and 'syntax' not in str(_oe).lower():
raise
conn.rollback()
cur = conn.execute('DELETE FROM email_codes WHERE email = ? AND code = ? AND purpose = ? AND expires_at > ?',
async with write_transaction() as conn:
cur = await conn.execute('DELETE FROM email_codes WHERE email = ? AND code = ? AND purpose = ? AND expires_at > ?',
(email, (code or '').strip(), purpose, int(time.time())))
conn.commit()
if cur.rowcount == 1:
_CODE_FAILS.pop(fail_key, None)
return 60
row = None
finally:
conn.close()
if row:
_CODE_FAILS.pop(fail_key, None)
return max(0, int(row[0]) - int(time.time()))
# 猜错:失败计数 +1,累计 5 次直接作废该用途的验证码
n = _CODE_FAILS.get(fail_key, 0) + 1
_CODE_FAILS[fail_key] = n
@@ -251,13 +259,13 @@ def check_verify_code(email: str, code: str, purpose: str = 'register') -> int:
if n >= 5:
_CODE_FAILS.pop(fail_key, None)
try:
_x('DELETE FROM email_codes WHERE email = ? AND purpose = ?', (email, purpose))
await _x('DELETE FROM email_codes WHERE email = ? AND purpose = ?', (email, purpose))
except Exception:
pass
return 0
def restore_verify_code(email: str, code: str, ttl_seconds: int, purpose: str = 'register') -> None:
async def restore_verify_code(email: str, code: str, ttl_seconds: int, purpose: str = 'register') -> None:
"""注册流程后续步骤失败时还原已消费的验证码(第二轮:建号失败不应吞掉验证码)"""
if ttl_seconds <= 0:
return
@@ -267,11 +275,5 @@ def restore_verify_code(email: str, code: str, ttl_seconds: int, purpose: str =
code = (code or '').strip()
if not email or not code:
return
with _DB_LOCK:
conn = get_conn()
try:
conn.execute('INSERT INTO email_codes (email, code, expires_at, created_at, purpose) VALUES (?, ?, ?, ?, ?)',
await _x('INSERT INTO email_codes (email, code, expires_at, created_at, purpose) VALUES (?, ?, ?, ?, ?)',
(email, code, int(time.time() + ttl_seconds), int(time.time()), purpose))
conn.commit()
finally:
conn.close()
+9 -9
View File
@@ -1,7 +1,7 @@
# -*- coding: utf-8 -*-
"""公告与站点设置。
(自 auth_system.py 拆出,行为与原实现一致。)
(自 auth_system.py 拆出,行为与原实现一致;DB 访问已迁移为原生异步。)
"""
import time
@@ -15,28 +15,28 @@ from .db import (
# ============ 公告 ============
def get_announcement() -> dict[str, Any]:
rows = _q('SELECT * FROM announcements WHERE id = 1')
async def get_announcement() -> dict[str, Any]:
rows = await _q('SELECT * FROM announcements WHERE id = 1')
return dict(rows[0]) if rows else {'title': '', 'content': '', 'popup_mode': 'off'}
def save_announcement(title: str, content: str, popup_mode: str) -> None:
async def save_announcement(title: str, content: str, popup_mode: str) -> None:
if popup_mode not in ('off', 'every', 'once'):
popup_mode = 'off'
# 版本号精确到毫秒,同秒多次保存也能区分(once 已读判断依赖该字段)
updated = time.strftime('%Y-%m-%d %H:%M:%S') + f'.{int(time.time() * 1000) % 1000:03d}'
_x('UPDATE announcements SET title = ?, content = ?, popup_mode = ?, updated_at = ? WHERE id = 1',
await _x('UPDATE announcements SET title = ?, content = ?, popup_mode = ?, updated_at = ? WHERE id = 1',
(title, content, popup_mode, updated))
# ============ 设置(键值) ============
def get_setting(key: str, default: str = '') -> str:
rows = _q('SELECT value FROM settings WHERE key = ?', (key,))
async def get_setting(key: str, default: str = '') -> str:
rows = await _q('SELECT value FROM settings WHERE key = ?', (key,))
return rows[0]['value'] if rows else default
def set_setting(key: str, value: str) -> None:
_x('INSERT INTO settings (key, value) VALUES (?, ?) '
async def set_setting(key: str, value: str) -> None:
await _x('INSERT INTO settings (key, value) VALUES (?, ?) '
'ON CONFLICT(key) DO UPDATE SET value = excluded.value', (key, str(value)))
+69 -77
View File
@@ -1,9 +1,10 @@
# -*- coding: utf-8 -*-
"""用户增删改查、登录校验与状态。
(自 auth_system.py 拆出,行为与原实现一致。)
(自 auth_system.py 拆出,行为与原实现一致;DB 访问已迁移为原生异步。)
"""
import asyncio
import json
import sqlite3
import time
@@ -11,10 +12,9 @@ from dataclasses import dataclass
from typing import Any
from .db import (
_DB_LOCK,
_q,
_x,
get_conn,
write_transaction,
)
from .security import (
_hash_new,
@@ -27,7 +27,7 @@ from .security import (
class RegistrationGuard:
"""注册风控上下文:同 IP 频率闸 + 设备赠送闸。
这些检查必须与建号 INSERT 在**同一把 _DB_LOCK 事务内**完成——锁外预检存在
这些检查必须与建号 INSERT 在**同一写事务内**完成——锁外预检存在
TOCTOU(并发同设备双注册各得一次赠送、并发同 IP 挤过限额),所以它们不是
「可选的附加参数」,而是事务内必须可见的输入。管理建号无风控需求,不传即可。
"""
@@ -40,7 +40,7 @@ class RegistrationGuard:
device_once: bool = False
def create_user(username: str, password: str, email: str = '', credits: int = 0,
async def create_user(username: str, password: str, email: str = '', credits: int = 0,
role: str = 'user', *,
guard: RegistrationGuard | None = None) -> sqlite3.Row | None:
# 余额只接受 -1(不限)或 >= 0;非法负值一律按 0 处理,禁止出现 -2 这类"假不限"
@@ -51,165 +51,157 @@ def create_user(username: str, password: str, email: str = '', credits: int = 0,
if credits < -1:
credits = 0
# 注册入口传 guard=RegistrationGuard(ip_limit=3, device_once=True)——同 IP 24h
# 注册数检查在 _DB_LOCK 事务内与 INSERT 原子完成,堵住「锁外检查 + 并发插入」
# 注册数检查在写事务内与 INSERT 原子完成,堵住「锁外检查 + 并发插入」
# 的 TOCTOU 刷号;管理建号不传 guard(即不限制)。超限抛 ValueError 由调用方转 400/429。
guard = guard or RegistrationGuard()
now = int(time.time())
with _DB_LOCK:
conn = get_conn()
try:
# PBKDF2 10 万次迭代是 CPU 密集(~50ms):在事务外先算好,避免持写锁时
# 阻塞事件循环(正当的 CPU 卸载,非同步门面)。
phash = await asyncio.to_thread(_hash_new, password)
async with write_transaction() as conn:
if guard.ip_limit > 0 and guard.ip:
n = conn.execute('SELECT COUNT(*) AS n FROM users WHERE register_ip = ? AND created_at > ?',
(guard.ip, now - guard.ip_window)).fetchone()[0]
cur = await conn.execute(
'SELECT COUNT(*) AS n FROM users WHERE register_ip = ? AND created_at > ?',
(guard.ip, now - guard.ip_window))
row = await cur.fetchone()
n = int(row['n']) if row else 0
if n >= guard.ip_limit:
raise ValueError('当前网络注册过于频繁,请 24 小时后再试或联系管理员')
if guard.device_once and guard.device_id:
# 设备赠送闸并入同一事务——锁外 device_used 预检有 TOCTOU,
# 并发同设备双注册可各得一次赠送;锁内复查命中则本次建号不赠送(账号照常创建)
n = conn.execute("SELECT COUNT(*) AS n FROM users WHERE trial_device = ? AND role != 'admin'",
(guard.device_id,)).fetchone()[0]
if n > 0:
cur = await conn.execute(
"SELECT COUNT(*) AS n FROM users WHERE trial_device = ? AND role != 'admin'",
(guard.device_id,))
row = await cur.fetchone()
if row and int(row['n']) > 0:
credits = 0
conn.execute(
await conn.execute(
'INSERT INTO users (username, password_hash, salt, email, role, duration_seconds, '
'trial_device, register_ip, register_ua, last_active, created_at) '
'VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)',
(username, _hash_new(password), '', email, role,
(username, phash, '', email, role,
-1 if role == 'admin' else int(credits),
guard.device_id, guard.ip, guard.ua, now, now))
conn.commit()
finally:
conn.close()
return get_user_by_name(username)
return await get_user_by_name(username)
def get_user_by_name(username: str) -> sqlite3.Row | None:
rows = _q('SELECT * FROM users WHERE username = ?', (username,))
async def get_user_by_name(username: str) -> sqlite3.Row | None:
rows = await _q('SELECT * FROM users WHERE username = ?', (username,))
return rows[0] if rows else None
def get_user_by_email(email: str) -> sqlite3.Row | None:
async def get_user_by_email(email: str) -> sqlite3.Row | None:
"""按邮箱查用户(注册/建号邮箱查重的友好提示;库侧另有唯一索引兜底并发)"""
email = _normalize_email(email)
if not email:
return None
rows = _q('SELECT * FROM users WHERE email = ? LIMIT 1', (email,))
rows = await _q('SELECT * FROM users WHERE email = ? LIMIT 1', (email,))
return rows[0] if rows else None
def register_ip_exceeded(ip: str, limit: int = 3, window: int = 86400) -> bool:
async def register_ip_exceeded(ip: str, limit: int = 3, window: int = 86400) -> bool:
"""同 IP 24h 内注册数超限(默认 3 次)——防 IP 轮换刷免费赠送次数;
设备指纹可被客户端伪造,IP 维度是第二道闸。家庭 NAT 场景超限可由管理员建号兜底。"""
if not ip:
return False
rows = _q('SELECT COUNT(*) AS n FROM users WHERE register_ip = ? AND created_at > ?',
rows = await _q('SELECT COUNT(*) AS n FROM users WHERE register_ip = ? AND created_at > ?',
(ip, int(time.time() - window)))
return bool(rows and rows[0]['n'] >= limit)
def get_user_by_id(uid: int) -> sqlite3.Row | None:
rows = _q('SELECT * FROM users WHERE id = ?', (uid,))
async def get_user_by_id(uid: int) -> sqlite3.Row | None:
rows = await _q('SELECT * FROM users WHERE id = ?', (uid,))
return rows[0] if rows else None
def verify_login(username: str, password: str) -> sqlite3.Row | None:
async def verify_login(username: str, password: str) -> sqlite3.Row | None:
"""校验用户名密码。密码正确即返回用户行(含被禁用/封禁账号——
状态语义由登录路由判断:禁用显示"已被禁用"、封禁显示封禁原因。
原实现把禁用账号提前过滤成 None,路由的"账号已被禁用"分支永不可达)"""
user = get_user_by_name(username)
user = await get_user_by_name(username)
if user:
ok, needs_upgrade = _verify_password(user['password_hash'], password, user['salt'])
# PBKDF2 校验是 CPU 密集(~50ms),卸载到线程避免阻塞事件循环(正当 CPU 卸载)
ok, needs_upgrade = await asyncio.to_thread(
_verify_password, user['password_hash'], password, user['salt'])
if ok:
if needs_upgrade:
# 存量 SHA-256 哈希在登录成功时透明升级为 PBKDF2
_x('UPDATE users SET password_hash = ?, salt = \'\' WHERE id = ?',
(_hash_new(password), user['id']))
new_hash = await asyncio.to_thread(_hash_new, password)
await _x('UPDATE users SET password_hash = ?, salt = \'\' WHERE id = ?',
(new_hash, user['id']))
return user
return None
def list_users() -> list[dict[str, Any]]:
rows = _q('SELECT * FROM users ORDER BY id DESC')
async def list_users() -> list[dict[str, Any]]:
rows = await _q('SELECT * FROM users ORDER BY id DESC')
return [_user_public(r, full=True) for r in rows]
def delete_user(uid: int) -> bool:
async def delete_user(uid: int) -> bool:
"""删除用户(卡密 used_by 为用户名字符串无外键——删除时同步
标注"(已注销#id)",管理端卡密列表不再显示悬空的已删用户名)"""
u = get_user_by_id(uid)
u = await get_user_by_id(uid)
if not u or u['role'] == 'admin':
return False
with _DB_LOCK:
conn = get_conn()
try:
async with write_transaction() as conn:
name = u['username']
conn.execute('DELETE FROM users WHERE id = ? AND role != ?', (uid, 'admin'))
conn.execute("UPDATE cards SET used_by = used_by || ? WHERE used_by = ?",
await conn.execute('DELETE FROM users WHERE id = ? AND role != ?', (uid, 'admin'))
await conn.execute("UPDATE cards SET used_by = used_by || ? WHERE used_by = ?",
(f'(已注销#{uid})', name))
conn.commit()
finally:
conn.close()
return True
def reset_password(uid: int, new_password: str) -> bool:
async def reset_password(uid: int, new_password: str) -> bool:
"""重置密码(同时递增 login_seq,让所有已签发会话立即失效)"""
with _DB_LOCK:
conn = get_conn()
try:
conn.execute('UPDATE users SET password_hash = ?, salt = \'\', login_seq = COALESCE(login_seq, 0) + 1 WHERE id = ?',
(_hash_new(new_password), uid))
conn.commit()
new_hash = await asyncio.to_thread(_hash_new, new_password)
await _x('UPDATE users SET password_hash = ?, salt = \'\', '
'login_seq = COALESCE(login_seq, 0) + 1 WHERE id = ?',
(new_hash, uid))
return True
finally:
conn.close()
def set_user_status(uid: int, status: int) -> bool:
async def set_user_status(uid: int, status: int) -> bool:
"""启用 / 禁用账号(1 启用,0 禁用);管理员账号不可禁用"""
if status not in (0, 1):
return False
row = _q('SELECT role FROM users WHERE id = ?', (uid,))
row = await _q('SELECT role FROM users WHERE id = ?', (uid,))
if row and row[0]['role'] == 'admin':
return False
return bool(_x('UPDATE users SET status = ? WHERE id = ?', (status, uid)))
return bool(await _x('UPDATE users SET status = ? WHERE id = ?', (status, uid)))
def ban_user(uid: int, reason: str = '') -> bool:
async def ban_user(uid: int, reason: str = '') -> bool:
"""封禁账号(管理员不可封)"""
row = _q('SELECT role FROM users WHERE id = ?', (uid,))
row = await _q('SELECT role FROM users WHERE id = ?', (uid,))
if not row or row[0]['role'] == 'admin':
return False
return bool(_x('UPDATE users SET status = 2, ban_reason = ? WHERE id = ?',
return bool(await _x('UPDATE users SET status = 2, ban_reason = ? WHERE id = ?',
(str(reason or '')[:200], uid)))
def unban_user(uid: int) -> bool:
async def unban_user(uid: int) -> bool:
"""解封账号,恢复正常(清空触发封禁的省份历史,否则下次登录立即再封)"""
return bool(_x('UPDATE users SET status = 1, ban_reason = \'\', recent_provinces = \'[]\' WHERE id = ?', (uid,)))
return bool(await _x('UPDATE users SET status = 1, ban_reason = \'\', recent_provinces = \'[]\' WHERE id = ?', (uid,)))
def set_user_credits(uid: int, count: int) -> None:
async def set_user_credits(uid: int, count: int) -> None:
"""设置用户剩余次数(-1 表示不限)"""
count = -1 if count < 0 else int(count)
_x('UPDATE users SET duration_seconds = ? WHERE id = ?', (count, uid))
await _x('UPDATE users SET duration_seconds = ? WHERE id = ?', (count, uid))
def set_user_password(uid: int, password: str) -> None:
async def set_user_password(uid: int, password: str) -> None:
# 新哈希 + 递增 login_seq(所有旧会话立即失效)
with _DB_LOCK:
conn = get_conn()
try:
conn.execute('UPDATE users SET password_hash = ?, salt = \'\', '
new_hash = await asyncio.to_thread(_hash_new, password)
await _x('UPDATE users SET password_hash = ?, salt = \'\', '
'login_seq = COALESCE(login_seq, 0) + 1 WHERE id = ?',
(_hash_new(password), uid))
conn.commit()
finally:
conn.close()
(new_hash, uid))
def set_user_netease_cookie(username: str, cookie: str) -> None:
_x('UPDATE users SET netease_cookie = ? WHERE username = ?', (cookie, username))
async def set_user_netease_cookie(username: str, cookie: str) -> None:
await _x('UPDATE users SET netease_cookie = ? WHERE username = ?', (cookie, username))
def _user_public(r: sqlite3.Row, full: bool = False) -> dict[str, Any]:
@@ -249,8 +241,8 @@ def _last_province(r: sqlite3.Row) -> str:
# ============ 免费体验(设备识别) ============
def device_used(device_id: str) -> bool:
async def device_used(device_id: str) -> bool:
if not device_id:
return False
rows = _q("SELECT id FROM users WHERE trial_device = ? AND role != 'admin' LIMIT 1", (device_id,))
rows = await _q("SELECT id FROM users WHERE trial_device = ? AND role != 'admin' LIMIT 1", (device_id,))
return bool(rows)
-142
View File
@@ -1,142 +0,0 @@
# -*- coding: utf-8 -*-
"""requests 兼容的异步 HTTP 垫片(基于 aiohttp)。
目的:让既有大量 `requests.post/get + response.json()/raise_for_status()` 风格的
代码**以最小改动**迁移到异步——把 `import requests` 换成
`from core import aiohttp_compat as requests`,加 `await`、加 `async def` 即可,
其余 `.text/.json()/.status_code/.raise_for_status()` 语义保持不变。
与 requests 的语义对齐点(很关键,不得改动):
- 请求本身**不**因 4xx/5xx 抛异常;只有显式 `raise_for_status()` 才抛 `HTTPError`
(上游风控 429/460 的重试/不重试判断依赖调用方读取 `status_code`,见 music_api.HTTPClient)。
- 网络异常/超时抛 `RequestException`(含 `Timeout`)。
- `Response` 一次性读全 body(与 requests 一致),暴露 text/json/content/headers/cookies/url。
连接按事件循环隔离(见 core.http_client 的同样设计)。
"""
import asyncio
import json as _json
from collections.abc import Mapping
from typing import Any
import aiohttp
from core.http_errors import ConnectionError, HTTPError, RequestException, Timeout
DEFAULT_TIMEOUT = 30
DEFAULT_MAX_BYTES = 8 * 1024 * 1024
class Response:
"""requests.Response 的异步子集:body 已读全,属性同步访问。"""
def __init__(self, status_code: int, url: str, content: bytes,
headers: Mapping[str, str], cookies: Mapping[str, str], reason: str = ''):
self.status_code = status_code
self.url = url
self.content = content
self.headers = dict(headers)
self.cookies = dict(cookies)
self.reason = reason
@property
def text(self) -> str:
return self.content.decode('utf-8', errors='replace')
def json(self, **kw) -> Any:
return _json.loads(self.text, **kw)
def raise_for_status(self) -> None:
if 400 <= self.status_code:
raise HTTPError(f"{self.status_code} {self.reason}".strip(), self)
# ---------- 会话(按 loop 隔离的 aiohttp session) ----------
class _SessionState:
def __init__(self):
self._sessions: dict[int, aiohttp.ClientSession] = {}
self._lock: asyncio.Lock | None = None
def _get_lock(self) -> asyncio.Lock:
if self._lock is None:
self._lock = asyncio.Lock()
return self._lock
async def session(self) -> aiohttp.ClientSession:
key = id(asyncio.get_running_loop())
sess = self._sessions.get(key)
if sess is not None and not sess.closed:
return sess
async with self._get_lock():
sess = self._sessions.get(key)
if sess is None or sess.closed:
sess = aiohttp.ClientSession(
connector=aiohttp.TCPConnector(limit=100, limit_per_host=32, ttl_dns_cache=300),
)
self._sessions[key] = sess
return sess
async def close_all(self) -> None:
for sess in self._sessions.values():
if not sess.closed:
await sess.close()
self._sessions.clear()
_state = _SessionState()
async def close() -> None:
"""关闭所有连接(应用关闭时调用)。"""
await _state.close_all()
async def request(
method: str,
url: str,
*,
params: Any = None,
data: Any = None,
json: Any = None,
headers: Mapping[str, str] | None = None,
cookies: Mapping[str, str] | None = None,
timeout: float | None = DEFAULT_TIMEOUT,
allow_redirects: bool = True,
max_bytes: int = DEFAULT_MAX_BYTES,
) -> Response:
"""发起单次异步请求,返回 Response(不因 4xx/5xx 抛异常,同 requests)。"""
sess = await _state.session()
req_timeout = aiohttp.ClientTimeout(total=timeout) if timeout else None
try:
async with sess.request(
method, url,
params=params, data=data, json=json,
headers=dict(headers) if headers else None,
cookies=dict(cookies) if cookies else None,
timeout=req_timeout, allow_redirects=allow_redirects,
) as resp:
body = await resp.read()
if len(body) > max_bytes:
raise RequestException(f"响应体超过上限 {max_bytes} 字节")
return Response(
status_code=resp.status,
url=str(resp.url),
content=body,
headers=dict(resp.headers),
cookies={k: v.value for k, v in resp.cookies.items()},
reason=resp.reason or '',
)
except aiohttp.ClientError as e:
raise ConnectionError(str(e)) from e
except TimeoutError as e:
raise Timeout('请求超时') from e
async def get(url: str, **kw) -> Response:
return await request('GET', url, **kw)
async def post(url: str, **kw) -> Response:
return await request('POST', url, **kw)
-61
View File
@@ -1,61 +0,0 @@
# -*- coding: utf-8 -*-
"""同步 ↔ 异步桥。
场景:部分调用方仍是同步代码(线程池下载器、后台任务线程),而服务层已异步。
`run_sync(coro)` 把协程提交到**专用后台事件循环线程**执行并阻塞等待结果,
使同步代码得以调用异步实现,无需把调用链一次性全部改异步。
- 后台 loop 常驻单例(daemon 线程),避免每个请求新建 loop 的开销与
aiohttp 连接池失效;
- 在已有事件循环中调用 `run_sync` 会死锁,故检测到运行中的 loop 时直接报错,
提示改用 `await`。
"""
import asyncio
import threading
from collections.abc import Coroutine
from typing import Any
_loop: asyncio.AbstractEventLoop | None = None
_loop_lock = threading.Lock()
def _get_loop() -> asyncio.AbstractEventLoop:
global _loop
if _loop is not None and not _loop.is_closed():
return _loop
with _loop_lock:
if _loop is None or _loop.is_closed():
loop = asyncio.new_event_loop()
t = threading.Thread(target=loop.run_forever, name='async-bridge', daemon=True)
t.start()
_loop = loop
return _loop
def get_loop() -> asyncio.AbstractEventLoop:
"""后台事件循环(供需要直接提交协程的调用方使用)。"""
return _get_loop()
def run_sync[T](coro: Coroutine[Any, Any, T], timeout: float | None = None) -> T:
"""在同步代码中运行协程并等待结果。
Raises:
RuntimeError: 在已有事件循环内调用(应直接 await 而非桥接)。
"""
try:
asyncio.get_running_loop()
except RuntimeError:
pass
else:
raise RuntimeError('run_sync 不能在事件循环内调用,请直接 await 协程')
fut = asyncio.run_coroutine_threadsafe(coro, _get_loop())
return fut.result(timeout)
def shutdown() -> None:
"""停止后台 loop(应用退出时调用)。"""
global _loop
if _loop is not None and not _loop.is_closed():
_loop.call_soon_threadsafe(_loop.stop)
_loop = None
+21 -2
View File
@@ -143,7 +143,7 @@ class AsyncHTTPClient:
# ---------- 底层:单次请求(不重试) ----------
async def _once(
async def request_once(
self,
method: str,
url: str,
@@ -156,6 +156,11 @@ class AsyncHTTPClient:
timeout: float | None = None,
allow_redirects: bool = True,
) -> HTTPResult:
"""发起一次请求,**不做重试**(自管重试策略的调用方用这个)。
非 2xx 一律抛 UpstreamHTTPError(含 5xx)——调用方按 .status 自行决定是否重试。
request() 在此基础上叠加「网络异常/5xx 退避重试一次」。
"""
session = await self._session_for_loop()
req_timeout = aiohttp.ClientTimeout(total=timeout) if timeout else None
try:
@@ -211,7 +216,7 @@ class AsyncHTTPClient:
# ---------- 高层 API ----------
async def request(self, method: str, url: str, **kw) -> HTTPResult:
return await self._with_retry(lambda: self._once(method, url, **kw))
return await self._with_retry(lambda: self.request_once(method, url, **kw))
async def get_text(self, url: str, **kw) -> str:
return (await self.request('GET', url, **kw)).text
@@ -284,5 +289,19 @@ class AsyncHTTPClient:
raise UpstreamNetworkError(str(e)) from e
async def iter_chunks(resp: aiohttp.ClientResponse, chunk_size: int) -> AsyncGenerator[bytes, None]:
"""逐块读取流式响应体,把 aiohttp 的传输异常统一翻译为 UpstreamNetworkError。
请求建立阶段的失败由 open()/request() 翻成 UpstreamError;但**流式读取期间**
断流抛的是 aiohttp.ClientPayloadError(ClientError 子类),不在那条路径上——
调用方(音频下载、播客落盘)只认 UpstreamError 层次,故在此收敛。
"""
try:
async for chunk in resp.content.iter_chunked(chunk_size):
yield chunk
except aiohttp.ClientError as e:
raise UpstreamNetworkError(str(e)) from e
# 进程级共享客户端(UA/Referer 由 music_api 在初始化后注入)
client = AsyncHTTPClient()
-38
View File
@@ -1,38 +0,0 @@
# -*- coding: utf-8 -*-
"""requests 同形异常层次(同步 / 异步两个 HTTP 垫片共用)。
`core.aiohttp_compat`(异步)与 `core.sync_http`(同步)此前各自定义了一份
`RequestException` / `HTTPError`——两份定义互不相干,于是同一条调用链上
「同步段抛的异常捕获不到异步段的同类失败」,调用方被迫记住自己走的是哪条垫片。
此处收敛为唯一来源;命名沿用 requests(`Timeout` / `ConnectionError` 与内置
异常同名,这是 requests 自身的约定——调用方应带模块前缀引用,例如
`requests.ConnectionError`,裸 `ConnectionError` 会命中内置的那个)。
层次:
RequestException 网络 / 协议层失败(可捕获的总入口)
├── HTTPError raise_for_status() 抛出的状态异常(携带 response)
├── Timeout 请求超时
└── ConnectionError 连接错误
"""
from typing import Any
class RequestException(Exception):
"""请求异常基类(网络/协议层)。"""
class HTTPError(RequestException):
"""raise_for_status() 抛出的 HTTP 状态异常。"""
def __init__(self, message: str, response: Any = None):
super().__init__(message)
self.response = response
class Timeout(RequestException):
"""请求超时。"""
class ConnectionError(RequestException):
"""连接错误。"""
+3 -1
View File
@@ -92,7 +92,9 @@ GLOBAL_MUSIC_U_CACHE: _TimedValue = {'v': None, 'ts': 0.0}
# 服务器网易云账号昵称缓存(10 分钟):非管理员可见响应中遮蔽真实昵称
SERVER_NICK_CACHE: _NickCache = {'nick': None, 'ts': 0.0}
SERVER_NICK_LOCK = threading.Lock() # 刷新单飞锁,防并发重复打上游
VIP_PROBE_LOCK = threading.Lock() # 音质探测单飞锁(有界等待)
# 注:音质探测的单飞锁(原 VIP_PROBE_LOCK)已移出本模块——本模块声明「依赖仅限
# 标准库 threading/collections」,而那是把 asyncio 锁;且它全仓只有
# web/routers/account.py 一个消费者(sync 时代遗留的「共享」注释已不成立)。
# 全局账号 VIP 探测缓存(5 分钟):probe 一次要打 8 次网易云接口,
# 公网下账号栏高频刷新会连累服务器账号触发风控。
-196
View File
@@ -1,196 +0,0 @@
# -*- coding: utf-8 -*-
"""同步 HTTP 门面(底层 aiohttp)——彻底移除 requests 依赖。
为什么需要它:部分代码天然是同步的(worker 线程里的流式下载、纯函数里的短链展开/
IP 归属地查询)。本模块提供 requests 风格的同步 API,但底层复用 aiohttp 连接池,
不再引入 requests。
流式读取的实现:在后台事件循环起一个读取协程,把块推入有界队列(背压),
同步侧从队列消费——每次搬运只过一次队列,不做「每块一次跨线程调度」。
"""
import asyncio
import queue
import threading
from collections.abc import Iterator, Mapping
from typing import Any
from core import async_bridge
from core.http_client import client as _client
from core.http_errors import HTTPError, RequestException
# 异常层次与异步垫片(core.aiohttp_compat)共用同一份定义——调用方可用同一组
# 类型捕获两条通道的失败。RequestException 在此再导出:music_downloader.models
# 的 _RetryFromScratch 继承它,既有 except 与测试也直接导入它。
__all__ = ['HTTPError', 'RequestException', 'Session', 'SyncResponse', 'get', 'post']
_DEFAULT_TIMEOUT = 30
class SyncResponse:
"""requests.Response 的同步子集(底层 aiohttp)。"""
def __init__(self, status: int, headers: Mapping[str, str], content: bytes | None,
url: str, stream_state: dict | None = None):
self.status_code = status
self.headers = dict(headers)
self.url = url
self._content = content
self._stream = stream_state
@property
def content(self) -> bytes:
if self._content is None:
self._content = b''.join(self.iter_content(64 * 1024))
return self._content
@property
def text(self) -> str:
return self.content.decode('utf-8', errors='replace')
def json(self, **kw) -> Any:
import json as _json
return _json.loads(self.text, **kw)
def raise_for_status(self) -> None:
if self.status_code >= 400:
raise HTTPError(f'HTTP {self.status_code}', self)
def iter_content(self, chunk_size: int = 64 * 1024) -> Iterator[bytes]:
"""流式读取响应体(大小写与 requests 一致:chunk_size 为字节数)。"""
if self._content is not None:
for i in range(0, len(self._content), chunk_size):
yield self._content[i:i + chunk_size]
return
st = self._stream
assert st is not None
q: queue.Queue = st['queue']
while True:
item = q.get()
if item is None: # 正常结束
break
if isinstance(item, BaseException): # 读取出错
raise item
yield item
def close(self) -> None:
st = self._stream
if st and st.get('cancel') is not None:
try:
async_bridge.run_sync(_cancel(st))
except Exception:
pass
def __enter__(self) -> 'SyncResponse':
return self
def __exit__(self, *exc) -> bool:
self.close()
return False
async def _cancel(st: dict) -> None:
st['cancelled'] = True
resp = st.get('resp')
if resp is not None:
resp.close()
def _stream_reader(st: dict, chunk_size: int) -> None:
"""在后台事件循环读块 → 有界队列(背压),同步侧消费。"""
async def _run():
try:
async for chunk in st['resp'].content.iter_chunked(chunk_size):
if st.get('cancelled'):
break
while True: # 背压:队列满则稍等,避免内存无界
try:
st['queue'].put(chunk, timeout=0.5)
break
except queue.Full:
if st.get('cancelled'):
return
except Exception as e:
st['queue'].put(e)
finally:
try:
st['resp'].close()
except Exception:
pass
st['queue'].put(None)
asyncio.run_coroutine_threadsafe(_run(), async_bridge.get_loop())
class Session:
"""requests.Session 的同步子集(连接池由底层 aiohttp 统一维护)。"""
trust_env = False
def __init__(self, **kw) -> None:
# headers:会话级默认头(与 requests.Session().headers.update 等价)
self.headers: dict[str, str] = dict(kw.pop('headers', {}))
def _merge(self, kw: dict) -> dict:
h = dict(self.headers)
h.update(kw.pop('headers', None) or {})
if h:
kw['headers'] = h
return kw
def get(self, url: str, **kw) -> SyncResponse:
return _request('GET', url, **self._merge(kw))
def post(self, url: str, **kw) -> SyncResponse:
return _request('POST', url, **self._merge(kw))
def close(self) -> None:
pass
def _norm_timeout(timeout):
"""requests 允许 (connect, read) 元组;aiohttp 只接受单一总时长——取较大者。"""
if isinstance(timeout, (tuple, list)):
vals = [float(v) for v in timeout if isinstance(v, (int, float))]
return max(vals) if vals else _DEFAULT_TIMEOUT
return timeout
def _request(method: str, url: str, *, headers: Mapping[str, str] | None = None,
cookies: Mapping[str, str] | None = None, params: Mapping[str, Any] | None = None,
data: Any = None, json: Any = None, timeout: float | None = _DEFAULT_TIMEOUT,
allow_redirects: bool = True, stream: bool = False) -> SyncResponse:
timeout = _norm_timeout(timeout)
if stream:
return _open_stream(method, url, headers=headers, cookies=cookies, params=params,
data=data, json=json, timeout=timeout,
allow_redirects=allow_redirects)
resp = async_bridge.run_sync(_client.request(
method, url, headers=headers, cookies=cookies, params=params, data=data,
json_body=json, timeout=timeout, allow_redirects=allow_redirects))
return SyncResponse(resp.status, resp.headers, resp.text.encode('utf-8'),
resp.url)
def _open_stream(method: str, url: str, **kw) -> SyncResponse:
timeout = _norm_timeout(kw.pop('timeout', _DEFAULT_TIMEOUT))
# AsyncHTTPClient.open 只接受 headers/cookies/timeout/allow_redirects;
# 流式场景不支持 params/data/json(调用方直接把查询串拼进 URL)
for unsupported in ('params', 'data', 'json'):
kw.pop(unsupported, None)
resp = async_bridge.run_sync(_client.open(method, url, timeout=timeout, **kw))
st = {'resp': resp, 'queue': queue.Queue(maxsize=8), 'cancel': object(), 'cancelled': False}
th = threading.Thread(target=_stream_reader, args=(st, 64 * 1024), daemon=True)
th.start()
return SyncResponse(resp.status, resp.headers, None, str(resp.url), stream_state=st)
# 模块级便捷函数(等价于共享 Session 的 get/post)
_default_session = Session()
def get(url: str, **kw) -> SyncResponse:
return _default_session.get(url, **kw)
def post(url: str, **kw) -> SyncResponse:
return _default_session.post(url, **kw)
+1 -31
View File
@@ -1,11 +1,10 @@
# -*- coding: utf-8 -*-
"""纯函数工具(框架无关,无 Web 框架依赖)。
SRT 时间轴、LRC→SRT 转换、字段参数解析、163cn.tv 短链展开(含私有缓存)。
SRT 时间轴、LRC→SRT 转换、字段参数解析。
"""
import re
import threading
def srt_time(t: float) -> str:
@@ -69,32 +68,3 @@ def parse_meta_fields(raw) -> list[str] | None:
return list(raw) or None
except Exception:
return None
# 短链展开结果缓存:用户反复解析同一分享链接不再重复请求 163cn.tv(有界防膨胀)
_SHORTLINK_CACHE: dict[str, str] = {}
_SHORTLINK_CACHE_LOCK = threading.Lock()
def expand_shortlink(url: str) -> str:
"""展开 163cn.tv 短链(带进程内缓存;失败原样返回)"""
with _SHORTLINK_CACHE_LOCK:
hit = _SHORTLINK_CACHE.get(url)
if hit is not None:
return hit
expanded = url
try:
from core import sync_http
response = sync_http.get(url, allow_redirects=False, timeout=10)
expanded = response.headers.get('Location', url)
except Exception:
expanded = url
# 展开失败(Location 缺失/网络异常,expanded 原样等于 url)不写
# 缓存——旧口径一次网络抖动会让该分享链接在整个缓存存续期内必然解析失败
if expanded != url:
with _SHORTLINK_CACHE_LOCK:
if len(_SHORTLINK_CACHE) > 500:
_SHORTLINK_CACHE.clear()
_SHORTLINK_CACHE[url] = expanded
return expanded
-6
View File
@@ -18,14 +18,10 @@ from .api import (
NeteaseAPI,
)
from .client import (
_MAX_UPSTREAM_JSON_BYTES,
HTTPClient,
_as_dict,
_ensure_dict,
)
from .compat import (
url_v1,
)
from .constants import (
APIConstants,
QualityLevel,
@@ -48,7 +44,6 @@ from .qr import (
__all__ = [
'DEFAULT_COOKIE_FILE',
'_MAX_UPSTREAM_JSON_BYTES',
'APIConstants',
'APIException',
'CryptoUtils',
@@ -61,5 +56,4 @@ __all__ = [
'_ensure_dict',
'load_cookies',
'parse_cookie_string',
'url_v1',
]
+28 -41
View File
@@ -10,7 +10,8 @@ from collections.abc import Callable
from random import randrange
from typing import Any
from core import aiohttp_compat as requests # 异步 HTTP 垫片(requests 同形 API)
from core.http_client import UpstreamError
from core.http_client import client as _http
from .client import (
HTTPClient,
@@ -71,9 +72,8 @@ async def _collect_playlist_tracks(track_ids: list[str], headers: dict[str, str]
except (TypeError, ValueError):
continue
song_resp = await requests.post(APIConstants.SONG_DETAIL_V3, data=song_data,
song_resp = await _http.request('POST', APIConstants.SONG_DETAIL_V3, data=song_data,
headers=headers, cookies=cookies, timeout=30)
song_resp.raise_for_status()
song_result = song_resp.json()
if not isinstance(song_result, dict):
@@ -222,9 +222,8 @@ class NeteaseAPI:
'User-Agent': APIConstants.USER_AGENT,
'Referer': APIConstants.REFERER
}
response = await requests.post(APIConstants.SONG_DETAIL_V3, data=data,
response = await _http.request('POST', APIConstants.SONG_DETAIL_V3, data=data,
headers=headers, timeout=30)
response.raise_for_status()
result = response.json()
# 合法 JSON 但非对象(null/数组)时给明确错误,不让 AttributeError 失真
@@ -234,7 +233,7 @@ class NeteaseAPI:
raise APIException(f"获取歌曲详情失败: {result.get('message', '未知错误')}")
return result
except requests.RequestException as e:
except UpstreamError as e:
raise APIException(f"获取歌曲详情请求失败: {e}") from e
except json.JSONDecodeError as e:
raise APIException(f"解析歌曲详情响应失败: {e}") from e
@@ -270,9 +269,8 @@ class NeteaseAPI:
'Referer': APIConstants.REFERER
}
response = await requests.post(APIConstants.LYRIC_API, data=data,
response = await _http.request('POST', APIConstants.LYRIC_API, data=data,
headers=headers, cookies=cookies, timeout=30)
response.raise_for_status()
result = response.json()
_ensure_dict(result, 'get_lyric 接口响应异常(非标准 JSON 对象)')
@@ -280,7 +278,7 @@ class NeteaseAPI:
raise APIException(f"获取歌词失败: {result.get('message', '未知错误')}")
return result
except requests.RequestException as e:
except UpstreamError as e:
raise APIException(f"获取歌词请求失败: {e}") from e
except json.JSONDecodeError as e:
raise APIException(f"解析歌词响应失败: {e}") from e
@@ -310,9 +308,8 @@ class NeteaseAPI:
'Referer': APIConstants.REFERER
}
response = await requests.post(APIConstants.SEARCH_API, data=data,
response = await _http.request('POST', APIConstants.SEARCH_API, data=data,
headers=headers, cookies=cookies, timeout=30)
response.raise_for_status()
result = response.json()
# 非 dict 响应(null/数组)给明确错误
@@ -323,7 +320,7 @@ class NeteaseAPI:
result_data = result.get('result') or {}
return _collect_search_items(result_data, search_type)
except requests.RequestException as e:
except UpstreamError as e:
raise APIException(f"搜索请求失败: {e}") from e
except (json.JSONDecodeError, KeyError) as e:
raise APIException(f"解析搜索响应失败: {e}") from e
@@ -348,9 +345,8 @@ class NeteaseAPI:
'Referer': APIConstants.REFERER
}
response = await requests.post(APIConstants.PLAYLIST_DETAIL_API, data=data,
response = await _http.request('POST', APIConstants.PLAYLIST_DETAIL_API, data=data,
headers=headers, cookies=cookies, timeout=30)
response.raise_for_status()
result = response.json()
# 非 dict 响应(null/数组)给明确错误
@@ -384,7 +380,7 @@ class NeteaseAPI:
info['failed_batches'] = failed_batches
return info
except requests.RequestException as e:
except UpstreamError as e:
raise APIException(f"获取歌单详情请求失败: {e}") from e
except (json.JSONDecodeError, KeyError) as e:
raise APIException(f"解析歌单详情响应失败: {e}") from e
@@ -409,8 +405,7 @@ class NeteaseAPI:
'Referer': APIConstants.REFERER
}
response = await requests.get(url, headers=headers, cookies=cookies, timeout=30)
response.raise_for_status()
response = await _http.request('GET', url, headers=headers, cookies=cookies, timeout=30)
result = response.json()
_ensure_dict(result, 'get_album_detail 接口响应异常(非标准 JSON 对象)')
@@ -440,7 +435,7 @@ class NeteaseAPI:
})
return info
except requests.RequestException as e:
except UpstreamError as e:
raise APIException(f"获取专辑详情请求失败: {e}") from e
except (json.JSONDecodeError, KeyError) as e:
raise APIException(f"解析专辑详情响应失败: {e}") from e
@@ -460,8 +455,7 @@ class NeteaseAPI:
'Referer': APIConstants.REFERER
}
response = await requests.get(APIConstants.TOPLIST_API, headers=headers, timeout=30)
response.raise_for_status()
response = await _http.request('GET', APIConstants.TOPLIST_API, headers=headers, timeout=30)
result = response.json()
_ensure_dict(result, 'get_toplist 接口响应异常(非标准 JSON 对象)')
@@ -481,7 +475,7 @@ class NeteaseAPI:
'topTitle': item.get('ToplistType', '')
})
return toplists
except requests.RequestException as e:
except UpstreamError as e:
raise APIException(f"获取榜单请求失败: {e}") from e
except (json.JSONDecodeError, KeyError) as e:
raise APIException(f"解析榜单响应失败: {e}") from e
@@ -505,8 +499,7 @@ class NeteaseAPI:
'Referer': APIConstants.REFERER
}
response = await requests.get(url, headers=headers, timeout=30)
response.raise_for_status()
response = await _http.request('GET', url, headers=headers, timeout=30)
result = response.json()
_ensure_dict(result, 'get_artist_hot_songs 接口响应异常(非标准 JSON 对象)')
@@ -535,7 +528,7 @@ class NeteaseAPI:
},
'songs': songs
}
except requests.RequestException as e:
except UpstreamError as e:
raise APIException(f"获取歌手信息请求失败: {e}") from e
except (json.JSONDecodeError, KeyError) as e:
raise APIException(f"解析歌手信息响应失败: {e}") from e
@@ -561,8 +554,7 @@ class NeteaseAPI:
'Referer': APIConstants.REFERER
}
response = await requests.get(url, headers=headers, timeout=30)
response.raise_for_status()
response = await _http.request('GET', url, headers=headers, timeout=30)
result = response.json()
_ensure_dict(result, 'get_song_comments 接口响应异常(非标准 JSON 对象)')
@@ -602,7 +594,7 @@ class NeteaseAPI:
'hotComments': [_comment_dict(c) for c in (result.get('hotComments') or [])],
'comments': [_comment_dict(c) for c in (result.get('comments') or [])],
}
except requests.RequestException as e:
except UpstreamError as e:
raise APIException(f"获取评论请求失败: {e}") from e
except (json.JSONDecodeError, KeyError) as e:
raise APIException(f"解析评论响应失败: {e}") from e
@@ -622,9 +614,8 @@ class NeteaseAPI:
'Referer': APIConstants.REFERER
}
response = await requests.get(APIConstants.USER_ACCOUNT_API, headers=headers,
response = await _http.request('GET', APIConstants.USER_ACCOUNT_API, headers=headers,
cookies=cookies, timeout=30)
response.raise_for_status()
result = response.json()
_ensure_dict(result, 'get_account_info 接口响应异常(非标准 JSON 对象)')
@@ -674,9 +665,8 @@ class NeteaseAPI:
page_size = 100
for _ in range(20): # 有界翻页,防上游 more 异常导致无限循环
params = {'uid': user_id, 'limit': page_size, 'offset': offset}
response = await requests.get(APIConstants.USER_PLAYLIST_API, params=params,
response = await _http.request('GET', APIConstants.USER_PLAYLIST_API, params=params,
headers=headers, cookies=cookies, timeout=30)
response.raise_for_status()
result = response.json()
_ensure_dict(result, 'get_user_playlists 接口响应异常(非标准 JSON 对象)')
@@ -706,7 +696,7 @@ class NeteaseAPI:
break
offset += page_size
return playlists
except requests.RequestException as e:
except UpstreamError as e:
raise APIException(f"获取用户歌单请求失败: {e}") from e
except (json.JSONDecodeError, KeyError) as e:
raise APIException(f"解析用户歌单响应失败: {e}") from e
@@ -790,15 +780,14 @@ class NeteaseAPI:
'User-Agent': APIConstants.USER_AGENT,
'Referer': APIConstants.REFERER
}
response = await requests.post(APIConstants.PLAYLIST_SUBSCRIBE_API, data=data,
response = await _http.request('POST', APIConstants.PLAYLIST_SUBSCRIBE_API, data=data,
headers=headers, cookies=cookies, timeout=30)
response.raise_for_status()
result = response.json()
_ensure_dict(result, 'subscribe_playlist 接口响应异常(非标准 JSON 对象)')
if result.get('code') != 200:
raise APIException(f"收藏歌单失败: {result.get('message', '未知错误')}")
return result
except requests.RequestException as e:
except UpstreamError as e:
raise APIException(f"收藏歌单请求失败: {e}") from e
except (json.JSONDecodeError, KeyError) as e:
raise APIException(f"解析收藏歌单响应失败: {e}") from e
@@ -824,15 +813,14 @@ class NeteaseAPI:
'User-Agent': APIConstants.USER_AGENT,
'Referer': APIConstants.REFERER
}
response = await requests.post(APIConstants.ALBUM_SUBSCRIBE_API, data=data,
response = await _http.request('POST', APIConstants.ALBUM_SUBSCRIBE_API, data=data,
headers=headers, cookies=cookies, timeout=30)
response.raise_for_status()
result = response.json()
_ensure_dict(result, 'subscribe_album 接口响应异常(非标准 JSON 对象)')
if result.get('code') != 200:
raise APIException(f"收藏专辑失败: {result.get('message', '未知错误')}")
return result
except requests.RequestException as e:
except UpstreamError as e:
raise APIException(f"收藏专辑请求失败: {e}") from e
except (json.JSONDecodeError, KeyError) as e:
raise APIException(f"解析收藏专辑响应失败: {e}") from e
@@ -875,9 +863,8 @@ class NeteaseAPI:
'User-Agent': APIConstants.USER_AGENT,
'Referer': APIConstants.REFERER,
}
response = await requests.post(APIConstants.COMMENT_POST_API, data=body,
response = await _http.request('POST', APIConstants.COMMENT_POST_API, data=body,
headers=headers, cookies=cookies or {}, timeout=30)
response.raise_for_status()
text = response.text.strip()
if not text:
@@ -889,7 +876,7 @@ class NeteaseAPI:
if result.get('code') != 200:
raise APIException(f"评论失败: {result.get('message', '未知错误')}")
return result
except requests.RequestException as e:
except UpstreamError as e:
raise APIException(f"评论请求失败: {e}") from e
except (json.JSONDecodeError, KeyError) as e:
raise APIException(f"解析评论响应失败: {e}") from e
+32 -25
View File
@@ -7,7 +7,13 @@
import asyncio
from typing import Any
from core import aiohttp_compat as requests # 异步 HTTP 垫片(requests 同形 API)
from core.http_client import (
HTTPResult,
UpstreamError,
UpstreamHTTPError,
UpstreamTooLarge,
)
from core.http_client import client as _http
from .constants import (
APIConstants,
@@ -41,7 +47,12 @@ class HTTPClient:
async def post_request(url: str, params: str, cookies: dict[str, str]) -> str:
"""发送POST请求并返回文本响应。
网络异常/5xx 做一次 1 秒退避重试(全站 API 调用此前零重试,
上游抖动即整首失败);4xx(429/460 风控)不重试——立即重试会加剧风控。"""
上游抖动即整首失败);4xx(429/460 风控)不重试——立即重试会加剧风控。
走 core.http_client.request_once(单次、不重试):重试策略由本函数自管,
非 2xx 由 request_once 抛 UpstreamHTTPError,按 .status 分流——
与原先「垫片只在 raise_for_status() 时才抛 HTTPError」的分流等价。
"""
headers = {
'User-Agent': APIConstants.USER_AGENT,
'Referer': APIConstants.REFERER,
@@ -53,20 +64,21 @@ class HTTPClient:
last_err = None
for attempt in range(2):
try:
response = await requests.post(url, headers=headers, cookies=request_cookies,
response = await _http.request_once(
'POST', url, headers=headers, cookies=request_cookies,
data={"params": params}, timeout=30)
if response.status_code >= 500 and attempt == 0:
return response.text
except UpstreamHTTPError as e:
if e.status >= 500 and attempt == 0:
await asyncio.sleep(1.0)
continue
response.raise_for_status()
if len(response.content) > _MAX_UPSTREAM_JSON_BYTES:
raise APIException("上游响应体异常(超过大小上限)")
return response.text
except requests.HTTPError as e:
# 4xx(429/460 风控)不重试——原实现 HTTPError 落入
# 下方 RequestException 分支被重试一次,与 意图相反
# 下方异常分支被重试一次,与意图相反
raise APIException(f"HTTP请求失败: {e}") from e
except requests.RequestException as e:
except UpstreamTooLarge as e:
# 响应体超限是确定性失败,重试无意义
raise APIException("上游响应体异常(超过大小上限)") from e
except UpstreamError as e:
last_err = e
if attempt == 0:
await asyncio.sleep(1.0)
@@ -74,7 +86,7 @@ class HTTPClient:
raise APIException(f"HTTP请求失败: {last_err}")
@staticmethod
async def post_request_full(url: str, params: str, cookies: dict[str, str]) -> requests.Response:
async def post_request_full(url: str, params: str, cookies: dict[str, str]) -> HTTPResult:
"""发送POST请求并返回完整响应对象(同 post_request 的有界重试)"""
headers = {
'User-Agent': APIConstants.USER_AGENT,
@@ -87,26 +99,21 @@ class HTTPClient:
last_err = None
for attempt in range(2):
try:
response = await requests.post(url, headers=headers, cookies=request_cookies,
response = await _http.request_once(
'POST', url, headers=headers, cookies=request_cookies,
data={"params": params}, timeout=30)
if response.status_code >= 500 and attempt == 0:
return response
except UpstreamHTTPError as e:
if e.status >= 500 and attempt == 0:
await asyncio.sleep(1.0)
continue
response.raise_for_status()
if len(response.content) > _MAX_UPSTREAM_JSON_BYTES:
raise APIException("上游响应体异常(超过大小上限)")
return response
except requests.HTTPError as e:
# 4xx 不重试(同上)
raise APIException(f"HTTP请求失败: {e}") from e
except requests.RequestException as e:
except UpstreamTooLarge as e:
raise APIException("上游响应体异常(超过大小上限)") from e
except UpstreamError as e:
last_err = e
if attempt == 0:
await asyncio.sleep(1.0)
continue
raise APIException(f"HTTP请求失败: {last_err}")
# 上游 JSON 响应体上限——requests 已全量入内存,此处拦截的是
# 异常/被劫持上游的超大 body 进入后续 .json() 解析与传播(音频/封面已有字节预算)
_MAX_UPSTREAM_JSON_BYTES = 8 * 1024 * 1024
-25
View File
@@ -1,25 +0,0 @@
# -*- coding: utf-8 -*-
"""同步兼容函数接口(内部经 run_sync 桥接异步实现)。
(自 music_api.py 拆出,行为与原实现一致。)
⚠️ 迁移期残留:本模块的定位是「同步代码调用异步实现」的桥接门面。调用方陆续
改为原生异步(`await NeteaseAPI().xxx()`)后,这里的方法即成为死代码,应删除而
非新增。当前仅 `url_v1` 仍有唯一调用方(`music_api.podcast` 的同步解析路径),
故本文件只保留它——其余 16 个同步门面已随调用方异步化一并移除。
"""
from typing import Any
from core.async_bridge import run_sync # 模块级同步兼容接口用的同步/异步桥
from .api import (
NeteaseAPI,
)
# 向后兼容的函数接口
def url_v1(song_id: int, level: str, cookies: dict[str, str]) -> dict[str, Any]:
"""获取歌曲URL(向后兼容)"""
api = NeteaseAPI()
return run_sync(api.get_song_url(song_id, level, cookies))
+46 -53
View File
@@ -10,10 +10,12 @@
说明:
- 播客单集的音频本质上是网易云曲库中的 song(program.mainSong),
因此直链复用 music_api.url_v1(eapi song/enhance/player/url/v1)获取。
因此直链复用 NeteaseAPI.get_song_url(eapi song/enhance/player/url/v1)获取。
- 支持格式:mp3 / m4a / aac 等(以接口返回 type 为准)。
- 支持链接协议:music.163.com/djradio?id=(电台)、/program?id= 与 /sound?id=(单集)、
以及 163cn.tv 短链。
- 本模块**整体异步**:HTTP 统一走 core.http_client(进程级连接池 + 有界重试 +
8MB 响应体上限),调用方直接 await,不再需要同步门面/线程包装。
"""
import json
@@ -21,9 +23,10 @@ import re
import time
from typing import Any
from core import sync_http as requests # 同步门面(底层 aiohttp)
from core.http_client import UpstreamError
from core.http_client import client as _http
from .compat import url_v1
from .api import NeteaseAPI
from .constants import APIConstants
from .errors import APIException
@@ -51,26 +54,19 @@ class PodcastConstants:
DEFAULT_HOT_CATE_IDS = [6, 3, 8, 10001]
# 与 music_api 同口径的上游 JSON 响应体上限
_MAX_UPSTREAM_JSON_BYTES = 8 * 1024 * 1024
def _post_json(url: str, data: dict[str, Any], cookies: dict[str, str] | None = None) -> dict[str, Any]:
"""发送表单 POST 请求并返回 JSON"""
async def _post_json(url: str, data: dict[str, Any], cookies: dict[str, str] | None = None) -> dict[str, Any]:
"""发送表单 POST 请求并返回 JSON(响应体上限由 core.http_client 统一把关)"""
headers = {
'User-Agent': APIConstants.USER_AGENT,
'Referer': APIConstants.REFERER,
}
try:
response = requests.post(url, data=data, headers=headers,
response = await _http.request('POST', url, data=data, headers=headers,
cookies=cookies or {}, timeout=30)
response.raise_for_status()
if len(response.content) > _MAX_UPSTREAM_JSON_BYTES:
raise APIException('播客接口响应体异常(超过大小上限)')
result = response.json()
if not isinstance(result, dict):
raise APIException('播客接口响应异常(非标准 JSON 对象)')
except requests.RequestException as e:
except UpstreamError as e:
raise APIException(f"播客接口请求失败: {e}") from e
except json.JSONDecodeError as e:
raise APIException(f"解析播客接口响应失败: {e}") from e
@@ -180,7 +176,7 @@ def _norm_program(raw: dict[str, Any]) -> dict[str, Any]:
_SHORTLINK_CACHE: dict[str, str] = {}
def _resolve_short_link(url: str) -> str:
async def _resolve_short_link(url: str) -> str:
"""解析 163cn.tv 短链,返回跳转后的真实链接(仅接受整串短链,子串不触发请求)"""
if not re.match(r'^https?://(www\.)?163cn\.tv/\S+$', str(url or '').strip(), re.IGNORECASE):
return url
@@ -192,10 +188,10 @@ def _resolve_short_link(url: str) -> str:
return cached
resp = None
try:
resp = requests.get(url, allow_redirects=False, timeout=10,
resp = await _http.open('GET', url, allow_redirects=False, timeout=10,
headers={'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64)'})
real = resp.headers.get('Location', url)
except requests.RequestException:
except UpstreamError:
return url
finally:
if resp is not None:
@@ -210,7 +206,7 @@ def _resolve_short_link(url: str) -> str:
return real
def extract_podcast_ref(raw: str) -> dict[str, str]:
async def extract_podcast_ref(raw: str) -> dict[str, str]:
"""从用户输入中提取播客引用(正规 URL 解析,替代子串匹配与非数字剔除)
Returns:
@@ -223,7 +219,7 @@ def extract_podcast_ref(raw: str) -> dict[str, str]:
text = str(raw or '').strip()
# 短链先还原(仅当整串就是一个短链 URL)
if '163cn.tv' in text.lower():
text = _resolve_short_link(text)
text = await _resolve_short_link(text)
# 有 URL 结构:按路径判断类型,query 正规解析 id(参数顺序无关)
if '://' in text or text.startswith('//'):
@@ -256,7 +252,7 @@ def extract_podcast_ref(raw: str) -> dict[str, str]:
# ============ 核心 API ============
def search_podcast(keywords: str, cookies: dict[str, str] | None = None,
async def search_podcast(keywords: str, cookies: dict[str, str] | None = None,
limit: int = 20, offset: int = 0,
search_type: str = 'all') -> dict[str, Any]:
"""搜索播客
@@ -278,18 +274,17 @@ def search_podcast(keywords: str, cookies: dict[str, str] | None = None,
result: dict[str, Any] = {'radios': [], 'voices': []}
def _do_search(stype: int) -> dict[str, Any]:
return _post_json(PodcastConstants.SEARCH_API,
{'s': keywords, 'type': stype, 'limit': limit, 'offset': offset},
cookies)
if search_type in ('all', 'radio'):
res = _do_search(1009)
res = await _post_json(PodcastConstants.SEARCH_API,
{'s': keywords, 'type': 1009, 'limit': limit, 'offset': offset},
cookies)
radios = (res.get('result') or {}).get('djRadios') or []
result['radios'] = [_norm_radio(r) for r in radios]
if search_type in ('all', 'voice'):
res = _do_search(2000)
res = await _post_json(PodcastConstants.SEARCH_API,
{'s': keywords, 'type': 2000, 'limit': limit, 'offset': offset},
cookies)
rdata = res.get('result') or {}
voices = rdata.get('songs') or rdata.get('voices') or rdata.get('programs') or []
result['voices'] = [_norm_program(v) for v in voices]
@@ -297,13 +292,13 @@ def search_podcast(keywords: str, cookies: dict[str, str] | None = None,
return result
def get_radio_detail(radio_id: Any, cookies: dict[str, str] | None = None) -> dict[str, Any]:
async def get_radio_detail(radio_id: Any, cookies: dict[str, str] | None = None) -> dict[str, Any]:
"""获取播客电台(djradio)详情"""
try:
rid = int(str(radio_id).strip())
except (TypeError, ValueError):
raise APIException('无效的电台ID') from None
result = _post_json(PodcastConstants.DJRADIO_GET_API, {'id': rid}, cookies)
result = await _post_json(PodcastConstants.DJRADIO_GET_API, {'id': rid}, cookies)
# 正常返回键为 djRadio;不存在时 code=404(_post_json 已抛错)或 data 为空
data = result.get('djRadio') or result.get('data') or {}
if not data:
@@ -311,7 +306,7 @@ def get_radio_detail(radio_id: Any, cookies: dict[str, str] | None = None) -> di
return _norm_radio(data)
def get_radio_programs(radio_id: Any, cookies: dict[str, str] | None = None,
async def get_radio_programs(radio_id: Any, cookies: dict[str, str] | None = None,
limit: int = 30, offset: int = 0, asc: bool = True) -> dict[str, Any]:
"""获取电台下的单集列表(分页)
@@ -329,7 +324,7 @@ def get_radio_programs(radio_id: Any, cookies: dict[str, str] | None = None,
except (TypeError, ValueError):
raise APIException('无效的电台ID') from None
limit = max(1, min(int(limit or 30), 100))
result = _post_json(PodcastConstants.DJ_PROGRAM_BYRADIO_API,
result = await _post_json(PodcastConstants.DJ_PROGRAM_BYRADIO_API,
{'radioId': rid, 'limit': limit, 'offset': max(0, int(offset or 0)),
'asc': 'true' if asc else 'false'},
cookies)
@@ -343,20 +338,20 @@ def get_radio_programs(radio_id: Any, cookies: dict[str, str] | None = None,
}
def get_program_detail(program_id: Any, cookies: dict[str, str] | None = None) -> dict[str, Any]:
async def get_program_detail(program_id: Any, cookies: dict[str, str] | None = None) -> dict[str, Any]:
"""获取单集(program)详情"""
try:
pid = int(str(program_id).strip())
except (TypeError, ValueError):
raise APIException('无效的单集ID') from None
result = _post_json(PodcastConstants.DJ_PROGRAM_DETAIL_API, {'id': pid}, cookies)
result = await _post_json(PodcastConstants.DJ_PROGRAM_DETAIL_API, {'id': pid}, cookies)
program = result.get('program') or {}
if not program:
raise APIException('未找到该播客单集(可能已下架或ID错误)')
return _norm_program(program)
def get_categories() -> list[dict[str, Any]]:
async def get_categories() -> list[dict[str, Any]]:
"""获取播客分类列表(GET 接口)
Returns:
@@ -367,13 +362,10 @@ def get_categories() -> list[dict[str, Any]]:
'Referer': APIConstants.REFERER,
}
try:
response = requests.get(PodcastConstants.DJRADIO_CATEGORY_API,
response = await _http.request('GET', PodcastConstants.DJRADIO_CATEGORY_API,
headers=headers, timeout=30)
response.raise_for_status()
if len(response.content) > _MAX_UPSTREAM_JSON_BYTES:
raise APIException('播客接口响应体异常(超过大小上限)')
result = response.json()
except requests.RequestException as e:
except UpstreamError as e:
raise APIException(f"获取播客分类失败: {e}") from e
except json.JSONDecodeError as e:
raise APIException(f"解析播客分类响应失败: {e}") from e
@@ -396,7 +388,7 @@ def get_categories() -> list[dict[str, Any]]:
return cats
def get_hot_radios(cookies: dict[str, str] | None = None,
async def get_hot_radios(cookies: dict[str, str] | None = None,
limit: int = 30, offset: int = 0,
cate_id: int | None = None) -> list[dict[str, Any]]:
"""获取分类下热门播客
@@ -423,7 +415,7 @@ def get_hot_radios(cookies: dict[str, str] | None = None,
cate_ids.append(cid)
for cid in cate_ids:
result = _post_json(PodcastConstants.DJRADIO_HOT_API,
result = await _post_json(PodcastConstants.DJRADIO_HOT_API,
{'limit': limit, 'offset': max(0, int(offset or 0)), 'cateId': cid},
cookies)
radios = [_norm_radio(r) for r in (result.get('djRadios') or [])]
@@ -455,7 +447,7 @@ def _falldown_order(level: str) -> list[str]:
return list(dict.fromkeys(order)) # 去重保序
def resolve_episode_url(song_id: Any, cookies: dict[str, str] | None = None,
async def resolve_episode_url(song_id: Any, cookies: dict[str, str] | None = None,
level: str = 'auto') -> dict[str, Any] | None:
"""解析单集音频直链(带音质降级)
@@ -473,9 +465,10 @@ def resolve_episode_url(song_id: Any, cookies: dict[str, str] | None = None,
sid = int(str(song_id).strip())
except (TypeError, ValueError):
return None
api = NeteaseAPI()
for lv in order:
try:
res = url_v1(sid, lv, cookies or {})
res = await api.get_song_url(sid, lv, cookies or {})
except (APIException, ValueError):
continue
data = (res or {}).get('data') or []
@@ -495,7 +488,7 @@ def resolve_episode_url(song_id: Any, cookies: dict[str, str] | None = None,
# ============ 组合能力 ============
def parse_podcast_ref(raw: str, cookies: dict[str, str] | None = None,
async def parse_podcast_ref(raw: str, cookies: dict[str, str] | None = None,
page_size: int = 30) -> dict[str, Any]:
"""智能解析用户输入:电台链接 → 电台详情+单集列表;单集链接 → 单集详情
@@ -503,27 +496,27 @@ def parse_podcast_ref(raw: str, cookies: dict[str, str] | None = None,
{'kind': 'radio', 'radio': {...}, 'programs': [...], 'count': n, 'has_more': bool}
或 {'kind': 'program', 'program': {...}}
"""
ref = extract_podcast_ref(raw)
ref = await extract_podcast_ref(raw)
if ref['kind'] == 'radio':
radio = get_radio_detail(ref['id'], cookies)
page = get_radio_programs(ref['id'], cookies, limit=page_size, offset=0, asc=True)
radio = await get_radio_detail(ref['id'], cookies)
page = await get_radio_programs(ref['id'], cookies, limit=page_size, offset=0, asc=True)
return {'kind': 'radio', 'radio': radio, **page}
if ref['kind'] == 'program':
return {'kind': 'program', 'program': get_program_detail(ref['id'], cookies)}
return {'kind': 'program', 'program': await get_program_detail(ref['id'], cookies)}
# 纯数字:先试单集,再试电台
try:
return {'kind': 'program', 'program': get_program_detail(ref['id'], cookies)}
return {'kind': 'program', 'program': await get_program_detail(ref['id'], cookies)}
except APIException:
pass
radio = get_radio_detail(ref['id'], cookies)
page = get_radio_programs(ref['id'], cookies, limit=page_size, offset=0, asc=True)
radio = await get_radio_detail(ref['id'], cookies)
page = await get_radio_programs(ref['id'], cookies, limit=page_size, offset=0, asc=True)
return {'kind': 'radio', 'radio': radio, **page}
def list_radio_all_program_ids(radio_id: Any, cookies: dict[str, str] | None = None,
async def list_radio_all_program_ids(radio_id: Any, cookies: dict[str, str] | None = None,
max_count: int = 200) -> list[dict[str, Any]]:
"""拉取电台全部单集(用于整台打包,正序),返回 [{'id','name','main_song_id','serial_num','cover_url'}]
@@ -537,7 +530,7 @@ def list_radio_all_program_ids(radio_id: Any, cookies: dict[str, str] | None = N
page_size = 100
max_pages = 30 # 有界保护:即使上游 has_more 恒真也最多翻 30 页
for _ in range(max_pages):
page = get_radio_programs(radio_id, cookies, limit=page_size, offset=offset, asc=True)
page = await get_radio_programs(radio_id, cookies, limit=page_size, offset=offset, asc=True)
new_count = 0
for p in page['programs']:
if p['id'] in seen_ids:
+10 -5
View File
@@ -293,19 +293,24 @@ class QRLoginClient:
print(f"获取登录信息失败: {e}")
def logout(self) -> bool:
"""登出(清除 Cookie)"""
"""登出(清除 Cookie)。
输出走 logger 而不是 print:本方法既被 CLI(main)调用,也被 admin 端点
(/api/v1/admin/cookie/delete)调用——在 web 语境里 print 会把内容直接喷到
服务端 stdout,绕过 logging 配置、无法被收集,也会污染容器日志。
"""
try:
if self.cookie_file.exists():
backup = self._backup_cookie("logout")
if backup:
print(f"Cookie 已备份到: {backup}")
self.logger.info(f"Cookie 已备份到: {backup}")
if self._clear_cookie():
print("已成功登出")
self.logger.info("已成功登出")
return True
print("登出失败")
self.logger.error("登出失败")
return False
except Exception as e:
print(f"登出时发生错误: {e}")
self.logger.error(f"登出时发生错误: {e}")
return False
+3 -7
View File
@@ -2,9 +2,8 @@
"""网易云音乐下载器(标签写入基于 mutagen,纯 Python)
提供功能:
- 同步 / 异步下载(本地落盘 / 内存返回)
- 批量并发下载
- 进度查询
- 异步下载(本地落盘)
- 批量并发下载(由上层服务编排)
- 基于 mutagen 的标签 + 封面写入(MP3/FLAC/M4A/MP4/OGG/Opus)
设计要点:
@@ -12,6 +11,7 @@
2. 传输通道二选一:aria2c 常驻 RPC(多连接)优先,不可用时回落内置 aiohttp 流式
3. 标签写入按容器分派(ID3v2.3 / Vorbis comment / iTunes ilst),全部原子替换
4. 扩展名优先使用 API 返回的 type 字段,URL/Content-Type 兜底
5. 全链路异步:HTTP 走 core.http_client,阻塞 IO/CPU 经 asyncio.to_thread 卸载
"""
# 兼容门面:实现已按职责拆分到同包各子模块,此处原样再导出。
@@ -27,11 +27,9 @@ from ._concurrency import (
get_file_lock,
)
from ._http import (
_SESSION_LOCAL,
DEFAULT_UA,
DL_CHUNK_SIZE,
_sniff_audio_header,
get_http_session,
)
from ._log import (
logger,
@@ -68,7 +66,6 @@ __all__ = [
'_COVER_CACHE_ORDER',
'_FILE_LOCKS',
'_FILE_LOCKS_GUARD',
'_SESSION_LOCAL',
'CountingSemaphore',
'DownloadCanceled',
'DownloadException',
@@ -82,6 +79,5 @@ __all__ = [
'_sniff_audio_header',
'fetch_cover_bytes',
'get_file_lock',
'get_http_session',
'logger',
]
+36 -26
View File
@@ -3,6 +3,10 @@
from __future__ import annotations
import asyncio
import sys
from dataclasses import asdict
from .downloader import (
MusicDownloader,
)
@@ -12,44 +16,45 @@ from .models import (
# ============ CLI 入口 ============
if __name__ == "__main__":
import sys
from dataclasses import asdict
downloader = MusicDownloader()
# 无参数:打印模块信息
if len(sys.argv) < 2:
def _print_usage() -> None:
"""无参数时打印模块信息与用法。"""
print("网易云音乐下载器")
print("支持的音质: standard / exhigh / lossless / hires / sky / jyeffect / jymaster / dolby")
print("支持的功能: 同步/批量下载、标签与封面写入(mutagen 纯 Python)")
print("支持的功能: 单曲/批量下载、标签与封面写入(mutagen 纯 Python)")
print()
print("用法:")
print(" python music_downloader.py <music_id> [quality] # 仅打印信息")
print(" python music_downloader.py <music_id> [quality] --download # 打印并下载")
print(" python -m music_downloader <music_id> [quality] # 仅打印信息")
print(" python -m music_downloader <music_id> [quality] --download # 打印并下载")
print()
print("示例:")
print(" python music_downloader.py 123456")
print(" python music_downloader.py 123456 lossless")
print(" python music_downloader.py 123456 hires --download")
sys.exit(0)
print(" python -m music_downloader 123456")
print(" python -m music_downloader 123456 lossless")
print(" python -m music_downloader 123456 hires --download")
async def _main(argv: list[str]) -> int:
"""CLI 主体(下载链路是协程,故整体在 asyncio.run 里跑)。返回进程退出码。"""
downloader = MusicDownloader()
if len(argv) < 2:
_print_usage()
return 0
# 解析参数
try:
music_id = int(sys.argv[1])
music_id = int(argv[1])
except ValueError:
print(f"错误: music_id 必须是整数,收到: {sys.argv[1]!r}")
sys.exit(1)
print(f"错误: music_id 必须是整数,收到: {argv[1]!r}")
return 1
quality = sys.argv[2] if len(sys.argv) > 2 and not sys.argv[2].startswith("--") else "lossless"
do_download = "--download" in sys.argv
quality = argv[2] if len(argv) > 2 and not argv[2].startswith("--") else "lossless"
do_download = "--download" in argv
# 打印音乐信息
try:
info = downloader.get_music_info(music_id, quality)
info = await downloader.get_music_info(music_id, quality)
except DownloadException as e:
print(f"获取音乐信息失败: {e}")
sys.exit(1)
return 1
print()
print("=" * 70)
@@ -72,13 +77,18 @@ if __name__ == "__main__":
print("=" * 70)
# 可选:触发下载
if do_download:
print()
print("开始下载...")
result = downloader.download_music_file(music_id, quality)
result = await downloader.download_music_file(music_id, quality)
if result.success:
print(f"下载完成: {result.file_path} ({result.file_size} 字节)")
else:
print(f"下载失败: {result.error_message}")
sys.exit(1)
return 1
return 0
if __name__ == "__main__":
sys.exit(asyncio.run(_main(sys.argv)))
+191 -49
View File
@@ -9,13 +9,21 @@
- RPC 只监听 127.0.0.1 并强制随机 secret —— 无鉴权的 aria2 RPC 允许本机任意
进程操纵下载队列(读写任意文件),暴露到容器外等于开了一个后门;
- aria2c 缺失 / 进程早退 / RPC 未就绪,一律返回 None 静默降级,
由调用方回落到既有的 core.sync_http 流式通道;
由调用方回落到内置的 core.http_client 流式通道;
- aria2 没有「单文件体积上限」选项,故用轮询 tellStatus 实现体积守卫与取消响应。
异步口径:RPC 由本模块直接打 aria2 的 JSON-RPC(HTTP POST + JSON-RPC 2.0),
全程 await——原先依赖的 aria2p 内部是 requests + websocket 的同步实现,每次
RPC 都要经 asyncio.to_thread 卸载,是本项目最后一处「把同步 HTTP 藏在 to_thread
里」的地方。轮询间隔用 asyncio.sleep 让出事件循环。剩下的 to_thread 只有两处,
都是真正的阻塞卸载:进程创建(subprocess.Popen)与强杀回收(kill + wait)。
"""
from __future__ import annotations
import asyncio
import atexit
import contextlib
import secrets
import shutil
import socket
@@ -26,7 +34,8 @@ from collections.abc import Callable
from pathlib import Path
from typing import Any, Protocol
from core import sync_http as requests # 同步门面(底层 aiohttp)
from core.http_client import UpstreamNetworkError
from core.http_client import client as _http
from ._http import DEFAULT_UA
from ._log import logger
@@ -40,6 +49,8 @@ class _ProcessHandle(Protocol):
也让测试可以用轻量替身驱动生命周期逻辑。
"""
returncode: int | None
def poll(self) -> int | None: ...
def terminate(self) -> None: ...
def kill(self) -> None: ...
@@ -59,11 +70,30 @@ _DEFAULT_MAX_BYTES = 500 * 1024 * 1024
_SPLIT = 5
_MIN_SPLIT_SIZE = '1M'
_lock = threading.Lock()
# 单次 RPC 超时:本机回环,正常在毫秒级;给足余量以免大库时误判网络失败
_RPC_TIMEOUT = 10.0
# tellStatus 返回的**终态**:任务已不在下载队列,只能清结果表
_TERMINAL_STATUSES = frozenset({'complete', 'error', 'removed'})
_backend: Aria2Backend | None = None
_unavailable = False # 启动失败后置位,不再反复重试
# 启动单飞锁按事件循环隔离:asyncio.Lock 绑定创建它的 loop,应用主 loop 与测试
# 各自的 loop 并存,跨 loop 复用会抛 "bound to a different event loop"。
_START_LOCKS: dict[int, asyncio.Lock] = {}
def _start_lock() -> asyncio.Lock:
key = id(asyncio.get_running_loop())
lock = _START_LOCKS.get(key)
if lock is None:
lock = asyncio.Lock()
_START_LOCKS[key] = lock
return lock
def _free_port() -> int:
"""向内核要一个空闲端口(bind 后立即释放,交给 aria2c 监听)。"""
with socket.socket() as s:
@@ -71,11 +101,86 @@ def _free_port() -> int:
return int(s.getsockname()[1])
class Aria2RpcError(RuntimeError):
"""aria2 在 JSON-RPC 响应体里回报的 error 对象(协议层失败,非 HTTP 层)。"""
class _Aria2Rpc:
"""aria2 JSON-RPC 极简客户端(只覆盖本模块用到的 5 个方法)。
替代 aria2p:后者内部是 requests + websocket 的同步实现,每个 RPC 都得
to_thread 卸载;而 aria2 的 RPC 就是朴素的 HTTP POST + JSON-RPC 2.0,
直接复用 core.http_client 的进程级连接池即可,且连接池本就为异步而生。
一律走 ``request_once``(**不重试**),与 aria2p 口径一致:addUri /
forceRemove 这类变更调用若被重发可能重复提交,RPC 抖动由上层 ``fetch``
收敛成 UpstreamNetworkError、交给下载器的退避重试统一处理。
"""
def __init__(self, port: int, secret: str) -> None:
self._url = f'http://127.0.0.1:{port}/jsonrpc'
self._token = f'token:{secret}'
self._seq = 0
async def _call(self, method: str, *params: Any) -> Any:
self._seq += 1
body = {
'jsonrpc': '2.0',
'id': str(self._seq),
'method': f'aria2.{method}',
'params': [self._token, *params],
}
resp = await _http.request_once(
'POST', self._url, json_body=body, timeout=_RPC_TIMEOUT)
data = resp.json()
if not isinstance(data, dict):
raise Aria2RpcError(f'aria2 RPC 响应不是 JSON 对象: {resp.text[:200]!r}')
err = data.get('error')
if err:
raise Aria2RpcError(f'aria2 RPC {method} 失败: {err}')
return data.get('result')
async def get_version(self) -> str:
"""探活(启动时就绪判定)。"""
return str(await self._call('getVersion'))
async def add_uri(self, url: str, options: dict[str, str]) -> str:
"""提交下载,返回 GID。"""
return str(await self._call('addUri', [url], options))
async def tell_status(self, gid: str) -> dict[str, Any]:
"""查询任务状态(不传 keys,取全量字段)。"""
res = await self._call('tellStatus', gid)
return res if isinstance(res, dict) else {}
async def force_remove(self, gid: str) -> str:
"""停掉在途任务(不做联系 tracker 等耗时动作)。"""
return str(await self._call('forceRemove', gid))
async def remove_download_result(self, gid: str) -> str:
"""清掉已终结任务在结果表里的条目(否则结果表只增不减)。"""
return str(await self._call('removeDownloadResult', gid))
class _Rpc(Protocol):
"""aria2 RPC 的最小接口。
与 _ProcessHandle 同理:只声明本模块真正调用的方法,既不绑死具体实现,
也让测试能用轻量替身驱动整套状态机(不必起 HTTP 假服务器)。
"""
async def get_version(self) -> str: ...
async def add_uri(self, url: str, options: dict[str, str]) -> str: ...
async def tell_status(self, gid: str) -> dict[str, Any]: ...
async def force_remove(self, gid: str) -> str: ...
async def remove_download_result(self, gid: str) -> str: ...
class Aria2Backend:
"""aria2c RPC 后端。请通过 get_backend() 获取单例,不要直接构造。"""
def __init__(self, api: Any, proc: _ProcessHandle, port: int) -> None:
self._api = api
def __init__(self, rpc: _Rpc, proc: _ProcessHandle, port: int) -> None:
self._rpc = rpc
self._proc = proc
self.port = port
self._closed = False
@@ -83,14 +188,15 @@ class Aria2Backend:
# ---------- 生命周期 ----------
def close(self) -> None:
"""关闭 RPC 客户端并终止 aria2c 进程(幂等)。"""
"""终止 aria2c 进程(幂等)。
**同步**:由 atexit 调用,那时已经没有事件循环,且进程回收本就是阻塞操作。
这里不关 RPC 连接——它走 core.http_client 的**进程级共享连接池**,关掉会
连带影响内置流式通道(原先 aria2p 关的是它自己私有的 requests 会话)。
"""
if self._closed:
return
self._closed = True
try:
self._api.client.close()
except Exception:
pass # 客户端关闭失败不影响后续进程回收
if self._proc.poll() is None:
self._proc.terminate()
try:
@@ -101,7 +207,7 @@ class Aria2Backend:
# ---------- 下载 ----------
def fetch(
async def fetch(
self,
url: str,
target: Path,
@@ -118,7 +224,7 @@ class Aria2Backend:
Raises:
DownloadCanceled: cancel_event 置位(残片一并删除)
DownloadException: 体积超限(残片一并删除)、aria2 报告任务失败、总超时
requests.RequestException: RPC 抖动等可重试的网络类失败(保留残片续传)
UpstreamNetworkError: RPC 抖动等可重试的网络类失败(保留残片续传)
"""
options = {
'dir': str(target.parent),
@@ -137,52 +243,79 @@ class Aria2Backend:
options['referer'] = referer
try:
download = self._api.add_uris([url], options=options)
gid = await self._rpc.add_uri(url, options)
except Exception as e:
raise requests.RequestException(f'aria2 提交任务失败: {e}') from e
raise UpstreamNetworkError(f'aria2 提交任务失败: {e}') from e
deadline = time.monotonic() + timeout
status = ''
while True:
if cancel_event is not None and cancel_event.is_set():
self._abort(download, drop_files=True)
await self._abort(gid, target, drop_files=True, status=status)
raise DownloadCanceled('下载已取消')
try:
download.update()
st = await self._rpc.tell_status(gid)
except Exception as e:
# RPC 抖动按可重试的网络失败上报,交给外层退避重试
raise requests.RequestException(f'aria2 状态查询失败: {e}') from e
raise UpstreamNetworkError(f'aria2 状态查询失败: {e}') from e
done = int(download.completed_length or 0)
total = int(download.total_length or 0)
status = str(st.get('status') or '')
done = int(st.get('completedLength') or 0)
total = int(st.get('totalLength') or 0)
if progress_cb is not None:
try:
progress_cb(done, total)
except Exception:
pass # 进度回调绝不影响下载主流程
if done > max_bytes:
self._abort(download, drop_files=True)
await self._abort(gid, target, drop_files=True, status=status)
raise DownloadException(
f'文件超过大小上限 {max_bytes // (1024 * 1024)}MB,已中止')
if download.is_complete:
if status == 'complete':
self._drop_control_file(target)
return
if download.has_failed:
# 保留残片与控制文件,等待外层重试时续传
raise requests.RequestException(
f'aria2 下载失败: {download.error_message or download.status}')
if status == 'error':
# 保留残片与控制文件,等待外层重试时续传(**不调 _abort**)
raise UpstreamNetworkError(
f'aria2 下载失败: {st.get("errorMessage") or status}')
if time.monotonic() > deadline:
self._abort(download, drop_files=False)
raise requests.RequestException('aria2 下载超时')
time.sleep(_POLL_INTERVAL)
await self._abort(gid, target, drop_files=False, status=status)
raise UpstreamNetworkError('aria2 下载超时')
await asyncio.sleep(_POLL_INTERVAL)
# ---------- 内部 ----------
def _abort(self, download: Any, *, drop_files: bool) -> None:
"""强制移除任务。drop_files=True 时连同已下载字节一起删除。"""
async def _abort(
self, gid: str, target: Path, *, drop_files: bool, status: str,
) -> None:
"""移除任务。与 ``aria2p.remove(force=True, files=drop_files, clean=True)``
逐条对齐(该库的默认行为就是还要清控制文件):
- 已终结(complete/error/removed)→ 只能清结果表(removeDownloadResult);
在途 → forceRemove 先停掉,再顺手清结果表;
- 控制文件(target.aria2)**总是**删除——这是 aria2p 的 clean=True 默认值,
也是「取消 / 体积超限」后下轮必须重新下载的依据;
- 只有 drop_files 才删已落地的数据文件(超时场景保留残片供续传)。
任一步失败只告警、不影响主流程(与 aria2p 的 logger.exception 同口径),
且与 aria2p 一致:主移除动作失败时**不**删数据文件。
"""
removed = True
try:
self._api.remove([download], force=True, files=drop_files)
if status in _TERMINAL_STATUSES:
await self._rpc.remove_download_result(gid)
else:
await self._rpc.force_remove(gid)
with contextlib.suppress(Exception):
await self._rpc.remove_download_result(gid)
except Exception as e:
removed = False
logger.warning(f'终止 aria2 任务失败(不影响主流程): {e}')
self._drop_control_file(target)
if drop_files and removed:
try:
target.unlink(missing_ok=True)
except OSError as e:
logger.warning(f'删除 aria2 残片失败(不影响主流程): {e}')
@staticmethod
def _drop_control_file(target: Path) -> None:
@@ -194,37 +327,45 @@ class Aria2Backend:
pass
def get_backend() -> Aria2Backend | None:
async def get_backend() -> Aria2Backend | None:
"""返回 aria2 后端单例;aria2c 不可用/启动失败时返回 None(调用方降级)。
首次调用尝试拉起常驻进程,之后命中缓存;失败结果进程内记忆,不反复重试。
首次调用尝试拉起常驻进程并等 RPC 就绪(最长 ``_RPC_READY_TIMEOUT`` 秒,
等待用 ``await asyncio.sleep`` 让出事件循环,故本函数是**原生异步**、
不需要调用方再包 to_thread);之后命中缓存。失败结果进程内记忆,
不反复重试。并发首调由启动单飞锁收敛,只拉起一个进程。
"""
global _backend, _unavailable
if _unavailable:
return None
if _backend is not None:
return _backend
with _lock:
async with _start_lock():
if _unavailable:
return None
if _backend is None:
_backend = _try_start()
_backend = await _try_start()
if _backend is None:
_unavailable = True
return _backend
def _try_start() -> Aria2Backend | None:
def _reap(proc: _ProcessHandle) -> None:
"""强杀并回收子进程(阻塞)。只应由 asyncio.to_thread 调用。"""
try:
proc.kill()
except OSError:
return
with contextlib.suppress(Exception):
proc.wait(timeout=5)
async def _try_start() -> Aria2Backend | None:
"""拉起 aria2c --enable-rpc 并等待 RPC 就绪;任何失败都返回 None。"""
exe = shutil.which(ARIA2_BIN)
if not exe:
logger.info('未检测到 aria2c,下载走内置流式通道')
return None
try:
import aria2p
except ImportError as e:
logger.info(f'aria2p 不可用,下载走内置流式通道: {e}')
return None
port = _free_port()
secret = secrets.token_urlsafe(24)
@@ -239,33 +380,34 @@ def _try_start() -> Aria2Backend | None:
'--quiet=true',
]
try:
# 进程创建是真阻塞调用(Windows CreateProcess 数十毫秒),
# 且本函数在事件循环内被 await,故经 to_thread 卸载(真正的阻塞卸载)。
# exe 来自 shutil.which,参数全部由本模块构造,不经任何用户输入
proc = subprocess.Popen(
cmd, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL)
proc = await asyncio.to_thread(
subprocess.Popen, cmd, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL)
except OSError as e:
logger.warning(f'aria2c 启动失败,降级内置下载: {e}')
return None
client = aria2p.Client(host='http://127.0.0.1', port=port, secret=secret)
rpc = _Aria2Rpc(port, secret)
deadline = time.monotonic() + _RPC_READY_TIMEOUT
while True:
if proc.poll() is not None:
logger.warning(f'aria2c 进程提前退出(code={proc.returncode}),降级内置下载')
return None
try:
client.get_version()
await rpc.get_version()
break
except Exception:
if time.monotonic() > deadline:
logger.warning(f'aria2c RPC {_RPC_READY_TIMEOUT:.0f}s 内未就绪,降级内置下载')
proc.kill()
proc.wait(timeout=5)
await asyncio.to_thread(_reap, proc)
return None
time.sleep(0.2)
await asyncio.sleep(0.2)
atexit.register(_shutdown)
logger.info(f'aria2c 常驻 RPC 已就绪(127.0.0.1:{port},{_SPLIT} 连接/文件)')
return Aria2Backend(aria2p.API(client), proc, port)
return Aria2Backend(rpc, proc, port)
def _shutdown() -> None:
+26 -49
View File
@@ -1,11 +1,18 @@
# -*- coding: utf-8 -*-
"""文件锁与计数信号量。
"""文件锁与并发闸(asyncio 原语)。
(自 music_downloader.py 拆出,行为与原实现一致。)
(自 music_downloader.py 拆出。)
下载链路已整体异步化:同一进程、同一事件循环里跑多个任务,互斥不再靠线程
阻塞,而是靠 ``await`` 让出控制权。故此处两个原语都换成 asyncio 实现——
用 threading 锁会「持锁方在等事件循环、等待方又在事件循环上」而直接死锁。
两个计数器字典仍用 threading.Lock 保护:临界区只有字典读写、内部无 await,
不会把事件循环卡住,也让仍在同步线程里跑的监控循环能安全读取。
"""
from __future__ import annotations
import asyncio
import threading
from pathlib import Path
@@ -13,7 +20,7 @@ from pathlib import Path
# 防止并发下载同名歌曲(不同用户/任务)互相覆盖导致文件损坏
# 锁表带引用计数与回收——无人引用时从表中移除,防止长期运行无界增长
# 计数在注册表守卫锁内完成("取出引用"即计数 +1),消除"取引用后、acquire 前
# 被另一个线程回收、第三个线程拿到新锁对象"的互斥破坏窗口
# 被另一个任务回收、第三个任务拿到新锁对象"的互斥破坏窗口
_FILE_LOCKS: dict[str, RefCountedFileLock] = {}
_FILE_LOCKS_GUARD = threading.Lock()
@@ -23,11 +30,11 @@ class RefCountedFileLock:
"""带引用计数的文件锁:refs 表示"已持有或已登记即将持有"的数量,归零可回收"""
def __init__(self):
self._lock = threading.Lock()
self._lock = asyncio.Lock()
self.refs = 0
def acquire(self) -> None:
self._lock.acquire()
async def acquire(self) -> None:
await self._lock.acquire()
def release(self) -> None:
self._lock.release()
@@ -37,14 +44,14 @@ class RefCountedFileLock:
class _FileLockHandle:
"""兼容 `with get_file_lock(path):` 旧用法的句柄:语义与 file_lock_scope 一致"""
"""`async with get_file_lock(path):` 句柄:语义与引用计数回收口径同旧实现"""
def __init__(self, path: Path):
self._path = path
self._lock = None
self._key = ''
def __enter__(self) -> RefCountedFileLock:
async def __aenter__(self) -> RefCountedFileLock:
key = str(self._path)
with _FILE_LOCKS_GUARD:
lock = _FILE_LOCKS.get(key)
@@ -55,7 +62,7 @@ class _FileLockHandle:
self._lock = lock
self._key = key
try:
lock.acquire()
await lock.acquire()
except BaseException:
# acquire 被打断时回滚引用计数并摘除空表项,防锁表条目永久泄漏
with _FILE_LOCKS_GUARD:
@@ -67,7 +74,7 @@ class _FileLockHandle:
raise
return lock
def __exit__(self, exc_type, exc_val, exc_tb) -> None:
async def __aexit__(self, exc_type, exc_val, exc_tb) -> None:
if self._lock is None:
return
self._lock.release()
@@ -79,72 +86,42 @@ class _FileLockHandle:
def get_file_lock(path: Path) -> _FileLockHandle:
"""兼容入口:返回可用 with 语句的锁句柄(引用计数在守卫锁内登记,无回收竞态)"""
"""兼容入口:返回可用 `async with` 的锁句柄(引用计数在守卫锁内登记,无回收竞态)"""
return _FileLockHandle(path)
class CountingSemaphore:
"""带活跃/等待计数的信号量(线程安全),用于实时展示排队情况
"""带活跃/等待计数的并发闸(asyncio),用于实时展示排队情况
用法与 threading.BoundedSemaphore 一致(支持 with 语句),
额外维护 active(正在执行)/ waiting(排队等待)两个计数,
供前端实时展示排队人数与预计等待时间。
容量由 ``active`` 计数实现,等待靠 ``await asyncio.sleep`` 让出事件循环。
与旧实现(threading.BoundedSemaphore + 轮询 ``acquire(blocking=False)``)
同为非公平抢占、同样按 poll 粒度醒来,语义不变;差别只在等待不再阻塞线程。
"""
def __init__(self, value: int = 1):
self.capacity = max(1, int(value))
self._semaphore = threading.BoundedSemaphore(self.capacity)
self._lock = threading.Lock()
self.active = 0
self.waiting = 0
def acquire(self, blocking: bool = True) -> bool:
if not blocking:
if not self._semaphore.acquire(blocking=False):
return False
with self._lock:
self.active += 1
return True
with self._lock:
self.waiting += 1
try:
self._semaphore.acquire()
except BaseException:
with self._lock:
self.waiting = max(0, self.waiting - 1)
raise
with self._lock:
self.waiting = max(0, self.waiting - 1)
self.active += 1
return True
def acquire_cancellable(self, cancel_ev, poll: float = 1.0) -> bool:
async def acquire_cancellable(self, cancel_ev, poll: float = 1.0) -> bool:
"""可取消地等待名额:等待期间 cancel_ev 置位立即返回 False,
不再无期限阻塞线程与退款。返回 True 表示已获得名额。"""
import time as _time
不再无期限等待。返回 True 表示已获得名额。"""
with self._lock:
self.waiting += 1
try:
while True:
if cancel_ev is not None and cancel_ev.is_set():
return False
if self._semaphore.acquire(blocking=False):
with self._lock:
if self.active < self.capacity:
self.active += 1
return True
_time.sleep(poll)
await asyncio.sleep(poll)
finally:
with self._lock:
self.waiting = max(0, self.waiting - 1)
def release(self) -> None:
self._semaphore.release()
with self._lock:
self.active = max(0, self.active - 1)
def __enter__(self) -> CountingSemaphore:
self.acquire()
return self
def __exit__(self, exc_type, exc_val, exc_tb) -> None:
self.release()
+7 -23
View File
@@ -1,20 +1,18 @@
# -*- coding: utf-8 -*-
"""HTTP 会话复用与音频内容嗅探。
"""浏览器 UA、下载分块与音频内容嗅探。
(自 music_downloader.py 拆出,行为与原实现一致。)
(自 music_downloader.py 拆出。)
HTTP 本身已改走 core.http_client(进程级异步连接池),本模块不再持有任何
会话/连接状态——原先"线程本地 Session + keep-alive"的提速手段由 aiohttp
连接池统一承担,也就没有"同步 Session"这一层可留。
"""
from __future__ import annotations
import threading
import filetype
from core import sync_http as requests # 同步门面(底层 aiohttp,替代 requests)
# ============ HTTP 连接复用(提速核心) ============
# 常见浏览器 UA,避免直链 CDN 因空 UA 限速/拒绝
# 常见浏览器 UA,避免直链 CDN 因空 UA 限速/拒绝;封面等旁路请求也复用它
DEFAULT_UA = (
"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 "
"(KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36"
@@ -25,10 +23,6 @@ DEFAULT_UA = (
DL_CHUNK_SIZE = 256 * 1024
# 线程本地 Session:同线程内复用 TCP/TLS 连接(keep-alive),批量并发显著提速
_SESSION_LOCAL = threading.local()
# M4A/MP4 音频在 filetype 里被判为 video/mp4——mp4 是复合容器,仅凭魔数无法
# 区分音频轨与视频轨,故显式放行该 mime。
_CONTAINER_MIMES = frozenset({'video/mp4'})
@@ -51,13 +45,3 @@ def _sniff_audio_header(path) -> bool:
return False
mime = (kind.mime or '').lower()
return mime.startswith('audio/') or mime in _CONTAINER_MIMES
def get_http_session() -> requests.Session:
"""获取当前线程的共享同步会话(连接池由底层 aiohttp 统一维护)。"""
session: requests.Session | None = getattr(_SESSION_LOCAL, "session", None)
if session is None:
session = requests.Session()
session.headers.update({"User-Agent": DEFAULT_UA})
_SESSION_LOCAL.session = session
return session
+22 -12
View File
@@ -1,7 +1,7 @@
# -*- coding: utf-8 -*-
"""封面字节缓存(同专辑只从网络拉一次)。
(自 music_downloader.py 拆出,行为与原实现一致。)
(自 music_downloader.py 拆出,行为与原实现一致;HTTP 改走 core.http_client。)
"""
from __future__ import annotations
@@ -9,11 +9,14 @@ from __future__ import annotations
import threading
from collections import deque
from core.http_client import client as _http
from core.http_client import iter_chunks
from ._http import (
get_http_session,
DEFAULT_UA,
)
# 封面字节缓存:同一专辑封面只从网络拉取一次(ffmpeg 写元数据 与 ZIP 打包 共用)
# 封面字节缓存:同一专辑封面只从网络拉取一次(元数据写入 与 ZIP 打包 共用)
# 容量抽成常量:淘汰循环按它判定,不再读 deque.maxlen(该属性类型为 int|None,
# 一旦有人把 deque 改成无界,len() >= None 会直接 TypeError)
_COVER_CACHE_MAX = 128
@@ -22,10 +25,15 @@ _COVER_CACHE: dict[str, bytes] = {}
_COVER_CACHE_ORDER: deque[str] = deque(maxlen=_COVER_CACHE_MAX)
# 只保护字典/双端队列的读写,临界区内无 await——仍用 threading.Lock,
# 便于同步线程(ZIP 组装)读取缓存时一并复用
_COVER_CACHE_LOCK = threading.Lock()
# 封面字节预算:拒绝超大盘点阵/伪装文件
_COVER_MAX_BYTES = 10 * 1024 * 1024
def fetch_cover_bytes(pic_url: str) -> bytes | None:
async def fetch_cover_bytes(pic_url: str) -> bytes | None:
"""下载并缓存封面图片字节;失败返回 None(字节预算 10MB,拒绝超大盘点阵/伪装文件)"""
if not pic_url:
return None
@@ -34,24 +42,26 @@ def fetch_cover_bytes(pic_url: str) -> bytes | None:
if cached:
return cached
try:
resp = get_http_session().get(pic_url, timeout=15, stream=True)
resp = await _http.open('GET', pic_url,
headers={'User-Agent': DEFAULT_UA}, timeout=15)
try:
resp.raise_for_status()
if resp.status >= 400:
return None
try:
declared = int(resp.headers.get('Content-Length') or 0)
except ValueError:
declared = 0 # 垃圾 Content-Length 不炸,按未知长度走流式
if declared > 10 * 1024 * 1024:
if declared > _COVER_MAX_BYTES:
return None
# bytearray 就地追加(原 bytes += 是 O(n²) 拷贝,10MB 封面最坏多拷上百 MB)
data = bytearray()
for chunk in resp.iter_content(chunk_size=64 * 1024):
async for chunk in iter_chunks(resp, 64 * 1024):
data += chunk
if len(data) > 10 * 1024 * 1024:
if len(data) > _COVER_MAX_BYTES:
return None
if not data:
return None
data = bytes(data)
payload = bytes(data)
finally:
# 所有路径统一关闭响应(原 Content-Length 异常路径漏关)
resp.close()
@@ -63,6 +73,6 @@ def fetch_cover_bytes(pic_url: str) -> bytes | None:
while len(_COVER_CACHE_ORDER) >= _COVER_CACHE_MAX:
evict = _COVER_CACHE_ORDER.popleft()
_COVER_CACHE.pop(evict, None)
_COVER_CACHE[pic_url] = data
_COVER_CACHE[pic_url] = payload
_COVER_CACHE_ORDER.append(pic_url)
return data
return payload
+84 -86
View File
@@ -1,19 +1,24 @@
# -*- coding: utf-8 -*-
"""下载主类:落盘、元数据写入与 ZIP 打包。
"""下载主类:落盘、元数据写入与交付。
(自 music_downloader.py 拆出,行为与原实现一致。)
(自 music_downloader.py 拆出。)
异步口径:整个下载链路是协程。网络走 core.http_client(进程级连接池),
文件落盘/标签写入是阻塞 IO 与纯 Python CPU 工作,经 ``asyncio.to_thread`` 卸载;
互斥改 asyncio 锁(见 _concurrency),取消仍由 threading.Event 承载——
取消位由请求线程/监控线程置位,只有轮询读取,跨线程安全且无需跨循环唤醒。
"""
from __future__ import annotations
import asyncio
import os
import re
import threading
import time
from pathlib import Path
from core import sync_http as requests # 同步门面(底层 aiohttp,替代 requests)
from core.async_bridge import run_sync
from core.http_client import UpstreamError, UpstreamHTTPError, iter_chunks
from core.http_client import client as _http
from music_api import APIException, NeteaseAPI, load_cookies
from ._aria2 import Aria2Backend, get_backend
@@ -24,7 +29,6 @@ from ._concurrency import (
from ._http import (
DL_CHUNK_SIZE,
_sniff_audio_header,
get_http_session,
)
from ._log import (
logger,
@@ -52,7 +56,7 @@ class MusicDownloader:
用法:
downloader = MusicDownloader(download_dir="downloads")
result = downloader.download_music_file(music_id, quality="lossless")
result = await downloader.download_music_file(music_id, quality="lossless")
"""
def __init__(
@@ -66,7 +70,7 @@ class MusicDownloader:
download_dir: 下载目录
max_concurrent: 单个批量任务内的最大并发下载数
global_max_concurrent: 整个服务同时下载的文件数(含标签写入)。
所有请求共享该信号量,防止大量用户同时下载导致内存/CPU 耗尽。
所有请求共享该闸,防止大量用户同时下载导致内存/CPU 耗尽。
"""
self.download_dir = Path(download_dir)
self.download_dir.mkdir(exist_ok=True)
@@ -74,14 +78,14 @@ class MusicDownloader:
self.api = NeteaseAPI()
# 全局并发信号量:所有请求共用,限制同时进行的下载数
# 全局并发闸:所有请求共用,限制同时进行的下载数
# (带计数,供前端实时展示排队情况)
self._download_semaphore = CountingSemaphore(global_max_concurrent)
# 标签写入专用信号量:写标签会重写整个文件(IO 密集),
# 标签写入专用闸:写标签会重写整个文件(IO 密集),
# 与下载分离限流 —— 写满时只阻塞后来者,不占下载名额、不拖慢其它文件的传输
cpu_count = os.cpu_count() or 2
self._tag_semaphore = threading.BoundedSemaphore(max(1, min(4, cpu_count)))
self._tag_semaphore = asyncio.Semaphore(max(1, min(4, cpu_count)))
# ----------- 工具方法 -----------
@@ -145,24 +149,20 @@ class MusicDownloader:
# ----------- 信息获取 -----------
def get_music_info(self, music_id: int, quality: str = "standard",
async def get_music_info(self, music_id: int, quality: str = "standard",
cookies: dict[str, str] | None = None) -> MusicInfo:
"""获取音乐详细信息(同步,URL/详情/歌词 3 个接口并行请求,加快单曲/批量启动)"""
"""获取音乐详细信息(URL/详情/歌词 3 个接口并发请求,加快单曲/批量启动)"""
try:
cookies = cookies if cookies is not None else load_cookies()
# 3 个接口并行请求,单曲下载/批量预取都能明显提速
# URL/详情拿到后立即取消未完成的歌词请求,已知失败不再被拖着等完
# 异步化:改用 asyncio.gather 并发(api 方法已异步),失败语义与旧线程池一致
import asyncio
async def _gather_info():
url_t = self.api.get_song_url(music_id, quality, cookies)
detail_t = self.api.get_song_detail(music_id)
lyric_t = self.api.get_lyric(music_id, cookies)
return await asyncio.gather(url_t, detail_t, lyric_t, return_exceptions=True)
url_result, detail_result, lyric_result = run_sync(_gather_info())
# 3 个接口并发请求,单曲下载/批量预取都能明显提速
# URL/详情拿到后不再等待失败的歌词请求,已知失败不拖着等完
url_result, detail_result, lyric_result = await asyncio.gather(
self.api.get_song_url(music_id, quality, cookies),
self.api.get_song_detail(music_id),
self.api.get_lyric(music_id, cookies),
return_exceptions=True,
)
if isinstance(url_result, BaseException):
raise url_result
if isinstance(detail_result, BaseException):
@@ -255,9 +255,9 @@ class MusicDownloader:
safe_tag = self._sanitize_filename(tag, max_bytes=tag_budget)
return self.download_dir / f"{safe_base}{safe_tag}{ext}"
# ----------- 下载(同步) -----------
# ----------- 下载 -----------
def download_music_file(
async def download_music_file(
self,
music_id: int,
quality: str = "standard",
@@ -269,7 +269,7 @@ class MusicDownloader:
music_info: MusicInfo | None = None,
progress_cb=None,
) -> DownloadResult:
"""同步下载音乐到本地
"""下载音乐到本地
Args:
music_id: 音乐ID
@@ -288,15 +288,15 @@ class MusicDownloader:
- 流式下载到 .part 临时文件,完整下载后再原子改名,半截文件不会被当作已下载
- 断点续传:失败保留 .part,重试/下次下载从已写字节续传(服务器支持 Range 时)
- per-file 锁:同名歌曲并发下载互斥,防止 .part 被交叉写入损坏
- 连接复用:线程级 Session + keep-alive,避免每首歌重建 TCP/TLS 连接
- 网络传输与 ffmpeg 元数据写入分离限流,互不抢占对方资源
- 连接复用:core.http_client 的进程级连接池 + keep-alive
- 网络传输与标签写入分离限流,互不抢占对方资源
本函数只保留编排与重试策略:缓存校验、续传起点、传输通道、完整性判定、
元数据限流各自见对应私有方法的 docstring(此前是 205 行 / 圈复杂度 47
的单体,任何一处改动都要通读全部步骤)。
"""
if music_info is None:
music_info = self.get_music_info(music_id, quality, cookies)
music_info = await self.get_music_info(music_id, quality, cookies)
file_path = self._build_file_path(music_info)
part_path0 = file_path.with_name(file_path.name + ".part")
@@ -304,9 +304,9 @@ class MusicDownloader:
next_wait = 0.0
for attempt in range(retries + 1):
# 重试退避睡眠在文件锁外执行——同名并发任务不再陪睡
# 重试退避等待在文件锁外执行——同名并发任务不再陪睡
if next_wait > 0:
time.sleep(next_wait)
await asyncio.sleep(next_wait)
next_wait = 0.0
# 重试轮次顶部的取消检查也要清理 .part(原实现在 try 外抛出会绕过清理)
if cancel_event is not None and cancel_event.is_set():
@@ -319,22 +319,22 @@ class MusicDownloader:
part_path = part_path0
# 同一目标文件的下载者串行化(不同文件之间仍然并行)
with get_file_lock(file_path):
async with get_file_lock(file_path):
try:
# 其它线程可能刚好完成此文件,锁内复查(命中即返回,不再下载)
cached = self._cached_result(file_path, music_info,
# 其它任务可能刚好完成此文件,锁内复查(命中即返回,不再下载)
cached = await self._cached_result(file_path, music_info,
write_meta, meta_fields)
if cached is not None:
return cached
# 上次已下完但未改名的 .part:直接转正,不发起网络请求
promoted = self._promote_complete_part(
promoted = await self._promote_complete_part(
part_path, file_path, music_info, write_meta, meta_fields)
if promoted is not None:
return promoted
# part_path 已在文件锁外绑定为 part_path0(同一路径)
self._transfer(music_info, part_path,
await self._transfer(music_info, part_path,
self._resume_offset(part_path, music_info),
cancel_event, progress_cb)
@@ -347,7 +347,7 @@ class MusicDownloader:
continue
# 完整性通过 → 嗅探 → 原子转正 → 写元数据
return self._publish(part_path, file_path, music_info,
return await self._publish(part_path, file_path, music_info,
cancel_event, write_meta, meta_fields)
except DownloadCanceled:
@@ -363,7 +363,7 @@ class MusicDownloader:
last_error = "续传起点失效,重新下载"
next_wait = self._backoff_wait(attempt, retries)
continue
except (requests.RequestException, OSError) as e:
except (UpstreamError, OSError) as e:
# 网络错误:保留 .part 支持断点续传;
# 但若任务已被取消,.part 没有任何续传价值,一并清理
if cancel_event is not None and cancel_event.is_set():
@@ -385,7 +385,7 @@ class MusicDownloader:
# ----------- 下载分步实现(download_music_file 的组成件) -----------
def _cached_result(
async def _cached_result(
self,
file_path: Path,
music_info: MusicInfo,
@@ -406,9 +406,9 @@ class MusicDownloader:
if cache_bad:
self._unlink_quiet(file_path)
return None
# 缓存命中也尊重本次的元数据请求(重跑一次 ffmpeg,幂等)
# 缓存命中也尊重本次的元数据请求(重跑一次写标签,幂等)
if write_meta:
self._write_metadata_locked(file_path, music_info, meta_fields)
await self._write_metadata_locked(file_path, music_info, meta_fields)
return DownloadResult(
success=True,
file_path=str(file_path),
@@ -416,7 +416,7 @@ class MusicDownloader:
music_info=music_info,
)
def _promote_complete_part(
async def _promote_complete_part(
self,
part_path: Path,
file_path: Path,
@@ -437,7 +437,7 @@ class MusicDownloader:
return None
os.replace(part_path, file_path)
if write_meta:
self._write_metadata_locked(file_path, music_info, meta_fields)
await self._write_metadata_locked(file_path, music_info, meta_fields)
return DownloadResult(
success=True,
file_path=str(file_path),
@@ -460,7 +460,7 @@ class MusicDownloader:
return 0
return existing
def _transfer(
async def _transfer(
self,
music_info: MusicInfo,
part_path: Path,
@@ -474,17 +474,19 @@ class MusicDownloader:
"""
# 排队用可取消等待:原阻塞式 with 使取消后仍要等全局 10 名额腾出,
# 任务卡死、退款延迟
if not self._download_semaphore.acquire_cancellable(cancel_event, poll=0.5):
if not await self._download_semaphore.acquire_cancellable(cancel_event, poll=0.5):
raise DownloadCanceled("下载已取消")
try:
aria2_backend = get_backend()
# get_backend 首次调用要拉起 aria2c 并等 RPC 就绪(最长 8s);
# 它已是原生异步(等待用 await asyncio.sleep 让出事件循环),直接 await
aria2_backend = await get_backend()
if aria2_backend is not None:
# aria2c 通道:多连接 + 原生断点续传
self._fetch_via_aria2(aria2_backend, music_info, part_path,
await self._fetch_via_aria2(aria2_backend, music_info, part_path,
cancel_event, progress_cb)
else:
# 内置通道:aiohttp 流式落盘 + Range 起点校验
self._fetch_via_http(music_info, part_path, resume_from,
await self._fetch_via_http(music_info, part_path, resume_from,
cancel_event, progress_cb)
finally:
# 与 acquire_cancellable 配对释放(原 with 语义)
@@ -506,7 +508,7 @@ class MusicDownloader:
return (f"下载不完整: 期望 {music_info.file_size or '未知'} 字节, "
f"实际 {actual_size} 字节")
def _publish(
async def _publish(
self,
part_path: Path,
file_path: Path,
@@ -527,14 +529,14 @@ class MusicDownloader:
os.replace(part_path, file_path)
# 元数据写入移出下载信号量(CPU 操作不占网络名额),
# 元数据写入移出下载闸(CPU 操作不占网络名额),
# 但仍在文件锁内执行,保证其它请求看到的是已写入标签的最终文件
if write_meta:
# 元数据段入口补取消检查点——已取消的任务不再
# 进入信号量排队/ffmpeg 等待(有界但可达 120s),收尾不再延迟
# 进入标签闸排队/写盘等待(有界但可达数十秒),收尾不再延迟
if cancel_event is not None and cancel_event.is_set():
raise DownloadCanceled("下载已取消")
self._write_metadata_locked(file_path, music_info, meta_fields)
await self._write_metadata_locked(file_path, music_info, meta_fields)
return DownloadResult(
success=True,
@@ -543,26 +545,26 @@ class MusicDownloader:
music_info=music_info,
)
def _write_metadata_locked(
async def _write_metadata_locked(
self,
file_path: Path,
music_info: MusicInfo,
meta_fields: list[str] | None,
) -> None:
"""在标签信号量内写元数据:写标签会重写整个文件(IO 密集),与下载分离
"""在标签闸内写元数据:写标签会重写整个文件(IO 密集),与下载分离
限流——写满时只阻塞后来者,不占下载名额、不拖慢其它文件的传输。"""
with self._tag_semaphore:
self._write_metadata(file_path, music_info, meta_fields)
async with self._tag_semaphore:
await self._write_metadata(file_path, music_info, meta_fields)
@staticmethod
def _backoff_wait(attempt: int, retries: int) -> float:
"""失败后的退避秒数;已是最后一轮则不等待。锁外睡眠——同名并发任务不陪睡。"""
"""失败后的退避秒数;已是最后一轮则不等待。锁外等待——同名并发任务不陪睡。"""
return 0.5 * (attempt + 1) if attempt < retries else 0.0
# ----------- 传输通道(二选一,产物都是 part_path) -----------
def _fetch_via_http(
async def _fetch_via_http(
self,
music_info: MusicInfo,
part_path: Path,
@@ -570,7 +572,7 @@ class MusicDownloader:
cancel_event: threading.Event | None,
progress_cb,
) -> None:
"""内置流式通道:core.sync_http 逐块落盘,支持 Range 断点续传。
"""内置流式通道:core.http_client 逐块落盘,支持 Range 断点续传。
四步:构造 Range 头 → 发请求 → 定追加/重写并校验分片起点 → 流式落盘
(含体积上限与取消)。HTTP 错误在此分类成「确定性失败」或交外层重试。
@@ -579,23 +581,22 @@ class MusicDownloader:
DownloadCanceled: 取消事件置位
DownloadException: 体积超限、CDN 4xx 等确定性失败
_RetryFromScratch: 续传起点失效(残片已清,外层从头重下)
UpstreamError: 网络/5xx 等可重试失败(保留残片续传)
"""
expected_size = music_info.file_size or 0
self._safe_progress(progress_cb, resume_from, expected_size)
resp: requests.SyncResponse | None = None
resp = None
try:
resp = get_http_session().get(
music_info.download_url,
stream=True,
timeout=(10, 60),
headers=self._range_headers(resume_from),
)
headers = self._range_headers(resume_from)
resp = await _http.open('GET', music_info.download_url,
headers=headers or None, timeout=60)
mode = self._resolve_write_mode(resp, resume_from, part_path)
resp.raise_for_status()
self._stream_body(resp, part_path, mode, resume_from,
if resp.status >= 400:
raise UpstreamHTTPError(resp.status, resp.reason or '')
await self._stream_body(resp, part_path, mode, resume_from,
expected_size, cancel_event, progress_cb)
except requests.HTTPError as e:
status = self._http_status(e)
except UpstreamHTTPError as e:
status = e.status
if status == 416:
# Range 起点无效(如 CDN 环节变化):清除旧 part,从头重新下载
self._unlink_quiet(part_path)
@@ -622,7 +623,7 @@ class MusicDownloader:
服务器忽略 Range 返回 200 时若继续追加,会把整段拼在残片后面;206 还须
校验分片起点(见 _verify_content_range)。
"""
if resume_from <= 0 or resp.status_code != 206:
if resume_from <= 0 or resp.status != 206:
return "wb"
self._verify_content_range(resp, resume_from, part_path)
return "ab"
@@ -643,7 +644,7 @@ class MusicDownloader:
raise _RetryFromScratch(
f"Content-Range 起点错位({m.group(1)} != {resume_from}),已从头重新下载")
def _stream_body(
async def _stream_body(
self,
resp,
part_path: Path,
@@ -662,7 +663,7 @@ class MusicDownloader:
written = resume_from if mode == "ab" else 0
over_limit = False
with open(part_path, mode) as f:
for chunk in resp.iter_content(chunk_size=DL_CHUNK_SIZE):
async for chunk in iter_chunks(resp, DL_CHUNK_SIZE):
if cancel_event is not None and cancel_event.is_set():
raise DownloadCanceled("下载已取消")
if not chunk:
@@ -688,11 +689,6 @@ class MusicDownloader:
except Exception:
pass
@staticmethod
def _http_status(exc: BaseException) -> int | None:
"""从 HTTPError 上取响应状态码(无响应对象时返回 None)。"""
return getattr(getattr(exc, "response", None), "status_code", None)
@staticmethod
def _is_deterministic_client_error(status: int | None) -> bool:
"""确定性 4xx(403 防盗链 / 429 风控等):对同一失效直链重试只会加剧风控、
@@ -700,7 +696,7 @@ class MusicDownloader:
"""
return status is not None and 400 <= status < 500 and status != 408
def _fetch_via_aria2(
async def _fetch_via_aria2(
self,
backend: Aria2Backend,
music_info: MusicInfo,
@@ -711,10 +707,10 @@ class MusicDownloader:
"""aria2c 通道:多连接加速 + 原生断点续传。
异常语义与内置通道对齐:取消抛 DownloadCanceled、体积超限抛
DownloadException(均清残片),网络类失败抛 RequestException
DownloadException(均清残片),网络类失败抛 UpstreamNetworkError
(保留残片,由外层退避重试续传)。
"""
backend.fetch(
await backend.fetch(
music_info.download_url,
part_path,
cancel_event=cancel_event,
@@ -728,7 +724,7 @@ class MusicDownloader:
# 三个方法已删除——全服务端与验证脚本零调用(CLI 遗留),且 to_memory 不接受
# cookies 参数、真被启用会有行为差异;如需内存下载/异步批量再按当前接口重写
def _write_metadata(
async def _write_metadata(
self,
file_path: Path,
music_info: MusicInfo,
@@ -743,11 +739,13 @@ class MusicDownloader:
封面走共享缓存(与 ZIP 打包共用一次网络请求);各容器的落盘细节见
tagging.py——按扩展名分派到 ID3v2.3 / Vorbis comment / iTunes ilst。
写盘是阻塞 IO + 纯 Python CPU(mutagen 全量重写文件),经 to_thread 卸载。
"""
include_cover = fields is None or "cover" in fields
cover: bytes | None = None
if include_cover and music_info.pic_url:
cover = fetch_cover_bytes(music_info.pic_url)
cover = await fetch_cover_bytes(music_info.pic_url)
if not cover:
logger.warning("封面下载失败,跳过封面写入")
return write_tags(file_path, build_tags(music_info, fields), cover)
return await asyncio.to_thread(
write_tags, file_path, build_tags(music_info, fields), cover)
+8 -4
View File
@@ -9,8 +9,6 @@ from __future__ import annotations
from dataclasses import dataclass
from enum import Enum
from core import sync_http as requests # 同步门面(底层 aiohttp,替代 requests)
# ============ 类型定义 ============
class QualityLevel(Enum):
@@ -64,8 +62,14 @@ class DownloadCanceled(DownloadException):
pass
class _RetryFromScratch(requests.RequestException):
"""内部信号:断点续传起点失效(HTTP 416 等),清除 .part 后从头重试"""
class _RetryFromScratch(Exception):
"""内部信号:断点续传起点失效(HTTP 416 / Content-Range 错位),清除 .part 后从头重试。
刻意**不**继承 DownloadException 或 UpstreamError:外层
``except DownloadException`` 的语义是"确定性失败,不重试",本信号恰恰要求
从头重来;``except UpstreamError`` 的语义是"网络失败,保留 .part 续传",
而本信号已经把残片清掉了。两条语义都不是它,故独立成类、单独先于二者捕获。
"""
pass
+18 -5
View File
@@ -9,14 +9,12 @@ dependencies = [
"uvicorn[standard]>=0.30,<1.0",
"aiohttp>=3.10,<4.0",
"aiosqlite>=0.20,<1.0",
"aiofiles>=24.1",
"aiosmtplib>=5.1.3,<6.0",
"itsdangerous>=2.2,<3.0",
"cryptography>=50.0,<51.0",
"qrcode>=7.0,<8.3",
"python-multipart>=0.0.9",
"jinja2>=3.1,<4.0",
"filetype>=1.2.0",
"aria2p>=0.12.1",
"mutagen>=1.48.1",
]
@@ -67,7 +65,22 @@ ignore = [
# ASYNC109:timeout 参数是有意 API——直接转发给 aiohttp 的 ClientTimeout,
# 不是「用 sleep 当超时」的反模式。
"core/http_client.py" = ["ASYNC109"]
"core/aiohttp_compat.py" = ["ASYNC109"]
# ASYNC109:timeout 是**总下载时限**(交给 aria2 轮询循环判定),不是「用 sleep 当超时」。
# ASYNC240:搬迁后才出现的阻塞文件操作只有 unlink(删残片/控制文件)——目录项删除,
# 微秒级且不碰网络,卸载到线程的调度开销大于操作本身,与 downloader.py 同口径。
"music_downloader/_aria2.py" = ["ASYNC109", "ASYNC240"]
# ASYNC230/240:残留的阻塞文件系统调用都是**本地元数据/短写**——exists()/stat()/
# os.replace()/往 .part 顺序写分块,微秒级且不碰网络,卸载到线程的调度开销大于操作
# 本身。真正的重活(mutagen 全量重写标签、zipfile 整段拷贝、shutil.rmtree、同步
# SQLite 落库)都已各自 to_thread 卸载;aria2 的 RPC 已改为手写原生异步实现
# (music_downloader/_aria2.py),不再是阻塞调用。
"music_downloader/downloader.py" = ["ASYNC230", "ASYNC240"]
"services/podcast.py" = ["ASYNC230", "ASYNC240"]
"services/batch.py" = ["ASYNC240"]
# ASYNC110:暂停闸门轮询的是**跨线程**置位的 threading.Event 与任务表状态——置位方
# 在请求线程/监控线程,不属于本事件循环,故不能用 asyncio.Event 取代(见 _taskcore
# 模块 docstring 的取消/暂停载体说明)。
"services/_taskcore.py" = ["ASYNC110"]
[tool.ruff.lint.isort]
known-first-party = ["core", "web", "services", "music_api", "auth_system", "music_downloader"]
@@ -79,7 +92,7 @@ quote-style = "preserve"
[tool.pyright]
# 全量纳入(Flask 巨石删除后不再有例外清单)
# 顶层模块与 tests 此前不在名单里——「pyright 0 错误」只覆盖了 core/web/services,
# 而 requests→sync_http 迁移的遗留恰好全部躲在漏检的顶层模块中,故补齐。
# 而早期 requests→aiohttp 迁移的遗留恰好全部躲在漏检的顶层模块中,故补齐。
include = [
"core",
"web",
+4 -1
View File
@@ -10,8 +10,11 @@
- ``downloads``:单曲下载、账号探测与任务交付
- ``batch``:歌单/专辑批量打包 worker
- ``podcast``:播客单集下载与整台打包 worker
- ``streaming``:同步流式响应垫片(音频分段透传)
- ``tasks``:任务监控循环、设备心跳与缓存清扫
worker 与打包引擎都是协程,由 ``_taskcore._start_task`` 作为 asyncio 任务挂到
应用事件循环上(不再起线程);监控循环仍是一条常驻线程——它每轮只做周期性的
文件/库维护,不参与请求链路,留在线程里既不占事件循环也不额外引入 to_thread。
"""
# ============ 依赖自检 ============
+53 -24
View File
@@ -1,7 +1,17 @@
# -*- coding: utf-8 -*-
"""任务生命周期原语(启动 / 终态 / 文件收尾 / 结算)。
worker 自本轮起是协程:``_start_task`` 把 ``worker(*args)`` 包成 asyncio 任务挂到
应用事件循环上(原先起一条线程)。任务字典里的句柄随之由 ``worker_thread`` 换成
``worker_task``,监控循环对「worker 已结束但没人走终态结算」的兜底判断改为
``task.done()``——语义不变(settle 幂等,无双退)。
取消/暂停位仍是 ``threading.Event``:置位方可能是监控线程或请求线程,读取方是
worker 协程,只有"置位"与"轮询读取"两种操作,既不需要跨循环唤醒也不会被事件
循环卡住,比 asyncio.Event 更省心。
"""
import asyncio
import threading
import time
import traceback
@@ -21,36 +31,55 @@ from .api_service import (
api_service,
)
# 存活期的后台任务强引用——asyncio 只持弱引用,不留引用的话任务可能中途被 GC
_BACKGROUND_TASKS: set[asyncio.Task] = set()
def _start_task_thread(task_id: str, target, args=()) -> str | None:
def _log_worker_crash(handle: asyncio.Task) -> None:
"""worker 的异常逃逸到任务层时留痕。
正常情况下 worker 自带四路终态收敛,不会抛到这里;真抛出来说明收尾路径本身
出了问题,必须留下 traceback,否则任务永远停在 running 且无人知晓。
"""
if handle.cancelled():
return
exc = handle.exception()
if exc is not None:
api_service.logger.error(f"任务 worker 异常退出: {exc}", exc_info=exc)
async def _start_task(task_id: str, worker, args: tuple = ()) -> str | None:
"""启动任务 worker(启动失败时收敛任务终态并按账本退费,不留僵尸 queued 记录)。
退款只走账本(本处 settle 后装饰器看到账已关,不会重复退)。
线程句柄存入任务字典——监控循环对 4h 超时任务若发现 worker 线程已死
任务句柄存入任务字典——监控循环对 4h 超时任务若发现 worker 已结束
(无法自行走到终态结算),可代为按账本结算退款(settle 幂等,无双退)。"""
try:
th = threading.Thread(target=target, args=args, daemon=True)
with DOWNLOAD_TASKS_LOCK:
t = DOWNLOAD_TASKS.get(task_id)
if t is not None:
t['worker_thread'] = th
th.start()
return None
loop = asyncio.get_running_loop()
handle = loop.create_task(worker(*args), name=f'task:{task_id}')
except Exception as e:
with DOWNLOAD_TASKS_LOCK:
t = DOWNLOAD_TASKS.pop(task_id, None)
if t:
try:
auth_system.ledger_settle(task_id, 0, 'submit_failed')
await auth_system.ledger_settle(task_id, 0, 'submit_failed')
except Exception:
pass
api_service.logger.error(f"任务线程启动失败: {task_id} {e}")
api_service.logger.error(f"任务启动失败: {task_id} {e}")
return f'任务启动失败:{e}'
_BACKGROUND_TASKS.add(handle)
handle.add_done_callback(_BACKGROUND_TASKS.discard)
handle.add_done_callback(_log_worker_crash)
with DOWNLOAD_TASKS_LOCK:
t = DOWNLOAD_TASKS.get(task_id)
if t is not None:
t['worker_task'] = handle
return None
def _mark_task_terminal(task_id: str, status: str, msg: str = '') -> None:
async def _mark_task_terminal(task_id: str, status: str, msg: str = '') -> None:
"""任务进入终态的统一入口(记录 completed_at,保留期从完成时刻起算,
而不是从提交时刻起算)。
/F-E:置 error/canceled 终态时若 worker 线程已死(无人再走正常结算),
/F-E:置 error/canceled 终态时若 worker 已结束(无人再走正常结算),
代为按账本结算退款——ledger_settle 幂等,worker 已自行结算过则此处为 0。
终态先到先得——已终态(done/error/canceled)的任务
不再被覆盖(堵住 canceled→error 的非法迁移边:取消后解析阶段的自身失败
@@ -65,15 +94,15 @@ def _mark_task_terminal(task_id: str, status: str, msg: str = '') -> None:
t['msg'] = msg or t.get('msg', '')
t['completed_at'] = time.time()
if status in ('canceled', 'error'):
th = t.get('worker_thread')
if th is not None and not th.is_alive():
wt = t.get('worker_task')
if wt is not None and wt.done():
need_settle = True
# settle 移到任务锁外——ledger_settle 内部要抢 _DB_LOCK 写库
# settle 移到任务锁外——ledger_settle 内部要抢 SQLite 写锁
# (busy 最长 15s),持任务锁调用会让全站进度/提交/取消/领取随库停摆;
# settle 幂等,锁外调用无双退
# settle 幂等,锁外调用无双退。落库已迁移为原生异步,直接 await。
if need_settle:
try:
auth_system.ledger_settle(task_id, 0, status)
await auth_system.ledger_settle(task_id, 0, status)
except Exception:
pass
@@ -88,11 +117,11 @@ def _task_files(task: dict[str, Any]) -> list[str]:
return files
def _settle_batch_billing(task_id: str, ok_count: int, final_status: str = 'settled') -> int:
async def _settle_batch_billing(task_id: str, ok_count: int, final_status: str = 'settled') -> int:
"""批量任务结算(走持久账本,按实际预扣-实际交付幂等退款;
重复调用/重启后调用都只会退一次)"""
try:
refund = auth_system.ledger_settle(task_id, ok_count, final_status)
refund = await auth_system.ledger_settle(task_id, ok_count, final_status)
except Exception:
# 本函数多在异常处理器内被调用——结算再抛错(库忙超时)
# 会杀死 worker 收尾路径,跳过终态标记;失败留周期对账兜底(幂等)
@@ -104,10 +133,10 @@ def _settle_batch_billing(task_id: str, ok_count: int, final_status: str = 'sett
with DOWNLOAD_TASKS_LOCK:
t = DOWNLOAD_TASKS.get(task_id) or {}
uid = int(t.get('billing_user_id') or 0)
u = auth_system.get_user_by_id(uid) if uid else None
u = await auth_system.get_user_by_id(uid) if uid else None
uname = u['username'] if u else 'system'
remaining = u['duration_seconds'] if u else '?'
auth_system.add_log('success', uname, 'system', '批量退费',
await auth_system.add_log('success', uname, 'system', '批量退费',
f'失败部分 {refund} 次已退回,剩余 {remaining} 次')
except Exception:
pass
@@ -127,7 +156,7 @@ def _task_work_dir(task_id: str) -> Path:
return path
def _wait_if_paused(task: dict[str, Any], pause_ev: threading.Event,
async def _wait_if_paused(task: dict[str, Any], pause_ev: threading.Event,
cancel_ev: threading.Event, done: int) -> None:
"""断线暂停闸门:置 paused 态并等待恢复或取消,恢复后回到 running。
@@ -147,7 +176,7 @@ def _wait_if_paused(task: dict[str, Any], pause_ev: threading.Event,
task['done'] = done
task['msg'] = '已暂停(重新打开页面自动恢复)'
while not cancel_ev.is_set() and (pause_ev.is_set() or task.get('status') == 'paused'):
time.sleep(1)
await asyncio.sleep(1)
if cancel_ev.is_set():
raise DownloadCanceled("任务已取消")
pause_ev.clear()
+85 -91
View File
@@ -13,25 +13,25 @@
术语:下文「条目」指一个下载单元——音乐歌单里是一首歌,播客整台里是一集。
线程模型(本轮最容易踩坏的地方,改动前请先读):
- 下载 I/O 在 ``ThreadPoolExecutor`` 里跑,窗口大小 = ``job.concurrency``;
- **ZIP 写入与包内名去重只发生在调用线程**(``as_completed`` 消费结果时),
:class:`zipfile.ZipFile` 不是线程安全的,且去重读改写 ``used_arcnames``
需要串行——故下载与落包被拆成 ``download_one`` / ``write_entry`` 两个钩子,
只有前者进线程池。
并发模型(本轮最容易踩坏的地方,改动前请先读):
- 一个窗口 = ``job.concurrency`` 个条目,窗口内 ``asyncio.gather`` 并发下载;
- **ZIP 写入与包内名去重只在引擎协程里串行发生**(按窗口顺序消费结果)——
:class:`zipfile.ZipFile` 不是并发安全的,且去重读改写 ``used_arcnames``
需要串行;因此 ``download_one``(并发)与 ``write_entry``(串行)是两个钩子,
只有前者进 gather。``write_entry`` 里的重型落盘(几百 MB 的音频条目)
由实现方自行经 ``asyncio.to_thread`` 卸载,避免卡住事件循环。
``run_zip_job`` 的四个终态(成功 / 全失败 / 取消 / 异常)原先各写一段 8~20 行的
收尾内联在 try/except 里。此处把「一次运行」的状态收进 :class:`_ZipRun`,
四个终态各自成函数——改账本口径或补一种终态时,不必再在 167 行里找落点。
"""
import asyncio
import os
import tempfile
import threading
import time
import traceback
import zipfile
from collections.abc import Callable
from concurrent.futures import ThreadPoolExecutor, as_completed
from collections.abc import Awaitable, Callable
from dataclasses import dataclass, field
from pathlib import Path
from typing import Any
@@ -80,7 +80,7 @@ class ZipContext:
task_id: str
params: dict[str, Any]
cancel_ev: threading.Event
cancel_ev: Any
work_dir: Path
zf: zipfile.ZipFile
progress: Callable[..., None]
@@ -106,7 +106,7 @@ class ZipJob:
子类至少实现 :meth:`prepare` / :meth:`download_one` / :meth:`write_entry` /
:meth:`zip_filename`,其余钩子按需覆写。**实例每次运行新建**——下载器信息
预取池等运行期状态就挂在实例上。
预取任务等运行期状态就挂在实例上。
"""
tag = '批量打包' # 日志前缀
@@ -119,7 +119,7 @@ class ZipJob:
# ---- 必须实现 ----
def prepare(self, params: dict[str, Any], task_id: str, failed: list[str]) -> list[Any]:
async def prepare(self, params: dict[str, Any], task_id: str, failed: list[str]) -> list[Any]:
"""组装条目清单。可抛 ``DownloadException`` 让任务按业务失败收敛。
``failed`` 可被直接追加——清单阶段就有条目已知失败(如按 ID 列表时个别
@@ -127,8 +127,8 @@ class ZipJob:
"""
raise NotImplementedError
def download_one(self, item: Any, idx: int, ctx: ZipContext) -> Any:
"""下载单个条目(**在下载线程池内执行**)。
async def download_one(self, item: Any, idx: int, ctx: ZipContext) -> Any:
"""下载单个条目(**在窗口内并发执行**)。
返回的 ``payload`` 会原样交给 :meth:`write_entry`(放临时文件名、
已取到的元信息等)。失败请抛异常:``ItemFailed`` 用其自带文案,
@@ -136,10 +136,12 @@ class ZipJob:
"""
raise NotImplementedError
def write_entry(self, item: Any, idx: int, payload: Any, ctx: ZipContext) -> None:
"""把下载结果写入 ZIP(**在引擎调用线程内执行,天然串行**)。
async def write_entry(self, item: Any, idx: int, payload: Any, ctx: ZipContext) -> None:
"""把下载结果写入 ZIP(**由引擎按窗口顺序串行调用**)。
包内名去重在这里做——需要读写 ``ctx.used_arcnames``,必须串行。
重型落盘请自行 ``await asyncio.to_thread(...)``:直接在协程里写
几百 MB 会把事件循环卡住。
"""
raise NotImplementedError
@@ -149,7 +151,7 @@ class ZipJob:
# ---- 可选钩子 ----
def prefetch(self, window: list[Any], ctx: ZipContext) -> None:
async def prefetch(self, window: list[Any], ctx: ZipContext) -> None:
"""窗口级预热(每窗口一次、提交下载前调用)。默认无操作。"""
def fail_label(self, item: Any) -> str:
@@ -174,14 +176,14 @@ class ZipJob:
return f"部分{self.item_noun}已打包完成,可领取" + (
f",退回 {refunded} 次" if refunded else '')
def supplement(self, task_id: str, params: dict[str, Any], count: int) -> None:
async def supplement(self, task_id: str, params: dict[str, Any], count: int) -> None:
"""清单确定后的计费补扣(默认无操作)。"""
def on_delivered(self, items: list[Any], task_id: str, params: dict[str, Any]) -> None:
async def on_delivered(self, items: list[Any], task_id: str, params: dict[str, Any]) -> None:
"""交付落定后的操作记录(默认无操作)。"""
def cleanup(self) -> None:
"""释放运行期资源(如信息预取池)。默认无操作。"""
async def cleanup(self) -> None:
"""释放运行期资源(如信息预取任务)。默认无操作。"""
def _discard(path: str | None) -> None:
@@ -206,12 +208,12 @@ class _ZipRun:
job: ZipJob
task_id: str
settle: Callable[[str, int, str], int]
settle: Callable[[str, int, str], Awaitable[int]]
tally: _Tally = field(default_factory=_Tally)
task: dict[str, Any] = field(default_factory=dict)
params: dict[str, Any] = field(default_factory=dict)
cancel_ev: threading.Event | None = None
pause_ev: threading.Event | None = None
cancel_ev: Any = None
pause_ev: Any = None
items: list[Any] = field(default_factory=list)
zip_tmp_path: str | None = None
zip_valid: bool = False
@@ -232,7 +234,7 @@ def _progress_of(run: _ZipRun) -> Callable[..., None]:
return _progress
def _start_zip_run(run: _ZipRun) -> None:
async def _start_zip_run(run: _ZipRun) -> None:
"""置 running、组装清单、写回 total、执行计费补扣。"""
with DOWNLOAD_TASKS_LOCK:
task = DOWNLOAD_TASKS[run.task_id]
@@ -245,13 +247,13 @@ def _start_zip_run(run: _ZipRun) -> None:
# 计耗时只算「下载+打包」这一段(不含排队),故不用 task['start']
run.t0 = time.time()
run.items = run.job.prepare(run.params, run.task_id, run.tally.failed)
run.items = await run.job.prepare(run.params, run.task_id, run.tally.failed)
with DOWNLOAD_TASKS_LOCK:
DOWNLOAD_TASKS[run.task_id]['total'] = len(run.items)
run.job.supplement(run.task_id, run.params, len(run.items))
await run.job.supplement(run.task_id, run.params, len(run.items))
def _build_zip_archive(run: _ZipRun) -> None:
async def _build_zip_archive(run: _ZipRun) -> None:
"""建临时 ZIP 并逐窗口填充。返回时 ``zip_valid`` 才置位。
ZIP 构建放 downloads 卷(原实现落系统 %TEMP%——500 首无损可达 GB 级,
@@ -269,7 +271,7 @@ def _build_zip_archive(run: _ZipRun) -> None:
ctx = ZipContext(task_id=run.task_id, params=run.params,
cancel_ev=run.cancel_ev, work_dir=work_dir, zf=zf,
progress=_progress_of(run), total=len(run.items))
_fill_zip(job, run.items, ctx, run.tally, run.task, run.pause_ev)
await _fill_zip(job, run.items, ctx, run.tally, run.task, run.pause_ev)
# zip_valid 只在 ZIP with 块正常退出(中央目录完整写出)后才置 True。
# 此前两个 worker 在这之后还各留了一段 `if not zip_valid:` 失败处理——
# 紧跟赋值、中间无任何可改zip_valid的语句,是**不可达**的复审残留,已删。
@@ -280,7 +282,7 @@ def _build_zip_archive(run: _ZipRun) -> None:
BATCH_DURATIONS.append(time.time() - run.t0)
def _finish_all_failed(run: _ZipRun) -> None:
async def _finish_all_failed(run: _ZipRun) -> None:
"""全部失败(含清单非空但一条都没成):不发布空 ZIP、不按交付结算,转 error 并全额退费。
原音乐批量用 ``ok==0 and failed``、播客用早抛 ``DownloadException``——这里
@@ -289,8 +291,8 @@ def _finish_all_failed(run: _ZipRun) -> None:
job, tally = run.job, run.tally
_discard(run.zip_tmp_path)
run.zip_tmp_path = None
refunded = run.settle(run.task_id, 0, 'error')
_mark_task_terminal(run.task_id, 'error', job.all_failed_msg(tally.failed, refunded))
refunded = await run.settle(run.task_id, 0, 'error')
await _mark_task_terminal(run.task_id, 'error', job.all_failed_msg(tally.failed, refunded))
with DOWNLOAD_TASKS_LOCK:
t = DOWNLOAD_TASKS.get(run.task_id)
if t is not None:
@@ -300,7 +302,7 @@ def _finish_all_failed(run: _ZipRun) -> None:
f"{job.tag}全部失败: {run.task_id} 失败{len(tally.failed)} 退回{refunded}次")
def _publish_delivery(run: _ZipRun) -> None:
async def _publish_delivery(run: _ZipRun) -> None:
"""正常交付:按入包成功数结算,锁内复查取消/终态后发布结果。
``'settled'`` 是 ``_settle_batch_billing`` 的默认值,此处显式写出,让
@@ -308,7 +310,7 @@ def _publish_delivery(run: _ZipRun) -> None:
"""
job, tally = run.job, run.tally
assert run.cancel_ev is not None
refunded = run.settle(run.task_id, tally.ok, 'settled')
refunded = await run.settle(run.task_id, tally.ok, 'settled')
with DOWNLOAD_TASKS_LOCK:
t = DOWNLOAD_TASKS.get(run.task_id)
# 发布前锁内复查取消/失效状态——取消请求落在「检查后、拿锁前」的窗口
@@ -329,22 +331,22 @@ def _publish_delivery(run: _ZipRun) -> None:
'filename': job.zip_filename(tally.ok),
'audio_files': [], # 缓存文件不归任务所有,不进入清理清单
}
job.on_delivered(run.items, run.task_id, run.params)
await job.on_delivered(run.items, run.task_id, run.params)
api_service.logger.info(
f"{job.tag}完成: {job.log_label()}成功{tally.ok} 失败{len(tally.failed)} "
f"退回{refunded}次 ({run.task_id})")
def _finish_canceled(run: _ZipRun) -> None:
async def _finish_canceled(run: _ZipRun) -> None:
"""部分成功后取消——已入包部分照常交付(成果守恒),未完成部分按账本退费。
ZIP 未正常完成(写坏)时不按 ``ok_count`` 交付——按全额退费收敛。
"""
job, tally = run.job, run.tally
delivered = tally.ok if run.zip_valid else 0
refunded = run.settle(run.task_id, delivered, 'canceled')
refunded = await run.settle(run.task_id, delivered, 'canceled')
if delivered > 0 and run.zip_tmp_path:
_mark_task_terminal(run.task_id, 'canceled',
await _mark_task_terminal(run.task_id, 'canceled',
f"已取消(已打包 {tally.ok} {job.noun}可领取)" +
(f",退回 {refunded} 次" if refunded else ''))
with DOWNLOAD_TASKS_LOCK:
@@ -358,7 +360,7 @@ def _finish_canceled(run: _ZipRun) -> None:
}
else:
_discard(run.zip_tmp_path)
_mark_task_terminal(run.task_id, 'canceled',
await _mark_task_terminal(run.task_id, 'canceled',
'任务已取消' + (
f"(已退回 {refunded} 次下载次数)" if refunded else ''))
with DOWNLOAD_TASKS_LOCK:
@@ -369,29 +371,29 @@ def _finish_canceled(run: _ZipRun) -> None:
f"{job.tag}已取消: {run.task_id}(成功{tally.ok} 退回{refunded}次)")
def _finish_business_error(run: _ZipRun, exc: DownloadException) -> None:
async def _finish_business_error(run: _ZipRun, exc: DownloadException) -> None:
"""业务失败(清单阶段:无可下载条目 / 补扣下载次数失败 / 上游不可用)。
文案是我们自己拼的用户可读文本,保留原文(同播客单集下载)。
"""
job, tally = run.job, run.tally
delivered = tally.ok if run.zip_valid else 0
refunded = run.settle(run.task_id, delivered, 'error')
_apply_error(run.task_id, str(exc), refunded)
refunded = await run.settle(run.task_id, delivered, 'error')
await _apply_error(run.task_id, str(exc), refunded)
_discard(run.zip_tmp_path)
api_service.logger.error(
f"{job.tag}失败: {run.task_id} {exc}(成功{tally.ok} 退回{refunded}次)")
def _finish_unexpected(run: _ZipRun, exc: Exception) -> None:
async def _finish_unexpected(run: _ZipRun, exc: Exception) -> None:
"""非预期异常:原文 logger 先行(结算本身可能再抛,先落下 traceback),
用户可见文案泛化;带上退款结果再补一条,避免排障时只看到「异常」。"""
job, tally = run.job, run.tally
api_service.logger.error(f"{job.tag}异常: {run.task_id} {exc}\n{traceback.format_exc()}")
delivered = tally.ok if run.zip_valid else 0
refunded = run.settle(run.task_id, delivered, 'error')
refunded = await run.settle(run.task_id, delivered, 'error')
if delivered > 0 and run.zip_tmp_path:
_apply_error(run.task_id, job.partial_msg(refunded), refunded)
await _apply_error(run.task_id, job.partial_msg(refunded), refunded)
with DOWNLOAD_TASKS_LOCK:
t = DOWNLOAD_TASKS.get(run.task_id)
if t is not None:
@@ -401,16 +403,16 @@ def _finish_unexpected(run: _ZipRun, exc: Exception) -> None:
'audio_files': [],
}
else:
_apply_error(run.task_id, '打包失败,请稍后重试' + (
await _apply_error(run.task_id, '打包失败,请稍后重试' + (
f"(已退回 {refunded} 次下载次数)" if refunded else ''), refunded)
_discard(run.zip_tmp_path)
api_service.logger.error(
f"{job.tag}异常: {run.task_id}(成功{tally.ok} 退回{refunded}次)")
def run_zip_job(job: ZipJob, task_id: str, *,
settle: Callable[[str, int, str], int] = _settle_batch_billing) -> None:
"""执行一次批量打包任务(在 worker 线程内调用,自身不向外抛异常)。
async def run_zip_job(job: ZipJob, task_id: str, *,
settle: Callable[[str, int, str], Awaitable[int]] = _settle_batch_billing) -> None:
"""执行一次批量打包任务(在 worker 协程内调用,自身不向外抛异常)。
``settle`` 由调用方注入,默认走 :func:`services._taskcore._settle_batch_billing`。
做成参数是为了让测试能在**模块级**替换结算函数(包装层在调用时读取模块全局,
@@ -424,60 +426,58 @@ def run_zip_job(job: ZipJob, task_id: str, *,
"""
run = _ZipRun(job=job, task_id=task_id, settle=settle)
try:
_start_zip_run(run)
_build_zip_archive(run)
await _start_zip_run(run)
await _build_zip_archive(run)
if run.tally.ok == 0:
_finish_all_failed(run)
await _finish_all_failed(run)
return
_publish_delivery(run)
await _publish_delivery(run)
except DownloadCanceled:
_finish_canceled(run)
await _finish_canceled(run)
except DownloadException as e:
_finish_business_error(run, e)
await _finish_business_error(run, e)
except Exception as e:
_finish_unexpected(run, e)
await _finish_unexpected(run, e)
finally:
# 回收信息预取线程池,避免线程泄漏(未启动的预取任务一并取消)
job.cleanup()
# 回收信息预取任务,避免任务泄漏(未启动的预取任务一并取消)
await job.cleanup()
def _apply_error(task_id: str, msg: str, refunded: int) -> None:
async def _apply_error(task_id: str, msg: str, refunded: int) -> None:
"""异常/业务失败的终态收敛:置 error(终态先到先得)、记账退款额。
走 :func:`_mark_task_terminal` 而非就地写任务字典——后者是播客侧的旧写法,
除了重复一遍「已终态不覆盖」判断外,还漏掉了 worker 线程已死时按账本补结算
的兜底(``/F-E``)。
除了重复一遍「已终态不覆盖」判断外,还漏掉了 worker 已结束未结算时按账本
补结算的兜底(``/F-E``)。
"""
_mark_task_terminal(task_id, 'error', msg)
await _mark_task_terminal(task_id, 'error', msg)
with DOWNLOAD_TASKS_LOCK:
t = DOWNLOAD_TASKS.get(task_id)
if t is not None:
t['refunded'] = refunded
def _fill_zip(job: ZipJob, items: list[Any], ctx: ZipContext, tally: _Tally,
task: dict[str, Any], pause_ev: threading.Event) -> None:
async def _fill_zip(job: ZipJob, items: list[Any], ctx: ZipContext, tally: _Tally,
task: dict[str, Any], pause_ev) -> None:
"""逐窗口下载并写入 ZIP(返回时中央目录尚未写出,由调用方的 with 退出时落盘)。
窗口 = ``job.concurrency`` 个条目:下载并行、落包串行。窗口大小取 1 时
窗口 = ``job.concurrency`` 个条目:下载并发、落包串行。窗口大小取 1 时
退化为顺序下载(播客整台即如此)。
取消语义与旧的线程池版本对齐:等待名额期间可被取消;窗口内某条抛
DownloadCanceled 时整批中止(在途下载因每次分块都查取消位而很快退出)。
"""
concurrency = max(1, int(job.concurrency))
pool: ThreadPoolExecutor | None = None
try:
# 不用 with 管理下载池——取消时 with 退出会 wait=True 等在途 future
# (其内部阻塞在预取 HTTP 上最长 30s),改为不等待 + 取消排队任务
pool = ThreadPoolExecutor(max_workers=concurrency)
offset = 0
while offset < len(items):
if ctx.cancel_ev.is_set():
raise DownloadCanceled("任务已取消")
# 断线暂停:释放名额等待恢复/取消,恢复后重新进入排队
_wait_if_paused(task, pause_ev, ctx.cancel_ev, tally.ok + len(tally.failed))
await _wait_if_paused(task, pause_ev, ctx.cancel_ev, tally.ok + len(tally.failed))
# 每个窗口持有 BATCH_ZIP_SEM(同一时刻全局最多 N 个批量任务在下载)
# 等待名额期间可被取消,不再无限阻塞线程与退款
if not BATCH_ZIP_SEM.acquire_cancellable(ctx.cancel_ev):
# 等待名额期间可被取消,不再无限等待与延迟退款
if not await BATCH_ZIP_SEM.acquire_cancellable(ctx.cancel_ev):
raise DownloadCanceled("任务已取消")
try:
# 暂停/取消落在「等待恢复返回」与「拿到名额」之间:立刻放名额回到
@@ -488,19 +488,21 @@ def _fill_zip(job: ZipJob, items: list[Any], ctx: ZipContext, tally: _Tally,
window = items[offset:offset + concurrency]
start_idx = offset + 1 # 条目序号从 1 起(进度文案用)
offset += len(window)
job.prefetch(window, ctx) # 只预取本窗口
futures = {
pool.submit(job.download_one, item, start_idx + i, ctx): (item, start_idx + i)
for i, item in enumerate(window)
}
for future in as_completed(futures):
await job.prefetch(window, ctx) # 只预取本窗口
results = await asyncio.gather(
*(job.download_one(item, start_idx + i, ctx)
for i, item in enumerate(window)),
return_exceptions=True,
)
for i, result in enumerate(results):
if ctx.cancel_ev.is_set():
raise DownloadCanceled("任务已取消")
item, idx = futures[future]
item, idx = window[i], start_idx + i
try:
payload = future.result()
# 落包在主线程:ZipFile 非线程安全 + 包内名去重要串行
job.write_entry(item, idx, payload, ctx)
if isinstance(result, BaseException):
raise result
# 落包串行:ZipFile 非并发安全 + 包内名去重要按序
await job.write_entry(item, idx, result, ctx)
except DownloadCanceled:
raise
except ItemFailed as f:
@@ -512,11 +514,3 @@ def _fill_zip(job: ZipJob, items: list[Any], ctx: ZipContext, tally: _Tally,
ctx.progress(job.done_msg(item, idx, ctx))
finally:
BATCH_ZIP_SEM.release()
finally:
if pool is not None:
try:
# 取消/异常时不等待在途下载线程(其可能阻塞在预取 HTTP 上),
# 排队任务一并取消;在途线程自行收尾(.part 残片由续传/清扫兜底)
pool.shutdown(wait=False, cancel_futures=True)
except TypeError:
pool.shutdown(wait=False)
-34
View File
@@ -3,14 +3,11 @@
"""
import logging
import re
from pathlib import Path
from urllib.parse import parse_qs, urlsplit
from core.config import (
Settings,
)
from core.utils import expand_shortlink as _expand_shortlink
from music_api import (
NeteaseAPI,
load_cookies,
@@ -80,37 +77,6 @@ class MusicAPIService:
self.logger.error(f"读取 cookie.txt 异常: {e}")
return {}
def _extract_music_id(self, id_or_url: str) -> str:
"""提取音乐ID(正规 URL 解析替代子串/删除非数字启发式)
- 短链仅当整串就是一个 163cn.tv 短链 URL 时才请求展开(防 SSRF)
- music.163.com 链接用 parse_qs 取 id 参数(参数顺序无关)
"""
raw = str(id_or_url or '').strip()
try:
# 短链展开:仅接受"整个输入就是 163cn.tv 链接"(子串包含不算)
if re.match(r'^https?://(www\.)?163cn\.tv/\S+$', raw, re.IGNORECASE):
raw = _expand_shortlink(raw)
# 网易云链接:正规解析 query 取 id(不再 find('id=') 误吃 userid= 尾部)
if 'music.163.com' in raw:
try:
q = parse_qs(urlsplit(raw).query)
for key in ('id', 'songid'):
if q.get(key):
return str(q[key][0]).strip()
except Exception:
pass
# 兜底:老格式 #/song?id=xxx 的片段部分
m = re.search(r'[?&#]id=(\d+)', raw)
if m:
return m.group(1)
# 纯数字/普通文本直接返回
return raw
except Exception as e:
self.logger.error(f"提取音乐ID失败: {e}")
return str(id_or_url or '').strip()
def _format_file_size(self, size_bytes) -> str:
"""格式化文件大小(上游 size 可能为 null/垃圾值:容错返回"大小未知"而非 500)"""
try:
+40 -32
View File
@@ -5,11 +5,8 @@
本模块只保留「音乐歌单」这一侧的策略:窗口化预取与并行下载、每首三条目
(音频 + 歌词 + 封面)入包、同名去重、包名与文案。
"""
import asyncio
import zipfile
from concurrent.futures import (
Future,
ThreadPoolExecutor,
)
from dataclasses import dataclass
from pathlib import Path
from typing import Any
@@ -38,7 +35,7 @@ from .api_service import (
@dataclass(frozen=True)
class _SongPayload:
"""一首歌下载完成后的入包素材(下载线程产出,落包线程消费)。"""
"""一首歌下载完成后的入包素材(下载协程产出,落包协程消费)。"""
path: str
name: str
@@ -59,36 +56,41 @@ class _SonglistZipJob(ZipJob):
record_duration = True
def __init__(self) -> None:
self._info_pool: ThreadPoolExecutor | None = None
self._info_futures: dict[Any, Future[Any]] = {}
self._info_sem: asyncio.Semaphore | None = None
self._info_tasks: dict[Any, asyncio.Task[Any]] = {}
def prepare(self, params: dict[str, Any], task_id: str, failed: list[str]) -> list[Any]:
async def prepare(self, params: dict[str, Any], task_id: str, failed: list[str]) -> list[Any]:
self.concurrency = int(params['concurrency'])
return list(params['ids'])
def prefetch(self, window: list[Any], ctx: ZipContext) -> None:
async def prefetch(self, window: list[Any], ctx: ZipContext) -> None:
"""按窗口预取歌曲信息。
原实现一次性提交全部(最多 500 首 ×3 接口)预取任务,大歌单瞬间以 10 并发
连发上游约 1500 次请求打风控;改为每个下载窗口只预取本窗口的歌(与下载节奏
对齐),``download_one`` 语义不变(等预取结果再下载)。
对齐),并发另用信号量钳在 4,``download_one`` 语义不变(等预取结果再下载)。
"""
if self._info_pool is None:
self._info_pool = ThreadPoolExecutor(max_workers=min(4, max(1, self.concurrency)))
if self._info_sem is None:
self._info_sem = asyncio.Semaphore(min(4, max(1, self.concurrency)))
quality = ctx.params['quality']
cookies = ctx.params['cookies']
for mid in window:
if mid not in self._info_futures:
self._info_futures[mid] = self._info_pool.submit(
api_service.downloader.get_music_info, mid, quality, cookies
)
if mid not in self._info_tasks:
self._info_tasks[mid] = asyncio.create_task(
self._load_info(mid, quality, cookies))
def download_one(self, item: Any, idx: int, ctx: ZipContext) -> _SongPayload:
async def _load_info(self, mid: Any, quality: str, cookies: dict[str, str]) -> Any:
"""取单曲信息;并发由 _info_sem 钳制(避免预取风暴打风控)。"""
assert self._info_sem is not None
async with self._info_sem:
return await api_service.downloader.get_music_info(mid, quality, cookies)
async def download_one(self, item: Any, idx: int, ctx: ZipContext) -> _SongPayload:
"""并行下载单元:等待预取信息 → 下载 → 复用预取歌词与缓存封面(零重复请求)。"""
mid = item
info = self._info_futures[mid].result()
info = await self._info_tasks[mid]
params = ctx.params
result = api_service.downloader.download_music_file(
result = await api_service.downloader.download_music_file(
mid, params['quality'],
write_meta=params['write_meta'],
meta_fields=params['meta_fields'],
@@ -103,7 +105,7 @@ class _SonglistZipJob(ZipJob):
lyric = getattr(info, 'lyric', '') or ''
# 封面走共享缓存(同一专辑只拉一次网络);取消后不再发起新请求
if not ctx.cancel_ev.is_set() and info.pic_url:
cover = fetch_cover_bytes(info.pic_url) or b''
cover = await fetch_cover_bytes(info.pic_url) or b''
if not result.success:
raise ItemFailed(str(mid))
assert result.file_path is not None
@@ -118,7 +120,7 @@ class _SonglistZipJob(ZipJob):
return _SongPayload(path=result.file_path, name=mi.name, artists=mi.artists,
file_type=mi.file_type, lyric=lyric, cover=cover)
def write_entry(self, item: Any, idx: int, payload: _SongPayload, ctx: ZipContext) -> None:
async def write_entry(self, item: Any, idx: int, payload: _SongPayload, ctx: ZipContext) -> None:
mid = item
base = _build_download_base(ctx.params['filename_format'],
{'name': payload.name, 'artist_string': payload.artists})
@@ -139,13 +141,15 @@ class _SonglistZipJob(ZipJob):
k += 1
ctx.used_arcnames.update((final_name, lrc_name, jpg_name))
# F-D:exists 与 zf.write 之间的缓存竞态(另一任务缓存 miss 删文件)
# 按单曲失败计,不再让整批转 error
# 按单曲失败计,不再让整批转 error。音频条目可达上百 MB,落盘经 to_thread
# 卸载,避免在事件循环里做整段文件拷贝
try:
ctx.zf.write(payload.path, arcname=final_name, compress_type=zipfile.ZIP_STORED)
await asyncio.to_thread(ctx.zf.write, payload.path, arcname=final_name,
compress_type=zipfile.ZIP_STORED)
except OSError as e:
raise ItemFailed(f"{mid}(缓存文件读取失败: {e})") from e
# 缓存文件是跨任务共享资源,任务不得删除;磁盘回收交给监控线程的下载缓存
# 清扫(按文件年龄)。歌词与封面已在并行下载线程中拉取,这里直接写包(无网络等待)
# 清扫(按文件年龄)。歌词与封面已在并行下载协程中拉取,这里直接写包(无网络等待)
if payload.lyric:
try:
ctx.zf.writestr(lrc_name, _format_lyric(payload.lyric, lyric_format),
@@ -161,19 +165,23 @@ class _SonglistZipJob(ZipJob):
def zip_filename(self, ok_count: int) -> str:
return f"批量下载_{ok_count}首.zip"
def cleanup(self) -> None:
if self._info_pool is not None:
try:
self._info_pool.shutdown(wait=False, cancel_futures=True)
except TypeError:
self._info_pool.shutdown(wait=False)
async def cleanup(self) -> None:
"""取消在途预取任务(窗口因暂停/取消被跳过时它们没人 await)。
取消后不再 await:asyncio 任务被 cancel 不会产生"异常未被取用"告警,
而 await 会在此处引入最长一次上游请求的等待。
"""
for task in self._info_tasks.values():
if not task.done():
task.cancel()
self._info_tasks.clear()
def _batch_zip_worker(task_id: str) -> None:
async def _batch_zip_worker(task_id: str) -> None:
"""后台批量打包:窗口化下载 → 写 ZIP(音频免压缩),支持取消与断线暂停
- 断线暂停:设备心跳超时后,当前窗口下载完即释放 BATCH_ZIP_SEM 名额给下一位,
保存进度进入暂停等待;重新打开网站(心跳恢复)后自动回到队列继续下载
- 音频入包即删,缓解磁盘占用
"""
run_zip_job(_SonglistZipJob(), task_id, settle=_settle_batch_billing)
await run_zip_job(_SonglistZipJob(), task_id, settle=_settle_batch_billing)
+37 -22
View File
@@ -7,8 +7,12 @@
``_single_download_worker`` 原是 177 行 / 圈复杂度 30 的单函数,三个 except 分支
各写一遍「结算 + 删任务目录」,正文里又散着四处取消检查。此处按「一步一个具名
函数」拆开:主函数只留线性流程,失败收尾收敛到 ``_fail_single_task`` 一处。
worker 自本轮起是协程:下载链路本身已异步,落库已迁移为原生异步直接 await;
删目录/写 ZIP 等阻塞文件 IO 经 ``asyncio.to_thread`` 卸载。
"""
import asyncio
import os
import shutil
import threading
@@ -114,13 +118,13 @@ def _begin_single_task(task_id: str) -> SingleTaskParams:
)
def _fetch_single_metadata(p: SingleTaskParams) -> tuple[MusicInfo, dict[str, Any], str, str]:
"""一次预取歌曲完整信息(URL/详情/歌词三接口并行),并算出交付基础名。
async def _fetch_single_metadata(p: SingleTaskParams) -> tuple[MusicInfo, dict[str, Any], str, str]:
"""一次预取歌曲完整信息(URL/详情/歌词三接口并发),并算出交付基础名。
替代原先 name_v1 + url_v1 + 下载器内部 get_music_info 的重复串行调用。
返回 ``(MusicInfo, 下载器口径的 music_info, base, filename)``。
"""
mi = api_service.downloader.get_music_info(p.music_id, p.quality, p.cookies)
mi = await api_service.downloader.get_music_info(p.music_id, p.quality, p.cookies)
music_info = {
'id': p.music_id,
'name': mi.name,
@@ -188,28 +192,36 @@ def _single_file_result(file_path_str: str, filename: str,
}
def _pack_single_zip(file_path_str: str, base: str, filename: str, mi: MusicInfo,
async def _pack_single_zip(file_path_str: str, base: str, filename: str, mi: MusicInfo,
music_info: dict[str, Any], p: SingleTaskParams) -> dict[str, Any]:
"""把音频 + 歌词 + 封面 + info.txt 打成单曲 ZIP 交付包。
歌词与封面都是**尽力而为**:歌词在预取时已一并拿到(无需再请求一次),
封面走共享缓存;两者拿不到就不写对应条目,不影响音频交付。
ZIP 组装是整段文件拷贝的阻塞 IO,经 to_thread 卸载。
"""
import tempfile as _tempfile
# 封面走共享缓存(与元数据写入共用同一次网络请求)——先取好再落包,
# 网络等待留在事件循环里(协程),文件写入整段交给线程
cover: bytes = b''
if mi.pic_url:
cover = await fetch_cover_bytes(mi.pic_url) or b''
fd, zip_tmp_path = _tempfile.mkstemp(prefix='single_zip_', suffix='.zip',
dir=str(Path(file_path_str).parent))
os.close(fd)
def _write_zip() -> None:
with zipfile.ZipFile(zip_tmp_path, 'w', zipfile.ZIP_DEFLATED) as zf:
zf.write(file_path_str, arcname=filename)
# 歌词已在预取时一并拿到,无需再请求一次
if mi.lyric:
try:
zf.writestr(f"{base}.{p.lyric_format}", _format_lyric(mi.lyric, p.lyric_format))
zf.writestr(f"{base}.{p.lyric_format}",
_format_lyric(mi.lyric, p.lyric_format))
except Exception:
pass
# 封面走共享缓存(与元数据写入共用同一次网络请求)
cover = fetch_cover_bytes(mi.pic_url)
if cover:
zf.writestr(f"{base}.jpg", cover)
info_txt = (
@@ -219,6 +231,8 @@ def _pack_single_zip(file_path_str: str, base: str, filename: str, mi: MusicInfo
f"音质: {api_service._get_quality_display_name(p.quality)}\n"
)
zf.writestr("info.txt", info_txt)
await asyncio.to_thread(_write_zip)
return {
'file_path': file_path_str,
'zip_path': zip_tmp_path,
@@ -228,7 +242,7 @@ def _pack_single_zip(file_path_str: str, base: str, filename: str, mi: MusicInfo
}
def _publish_single_task(task_id: str, result: dict[str, Any], remote_ip: str,
async def _publish_single_task(task_id: str, result: dict[str, Any], remote_ip: str,
name: str, filename: str) -> None:
"""最终发布:置 done(已终态不覆盖)→ 结算 → 记操作日志。
@@ -246,7 +260,7 @@ def _publish_single_task(task_id: str, result: dict[str, Any], remote_ip: str,
t['msg'] = '下载完成'
t['result'] = result
try:
auth_system.ledger_settle(task_id, 1, 'delivered')
await auth_system.ledger_settle(task_id, 1, 'delivered')
except Exception as _e:
api_service.logger.error(f"单曲交付结算失败(重启对账兜底): {task_id} {_e}")
try:
@@ -256,30 +270,30 @@ def _publish_single_task(task_id: str, result: dict[str, Any], remote_ip: str,
api_service.logger.info(f"单曲下载完成: {filename} ({task_id})")
def _fail_single_task(task_id: str, status: str, msg: str, ledger_status: str,
async def _fail_single_task(task_id: str, status: str, msg: str, ledger_status: str,
task_dir_path: str | None) -> None:
"""失败/取消的统一收尾:置终态 → 退款 → 清掉任务目录。
这三个 except 分支原先各写一遍(共 9 行 × 3 处);收敛后新增一种失败
只需要挑一个 ``ledger_status``,不会再漏掉清理或退款。
"""
_mark_task_terminal(task_id, status, msg)
await _mark_task_terminal(task_id, status, msg)
try:
auth_system.ledger_settle(task_id, 0, ledger_status)
await auth_system.ledger_settle(task_id, 0, ledger_status)
except Exception:
pass
if task_dir_path:
shutil.rmtree(task_dir_path, ignore_errors=True)
await asyncio.to_thread(shutil.rmtree, task_dir_path, ignore_errors=True)
def _single_download_worker(task_id: str) -> None:
async def _single_download_worker(task_id: str) -> None:
"""后台单曲下载:下载音频(可选 ZIP 打包),支持取消;完成后由 /download/result 交付,35 分钟保留期清理"""
task_dir_path: str | None = None
try:
p = _begin_single_task(task_id)
mi, music_info, base, filename = _fetch_single_metadata(p)
mi, music_info, base, filename = await _fetch_single_metadata(p)
download_result = api_service.downloader.download_music_file(
download_result = await api_service.downloader.download_music_file(
p.music_id, p.quality,
write_meta=p.write_meta,
meta_fields=p.meta_fields,
@@ -293,25 +307,26 @@ def _single_download_worker(task_id: str) -> None:
_abort_if_canceled(p.cancel_ev)
assert download_result.file_path is not None
# 交付文件用任务专属副本,共享缓存不进清理清单
file_path_str = _task_delivery_copy(task_id, Path(download_result.file_path), filename)
file_path_str = await asyncio.to_thread(
_task_delivery_copy, task_id, Path(download_result.file_path), filename)
task_dir_path = str(Path(file_path_str).parent)
if p.need_zip:
result = _pack_single_zip(file_path_str, base, filename, mi, music_info, p)
result = await _pack_single_zip(file_path_str, base, filename, mi, music_info, p)
else:
result = _single_file_result(file_path_str, filename, music_info)
# 打包阶段同样以取消检查收口
_abort_if_canceled(p.cancel_ev)
_publish_single_task(task_id, result, p.remote_ip, music_info['name'], filename)
await _publish_single_task(task_id, result, p.remote_ip, music_info['name'], filename)
except DownloadCanceled:
api_service.logger.info(f"单曲下载已取消: {task_id}")
_fail_single_task(task_id, 'canceled', '任务已取消', 'canceled', task_dir_path)
await _fail_single_task(task_id, 'canceled', '任务已取消', 'canceled', task_dir_path)
except DownloadException as e:
api_service.logger.error(f"单曲下载失败: {task_id} {e}")
_fail_single_task(task_id, 'error', str(e), 'error', task_dir_path)
await _fail_single_task(task_id, 'error', str(e), 'error', task_dir_path)
except Exception as e:
# 非预期异常的 repr 可能含上游主机/连接细节——用户可见文案
# 泛化为固定文案,原文 logger 先行留档(教训:先记录再泛化)
api_service.logger.error(f"单曲下载异常: {task_id} {e}\n{traceback.format_exc()}")
_fail_single_task(task_id, 'error', '下载失败,请稍后重试', 'error', task_dir_path)
await _fail_single_task(task_id, 'error', '下载失败,请稍后重试', 'error', task_dir_path)
+77 -59
View File
@@ -4,7 +4,12 @@
整台打包的骨架(状态机 / 暂停 / 名额 / 终态收敛 / 结算)在
:mod:`services._zipcore`,本模块只保留播客侧策略:清单组装(整台或指定单集)、
逐集顺序下载、按实际集数补扣、单条目入包(首集附封面)。
下载走 core.http_client 的流式响应(原先是 services.streaming 这个同步流垫片),
落盘写文件是阻塞 IO、ZIP 组装是整段拷贝,均经 ``asyncio.to_thread`` 卸载。
"""
import asyncio
import os
import shutil
import tempfile
@@ -17,6 +22,8 @@ from pathlib import Path
from typing import Any
import auth_system
from core.http_client import client as _http
from core.http_client import iter_chunks
from core.state import (
DOWNLOAD_TASKS,
DOWNLOAD_TASKS_LOCK,
@@ -49,9 +56,13 @@ from .api_service import (
from .downloads import (
_abort_if_canceled,
)
from .streaming import (
_stream_open,
)
# 播客流式下载的上游头(与音频直链 CDN 的口径一致)
_STREAM_HEADERS = {
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 '
'(KHTML, like Gecko) Safari/537.36',
'Referer': 'https://music.163.com/',
}
def _podcast_safe_name(s: str, fallback: str = 'unknown') -> str:
@@ -92,7 +103,7 @@ def _report_progress(task_id: str, progress_label: str, done: int, total: int) -
t['live_percent'] = min(99.0, float(pct))
def _stream_to_part(url: str, part: Path, total_size: int, cancel_ev: threading.Event,
async def _stream_to_part(url: str, part: Path, total_size: int, cancel_ev: threading.Event,
task_id: str, progress_label: str) -> tuple[int, int]:
"""把响应体流式写入 part,返回 (已写字节数, 期望总大小)。
@@ -100,12 +111,13 @@ def _stream_to_part(url: str, part: Path, total_size: int, cancel_ev: threading.
"""
done = 0
last_update = 0.0
with _stream_open(url, timeout=60) as r:
if r.status >= 400:
raise DownloadException(f"上游返回 {r.status}")
total = total_size or int(r.headers.get('Content-Length') or 0)
resp = await _http.open('GET', url, headers=_STREAM_HEADERS, timeout=60)
try:
if resp.status >= 400:
raise DownloadException(f"上游返回 {resp.status}")
total = total_size or int(resp.headers.get('Content-Length') or 0)
with open(part, 'wb') as f:
for chunk in r.iter_content(chunk_size=256 * 1024):
async for chunk in iter_chunks(resp, 256 * 1024):
if cancel_ev.is_set():
raise DownloadCanceled("任务已取消")
if not chunk:
@@ -116,13 +128,15 @@ def _stream_to_part(url: str, part: Path, total_size: int, cancel_ev: threading.
if now - last_update > 0.5:
last_update = now
_report_progress(task_id, progress_label, done, total)
finally:
resp.close()
return done, total
def _verify_downloaded(target: Path, done: int, total_size: int) -> None:
"""完整性校验:不达标则删掉已落地的文件并抛 DownloadException。
为什么非要校验:CDN 提前断流时 iter_content 会「正常」结束,文件静默截断。
为什么非要校验:CDN 提前断流时流式读取会「正常」结束,文件静默截断。
已知期望大小时要求**精确相等**(API 报的大小与实际一致;放宽到 95% 容差会
交付截断的坏文件还照扣次数);期望未知时退化为非空校验——Content-Length
兜底值可能是压缩口径,不做精确比对。
@@ -135,21 +149,21 @@ def _verify_downloaded(target: Path, done: int, total_size: int) -> None:
raise DownloadException("下载内容为空,请重试")
def _podcast_stream_download(url: str, target: Path, total_size: int,
async def _podcast_stream_download(url: str, target: Path, total_size: int,
cancel_ev: threading.Event,
task_id: str = '', progress_label: str = '') -> int:
"""流式下载播客音频到 target(先写 .part 再改名),返回实际字节数。
自动加入全局下载信号量排队;每 0.5s 更新一次任务进度消息。
自动加入全局下载闸排队;每 0.5s 更新一次任务进度消息。
取消时 .part 会留存(不删)——下次下载同名文件直接覆盖,是既有行为。
"""
part = target.with_name(target.name + '.part')
# 等待全局下载名额期间可被取消(与音乐批量 BATCH_ZIP_SEM 同语义)
sem = api_service.downloader._download_semaphore
if not sem.acquire_cancellable(cancel_ev):
if not await sem.acquire_cancellable(cancel_ev):
raise DownloadCanceled("任务已取消")
try:
done, _total = _stream_to_part(url, part, total_size, cancel_ev, task_id, progress_label)
done, _total = await _stream_to_part(url, part, total_size, cancel_ev, task_id, progress_label)
finally:
sem.release()
if part.exists():
@@ -158,13 +172,13 @@ def _podcast_stream_download(url: str, target: Path, total_size: int,
return done
def _podcast_resolve_audio(program: dict[str, Any], quality: str,
async def _podcast_resolve_audio(program: dict[str, Any], quality: str,
cookies: dict[str, str]) -> dict[str, Any]:
"""解析单集音频直链,失败抛 DownloadException"""
song_id = program.get('main_song_id')
if not song_id:
raise DownloadException("该单集缺少音频信息(mainSong)")
url_info = podcast.resolve_episode_url(song_id, cookies, quality)
url_info = await podcast.resolve_episode_url(song_id, cookies, quality)
if not url_info or not url_info.get('url'):
raise DownloadException("未获取到音频直链(单集可能已下架或受版权限制)")
return url_info
@@ -242,21 +256,16 @@ def _prepare_podcast_target(task_id: str, program_id: Any, program: dict[str, An
def _write_podcast_zip(zip_path: str, file_path_str: str, arcname: str, base: str,
name: str, program: dict[str, Any], url_info: dict[str, Any],
fallback_quality: str) -> None:
fallback_quality: str, cover: bytes = b'') -> None:
"""单集入包:音频 + 封面(可选,失败不炸任务)+ info.txt。
info.txt 不再写直链(真实 URL 不应随交付文件外泄)。
封面字节由调用方先取好(网络等待留在协程里),本函数只做阻塞的落盘组装。
"""
with zipfile.ZipFile(zip_path, 'w', zipfile.ZIP_DEFLATED) as zf:
zf.write(file_path_str, arcname=arcname)
cover_url = program.get('cover_url') or ''
if cover_url:
try:
cover = fetch_cover_bytes(cover_url)
if cover:
zf.writestr(f"{base}.jpg", cover)
except Exception:
pass
info_txt = (
f"单集: {name}\n"
f"播客: {program.get('radio_name') or ''}\n"
@@ -277,16 +286,24 @@ def _podcast_file_result(file_path_str: str, filename: str, ext: str) -> dict[st
}
def _podcast_zip_result(prep: _PodcastTarget, file_path_str: str,
async def _podcast_zip_result(prep: _PodcastTarget, file_path_str: str,
program: dict[str, Any], url_info: dict[str, Any],
spec: _PodcastTaskSpec) -> dict[str, Any]:
"""打包交付形状:ZIP 中间文件建在任务目录下(失败收尾 rmtree 一并覆盖)。"""
cover: bytes = b''
cover_url = program.get('cover_url') or ''
if cover_url:
try:
cover = await fetch_cover_bytes(cover_url) or b''
except Exception:
cover = b''
fd, zip_tmp_path = tempfile.mkstemp(prefix='podcast_zip_', suffix='.zip',
dir=str(prep.task_dir))
os.close(fd)
_write_podcast_zip(zip_tmp_path, file_path_str, arcname=prep.filename,
base=prep.base, name=prep.name, program=program,
url_info=url_info, fallback_quality=spec.quality)
await asyncio.to_thread(_write_podcast_zip, zip_tmp_path, file_path_str,
arcname=prep.filename, base=prep.base, name=prep.name,
program=program, url_info=url_info,
fallback_quality=spec.quality, cover=cover)
return {
'file_path': file_path_str,
'zip_path': zip_tmp_path,
@@ -296,7 +313,7 @@ def _podcast_zip_result(prep: _PodcastTarget, file_path_str: str,
}
def _publish_podcast_task(task_id: str, result: dict[str, Any], remote_ip: str,
async def _publish_podcast_task(task_id: str, result: dict[str, Any], remote_ip: str,
name: str, filename: str) -> None:
"""最终发布:置 done(已终态不覆盖)→ 结算 → 记操作日志。
@@ -313,7 +330,7 @@ def _publish_podcast_task(task_id: str, result: dict[str, Any], remote_ip: str,
t['msg'] = '下载完成'
t['result'] = result
try:
auth_system.ledger_settle(task_id, 1, 'delivered')
await auth_system.ledger_settle(task_id, 1, 'delivered')
except Exception as _e:
api_service.logger.error(f"播客单集交付结算失败(重启对账兜底): {task_id} {_e}")
try:
@@ -323,19 +340,19 @@ def _publish_podcast_task(task_id: str, result: dict[str, Any], remote_ip: str,
api_service.logger.info(f"播客单集下载完成: {filename} ({task_id})")
def _fail_podcast_task(task_id: str, status: str, msg: str, ledger_status: str,
async def _fail_podcast_task(task_id: str, status: str, msg: str, ledger_status: str,
task_dir: Path | None) -> None:
"""失败/取消的统一收尾:置终态 → 退款 → 清掉任务目录(含 ZIP 中间件)。"""
_mark_task_terminal(task_id, status, msg)
await _mark_task_terminal(task_id, status, msg)
try:
auth_system.ledger_settle(task_id, 0, ledger_status)
await auth_system.ledger_settle(task_id, 0, ledger_status)
except Exception:
pass
if task_dir is not None:
shutil.rmtree(str(task_dir), ignore_errors=True)
await asyncio.to_thread(shutil.rmtree, str(task_dir), ignore_errors=True)
def _podcast_download_worker(task_id: str) -> None:
async def _podcast_download_worker(task_id: str) -> None:
"""后台播客单集下载:解析单集 → 音质降级取直链 → 流式下载(可选 ZIP 打包)
任务 type 为 'single',完成后经 /download/result 交付,与单曲下载共用交付通道。
@@ -344,37 +361,37 @@ def _podcast_download_worker(task_id: str) -> None:
task_dir: Path | None = None
try:
spec = _begin_podcast_task(task_id)
program = podcast.get_program_detail(spec.program_id, spec.cookies)
url_info = _podcast_resolve_audio(program, spec.quality, spec.cookies)
program = await podcast.get_program_detail(spec.program_id, spec.cookies)
url_info = await _podcast_resolve_audio(program, spec.quality, spec.cookies)
prep = _prepare_podcast_target(task_id, spec.program_id, program, url_info)
task_dir = prep.task_dir
with DOWNLOAD_TASKS_LOCK:
t = DOWNLOAD_TASKS[task_id]
t['msg'] = '下载中...'
_podcast_stream_download(url_info['url'], prep.target, url_info.get('size', 0),
await _podcast_stream_download(url_info['url'], prep.target, url_info.get('size', 0),
spec.cancel_ev, task_id, '下载中')
_abort_if_canceled(spec.cancel_ev)
file_path_str = str(prep.target)
if spec.need_zip:
result = _podcast_zip_result(prep, file_path_str, program, url_info, spec)
result = await _podcast_zip_result(prep, file_path_str, program, url_info, spec)
else:
result = _podcast_file_result(file_path_str, prep.filename, prep.ext)
# 打包阶段同样以取消检查收口(末段取消不得覆盖为 done)
_abort_if_canceled(spec.cancel_ev)
_publish_podcast_task(task_id, result, spec.remote_ip, prep.name, prep.filename)
await _publish_podcast_task(task_id, result, spec.remote_ip, prep.name, prep.filename)
except DownloadCanceled:
api_service.logger.info(f"播客单集下载已取消: {task_id}")
_fail_podcast_task(task_id, 'canceled', '任务已取消', 'canceled', task_dir)
await _fail_podcast_task(task_id, 'canceled', '任务已取消', 'canceled', task_dir)
except DownloadException as e:
api_service.logger.error(f"播客单集下载失败: {task_id} {e}")
_fail_podcast_task(task_id, 'error', str(e), 'error', task_dir)
await _fail_podcast_task(task_id, 'error', str(e), 'error', task_dir)
except Exception as e:
# 非预期异常原文 logger 先行,用户可见文案泛化(教训)
api_service.logger.error(f"播客单集下载异常: {task_id} {e}\n{traceback.format_exc()}")
_fail_podcast_task(task_id, 'error', '下载失败,请稍后重试', 'error', task_dir)
await _fail_podcast_task(task_id, 'error', '下载失败,请稍后重试', 'error', task_dir)
def _podcast_cleanup_files(paths: list[str | None]) -> None:
@@ -397,7 +414,7 @@ def _podcast_cleanup_files(paths: list[str | None]) -> None:
@dataclass(frozen=True)
class _EpisodePayload:
"""一集下载完成后的入包素材(下载线程产出,落包线程消费)。"""
"""一集下载完成后的入包素材(下载协程产出,落包协程消费)。"""
path: str
base: str
@@ -409,7 +426,7 @@ class _PodcastZipJob(ZipJob):
并发刻意等于音乐批量:同样占用全局批量名额(BATCH_ZIP_SEM,全局 3),
逐集 acquire/release——暂停等待/取消时不长期占名额;音频本身仍受全局
下载信号量约束。原实现不受批量并发控制,多用户同时打包会无限并发。
下载闸约束。原实现不受批量并发控制,多用户同时打包会无限并发。
"""
tag = '播客打包'
@@ -422,20 +439,20 @@ class _PodcastZipJob(ZipJob):
def __init__(self) -> None:
self._radio_name = ''
def prepare(self, params: dict[str, Any], task_id: str, failed: list[str]) -> list[Any]:
async def prepare(self, params: dict[str, Any], task_id: str, failed: list[str]) -> list[Any]:
cookies = params['cookies']
radio_id = params['radio_id']
limit = params['limit']
# 组装单集清单:整台电台(正序全量)或指定单集ID列表
if radio_id:
episodes = podcast.list_radio_all_program_ids(radio_id, cookies, max_count=limit)
episodes = await podcast.list_radio_all_program_ids(radio_id, cookies, max_count=limit)
if not episodes:
raise DownloadException("该电台暂无可下载的单集")
else:
episodes = []
for pid in (params.get('ids') or []):
try:
prog = podcast.get_program_detail(pid, cookies)
prog = await podcast.get_program_detail(pid, cookies)
episodes.append({
'id': prog['id'],
'name': prog['name'],
@@ -453,7 +470,7 @@ class _PodcastZipJob(ZipJob):
self._radio_name = _podcast_safe_name(params.get('radio_name') or '', '播客')
return episodes
def supplement(self, task_id: str, params: dict[str, Any], count: int) -> None:
async def supplement(self, task_id: str, params: dict[str, Any], count: int) -> None:
"""按次计费补扣:整台电台提交时只预扣 1 次,此处按实际单集数补足差额。
补扣失败(余额不足/数据库异常)任务必须失败并按账本退费——不允许无账本继续交付。
@@ -462,7 +479,7 @@ class _PodcastZipJob(ZipJob):
_buid = params.get('billing_uid')
_extra = count - int(params.get('billed') or 1)
if _buid and _extra > 0:
if not auth_system.ledger_supplement(task_id, _extra):
if not await auth_system.ledger_supplement(task_id, _extra):
raise DownloadException(
f"补扣下载次数失败(需补扣 {_extra} 次),任务已取消,预扣部分将退回")
if _buid:
@@ -476,13 +493,13 @@ class _PodcastZipJob(ZipJob):
except Exception as e:
raise DownloadException(f"补扣下载次数异常: {e}") from e
def download_one(self, item: Any, idx: int, ctx: ZipContext) -> _EpisodePayload:
async def download_one(self, item: Any, idx: int, ctx: ZipContext) -> _EpisodePayload:
ep = item
label = f"[{idx}/{ctx.total}] {ep['name'][:24]}"
ctx.progress(f"{label} 解析中...")
cookies = ctx.params['cookies']
prog = ep.get('detail') or podcast.get_program_detail(ep['id'], cookies)
url_info = _podcast_resolve_audio(prog, ctx.params['quality'], cookies)
prog = ep.get('detail') or await podcast.get_program_detail(ep['id'], cookies)
url_info = await _podcast_resolve_audio(prog, ctx.params['quality'], cookies)
ctx.progress(f"{label} 下载中...")
ext = (url_info.get('type') or 'mp3').lower().lstrip('.')
serial = int(ep.get('serial_num') or 0)
@@ -492,7 +509,7 @@ class _PodcastZipJob(ZipJob):
# 单集(Linux 上 unlink 打开中的文件)
tmp_audio = ctx.work_dir / f"podcast_zip_tmp_{ctx.task_id}_{idx}.{ext}"
try:
_podcast_stream_download(url_info['url'], tmp_audio,
await _podcast_stream_download(url_info['url'], tmp_audio,
url_info.get('size', 0), ctx.cancel_ev)
except Exception:
# 下载失败即清(.part 残片同清);成功时由 write_entry 的 finally 清
@@ -500,7 +517,7 @@ class _PodcastZipJob(ZipJob):
raise
return _EpisodePayload(path=str(tmp_audio), base=base, ext=ext)
def write_entry(self, item: Any, idx: int, payload: _EpisodePayload, ctx: ZipContext) -> None:
async def write_entry(self, item: Any, idx: int, payload: _EpisodePayload, ctx: ZipContext) -> None:
ep = item
try:
# 重名条目自动追加序号(同名单集/同号不同集不得互相覆盖)
@@ -510,11 +527,12 @@ class _PodcastZipJob(ZipJob):
arcname = f"{payload.base} ({k}).{payload.ext}"
k += 1
ctx.used_arcnames.add(arcname)
ctx.zf.write(payload.path, arcname=arcname, compress_type=zipfile.ZIP_STORED)
await asyncio.to_thread(ctx.zf.write, payload.path, arcname=arcname,
compress_type=zipfile.ZIP_STORED)
# 首集附带封面,便于识别
if idx == 1 and ep.get('cover_url'):
try:
cover = fetch_cover_bytes(ep['cover_url'])
cover = await fetch_cover_bytes(ep['cover_url'])
if cover:
ctx.zf.writestr("cover.jpg", cover, compress_type=zipfile.ZIP_STORED)
except Exception:
@@ -541,7 +559,7 @@ class _PodcastZipJob(ZipJob):
return f"全部单集下载失败: {'; '.join(failed[:3])}" + (
f"(已退回 {refunded} 次下载次数)" if refunded else '')
def on_delivered(self, items: list[Any], task_id: str, params: dict[str, Any]) -> None:
async def on_delivered(self, items: list[Any], task_id: str, params: dict[str, Any]) -> None:
try:
first_name = items[0].get('name') if items else ''
_record_log('下载', params.get('remote_ip') or '-',
@@ -550,6 +568,6 @@ class _PodcastZipJob(ZipJob):
pass
def _podcast_zip_worker(task_id: str) -> None:
async def _podcast_zip_worker(task_id: str) -> None:
"""整台播客打包:逐集 下载→入包即删,支持取消与断线暂停(复用批量任务机制)"""
run_zip_job(_PodcastZipJob(), task_id, settle=_settle_batch_billing)
await run_zip_job(_PodcastZipJob(), task_id, settle=_settle_batch_billing)
-61
View File
@@ -1,61 +0,0 @@
# -*- coding: utf-8 -*-
"""同步流式响应垫片(音频分段透传)。
"""
class _SyncStreamResponse:
"""同步流式响应包装(把 aiohttp 的异步流按需拉成同步迭代)。
供流式下载使用:worker 是同步线程,这里经 core.async_bridge 在后台事件循环
逐块取数据,避免引入 requests 依赖(Flask 已删除、requests 已移出依赖)。
"""
def __init__(self, url: str, timeout: float):
from core import async_bridge
self._bridge = async_bridge
self._url = url
self._timeout = timeout
self._agent = None
self.status = 0
self.headers: dict[str, str] = {}
self._resp = None
def __enter__(self):
self._agent = self._bridge.run_sync(self._open())
self.status = self._agent[0]
self.headers = self._agent[1]
return self
async def _open(self):
from core.http_client import client
resp = await client.open('GET', self._url, headers={
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 '
'(KHTML, like Gecko) Safari/537.36',
'Referer': 'https://music.163.com/',
}, timeout=self._timeout)
return resp.status, dict(resp.headers), resp
def iter_content(self, chunk_size: int = 256 * 1024):
assert self._agent is not None
resp = self._agent[2]
it = resp.content.iter_chunked(chunk_size)
while True:
try:
chunk = self._bridge.run_sync(it.__anext__())
except StopAsyncIteration:
break
yield chunk
def __exit__(self, *exc):
if self._agent is not None and self._agent[2] is not None:
try:
self._agent[2].close()
except Exception:
pass
return False
def _stream_open(url: str, timeout: float = 60) -> _SyncStreamResponse:
"""打开同步可迭代的流式响应(见 _SyncStreamResponse)。"""
return _SyncStreamResponse(url, timeout)
+12 -7
View File
@@ -6,6 +6,7 @@
对账与缓存清扫,每项作业的异常兜底粒度也一目了然(见各自 docstring)。
"""
import asyncio
import os
import shutil
import threading
@@ -172,7 +173,7 @@ def _reap_finished_tasks(now: float) -> None:
def _cancel_overtime_tasks(now: float) -> None:
"""运行/排队/暂停超过硬上限仍未结束的任务 → 置取消并唤醒 cancel Event。
worker 线程已死则无人再走终态结算,监控代为按账本退款——ledger_settle 幂等,
worker 已结束则无人再走终态结算,监控代为按账本退款——ledger_settle 幂等,
worker 若已结算过此处返回 0,无双退。结算移到任务锁外(同 _mark_task_terminal
口径):库忙 15s 不再随任务锁波及全站任务端点。
"""
@@ -188,14 +189,17 @@ def _cancel_overtime_tasks(now: float) -> None:
task['cancel'].set()
except Exception:
pass
# worker 线程已死则无人再走终态结算——监控代为按账本结算退款
th = task.get('worker_thread')
if th is not None and not th.is_alive():
# worker 已结束则无人再走终态结算——监控代为按账本结算退款
wt = task.get('worker_task')
if wt is not None and wt.done():
timeout_settle.append(tid)
api_service.logger.warning(f"任务超时自动取消: {tid}")
for tid in timeout_settle:
try:
auth_system.ledger_settle(tid, 0, 'timeout')
# 监控在**独立线程**里跑(见 _task_monitor_loop),线程无法 await——
# auth_system 已迁移为原生异步,此处用 asyncio.run 起独立 loop 执行结算
# (线程边界的一次性同步→异步衔接,非运行时同步门面)。
asyncio.run(auth_system.ledger_settle(tid, 0, 'timeout'))
except Exception:
pass
@@ -215,10 +219,11 @@ def _run_periodic_ledger_jobs(now: float) -> None:
try:
if now - _LEDGER_LAST_RECOVER['ts'] > _LEDGER_RECOVER_INTERVAL:
_LEDGER_LAST_RECOVER['ts'] = now
auth_system.ledger_recover_orphans(max_age_seconds=_LEDGER_ORPHAN_MAX_AGE)
# 监控线程内的同步→异步衔接:auth_system 已异步化,线程里用 asyncio.run 起 loop
asyncio.run(auth_system.ledger_recover_orphans(max_age_seconds=_LEDGER_ORPHAN_MAX_AGE))
if now - _LEDGER_LAST_CLEANUP['ts'] > _LEDGER_CLEANUP_INTERVAL:
_LEDGER_LAST_CLEANUP['ts'] = now
auth_system.ledger_cleanup_settled(_LEDGER_SETTLED_RETENTION)
asyncio.run(auth_system.ledger_cleanup_settled(_LEDGER_SETTLED_RETENTION))
except Exception as e:
api_service.logger.warning(f"账本周期兜底异常(下轮重试): {e}")
-1434
View File
File diff suppressed because it is too large. Load diff
-7704
View File
File diff suppressed because it is too large. Load diff
+2 -1
View File
@@ -10,6 +10,7 @@
本测试用真实 multipart 请求穿过端点,确保三种分支(接受 / 后缀拒 / 内容拒)行为正确。
"""
import asyncio
import io
import pytest
@@ -35,7 +36,7 @@ def client(tmp_path, monkeypatch):
monkeypatch.setattr(auth.db, 'SECRET_KEY_PATH', tmp_path / 'secret.key')
monkeypatch.setattr(db, 'DATA_DIR', tmp_path)
monkeypatch.setattr(db, 'DB_PATH', tmp_path / 'site.db')
auth.init_db()
asyncio.run(auth.init_db())
app = FastAPI()
app.include_router(admin_site.router)
-86
View File
@@ -1,86 +0,0 @@
# -*- coding: utf-8 -*-
"""兼容垫片与同步桥测试。"""
import asyncio
import pytest
from aiohttp import web
from core import aiohttp_compat as req
from core import async_bridge
@pytest.fixture()
async def server():
app = web.Application()
async def ok(request):
return web.json_response({'code': 200, 'data': 'hi'})
async def notfound(request):
return web.json_response({'code': 404}, status=404)
async def echo_post(request):
body = await request.text()
return web.json_response({'got': body, 'ua': request.headers.get('User-Agent')})
app.router.add_get('/ok', ok)
app.router.add_get('/nf', notfound)
app.router.add_post('/echo', echo_post)
runner = web.AppRunner(app)
await runner.setup()
site = web.TCPSite(runner, '127.0.0.1', 0)
await site.start()
port = runner.addresses[0][1]
try:
yield f'http://127.0.0.1:{port}'
finally:
await runner.cleanup()
async def test_get_json(server):
r = await req.get(server + '/ok')
assert r.status_code == 200
assert r.json() == {'code': 200, 'data': 'hi'}
assert 'code' in r.text
r.raise_for_status() # 200 不抛
async def test_4xx_does_not_raise_until_raise_for_status(server):
r = await req.get(server + '/nf')
assert r.status_code == 404 # 请求本身不抛(同 requests)
assert r.json() == {'code': 404}
with pytest.raises(req.HTTPError):
r.raise_for_status()
async def test_post_headers_forwarded(server):
r = await req.post(server + '/echo', data={'params': 'x'}, headers={'User-Agent': 'ua-test'})
body = r.json()
assert body['ua'] == 'ua-test'
assert 'params' in body['got']
async def test_network_error_raises_request_exception():
with pytest.raises(req.RequestException):
await req.get('http://127.0.0.1:1/nope', timeout=1)
def test_bridge_runs_coro_from_sync():
async def coro():
await asyncio.sleep(0.01)
return 42
assert async_bridge.run_sync(coro()) == 42
async def test_bridge_rejects_inside_loop():
async def coro():
return 1
c = coro()
try:
with pytest.raises(RuntimeError):
async_bridge.run_sync(c)
finally:
c.close() # run_sync 拒绝执行,需自行关闭以免「未 await」告警
+335 -87
View File
@@ -1,73 +1,78 @@
# -*- coding: utf-8 -*-
"""aria2 下载后端的单元测试(用假 api 驱动,不需要真实 aria2c 二进制)。
"""aria2 下载后端的单元测试(用假 RPC 驱动,不需要真实 aria2c 二进制)。
覆盖:
- aria2c 缺失时 get_backend 静默降级且失败结果被记忆
- 正常完成路径的进度回调与自动清理
- 正常完成路径的进度回调、控制文件兜底清理与选项下发
- 取消 / 体积超限 连带删除残片;任务失败 / 总超时 保留残片供续传
- 提交失败与 RPC 抖动统一上报为可重试异常
- 已终结任务只清结果表(removeDownloadResult),在途任务才 forceRemove
- JSON-RPC 信封(方法名 / token / params)与 error 对象的收敛
替身边界:本模块已不再依赖 aria2p——RPC 由 ``_Aria2Rpc`` 直接打
core.http_client。故替身只替换 ``_Aria2Rpc`` 这一层的四个方法,
以及(信封测试里)``_http.request_once``。
"""
from __future__ import annotations
import json
import threading
import pytest
import music_downloader._aria2 as a2
from core.sync_http import RequestException
from core.http_client import HTTPResult, UpstreamNetworkError
from music_downloader import DownloadCanceled, DownloadException
class _FakeDownload:
"""按预设步进产出状态,模拟 aria2 的 tellStatus 轮询。"""
def _st(**kw):
"""构造一条 tellStatus 结果(aria2 的数值字段都是字符串)。"""
base = {'status': 'active', 'completedLength': '0', 'totalLength': '100'}
base.update(kw)
return base
def __init__(self, steps):
self.gid = 'g1'
class _FakeRpc:
"""按预设步进产出 tellStatus 结果,并记录移除类调用。"""
def __init__(self, steps, *, fail_add=False, fail_tell=False):
self._steps = list(steps)
self._i = 0
self.completed_length = 0
self.total_length = 100
self.is_complete = False
self.has_failed = False
self.error_message = ''
self.status = 'active'
self._fail_add = fail_add
self._fail_tell = fail_tell
self.added: tuple[str, dict[str, str]] | None = None
self.removed: list[tuple[str, str]] = [] # ('forceRemove'|'removeDownloadResult', gid)
def update(self):
idx = min(self._i, len(self._steps) - 1)
done, complete, failed = self._steps[idx]
self._i += 1
self.completed_length = done
self.is_complete = complete
self.has_failed = failed
if failed:
self.status = 'error'
self.error_message = 'boom'
async def get_version(self) -> str:
return "1.36.0"
class _FakeAPI:
def __init__(self, download, fail_submit=False, fail_update=False):
self.download = download
self.removed: list[tuple[str, bool, bool]] = []
self.submitted: tuple[list[str], dict[str, str]] | None = None
self._fail_submit = fail_submit
self._fail_update = fail_update
def add_uris(self, uris, options=None):
if self._fail_submit:
async def add_uri(self, url: str, options: dict[str, str]) -> str:
if self._fail_add:
raise RuntimeError('rpc down')
self.submitted = (list(uris), options or {})
return self.download
self.added = (url, options)
return 'g1'
def remove(self, downloads, force=False, files=False):
self.removed.append((downloads[0].gid, force, files))
class _FailUpdateDownload(_FakeDownload):
def update(self):
async def tell_status(self, gid: str) -> dict:
if self._fail_tell:
raise RuntimeError('rpc jitter')
idx = min(self._i, len(self._steps) - 1)
self._i += 1
return dict(self._steps[idx])
async def force_remove(self, gid: str) -> str:
self.removed.append(('forceRemove', gid))
return gid
async def remove_download_result(self, gid: str) -> str:
self.removed.append(('removeDownloadResult', gid))
return 'OK'
class _FakeProc:
"""已退出的假进程:poll() 返回非 None,故 _try_start 会立即判定「提前退出」。"""
returncode: int | None = 0
def poll(self):
return 0
@@ -81,98 +86,341 @@ class _FakeProc:
return 0
class _LiveProc:
"""仍在运行的假进程:poll() 返回 None(用于 _try_start 的就绪等待)。"""
returncode: int | None = None
def poll(self):
return None
def terminate(self):
pass
def kill(self):
pass
def wait(self, timeout=None):
return 0
def _backend(monkeypatch, steps, **kw):
monkeypatch.setattr(a2.time, 'sleep', lambda _s: None)
dl = _FailUpdateDownload([]) if kw.get('fail_update') else _FakeDownload(steps)
api = _FakeAPI(dl, fail_submit=kw.get('fail_submit', False))
return a2.Aria2Backend(api, _FakeProc(), 6800), api
# fetch 的轮询改 asyncio.sleep(不再 time.sleep):把间隔压到 0 让状态机快速走完
monkeypatch.setattr(a2, '_POLL_INTERVAL', 0.0)
rpc = _FakeRpc(steps, fail_add=kw.get('fail_add', False),
fail_tell=kw.get('fail_tell', False))
return a2.Aria2Backend(rpc, _FakeProc(), 6800), rpc
# ---------- 降级 ----------
def test_get_backend_degrades_without_binary(monkeypatch):
async def test_get_backend_degrades_without_binary(monkeypatch):
monkeypatch.setattr(a2.shutil, 'which', lambda _n: None)
monkeypatch.setattr(a2, '_backend', None)
monkeypatch.setattr(a2, '_unavailable', False)
assert a2.get_backend() is None
assert a2.get_backend() is None # 失败结果被记忆,不反复拉起进程
assert await a2.get_backend() is None
assert await a2.get_backend() is None # 失败结果被记忆,不反复拉起进程
def test_free_port_is_in_valid_range():
assert 0 < a2._free_port() < 65536
async def test_try_start_waits_for_rpc_ready(monkeypatch):
"""RPC 就绪前的探活失败要被容忍并重试;就绪后返回可用后端(不需真 aria2c)。
这条覆盖的是「首次调用最长阻塞 8s」那条路径——它已从「同步 sleep + to_thread」
改成「await asyncio.sleep 让出事件循环」,用例钉住状态机不变。
"""
probes = {'n': 0}
class _RpcStub:
def __init__(self, port, secret):
self.port = port
self.secret = secret
async def get_version(self):
probes['n'] += 1
if probes['n'] < 2:
raise RuntimeError('not ready yet')
return '1.36.0'
monkeypatch.setattr(a2.shutil, 'which', lambda _n: 'aria2c')
monkeypatch.setattr(a2, '_free_port', lambda: 6800)
monkeypatch.setattr(a2, '_Aria2Rpc', _RpcStub)
monkeypatch.setattr(a2.subprocess, 'Popen', lambda *a, **kw: _LiveProc())
monkeypatch.setattr(a2.atexit, 'register', lambda _f: None)
monkeypatch.setattr(a2, '_RPC_READY_TIMEOUT', 3.0)
be = await a2._try_start()
assert be is not None
assert be.port == 6800
assert probes['n'] == 2 # 第一次没就绪、重试后才成功
async def test_try_start_gives_up_when_rpc_never_ready(monkeypatch):
"""超过上限仍未就绪:返回 None 走降级,并回收子进程(不留下孤儿 aria2c)。"""
killed = []
proc = _LiveProc()
monkeypatch.setattr(proc, 'kill', lambda: killed.append(1))
class _NeverReady:
def __init__(self, port, secret):
pass
async def get_version(self):
raise RuntimeError('not ready')
monkeypatch.setattr(a2.shutil, 'which', lambda _n: 'aria2c')
monkeypatch.setattr(a2, '_free_port', lambda: 6800)
monkeypatch.setattr(a2, '_Aria2Rpc', _NeverReady)
monkeypatch.setattr(a2.subprocess, 'Popen', lambda *a, **kw: proc)
monkeypatch.setattr(a2.atexit, 'register', lambda _f: None)
monkeypatch.setattr(a2, '_RPC_READY_TIMEOUT', 0.0)
assert await a2._try_start() is None
assert killed == [1] # 已强杀回收
async def test_try_start_returns_none_when_process_exits_early(monkeypatch):
"""aria2c 进程提前退出:立即降级,不等满超时。"""
class _RpcStub:
def __init__(self, port, secret):
pass
async def get_version(self):
raise RuntimeError('should not matter')
monkeypatch.setattr(a2.shutil, 'which', lambda _n: 'aria2c')
monkeypatch.setattr(a2, '_free_port', lambda: 6800)
monkeypatch.setattr(a2, '_Aria2Rpc', _RpcStub)
monkeypatch.setattr(a2.subprocess, 'Popen', lambda *a, **kw: _FakeProc())
monkeypatch.setattr(a2.atexit, 'register', lambda _f: None)
assert await a2._try_start() is None # _FakeProc.poll() != None → 提前退出
# ---------- 完成路径 ----------
def test_fetch_reports_progress_and_completes(monkeypatch, tmp_path):
be, api = _backend(monkeypatch, [(50, False, False), (100, True, False)])
async def test_fetch_reports_progress_and_completes(monkeypatch, tmp_path):
be, rpc = _backend(monkeypatch, [
_st(completedLength='50'),
_st(status='complete', completedLength='100'),
])
target = tmp_path / 'y.part'
ctrl = tmp_path / 'y.part.aria2'
ctrl.write_bytes(b'aria2 ctrl') # 预置控制文件,验证兜底清理
seen = []
be.fetch('http://x/y.mp3', tmp_path / 'y.part',
await be.fetch('http://x/y.mp3', target,
progress_cb=lambda done, total: seen.append((done, total)))
assert seen == [(50, 100), (100, 100)]
assert api.removed == [] # 完成不需要中止
assert not (tmp_path / 'y.part.aria2').exists() # 控制文件已兜底清理
submitted = api.submitted
assert submitted is not None
options = submitted[1]
assert rpc.removed == [] # 完成不需要中止
assert not ctrl.exists() # 控制文件已兜底清理
assert rpc.added is not None
url, options = rpc.added
assert url == 'http://x/y.mp3'
assert options['continue'] == 'true' # 续传选项已下发
assert options['max-connection-per-server'] == str(a2._SPLIT)
assert options['out'] == 'y.part'
assert options['auto-file-renaming'] == 'false' # 与 .part 语义不冲突
def test_progress_callback_exception_is_swallowed(monkeypatch, tmp_path):
be, _api = _backend(monkeypatch, [(100, True, False)])
async def test_progress_callback_exception_is_swallowed(monkeypatch, tmp_path):
be, _rpc = _backend(monkeypatch, [_st(status='complete', completedLength='100')])
def boom(_d, _t):
raise RuntimeError('cb')
be.fetch('http://x/y.mp3', tmp_path / 'y.part', progress_cb=boom) # 不应抛出
# 不应抛出
await be.fetch('http://x/y.mp3', tmp_path / 'y.part', progress_cb=boom)
async def test_referer_is_forwarded_when_given(monkeypatch, tmp_path):
be, rpc = _backend(monkeypatch, [_st(status='complete', completedLength='100')])
await be.fetch('http://x/y.mp3', tmp_path / 'y.part', referer='http://ref/')
assert rpc.added is not None
assert rpc.added[1]['referer'] == 'http://ref/'
# ---------- 取消 / 守卫 / 失败 ----------
def test_fetch_maps_cancel_and_drops_partial(monkeypatch, tmp_path):
be, api = _backend(monkeypatch, [(10, False, False)])
async def test_fetch_maps_cancel_and_drops_partial(monkeypatch, tmp_path):
be, rpc = _backend(monkeypatch, [_st(completedLength='10')])
target = tmp_path / 'y.part'
target.write_bytes(b'partial')
ctrl = tmp_path / 'y.part.aria2'
ctrl.write_bytes(b'aria2 ctrl')
ev = threading.Event()
ev.set()
with pytest.raises(DownloadCanceled):
be.fetch('http://x/y.mp3', tmp_path / 'y.part', cancel_event=ev)
assert api.removed == [('g1', True, True)]
await be.fetch('http://x/y.mp3', target, cancel_event=ev)
# 在途分支:先 forceRemove 停掉,再清结果表(与 aria2p 的 453-454 行一致)
assert rpc.removed == [('forceRemove', 'g1'),
('removeDownloadResult', 'g1')]
assert not target.exists() # 残片一并删除
assert not ctrl.exists() # 控制文件总是删除
def test_fetch_guards_size_and_drops_partial(monkeypatch, tmp_path):
be, api = _backend(monkeypatch, [(700, False, False)])
async def test_fetch_guards_size_and_drops_partial(monkeypatch, tmp_path):
be, rpc = _backend(monkeypatch, [_st(completedLength='700')])
target = tmp_path / 'y.part'
target.write_bytes(b'x' * 10)
with pytest.raises(DownloadException):
be.fetch('http://x/y.mp3', tmp_path / 'y.part', max_bytes=500)
assert api.removed == [('g1', True, True)]
await be.fetch('http://x/y.mp3', target, max_bytes=500)
assert rpc.removed == [('forceRemove', 'g1'),
('removeDownloadResult', 'g1')]
assert not target.exists()
def test_fetch_failure_is_retryable_and_keeps_partial(monkeypatch, tmp_path):
be, api = _backend(monkeypatch, [(10, False, True)])
with pytest.raises(RequestException):
be.fetch('http://x/y.mp3', tmp_path / 'y.part')
assert api.removed == [] # 残片保留,供下轮续传
async def test_size_guard_after_complete_clears_result_only(monkeypatch, tmp_path):
"""体积守卫在「完成」判定之前:任务已终结时只能清结果表,不是 forceRemove。
这条分支与 aria2p.remove() 的 if download.is_complete 分支对齐——
对已终结的 GID 调 forceRemove 会报错,故必须走 removeDownloadResult。
"""
be, rpc = _backend(monkeypatch, [_st(status='complete', completedLength='700')])
target = tmp_path / 'y.part'
target.write_bytes(b'x' * 10)
with pytest.raises(DownloadException):
await be.fetch('http://x/y.mp3', target, max_bytes=500)
assert rpc.removed == [('removeDownloadResult', 'g1')]
assert not target.exists()
def test_fetch_timeout_is_retryable(monkeypatch, tmp_path):
be, api = _backend(monkeypatch, [(10, False, False)]) # 永不完成
with pytest.raises(RequestException):
be.fetch('http://x/y.mp3', tmp_path / 'y.part', timeout=-1)
assert api.removed == [('g1', True, False)] # 超时中止但保留残片
async def test_fetch_failure_is_retryable_and_keeps_partial(monkeypatch, tmp_path):
be, rpc = _backend(monkeypatch, [
_st(status='error', completedLength='10', errorMessage='boom')])
target = tmp_path / 'y.part'
target.write_bytes(b'partial')
ctrl = tmp_path / 'y.part.aria2'
ctrl.write_bytes(b'aria2 ctrl')
with pytest.raises(UpstreamNetworkError):
await be.fetch('http://x/y.mp3', target)
assert rpc.removed == [] # 不调 _abort:残片与控制文件都保留,供下轮续传
assert target.exists()
assert ctrl.exists()
def test_fetch_submit_failure_is_retryable(monkeypatch, tmp_path):
be, _api = _backend(monkeypatch, [], fail_submit=True)
with pytest.raises(RequestException):
be.fetch('http://x/y.mp3', tmp_path / 'y.part')
async def test_fetch_timeout_is_retryable_and_keeps_partial(monkeypatch, tmp_path):
be, rpc = _backend(monkeypatch, [_st(completedLength='10')]) # 永不完成
target = tmp_path / 'y.part'
target.write_bytes(b'partial')
ctrl = tmp_path / 'y.part.aria2'
ctrl.write_bytes(b'aria2 ctrl')
with pytest.raises(UpstreamNetworkError):
await be.fetch('http://x/y.mp3', target, timeout=-1)
assert rpc.removed == [('forceRemove', 'g1'),
('removeDownloadResult', 'g1')]
assert target.exists() # 超时保留残片(与 aria2p 的 files=False 对齐)
assert not ctrl.exists() # 但控制文件仍删(aria2p 的 clean=True 默认)
def test_fetch_update_failure_is_retryable(monkeypatch, tmp_path):
be, _api = _backend(monkeypatch, [], fail_update=True)
with pytest.raises(RequestException):
be.fetch('http://x/y.mp3', tmp_path / 'y.part')
async def test_fetch_submit_failure_is_retryable(monkeypatch, tmp_path):
be, _rpc = _backend(monkeypatch, [], fail_add=True)
with pytest.raises(UpstreamNetworkError):
await be.fetch('http://x/y.mp3', tmp_path / 'y.part')
async def test_fetch_update_failure_is_retryable(monkeypatch, tmp_path):
be, _rpc = _backend(monkeypatch, [], fail_tell=True)
with pytest.raises(UpstreamNetworkError):
await be.fetch('http://x/y.mp3', tmp_path / 'y.part')
async def test_abort_failure_does_not_delete_partial(monkeypatch, tmp_path):
"""主移除动作失败时**不**删数据文件(与 aria2p 的 `files and result[-1]` 对齐)。"""
be, rpc = _backend(monkeypatch, [_st(completedLength='10')])
target = tmp_path / 'y.part'
target.write_bytes(b'partial')
async def boom(_gid):
raise RuntimeError('rpc down')
monkeypatch.setattr(rpc, 'force_remove', boom)
with pytest.raises(UpstreamNetworkError):
await be.fetch('http://x/y.mp3', target, timeout=-1)
assert target.exists()
def test_close_is_idempotent(monkeypatch):
be, _api = _backend(monkeypatch, [])
be, _rpc = _backend(monkeypatch, [])
be.close()
be.close()
# ---------- JSON-RPC 信封 ----------
class _FakeHTTP:
"""捕获 request_once 的入参,并按预设响应体返回。"""
def __init__(self, payload):
self.payload = payload
self.calls: list[dict] = []
async def __call__(self, method, url, **kw):
self.calls.append({'method': method, 'url': url, **kw})
return HTTPResult(status=200, url=url, text=json.dumps(self.payload))
async def test_rpc_envelope_shape(monkeypatch):
"""信封必须是 aria2 要求的 {"jsonrpc","id","method","params":["token:…",…]}。"""
fake = _FakeHTTP({'jsonrpc': '2.0', 'id': '1', 'result': 'gid-1'})
monkeypatch.setattr(a2._http, 'request_once', fake)
rpc = a2._Aria2Rpc(6800, 'SEC')
gid = await rpc.add_uri('http://u/a.mp3', {'out': 'a.part'})
assert gid == 'gid-1'
assert len(fake.calls) == 1
call = fake.calls[0]
assert call['method'] == 'POST'
assert call['url'] == 'http://127.0.0.1:6800/jsonrpc'
body = call['json_body']
assert body['jsonrpc'] == '2.0'
assert body['method'] == 'aria2.addUri'
assert body['params'][0] == 'token:SEC' # secret 以 token: 前缀入参
assert body['params'][1] == ['http://u/a.mp3']
assert body['params'][2] == {'out': 'a.part'}
async def test_rpc_call_impl_does_not_retry(monkeypatch):
"""变更类 RPC 必须走 request_once(不重试),避免 addUri 被重复提交。"""
fake = _FakeHTTP({'jsonrpc': '2.0', 'id': '1', 'result': 'OK'})
monkeypatch.setattr(a2._http, 'request_once', fake)
rpc = a2._Aria2Rpc(6800, 'SEC')
await rpc.force_remove('g1')
assert len(fake.calls) == 1
async def test_rpc_error_object_becomes_rpc_error(monkeypatch):
"""aria2 的协议层错误是 HTTP 200 + error 对象,必须收敛成 Aria2RpcError。"""
fake = _FakeHTTP({'jsonrpc': '2.0', 'id': '1',
'error': {'code': 1, 'message': 'GID not found'}})
monkeypatch.setattr(a2._http, 'request_once', fake)
rpc = a2._Aria2Rpc(6800, 'SEC')
with pytest.raises(a2.Aria2RpcError) as ei:
await rpc.tell_status('nope')
assert 'GID not found' in str(ei.value)
async def test_rpc_non_object_response_is_rpc_error(monkeypatch):
fake = _FakeHTTP([1, 2, 3]) # 合法 JSON 但不是对象
monkeypatch.setattr(a2._http, 'request_once', fake)
rpc = a2._Aria2Rpc(6800, 'SEC')
with pytest.raises(a2.Aria2RpcError):
await rpc.get_version()
+14 -26
View File
@@ -1,8 +1,8 @@
# -*- coding: utf-8 -*-
"""auth_system 同步栈的读写并发验证:读不持锁、写仍串行(WAL)。"""
"""auth_system 异步栈的读写并发验证:读不持锁、写仍串行(WAL)。"""
import asyncio
import sqlite3
import threading
import time
import pytest
@@ -15,33 +15,21 @@ def temp_site(tmp_path, monkeypatch):
monkeypatch.setattr(auth_system.db, 'DB_PATH', tmp_path / 'site.db')
monkeypatch.setattr(auth_system.db, 'BACKUP_DIR', tmp_path / 'backups')
monkeypatch.setattr(auth_system.db, 'SECRET_KEY_PATH', tmp_path / 'secret.key')
auth_system.init_db()
asyncio.run(auth_system.init_db())
return tmp_path / 'site.db'
def test_q_does_not_hold_lock(temp_site):
"""_q 读不应获取 _DB_LOCK(否则读会被写挡住)。"""
# 让另一个线程持有写锁,读仍应在短时间返回(不被阻塞)
held = threading.Event()
release = threading.Event()
"""_q 读不应获取写锁(否则读会被写挡住)。"""
def holder():
with auth_system._DB_LOCK:
held.set()
release.wait(5)
t = threading.Thread(target=holder, daemon=True)
t.start()
held.wait(2)
try:
start = time.monotonic()
rows = auth_system._q('SELECT COUNT(*) AS n FROM users')
elapsed = time.monotonic() - start
async def _main():
# 已在写锁内:读仍应立即返回(读路径不取写锁)
async with auth_system.db._write_lock():
rows = await auth_system._q('SELECT COUNT(*) AS n FROM users')
assert rows is not None
assert elapsed < 1.0, f'读被写锁阻塞了 {elapsed:.2f}s(WAL 下不应发生)'
finally:
release.set()
t.join(2)
assert rows[0]['n'] >= 0
asyncio.run(_main())
def test_concurrent_reads_and_writes(temp_site):
@@ -52,7 +40,7 @@ def test_concurrent_reads_and_writes(temp_site):
def writer():
try:
for i in range(50):
auth_system.set_setting(f'k{i}', str(i))
asyncio.run(auth_system.set_setting(f'k{i}', str(i)))
except Exception as e:
errors.append(('w', e))
finally:
@@ -61,7 +49,7 @@ def test_concurrent_reads_and_writes(temp_site):
def reader():
while not stop.is_set():
try:
auth_system.get_setting('k0')
asyncio.run(auth_system.get_setting('k0'))
except Exception as e:
errors.append(('r', e))
@@ -75,7 +63,7 @@ def test_concurrent_reads_and_writes(temp_site):
r.join(2)
assert not errors, f'并发读写异常: {errors[:3]}'
assert auth_system.get_setting('k49') == '49'
assert asyncio.run(auth_system.get_setting('k49')) == '49'
def test_wal_enabled(temp_site):
+10 -9
View File
@@ -8,6 +8,7 @@
按惯例:先在未改动的原始实现上跑绿,重构时这批测试即等价性验证器。
"""
import asyncio
import sqlite3
from typing import Any
@@ -90,7 +91,7 @@ def _tables(dbx: dict[str, Any]) -> set[str]:
def test_fresh_db_creates_tables_and_admin(dbx, capsys):
"""全新库:全部表就位、单行配置有种子行、初始管理员 role=admin 且余额 -1。"""
auth.init_db()
asyncio.run(auth.init_db())
tables = _tables(dbx)
for t in ('users', 'email_codes', 'cards', 'logs',
@@ -115,7 +116,7 @@ def test_fresh_db_creates_tables_and_admin(dbx, capsys):
def test_second_run_is_idempotent(dbx):
"""二次初始化:不报错、不重建管理员、已有数据原样保留。"""
auth.init_db()
asyncio.run(auth.init_db())
conn = _conn(dbx)
try:
conn.execute("UPDATE users SET duration_seconds = 42 WHERE role = 'admin'")
@@ -123,7 +124,7 @@ def test_second_run_is_idempotent(dbx):
finally:
conn.close()
auth.init_db() # 不应抛
asyncio.run(auth.init_db()) # 不应抛
conn = _conn(dbx)
try:
@@ -146,7 +147,7 @@ def test_legacy_db_columns_get_added(dbx):
conn.commit()
conn.close()
auth.init_db() # 不应抛
asyncio.run(auth.init_db()) # 不应抛
assert {'status', 'login_seq', 'recent_provinces', 'ban_reason'} <= _cols(dbx, 'users')
assert {'note', 'expire_at', 'disabled'} <= _cols(dbx, 'cards')
@@ -166,7 +167,7 @@ def test_legacy_db_columns_get_added(dbx):
def test_renamed_admin_not_recreated(dbx):
"""管理员改名(admin→qfbsh)后重启:按「无管理员角色」判断,不得再建新 admin。"""
auth.init_db()
asyncio.run(auth.init_db())
conn = _conn(dbx)
try:
conn.execute("UPDATE users SET username = 'qfbsh' WHERE role = 'admin'")
@@ -174,7 +175,7 @@ def test_renamed_admin_not_recreated(dbx):
finally:
conn.close()
auth.init_db()
asyncio.run(auth.init_db())
conn = _conn(dbx)
try:
@@ -200,7 +201,7 @@ def test_duplicate_nonempty_email_warns_and_continues(dbx, capsys):
conn.commit()
conn.close()
auth.init_db() # 不应抛(查重兜底在注册/建号路由,此处只告警)
asyncio.run(auth.init_db()) # 不应抛(查重兜底在注册/建号路由,此处只告警)
out = capsys.readouterr().out
assert '重复的非空邮箱' in out, f'应打印人工清理提示,实际输出: {out!r}'
@@ -225,7 +226,7 @@ def test_email_normalization_migrates_plus_tag(dbx):
conn.commit()
conn.close()
auth.init_db()
asyncio.run(auth.init_db())
conn = _conn(dbx)
try:
@@ -246,7 +247,7 @@ def test_email_normalization_collision_keeps_original(dbx, capsys):
conn.commit()
conn.close()
auth.init_db() # 不应抛
asyncio.run(auth.init_db()) # 不应抛
out = capsys.readouterr().out
assert '重复' in out, f'应打印撞行告警,实际输出: {out!r}'
+20 -6
View File
@@ -5,6 +5,9 @@
退出登录递增 login_seq、heartbeat 次数校验。
"""
import asyncio
import sqlite3
import pytest
from fastapi.testclient import TestClient
@@ -16,6 +19,17 @@ USER_PW = 'UserPass123'
ADMIN_PW = 'AdminPass123'
async def _fake_secret_key() -> str:
return SECRET
def _connect() -> sqlite3.Connection:
"""同步读沙箱库(auth.db.DB_PATH 已被 fixture 指向临时库)。"""
conn = sqlite3.connect(str(auth.db.DB_PATH))
conn.row_factory = sqlite3.Row
return conn
@pytest.fixture()
def app_env(tmp_path, monkeypatch):
"""沙箱站点库 + 独立密钥,隔离生产数据。"""
@@ -26,9 +40,9 @@ def app_env(tmp_path, monkeypatch):
monkeypatch.setattr(auth.db, 'SECRET_KEY_PATH', tmp_path / 'secret.key')
monkeypatch.setattr(db, 'DATA_DIR', tmp_path)
monkeypatch.setattr(db, 'DB_PATH', db_path)
auth.init_db()
asyncio.run(auth.init_db())
conn = auth.get_conn()
conn = _connect()
conn.execute('INSERT INTO users (username, password_hash, salt, role, duration_seconds, '
'status, login_seq, created_at) VALUES (?,?,?,?,?,1,0,?)',
('normaluser', auth._hash_new(USER_PW), '', 'user', 97, 1730000000))
@@ -39,7 +53,7 @@ def app_env(tmp_path, monkeypatch):
conn.commit()
conn.close()
monkeypatch.setattr('web.deps.secret_key', lambda: SECRET)
monkeypatch.setattr('web.deps.secret_key', _fake_secret_key)
# 清空登录失败锁定/2FA 的模块级内存态,避免测试间串扰
import web.auth_state as auth_routes
@@ -83,13 +97,13 @@ def test_admin_login_returns_2fa_ticket(app_env):
def test_logout_bumps_login_seq_and_clears(app_env):
app_env.post('/auth/login', json={'username': 'normaluser', 'password': USER_PW})
conn = auth.get_conn()
conn = _connect()
seq_before = conn.execute(
"SELECT login_seq FROM users WHERE username='normaluser'").fetchone()[0]
conn.close()
r = app_env.post('/auth/logout', json={})
assert r.status_code == 200
conn = auth.get_conn()
conn = _connect()
seq_after = conn.execute(
"SELECT login_seq FROM users WHERE username='normaluser'").fetchone()[0]
conn.close()
@@ -100,7 +114,7 @@ def test_logout_bumps_login_seq_and_clears(app_env):
def test_heartbeat_expired_credits(app_env):
conn = auth.get_conn()
conn = _connect()
conn.execute("UPDATE users SET duration_seconds=0 WHERE username='normaluser'")
conn.commit()
conn.close()
+23 -30
View File
@@ -5,6 +5,8 @@
settle/rename_to 消费过的行天然不在退款范围,因此自动退与成功路径
互不干扰,调用方无需手动取消。测试直接对真实账本(沙箱库)断言。
"""
import asyncio
import pytest
import auth_system as auth
@@ -18,22 +20,22 @@ def ledger_env(tmp_path, monkeypatch):
monkeypatch.setattr(auth.db, 'DB_PATH', tmp_path / 'site.db')
monkeypatch.setattr(auth.db, 'BACKUP_DIR', tmp_path / 'backups')
monkeypatch.setattr(auth.db, 'SECRET_KEY_PATH', tmp_path / 'secret.key')
auth.init_db()
row = auth.create_user('payer', 'Passw0rd12', email='p@example.com', credits=5)
asyncio.run(auth.init_db())
row = asyncio.run(auth.create_user('payer', 'Passw0rd12', email='p@example.com', credits=5))
assert row is not None, '建号应成功'
yield {'uid': int(row['id'])}
# 无需清理:tmp_path 由 pytest 回收
def _row(key: str):
async def _row(key: str):
"""取账本行,不存在则测试失败(供「行必须存在」的断言用)。"""
row = auth.ledger_get(key)
row = await auth.ledger_get(key)
assert row is not None, f'账本行 {key} 应存在'
return row
def _user_credits() -> int:
u = auth.get_user_by_name('payer')
async def _user_credits() -> int:
u = await auth.get_user_by_name('payer')
assert u is not None, '用户 payer 应存在'
return int(u['duration_seconds'] or 0)
@@ -43,81 +45,72 @@ def _make_billing(ledger_env, key: str | None) -> Billing:
return Billing(user=user, ledger_key=key, need=1, free=False)
@pytest.mark.anyio
async def test_scope_auto_refunds_open_row(ledger_env):
"""正常退出作用域:仍 open 的行被退(rejected),余额恢复。"""
key = 'req-aaa'
reserved = auth.reserve_and_open(ledger_env['uid'], 1, key)
reserved = await auth.reserve_and_open(ledger_env['uid'], 1, key)
assert reserved is not None
assert _row(key)['status'] == 'open'
assert (await _row(key))['status'] == 'open'
billing = _make_billing(ledger_env, key)
async with billing.scope():
pass # 业务体正常走完但未结算(等价于路由里的错误早退)
assert _row(key)['status'] == 'rejected'
assert _user_credits() == 5
assert (await _row(key))['status'] == 'rejected'
assert await _user_credits() == 5
@pytest.mark.anyio
async def test_scope_noop_after_settle(ledger_env):
"""作用域内已结算:退出自动退为 no-op,交付行不受影响。"""
key = 'req-bbb'
auth.reserve_and_open(ledger_env['uid'], 1, key)
await auth.reserve_and_open(ledger_env['uid'], 1, key)
billing = _make_billing(ledger_env, key)
async with billing.scope():
await billing.settle(1, 'delivered')
assert _row(key)['status'] == 'delivered'
assert _user_credits() == 4 # 恰好扣 1,未被二次退
assert (await _row(key))['status'] == 'delivered'
assert await _user_credits() == 4 # 恰好扣 1,未被二次退
@pytest.mark.anyio
async def test_scope_noop_after_rename(ledger_env):
"""作用域内账本已改挂 task_id:请求级行不存在,自动退为 no-op;
任务级行保持 open 交给 worker 结算——与迁移前手工语义逐路径一致。"""
key = 'req-ccc'
auth.reserve_and_open(ledger_env['uid'], 1, key)
await auth.reserve_and_open(ledger_env['uid'], 1, key)
billing = _make_billing(ledger_env, key)
async with billing.scope():
ok = await billing.rename_to('task-xyz')
assert ok is True
assert auth.ledger_get(key) is None
moved = _row('task-xyz')
assert await auth.ledger_get(key) is None
moved = await _row('task-xyz')
assert moved['status'] == 'open'
@pytest.mark.anyio
async def test_scope_refunds_on_exception_and_reraises(ledger_env):
"""业务体抛异常:先退款,异常原样穿透(路由自行转错误响应)。"""
key = 'req-ddd'
auth.reserve_and_open(ledger_env['uid'], 1, key)
await auth.reserve_and_open(ledger_env['uid'], 1, key)
billing = _make_billing(ledger_env, key)
with pytest.raises(ValueError, match='boom'):
async with billing.scope():
raise ValueError('boom')
assert _row(key)['status'] == 'rejected'
assert (await _row(key))['status'] == 'rejected'
@pytest.mark.anyio
async def test_scope_free_billing_noop(ledger_env):
"""free/unlimited(ledger_key=None):作用域进出零动作。"""
billing = _make_billing(ledger_env, None)
async with billing.scope():
pass
# 无账本行可动,不抛异常即为通过
assert _open_rows() == []
assert await _open_rows() == []
def _open_rows():
conn = auth.get_conn()
try:
rows = conn.execute("SELECT task_id FROM billing_ledger WHERE status='open'").fetchall()
async def _open_rows():
rows = await auth._q("SELECT task_id FROM billing_ledger WHERE status='open'")
return [dict(r) for r in rows]
finally:
conn.close()
+94 -63
View File
@@ -1,74 +1,83 @@
"""get_conn 连接策略契约:WAL 首连设置一次 + 目录自动创建。
"""db.connect 连接策略契约:WAL 首连设置一次 + 目录自动创建。
背景:get_conn 原实现每个连接都执行 mkdir + PRAGMA journal_mode=WAL,
实测每次多付 ~1.2ms(mkdir 190us + 新连接上 PRAGMA ~1ms)。WAL 模式
持久化在库文件头里,与 core/db.py 的 _WAL_READY_FOR 同款,按路径设一次即可。
背景:原同步实现每个连接都执行 mkdir + PRAGMA journal_mode=WAL,实测每次
多付 ~1.2ms(mkdir 190us + 新连接上 PRAGMA ~1ms)。WAL 模式持久化在库文件头
里,与 core/db.py 的 _WAL_READY_FOR 同款,按路径设一次即可。迁移到 aiosqlite
后策略不变:`_configure` 在专用锁内首连设置一次,库文件被删重建时(force_wal)
再补设。
"""
import sqlite3
import asyncio
import threading
from pathlib import Path
import aiosqlite
from auth_system import db
def _point_db_to(monkeypatch, tmp_path: Path, conn_counter: list[int] | None = None):
def _point_db_to(monkeypatch, tmp_path: Path, statements: list[str] | None = None):
"""把 DB_PATH 指到临时库,并清空 WAL 已就绪缓存。"""
new_path = tmp_path / 'site.db'
monkeypatch.setattr(db, 'DB_PATH', new_path)
monkeypatch.setattr(db, '_WAL_READY', set())
if conn_counter is not None:
real_connect = sqlite3.connect
monkeypatch.setattr(db, '_WAL_READY_FOR', None)
if statements is not None:
real_connect = aiosqlite.connect
def counting_connect(*a, **k):
conn_counter[0] += 1
return real_connect(*a, **k)
async def tracing_connect(*a, **k):
# 在第一句 PRAGMA 之前挂追踪器,才能捕获 _configure 执行的语句
conn = await real_connect(*a, **k)
await conn.set_trace_callback(statements.append)
return conn
monkeypatch.setattr(db.sqlite3, 'connect', counting_connect)
monkeypatch.setattr(db.aiosqlite, 'connect', tracing_connect)
return new_path
async def _open_close() -> None:
conn = await db.connect()
await conn.close()
async def _journal_mode() -> str:
conn = await db.connect()
try:
row = await (await conn.execute('PRAGMA journal_mode')).fetchone()
assert row is not None
return str(row[0]).lower()
finally:
await conn.close()
def test_wal_mode_active_on_fresh_db(monkeypatch, tmp_path):
"""新路径首连:WAL 生效。"""
_point_db_to(monkeypatch, tmp_path)
conn = db.get_conn()
try:
mode = conn.execute('PRAGMA journal_mode').fetchone()[0]
assert mode.lower() == 'wal'
finally:
conn.close()
assert asyncio.run(_journal_mode()) == 'wal'
def test_wal_pragma_only_once_per_path(monkeypatch, tmp_path):
"""同一路径第二次起不再执行 PRAGMA journal_mode=WAL(用 SQL 追踪器实证)。"""
_point_db_to(monkeypatch, tmp_path)
conn1 = db.get_conn()
conn1.close()
conn2 = db.get_conn()
conn2.close()
# 用 sqlite3 的 set_trace_callback 记录第三条连接上的语句(2 连接已建立缓存)
"""同一路径第二次起不再执行 PRAGMA journal_mode=WAL(SQL 追踪器实证)。"""
statements: list[str] = []
conn3 = db.get_conn()
conn3.set_trace_callback(statements.append)
conn3.execute('SELECT 1').fetchall()
conn3.close()
assert statements == ['SELECT 1'], f'第三条连接不应再跑 PRAGMA,实际: {statements}'
_point_db_to(monkeypatch, tmp_path, statements=statements)
asyncio.run(_open_close())
assert any('journal_mode=WAL' in s for s in statements), f'首连必须设 WAL,实际: {statements}'
statements.clear()
asyncio.run(_open_close())
assert not any('journal_mode=WAL' in s for s in statements), \
f'第二条连接不应再设 WAL,实际: {statements}'
# 常规 PRAGMA(busy_timeout 等)每连接仍照设
assert any('busy_timeout' in s for s in statements), statements
def test_new_path_resets_wal_cache(monkeypatch, tmp_path):
"""换库路径(测试隔离场景):新路径重新设置 WAL。"""
_point_db_to(monkeypatch, tmp_path / 'a')
c = db.get_conn()
c.close()
asyncio.run(_open_close())
monkeypatch.setattr(db, 'DB_PATH', tmp_path / 'b' / 'site.db')
c = db.get_conn()
try:
mode = c.execute('PRAGMA journal_mode').fetchone()[0]
assert mode.lower() == 'wal'
finally:
c.close()
assert asyncio.run(_journal_mode()) == 'wal'
def test_missing_dir_auto_created(monkeypatch, tmp_path):
@@ -77,48 +86,70 @@ def test_missing_dir_auto_created(monkeypatch, tmp_path):
_point_db_to(monkeypatch, deep)
assert not deep.exists()
conn = db.get_conn()
async def _main():
conn = await db.connect()
try:
conn.execute('CREATE TABLE t (v)').close()
conn.commit()
await conn.execute('CREATE TABLE t (v)')
await conn.commit()
finally:
conn.close()
await conn.close()
asyncio.run(_main())
assert (deep / 'site.db').exists()
def test_deleted_db_file_recovers(monkeypatch, tmp_path):
"""库文件被删(目录还在):重连自动重建且恢复 WAL 模式,不崩。"""
_point_db_to(monkeypatch, tmp_path)
c = db.get_conn()
c.close()
asyncio.run(_open_close())
(tmp_path / 'site.db').unlink()
conn = db.get_conn()
try:
conn.execute('SELECT 1').fetchall()
mode = conn.execute('PRAGMA journal_mode').fetchone()[0]
assert mode.lower() == 'wal', '重建的库文件必须恢复 WAL(journal_mode 不持久于新文件)'
finally:
conn.close()
assert asyncio.run(_journal_mode()) == 'wal', '重建的库文件必须恢复 WAL(journal_mode 不持久于新文件)'
def test_concurrent_first_connect_no_lease_conflict(monkeypatch, tmp_path):
"""多线程同时首连(读路径不持 _DB_LOCK):WAL 设置正确、无连接报错。
core/db.py 记录过并发 PRAGMA 争排他锁报 database is locked 的实测坑,
同步层用专用锁 + 二次复查同款防护。
"""**单事件循环内**并发首连(生产模型:一个 loop 上并发多个请求):
专用锁 + 二次复查保证 WAL 只切一次,无连接报错。
"""
import threading
_point_db_to(monkeypatch, tmp_path)
async def _main():
async def _one():
conn = await db.connect()
try:
await (await conn.execute('SELECT 1')).fetchall()
finally:
await conn.close()
await asyncio.gather(*[_one() for _ in range(16)])
asyncio.run(_main())
def test_concurrent_connect_from_threads_after_wal_ready(monkeypatch, tmp_path):
"""WAL 已就绪后,多线程各自独立 loop 建连不再触排他锁争用(读路径并发)。
为何不再钉「多线程**同时首连**」:异步层的 WAL 切换锁按事件循环隔离
(见 auth_system/db.py 的 _loop_lock,与 core/db.py 同款),跨 loop 的首连
无法用同一把 asyncio.Lock 串行化——那是同步栈 threading.Lock 才有的保证。
生产只有一个事件循环(任务监控线程的 asyncio.run 只在 init_db 之后启动),
首连争用实际发生在单 loop 内,由上一条用例覆盖。
"""
_point_db_to(monkeypatch, tmp_path)
asyncio.run(_open_close()) # 先把库切到 WAL,置 _WAL_READY_FOR
errors: list[Exception] = []
def worker():
async def _one():
conn = await db.connect()
try:
await (await conn.execute('SELECT 1')).fetchall()
finally:
await conn.close()
try:
for _ in range(5):
c = db.get_conn()
c.execute('SELECT 1').fetchall()
c.close()
asyncio.run(_one())
except Exception as e: # 收集全部失败供断言
errors.append(e)
+6 -1
View File
@@ -13,6 +13,11 @@ from web.errors import AppError
SECRET = 'deps-test-secret'
async def _fake_secret_key() -> str:
"""web.deps.secret_key 已异步化:替身须为协程函数(否则 await 处 TypeError)。"""
return SECRET
@pytest.fixture()
def site(tmp_path, monkeypatch):
path = tmp_path / 'site.db'
@@ -31,7 +36,7 @@ def site(tmp_path, monkeypatch):
conn.close()
monkeypatch.setattr(db, 'DATA_DIR', tmp_path)
monkeypatch.setattr(db, 'DB_PATH', path)
monkeypatch.setattr('web.deps.secret_key', lambda: SECRET)
monkeypatch.setattr('web.deps.secret_key', _fake_secret_key)
return path
+14 -9
View File
@@ -8,6 +8,8 @@
- 批量超 500:400 且退款
- 参数校验失败各分支均按「只退 open 行」退费
"""
import asyncio
import pytest
from fastapi.testclient import TestClient
@@ -18,6 +20,10 @@ SECRET = 'download-test-secret'
PW = 'Passw0rd12'
async def _fake_secret_key() -> str:
return SECRET
@pytest.fixture()
def env(tmp_path, monkeypatch):
db_path = tmp_path / 'site.db'
@@ -27,15 +33,15 @@ def env(tmp_path, monkeypatch):
monkeypatch.setattr(auth.db, 'SECRET_KEY_PATH', tmp_path / 'secret.key')
monkeypatch.setattr(db, 'DATA_DIR', tmp_path)
monkeypatch.setattr(db, 'DB_PATH', db_path)
auth.init_db()
asyncio.run(auth.init_db())
# 单一密钥源:web 层统一经 web.deps.secret_key 取密钥(调用期解析),
# patch 它即让登录签发与依赖校验同源
monkeypatch.setattr('web.deps.secret_key', lambda: SECRET)
monkeypatch.setattr('web.deps.secret_key', _fake_secret_key)
# 造 3 次余额用户 + 0 次余额用户
auth.create_user('payer', PW, email='p@example.com', credits=3)
auth.create_user('broke', PW, email='b@example.com', credits=0)
auth.create_user('rich', PW, email='r@example.com', credits=600)
asyncio.run(auth.create_user('payer', PW, email='p@example.com', credits=3))
asyncio.run(auth.create_user('broke', PW, email='b@example.com', credits=0))
asyncio.run(auth.create_user('rich', PW, email='r@example.com', credits=600))
from web.app import create_app
return TestClient(create_app())
@@ -61,15 +67,14 @@ def _login(env, username):
def _balance(username):
u = auth.get_user_by_name(username)
u = asyncio.run(auth.get_user_by_name(username))
assert u is not None, f'用户 {username} 不存在,余额无从校验'
return int(u['duration_seconds'] or 0)
def _open_ledger_rows():
conn = auth.get_conn()
rows = conn.execute("SELECT task_id, reserved, status FROM billing_ledger WHERE status='open'").fetchall()
conn.close()
rows = asyncio.run(auth._q(
"SELECT task_id, reserved, status FROM billing_ledger WHERE status='open'"))
return [dict(r) for r in rows]
+14 -10
View File
@@ -11,6 +11,8 @@ test_download_billing.py 覆盖参数校验与退费口径;本文件钉住 for
(路由内延迟导入取的就是这个实例属性),MusicInfo/DownloadResult 用
真实 dataclass 构造。
"""
import asyncio
import pytest
from fastapi.testclient import TestClient
@@ -22,6 +24,10 @@ SECRET = 'download-sync-test-secret'
PW = 'Passw0rd12'
async def _fake_secret_key() -> str:
return SECRET
def _info() -> MusicInfo:
return MusicInfo(id=347230, name='晴天', artists='周杰伦', album='叶惠美',
pic_url='', duration=269, track_number=1,
@@ -44,19 +50,19 @@ def env(tmp_path, monkeypatch):
monkeypatch.setattr(auth.db, 'SECRET_KEY_PATH', tmp_path / 'secret.key')
monkeypatch.setattr(db, 'DATA_DIR', tmp_path)
monkeypatch.setattr(db, 'DB_PATH', db_path)
auth.init_db()
monkeypatch.setattr('web.deps.secret_key', lambda: SECRET)
auth.create_user('payer', PW, email='p@example.com', credits=3)
asyncio.run(auth.init_db())
monkeypatch.setattr('web.deps.secret_key', _fake_secret_key)
asyncio.run(auth.create_user('payer', PW, email='p@example.com', credits=3))
state: dict = {'info_exc': None, 'dl_result': _dl(), 'dl_exc': None}
class _FakeDownloader:
def get_music_info(self, music_id, quality, cookies):
async def get_music_info(self, music_id, quality, cookies):
if state['info_exc'] is not None:
raise state['info_exc']
return state['info']
def download_music_file(self, music_id, quality, **kw):
async def download_music_file(self, music_id, quality, **kw):
if state['dl_exc'] is not None:
raise state['dl_exc']
return state['dl_result']
@@ -81,16 +87,14 @@ def env(tmp_path, monkeypatch):
def _balance(username: str = 'payer') -> int:
u = auth.get_user_by_name(username)
u = asyncio.run(auth.get_user_by_name(username))
assert u is not None, f'用户 {username} 不存在'
return int(u['duration_seconds'] or 0)
def _open_rows() -> list[dict]:
conn = auth.get_conn()
rows = conn.execute(
"SELECT task_id, status FROM billing_ledger WHERE status='open'").fetchall()
conn.close()
rows = asyncio.run(auth._q(
"SELECT task_id, status FROM billing_ledger WHERE status='open'"))
return [dict(r) for r in rows]
+143 -120
View File
@@ -9,64 +9,83 @@ ZIP 都经它,约 205 行 / 圈复杂度 47,此前**零直接测试覆盖**
断言只依赖「调用后世界里发生了什么」——返回值、磁盘上的文件、发往网络的请求——
不引用任何内部函数名,因此本文件可作为后续拆分的等价性验证器。
替身约定(三处,均为模块级符号,替换点在 downloader 模块命名空间):
- 网络:``get_http_session`` / ``get_backend``(后者返回 None = 走内置 HTTP 通道)
替身约定(均为模块级符号,替换点在 downloader 模块命名空间):
- 网络:``_http``(core.http_client 的进程级异步客户端)——替身提供
``async open(method, url, **kw)``,返回带 ``status``/``headers``/``content``
的未关闭响应;分块读取走**真实的** ``iter_chunks``,顺带钉住「读取期异常
收敛为 UpstreamNetworkError」这条边界。
- aria2:``get_backend``(**协程**替身,返回 None = 强制走内置 HTTP 通道)
- 内容嗅探:``_sniff_audio_header``(默认放行,按需置 False)
- 元数据写入:``MusicDownloader._write_metadata``(不真跑 mutagen)
"""
from __future__ import annotations
import threading
from pathlib import Path
from typing import Any
import aiohttp
import pytest
from core import sync_http
from music_downloader import DownloadCanceled, DownloadException, MusicInfo
from music_downloader.downloader import MusicDownloader
# ============ 替身 ============
class _FakeResponse:
"""core.sync_http.SyncResponse 的替身。
class _FakeContent:
"""``aiohttp.ClientResponse.content`` 的替身。
chunks 保留原始 iterable(**不要** list() 归一化):生成器的副作用
(如中途置取消位)若在构造时刻就求值,「中途取消」会被悄悄测成
「开始前取消」——两条不同的路径混成一条,而测试仍是绿的。
元素约定:可调用对象 → 调用一次(注入副作用,如 cancel.set);
异常实例 → 抛出(模拟读取期断流);其余 → 作为分块产出。
"""
def __init__(self, chunks, status=200, headers=None):
def __init__(self, chunks):
self._chunks = chunks
self.status_code = status
async def iter_chunked(self, chunk_size=0):
for item in self._chunks:
if callable(item):
item()
continue
if isinstance(item, BaseException):
raise item
yield item
class _FakeResponse:
"""``core.http_client.open`` 返回的未关闭流式响应替身。"""
def __init__(self, chunks, status=200, headers=None, reason=''):
self.status = status
self.reason = reason
self.headers = dict(headers or {})
self.content = _FakeContent(chunks)
self.closed = False
def iter_content(self, chunk_size=0):
yield from self._chunks
def raise_for_status(self):
if self.status_code >= 400:
raise sync_http.HTTPError(f'HTTP {self.status_code}', self)
def close(self):
self.closed = True
class _FakeSession:
"""记录每次 GET(含 headers)并按序产出响应。"""
class _FakeClient:
"""记录每次 open(含 kw)并按序产出响应;responder 可随用例重装。"""
def __init__(self, responder):
self._responder = responder
def __init__(self):
self._responder = None
self.calls: list[tuple[str, dict[str, Any]]] = []
self.closed = False
def get(self, url, **kw):
def set_responder(self, responder):
self._responder = responder
async def open(self, method, url, **kw):
self.calls.append((url, kw))
assert self._responder is not None, '本用例未安装网络替身'
return self._responder(len(self.calls), url, kw)
def close(self):
self.closed = True
def _body(size, status=200, content_range=None):
"""固定返回 size 字节的响应工厂(每次调用产出新响应对象)。"""
@@ -78,6 +97,11 @@ def _body(size, status=200, content_range=None):
return _mk
def _headers(kw) -> dict[str, str]:
"""取一次调用的请求头——没带 Range 时 downloader 传的是 None。"""
return kw.get('headers') or {}
def _info(mid=101, size=20, file_type='mp3'):
return MusicInfo(
id=mid, name='测试曲', artists='歌手', album='专辑', pic_url='',
@@ -98,22 +122,27 @@ def env(tmp_path, monkeypatch):
- state['sniff']:内容嗅探结果,默认 True
- state['meta']:_write_metadata 的调用记录 [(文件路径, fields), ...]
- install(responder) -> _FakeSession:安装网络替身,返回该替身以便断言
- install(responder) -> _FakeClient:安装网络替身,返回它以便断言
"""
state: dict[str, Any] = {'sniff': True, 'meta': []}
box: dict[str, _FakeSession] = {}
client = _FakeClient()
def install(responder):
box['session'] = _FakeSession(responder)
return box['session']
client.set_responder(responder)
return client
def _record_meta(self, file_path, music_info, fields=None):
async def _record_meta(self, file_path, music_info, fields=None):
state['meta'].append((str(file_path), fields))
return True
monkeypatch.setattr('music_downloader.downloader.get_http_session',
lambda: box['session'])
monkeypatch.setattr('music_downloader.downloader.get_backend', lambda: None)
monkeypatch.setattr('music_downloader.downloader._http', client)
# get_backend 已是原生异步(调用点直接 await)——替身也必须是协程,
# 同步 lambda 会被 await 成 None 而报 "object NoneType can't be used in 'await'"
async def _no_aria2():
return None
monkeypatch.setattr('music_downloader.downloader.get_backend', _no_aria2)
monkeypatch.setattr('music_downloader.downloader._sniff_audio_header',
lambda _p: state['sniff'])
monkeypatch.setattr(MusicDownloader, '_write_metadata', _record_meta)
@@ -126,48 +155,48 @@ def env(tmp_path, monkeypatch):
# ============ 正常落盘 ============
def test_success_promotes_part_to_final(env):
async def test_success_promotes_part_to_final(env):
dl, _state, install = env
info = _info(size=20)
fp, part = _paths(dl, info)
install(_body(20))
r = dl.download_music_file(101, music_info=info)
r = await dl.download_music_file(101, music_info=info)
assert r.success and r.error_message == ''
assert Path(r.file_path) == fp
assert Path(r.file_path or '') == fp
assert r.file_size == 20 and fp.read_bytes() == b'x' * 20
assert not part.exists(), '完成后 .part 应改名转正'
def test_empty_chunks_skipped(env):
async def test_empty_chunks_skipped(env):
dl, _state, install = env
info = _info(size=20)
fp, _ = _paths(dl, info)
install(lambda a, u, kw: _FakeResponse([b'', b'x' * 20, b'']))
seen: list[int] = []
r = dl.download_music_file(101, music_info=info,
r = await dl.download_music_file(101, music_info=info,
progress_cb=lambda w, _t: seen.append(w))
assert r.success and fp.stat().st_size == 20
assert seen == [0, 20], '空分块不写盘也不触发进度回调'
def test_progress_reports_resume_start_then_each_chunk(env):
async def test_progress_reports_resume_start_then_each_chunk(env):
dl, _state, install = env
info = _info(size=20)
install(lambda a, u, kw: _FakeResponse([b'a' * 8, b'b' * 12]))
seen: list[tuple[int, int]] = []
r = dl.download_music_file(101, music_info=info,
r = await dl.download_music_file(101, music_info=info,
progress_cb=lambda w, t: seen.append((w, t)))
assert r.success
assert seen == [(0, 20), (8, 20), (20, 20)], '起始回调带续传起点,此后每块一次'
def test_progress_callback_exception_never_breaks_download(env):
async def test_progress_callback_exception_never_breaks_download(env):
dl, _state, install = env
info = _info(size=20)
install(_body(20))
@@ -175,20 +204,24 @@ def test_progress_callback_exception_never_breaks_download(env):
def _boom(_w, _t):
raise RuntimeError('回调炸了')
r = dl.download_music_file(101, music_info=info, progress_cb=_boom)
r = await dl.download_music_file(101, music_info=info, progress_cb=_boom)
assert r.success, '进度回调异常绝不影响下载主流程'
def test_music_info_fetched_when_not_provided(env, monkeypatch):
async def test_music_info_fetched_when_not_provided(env, monkeypatch):
dl, _state, install = env
info = _info(size=20)
seen: list[tuple] = []
monkeypatch.setattr(dl, 'get_music_info',
lambda mid, q, c=None: (seen.append((mid, q)), info)[1])
async def _fake_info(mid, q, c=None):
seen.append((mid, q))
return info
monkeypatch.setattr(dl, 'get_music_info', _fake_info)
install(_body(20))
r = dl.download_music_file(101, 'lossless')
r = await dl.download_music_file(101, 'lossless')
assert r.success and seen == [(101, 'lossless')]
@@ -196,40 +229,40 @@ def test_music_info_fetched_when_not_provided(env, monkeypatch):
# ============ 缓存校验 ============
def test_cache_hit_skips_network(env):
async def test_cache_hit_skips_network(env):
dl, _state, install = env
info = _info(size=20)
fp, _ = _paths(dl, info)
fp.write_bytes(b'c' * 20)
session = install(lambda *a: pytest.fail('缓存命中不应发网络请求'))
r = dl.download_music_file(101, music_info=info)
r = await dl.download_music_file(101, music_info=info)
assert r.success and r.file_size == 20
assert session.calls == []
def test_cache_size_mismatch_triggers_redownload(env):
async def test_cache_size_mismatch_triggers_redownload(env):
dl, _state, install = env
info = _info(size=20)
fp, _ = _paths(dl, info)
fp.write_bytes(b'c' * 15) # 与本次期望不符 = 缓存 miss
session = install(_body(20))
r = dl.download_music_file(101, music_info=info)
r = await dl.download_music_file(101, music_info=info)
assert r.success and fp.read_bytes() == b'x' * 20
assert len(session.calls) == 1, '坏缓存应被删除并重下,而不是当结果交付'
def test_zero_byte_cache_rejected_when_expected_size_unknown(env):
async def test_zero_byte_cache_rejected_when_expected_size_unknown(env):
dl, _state, install = env
info = _info(size=0)
fp, _ = _paths(dl, info)
fp.write_bytes(b'')
session = install(lambda a, u, kw: _FakeResponse([b'good']))
r = dl.download_music_file(101, music_info=info)
r = await dl.download_music_file(101, music_info=info)
assert r.success and fp.read_bytes() == b'good'
assert len(session.calls) == 1, '期望未知时至少拒绝 0 字节坏文件'
@@ -238,7 +271,7 @@ def test_zero_byte_cache_rejected_when_expected_size_unknown(env):
# ============ 断点续传 ============
def test_resume_sends_range_and_appends(env):
async def test_resume_sends_range_and_appends(env):
dl, _state, install = env
info = _info(size=20)
fp, part = _paths(dl, info)
@@ -247,67 +280,67 @@ def test_resume_sends_range_and_appends(env):
[b'q' * 10], status=206, headers={'Content-Range': 'bytes 10-19/20'}))
seen: list[tuple[int, int]] = []
r = dl.download_music_file(101, music_info=info,
r = await dl.download_music_file(101, music_info=info,
progress_cb=lambda w, t: seen.append((w, t)))
assert r.success
assert session.calls[0][1]['headers'].get('Range') == 'bytes=10-'
assert _headers(session.calls[0][1]).get('Range') == 'bytes=10-'
assert fp.read_bytes() == b'p' * 10 + b'q' * 10, '续传应追加而非覆盖'
assert seen[0] == (10, 20), '起始进度应带续传起点'
def test_server_ignoring_range_rewrites_from_zero(env):
async def test_server_ignoring_range_rewrites_from_zero(env):
dl, _state, install = env
info = _info(size=20)
fp, part = _paths(dl, info)
part.write_bytes(b'p' * 10)
install(lambda a, u, kw: _FakeResponse([b'r' * 20], status=200))
r = dl.download_music_file(101, music_info=info)
r = await dl.download_music_file(101, music_info=info)
assert r.success and fp.read_bytes() == b'r' * 20, '服务器返回 200 须从头重写'
def test_complete_part_promoted_without_network(env):
async def test_complete_part_promoted_without_network(env):
dl, _state, install = env
info = _info(size=20)
fp, part = _paths(dl, info)
part.write_bytes(b'z' * 20) # 已下完但未改名
session = install(lambda *a: pytest.fail('完整残片不应发网络请求'))
r = dl.download_music_file(101, music_info=info)
r = await dl.download_music_file(101, music_info=info)
assert r.success and r.file_size == 20
assert session.calls == []
assert fp.read_bytes() == b'z' * 20 and not part.exists()
def test_oversized_part_discarded_starts_from_zero(env):
async def test_oversized_part_discarded_starts_from_zero(env):
dl, _state, install = env
info = _info(size=20)
_fp, part = _paths(dl, info)
part.write_bytes(b'o' * 30) # 超尺寸残片没有续传价值
session = install(_body(20))
r = dl.download_music_file(101, music_info=info)
r = await dl.download_music_file(101, music_info=info)
assert r.success
assert 'Range' not in session.calls[0][1]['headers']
assert 'Range' not in _headers(session.calls[0][1])
def test_zero_byte_part_discarded(env):
async def test_zero_byte_part_discarded(env):
dl, _state, install = env
info = _info(size=20)
_fp, part = _paths(dl, info)
part.write_bytes(b'')
session = install(_body(20))
r = dl.download_music_file(101, music_info=info)
r = await dl.download_music_file(101, music_info=info)
assert r.success and 'Range' not in session.calls[0][1]['headers']
assert r.success and 'Range' not in _headers(session.calls[0][1])
def test_content_range_offset_mismatch_restarts_from_scratch(env):
async def test_content_range_offset_mismatch_restarts_from_scratch(env):
dl, _state, install = env
info = _info(size=20)
fp, part = _paths(dl, info)
@@ -321,14 +354,14 @@ def test_content_range_offset_mismatch_restarts_from_scratch(env):
session = install(responder)
r = dl.download_music_file(101, music_info=info, retries=1)
r = await dl.download_music_file(101, music_info=info, retries=1)
assert r.success and len(session.calls) == 2
assert 'Range' not in session.calls[1][1]['headers'], '清残片后应从头下'
assert 'Range' not in _headers(session.calls[1][1]), '清残片后应从头下'
assert fp.read_bytes() == b'f' * 20
def test_206_without_content_range_restarts_from_scratch(env):
async def test_206_without_content_range_restarts_from_scratch(env):
dl, _state, install = env
info = _info(size=20)
_fp, part = _paths(dl, info)
@@ -341,13 +374,13 @@ def test_206_without_content_range_restarts_from_scratch(env):
session = install(responder)
r = dl.download_music_file(101, music_info=info, retries=1)
r = await dl.download_music_file(101, music_info=info, retries=1)
assert r.success and len(session.calls) == 2
assert 'Range' not in session.calls[1][1]['headers']
assert 'Range' not in _headers(session.calls[1][1])
def test_http_416_restarts_from_scratch(env):
async def test_http_416_restarts_from_scratch(env):
dl, _state, install = env
info = _info(size=20)
fp, part = _paths(dl, info)
@@ -360,45 +393,45 @@ def test_http_416_restarts_from_scratch(env):
session = install(responder)
r = dl.download_music_file(101, music_info=info, retries=1)
r = await dl.download_music_file(101, music_info=info, retries=1)
assert r.success and len(session.calls) == 2
assert 'Range' not in session.calls[1][1]['headers']
assert 'Range' not in _headers(session.calls[1][1])
assert fp.read_bytes() == b'f' * 20
# ============ 失败与重试 ============
def test_cdn_4xx_is_deterministic_and_not_retried(env):
async def test_cdn_4xx_is_deterministic_and_not_retried(env):
dl, _state, install = env
info = _info(size=20)
session = install(lambda a, u, kw: _FakeResponse([], status=403))
with pytest.raises(DownloadException) as ei:
dl.download_music_file(101, music_info=info, retries=2)
await dl.download_music_file(101, music_info=info, retries=2)
assert 'HTTP 403' in str(ei.value)
assert len(session.calls) == 1, '确定性失败对同一失效直链白重试毫无意义'
def test_408_is_retried_then_reported_as_failure(env):
async def test_408_is_retried_then_reported_as_failure(env):
dl, _state, install = env
info = _info(size=20)
session = install(lambda a, u, kw: _FakeResponse([], status=408))
r = dl.download_music_file(101, music_info=info, retries=1)
r = await dl.download_music_file(101, music_info=info, retries=1)
assert not r.success
assert '下载请求失败' in r.error_message
assert len(session.calls) == 2, '408 请求超时应保留可重试'
def test_5xx_is_retried_and_keeps_part(env):
async def test_5xx_is_retried_and_keeps_part(env):
"""5xx 是服务端瞬时故障(非确定性失败):应可重试,且保留 .part 供续传。
与 4xx 的分界在这里钉住:error 分类只看 400<=status<500,5xx 落回裸重抛,
由外层 RequestException 分支退避重试。此前这条路径无测试覆盖。
由外层 UpstreamError 分支退避重试。此前这条路径无测试覆盖。
"""
dl, _state, install = env
info = _info(size=100)
@@ -406,7 +439,7 @@ def test_5xx_is_retried_and_keeps_part(env):
part.write_bytes(b'p' * 10)
session = install(lambda a, u, kw: _FakeResponse([], status=500))
r = dl.download_music_file(101, music_info=info, retries=1)
r = await dl.download_music_file(101, music_info=info, retries=1)
assert not r.success and '下载请求失败' in r.error_message
assert len(session.calls) == 2, '5xx 非确定性失败,应重试'
@@ -414,20 +447,20 @@ def test_5xx_is_retried_and_keeps_part(env):
assert part.stat().st_size == 10, '.part 应保留供下轮续传'
def test_incomplete_size_keeps_part_and_reports(env):
async def test_incomplete_size_keeps_part_and_reports(env):
dl, _state, install = env
info = _info(size=100)
fp, part = _paths(dl, info)
install(_body(10))
r = dl.download_music_file(101, music_info=info, retries=0)
r = await dl.download_music_file(101, music_info=info, retries=0)
assert not r.success and '下载不完整' in r.error_message
assert part.exists() and part.stat().st_size == 10, '.part 应保留以供续传'
assert not fp.exists(), '不完整的下载不得转正交付'
def test_incomplete_then_resumes_on_next_attempt(env):
async def test_incomplete_then_resumes_on_next_attempt(env):
dl, _state, install = env
info = _info(size=100)
_fp, part = _paths(dl, info)
@@ -440,25 +473,24 @@ def test_incomplete_then_resumes_on_next_attempt(env):
session = install(responder)
r = dl.download_music_file(101, music_info=info, retries=1)
r = await dl.download_music_file(101, music_info=info, retries=1)
assert not r.success and '下载不完整' in r.error_message
assert session.calls[1][1]['headers'].get('Range') == 'bytes=10-', '下轮应续传'
assert _headers(session.calls[1][1]).get('Range') == 'bytes=10-', '下轮应续传'
assert part.stat().st_size == 15
def test_network_error_keeps_part_for_resume(env):
async def test_network_error_keeps_part_for_resume(env):
"""读取期断流:真实的 iter_chunks 把 aiohttp.ClientError 收敛成
UpstreamNetworkError,外层据此保留 .part 供续传。"""
dl, _state, install = env
info = _info(size=100)
fp, part = _paths(dl, info)
def chunks():
yield b'a' * 10
raise sync_http.RequestException('连接中断')
install(lambda a, u, kw: _FakeResponse(
[b'a' * 10, aiohttp.ClientPayloadError('连接中断')]))
install(lambda a, u, kw: _FakeResponse(chunks()))
r = dl.download_music_file(101, music_info=info, retries=0)
r = await dl.download_music_file(101, music_info=info, retries=0)
assert not r.success and '下载请求失败' in r.error_message
assert part.exists() and part.stat().st_size == 10, '网络错误应保留 .part'
@@ -468,7 +500,7 @@ def test_network_error_keeps_part_for_resume(env):
# ============ 内容校验与体积上限 ============
def test_unknown_size_non_audio_content_rejected(env):
async def test_unknown_size_non_audio_content_rejected(env):
dl, state, install = env
info = _info(size=0)
fp, part = _paths(dl, info)
@@ -476,25 +508,25 @@ def test_unknown_size_non_audio_content_rejected(env):
install(lambda a, u, kw: _FakeResponse([b'<html>blocked</html>']))
with pytest.raises(DownloadException) as ei:
dl.download_music_file(101, music_info=info)
await dl.download_music_file(101, music_info=info)
assert '非音频数据' in str(ei.value)
assert not part.exists() and not fp.exists()
def test_unknown_size_audio_content_accepted(env):
async def test_unknown_size_audio_content_accepted(env):
dl, state, install = env
info = _info(size=0)
fp, _ = _paths(dl, info)
state['sniff'] = True
install(lambda a, u, kw: _FakeResponse([b'ID3audio']))
r = dl.download_music_file(101, music_info=info)
r = await dl.download_music_file(101, music_info=info)
assert r.success and fp.read_bytes() == b'ID3audio'
def test_size_limit_discards_part_and_fails(env, monkeypatch):
async def test_size_limit_discards_part_and_fails(env, monkeypatch):
dl, _state, install = env
monkeypatch.setattr('music_downloader.downloader._MAX_DOWNLOAD_BYTES', 16)
info = _info(size=0)
@@ -502,7 +534,7 @@ def test_size_limit_discards_part_and_fails(env, monkeypatch):
install(lambda a, u, kw: _FakeResponse([b'x' * 100]))
with pytest.raises(DownloadException) as ei:
dl.download_music_file(101, music_info=info)
await dl.download_music_file(101, music_info=info)
assert '大小上限' in str(ei.value)
assert not part.exists() and not fp.exists(), '超限残片无续传价值,须清理'
@@ -511,7 +543,7 @@ def test_size_limit_discards_part_and_fails(env, monkeypatch):
# ============ 取消 ============
def test_cancel_before_transfer_clears_part_and_skips_network(env):
async def test_cancel_before_transfer_clears_part_and_skips_network(env):
dl, _state, install = env
info = _info(size=20)
_fp, part = _paths(dl, info)
@@ -521,51 +553,42 @@ def test_cancel_before_transfer_clears_part_and_skips_network(env):
session = install(lambda *a: pytest.fail('已取消不应发网络请求'))
with pytest.raises(DownloadCanceled):
dl.download_music_file(101, music_info=info, cancel_event=cancel)
await dl.download_music_file(101, music_info=info, cancel_event=cancel)
assert session.calls == []
assert not part.exists(), '取消须清掉 .part'
def test_cancel_midway_clears_half_written_file(env):
async def test_cancel_midway_clears_half_written_file(env):
dl, _state, install = env
info = _info(size=20)
fp, part = _paths(dl, info)
cancel = threading.Event()
def chunks():
yield b'a' * 5
cancel.set()
yield b'b' * 15
install(lambda a, u, kw: _FakeResponse(chunks()))
# 第二块产出**之前**置取消位:取消发生在迭代过程中(.part 已建立)
install(lambda a, u, kw: _FakeResponse([b'a' * 5, cancel.set, b'b' * 15]))
with pytest.raises(DownloadCanceled):
dl.download_music_file(101, music_info=info, cancel_event=cancel)
await dl.download_music_file(101, music_info=info, cancel_event=cancel)
assert not part.exists() and not fp.exists()
def test_network_error_while_canceled_reports_canceled(env):
async def test_network_error_while_canceled_reports_canceled(env):
dl, _state, install = env
info = _info(size=20)
fp, part = _paths(dl, info)
cancel = threading.Event()
def chunks():
yield b'a' * 5
cancel.set()
raise OSError('句柄异常')
install(lambda a, u, kw: _FakeResponse(chunks()))
install(lambda a, u, kw: _FakeResponse([b'a' * 5, cancel.set, OSError('句柄异常')]))
with pytest.raises(DownloadCanceled):
dl.download_music_file(101, music_info=info, cancel_event=cancel, retries=2)
await dl.download_music_file(101, music_info=info, cancel_event=cancel, retries=2)
assert not part.exists() and not fp.exists(), '已取消时残片没有续传价值'
def test_cancel_after_promote_keeps_file_but_skips_metadata(env):
async def test_cancel_after_promote_keeps_file_but_skips_metadata(env):
dl, state, install = env
info = _info(size=20)
fp, part = _paths(dl, info)
@@ -578,7 +601,7 @@ def test_cancel_after_promote_keeps_file_but_skips_metadata(env):
install(_body(20))
with pytest.raises(DownloadCanceled):
dl.download_music_file(101, music_info=info, cancel_event=cancel,
await dl.download_music_file(101, music_info=info, cancel_event=cancel,
progress_cb=cb)
assert fp.exists() and not part.exists(), '已转正的文件不回滚'
@@ -588,14 +611,14 @@ def test_cancel_after_promote_keeps_file_but_skips_metadata(env):
# ============ 元数据 ============
def test_metadata_written_only_when_requested(env):
async def test_metadata_written_only_when_requested(env):
dl, state, install = env
install(_body(20))
dl.download_music_file(101, music_info=_info(mid=101), write_meta=False)
await dl.download_music_file(101, music_info=_info(mid=101), write_meta=False)
assert state['meta'] == [], 'write_meta=False 不应写元数据'
r = dl.download_music_file(102, music_info=_info(mid=102), write_meta=True,
r = await dl.download_music_file(102, music_info=_info(mid=102), write_meta=True,
meta_fields=['title'])
assert r.success
assert [f for _p, f in state['meta']] == [['title']], 'fields 应原样透传'
+8 -2
View File
@@ -31,6 +31,12 @@ _GOLDEN = Path(__file__).parent / 'golden' / 'responses.json'
SECRET = 'golden-secret'
async def _fake_secret_key() -> str:
"""web.deps.secret_key / auth_system.get_secret_key 均已异步化:
替身须为协程函数,否则 await / asyncio.run 处会报 TypeError。"""
return SECRET
def _load_cases():
return json.loads(_GOLDEN.read_text(encoding='utf-8'))
@@ -80,8 +86,8 @@ def client(tmp_path, monkeypatch):
monkeypatch.setattr(auth_system.db, 'DB_PATH', tmp_path / 'site.db')
monkeypatch.setattr(core_db, 'DATA_DIR', tmp_path)
monkeypatch.setattr(core_db, 'DB_PATH', tmp_path / 'site.db')
monkeypatch.setattr('web.deps.secret_key', lambda: SECRET)
monkeypatch.setattr(auth_system, 'get_secret_key', lambda: SECRET)
monkeypatch.setattr('web.deps.secret_key', _fake_secret_key)
monkeypatch.setattr(auth_system, 'get_secret_key', _fake_secret_key)
from web.app import create_app
return TestClient(create_app(api_prefix=''))
+25 -12
View File
@@ -4,6 +4,9 @@
test_auth_routes.py 覆盖成功主路径;本文件钉住 login 的各拒绝分支,
作为 login 拆解的等价性验证器。
"""
import asyncio
import sqlite3
import pytest
from fastapi.testclient import TestClient
@@ -15,6 +18,16 @@ USER_PW = 'UserPass123'
ADMIN_PW = 'AdminPass123'
async def _fake_secret_key() -> str:
return SECRET
def _connect() -> sqlite3.Connection:
conn = sqlite3.connect(str(auth.db.DB_PATH))
conn.row_factory = sqlite3.Row
return conn
@pytest.fixture()
def env(tmp_path, monkeypatch):
db_path = tmp_path / 'site.db'
@@ -24,9 +37,9 @@ def env(tmp_path, monkeypatch):
monkeypatch.setattr(auth.db, 'SECRET_KEY_PATH', tmp_path / 'secret.key')
monkeypatch.setattr(db, 'DATA_DIR', tmp_path)
monkeypatch.setattr(db, 'DB_PATH', db_path)
auth.init_db()
asyncio.run(auth.init_db())
conn = auth.get_conn()
conn = _connect()
conn.execute('INSERT INTO users (username, password_hash, salt, role, duration_seconds, '
'status, login_seq, created_at, email) VALUES (?,?,?,?,?,1,0,?,?)',
('mailuser', auth._hash_new(USER_PW), '', 'user', 97, 1730000000,
@@ -37,7 +50,7 @@ def env(tmp_path, monkeypatch):
conn.commit()
conn.close()
monkeypatch.setattr('web.deps.secret_key', lambda: SECRET)
monkeypatch.setattr('web.deps.secret_key', _fake_secret_key)
import web.auth_state as auth_state
auth_state._LOGIN_FAILS.clear()
@@ -55,7 +68,7 @@ def _logs(conn, action: str) -> list[tuple[str, str]]:
def test_overlong_password_rejected_without_pbkdf2(env, monkeypatch):
"""超长密码(>128):不碰 PBKDF2,直接按密码错误拒绝。"""
def _boom(*a, **k):
async def _boom(*a, **k):
raise AssertionError('verify_login must not be called')
monkeypatch.setattr(auth, 'verify_login', _boom)
@@ -90,7 +103,7 @@ def test_locked_log_written_once(env):
env.post('/auth/login', json={'username': 'mailuser', 'password': 'nope'})
for _ in range(3):
env.post('/auth/login', json={'username': 'mailuser', 'password': USER_PW})
conn = auth.get_conn()
conn = _connect()
locked_logs = [d for t, d in _logs(conn, '登录') if d == '失败次数过多,临时锁定']
conn.close()
assert len(locked_logs) == 1
@@ -98,21 +111,21 @@ def test_locked_log_written_once(env):
def test_banned_account_message(env, monkeypatch):
"""封禁账号(status=2):文案带原因 + 失败日志。"""
conn = auth.get_conn()
conn = _connect()
conn.execute("UPDATE users SET status=2, ban_reason='刷单' WHERE username='mailuser'")
conn.commit()
conn.close()
r = env.post('/auth/login', json={'username': 'mailuser', 'password': USER_PW})
assert r.json()['message'] == '账号已被封禁:刷单,请联系管理员'
conn = auth.get_conn()
conn = _connect()
assert ('fail', '封禁账号尝试登录: 刷单') in _logs(conn, '登录')
conn.close()
def test_disabled_account_message(env):
"""禁用账号(status=0):固定文案,无原因拼接。"""
conn = auth.get_conn()
conn = _connect()
conn.execute("UPDATE users SET status=0 WHERE username='mailuser'")
conn.commit()
conn.close()
@@ -126,21 +139,21 @@ def test_admin_without_email_blocked_fail_closed(env):
r = env.post('/auth/login', json={'username': 'bossuser', 'password': ADMIN_PW})
assert r.json()['message'] == '管理员账号未绑定邮箱,请联系管理员先完成绑定'
assert 'session' not in r.cookies
conn = auth.get_conn()
conn = _connect()
assert ('fail', '未绑定邮箱,两步验证不可用') in _logs(conn, '管理员登录')
conn.close()
def test_geo_ban_bumps_seq_again(env, monkeypatch):
"""异地多省封禁:login_seq 再递增一次(已签发中的会话全部顶下线)。"""
def _banned(user, ip):
async def _banned(user, ip):
return True, {'province': 'X'}
monkeypatch.setattr(auth, 'record_login_location', _banned)
r = env.post('/auth/login', json={'username': 'mailuser', 'password': USER_PW})
assert r.json()['message'] == '检测到账号在多个省份异常登录,已自动封禁,请联系管理员'
conn = auth.get_conn()
conn = _connect()
seq = conn.execute("SELECT login_seq FROM users WHERE username='mailuser'").fetchone()[0]
conn.close()
assert seq == 2 # 会话递增 + 封禁再递增
@@ -148,7 +161,7 @@ def test_geo_ban_bumps_seq_again(env, monkeypatch):
def test_unexpected_error_is_500(env, monkeypatch):
"""非预期异常:泛化文案 + HTTP 500。"""
def _boom(*a, **k):
async def _boom(*a, **k):
raise RuntimeError('db busy')
monkeypatch.setattr(auth, 'verify_login', _boom)
+112
View File
@@ -0,0 +1,112 @@
# -*- coding: utf-8 -*-
"""SMTP 发送路径回归:钉住「端口 → TLS 方式」的映射。
为什么值得单独钉:587/25 上必须先 STARTTLS 再 LOGIN,否则 SMTP 授权码会明文过网;
465 则是连上即握手(隐式 TLS)。这两种映射一旦被改反,除了本文件之外没有任何地方
会报警——而它是「凭据是否明文出网」级别的差异。
本测试用假 SMTP 替身记录构造参数与调用顺序,全程不触网(也就不会因外部 SMTP
不可达而失败)。
"""
import ssl
from typing import Any
import pytest
from auth_system import mail
class _FakeSMTP:
"""只记录行为、不做任何网络动作的 aiosmtplib.SMTP 替身。"""
instances: list['_FakeSMTP'] = []
def __init__(self, **kwargs: Any) -> None:
self.kwargs: dict[str, Any] = kwargs
self.calls: list[Any] = []
_FakeSMTP.instances.append(self)
async def connect(self) -> None:
self.calls.append('connect')
async def login(self, username: str, password: str) -> None:
self.calls.append(('login', username, password))
async def sendmail(self, sender: str, recipients: list[str], message: str) -> None:
self.calls.append(('sendmail', sender, tuple(recipients), message))
async def quit(self) -> None:
self.calls.append('quit')
@pytest.fixture()
def fake_smtp(monkeypatch: pytest.MonkeyPatch) -> type[_FakeSMTP]:
_FakeSMTP.instances.clear()
monkeypatch.setattr(mail.aiosmtplib, 'SMTP', _FakeSMTP)
return _FakeSMTP
CFG: dict[str, Any] = {
'host': 'smtp.example.com', 'port': 587,
'sender': 'a@example.com', 'password': 'pwd',
}
def _step_name(call: Any) -> str:
return call if isinstance(call, str) else call[0]
async def test_port_587_uses_starttls_before_login(fake_smtp: type[_FakeSMTP]) -> None:
"""587:显式 STARTTLS,且必须握手完成之后才发 LOGIN(凭据不得先出网)。"""
await mail._smtp_send(CFG, CFG['sender'], 'to@example.com', 'Raw: x')
(s,) = fake_smtp.instances
assert s.kwargs['port'] == 587
assert s.kwargs['use_tls'] is False
assert s.kwargs['start_tls'] is True
# 显式构造的 TLS 上下文——不依赖 aiosmtplib / 运行时的默认校验口径
assert isinstance(s.kwargs['tls_context'], ssl.SSLContext)
assert s.kwargs['timeout'] == 15
assert [_step_name(c) for c in s.calls] == ['connect', 'login', 'sendmail', 'quit']
async def test_port_465_uses_implicit_tls(fake_smtp: type[_FakeSMTP]) -> None:
"""465:隐式 TLS(连上即握手),不能再叠一次 STARTTLS。"""
cfg = dict(CFG, port=465)
await mail._smtp_send(cfg, cfg['sender'], 'to@example.com', 'Raw: x')
(s,) = fake_smtp.instances
assert s.kwargs['port'] == 465
assert s.kwargs['use_tls'] is True
assert s.kwargs['start_tls'] is False
async def test_missing_port_falls_back_to_587(fake_smtp: type[_FakeSMTP]) -> None:
"""端口缺省/为空时回落到 587,并仍走 STARTTLS 分支。"""
cfg = dict(CFG, port='')
await mail._smtp_send(cfg, cfg['sender'], 'to@example.com', 'Raw: x')
(s,) = fake_smtp.instances
assert s.kwargs['port'] == 587
assert s.kwargs['start_tls'] is True
async def test_quit_failure_does_not_mask_send_result(
fake_smtp: type[_FakeSMTP], monkeypatch: pytest.MonkeyPatch) -> None:
"""关连接失败不影响「信已发出」的结论,更不得盖掉真实异常。"""
async def _boom(self: _FakeSMTP) -> None:
raise OSError('connection already closed')
monkeypatch.setattr(_FakeSMTP, 'quit', _boom)
await mail._smtp_send(CFG, CFG['sender'], 'to@example.com', 'Raw: x') # 不应抛出
async def test_send_failure_propagates_and_still_quits(
fake_smtp: type[_FakeSMTP], monkeypatch: pytest.MonkeyPatch) -> None:
"""发信失败必须原样上抛(send_verify_code 依赖它决定「不写库不入额度」)。"""
async def _boom(self: _FakeSMTP, sender: str, recipients: list[str], message: str) -> None:
raise RuntimeError('smtp rejected')
monkeypatch.setattr(_FakeSMTP, 'sendmail', _boom)
with pytest.raises(RuntimeError, match='smtp rejected'):
await mail._smtp_send(CFG, CFG['sender'], 'to@example.com', 'Raw: x')
(s,) = fake_smtp.instances
assert 'quit' in [_step_name(c) for c in s.calls] # finally 里仍关连接
+52 -32
View File
@@ -13,6 +13,7 @@
账本口径同 tests/test_zip_workers.py:一律走 ``reserve_and_open`` 真预扣。
"""
import asyncio
import os
import threading
import time
@@ -71,8 +72,8 @@ def sandbox(tmp_path, monkeypatch):
monkeypatch.setattr(auth.db, 'SECRET_KEY_PATH', tmp_path / 'secret.key')
monkeypatch.setattr(core_db, 'DATA_DIR', tmp_path)
monkeypatch.setattr(core_db, 'DB_PATH', db_path)
auth.init_db()
auth.create_user('payer', PW, email='p@example.com', credits=CREDITS)
asyncio.run(auth.init_db())
asyncio.run(auth.create_user('payer', PW, email='p@example.com', credits=CREDITS))
downloads = tmp_path / 'downloads'
downloads.mkdir()
@@ -180,23 +181,41 @@ def test_reap_retries_then_gives_up(sandbox):
# ============ ② 超时任务取消 ============
def _finished_thread() -> threading.Thread:
"""已结束的线程替身——对应生产里 worker 崩溃/退出的情形(is_alive() 为 False)。"""
th = threading.Thread(target=lambda: None)
th.start()
th.join()
return th
class _WorkerHandle:
"""worker 句柄替身——生产里是 asyncio.Task,监控只看它的 ``done()``。
这里用轻量替身而不是真造一个已结束的 Task:Task 的构造依赖运行中的事件循环,
而监控循环跑在常驻线程里、与任务所在的循环无关;本文件测的是「句柄状态 →
是否代结算」这段判断,只要 done() 可控即可。
"""
def __init__(self, done: bool) -> None:
self._done = done
def done(self) -> bool:
return self._done
def _finished_worker() -> _WorkerHandle:
"""已结束的 worker 替身——对应生产里 worker 崩溃/退出的情形(done() 为 True)。"""
return _WorkerHandle(True)
def _live_worker() -> _WorkerHandle:
return _WorkerHandle(False)
def test_cancel_overtime_settles_dead_worker(sandbox, monkeypatch):
"""超硬上限且 worker 线程已死:置取消、唤醒 cancel Event、代为退款。"""
"""超硬上限且 worker 已结束:置取消、唤醒 cancel Event、代为退款。"""
settled: list[tuple] = []
monkeypatch.setattr(auth, 'ledger_settle',
lambda tid, ok, st: settled.append((tid, ok, st)))
async def _settle(tid, ok, st):
settled.append((tid, ok, st))
monkeypatch.setattr(auth, 'ledger_settle', _settle)
task = _add_task('t-slow', status='running',
start=time.time() - _HARD_TIMEOUT - 60,
worker_thread=_finished_thread())
worker_task=_finished_worker())
services_tasks._cancel_overtime_tasks(time.time())
assert task['status'] == 'canceled'
@@ -204,12 +223,14 @@ def test_cancel_overtime_settles_dead_worker(sandbox, monkeypatch):
assert settled == [('t-slow', 0, 'timeout')]
def test_cancel_overtime_without_worker_thread_does_not_settle(sandbox, monkeypatch):
"""无 worker_thread 字段的任务不代结算——代结算的前提是「线程曾存在且已死」,
没有线程句柄就无法判断 worker 是否已在别处结算过,贸然退款会双退。"""
def test_cancel_overtime_without_worker_task_does_not_settle(sandbox, monkeypatch):
"""无 worker_task 字段的任务不代结算——代结算的前提是「句柄曾存在且已结束」,
没有句柄就无法判断 worker 是否已在别处结算过,贸然退款会双退。"""
settled: list[tuple] = []
monkeypatch.setattr(auth, 'ledger_settle',
lambda tid, ok, st: settled.append((tid, ok, st)))
async def _settle(tid, ok, st):
settled.append((tid, ok, st))
monkeypatch.setattr(auth, 'ledger_settle', _settle)
task = _add_task('t-noth', status='queued',
start=time.time() - _HARD_TIMEOUT - 60)
@@ -222,21 +243,16 @@ def test_cancel_overtime_without_worker_thread_does_not_settle(sandbox, monkeypa
def test_cancel_overtime_skips_live_worker(sandbox, monkeypatch):
"""worker 仍在跑:只置取消,不代结算(交 worker 自己收敛,避免双退)。"""
settled: list[tuple] = []
monkeypatch.setattr(auth, 'ledger_settle',
lambda tid, ok, st: settled.append((tid, ok, st)))
async def _settle(tid, ok, st):
settled.append((tid, ok, st))
monkeypatch.setattr(auth, 'ledger_settle', _settle)
stop = threading.Event()
alive = threading.Thread(target=stop.wait, daemon=True)
alive.start()
try:
task = _add_task('t-live', status='running',
start=time.time() - _HARD_TIMEOUT - 60, worker_thread=alive)
start=time.time() - _HARD_TIMEOUT - 60, worker_task=_live_worker())
services_tasks._cancel_overtime_tasks(time.time())
assert task['status'] == 'canceled'
assert settled == [], 'worker 活着 → 不应代结算'
finally:
stop.set()
alive.join(timeout=5)
def test_cancel_overtime_ignores_recent(sandbox):
@@ -251,10 +267,14 @@ def test_cancel_overtime_ignores_recent(sandbox):
def test_ledger_jobs_respect_intervals(sandbox, monkeypatch):
"""时间戳门槛:到期执行一次并推进门槛,未到期不重复执行。"""
calls: list[str] = []
monkeypatch.setattr(auth, 'ledger_recover_orphans',
lambda max_age_seconds=0: calls.append(f'recover:{max_age_seconds}'))
monkeypatch.setattr(auth, 'ledger_cleanup_settled',
lambda days: calls.append(f'cleanup:{days}'))
async def _recover(max_age_seconds=0):
calls.append(f'recover:{max_age_seconds}')
async def _cleanup(days):
calls.append(f'cleanup:{days}')
monkeypatch.setattr(auth, 'ledger_recover_orphans', _recover)
monkeypatch.setattr(auth, 'ledger_cleanup_settled', _cleanup)
now = time.time()
services_tasks._run_periodic_ledger_jobs(now)
@@ -272,7 +292,7 @@ def test_ledger_jobs_respect_intervals(sandbox, monkeypatch):
def test_ledger_jobs_swallow_exception(sandbox, monkeypatch):
"""账本作业抛错由自身吞掉(不能让本轮其余作业一起落空)。"""
def boom(max_age_seconds=0):
async def boom(max_age_seconds=0):
raise RuntimeError('库忙')
monkeypatch.setattr(auth, 'ledger_recover_orphans', boom)
+10 -15
View File
@@ -9,14 +9,15 @@
- trackIds 数与实得数不一致时显式标 incomplete/missing
按惯例:先在未改动的原始实现上跑绿,重构时这批测试即等价性验证器。
替身边界:monkeypatch ``music_api.api.requests.post``,按 URL 分流——
PLAYLIST_DETAIL_API 与 SONG_DETAIL_V3 分别给响应替身。
替身边界:monkeypatch ``music_api.api._http.request``(core.http_client 的进程级
客户端),按 URL 分流——PLAYLIST_DETAIL_API 与 SONG_DETAIL_V3 分别给响应替身。
"""
import json
from typing import Any
import pytest
from core.http_client import UpstreamNetworkError
from music_api import APIException
from music_api import api as api_mod
from music_api.api import NeteaseAPI
@@ -26,19 +27,13 @@ _API = NeteaseAPI()
class _FakeResp:
"""``requests.post`` 响应替身:json 载荷或注入异常。"""
"""请求响应替身(``HTTPResult`` 的形状子集):json 载荷或注入异常。"""
def __init__(self, payload: Any = None, *,
status_error: Exception | None = None,
json_error: Exception | None = None):
self.payload = payload
self.status_error = status_error
self.json_error = json_error
def raise_for_status(self) -> None:
if self.status_error is not None:
raise self.status_error
def json(self) -> Any:
if self.json_error is not None:
raise self.json_error
@@ -47,14 +42,14 @@ class _FakeResp:
@pytest.fixture()
def api_capture(monkeypatch):
"""按 URL 分流的 requests.post 替身:歌单详情响应固定,批次响应按序消费。"""
"""按 URL 分流的请求替身:歌单详情响应固定,批次响应按序消费。"""
calls: list[dict[str, Any]] = []
state: dict[str, Any] = {
'playlist_resp': _FakeResp({'code': 200, 'playlist': {}}),
'batch_resps': [_FakeResp({'code': 200, 'songs': []})],
}
async def _fake_post(url: str, **kw: Any):
async def _fake_request(method: str, url: str, **kw: Any):
calls.append({'url': url, 'data': kw.get('data')})
if url == APIConstants.PLAYLIST_DETAIL_API:
return state['playlist_resp']
@@ -63,7 +58,7 @@ def api_capture(monkeypatch):
raise AssertionError('批次响应耗尽')
return state['batch_resps'].pop(0)
monkeypatch.setattr(api_mod.requests, 'post', _fake_post)
monkeypatch.setattr(api_mod._http, 'request', _fake_request)
state['calls'] = calls
return state
@@ -197,10 +192,10 @@ async def test_batches_of_100(api_capture):
async def test_network_error_maps_to_request_failed(api_capture, monkeypatch):
"""网络层异常(含批次阶段的):统一转「获取歌单详情请求失败」。"""
async def _boom(url: str, **kw: Any):
raise api_mod.requests.RequestException('connection reset')
async def _boom(method: str, url: str, **kw: Any):
raise UpstreamNetworkError('connection reset')
monkeypatch.setattr(api_mod.requests, 'post', _boom)
monkeypatch.setattr(api_mod._http, 'request', _boom)
with pytest.raises(APIException, match='获取歌单详情请求失败'):
await _API.get_playlist_detail(123, {})
+52 -37
View File
@@ -8,6 +8,9 @@
断言只看「调用后世界里发生了什么」(文件系统 + 任务表),不引用私有函数名,
故可跨后续重构复用。
替身边界:monkeypatch ``services.podcast._http``(core.http_client 的进程级异步
客户端),测试用 ``holder['resp']`` 指定本次响应。
"""
import threading
@@ -18,8 +21,19 @@ from music_downloader import DownloadCanceled, DownloadException
from services import podcast as podcast_svc
class _FakeContent:
"""``aiohttp.ClientResponse.content`` 的替身(逐块产出,惰性求值)。"""
def __init__(self, chunks):
self._chunks = chunks
async def iter_chunked(self, chunk_size=0):
for item in self._chunks:
yield item
class _FakeResp:
"""`_stream_open` 的替身:可控状态码、Content-Length 与分块序列。
"""``core.http_client.open`` 的替身:可控状态码、Content-Length 与分块序列。
注意不能用 list(chunks) 归一化——那会把传入的生成器**立即求值**,使「迭代中
才触发的副作用」(如中途置取消位)提前到构造时刻,测出来的就不是想测的路径。
@@ -28,16 +42,13 @@ class _FakeResp:
def __init__(self, chunks, status=200, content_length=None):
self._chunks = chunks
self.status = status
self.reason = ''
self.headers = {} if content_length is None else {'Content-Length': str(content_length)}
self.content = _FakeContent(chunks)
self.closed = False
def __enter__(self):
return self
def __exit__(self, *exc):
return False
def iter_content(self, chunk_size=0):
yield from self._chunks
def close(self):
self.closed = True
class _CancelAfterFirstChunk:
@@ -66,14 +77,15 @@ class _CancelAfterFirstChunk:
@pytest.fixture()
def fake_stream(monkeypatch):
"""替换 _stream_open;测试通过 holder['resp'] 指定本次响应。"""
"""替换 podcast 模块的 _http 客户端;测试通过 holder['resp'] 指定本次响应。"""
holder: dict = {}
def _open(url, timeout=60):
class _Client:
async def open(self, method, url, **kw):
assert url == holder.get('url', url)
return holder['resp']
monkeypatch.setattr(podcast_svc, '_stream_open', _open)
monkeypatch.setattr(podcast_svc, '_http', _Client())
return holder
@@ -97,91 +109,93 @@ def task_table():
# ============ 正常路径 ============
def test_writes_target_and_renames_part(tmp_path, fake_stream):
async def test_writes_target_and_renames_part(tmp_path, fake_stream):
"""正常下载:返回实际字节数,落地 target,不残留 .part。"""
fake_stream['resp'] = _FakeResp([b'a' * 600, b'b' * 400])
target = tmp_path / 'ep.mp3'
done = podcast_svc._podcast_stream_download('http://u', target, 1000, threading.Event())
done = await podcast_svc._podcast_stream_download('http://u', target, 1000, threading.Event())
assert done == 1000
assert target.read_bytes() == b'a' * 600 + b'b' * 400
assert not target.with_name('ep.mp3.part').exists()
def test_empty_chunks_are_skipped(tmp_path, fake_stream):
async def test_empty_chunks_are_skipped(tmp_path, fake_stream):
"""空分块不计入字节数(上游偶发发空 chunk)。"""
fake_stream['resp'] = _FakeResp([b'', b'x' * 50, b''])
target = tmp_path / 'ep.mp3'
assert podcast_svc._podcast_stream_download('http://u', target, 50, threading.Event()) == 50
assert await podcast_svc._podcast_stream_download(
'http://u', target, 50, threading.Event()) == 50
def test_content_length_used_when_total_unknown(tmp_path, fake_stream):
async def test_content_length_used_when_total_unknown(tmp_path, fake_stream):
"""未传期望大小时用 Content-Length 兜底,并据此校验完整性。"""
fake_stream['resp'] = _FakeResp([b'x' * 30], content_length=30)
target = tmp_path / 'ep.mp3'
assert podcast_svc._podcast_stream_download('http://u', target, 0, threading.Event()) == 30
assert await podcast_svc._podcast_stream_download(
'http://u', target, 0, threading.Event()) == 30
# ============ 完整性校验 ============
def test_size_mismatch_removes_delivered_file(tmp_path, fake_stream):
async def test_size_mismatch_removes_delivered_file(tmp_path, fake_stream):
"""字节数不符:删掉已落地的文件并报错(不交付截断的坏文件)。"""
fake_stream['resp'] = _FakeResp([b'x' * 100])
target = tmp_path / 'ep.mp3'
with pytest.raises(DownloadException) as ei:
podcast_svc._podcast_stream_download('http://u', target, 200, threading.Event())
await podcast_svc._podcast_stream_download('http://u', target, 200, threading.Event())
assert '下载不完整' in str(ei.value)
assert not target.exists(), '坏文件必须被清掉'
def test_empty_download_raises_and_removes(tmp_path, fake_stream):
async def test_empty_download_raises_and_removes(tmp_path, fake_stream):
"""期望大小未知且一个字节都没拿到 → 报「下载内容为空」。"""
fake_stream['resp'] = _FakeResp([])
target = tmp_path / 'ep.mp3'
with pytest.raises(DownloadException) as ei:
podcast_svc._podcast_stream_download('http://u', target, 0, threading.Event())
await podcast_svc._podcast_stream_download('http://u', target, 0, threading.Event())
assert '下载内容为空' in str(ei.value)
assert not target.exists()
def test_known_size_does_not_tolerate_partial(tmp_path, fake_stream):
async def test_known_size_does_not_tolerate_partial(tmp_path, fake_stream):
"""已知期望大小时要求精确相等——95% 容差会放行截断的坏文件。"""
fake_stream['resp'] = _FakeResp([b'x' * 95])
target = tmp_path / 'ep.mp3'
with pytest.raises(DownloadException):
podcast_svc._podcast_stream_download('http://u', target, 100, threading.Event())
await podcast_svc._podcast_stream_download('http://u', target, 100, threading.Event())
# ============ 上游与取消 ============
def test_upstream_error_status_raises(tmp_path, fake_stream):
async def test_upstream_error_status_raises(tmp_path, fake_stream):
fake_stream['resp'] = _FakeResp([], status=404)
target = tmp_path / 'ep.mp3'
with pytest.raises(DownloadException) as ei:
podcast_svc._podcast_stream_download('http://u', target, 100, threading.Event())
await podcast_svc._podcast_stream_download('http://u', target, 100, threading.Event())
assert '上游返回 404' in str(ei.value)
assert not target.exists()
def test_cancel_before_first_chunk(tmp_path, fake_stream):
async def test_cancel_before_first_chunk(tmp_path, fake_stream):
"""取消已置位:抛 DownloadCanceled,不发布 target。"""
cancel = threading.Event()
cancel.set()
fake_stream['resp'] = _FakeResp([b'x' * 10])
target = tmp_path / 'ep.mp3'
with pytest.raises(DownloadCanceled):
podcast_svc._podcast_stream_download('http://u', target, 10, cancel)
await podcast_svc._podcast_stream_download('http://u', target, 10, cancel)
assert not target.exists()
def test_cancel_midway_leaves_part_file(tmp_path, fake_stream):
async def test_cancel_midway_leaves_part_file(tmp_path, fake_stream):
"""下载中途取消:.part 残留(不删)——改名不发生,下次同名下载直接覆盖。"""
cancel = threading.Event()
fake_stream['resp'] = _FakeResp(_CancelAfterFirstChunk(cancel))
target = tmp_path / 'ep.mp3'
with pytest.raises(DownloadCanceled):
podcast_svc._podcast_stream_download('http://u', target, 20, cancel)
await podcast_svc._podcast_stream_download('http://u', target, 20, cancel)
assert not target.exists()
assert target.with_name('ep.mp3.part').exists(), '既有行为:取消时 .part 留存'
@@ -189,7 +203,7 @@ def test_cancel_midway_leaves_part_file(tmp_path, fake_stream):
# ============ 进度消息 ============
def test_progress_written_for_running_single_task(tmp_path, fake_stream, task_table):
async def test_progress_written_for_running_single_task(tmp_path, fake_stream, task_table):
"""running 的单集任务:写 msg,并按字节级口径写 live_percent。
单块即完成 → 百分比被 min(99, ...) 封在 99(进度条不预支 100%,收尾靠终态)。
@@ -197,24 +211,25 @@ def test_progress_written_for_running_single_task(tmp_path, fake_stream, task_ta
task = task_table()
fake_stream['resp'] = _FakeResp([b'x' * 100])
target = tmp_path / 'ep.mp3'
podcast_svc._podcast_stream_download('http://u', target, 100, threading.Event(),
await podcast_svc._podcast_stream_download('http://u', target, 100, threading.Event(),
task_id=task['id'], progress_label='下载中')
assert '下载中' in task['msg'] and '99%' in task['msg']
assert task['live_percent'] == 99.0
def test_progress_not_written_for_finished_task(tmp_path, fake_stream, task_table):
async def test_progress_not_written_for_finished_task(tmp_path, fake_stream, task_table):
"""已终态的任务不再被写进度(避免覆盖终态消息),但下载照常完成。"""
task = task_table(status='done', msg='原消息')
fake_stream['resp'] = _FakeResp([b'x' * 50])
target = tmp_path / 'ep.mp3'
assert podcast_svc._podcast_stream_download('http://u', target, 50, threading.Event(),
task_id=task['id']) == 50
assert await podcast_svc._podcast_stream_download(
'http://u', target, 50, threading.Event(), task_id=task['id']) == 50
assert task['msg'] == '原消息'
def test_progress_skipped_without_task_id(tmp_path, fake_stream):
async def test_progress_skipped_without_task_id(tmp_path, fake_stream):
"""task_id 为空时只下载不写任务表(不得因缺 id 而报错)。"""
fake_stream['resp'] = _FakeResp([b'x' * 50])
target = tmp_path / 'ep.mp3'
assert podcast_svc._podcast_stream_download('http://u', target, 50, threading.Event()) == 50
assert await podcast_svc._podcast_stream_download(
'http://u', target, 50, threading.Event()) == 50
+31 -20
View File
@@ -19,6 +19,7 @@ except 分支各写一遍「结算 + 删任务目录」,正文里散着三处
账本口径:一律用 ``reserve_and_open`` 真预扣(与 test_single_worker.py 同理由)。
"""
import asyncio
import threading
import time
import zipfile
@@ -83,7 +84,7 @@ class _FakeStream:
self.on_call = on_call
self.calls: list[dict[str, Any]] = []
def __call__(self, url: str, target: Path, total_size: int,
async def __call__(self, url: str, target: Path, total_size: int,
cancel_ev: threading.Event, task_id: str = '',
progress_label: str = '') -> int:
self.calls.append({'url': url, 'target': Path(target),
@@ -110,8 +111,8 @@ def sandbox(tmp_path, monkeypatch):
monkeypatch.setattr(auth.db, 'SECRET_KEY_PATH', tmp_path / 'secret.key')
monkeypatch.setattr(core_db, 'DATA_DIR', tmp_path)
monkeypatch.setattr(core_db, 'DB_PATH', db_path)
auth.init_db()
auth.create_user('payer', PW, email='p@example.com', credits=CREDITS)
asyncio.run(auth.init_db())
asyncio.run(auth.create_user('payer', PW, email='p@example.com', credits=CREDITS))
downloads = tmp_path / 'downloads'
downloads.mkdir()
@@ -130,13 +131,13 @@ def sandbox(tmp_path, monkeypatch):
# ============ 工具 ============
def _uid(username: str = 'payer') -> int:
u = auth.get_user_by_name(username)
u = asyncio.run(auth.get_user_by_name(username))
assert u is not None
return int(u['id'])
def _balance(username: str = 'payer') -> int:
u = auth.get_user_by_name(username)
u = asyncio.run(auth.get_user_by_name(username))
assert u is not None
return int(u['duration_seconds'] or 0)
@@ -149,7 +150,7 @@ def _status(task_id: str) -> str:
def _register_podcast(sandbox, *, program_id: int = 777, user_id: int,
reserved: int = 1, **params: Any) -> tuple[str, dict[str, Any]]:
tid = 'p' + str(int(time.time() * 1000000) % 10 ** 10)
left = auth.reserve_and_open(user_id, reserved, tid)
left = asyncio.run(auth.reserve_and_open(user_id, reserved, tid))
assert left is not None, '预扣失败(余额不足,检查 fixture 的 credits)'
task: dict[str, Any] = {
'type': 'single', 'task_id': tid,
@@ -172,13 +173,20 @@ def _register_podcast(sandbox, *, program_id: int = 777, user_id: int,
def _install_upstream(monkeypatch, *, program: dict[str, Any] | None = None,
url_info: dict[str, Any] | None = None,
stream: Any = None,
cover: Any = lambda url: b'\xff\xd8\xff cover') -> None:
cover: Any = None) -> None:
"""装上播客上游替身(节目详情 / 直链 / 流式下载 / 封面)。"""
monkeypatch.setattr(podcast_svc.podcast, 'get_program_detail',
lambda pid, cookies: program if program is not None else _program())
monkeypatch.setattr(podcast_svc.podcast, 'resolve_episode_url',
lambda sid, cookies, quality:
url_info if url_info is not None else _url_info())
async def _detail(pid: Any, cookies: Any) -> dict[str, Any]:
return program if program is not None else _program()
async def _resolve(sid: Any, cookies: Any, quality: str) -> dict[str, Any]:
return url_info if url_info is not None else _url_info()
monkeypatch.setattr(podcast_svc.podcast, 'get_program_detail', _detail)
monkeypatch.setattr(podcast_svc.podcast, 'resolve_episode_url', _resolve)
if cover is None:
async def _default_cover(_url: str) -> bytes:
return b'\xff\xd8\xff cover'
cover = _default_cover
if stream is None:
stream = _FakeStream()
monkeypatch.setattr(podcast_svc, '_podcast_stream_download', stream)
@@ -186,8 +194,11 @@ def _install_upstream(monkeypatch, *, program: dict[str, Any] | None = None,
def _run(task_id: str) -> threading.Thread:
th = threading.Thread(target=podcast_svc._podcast_download_worker,
args=(task_id,), daemon=True)
"""把 worker 协程跑在独立线程的私有事件循环里(等价于生产的任务循环)。"""
def _target() -> None:
asyncio.run(podcast_svc._podcast_download_worker(task_id))
th = threading.Thread(target=_target, daemon=True)
th.start()
return th
@@ -227,7 +238,7 @@ def test_podcast_download_completes(sandbox, monkeypatch):
assert delivered.parent == sandbox['downloads'] / 'tasks' / tid
assert delivered.read_bytes() == b'audio-bytes'
row = auth.ledger_get(tid)
row = asyncio.run(auth.ledger_get(tid))
assert row is not None and row['status'] != 'open'
assert row['delivered'] == 1
assert _balance() == before - 1, '交付成功:只扣预扣的 1 次,无退款'
@@ -332,7 +343,7 @@ def test_cover_fetch_failure_still_delivers_zip(sandbox, monkeypatch):
"""取封面抛异常:被吞掉,ZIP 照常交付(历史行为:封面失败不炸任务)。"""
uid = _uid()
def _boom(url: str) -> bytes:
async def _boom(url: str) -> bytes:
raise RuntimeError('cover cdn down')
_install_upstream(monkeypatch, cover=_boom)
@@ -361,7 +372,7 @@ def test_missing_song_id_marks_error_and_refunds(sandbox, monkeypatch):
assert task['msg'] == '该单集缺少音频信息(mainSong)'
assert task['result'] is None
assert _balance() == before, '失败应全额退回预扣'
row = auth.ledger_get(tid)
row = asyncio.run(auth.ledger_get(tid))
assert row is not None and row['delivered'] == 0 and row['status'] != 'open'
assert not (sandbox['downloads'] / 'tasks' / tid).exists(), '失败应清掉任务目录'
@@ -402,7 +413,7 @@ def test_unexpected_exception_uses_generic_message(sandbox, monkeypatch):
uid = _uid()
before = _balance()
def _boom(pid: Any, cookies: Any) -> dict[str, Any]:
async def _boom(pid: Any, cookies: Any) -> dict[str, Any]:
raise RuntimeError('upstream host=10.0.0.9 refused')
_install_upstream(monkeypatch)
@@ -468,7 +479,7 @@ def test_cancel_during_zip_stage(sandbox, monkeypatch):
before = _balance()
holder: dict[str, Any] = {}
def _cancel_in_cover(url: str) -> bytes:
async def _cancel_in_cover(url: str) -> bytes:
holder['task']['cancel'].set()
return b'\xff\xd8\xff cover'
@@ -545,6 +556,6 @@ def test_ledger_open_when_worker_crashes_before_settle(sandbox, monkeypatch):
th.join(timeout=30)
assert _status(tid) == 'error'
row = auth.ledger_get(tid)
row = asyncio.run(auth.ledger_get(tid))
assert row is not None
assert row['status'] == 'open', f"应留给对账兜底,实际 status={row['status']}"
+20 -7
View File
@@ -8,6 +8,8 @@
- 找回密码:重置后可用新密码登录、旧会话失效
- 卡密充值
"""
import asyncio
import sqlite3
import time
import pytest
@@ -19,6 +21,17 @@ import core.db as db
SECRET = 'register-test-secret'
async def _fake_secret_key() -> str:
return SECRET
def _connect() -> sqlite3.Connection:
"""直连沙箱库做只读/播种(auth.get_conn 已异步化,测试用裸连接更直接)。"""
conn = sqlite3.connect(str(auth.db.DB_PATH))
conn.row_factory = sqlite3.Row
return conn
@pytest.fixture()
def env(tmp_path, monkeypatch):
db_path = tmp_path / 'site.db'
@@ -28,8 +41,8 @@ def env(tmp_path, monkeypatch):
monkeypatch.setattr(auth.db, 'SECRET_KEY_PATH', tmp_path / 'secret.key')
monkeypatch.setattr(db, 'DATA_DIR', tmp_path)
monkeypatch.setattr(db, 'DB_PATH', db_path)
auth.init_db()
monkeypatch.setattr('web.deps.secret_key', lambda: SECRET)
asyncio.run(auth.init_db())
monkeypatch.setattr('web.deps.secret_key', _fake_secret_key)
import web.auth_state as auth_routes
auth_routes._SEND_CODE_IP.clear()
from web.app import create_app
@@ -37,7 +50,7 @@ def env(tmp_path, monkeypatch):
def _seed_code(email: str, code: str, purpose: str = 'register'):
conn = auth.get_conn()
conn = _connect()
conn.execute('DELETE FROM email_codes WHERE email = ? AND purpose = ?', (email, purpose))
conn.execute('INSERT INTO email_codes (email, code, expires_at, created_at, purpose) '
'VALUES (?, ?, ?, ?, ?)',
@@ -84,7 +97,7 @@ def test_register_bad_code(env):
def test_forgot_password_resets(env):
# 建一个用户
auth.create_user('forgetme', 'OldPass123', email='f@example.com', credits=5)
asyncio.run(auth.create_user('forgetme', 'OldPass123', email='f@example.com', credits=5))
_seed_code('f@example.com', '654321', purpose='reset')
r = env.post('/auth/forgot_password', json={
'email': 'f@example.com', 'code': '654321', 'password': 'NewPass456'})
@@ -96,7 +109,7 @@ def test_forgot_password_resets(env):
def test_forgot_password_admin_blocked(env):
conn = auth.get_conn()
conn = _connect()
conn.execute("UPDATE users SET role='admin', email='boss@example.com' WHERE id=1")
conn.commit()
conn.close()
@@ -109,8 +122,8 @@ def test_forgot_password_admin_blocked(env):
def test_redeem_card(env):
# 造一张卡密并让用户登录
auth.create_user('buyer', 'Passw0rd12', email='b@example.com', credits=0)
code = auth.generate_cards(1, 10, note='test')[0]
asyncio.run(auth.create_user('buyer', 'Passw0rd12', email='b@example.com', credits=0))
code = asyncio.run(auth.generate_cards(1, 10, note='test'))[0]
env.post('/auth/login', json={'username': 'buyer', 'password': 'Passw0rd12'})
r = env.post('/auth/redeem', json={'code': code})
assert r.status_code == 200
+28 -10
View File
@@ -7,6 +7,7 @@ create_user 的三种异常映射,作为 register 拆解的等价性验证器
验证码由 check_verify_code 以 DELETE 原子消费:失败路径后 email_codes 行
仍然存在 ⇔ restore_verify_code 确实被调用。以此做实证断言,不 mock 还原。
"""
import asyncio
import sqlite3
import time
@@ -19,6 +20,17 @@ import core.db as db
SECRET = 'register-flow-test-secret'
async def _fake_secret_key() -> str:
return SECRET
def _connect() -> sqlite3.Connection:
"""直连沙箱库做只读/播种(auth.get_conn 已异步化,测试用裸连接更直接)。"""
conn = sqlite3.connect(str(auth.db.DB_PATH))
conn.row_factory = sqlite3.Row
return conn
@pytest.fixture()
def env(tmp_path, monkeypatch):
db_path = tmp_path / 'site.db'
@@ -28,8 +40,8 @@ def env(tmp_path, monkeypatch):
monkeypatch.setattr(auth.db, 'SECRET_KEY_PATH', tmp_path / 'secret.key')
monkeypatch.setattr(db, 'DATA_DIR', tmp_path)
monkeypatch.setattr(db, 'DB_PATH', db_path)
auth.init_db()
monkeypatch.setattr('web.deps.secret_key', lambda: SECRET)
asyncio.run(auth.init_db())
monkeypatch.setattr('web.deps.secret_key', _fake_secret_key)
import web.auth_state as auth_state
auth_state._SEND_CODE_IP.clear()
from web.app import create_app
@@ -37,7 +49,7 @@ def env(tmp_path, monkeypatch):
def _seed_code(email: str, code: str, purpose: str = 'register'):
conn = auth.get_conn()
conn = _connect()
conn.execute('DELETE FROM email_codes WHERE email = ? AND purpose = ?', (email, purpose))
conn.execute('INSERT INTO email_codes (email, code, expires_at, created_at, purpose) '
'VALUES (?, ?, ?, ?, ?)',
@@ -47,7 +59,7 @@ def _seed_code(email: str, code: str, purpose: str = 'register'):
def _code_rows(email: str, purpose: str = 'register') -> list[str]:
conn = auth.get_conn()
conn = _connect()
rows = conn.execute('SELECT code FROM email_codes WHERE email = ? AND purpose = ?',
(email, purpose)).fetchall()
conn.close()
@@ -55,7 +67,7 @@ def _code_rows(email: str, purpose: str = 'register') -> list[str]:
def _register_logs() -> list[tuple[str, str]]:
conn = auth.get_conn()
conn = _connect()
rows = conn.execute("SELECT type, detail FROM logs WHERE action = '注册'").fetchall()
conn.close()
return [(r['type'], r['detail']) for r in rows]
@@ -86,7 +98,7 @@ def test_register_input_validation_order(env):
def test_dup_username_restores_code_end_to_end(env):
"""用户名重名拒绝后,同一验证码无需重发即可再次使用(restore 实证)。"""
auth.create_user('taken', 'Passw0rd12')
asyncio.run(auth.create_user('taken', 'Passw0rd12'))
_seed_code('x@example.com', '123456')
r = _post(env, 'taken', 'x@example.com', '123456')
@@ -98,7 +110,7 @@ def test_dup_username_restores_code_end_to_end(env):
def test_dup_email_keeps_code_row(env):
"""邮箱重名拒绝后验证码行仍在(被 check_verify_code 消费后又还原)。"""
auth.create_user('holder', 'Passw0rd12', email='y@example.com')
asyncio.run(auth.create_user('holder', 'Passw0rd12', email='y@example.com'))
_seed_code('y@example.com', '111111')
r = _post(env, 'maildup', 'y@example.com', '111111')
@@ -108,7 +120,10 @@ def test_dup_email_keeps_code_row(env):
def test_ip_exceeded_restores_code_and_logs(env, monkeypatch):
"""IP 闸触发:还原验证码 + 写失败日志 + 专属文案。"""
monkeypatch.setattr(auth, 'register_ip_exceeded', lambda ip, **k: True)
async def _exceeded(ip, **k):
return True
monkeypatch.setattr(auth, 'register_ip_exceeded', _exceeded)
_seed_code('ip@example.com', '222222')
r = _post(env, 'ipuser', 'ip@example.com', '222222')
@@ -130,7 +145,7 @@ def test_device_used_grants_zero_credits(env):
assert r2.json()['success'] is True
assert r2.json()['message'] == '注册成功'
conn = auth.get_conn()
conn = _connect()
row = conn.execute("SELECT duration_seconds FROM users WHERE username = 'seconddev'").fetchone()
conn.close()
assert row['duration_seconds'] == 0
@@ -167,7 +182,10 @@ def test_valueerror_maps_and_restores(env, monkeypatch):
def test_user_none_maps_and_restores(env, monkeypatch):
"""建号事务提交却读不回:按注册失败处理 + 还原验证码 + 失败日志。"""
monkeypatch.setattr(auth, 'create_user', lambda *a, **k: None)
async def _none(*a, **k):
return None
monkeypatch.setattr(auth, 'create_user', _none)
_seed_code('none@example.com', '777777')
r = _post(env, 'noneuser', 'none@example.com', '777777')
+20 -15
View File
@@ -8,8 +8,9 @@
- 未知音质 → 该音质优先 + 通用阶梯
- song_id 垃圾值 → None 且不发任何请求
替身边界:monkeypatch ``podcast.url_v1``(模块级名字,podcast.py 从
.compat 导入后的绑定),按 level 返回结果或抛异常,并记录调用序列。
替身边界:monkeypatch ``music_api.podcast.NeteaseAPI``——resolve_episode_url 内部
现构造该客户端并 ``await api.get_song_url(...)``,替身按 level 返回结果或抛异常,
并记录调用序列。
"""
import pytest
@@ -23,14 +24,18 @@ def url_calls(monkeypatch):
calls: list[tuple[int, str]] = []
state: dict = {'results': {}}
def _fake_url_v1(sid, level, cookies):
async def _fake_get_song_url(sid, level, cookies):
calls.append((sid, level))
r = state['results'].get(level)
if isinstance(r, Exception):
raise r
return r
monkeypatch.setattr(podcast_mod, 'url_v1', _fake_url_v1)
class _FakeAPI:
async def get_song_url(self, sid, level, cookies):
return await _fake_get_song_url(sid, level, cookies)
monkeypatch.setattr(podcast_mod, 'NeteaseAPI', _FakeAPI)
state['calls'] = calls
return state
@@ -42,38 +47,38 @@ def _ok(level: str = 'standard') -> dict:
async def test_auto_tries_quality_ladder_in_order(url_calls):
"""auto:按 QUALITY_LADDER 原序尝试,全失败返回 None。"""
assert resolve_episode_url(101, {}, 'auto') is None
assert await resolve_episode_url(101, {}, 'auto') is None
assert url_calls['calls'] == [(101, 'exhigh'), (101, 'standard')]
async def test_explicit_level_falls_down_only(url_calls):
"""显式 lossless:向后降级 exhigh/standard,绝不向上尝试 hires。"""
assert resolve_episode_url(101, {}, 'lossless') is None
assert await resolve_episode_url(101, {}, 'lossless') is None
assert url_calls['calls'] == [(101, 'lossless'), (101, 'exhigh'), (101, 'standard')]
async def test_standard_never_falls_up(url_calls):
"""显式 standard:只试一次(历史坑:原实现会回退到更高的 exhigh)。"""
assert resolve_episode_url(101, {}, 'standard') is None
assert await resolve_episode_url(101, {}, 'standard') is None
assert url_calls['calls'] == [(101, 'standard')]
async def test_unknown_level_prepends_generic_ladder(url_calls):
"""未知音质:先试该音质,再退通用阶梯(去重保序)。"""
assert resolve_episode_url(101, {}, 'sky') is None
assert await resolve_episode_url(101, {}, 'sky') is None
assert url_calls['calls'] == [(101, 'sky'), (101, 'exhigh'), (101, 'standard')]
async def test_first_hit_returns_mapped_result(url_calls):
url_calls['results']['exhigh'] = _ok('hires')
out = resolve_episode_url(101, {}, 'auto')
out = await resolve_episode_url(101, {}, 'auto')
assert out == {'url': 'https://x/1.mp3', 'level': 'hires', 'type': 'mp3',
'size': 123, 'br': 128000, 'md5': 'abc', 'requested_level': 'exhigh'}
async def test_level_defaults_to_requested_when_missing(url_calls):
url_calls['results']['exhigh'] = {'data': [{'url': 'https://x/1.mp3'}]}
out = resolve_episode_url(101, {}, 'auto')
out = await resolve_episode_url(101, {}, 'auto')
assert out is not None
assert out['level'] == 'exhigh'
assert out['type'] == 'mp3'
@@ -84,7 +89,7 @@ async def test_level_defaults_to_requested_when_missing(url_calls):
async def test_api_exception_tries_next_level(url_calls):
url_calls['results']['exhigh'] = APIException('无版权')
url_calls['results']['standard'] = _ok()
out = resolve_episode_url(101, {}, 'auto')
out = await resolve_episode_url(101, {}, 'auto')
assert out is not None
assert out['requested_level'] == 'standard'
assert url_calls['calls'] == [(101, 'exhigh'), (101, 'standard')]
@@ -94,7 +99,7 @@ async def test_missing_url_tries_next_level(url_calls):
"""data[0] 无 url:视同失败,继续降级。"""
url_calls['results']['exhigh'] = {'data': [{'url': None}]}
url_calls['results']['standard'] = _ok()
out = resolve_episode_url(101, {}, 'auto')
out = await resolve_episode_url(101, {}, 'auto')
assert out is not None
assert out['requested_level'] == 'standard'
@@ -102,7 +107,7 @@ async def test_missing_url_tries_next_level(url_calls):
async def test_empty_data_tries_next_level(url_calls):
url_calls['results']['exhigh'] = None # res 为 None → data 为 []
url_calls['results']['standard'] = _ok()
out = resolve_episode_url(101, {}, 'auto')
out = await resolve_episode_url(101, {}, 'auto')
assert out is not None
assert out['requested_level'] == 'standard'
@@ -110,12 +115,12 @@ async def test_empty_data_tries_next_level(url_calls):
async def test_bad_song_id_returns_none_without_requests(url_calls):
"""song_id 垃圾值:None,且不发任何请求(防御原 TypeError 500)。"""
for bad in (None, 'abc', ''):
assert resolve_episode_url(bad, {}, 'auto') is None
assert await resolve_episode_url(bad, {}, 'auto') is None
assert url_calls['calls'] == []
async def test_numeric_string_song_id_accepted(url_calls):
url_calls['results']['exhigh'] = _ok()
out = resolve_episode_url(' 101 ', {}, 'auto')
out = await resolve_episode_url(' 101 ', {}, 'auto')
assert out is not None
assert url_calls['calls'][0] == (101, 'exhigh')
+7 -2
View File
@@ -1,5 +1,6 @@
# -*- coding: utf-8 -*-
"""纯 FastAPI 模式(api_prefix='')测试:前端全部路径由 FastAPI 服务。"""
import asyncio
import warnings
import pytest
@@ -41,8 +42,12 @@ def root_client(tmp_path, monkeypatch):
monkeypatch.setattr(auth.db, 'SECRET_KEY_PATH', tmp_path / 'secret.key')
monkeypatch.setattr(db, 'DATA_DIR', tmp_path)
monkeypatch.setattr(db, 'DB_PATH', tmp_path / 'site.db')
auth.init_db()
monkeypatch.setattr('web.deps.secret_key', lambda: 'root-mode-secret')
asyncio.run(auth.init_db())
async def _fake_secret_key() -> str:
return 'root-mode-secret'
monkeypatch.setattr('web.deps.secret_key', _fake_secret_key)
from web.app import create_app
return TestClient(create_app(api_prefix=''))
+17 -21
View File
@@ -3,20 +3,22 @@
为什么:``music_api/api.py::search_music`` 是站内搜索的唯一上游通道
(101 行 / 四路条目映射 / 嵌套 7),此前**零测试覆盖**。它的错误映射
(RequestException/JSONDecodeError→APIException)与四路字段的容错口径
(UpstreamError/JSONDecodeError→APIException)与四路字段的容错口径
(残缺单条跳过、空 artists 不兜底、alias 缺省 '' 等)都是对外可观察行为,
拆解时改坏任意一路都不会有测试报警。
按惯例:先在未改动的原始实现上跑绿,重构时这批测试即等价性验证器。
替身边界:monkeypatch ``music_api.api.requests.post``(aiohttp_compat 垫片
的同形入口),响应替身只实现 ``raise_for_status`` / ``json`` 两个同步方法——
``search_music`` 只用到这两个。
替身边界:monkeypatch ``music_api.api._http.request``(core.http_client 的进程级
客户端),响应替身实现 ``json`` 一个同步方法——``search_music`` 只用到它。
状态码错误不再由替身经 ``raise_for_status`` 模拟:请求入口自身即对非 2xx 抛
``UpstreamHTTPError``(见 core.http_client.request_once)。
"""
import json
from typing import Any
import pytest
from core.http_client import UpstreamNetworkError
from music_api import APIException
from music_api import api as api_mod
from music_api.api import NeteaseAPI
@@ -26,19 +28,13 @@ _API = NeteaseAPI()
class _FakeResp:
"""``requests.post`` 响应替身:json 载荷或注入异常。"""
"""请求响应替身(``HTTPResult`` 的形状子集):json 载荷或注入异常。"""
def __init__(self, payload: Any = None, *,
status_error: Exception | None = None,
json_error: Exception | None = None):
self.payload = payload
self.status_error = status_error
self.json_error = json_error
def raise_for_status(self) -> None:
if self.status_error is not None:
raise self.status_error
def json(self) -> Any:
if self.json_error is not None:
raise self.json_error
@@ -47,15 +43,16 @@ class _FakeResp:
@pytest.fixture()
def post_capture(monkeypatch):
"""替换 requests.post,捕获调用参数并按用例注入响应。"""
"""替换请求入口,捕获调用参数并按用例注入响应。"""
holder: dict[str, Any] = {'resp': _FakeResp({'code': 200, 'result': {}})}
async def _fake_post(url: str, **kw: Any):
async def _fake_request(method: str, url: str, **kw: Any):
holder['method'] = method
holder['url'] = url
holder['data'] = kw.get('data')
return holder['resp']
monkeypatch.setattr(api_mod.requests, 'post', _fake_post)
monkeypatch.setattr(api_mod._http, 'request', _fake_request)
return holder
@@ -178,16 +175,15 @@ async def test_non_dict_response_raises_format_error(post_capture):
async def test_request_error_maps_to_api_exception(post_capture, monkeypatch):
"""网络层异常(垫片 RequestException 族):转"搜索请求失败"。
"""网络层异常(core.http_client 的 UpstreamError 族):转"搜索请求失败"。
注意替身抛的必须是垫片自己的 ``RequestException``——真实链路里
aiohttp.ClientError 由 aiohttp_compat 内部翻译后才冒出来,直接抛底层
异常是不存在的形状(test_request_* 相关层次见 core.http_errors)。
注意替身抛的必须是 ``UpstreamNetworkError``——真实链路里 aiohttp.ClientError
由 core.http_client 内部翻译后才冒出来,直接抛底层异常是不存在的形状。
"""
async def _boom(url: str, **kw: Any):
raise api_mod.requests.RequestException('connection reset')
async def _boom(method: str, url: str, **kw: Any):
raise UpstreamNetworkError('connection reset')
monkeypatch.setattr(api_mod.requests, 'post', _boom)
monkeypatch.setattr(api_mod._http, 'request', _boom)
with pytest.raises(APIException, match='搜索请求失败'):
await _API.search_music('测试', {}, search_type=1)
+26 -15
View File
@@ -13,6 +13,7 @@
账本口径:一律用 ``reserve_and_open`` 真预扣(``ledger_open`` 只记账不扣款,
会让「退款」凭空给余额加次数,守恒断言就失去意义)。
"""
import asyncio
import threading
import time
import zipfile
@@ -62,7 +63,7 @@ class _FakeDownloader:
self.info_calls = 0
self.download_calls = 0
def get_music_info(self, mid: Any, quality: str, cookies: Any) -> MusicInfo:
async def get_music_info(self, mid: Any, quality: str, cookies: Any) -> MusicInfo:
self.info_calls += 1
if self.info_error is not None:
raise self.info_error
@@ -73,7 +74,7 @@ class _FakeDownloader:
quality='standard', lyric=self.lyric,
)
def download_music_file(self, mid: Any, quality: str, **kw: Any) -> DownloadResult:
async def download_music_file(self, mid: Any, quality: str, **kw: Any) -> DownloadResult:
self.download_calls += 1
if self.on_download is not None:
self.on_download(kw)
@@ -100,8 +101,8 @@ def sandbox(tmp_path, monkeypatch):
monkeypatch.setattr(auth.db, 'SECRET_KEY_PATH', tmp_path / 'secret.key')
monkeypatch.setattr(core_db, 'DATA_DIR', tmp_path)
monkeypatch.setattr(core_db, 'DB_PATH', db_path)
auth.init_db()
auth.create_user('payer', PW, email='p@example.com', credits=CREDITS)
asyncio.run(auth.init_db())
asyncio.run(auth.create_user('payer', PW, email='p@example.com', credits=CREDITS))
downloads = tmp_path / 'downloads'
downloads.mkdir()
@@ -110,7 +111,10 @@ def sandbox(tmp_path, monkeypatch):
monkeypatch.setattr(api_service, 'downloads_path', downloads)
monkeypatch.setattr(api_service, 'logger', _NullLogger())
# 封面走网络——测试里默认返回固定字节,个例可覆盖
monkeypatch.setattr(downloads_svc, 'fetch_cover_bytes', lambda url: b'\xff\xd8\xff cover')
async def _cover(_url: str) -> bytes:
return b'\xff\xd8\xff cover'
monkeypatch.setattr(downloads_svc, 'fetch_cover_bytes', _cover)
# 操作记录写进程内共享列表——隔离掉,断言改由个例捕获
monkeypatch.setattr(downloads_svc, '_record_log', lambda *a: None)
@@ -124,13 +128,13 @@ def sandbox(tmp_path, monkeypatch):
# ============ 工具 ============
def _uid(username: str = 'payer') -> int:
u = auth.get_user_by_name(username)
u = asyncio.run(auth.get_user_by_name(username))
assert u is not None
return int(u['id'])
def _balance(username: str = 'payer') -> int:
u = auth.get_user_by_name(username)
u = asyncio.run(auth.get_user_by_name(username))
assert u is not None
return int(u['duration_seconds'] or 0)
@@ -144,7 +148,7 @@ def _register_single(sandbox, *, music_id: int = 1, user_id: int,
reserved: int = 1, **params: Any) -> tuple[str, dict[str, Any]]:
tid = 's' + str(int(time.time() * 1000000) % 10 ** 10)
# reserve_and_open:真预扣 + 开账同事务(ledger_open 只记账不扣款,见模块 docstring)
left = auth.reserve_and_open(user_id, reserved, tid)
left = asyncio.run(auth.reserve_and_open(user_id, reserved, tid))
assert left is not None, '预扣失败(余额不足,检查 fixture 的 credits)'
task: dict[str, Any] = {
'type': 'single', 'task_id': tid,
@@ -167,8 +171,11 @@ def _register_single(sandbox, *, music_id: int = 1, user_id: int,
def _run(task_id: str) -> threading.Thread:
th = threading.Thread(target=downloads_svc._single_download_worker,
args=(task_id,), daemon=True)
"""把 worker 协程跑在独立线程的私有事件循环里(等价于生产的任务循环)。"""
def _target() -> None:
asyncio.run(downloads_svc._single_download_worker(task_id))
th = threading.Thread(target=_target, daemon=True)
th.start()
return th
@@ -208,7 +215,7 @@ def test_single_download_completes(sandbox, monkeypatch):
assert delivered.read_bytes() == (sandbox['cache'] / '101.mp3').read_bytes()
assert (sandbox['cache'] / '101.mp3').exists(), '共享缓存不应被搬走'
row = auth.ledger_get(tid)
row = asyncio.run(auth.ledger_get(tid))
assert row is not None and row['status'] != 'open'
assert row['delivered'] == 1
assert _balance() == before - 1, '交付成功:只扣预扣的 1 次,无退款'
@@ -275,7 +282,11 @@ def test_cover_failure_keeps_zip_without_cover(sandbox, monkeypatch):
"""
uid = _uid()
monkeypatch.setattr(api_service, 'downloader', _FakeDownloader(sandbox['cache']))
monkeypatch.setattr(downloads_svc, 'fetch_cover_bytes', lambda url: None)
async def _no_cover(_url: str) -> None:
return None
monkeypatch.setattr(downloads_svc, 'fetch_cover_bytes', _no_cover)
tid, task = _register_single(sandbox, music_id=105, user_id=uid, need_zip=True)
th = _run(tid)
th.join(timeout=30)
@@ -343,7 +354,7 @@ def test_download_failure_marks_error_and_refunds(sandbox, monkeypatch):
assert task['msg'] == '上游 404'
assert task['result'] is None
assert _balance() == before, '失败应全额退回预扣'
row = auth.ledger_get(tid)
row = asyncio.run(auth.ledger_get(tid))
assert row is not None and row['delivered'] == 0 and row['status'] != 'open'
assert not (sandbox['downloads'] / 'tasks' / tid).exists(), '失败应清掉任务目录'
@@ -388,7 +399,7 @@ def test_cancel_mid_download(sandbox, monkeypatch):
assert task['msg'] == '任务已取消'
assert task['result'] is None
assert _balance() == before, '取消应全额退回预扣'
row = auth.ledger_get(tid)
row = asyncio.run(auth.ledger_get(tid))
assert row is not None and row['delivered'] == 0
assert not (sandbox['downloads'] / 'tasks' / tid).exists()
@@ -575,6 +586,6 @@ def test_ledger_open_when_worker_crashes_before_settle(sandbox, monkeypatch):
th = _run(tid)
th.join(timeout=30)
row = auth.ledger_get(tid)
row = asyncio.run(auth.ledger_get(tid))
assert row is not None
assert row['status'] == 'open', f"应留给对账兜底,实际 status={row['status']}"
+6 -1
View File
@@ -33,6 +33,11 @@ SECRET = 'song-test-secret'
PATH = '/song'
async def _fake_secret_key() -> str:
"""web.deps.secret_key 已异步化:替身必须是协程函数,同步 lambda 会在 await 处报错。"""
return SECRET
class FakeAPI:
"""music_api.NeteaseAPI 的替身:记录调用,返回预置结果或抛预置异常。"""
@@ -121,7 +126,7 @@ def client(tmp_path, monkeypatch):
monkeypatch.setattr(core_db, 'DB_PATH', path)
monkeypatch.setattr(auth_system.db, 'DATA_DIR', tmp_path)
monkeypatch.setattr(auth_system.db, 'DB_PATH', path)
monkeypatch.setattr('web.deps.secret_key', lambda: SECRET)
monkeypatch.setattr('web.deps.secret_key', _fake_secret_key)
from web.app import create_app
app = TestClient(create_app(api_prefix=''))
cookie = cs.dumps(SECRET, {'site_uid': 1, 'site_uid_seq': 7, 'site_uid_ts': int(time.time())})
+43 -31
View File
@@ -16,6 +16,7 @@ ZIP 内容、账本守恒、暂停/恢复/取消语义。断言只依赖「调
账本口径:一律用 ``reserve_and_open`` 真预扣(``ledger_open`` 只记账不扣款,
会让「退款」凭空给余额加次数,守恒断言就失去意义)。
"""
import asyncio
import threading
import time
import zipfile
@@ -59,7 +60,7 @@ class _FakeDownloader:
self.calls: list[int] = []
self._lock = threading.Lock()
def get_music_info(self, mid: Any, quality: str, cookies: Any) -> MusicInfo:
async def get_music_info(self, mid: Any, quality: str, cookies: Any) -> MusicInfo:
return MusicInfo(
id=int(mid), name=f'歌{mid}', artists='歌手A', album='专辑A',
pic_url='', duration=200, track_number=1, download_url='',
@@ -67,7 +68,7 @@ class _FakeDownloader:
lyric='[00:01.00]第一行\n[00:02.00]第二行',
)
def download_music_file(self, mid: Any, quality: str, **kw: Any) -> DownloadResult:
async def download_music_file(self, mid: Any, quality: str, **kw: Any) -> DownloadResult:
with self._lock:
self.calls.append(int(mid))
seq = len(self.calls)
@@ -97,8 +98,8 @@ def sandbox(tmp_path, monkeypatch):
monkeypatch.setattr(auth.db, 'SECRET_KEY_PATH', tmp_path / 'secret.key')
monkeypatch.setattr(core_db, 'DATA_DIR', tmp_path)
monkeypatch.setattr(core_db, 'DB_PATH', db_path)
auth.init_db()
auth.create_user('payer', PW, email='p@example.com', credits=CREDITS)
asyncio.run(auth.init_db())
asyncio.run(auth.create_user('payer', PW, email='p@example.com', credits=CREDITS))
downloads = tmp_path / 'downloads'
downloads.mkdir()
@@ -117,13 +118,13 @@ def sandbox(tmp_path, monkeypatch):
# ============ 工具 ============
def _uid(username: str = 'payer') -> int:
u = auth.get_user_by_name(username)
u = asyncio.run(auth.get_user_by_name(username))
assert u is not None
return int(u['id'])
def _balance(username: str = 'payer') -> int:
u = auth.get_user_by_name(username)
u = asyncio.run(auth.get_user_by_name(username))
assert u is not None
return int(u['duration_seconds'] or 0)
@@ -155,7 +156,7 @@ def _register_batch(sandbox, ids: list[int], *, reserved: int, user_id: int,
tid = 't' + str(int(time.time() * 1000000) % 10 ** 10)
# 必须走 reserve_and_open(真预扣+开账同事务)——ledger_open 只记账不扣款,
# 用它会让「退款」凭空给余额加次数,账本守恒断言失去意义
left = auth.reserve_and_open(user_id, reserved, tid)
left = asyncio.run(auth.reserve_and_open(user_id, reserved, tid))
assert left is not None, '预扣失败(余额不足,检查 fixture 的 credits)'
task: dict[str, Any] = {
'type': 'batch', 'task_id': tid,
@@ -178,7 +179,7 @@ def _register_batch(sandbox, ids: list[int], *, reserved: int, user_id: int,
def _register_podcast(sandbox, pids: list[int], *, reserved: int, user_id: int) -> tuple[str, dict[str, Any]]:
tid = 'p' + str(int(time.time() * 1000000) % 10 ** 10)
left = auth.reserve_and_open(user_id, reserved, tid)
left = asyncio.run(auth.reserve_and_open(user_id, reserved, tid))
assert left is not None, '预扣失败(余额不足)'
task: dict[str, Any] = {
'type': 'podcast_batch', 'task_id': tid,
@@ -199,7 +200,11 @@ def _register_podcast(sandbox, pids: list[int], *, reserved: int, user_id: int)
def _run(worker, task_id: str) -> threading.Thread:
th = threading.Thread(target=worker, args=(task_id,), daemon=True)
"""把 worker 协程跑在独立线程的私有事件循环里(等价于生产的任务循环)。"""
def _target() -> None:
asyncio.run(worker(task_id))
th = threading.Thread(target=_target, daemon=True)
th.start()
return th
@@ -229,7 +234,7 @@ def test_batch_zip_completes(sandbox, monkeypatch):
assert sum(1 for n in names if n.endswith('.mp3')) == 3, names
assert sum(1 for n in names if n.endswith('.lrc')) == 3, names
row = auth.ledger_get(tid)
row = asyncio.run(auth.ledger_get(tid))
assert row is not None and row['status'] != 'open'
assert row['delivered'] == 3
assert _balance() == before - 3, '全部成功:只扣预扣的 3 次,无退款'
@@ -283,7 +288,7 @@ def test_batch_zip_pause_then_cancel(sandbox, monkeypatch):
assert task['result'] is None, '未交付不应发布 ZIP'
assert task['msg'] == '任务已取消(已退回 3 次下载次数)', task['msg']
assert _balance() == before, '取消后应全额退回预扣'
row = auth.ledger_get(tid)
row = asyncio.run(auth.ledger_get(tid))
assert row is not None and row['delivered'] == 0 and row['status'] != 'open'
@@ -302,7 +307,7 @@ def test_batch_zip_all_fail_refunds(sandbox, monkeypatch):
assert task['failed'] == 2
assert task['result'] is None, '全失败不应发布 ZIP'
assert _balance() == before, '全失败应全额退回预扣'
row = auth.ledger_get(tid)
row = asyncio.run(auth.ledger_get(tid))
assert row is not None and row['delivered'] == 0 and row['status'] != 'open'
@@ -331,7 +336,7 @@ def test_batch_zip_midway_cancel_refunds_all(sandbox, monkeypatch):
assert _status(tid) == 'canceled', f"终态={_status(tid)} msg={task.get('msg')}"
assert task['result'] is None, '当前实现:循环内取消不发布'
assert _balance() == before, '未交付部分全额退回'
row = auth.ledger_get(tid)
row = asyncio.run(auth.ledger_get(tid))
assert row is not None and row['delivered'] == 0
@@ -346,8 +351,9 @@ def test_batch_zip_cancel_at_publish_checkpoint_delivers(sandbox, monkeypatch):
holder: dict[str, Any] = {}
real_settle = batch_svc._settle_batch_billing
def settle_then_cancel(task_id: str, ok_count: int, final_status: str = 'settled') -> int:
refunded = real_settle(task_id, ok_count, final_status)
async def settle_then_cancel(task_id: str, ok_count: int,
final_status: str = 'settled') -> int:
refunded = await real_settle(task_id, ok_count, final_status)
holder['task']['cancel'].set() # 结算已完成、发布前一刻取消
return refunded
@@ -371,24 +377,27 @@ def test_batch_zip_cancel_at_publish_checkpoint_delivers(sandbox, monkeypatch):
@pytest.fixture()
def podcast_stub(monkeypatch, sandbox):
"""把播客上游与流式下载替换为本地确定性实现。"""
def fake_detail(pid: Any, cookies: Any) -> dict[str, Any]:
async def fake_detail(pid: Any, cookies: Any) -> dict[str, Any]:
n = int(pid)
return {'id': n, 'name': f'单集{n}', 'main_song_id': 1,
'serial_num': n, 'cover_url': '', 'songs': []}
def fake_resolve(prog: dict[str, Any], quality: str, cookies: Any) -> dict[str, Any]:
async def fake_resolve(prog: dict[str, Any], quality: str, cookies: Any) -> dict[str, Any]:
return {'url': f"http://example.invalid/{prog['id']}.mp3", 'size': 2048, 'type': 'mp3'}
def fake_stream(url: str, target: Path, total_size: int, cancel_ev: Any,
async def fake_stream(url: str, target: Path, total_size: int, cancel_ev: Any,
task_id: str = '', progress_label: str = '') -> int:
if cancel_ev is not None and cancel_ev.is_set():
raise DownloadCanceled('任务已取消')
target.write_bytes(b'\xff\xfb\x90\x00' + b'\x00' * 128)
return target.stat().st_size
async def _no_cover(_u: str) -> None:
return None
monkeypatch.setattr(podcast_svc, '_podcast_resolve_audio', fake_resolve)
monkeypatch.setattr(podcast_svc, '_podcast_stream_download', fake_stream)
monkeypatch.setattr(podcast_svc, 'fetch_cover_bytes', lambda _u: None)
monkeypatch.setattr(podcast_svc, 'fetch_cover_bytes', _no_cover)
monkeypatch.setattr(podcast_svc.podcast, 'get_program_detail', fake_detail)
return sandbox
@@ -406,7 +415,7 @@ def test_podcast_zip_completes(podcast_stub):
assert task['done'] == 3 and task['failed'] == 0
names = _zip_names(task['result']['zip_path'])
assert sum(1 for n in names if n.endswith('.mp3')) == 3, names
row = auth.ledger_get(tid)
row = asyncio.run(auth.ledger_get(tid))
assert row is not None and row['delivered'] == 3
assert _balance() == before - 3
@@ -418,7 +427,7 @@ def test_podcast_zip_pause_then_resume(podcast_stub, monkeypatch):
holder: dict[str, Any] = {}
fired = {'n': 0}
def _pause_once(url, target, total_size, cancel_ev, task_id='', progress_label=''):
async def _pause_once(url, target, total_size, cancel_ev, task_id='', progress_label=''):
target.write_bytes(b'\xff\xfb\x90\x00' + b'\x00' * 128)
fired['n'] += 1
if fired['n'] == 1: # 只在第一集触发——否则恢复后立刻又被暂停
@@ -447,7 +456,7 @@ def test_podcast_zip_all_fail_refunds(podcast_stub, monkeypatch):
uid = _uid()
before = _balance()
def _boom(prog, quality, cookies):
async def _boom(prog, quality, cookies):
raise RuntimeError('上游不可用')
monkeypatch.setattr(podcast_svc, '_podcast_resolve_audio', _boom)
@@ -458,7 +467,7 @@ def test_podcast_zip_all_fail_refunds(podcast_stub, monkeypatch):
assert _status(tid) == 'error', f"终态={_status(tid)} msg={task.get('msg')}"
assert _balance() == before, '全失败应全额退回'
row = auth.ledger_get(tid)
row = asyncio.run(auth.ledger_get(tid))
assert row is not None and row['delivered'] == 0
@@ -470,9 +479,9 @@ def test_podcast_zip_all_fail_refunds(podcast_stub, monkeypatch):
class _SameNameDownloader(_FakeDownloader):
"""所有歌曲同名同歌手、且都带封面——用于验证包内三条目共享序号后缀。"""
def get_music_info(self, mid: Any, quality: str, cookies: Any) -> MusicInfo:
return replace(super().get_music_info(mid, quality, cookies),
name='同名歌', artists='同歌手',
async def get_music_info(self, mid: Any, quality: str, cookies: Any) -> MusicInfo:
base = await super().get_music_info(mid, quality, cookies)
return replace(base, name='同名歌', artists='同歌手',
pic_url='http://example.invalid/cover.jpg')
@@ -485,7 +494,10 @@ def test_batch_zip_dedups_three_arcnames_with_shared_suffix(sandbox, monkeypatch
"""
uid = _uid()
monkeypatch.setattr(api_service, 'downloader', _SameNameDownloader(sandbox['cache']))
monkeypatch.setattr(batch_svc, 'fetch_cover_bytes', lambda _u: b'\xff\xd8\xff\xe0')
async def _cover(_u: str) -> bytes:
return b'\xff\xd8\xff\xe0'
monkeypatch.setattr(batch_svc, 'fetch_cover_bytes', _cover)
tid, task = _register_batch(sandbox, [701, 702], reserved=2, user_id=uid)
th = _run(batch_svc._batch_zip_worker, tid)
@@ -513,7 +525,7 @@ def test_batch_all_fail_message_and_ledger_status(sandbox, monkeypatch):
th.join(timeout=30)
assert task['msg'] == '全部失败(2 首),已退回 2 次', task['msg']
row = auth.ledger_get(tid)
row = asyncio.run(auth.ledger_get(tid))
assert row is not None and row['status'] == 'error'
@@ -528,7 +540,7 @@ def test_podcast_all_fail_message_and_ledger_status(podcast_stub, monkeypatch):
uid = _uid()
before = _balance()
def _boom(prog, quality, cookies):
async def _boom(prog, quality, cookies):
raise RuntimeError('上游不可用')
monkeypatch.setattr(podcast_svc, '_podcast_resolve_audio', _boom)
@@ -543,7 +555,7 @@ def test_podcast_all_fail_message_and_ledger_status(podcast_stub, monkeypatch):
), task['msg']
assert task['result'] is None, '不发布空包'
assert _balance() == before, '全失败应全额退回'
row = auth.ledger_get(tid)
row = asyncio.run(auth.ledger_get(tid))
assert row is not None and row['status'] == 'error' and row['delivered'] == 0
@@ -557,7 +569,7 @@ def test_podcast_cleans_temp_files_on_episode_failure(podcast_stub, monkeypatch)
uid = _uid()
before = _balance()
def _flaky(url: str, target: Path, total_size: int, cancel_ev: Any,
async def _flaky(url: str, target: Path, total_size: int, cancel_ev: Any,
task_id: str = '', progress_label: str = '') -> int:
if '/102.' in url:
target.write_bytes(b'\x00' * 64) # 正文残留
Generated
+11 -300
View File
@@ -2,15 +2,6 @@ version = 1
revision = 3
requires-python = ">=3.12"
[[package]]
name = "aiofiles"
version = "25.1.0"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/41/c3/534eac40372d8ee36ef40df62ec129bee4fdb5ad9706e58a29be53b2c970/aiofiles-25.1.0.tar.gz", hash = "sha256:a8d728f0a29de45dc521f18f07297428d56992a742f0cd2701ba86e44d23d5b2", size = 46354, upload-time = "2025-10-09T20:51:04.358Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/bc/8a/340a1555ae33d7354dbca4faa54948d76d89a27ceef032c8c3bc661d003e/aiofiles-25.1.0-py3-none-any.whl", hash = "sha256:abe311e527c862958650f9438e859c1fa7568a141b22abcd015e120e86a85695", size = 14668, upload-time = "2025-10-09T20:51:03.174Z" },
]
[[package]]
name = "aiohappyeyeballs"
version = "2.7.1"
@@ -133,6 +124,15 @@ wheels = [
{ url = "https://files.pythonhosted.org/packages/fb/76/641ae371508676492379f16e2fa48f4e2c11741bd63c48be4b12a6b09cba/aiosignal-1.4.0-py3-none-any.whl", hash = "sha256:053243f8b92b990551949e63930a839ff0cf0b0ebbe0597b0f3fb19e1a0fe82e", size = 7490, upload-time = "2025-07-03T22:54:42.156Z" },
]
[[package]]
name = "aiosmtplib"
version = "5.1.3"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/9b/5c/9cabc5db6d607616e81ba6d8f1f231cd5a75955807a308c1090a59072d6d/aiosmtplib-5.1.3.tar.gz", hash = "sha256:ac2b418d3260ba62d9cfd0fe7359726e9dc009a4e8e8d9909fdfae332f522a7c", size = 77010, upload-time = "2026-09-08T02:11:20.532Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/9c/0a/b56ab8163d54960337fdca475d3dfd56c8badf6172e79cf2ad00d5335dc1/aiosmtplib-5.1.3-py3-none-any.whl", hash = "sha256:f7d76ce3d4995a65a178c1f11e1bd1607706b921d00cb768e7a2c7f7ef5517a8", size = 30116, upload-time = "2026-09-08T02:11:19.352Z" },
]
[[package]]
name = "aiosqlite"
version = "0.22.1"
@@ -173,21 +173,6 @@ wheels = [
{ url = "https://files.pythonhosted.org/packages/12/b8/4bd346e22b28902df4d651910f5242c28d84e4a5c2435ca5c3f797ed7e2e/anyio-4.15.1-py3-none-any.whl", hash = "sha256:6152fdbbf9a77fdec97731721bebf7c4c44f7c29b424b0065826173efc7ed101", size = 132079, upload-time = "2026-09-05T10:42:37.923Z" },
]
[[package]]
name = "aria2p"
version = "0.12.1"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "loguru" },
{ name = "platformdirs" },
{ name = "requests" },
{ name = "websocket-client" },
]
sdist = { url = "https://files.pythonhosted.org/packages/f7/d5/68f38791f44e5caf5c7c9576aea61d77cf940a060135358dcadb5345dd9c/aria2p-0.12.1.tar.gz", hash = "sha256:1521ccba4a4bb164a023f4c4213513dc3e6f47b91927f5e49dd50678da8c1294", size = 270595, upload-time = "2024-12-25T17:06:34.571Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/dc/95/1ec57df6c52152fb4c776da5fbb2896f19af52de3776899d91e7140e7654/aria2p-0.12.1-py3-none-any.whl", hash = "sha256:96d31c1edd3c9e0edf725b7adeb0b9fb9b6ae102f3c228040d773b43837913a2", size = 88242, upload-time = "2024-12-25T17:06:30.995Z" },
]
[[package]]
name = "attrs"
version = "26.1.0"
@@ -291,137 +276,6 @@ wheels = [
{ url = "https://files.pythonhosted.org/packages/aa/29/35e016098c814cd93de9cd320c66b5bfba14dc6ecedd3cb518fa7c408c69/cffi-2.1.1-cp315-cp315t-win_arm64.whl", hash = "sha256:d18e5ac0f2f03f4f518d3e23db0f0cad7faa1da8620e9c09461d443bbf6e6692", size = 186360, upload-time = "2026-08-03T21:21:13.636Z" },
]
[[package]]
name = "charset-normalizer"
version = "3.5.1"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/e5/3f/143b048436775b0f76ac3eec145c019e8173ccc2885c8f20319b996d5e83/charset_normalizer-3.5.1.tar.gz", hash = "sha256:6117b84ea48435e5356dc737f5121485c30920ba43375fa7b434fd753df0eac3", size = 171764, upload-time = "2026-08-15T08:20:44.807Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/30/27/78873dc8b6a56357517b74b6bb9568b80450e7bb4f6ef7e3fa9d22aa0bd7/charset_normalizer-3.5.1-cp312-cp312-macosx_10_13_universal2.whl", hash = "sha256:5b6d1386bf0096d26d3a863dc0a487a5b4eb9aa93cf5ba69683d29dde6b9d60f", size = 344456, upload-time = "2026-08-15T08:17:10.072Z" },
{ url = "https://files.pythonhosted.org/packages/9a/4c/be49ada26b1f0232d57aa89bbebf997a5cc2332a5616b6eca26ff680044d/charset_normalizer-3.5.1-cp312-cp312-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:4582c27e8c889d64811987b5967fbd3ae0c823fe1fd933b543d55ac20bb475fa", size = 238530, upload-time = "2026-08-15T08:17:11.563Z" },
{ url = "https://files.pythonhosted.org/packages/76/84/6f1290fa07ae6978d3960caa3eb1b8019bf9284ab7c2297b00c099ef4250/charset_normalizer-3.5.1-cp312-cp312-manylinux2014_armv7l.manylinux_2_17_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:1d1c7a53a6c2103925cdd6d7229f8c567379f211c869793df679f2e9f738c369", size = 230200, upload-time = "2026-08-15T08:17:12.919Z" },
{ url = "https://files.pythonhosted.org/packages/e7/a0/47b18adeed31c8f16ba9700f32c1b18594cfa09f47eb672a488c273c22bf/charset_normalizer-3.5.1-cp312-cp312-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:e6621fb2a4988d6e53eedc455e5903e2679f3967b8acb3d639f1b63c14a2e893", size = 262222, upload-time = "2026-08-15T08:17:14.571Z" },
{ url = "https://files.pythonhosted.org/packages/38/fe/341861ac118dae06f3ec0eb487488af52128f2ef2faf0b11003944d22259/charset_normalizer-3.5.1-cp312-cp312-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:7c0c10730342b0c9b35dd1d619beb8214e520bd96a1f870f452680b238aab3e0", size = 258951, upload-time = "2026-08-15T08:17:16.158Z" },
{ url = "https://files.pythonhosted.org/packages/6f/89/bb5108dc6c3651dca963f2b0a3ba19bbcb370c94e1b6d3e0e844a58e6dca/charset_normalizer-3.5.1-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:b9af956078716df40d985fb0dfeb2c2120c5ca92ba4ff4b388acfd01cdc14d08", size = 248801, upload-time = "2026-08-15T08:17:17.683Z" },
{ url = "https://files.pythonhosted.org/packages/b1/ba/ef83ae3aca816393decfa3530976f38a79812d707b80b580ac33b83f9877/charset_normalizer-3.5.1-cp312-cp312-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:f9f8405c2c758532c74fed975dbee57be1f31a6e865c031870c79a6ed3212ada", size = 244070, upload-time = "2026-08-15T08:17:19.191Z" },
{ url = "https://files.pythonhosted.org/packages/f6/0b/c5292a2462d69b7378ea89793bbb5b2b6fcf6f7dd6d1667f9619094ad553/charset_normalizer-3.5.1-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:96fef3e886d6a9874b14f27fc193fbdc69d5d8035783d86aa4e1cea594e695f9", size = 240110, upload-time = "2026-08-15T08:17:20.547Z" },
{ url = "https://files.pythonhosted.org/packages/46/22/111e5be3b740d5c2a5bfcedb3d237b6591e5c2e82ae9d6ffcb121fe0909c/charset_normalizer-3.5.1-cp312-cp312-musllinux_1_2_armv7l.whl", hash = "sha256:5d8531a6569d025f68e2321e7638fb7978f23db58e5f69f56913837aae03816e", size = 232836, upload-time = "2026-08-15T08:17:21.895Z" },
{ url = "https://files.pythonhosted.org/packages/f9/d2/d2aad6fe0dbb44b194bf3becb60f5a0ac48446ade999a47fe7bb41eb09a7/charset_normalizer-3.5.1-cp312-cp312-musllinux_1_2_ppc64le.whl", hash = "sha256:aae2ee51122d3ae968a3837d97dc24a0aeebb0dea23694422cd172bd30017cd6", size = 262712, upload-time = "2026-08-15T08:17:23.727Z" },
{ url = "https://files.pythonhosted.org/packages/35/5a/337e4663a5eae6de99db940ee8066d4145caafb61327db62deda15313cce/charset_normalizer-3.5.1-cp312-cp312-musllinux_1_2_riscv64.whl", hash = "sha256:7235dc28fc6dd9d832ac7c7bce95367dedb85929f17368a0c2bee1e080b9acbf", size = 242977, upload-time = "2026-08-15T08:17:25.157Z" },
{ url = "https://files.pythonhosted.org/packages/ca/85/f82f8a92e31c7519410e2e1afdc630f28ec47490ce2c09a11c1a43cbb459/charset_normalizer-3.5.1-cp312-cp312-musllinux_1_2_s390x.whl", hash = "sha256:4abdc5f9ad448c1ecbfae2974b820535d6bc6e7eef63babbab3d81cf46968c71", size = 260207, upload-time = "2026-08-15T08:17:26.602Z" },
{ url = "https://files.pythonhosted.org/packages/b7/52/643d11ffd60e9ac2fd1fb87e167a19285b9eefeff4a40e63c87cbfbeab36/charset_normalizer-3.5.1-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:ba501e667c17d8411f98e67a022d9604ef179aff0e459b7e292c796837c13573", size = 250562, upload-time = "2026-08-15T08:17:27.971Z" },
{ url = "https://files.pythonhosted.org/packages/62/16/46556278c2168d12df9da7fede5dc6fc70e60301b26a82bbeec238c9cfe3/charset_normalizer-3.5.1-cp312-cp312-win32.whl", hash = "sha256:cfa1c0cc3a8f9f53f1243a5a99ac36fd003880199383b37672e86ddda9cb07e2", size = 178507, upload-time = "2026-08-15T08:17:29.277Z" },
{ url = "https://files.pythonhosted.org/packages/9d/7a/4c6c298171e6b3e745633180ff59350fc0ca0db1ffd28df1e369e0579f71/charset_normalizer-3.5.1-cp312-cp312-win_amd64.whl", hash = "sha256:3617ac3cfd8b9888f145ad89dd6e692285834b0201c6074a5eeaad3fd4d668c2", size = 200551, upload-time = "2026-08-15T08:17:30.668Z" },
{ url = "https://files.pythonhosted.org/packages/cd/d7/eb95a042f0dd22e304b0b6472b154f3546a1a039a9ee89ccb2a7f61591fc/charset_normalizer-3.5.1-cp312-cp312-win_arm64.whl", hash = "sha256:88e85ab89cb822c1e635f51d6d32e488f94e002e70e2f492bdb8b945543f345a", size = 180700, upload-time = "2026-08-15T08:17:32.028Z" },
{ url = "https://files.pythonhosted.org/packages/bc/61/2cb6ad133dbbb449fa2d37ccae973232f4827e799af258d15e589a3d1e9e/charset_normalizer-3.5.1-cp313-cp313-android_24_arm64_v8a.whl", hash = "sha256:4f298bdadb8f0b9e5672877f647d1be9373ef5320c9e2f049795e26cad28b6a9", size = 211584, upload-time = "2026-08-15T08:17:33.597Z" },
{ url = "https://files.pythonhosted.org/packages/18/57/a305c968be1ca13f3dd1b32f445877e97addf55d80b65c7cb35fac82b777/charset_normalizer-3.5.1-cp313-cp313-android_24_x86_64.whl", hash = "sha256:88ca277405c2d3b71c4e1c2ee0e7966e807bcba86a69d11e19ba199d18ae4491", size = 223359, upload-time = "2026-08-15T08:17:35.022Z" },
{ url = "https://files.pythonhosted.org/packages/09/0a/d3646670292ce8d8f8cc11ac067d44885e697a5591f57a9221128da5e7b3/charset_normalizer-3.5.1-cp313-cp313-ios_13_0_arm64_iphoneos.whl", hash = "sha256:9362dd90aa7dab48c0054a21187791ccf05473f7dba5d92b8033ae62164675e7", size = 194464, upload-time = "2026-08-15T08:17:36.452Z" },
{ url = "https://files.pythonhosted.org/packages/de/93/d51ec556e01042fed6f993ea859311bc7917b466684182fbbceb6ca24762/charset_normalizer-3.5.1-cp313-cp313-ios_13_0_arm64_iphonesimulator.whl", hash = "sha256:977cdbd483a9cff38179bea4fd754289a6f2195c7abd414aba85410b3e66cc5e", size = 197676, upload-time = "2026-08-15T08:17:37.819Z" },
{ url = "https://files.pythonhosted.org/packages/a4/a0/562247944386f7d4ef94467e84876600cc1e0f1b93239aaa9213d2bc3cbd/charset_normalizer-3.5.1-cp313-cp313-macosx_10_13_universal2.whl", hash = "sha256:e90251c0c7bdd54a100a0dce3c07b7e637278c93af29dbf78ebb89a58c4bac7d", size = 340473, upload-time = "2026-08-15T08:17:39.303Z" },
{ url = "https://files.pythonhosted.org/packages/31/e7/1d994be1b93d41e9502b8b0460eaa88a1dd8df335df415db87d6c3e91ab2/charset_normalizer-3.5.1-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:94d78ecec2605a8d0398b0f365d5f12a63248438516f5dac536a5eff7337df4a", size = 240156, upload-time = "2026-08-15T08:17:40.66Z" },
{ url = "https://files.pythonhosted.org/packages/09/53/27923ce5cc6cbccb832037b27dca98882d9c53e9b69e866bbbef4aae7fc8/charset_normalizer-3.5.1-cp313-cp313-manylinux2014_armv7l.manylinux_2_17_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:d59b75732e9b6f27388e10c14b0259cc5f2e48c78627d185e6a177b58ad3cffe", size = 228246, upload-time = "2026-08-15T08:17:42.003Z" },
{ url = "https://files.pythonhosted.org/packages/ce/48/5a97e84d63af1d55c07439cb80e56d99a8efb4295700eb4e18c0d1615d2c/charset_normalizer-3.5.1-cp313-cp313-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:0d929fc574b4d6fd9e7c0f5c2ede8716a41911923aa7fa5fce38e0818aa4a1ac", size = 263660, upload-time = "2026-08-15T08:17:43.627Z" },
{ url = "https://files.pythonhosted.org/packages/7a/c2/071575791dcc88316c0a9a65ce38897a82e4cfe4a325f0f7fe1b1ac47bcf/charset_normalizer-3.5.1-cp313-cp313-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:394fea06235c8543390050ed5f529187074b029fb027213f6c46ac11ab5d950e", size = 260354, upload-time = "2026-08-15T08:17:45.094Z" },
{ url = "https://files.pythonhosted.org/packages/fb/af/63240b0c0248c075c2535a1f1bd992821d8251b9f173abc13329661d09e4/charset_normalizer-3.5.1-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:62b55f6722735a6c472f88361cde6640608773d9443cebdbb51abf436a1fcdd3", size = 250638, upload-time = "2026-08-15T08:17:46.496Z" },
{ url = "https://files.pythonhosted.org/packages/4d/66/70dfad64f15be09c15ccfee81330a7e515895dbe296dd23114e9a231268a/charset_normalizer-3.5.1-cp313-cp313-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:fa48b1b63d639f9483e0633e092f5851e2348c352f1f9bb6c8182f87884ef876", size = 244583, upload-time = "2026-08-15T08:17:47.963Z" },
{ url = "https://files.pythonhosted.org/packages/c0/24/ef36367d38b9ddd4bccbf72888c342e8de1f5ae506fa0b2dcf970e2732a1/charset_normalizer-3.5.1-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:c71fb0d56c920c269cd3e2e3fe7c610e3f1fdb21a6ce60efa6430ff63676cea6", size = 242038, upload-time = "2026-08-15T08:17:49.481Z" },
{ url = "https://files.pythonhosted.org/packages/db/ab/55e683ba0fff2e43adafc10daa3001eac90fdaa419a97227d5a7067eedde/charset_normalizer-3.5.1-cp313-cp313-musllinux_1_2_armv7l.whl", hash = "sha256:485a0d363cafefcd2538a73c7c838daa2035f09b2c9f9b5e3133f80c6aeb84c2", size = 233677, upload-time = "2026-08-15T08:17:50.845Z" },
{ url = "https://files.pythonhosted.org/packages/bd/67/0f40eaf8d1b6e7cf15e82382a2965efaca787fc1c2794b7021d37aaf5036/charset_normalizer-3.5.1-cp313-cp313-musllinux_1_2_ppc64le.whl", hash = "sha256:5c0ea61a470e070686aa30892fed79e297d2c8d0ab46b8bcdf027d38c51da591", size = 264491, upload-time = "2026-08-15T08:17:52.61Z" },
{ url = "https://files.pythonhosted.org/packages/5c/64/12b4c2a11ee8df4fcc518c78b0d93e3a92bd3d5253d1617ce74ff0e8c7ef/charset_normalizer-3.5.1-cp313-cp313-musllinux_1_2_riscv64.whl", hash = "sha256:90b7481fb62fbe172c558bc6fd1c4c98d82004a54a7551f20e11ac9bf0b8708c", size = 245196, upload-time = "2026-08-15T08:17:54.023Z" },
{ url = "https://files.pythonhosted.org/packages/37/2e/651d910af6d0fba325eee1cda37ec5443462ed25360e666c144166eb6091/charset_normalizer-3.5.1-cp313-cp313-musllinux_1_2_s390x.whl", hash = "sha256:35fe081843b35aad20ffeccec3eeffbe637b15d14f3fb22cc1b59cd8ec17e93c", size = 261660, upload-time = "2026-08-15T08:17:55.491Z" },
{ url = "https://files.pythonhosted.org/packages/90/c6/b09e05e6db7f64338e0dc067c79577b1138da86c1e38369096851d96be88/charset_normalizer-3.5.1-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:fd0350afdc3aabd5576f60ea109228bd5538139713c7b094c5cd27c73a98bc6f", size = 252618, upload-time = "2026-08-15T08:17:57.025Z" },
{ url = "https://files.pythonhosted.org/packages/76/4e/362d4f9fdcdf5556fb2aa3ce7d4a58ebce03ed1ff03aa1d9aca8d02f13f3/charset_normalizer-3.5.1-cp313-cp313-pyemscripten_2025_0_wasm32.whl", hash = "sha256:9d9a0dc7cbe9bec24c3f767c9122c41fe5a1bc43f47cd099d00d393e09769de4", size = 140362, upload-time = "2026-08-15T08:17:58.425Z" },
{ url = "https://files.pythonhosted.org/packages/b4/d4/703be739b26acce318bd29eb3b25b7209e1b1f527f9eae3d1f1f01fdde2b/charset_normalizer-3.5.1-cp313-cp313-win32.whl", hash = "sha256:d63600d620ad0064c3a748b950ac5ea38a80190e5498532efefa4b7b3f1da1f3", size = 177755, upload-time = "2026-08-15T08:18:00.037Z" },
{ url = "https://files.pythonhosted.org/packages/8a/33/56d97ade41c8db611e727168c52ae46c9224c362ec28d4b65d7e9869e8da/charset_normalizer-3.5.1-cp313-cp313-win_amd64.whl", hash = "sha256:aea996a6aba25260827c9ea511d1addfde2da9eb686ac961838509086188b7e6", size = 199295, upload-time = "2026-08-15T08:18:01.506Z" },
{ url = "https://files.pythonhosted.org/packages/5b/75/5b20dd1e6573a01a08158fe104104fa2c8abf941745596954185726cd46c/charset_normalizer-3.5.1-cp313-cp313-win_arm64.whl", hash = "sha256:fd0a274c0e5f9a21565cd9d3dd749b61f96b7aa1e20a93aa1ba4029518f2e5c0", size = 179856, upload-time = "2026-08-15T08:18:02.929Z" },
{ url = "https://files.pythonhosted.org/packages/29/cd/2b812ce5e888f1ce69a5350281e58aab07ae64a958ecae8912f30865718e/charset_normalizer-3.5.1-cp314-cp314-android_24_arm64_v8a.whl", hash = "sha256:774d157f112367ff4abd29019f38f023c24e00e56edc7829c20e358a5a913ad8", size = 212318, upload-time = "2026-08-15T08:18:04.403Z" },
{ url = "https://files.pythonhosted.org/packages/9e/4a/a6ee107430768a5334e6d63f31f148a04a1a491ef161a1ac9415a73f2fa8/charset_normalizer-3.5.1-cp314-cp314-android_24_x86_64.whl", hash = "sha256:26422d45fd13551cf564c58932f7d72b4f58b93b0fcf18c35ba6be12b46bb102", size = 224897, upload-time = "2026-08-15T08:18:05.997Z" },
{ url = "https://files.pythonhosted.org/packages/c3/d9/35ae3f64f29d0179c35c3baefe575904df2913dde519129c7f75995a2b1d/charset_normalizer-3.5.1-cp314-cp314-ios_13_0_arm64_iphoneos.whl", hash = "sha256:09a7bba9f739468c8e78c36a75c33768e53cb1959fc638f510454c14683f00d5", size = 194848, upload-time = "2026-08-15T08:18:07.397Z" },
{ url = "https://files.pythonhosted.org/packages/74/76/f2fc7380f056cc273a53af37f50d08ad54b2c59f61078f31432edcf1c2bd/charset_normalizer-3.5.1-cp314-cp314-ios_13_0_arm64_iphonesimulator.whl", hash = "sha256:4c9548dc78002099910abaebc0a72ac58b7d30931869e0351c09b507dff4ece3", size = 198163, upload-time = "2026-08-15T08:18:08.989Z" },
{ url = "https://files.pythonhosted.org/packages/e9/40/095ce62fa078483cccc1fa2b36e6bc9580b85422a20ee9f925341c50e44f/charset_normalizer-3.5.1-cp314-cp314-macosx_10_15_universal2.whl", hash = "sha256:c428c6c31eb5f4277d7f8eccaf767fbd548ddd5ce3c8b4f4cbbfab3d96b5904c", size = 341823, upload-time = "2026-08-15T08:18:10.458Z" },
{ url = "https://files.pythonhosted.org/packages/f1/5a/0e58b1c04a1596e0256f407274a92d5fb2ee21324409d1fab1da48a65b5b/charset_normalizer-3.5.1-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:2f06b7eae9dbe77fe1d644ca244dad508de8d302870a43f3c559b521270938a0", size = 242458, upload-time = "2026-08-15T08:18:11.989Z" },
{ url = "https://files.pythonhosted.org/packages/22/95/b4618ce912e6db0b1aae89ba788e38e8a7eba0f3025cc66e8c0699f977b2/charset_normalizer-3.5.1-cp314-cp314-manylinux2014_armv7l.manylinux_2_17_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:6b7430cf5728e68f6c462254009a6ef4086e1bea43cf2f57aa9c55fb4f50ff96", size = 226717, upload-time = "2026-08-15T08:18:13.401Z" },
{ url = "https://files.pythonhosted.org/packages/8a/76/c681192bbda3d55356db5dadd64381d5202b37c6b598fcda5282e88b5d3d/charset_normalizer-3.5.1-cp314-cp314-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:ab743e9bc90c1f73552ec33e10e3331315acd2c397b36065b591b0181de533cc", size = 266111, upload-time = "2026-08-15T08:18:14.961Z" },
{ url = "https://files.pythonhosted.org/packages/88/be/55127bfca72c0cff6c022488d140d7c5b04c771e3b72e9bdb4836d54979d/charset_normalizer-3.5.1-cp314-cp314-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:f6f7deae3feb4edfa2efaf7c574fe88cbf055038a6abdb40188e4fff66d5699f", size = 263128, upload-time = "2026-08-15T08:18:16.515Z" },
{ url = "https://files.pythonhosted.org/packages/e0/91/39c3af510b0aa32bbda03374259200f28430febfd1bf5e511fe765282ce5/charset_normalizer-3.5.1-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:15f024313246a4ed976c60f440bb8d257815513a681d212ff74fd46f7d715a90", size = 251240, upload-time = "2026-08-15T08:18:18.127Z" },
{ url = "https://files.pythonhosted.org/packages/1c/a5/cbe418bbc6ecdfc3e05a0116002897c4b403a5e838d697e64c78e9f0190d/charset_normalizer-3.5.1-cp314-cp314-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:823f82903d189af463d7df250ef1f7f696f3cee08cc8d91deb565e8d425f6506", size = 245282, upload-time = "2026-08-15T08:18:19.625Z" },
{ url = "https://files.pythonhosted.org/packages/cc/a4/689bb42e8e7cd492f3cb64907c6bc00ad247ec9a3628cd3f8eed126e8ae1/charset_normalizer-3.5.1-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:01e93745f7f219b703b60ba7afead36cfc4242782be5af484673fc500df12da5", size = 244597, upload-time = "2026-08-15T08:18:21.121Z" },
{ url = "https://files.pythonhosted.org/packages/c1/ce/9962938e179cf9f699d3f1e7b3114b5d7642dee6a893745229f9dd04f274/charset_normalizer-3.5.1-cp314-cp314-musllinux_1_2_armv7l.whl", hash = "sha256:329fc3ccb63ad22d867d84c2adea759a64079a37ba4a343433b02c7a2816871e", size = 231376, upload-time = "2026-08-15T08:18:22.57Z" },
{ url = "https://files.pythonhosted.org/packages/85/54/46000450ada53bd9eac5429a2c8c54cd2d9b39c0c255f229aea9af0948a5/charset_normalizer-3.5.1-cp314-cp314-musllinux_1_2_ppc64le.whl", hash = "sha256:bb57753e36e4855b8ca375069482250a6246372331a3e4f3407eaebb007443f5", size = 266715, upload-time = "2026-08-15T08:18:24.235Z" },
{ url = "https://files.pythonhosted.org/packages/3d/bb/618749d70f792b44252a777bf89bfb86823b9bbc1ea13fe8ce759b07f38a/charset_normalizer-3.5.1-cp314-cp314-musllinux_1_2_riscv64.whl", hash = "sha256:fce8cbd4997efeb450bd298b54f755dcdff18d496f7a5ddbb4867c6d7c88fdc3", size = 245848, upload-time = "2026-08-15T08:18:25.726Z" },
{ url = "https://files.pythonhosted.org/packages/7e/3f/ffb64458527c7668031d5eb095d978de561958dc9f5b53f8e488a533e603/charset_normalizer-3.5.1-cp314-cp314-musllinux_1_2_s390x.whl", hash = "sha256:6c9cdde8becb25a7fde49924511aa2644d6f8081cc8df8e9452724303348d8e3", size = 264521, upload-time = "2026-08-15T08:18:27.193Z" },
{ url = "https://files.pythonhosted.org/packages/4f/ab/74a55fd803916a35ac461daf002708191aac19b546b80dc8cabfedc63d98/charset_normalizer-3.5.1-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:9ac4444d8d4fd4c4bd08bf451ed3167aa9e7ec6cdb41b648794f1d1103652e36", size = 253054, upload-time = "2026-08-15T08:18:28.568Z" },
{ url = "https://files.pythonhosted.org/packages/a0/2a/6a9034b7d3c60b17499afb482df5878bf9fa20b50cc3887d5ef017a833db/charset_normalizer-3.5.1-cp314-cp314-pyemscripten_2026_0_wasm32.whl", hash = "sha256:f03ac127268b43ef4fe9e6ab6794a6794b49485a0cc0c1db79876d2f33f75bc7", size = 140580, upload-time = "2026-08-15T08:18:30.214Z" },
{ url = "https://files.pythonhosted.org/packages/f3/46/1d362e1a00d035d66b9869e1281eee115907f7e390a16a07824ab5737360/charset_normalizer-3.5.1-cp314-cp314-win32.whl", hash = "sha256:1f5883d77fd409a261abb5dc8ccbe335720d798b1de4abb3b1d47ccbbc76b53b", size = 180325, upload-time = "2026-08-15T08:18:31.877Z" },
{ url = "https://files.pythonhosted.org/packages/7a/7c/4938c329b6a9d446f6a59aa2092ff7118f274209b5ed0e26893d1d30a63c/charset_normalizer-3.5.1-cp314-cp314-win_amd64.whl", hash = "sha256:c658c50ac0c98cd755a2dd50b7977d3bca7df401dcc47fbdfa87db53ef7d4e8b", size = 204175, upload-time = "2026-08-15T08:18:33.466Z" },
{ url = "https://files.pythonhosted.org/packages/ac/33/eeb384dbd8dec570661354592f4f2e1b2fcc92585624d146a000caf53841/charset_normalizer-3.5.1-cp314-cp314-win_arm64.whl", hash = "sha256:4bea7f8ebe90bbd7f0e4a2de42ca6924ba23e3e76418c408ff82f1d46fabd687", size = 184123, upload-time = "2026-08-15T08:18:34.913Z" },
{ url = "https://files.pythonhosted.org/packages/1c/6c/c73fa9d5a85f6ab05395de61c5f6984e0a9ff40bb5ff888d46dff02526c6/charset_normalizer-3.5.1-cp314-cp314t-macosx_10_15_universal2.whl", hash = "sha256:fbc597639158fd7c14d55e808718848319540f51b0e6746e3eefa59723a4a348", size = 381682, upload-time = "2026-08-15T08:18:36.349Z" },
{ url = "https://files.pythonhosted.org/packages/30/c7/63565f860921457feba93bae6c86fb7746deb4cffeed2f375cb845318146/charset_normalizer-3.5.1-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:e71c909f353863b2b89c83de2ebed71ea6d0df8a6ef65a128193c5e650766bef", size = 240826, upload-time = "2026-08-15T08:18:37.887Z" },
{ url = "https://files.pythonhosted.org/packages/06/ae/7ae8807410dfa33f8e6f1715740adeaafa8a816cc4cb33508f54b1f7c896/charset_normalizer-3.5.1-cp314-cp314t-manylinux2014_armv7l.manylinux_2_17_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:7ac76cf9afd34929d76eb7fcb63be476a4853d8a96f0dcf2d0db68a0cbdf9885", size = 227861, upload-time = "2026-08-15T08:18:39.315Z" },
{ url = "https://files.pythonhosted.org/packages/e9/a3/887c1642f0da26000b0e0652d91071113c0e72cea33952e225cf589f49a9/charset_normalizer-3.5.1-cp314-cp314t-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:a3a370082ce34d0612f421e15fe011c53bb1feff21a26d06ad4fb244dab5a375", size = 260758, upload-time = "2026-08-15T08:18:40.88Z" },
{ url = "https://files.pythonhosted.org/packages/3e/11/e6f5b9a3d0e55b0ef7505cd3765cdd48f22db89994c947b316f52f801fd8/charset_normalizer-3.5.1-cp314-cp314t-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:256dd4d85d9e4dc595e2bc983c980e73f62ddeb3165c58b4c3dfe78c5c8548c1", size = 259950, upload-time = "2026-08-15T08:18:42.351Z" },
{ url = "https://files.pythonhosted.org/packages/1b/ee/e4e10a94d51cd1ee638aa7e00b65399e6b2a4e8376ab6d2eac9f95586671/charset_normalizer-3.5.1-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:58d4aa13a59c969dbfdf9e6a9560e242cbfd9e8a8f50c2747714df1a423adf65", size = 249329, upload-time = "2026-08-15T08:18:43.914Z" },
{ url = "https://files.pythonhosted.org/packages/c4/25/d5f4198819e6059735a84e8d0bfb72dc33976da67b97adcd3fb5a5e07ec6/charset_normalizer-3.5.1-cp314-cp314t-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:0c6dfb5ca6723eeed15aa8e564a014d69fcb8812f94eef11fe3631e0508199f5", size = 243137, upload-time = "2026-08-15T08:18:45.368Z" },
{ url = "https://files.pythonhosted.org/packages/a5/e9/e925ca7569cf9fb9701fd82503fee73eea5268fdb856bdd64947092d3daa/charset_normalizer-3.5.1-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:c010f5581d9c612804cc59fcf7b524b707fbcb72828551237ab545bb5c7034af", size = 242820, upload-time = "2026-08-15T08:18:46.842Z" },
{ url = "https://files.pythonhosted.org/packages/34/17/672c251a888ed2aebcdd2fe830ad0104e25ff83c43f5c4f9c15e9fc6853c/charset_normalizer-3.5.1-cp314-cp314t-musllinux_1_2_armv7l.whl", hash = "sha256:52ec005752a56ae79547a05c0139ca2501a0c866390b6115008456b9f0e7cde1", size = 230504, upload-time = "2026-08-15T08:18:48.353Z" },
{ url = "https://files.pythonhosted.org/packages/3f/fc/f6a85abebd42ce4da2f1db0aa56cc6a0df1995e318b3875d14401b8381d1/charset_normalizer-3.5.1-cp314-cp314t-musllinux_1_2_ppc64le.whl", hash = "sha256:2bced4061f000f7187254a02ad3433ae17eaf991747ceea2f478422590a5bba9", size = 263087, upload-time = "2026-08-15T08:18:49.859Z" },
{ url = "https://files.pythonhosted.org/packages/98/66/7c42677e739ba66746b297e2046918d793078094dc239e1e72768cffccc6/charset_normalizer-3.5.1-cp314-cp314t-musllinux_1_2_riscv64.whl", hash = "sha256:9eea3ab2597a5e65fe65296e2d6a84570845a6b55532d90333d740d48bbc850a", size = 243269, upload-time = "2026-08-15T08:18:51.601Z" },
{ url = "https://files.pythonhosted.org/packages/de/d8/a50b79237f417af10f8c2a501ce8d1ca87829a22e69117891ca4ba20a69e/charset_normalizer-3.5.1-cp314-cp314t-musllinux_1_2_s390x.whl", hash = "sha256:496846868fea80e479324862fa877f02411f2fd0f83b79ccee2607aa68b2a032", size = 258766, upload-time = "2026-08-15T08:18:53.23Z" },
{ url = "https://files.pythonhosted.org/packages/2e/1d/0fc91aeaeb3c83b748f532399ce67cf84604b48297405d740000f7a9e786/charset_normalizer-3.5.1-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:85d5855daafc240cc045c026d7a15fd198a09b0fc8ff6f5ecbb5297b509cb11e", size = 250814, upload-time = "2026-08-15T08:18:54.768Z" },
{ url = "https://files.pythonhosted.org/packages/ae/10/3d8c777cf9024615295aa1b808324ad5b4a77855869c00824bad74ffaf8a/charset_normalizer-3.5.1-cp314-cp314t-win32.whl", hash = "sha256:58d3e12c88e0950bca850ae1f7c256055c097639c2edb9eb123af9807d8b15e4", size = 191074, upload-time = "2026-08-15T08:18:56.305Z" },
{ url = "https://files.pythonhosted.org/packages/4d/81/ae557d3c44d1a1d688696d60563413a0866a91b7ebc50f20df838be3d8c8/charset_normalizer-3.5.1-cp314-cp314t-win_amd64.whl", hash = "sha256:acaf604462bf330b0d07e7a07c1d6e4adac79e5fb13e9c5140590542cafacc00", size = 216476, upload-time = "2026-08-15T08:18:57.889Z" },
{ url = "https://files.pythonhosted.org/packages/27/e9/61c01fb8b804692569c036b3fc50495814502dcf13a60649c6055390b02c/charset_normalizer-3.5.1-cp314-cp314t-win_arm64.whl", hash = "sha256:fdb8a068947befafba9952162645dc2fecaeb400e64584829ed5e9b2fbe21a7f", size = 194115, upload-time = "2026-08-15T08:18:59.418Z" },
{ url = "https://files.pythonhosted.org/packages/4a/4e/8544831ef59d8f27ce92c80871380fdacc8076a8a56ed62f82e54f991333/charset_normalizer-3.5.1-cp315-cp315-macosx_10_15_universal2.whl", hash = "sha256:9085f87b0e38a2b92b8923059b4e8789fe40d9279712d15dcc670048d77079af", size = 342048, upload-time = "2026-08-15T08:19:01.054Z" },
{ url = "https://files.pythonhosted.org/packages/7f/a6/e3b46852424246065355644f4fb6dbccc0239a42a2eee27ecfc8957f0bcd/charset_normalizer-3.5.1-cp315-cp315-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:2679de311c7946dde5d3b6f44941844133ff5c7cb86099c0061ab1e8901c20a8", size = 242997, upload-time = "2026-08-15T08:19:02.492Z" },
{ url = "https://files.pythonhosted.org/packages/03/3b/0cc9a26777334ab2f2e3089b948bbf4e4fe72ea70b897715ef6415043ec8/charset_normalizer-3.5.1-cp315-cp315-manylinux2014_armv7l.manylinux_2_17_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:baf3775a2635e5a11fbd5e4e64ee69c7e86875d224a5c72aca4c141064589a90", size = 237014, upload-time = "2026-08-15T08:19:03.943Z" },
{ url = "https://files.pythonhosted.org/packages/8c/c2/027335f0aa337a2a2e121bac1ad88c4f02ba6053ea0926802784f3db11af/charset_normalizer-3.5.1-cp315-cp315-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:8ac8c94b6539074e0f40899301273ac8402b9b3e01c7b7ba269ff30340aaaf20", size = 266174, upload-time = "2026-08-15T08:19:05.598Z" },
{ url = "https://files.pythonhosted.org/packages/86/d3/e367787febe4e74769dec0f406f2c3c8d1b955fce5aee1fd0f94e8367a45/charset_normalizer-3.5.1-cp315-cp315-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:8fe532b3c966d1fb794e0698e4589d0444017ae77fc0b31edea13c0e35bcc449", size = 263361, upload-time = "2026-08-15T08:19:07.251Z" },
{ url = "https://files.pythonhosted.org/packages/af/3d/391b193eb9f3e84b02f9314088c386debdc0debee843535aaea2e2c6715d/charset_normalizer-3.5.1-cp315-cp315-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:5c84bec0ab5ae0c64bfe73a7d2adcb5ce73b467523fc27fd6a28ab2aa6cbe35a", size = 252143, upload-time = "2026-08-15T08:19:08.816Z" },
{ url = "https://files.pythonhosted.org/packages/2e/57/de221f1745a90d418199761967e2776bfe2c275a1194220985e8c1d37833/charset_normalizer-3.5.1-cp315-cp315-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:854066be00447fa8de2ccbbe893e2ffc4b123ef16d897af794c1e18bd4a714b0", size = 252086, upload-time = "2026-08-15T08:19:10.255Z" },
{ url = "https://files.pythonhosted.org/packages/c8/e3/d119f86a01f9331e8186175f24873b1d74a7ee9e2e4b4d68f9947dae5afd/charset_normalizer-3.5.1-cp315-cp315-musllinux_1_2_aarch64.whl", hash = "sha256:21b82d8082f6f5e7f456ef0bd16323d08de1266efbfeb476e64b2a91d1471a4e", size = 245231, upload-time = "2026-08-15T08:19:11.807Z" },
{ url = "https://files.pythonhosted.org/packages/26/de/d8e48c135ae480879539cdb179c8d3b50c7879497d75dd899b5763b69cee/charset_normalizer-3.5.1-cp315-cp315-musllinux_1_2_armv7l.whl", hash = "sha256:838648accb3a7fd9803fd45c87bce8509648eb0c11bc34e216141300977244f2", size = 241546, upload-time = "2026-08-15T08:19:13.416Z" },
{ url = "https://files.pythonhosted.org/packages/67/c4/217755fd1abc50d326c252922cd642002758095a81ff45010337b8b3ef65/charset_normalizer-3.5.1-cp315-cp315-musllinux_1_2_ppc64le.whl", hash = "sha256:195ce897c6153c0700078142cf8efe3e6454ca4cf4357499e4078dfd83396626", size = 267033, upload-time = "2026-08-15T08:19:14.981Z" },
{ url = "https://files.pythonhosted.org/packages/b8/d7/34d8e404e358d2adcc5a228c2134643af00104c8fb0bf525f3688d756f05/charset_normalizer-3.5.1-cp315-cp315-musllinux_1_2_riscv64.whl", hash = "sha256:978eab16f55b4ab2c2a745be9a0a840bf8f09a7f227d9c76eb30214d078865a5", size = 252045, upload-time = "2026-08-15T08:19:16.618Z" },
{ url = "https://files.pythonhosted.org/packages/5e/fa/40414471acf0aa0692ca77305aa00e434fcd8288f0941c93c30e9a5f8f2f/charset_normalizer-3.5.1-cp315-cp315-musllinux_1_2_s390x.whl", hash = "sha256:cc0329df4caaceb950d2f580b5ac716a377f7059624a0bafaeaf8a218c6ed774", size = 264866, upload-time = "2026-08-15T08:19:18.101Z" },
{ url = "https://files.pythonhosted.org/packages/32/90/fcc850bae791abd2e0c041847f13e270aa08692a79f3e00de6d2dce1cb50/charset_normalizer-3.5.1-cp315-cp315-musllinux_1_2_x86_64.whl", hash = "sha256:687c9ca3035544b113bea2055e180af96fb63c0c476e22a9180f51925186e7b7", size = 253932, upload-time = "2026-08-15T08:19:19.734Z" },
{ url = "https://files.pythonhosted.org/packages/af/af/53afe99068b3c10b4cbae592a52ef72a7c92c0188440e83ee3a078fd8f75/charset_normalizer-3.5.1-cp315-cp315-win32.whl", hash = "sha256:706bfd38730a5ac7a365793269a00f4e988178cec121391f4248d84ad8c972e9", size = 180320, upload-time = "2026-08-15T08:19:21.37Z" },
{ url = "https://files.pythonhosted.org/packages/c9/bc/f46a132041b29e4a8779ed712d3df1bf112e94ca8de58b66d7ec2c0cf8b9/charset_normalizer-3.5.1-cp315-cp315-win_amd64.whl", hash = "sha256:92caef967d287a407085d61176fce4012b1dd62daed4eb6d5ceb26d3d2538712", size = 204174, upload-time = "2026-08-15T08:19:23.088Z" },
{ url = "https://files.pythonhosted.org/packages/a1/5d/9ed554480eda8e447b673648628fdc29574d23dbad01fe11837adedd1cae/charset_normalizer-3.5.1-cp315-cp315-win_arm64.whl", hash = "sha256:5fc45d653ea8c9a20479167e11d4a0f8cb2fa3470737ab6f9c827532313187b7", size = 184126, upload-time = "2026-08-15T08:19:24.471Z" },
{ url = "https://files.pythonhosted.org/packages/3b/32/9b8929bf384061ee1fe5d9c27c6f9776d3d824039ad4e14c88ec00c7808e/charset_normalizer-3.5.1-cp315-cp315t-macosx_10_15_universal2.whl", hash = "sha256:59171c6e45bf07d0d5cab3b0bf81d945035530f6873398b3b531c31184d46663", size = 381441, upload-time = "2026-08-15T08:19:26.038Z" },
{ url = "https://files.pythonhosted.org/packages/96/10/e9aa7923d3ddac652c99a1c5f7be494e737e151566a44abe018daf757f2c/charset_normalizer-3.5.1-cp315-cp315t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:9dbdd9205662134957cf0c324f639bdc5031c0ca056e2369e238db75187c0f11", size = 241742, upload-time = "2026-08-15T08:19:27.532Z" },
{ url = "https://files.pythonhosted.org/packages/28/53/a2d249ebddf47b889a100c0bdcb61a2f9dbb8bc24ef325cc062e4f476877/charset_normalizer-3.5.1-cp315-cp315t-manylinux2014_armv7l.manylinux_2_17_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:e4b018dc5a0eee4676e38fe84a47a427816c590b93b55d9025274ec4d6ffc2dc", size = 235298, upload-time = "2026-08-15T08:19:29.274Z" },
{ url = "https://files.pythonhosted.org/packages/7d/07/469f78af590f7d5cd48e20d8dbfa3d66deeff9ba37768c04d886b5afd45c/charset_normalizer-3.5.1-cp315-cp315t-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:ced3fdd71aaa83ce593746c2edb42b7a59cb4c19c8b5c407781c72e493aae55a", size = 262500, upload-time = "2026-08-15T08:19:30.955Z" },
{ url = "https://files.pythonhosted.org/packages/55/66/3bb56a47f7dcba014055b1a1d33c6f08bbe9c1e74dba154cfa25f90ae885/charset_normalizer-3.5.1-cp315-cp315t-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:19a3dd5aa73cef1c99687c4fc57db016a9c17104ae1185da88ba566a5d3bebe4", size = 258888, upload-time = "2026-08-15T08:19:32.458Z" },
{ url = "https://files.pythonhosted.org/packages/ff/c1/2adc2800903fb013210349313b710a5376856578d9e33e6b9a1d8b36714a/charset_normalizer-3.5.1-cp315-cp315t-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:cc5d36d96478aa9c60654bd932525bf32964c62a7281eafdf16d85003a8d6004", size = 250243, upload-time = "2026-08-15T08:19:33.94Z" },
{ url = "https://files.pythonhosted.org/packages/95/b5/a18d0dd1157ab655cc2cb14a545f4a4784bbad70ab3502412e36097502d9/charset_normalizer-3.5.1-cp315-cp315t-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:04368edf83514385ffc3e1cfd4546e595f4f1272dd23ba437a93a9cc3741d47b", size = 249871, upload-time = "2026-08-15T08:19:35.413Z" },
{ url = "https://files.pythonhosted.org/packages/ad/c3/525f508cd1e58d0450ac55ed40ac75bc3a97482c59def5278456a5fbf03c/charset_normalizer-3.5.1-cp315-cp315t-musllinux_1_2_aarch64.whl", hash = "sha256:9b5db6052055d34d41230fb78d7c439c23dc536a9896f6cb039e8dd92cfc1263", size = 243580, upload-time = "2026-08-15T08:19:36.886Z" },
{ url = "https://files.pythonhosted.org/packages/7c/c1/49a91fe7e97c8140094ca5c64161ab623a70d9f636bf834eace14048acb5/charset_normalizer-3.5.1-cp315-cp315t-musllinux_1_2_armv7l.whl", hash = "sha256:252d099029bcbea642f2a06c4ed5046bdf8b5a8150b64afa5e027e88b106e5ee", size = 239807, upload-time = "2026-08-15T08:19:38.392Z" },
{ url = "https://files.pythonhosted.org/packages/d3/58/56a48c296601274c4689b864a8e2dfb209b81dfcb39472753ce95eea662b/charset_normalizer-3.5.1-cp315-cp315t-musllinux_1_2_ppc64le.whl", hash = "sha256:6199d5606e2bbf2b096cf64d03f8b6790c91081d5ac866b8e7bb6422738cc60c", size = 264083, upload-time = "2026-08-15T08:19:39.856Z" },
{ url = "https://files.pythonhosted.org/packages/10/4c/dc48409274a1817ff349711d26c62aa0c597df865d4d69ef79160c859193/charset_normalizer-3.5.1-cp315-cp315t-musllinux_1_2_riscv64.whl", hash = "sha256:77efcff2b23071c349402ac1066667a3d011f62398d81408c9b88ad991747c9e", size = 250317, upload-time = "2026-08-15T08:19:41.53Z" },
{ url = "https://files.pythonhosted.org/packages/81/58/d325912115caec62d6bdd77bbab5e0b7da5d234a9f20affdffcbcb530d0b/charset_normalizer-3.5.1-cp315-cp315t-musllinux_1_2_s390x.whl", hash = "sha256:a5cbd90ecf0fc62e64726917ad083b73001f0563657a87ec3c0b504e277dc90d", size = 258173, upload-time = "2026-08-15T08:19:43.07Z" },
{ url = "https://files.pythonhosted.org/packages/34/f7/b13b1ccae2c8ec63980d13be1890eb73f8aeabbfce02a24aabc0908788f5/charset_normalizer-3.5.1-cp315-cp315t-musllinux_1_2_x86_64.whl", hash = "sha256:4d26f14f041e83dd8edfd61f4cd4fa7285d31798b5bf1f28e70c367ba6c41d61", size = 251960, upload-time = "2026-08-15T08:19:44.587Z" },
{ url = "https://files.pythonhosted.org/packages/1e/25/ed3f9919c5aef8cc818be1f972f565f7610d7b2076b8ebb98839516ffc3c/charset_normalizer-3.5.1-cp315-cp315t-win32.whl", hash = "sha256:ac13b004224fb341e1e25a1ed5e19d32f57cdb2a403e01f003b46f051a550f6f", size = 191186, upload-time = "2026-08-15T08:19:46.293Z" },
{ url = "https://files.pythonhosted.org/packages/69/d5/43c2b3e9d8267092b913eb8b0603f0f71993c395632886bd37a7223f96cf/charset_normalizer-3.5.1-cp315-cp315t-win_amd64.whl", hash = "sha256:35aea775dc2bd5f54cd84a1cd2696cc3207c479cb9cf0bd346f0d343e4300ddb", size = 215947, upload-time = "2026-08-15T08:19:47.853Z" },
{ url = "https://files.pythonhosted.org/packages/a8/76/9aad3e9c8865e5e0efa9a7f6f81c37a67635a985145ecd44528a81e088ee/charset_normalizer-3.5.1-cp315-cp315t-win_arm64.whl", hash = "sha256:fb78f6e7fcd8ad785d28cd577168bc1aaee827b25bb8755638f694794ea98f0a", size = 193909, upload-time = "2026-08-15T08:19:49.383Z" },
{ url = "https://files.pythonhosted.org/packages/5b/97/fb4e82231aba271ffd775a1b4993b0defc4e3059f286ae41d9433409fe85/charset_normalizer-3.5.1-cp37-abi3-macosx_10_9_universal2.whl", hash = "sha256:41876ee62a3dddf48ff1121ad8f0798032aa03f2fd35f21f34a4cab14f18d8d2", size = 331467, upload-time = "2026-08-15T08:19:50.959Z" },
{ url = "https://files.pythonhosted.org/packages/9f/2f/fe3f187327aac18e2d54e9d2b08e15d27bf9b642d9e51c219f130fc34d1a/charset_normalizer-3.5.1-cp37-abi3-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl", hash = "sha256:a6dac12ff6b846103483683f60c5f8fee205121adc58ffd87e90a90a3af69e99", size = 253057, upload-time = "2026-08-15T08:19:52.654Z" },
{ url = "https://files.pythonhosted.org/packages/d7/c7/9e48cee5c161fe24da823b61bf381921d77cb994a0a4de148e95018c1984/charset_normalizer-3.5.1-cp37-abi3-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:cee5dd7c6fb5dd52a0fe2a740f9bc6e3593f5f8b1788bde49de02086f30182b2", size = 240930, upload-time = "2026-08-15T08:19:54.163Z" },
{ url = "https://files.pythonhosted.org/packages/49/e0/716601f3cc69be7b198951150c75ead1ece33c3c8036ff6ffa46029659a0/charset_normalizer-3.5.1-cp37-abi3-manylinux2014_armv7l.manylinux_2_17_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:343fb4f2821043bd87095f7b08a1a181febc8e36ac64212143bbfd0a0e1bc235", size = 230822, upload-time = "2026-08-15T08:19:55.807Z" },
{ url = "https://files.pythonhosted.org/packages/d3/05/71bfc5caa0abcc45aea1f6a4d50ac68e59605ddc7666fe8494f4cd229665/charset_normalizer-3.5.1-cp37-abi3-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:ae4a097991662cd4fff0ddc74e0fe7874f82e00042fa0ea00855645ed0c79598", size = 260037, upload-time = "2026-08-15T08:19:57.312Z" },
{ url = "https://files.pythonhosted.org/packages/c3/92/de7e32ed05341e7a9c4c877c318418197b7f2d66a3b68d561bf2ac57ca3e/charset_normalizer-3.5.1-cp37-abi3-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:4b599739b93b2cbeded49645ae3c8d1405c29ddfbceac1545c87a3f9580a9e96", size = 255097, upload-time = "2026-08-15T08:19:59.056Z" },
{ url = "https://files.pythonhosted.org/packages/f5/7b/ade0a122600319dfa0b1000ab0f9731c94a817904cf3c5de408c73a4ede7/charset_normalizer-3.5.1-cp37-abi3-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:b39b69b347e5e47a3b5b8cfc005c68c1ba347474e3960236c4944a8ecd174962", size = 250166, upload-time = "2026-08-15T08:20:00.612Z" },
{ url = "https://files.pythonhosted.org/packages/75/9c/019fbb9f4834491a160951349b1a3714439376f66e5f7cf18b4f18f0c7aa/charset_normalizer-3.5.1-cp37-abi3-musllinux_1_2_aarch64.whl", hash = "sha256:a2028475ba855475b8b4d3cfeb4994269c967aea8b9892dfba907f4263a863a3", size = 241821, upload-time = "2026-08-15T08:20:02.321Z" },
{ url = "https://files.pythonhosted.org/packages/2b/b8/11d4840bfc99330cc7fbcc2681ee5a044553a6e77655508d8f9b2bff7b34/charset_normalizer-3.5.1-cp37-abi3-musllinux_1_2_armv7l.whl", hash = "sha256:36047af20e17097c3bb9476c2b7655f2f7aa51322c0ba58c07695bedf755a950", size = 232529, upload-time = "2026-08-15T08:20:04.008Z" },
{ url = "https://files.pythonhosted.org/packages/18/96/2b3a21492d9f65171ac75d872f5018260013d00bfa0ff70ec9f179148cbd/charset_normalizer-3.5.1-cp37-abi3-musllinux_1_2_ppc64le.whl", hash = "sha256:4c4fb141a727957c93edfe5c32a26ceb6b5f6461d67146e2d39f51e16170bea8", size = 260348, upload-time = "2026-08-15T08:20:05.877Z" },
{ url = "https://files.pythonhosted.org/packages/d6/aa/a69a2028e8bd052476c245460ab19d7de595de084dd968f2d75cd50c3e25/charset_normalizer-3.5.1-cp37-abi3-musllinux_1_2_riscv64.whl", hash = "sha256:2f293479cce755c75f1697e87c409b7ae4c555c7dfecb6e988ad13abba943031", size = 247234, upload-time = "2026-08-15T08:20:07.487Z" },
{ url = "https://files.pythonhosted.org/packages/35/8a/3d130aeabcaf3d2466af76b7b141c08d9e89c9016ab4b7cdd0f7dc2d1c62/charset_normalizer-3.5.1-cp37-abi3-musllinux_1_2_s390x.whl", hash = "sha256:3588e376b3ea2eea84976f67273d679f229e24c66dce7b82ae45aef04ff6e072", size = 256917, upload-time = "2026-08-15T08:20:09.142Z" },
{ url = "https://files.pythonhosted.org/packages/80/c2/a7379b840292d0c1ab9fbd17d1f3967aa81794dc95bc74be8999d7fedcf7/charset_normalizer-3.5.1-cp37-abi3-musllinux_1_2_x86_64.whl", hash = "sha256:e199fb99720074809a7720f1c0b4d919eea8b87e88713e0f8f602f7bef543d9d", size = 254846, upload-time = "2026-08-15T08:20:10.727Z" },
{ url = "https://files.pythonhosted.org/packages/01/65/d43b714731bb2f40d4053dfa00ecfc1c5a301f8e3316c5db3a09af59fe94/charset_normalizer-3.5.1-cp37-abi3-win32.whl", hash = "sha256:dd732602a7009217f658d5863d12d79d373a4de0eebc111094bcdd3bb8e0a6cc", size = 174216, upload-time = "2026-08-15T08:20:12.334Z" },
{ url = "https://files.pythonhosted.org/packages/35/4f/b911ed898b26a09789eba9c9200c999aff6c61b4bafaf4838e56d1a1e1a3/charset_normalizer-3.5.1-cp37-abi3-win_amd64.whl", hash = "sha256:70055ff39b97c99e7ae40ea3e393fb62aa2e44dbd9b29f8d14f42fb0025c3959", size = 199764, upload-time = "2026-08-15T08:20:13.908Z" },
{ url = "https://files.pythonhosted.org/packages/f0/a7/920baf467bfd9bf689f3b318340f37aee4572a71f162bd8db51da55ba4fa/charset_normalizer-3.5.1-cp37-abi3-win_arm64.whl", hash = "sha256:87e4f41d375c0b9be2fb5251aee4b8a689169e134535aed81bf085c3b647451e", size = 287318, upload-time = "2026-08-15T08:20:15.551Z" },
{ url = "https://files.pythonhosted.org/packages/cc/61/d01fc49b8dea277640b55a9e15960dbca9fdc8c9fde18e572d39c59f4019/charset_normalizer-3.5.1-py3-none-any.whl", hash = "sha256:6df0ec430f9a831772c23ca5a224cba36517a58a84bb32c32bb59a9fa67c47f6", size = 68658, upload-time = "2026-08-15T08:20:43.306Z" },
]
[[package]]
name = "click"
version = "8.5.0"
@@ -743,94 +597,6 @@ wheels = [
{ url = "https://files.pythonhosted.org/packages/04/96/92447566d16df59b2a776c0fb82dbc4d9e07cd95062562af01e408583fc4/itsdangerous-2.2.0-py3-none-any.whl", hash = "sha256:c6242fc49e35958c8b15141343aa660db5fc54d4f13a1db01a3f5891b98700ef", size = 16234, upload-time = "2024-04-16T21:28:14.499Z" },
]
[[package]]
name = "jinja2"
version = "3.1.6"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "markupsafe" },
]
sdist = { url = "https://files.pythonhosted.org/packages/df/bf/f7da0350254c0ed7c72f3e33cef02e048281fec7ecec5f032d4aac52226b/jinja2-3.1.6.tar.gz", hash = "sha256:0137fb05990d35f1275a587e9aee6d56da821fc83491a0fb838183be43f66d6d", size = 245115, upload-time = "2025-03-05T20:05:02.478Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/62/a1/3d680cbfd5f4b8f15abc1d571870c5fc3e594bb582bc3b64ea099db13e56/jinja2-3.1.6-py3-none-any.whl", hash = "sha256:85ece4451f492d0c13c5dd7c13a64681a86afae63a5f347908daf103ce6d2f67", size = 134899, upload-time = "2025-03-05T20:05:00.369Z" },
]
[[package]]
name = "loguru"
version = "0.7.3"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "colorama", marker = "sys_platform == 'win32'" },
{ name = "win32-setctime", marker = "sys_platform == 'win32'" },
]
sdist = { url = "https://files.pythonhosted.org/packages/3a/05/a1dae3dffd1116099471c643b8924f5aa6524411dc6c63fdae648c4f1aca/loguru-0.7.3.tar.gz", hash = "sha256:19480589e77d47b8d85b2c827ad95d49bf31b0dcde16593892eb51dd18706eb6", size = 63559, upload-time = "2024-12-06T11:20:56.608Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/0c/29/0348de65b8cc732daa3e33e67806420b2ae89bdce2b04af740289c5c6c8c/loguru-0.7.3-py3-none-any.whl", hash = "sha256:31a33c10c8e1e10422bfd431aeb5d351c7cf7fa671e3c4df004162264b28220c", size = 61595, upload-time = "2024-12-06T11:20:54.538Z" },
]
[[package]]
name = "markupsafe"
version = "3.0.3"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/7e/99/7690b6d4034fffd95959cbe0c02de8deb3098cc577c67bb6a24fe5d7caa7/markupsafe-3.0.3.tar.gz", hash = "sha256:722695808f4b6457b320fdc131280796bdceb04ab50fe1795cd540799ebe1698", size = 80313, upload-time = "2025-09-27T18:37:40.426Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/5a/72/147da192e38635ada20e0a2e1a51cf8823d2119ce8883f7053879c2199b5/markupsafe-3.0.3-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:d53197da72cc091b024dd97249dfc7794d6a56530370992a5e1a08983ad9230e", size = 11615, upload-time = "2025-09-27T18:36:30.854Z" },
{ url = "https://files.pythonhosted.org/packages/9a/81/7e4e08678a1f98521201c3079f77db69fb552acd56067661f8c2f534a718/markupsafe-3.0.3-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:1872df69a4de6aead3491198eaf13810b565bdbeec3ae2dc8780f14458ec73ce", size = 12020, upload-time = "2025-09-27T18:36:31.971Z" },
{ url = "https://files.pythonhosted.org/packages/1e/2c/799f4742efc39633a1b54a92eec4082e4f815314869865d876824c257c1e/markupsafe-3.0.3-cp312-cp312-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:3a7e8ae81ae39e62a41ec302f972ba6ae23a5c5396c8e60113e9066ef893da0d", size = 24332, upload-time = "2025-09-27T18:36:32.813Z" },
{ url = "https://files.pythonhosted.org/packages/3c/2e/8d0c2ab90a8c1d9a24f0399058ab8519a3279d1bd4289511d74e909f060e/markupsafe-3.0.3-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:d6dd0be5b5b189d31db7cda48b91d7e0a9795f31430b7f271219ab30f1d3ac9d", size = 22947, upload-time = "2025-09-27T18:36:33.86Z" },
{ url = "https://files.pythonhosted.org/packages/2c/54/887f3092a85238093a0b2154bd629c89444f395618842e8b0c41783898ea/markupsafe-3.0.3-cp312-cp312-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:94c6f0bb423f739146aec64595853541634bde58b2135f27f61c1ffd1cd4d16a", size = 21962, upload-time = "2025-09-27T18:36:35.099Z" },
{ url = "https://files.pythonhosted.org/packages/c9/2f/336b8c7b6f4a4d95e91119dc8521402461b74a485558d8f238a68312f11c/markupsafe-3.0.3-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:be8813b57049a7dc738189df53d69395eba14fb99345e0a5994914a3864c8a4b", size = 23760, upload-time = "2025-09-27T18:36:36.001Z" },
{ url = "https://files.pythonhosted.org/packages/32/43/67935f2b7e4982ffb50a4d169b724d74b62a3964bc1a9a527f5ac4f1ee2b/markupsafe-3.0.3-cp312-cp312-musllinux_1_2_riscv64.whl", hash = "sha256:83891d0e9fb81a825d9a6d61e3f07550ca70a076484292a70fde82c4b807286f", size = 21529, upload-time = "2025-09-27T18:36:36.906Z" },
{ url = "https://files.pythonhosted.org/packages/89/e0/4486f11e51bbba8b0c041098859e869e304d1c261e59244baa3d295d47b7/markupsafe-3.0.3-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:77f0643abe7495da77fb436f50f8dab76dbc6e5fd25d39589a0f1fe6548bfa2b", size = 23015, upload-time = "2025-09-27T18:36:37.868Z" },
{ url = "https://files.pythonhosted.org/packages/2f/e1/78ee7a023dac597a5825441ebd17170785a9dab23de95d2c7508ade94e0e/markupsafe-3.0.3-cp312-cp312-win32.whl", hash = "sha256:d88b440e37a16e651bda4c7c2b930eb586fd15ca7406cb39e211fcff3bf3017d", size = 14540, upload-time = "2025-09-27T18:36:38.761Z" },
{ url = "https://files.pythonhosted.org/packages/aa/5b/bec5aa9bbbb2c946ca2733ef9c4ca91c91b6a24580193e891b5f7dbe8e1e/markupsafe-3.0.3-cp312-cp312-win_amd64.whl", hash = "sha256:26a5784ded40c9e318cfc2bdb30fe164bdb8665ded9cd64d500a34fb42067b1c", size = 15105, upload-time = "2025-09-27T18:36:39.701Z" },
{ url = "https://files.pythonhosted.org/packages/e5/f1/216fc1bbfd74011693a4fd837e7026152e89c4bcf3e77b6692fba9923123/markupsafe-3.0.3-cp312-cp312-win_arm64.whl", hash = "sha256:35add3b638a5d900e807944a078b51922212fb3dedb01633a8defc4b01a3c85f", size = 13906, upload-time = "2025-09-27T18:36:40.689Z" },
{ url = "https://files.pythonhosted.org/packages/38/2f/907b9c7bbba283e68f20259574b13d005c121a0fa4c175f9bed27c4597ff/markupsafe-3.0.3-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:e1cf1972137e83c5d4c136c43ced9ac51d0e124706ee1c8aa8532c1287fa8795", size = 11622, upload-time = "2025-09-27T18:36:41.777Z" },
{ url = "https://files.pythonhosted.org/packages/9c/d9/5f7756922cdd676869eca1c4e3c0cd0df60ed30199ffd775e319089cb3ed/markupsafe-3.0.3-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:116bb52f642a37c115f517494ea5feb03889e04df47eeff5b130b1808ce7c219", size = 12029, upload-time = "2025-09-27T18:36:43.257Z" },
{ url = "https://files.pythonhosted.org/packages/00/07/575a68c754943058c78f30db02ee03a64b3c638586fba6a6dd56830b30a3/markupsafe-3.0.3-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:133a43e73a802c5562be9bbcd03d090aa5a1fe899db609c29e8c8d815c5f6de6", size = 24374, upload-time = "2025-09-27T18:36:44.508Z" },
{ url = "https://files.pythonhosted.org/packages/a9/21/9b05698b46f218fc0e118e1f8168395c65c8a2c750ae2bab54fc4bd4e0e8/markupsafe-3.0.3-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:ccfcd093f13f0f0b7fdd0f198b90053bf7b2f02a3927a30e63f3ccc9df56b676", size = 22980, upload-time = "2025-09-27T18:36:45.385Z" },
{ url = "https://files.pythonhosted.org/packages/7f/71/544260864f893f18b6827315b988c146b559391e6e7e8f7252839b1b846a/markupsafe-3.0.3-cp313-cp313-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:509fa21c6deb7a7a273d629cf5ec029bc209d1a51178615ddf718f5918992ab9", size = 21990, upload-time = "2025-09-27T18:36:46.916Z" },
{ url = "https://files.pythonhosted.org/packages/c2/28/b50fc2f74d1ad761af2f5dcce7492648b983d00a65b8c0e0cb457c82ebbe/markupsafe-3.0.3-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:a4afe79fb3de0b7097d81da19090f4df4f8d3a2b3adaa8764138aac2e44f3af1", size = 23784, upload-time = "2025-09-27T18:36:47.884Z" },
{ url = "https://files.pythonhosted.org/packages/ed/76/104b2aa106a208da8b17a2fb72e033a5a9d7073c68f7e508b94916ed47a9/markupsafe-3.0.3-cp313-cp313-musllinux_1_2_riscv64.whl", hash = "sha256:795e7751525cae078558e679d646ae45574b47ed6e7771863fcc079a6171a0fc", size = 21588, upload-time = "2025-09-27T18:36:48.82Z" },
{ url = "https://files.pythonhosted.org/packages/b5/99/16a5eb2d140087ebd97180d95249b00a03aa87e29cc224056274f2e45fd6/markupsafe-3.0.3-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:8485f406a96febb5140bfeca44a73e3ce5116b2501ac54fe953e488fb1d03b12", size = 23041, upload-time = "2025-09-27T18:36:49.797Z" },
{ url = "https://files.pythonhosted.org/packages/19/bc/e7140ed90c5d61d77cea142eed9f9c303f4c4806f60a1044c13e3f1471d0/markupsafe-3.0.3-cp313-cp313-win32.whl", hash = "sha256:bdd37121970bfd8be76c5fb069c7751683bdf373db1ed6c010162b2a130248ed", size = 14543, upload-time = "2025-09-27T18:36:51.584Z" },
{ url = "https://files.pythonhosted.org/packages/05/73/c4abe620b841b6b791f2edc248f556900667a5a1cf023a6646967ae98335/markupsafe-3.0.3-cp313-cp313-win_amd64.whl", hash = "sha256:9a1abfdc021a164803f4d485104931fb8f8c1efd55bc6b748d2f5774e78b62c5", size = 15113, upload-time = "2025-09-27T18:36:52.537Z" },
{ url = "https://files.pythonhosted.org/packages/f0/3a/fa34a0f7cfef23cf9500d68cb7c32dd64ffd58a12b09225fb03dd37d5b80/markupsafe-3.0.3-cp313-cp313-win_arm64.whl", hash = "sha256:7e68f88e5b8799aa49c85cd116c932a1ac15caaa3f5db09087854d218359e485", size = 13911, upload-time = "2025-09-27T18:36:53.513Z" },
{ url = "https://files.pythonhosted.org/packages/e4/d7/e05cd7efe43a88a17a37b3ae96e79a19e846f3f456fe79c57ca61356ef01/markupsafe-3.0.3-cp313-cp313t-macosx_10_13_x86_64.whl", hash = "sha256:218551f6df4868a8d527e3062d0fb968682fe92054e89978594c28e642c43a73", size = 11658, upload-time = "2025-09-27T18:36:54.819Z" },
{ url = "https://files.pythonhosted.org/packages/99/9e/e412117548182ce2148bdeacdda3bb494260c0b0184360fe0d56389b523b/markupsafe-3.0.3-cp313-cp313t-macosx_11_0_arm64.whl", hash = "sha256:3524b778fe5cfb3452a09d31e7b5adefeea8c5be1d43c4f810ba09f2ceb29d37", size = 12066, upload-time = "2025-09-27T18:36:55.714Z" },
{ url = "https://files.pythonhosted.org/packages/bc/e6/fa0ffcda717ef64a5108eaa7b4f5ed28d56122c9a6d70ab8b72f9f715c80/markupsafe-3.0.3-cp313-cp313t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:4e885a3d1efa2eadc93c894a21770e4bc67899e3543680313b09f139e149ab19", size = 25639, upload-time = "2025-09-27T18:36:56.908Z" },
{ url = "https://files.pythonhosted.org/packages/96/ec/2102e881fe9d25fc16cb4b25d5f5cde50970967ffa5dddafdb771237062d/markupsafe-3.0.3-cp313-cp313t-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:8709b08f4a89aa7586de0aadc8da56180242ee0ada3999749b183aa23df95025", size = 23569, upload-time = "2025-09-27T18:36:57.913Z" },
{ url = "https://files.pythonhosted.org/packages/4b/30/6f2fce1f1f205fc9323255b216ca8a235b15860c34b6798f810f05828e32/markupsafe-3.0.3-cp313-cp313t-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:b8512a91625c9b3da6f127803b166b629725e68af71f8184ae7e7d54686a56d6", size = 23284, upload-time = "2025-09-27T18:36:58.833Z" },
{ url = "https://files.pythonhosted.org/packages/58/47/4a0ccea4ab9f5dcb6f79c0236d954acb382202721e704223a8aafa38b5c8/markupsafe-3.0.3-cp313-cp313t-musllinux_1_2_aarch64.whl", hash = "sha256:9b79b7a16f7fedff2495d684f2b59b0457c3b493778c9eed31111be64d58279f", size = 24801, upload-time = "2025-09-27T18:36:59.739Z" },
{ url = "https://files.pythonhosted.org/packages/6a/70/3780e9b72180b6fecb83a4814d84c3bf4b4ae4bf0b19c27196104149734c/markupsafe-3.0.3-cp313-cp313t-musllinux_1_2_riscv64.whl", hash = "sha256:12c63dfb4a98206f045aa9563db46507995f7ef6d83b2f68eda65c307c6829eb", size = 22769, upload-time = "2025-09-27T18:37:00.719Z" },
{ url = "https://files.pythonhosted.org/packages/98/c5/c03c7f4125180fc215220c035beac6b9cb684bc7a067c84fc69414d315f5/markupsafe-3.0.3-cp313-cp313t-musllinux_1_2_x86_64.whl", hash = "sha256:8f71bc33915be5186016f675cd83a1e08523649b0e33efdb898db577ef5bb009", size = 23642, upload-time = "2025-09-27T18:37:01.673Z" },
{ url = "https://files.pythonhosted.org/packages/80/d6/2d1b89f6ca4bff1036499b1e29a1d02d282259f3681540e16563f27ebc23/markupsafe-3.0.3-cp313-cp313t-win32.whl", hash = "sha256:69c0b73548bc525c8cb9a251cddf1931d1db4d2258e9599c28c07ef3580ef354", size = 14612, upload-time = "2025-09-27T18:37:02.639Z" },
{ url = "https://files.pythonhosted.org/packages/2b/98/e48a4bfba0a0ffcf9925fe2d69240bfaa19c6f7507b8cd09c70684a53c1e/markupsafe-3.0.3-cp313-cp313t-win_amd64.whl", hash = "sha256:1b4b79e8ebf6b55351f0d91fe80f893b4743f104bff22e90697db1590e47a218", size = 15200, upload-time = "2025-09-27T18:37:03.582Z" },
{ url = "https://files.pythonhosted.org/packages/0e/72/e3cc540f351f316e9ed0f092757459afbc595824ca724cbc5a5d4263713f/markupsafe-3.0.3-cp313-cp313t-win_arm64.whl", hash = "sha256:ad2cf8aa28b8c020ab2fc8287b0f823d0a7d8630784c31e9ee5edea20f406287", size = 13973, upload-time = "2025-09-27T18:37:04.929Z" },
{ url = "https://files.pythonhosted.org/packages/33/8a/8e42d4838cd89b7dde187011e97fe6c3af66d8c044997d2183fbd6d31352/markupsafe-3.0.3-cp314-cp314-macosx_10_13_x86_64.whl", hash = "sha256:eaa9599de571d72e2daf60164784109f19978b327a3910d3e9de8c97b5b70cfe", size = 11619, upload-time = "2025-09-27T18:37:06.342Z" },
{ url = "https://files.pythonhosted.org/packages/b5/64/7660f8a4a8e53c924d0fa05dc3a55c9cee10bbd82b11c5afb27d44b096ce/markupsafe-3.0.3-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:c47a551199eb8eb2121d4f0f15ae0f923d31350ab9280078d1e5f12b249e0026", size = 12029, upload-time = "2025-09-27T18:37:07.213Z" },
{ url = "https://files.pythonhosted.org/packages/da/ef/e648bfd021127bef5fa12e1720ffed0c6cbb8310c8d9bea7266337ff06de/markupsafe-3.0.3-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:f34c41761022dd093b4b6896d4810782ffbabe30f2d443ff5f083e0cbbb8c737", size = 24408, upload-time = "2025-09-27T18:37:09.572Z" },
{ url = "https://files.pythonhosted.org/packages/41/3c/a36c2450754618e62008bf7435ccb0f88053e07592e6028a34776213d877/markupsafe-3.0.3-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:457a69a9577064c05a97c41f4e65148652db078a3a509039e64d3467b9e7ef97", size = 23005, upload-time = "2025-09-27T18:37:10.58Z" },
{ url = "https://files.pythonhosted.org/packages/bc/20/b7fdf89a8456b099837cd1dc21974632a02a999ec9bf7ca3e490aacd98e7/markupsafe-3.0.3-cp314-cp314-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:e8afc3f2ccfa24215f8cb28dcf43f0113ac3c37c2f0f0806d8c70e4228c5cf4d", size = 22048, upload-time = "2025-09-27T18:37:11.547Z" },
{ url = "https://files.pythonhosted.org/packages/9a/a7/591f592afdc734f47db08a75793a55d7fbcc6902a723ae4cfbab61010cc5/markupsafe-3.0.3-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:ec15a59cf5af7be74194f7ab02d0f59a62bdcf1a537677ce67a2537c9b87fcda", size = 23821, upload-time = "2025-09-27T18:37:12.48Z" },
{ url = "https://files.pythonhosted.org/packages/7d/33/45b24e4f44195b26521bc6f1a82197118f74df348556594bd2262bda1038/markupsafe-3.0.3-cp314-cp314-musllinux_1_2_riscv64.whl", hash = "sha256:0eb9ff8191e8498cca014656ae6b8d61f39da5f95b488805da4bb029cccbfbaf", size = 21606, upload-time = "2025-09-27T18:37:13.485Z" },
{ url = "https://files.pythonhosted.org/packages/ff/0e/53dfaca23a69fbfbbf17a4b64072090e70717344c52eaaaa9c5ddff1e5f0/markupsafe-3.0.3-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:2713baf880df847f2bece4230d4d094280f4e67b1e813eec43b4c0e144a34ffe", size = 23043, upload-time = "2025-09-27T18:37:14.408Z" },
{ url = "https://files.pythonhosted.org/packages/46/11/f333a06fc16236d5238bfe74daccbca41459dcd8d1fa952e8fbd5dccfb70/markupsafe-3.0.3-cp314-cp314-win32.whl", hash = "sha256:729586769a26dbceff69f7a7dbbf59ab6572b99d94576a5592625d5b411576b9", size = 14747, upload-time = "2025-09-27T18:37:15.36Z" },
{ url = "https://files.pythonhosted.org/packages/28/52/182836104b33b444e400b14f797212f720cbc9ed6ba34c800639d154e821/markupsafe-3.0.3-cp314-cp314-win_amd64.whl", hash = "sha256:bdc919ead48f234740ad807933cdf545180bfbe9342c2bb451556db2ed958581", size = 15341, upload-time = "2025-09-27T18:37:16.496Z" },
{ url = "https://files.pythonhosted.org/packages/6f/18/acf23e91bd94fd7b3031558b1f013adfa21a8e407a3fdb32745538730382/markupsafe-3.0.3-cp314-cp314-win_arm64.whl", hash = "sha256:5a7d5dc5140555cf21a6fefbdbf8723f06fcd2f63ef108f2854de715e4422cb4", size = 14073, upload-time = "2025-09-27T18:37:17.476Z" },
{ url = "https://files.pythonhosted.org/packages/3c/f0/57689aa4076e1b43b15fdfa646b04653969d50cf30c32a102762be2485da/markupsafe-3.0.3-cp314-cp314t-macosx_10_13_x86_64.whl", hash = "sha256:1353ef0c1b138e1907ae78e2f6c63ff67501122006b0f9abad68fda5f4ffc6ab", size = 11661, upload-time = "2025-09-27T18:37:18.453Z" },
{ url = "https://files.pythonhosted.org/packages/89/c3/2e67a7ca217c6912985ec766c6393b636fb0c2344443ff9d91404dc4c79f/markupsafe-3.0.3-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:1085e7fbddd3be5f89cc898938f42c0b3c711fdcb37d75221de2666af647c175", size = 12069, upload-time = "2025-09-27T18:37:19.332Z" },
{ url = "https://files.pythonhosted.org/packages/f0/00/be561dce4e6ca66b15276e184ce4b8aec61fe83662cce2f7d72bd3249d28/markupsafe-3.0.3-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:1b52b4fb9df4eb9ae465f8d0c228a00624de2334f216f178a995ccdcf82c4634", size = 25670, upload-time = "2025-09-27T18:37:20.245Z" },
{ url = "https://files.pythonhosted.org/packages/50/09/c419f6f5a92e5fadde27efd190eca90f05e1261b10dbd8cbcb39cd8ea1dc/markupsafe-3.0.3-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:fed51ac40f757d41b7c48425901843666a6677e3e8eb0abcff09e4ba6e664f50", size = 23598, upload-time = "2025-09-27T18:37:21.177Z" },
{ url = "https://files.pythonhosted.org/packages/22/44/a0681611106e0b2921b3033fc19bc53323e0b50bc70cffdd19f7d679bb66/markupsafe-3.0.3-cp314-cp314t-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:f190daf01f13c72eac4efd5c430a8de82489d9cff23c364c3ea822545032993e", size = 23261, upload-time = "2025-09-27T18:37:22.167Z" },
{ url = "https://files.pythonhosted.org/packages/5f/57/1b0b3f100259dc9fffe780cfb60d4be71375510e435efec3d116b6436d43/markupsafe-3.0.3-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:e56b7d45a839a697b5eb268c82a71bd8c7f6c94d6fd50c3d577fa39a9f1409f5", size = 24835, upload-time = "2025-09-27T18:37:23.296Z" },
{ url = "https://files.pythonhosted.org/packages/26/6a/4bf6d0c97c4920f1597cc14dd720705eca0bf7c787aebc6bb4d1bead5388/markupsafe-3.0.3-cp314-cp314t-musllinux_1_2_riscv64.whl", hash = "sha256:f3e98bb3798ead92273dc0e5fd0f31ade220f59a266ffd8a4f6065e0a3ce0523", size = 22733, upload-time = "2025-09-27T18:37:24.237Z" },
{ url = "https://files.pythonhosted.org/packages/14/c7/ca723101509b518797fedc2fdf79ba57f886b4aca8a7d31857ba3ee8281f/markupsafe-3.0.3-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:5678211cb9333a6468fb8d8be0305520aa073f50d17f089b5b4b477ea6e67fdc", size = 23672, upload-time = "2025-09-27T18:37:25.271Z" },
{ url = "https://files.pythonhosted.org/packages/fb/df/5bd7a48c256faecd1d36edc13133e51397e41b73bb77e1a69deab746ebac/markupsafe-3.0.3-cp314-cp314t-win32.whl", hash = "sha256:915c04ba3851909ce68ccc2b8e2cd691618c4dc4c4232fb7982bca3f41fd8c3d", size = 14819, upload-time = "2025-09-27T18:37:26.285Z" },
{ url = "https://files.pythonhosted.org/packages/1a/8a/0402ba61a2f16038b48b39bccca271134be00c5c9f0f623208399333c448/markupsafe-3.0.3-cp314-cp314t-win_amd64.whl", hash = "sha256:4faffd047e07c38848ce017e8725090413cd80cbc23d86e55c587bf979e579c9", size = 15426, upload-time = "2025-09-27T18:37:27.316Z" },
{ url = "https://files.pythonhosted.org/packages/70/bc/6f1c2f612465f5fa89b95bead1f44dcb607670fd42891d8fdcd5d039f4f4/markupsafe-3.0.3-cp314-cp314t-win_arm64.whl", hash = "sha256:32001d6a8fc98c8cb5c947787c5d08b0a50663d139f1305bac5885d98d9b40fa", size = 14146, upload-time = "2025-09-27T18:37:28.327Z" },
]
[[package]]
name = "multidict"
version = "6.9.1"
@@ -996,15 +762,6 @@ wheels = [
{ url = "https://files.pythonhosted.org/packages/63/34/ba1c580383c9eada3711951fef0795c80b829a078d72188184bcab9dd527/packaging-26.3-py3-none-any.whl", hash = "sha256:d7193f7c8e4e93f444fde0262bf90af30e16fa0ad0ad44cb553c87339b23cd1c", size = 129956, upload-time = "2026-08-04T18:15:27.159Z" },
]
[[package]]
name = "platformdirs"
version = "4.11.15"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/5f/e0/7c20b5d0e0f147c40a934e8f9e9f717bd6e3e372c4d58ca5c2fcce2b1652/platformdirs-4.11.15.tar.gz", hash = "sha256:d7419e973b2b740d428200130f80c0e79304d1c71081001db911e15b26a3a6d4", size = 46059, upload-time = "2026-09-26T02:04:58.142Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/84/83/246d3052d043683cc5f400ab5748bd07a99fbdf311dc771cfad7e3f9e7e0/platformdirs-4.11.15-py3-none-any.whl", hash = "sha256:84508421c77d3c6462e71dd8c0d2776c2ace0e94eb006dc4fd383bc1dcf5a176", size = 26689, upload-time = "2026-09-26T02:04:56.71Z" },
]
[[package]]
name = "pluggy"
version = "1.6.0"
@@ -1351,21 +1108,6 @@ wheels = [
{ url = "https://files.pythonhosted.org/packages/dd/b8/d2d6d731733f51684bbf76bf34dab3b70a9148e8f2cef2bb544fccec681a/qrcode-8.2-py3-none-any.whl", hash = "sha256:16e64e0716c14960108e85d853062c9e8bba5ca8252c0b4d0231b9df4060ff4f", size = 45986, upload-time = "2025-05-01T15:44:22.781Z" },
]
[[package]]
name = "requests"
version = "2.34.2"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "certifi" },
{ name = "charset-normalizer" },
{ name = "idna" },
{ name = "urllib3" },
]
sdist = { url = "https://files.pythonhosted.org/packages/ac/c3/e2a2b89f2d3e2179abd6d00ebd70bff6273f37fb3e0cc209f48b39d00cbf/requests-2.34.2.tar.gz", hash = "sha256:f288924cae4e29463698d6d60bc6a4da69c89185ad1e0bcc4104f584e960b9ed", size = 142856, upload-time = "2026-05-14T19:25:27.735Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/a0/f4/c67b0b3f1b9245e8d266f0f112c500d50e5b4e83cb6f3b71b6528104182a/requests-2.34.2-py3-none-any.whl", hash = "sha256:2a0d60c172f83ac6ab31e4554906c0f3b3588d37b5cb939b1c061f4907e278e0", size = 73075, upload-time = "2026-05-14T19:25:26.443Z" },
]
[[package]]
name = "ruff"
version = "0.16.9"
@@ -1434,15 +1176,6 @@ wheels = [
{ url = "https://files.pythonhosted.org/packages/67/81/4add07e5172b7ac40d8ed5ff580409a7801a4fe26d529bdd915401dabfbe/typing_inspection-0.4.4-py3-none-any.whl", hash = "sha256:65b8397ba37ccbce054456aaccddfc91e6e3083c92824df348d96ca832f3f147", size = 14750, upload-time = "2026-08-12T12:37:24.648Z" },
]
[[package]]
name = "urllib3"
version = "2.8.0"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/e3/05/b17359e1cefb4f909b5e40b1b90a496d987258916dbbf88e842c729f510e/urllib3-2.8.0.tar.gz", hash = "sha256:63bf2ead4c879426ebf22ef2a781eeb4aa3b4ae798a0435506f8687fd5bb9b63", size = 458972, upload-time = "2026-09-15T19:29:36.253Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/92/9d/c4e665119135114480843e7ab388fa94d8480650450e6f8e26b70d323a4c/urllib3-2.8.0-py3-none-any.whl", hash = "sha256:0cf3cae568d36aa9576b28dfb35f11328f1cb974ca7647d9475ebb86c75ac6e3", size = 135717, upload-time = "2026-09-15T19:29:34.577Z" },
]
[[package]]
name = "uvicorn"
version = "0.54.0"
@@ -1537,15 +1270,6 @@ wheels = [
{ url = "https://files.pythonhosted.org/packages/46/dd/41c12ae9745d5d72f709333287679d71cbba68ad7678fed872207bba94f4/watchfiles-1.3.0-cp314-cp314t-win_arm64.whl", hash = "sha256:a5b631db08fd3032db57ec6a92d1777cd60f88803287e4e941a4858c53aab1e0", size = 276924, upload-time = "2026-09-21T09:08:48.976Z" },
]
[[package]]
name = "websocket-client"
version = "1.9.2"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/d8/cb/a5abcc2891249f393827c650c6296660ce40374ac22d99ab9aea41f9d2a2/websocket_client-1.9.2.tar.gz", hash = "sha256:0fcb57545848be86992e128218fd96dd87a6769ffdb1a968dff79632b85604d0", size = 84110, upload-time = "2026-08-31T14:08:40.964Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/d5/d2/cc4dc1271e464942db7ee278baae2daa99ee77cb2af744025c04da585a3e/websocket_client-1.9.2-py3-none-any.whl", hash = "sha256:e1a673830a9c7bfa47b1cd3d5e4178f4c9651d80a4eab02c9c23a1c3ec6250ce", size = 95786, upload-time = "2026-08-31T14:08:39.899Z" },
]
[[package]]
name = "websockets"
version = "17.1"
@@ -1675,15 +1399,6 @@ wheels = [
{ url = "https://files.pythonhosted.org/packages/41/63/23572870e01836a98346075b9e17a8bc24a6ddd9800a3204ceee58677f3c/websockets-17.1-py3-none-any.whl", hash = "sha256:f221081107b8c48184d99f7019604486376e7ef826037e70aad6b02540732c23", size = 211134, upload-time = "2026-08-26T17:25:31.397Z" },
]
[[package]]
name = "win32-setctime"
version = "1.2.0"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/b3/8f/705086c9d734d3b663af0e9bb3d4de6578d08f46b1b101c2442fd9aecaa2/win32_setctime-1.2.0.tar.gz", hash = "sha256:ae1fdf948f5640aae05c511ade119313fb6a30d7eabe25fef9764dca5873c4c0", size = 4867, upload-time = "2024-12-07T15:28:28.314Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/e1/07/c6fe3ad3e685340704d314d765b7912993bcb8dc198f0e7a89382d37974b/win32_setctime-1.2.0-py3-none-any.whl", hash = "sha256:95d644c4e708aba81dc3704a116d8cbc974d70b3bdb8be1d150e36be6e9d1390", size = 4083, upload-time = "2024-12-07T15:28:26.465Z" },
]
[[package]]
name = "yarl"
version = "1.25.1"
@@ -1805,15 +1520,13 @@ name = "yunmusic-assistant"
version = "3.0.0"
source = { virtual = "." }
dependencies = [
{ name = "aiofiles" },
{ name = "aiohttp" },
{ name = "aiosmtplib" },
{ name = "aiosqlite" },
{ name = "aria2p" },
{ name = "cryptography" },
{ name = "fastapi" },
{ name = "filetype" },
{ name = "itsdangerous" },
{ name = "jinja2" },
{ name = "mutagen" },
{ name = "python-multipart" },
{ name = "qrcode" },
@@ -1832,15 +1545,13 @@ dev = [
[package.metadata]
requires-dist = [
{ name = "aiofiles", specifier = ">=24.1" },
{ name = "aiohttp", specifier = ">=3.10,<4.0" },
{ name = "aiosmtplib", specifier = ">=5.1.3,<6.0" },
{ name = "aiosqlite", specifier = ">=0.20,<1.0" },
{ name = "aria2p", specifier = ">=0.12.1" },
{ name = "cryptography", specifier = ">=50.0,<51.0" },
{ name = "fastapi", specifier = ">=0.115,<1.0" },
{ name = "filetype", specifier = ">=1.2.0" },
{ name = "itsdangerous", specifier = ">=2.2,<3.0" },
{ name = "jinja2", specifier = ">=3.1,<4.0" },
{ name = "mutagen", specifier = ">=1.48.1" },
{ name = "python-multipart", specifier = ">=0.0.9" },
{ name = "qrcode", specifier = ">=7.0,<8.3" },
+11 -14
View File
@@ -40,7 +40,7 @@ def _make_lifespan(start_monitor: bool):
@asynccontextmanager
async def _lifespan(app: FastAPI):
"""应用生命周期:建表 → 启动任务监控 → 关闭时释放连接池与后台 loop。
"""应用生命周期:建表 → 启动任务监控 → 关闭时释放连接池。
启动必须建表。全新部署时宿主机的 data/ 往往是个空目录(compose 用
`./data:/app/data` 挂载,会盖掉镜像内的任何内容),此时若只等首次查询
@@ -55,20 +55,16 @@ def _make_lifespan(start_monitor: bool):
离线设备、超时任务不再取消、过期任务与下载缓存不再回收、账本孤儿对账
不再执行(预扣未退款的账永久悬挂)。详见 services.tasks.start_task_monitor。
"""
await asyncio.to_thread(auth_system.init_db)
await auth_system.init_db()
if start_monitor:
from services.tasks import start_task_monitor
await asyncio.to_thread(start_task_monitor)
yield
try:
from core import aiohttp_compat, async_bridge, http_client
from core import http_client
await http_client.client.close()
await aiohttp_compat.close()
# 停掉 sync↔async 桥的后台事件循环(daemon 线程不调也能随进程退出,
# 但显式关闭更干净)
async_bridge.shutdown()
except Exception:
pass
@@ -76,11 +72,17 @@ def _make_lifespan(start_monitor: bool):
def _init_audio_secret() -> None:
"""注入音频签名密钥(与 auth_system 同源,令直链 token 与服务端一致)。"""
"""注入音频签名密钥(与 auth_system 同源,令直链 token 与服务端一致)。
create_app 是**同步**的应用工厂入口(可能在无事件循环的模块导入期调用),
而 get_secret_key 已迁移为原生异步——此处用 asyncio.run 起一个临时 loop 完成
一次性的密钥读取(启动期一次性边界,非运行时同步门面)。若已处于事件循环中
无法 run,则跳过注入(由调用方另行初始化)。
"""
try:
import auth_system
init_secret_key(auth_system.get_secret_key())
init_secret_key(asyncio.run(auth_system.get_secret_key()))
except Exception:
pass
@@ -109,11 +111,6 @@ def create_app(api_prefix: str = '', *, start_monitor: bool = True) -> FastAPI:
# 响应侧统一处理:安全头 + 音频直链签名保护 + 版本指纹抹除
app.add_middleware(ResponseHygieneMiddleware)
# 静态资源(/static/site_auth.js 等)——与 Flask static_folder 同目录
_static_dir = Path(__file__).resolve().parent.parent / 'static'
if _static_dir.is_dir():
app.mount('/static', StaticFiles(directory=str(_static_dir)), name='static')
# React 构建产物(vite build → frontend/dist,hash 指纹资源挂 /assets)
_dist_dir = Path(__file__).resolve().parent.parent / 'frontend' / 'dist'
if (_dist_dir / 'assets').is_dir():
+8 -13
View File
@@ -9,7 +9,6 @@
金额相关逻辑一律走 auth_system 既有函数,本模块不新造账务语义。
"""
import asyncio
import logging
import secrets
from collections.abc import AsyncGenerator, Callable
@@ -36,33 +35,29 @@ class Billing:
key = self.ledger_key
if not key:
return
def _do() -> None:
try:
row = auth_system.ledger_get(key)
row = await auth_system.ledger_get(key)
if row and row['status'] == 'open':
auth_system.ledger_settle(key, 0, reason)
await auth_system.ledger_settle(key, 0, reason)
except Exception:
pass
await asyncio.to_thread(_do)
async def settle(self, delivered: int, status: str = 'delivered') -> None:
"""交付结算(json 模式等无 task_id 的场景)。"""
key = self.ledger_key
if not key:
return
try:
await asyncio.to_thread(auth_system.ledger_settle, key, delivered, status)
await auth_system.ledger_settle(key, delivered, status)
except Exception as e:
logger.error('账本结算失败(重启对账兜底): %s %s', key, e)
async def rename_to(self, task_id: str) -> bool:
"""把请求级账本行改挂 task_id(任务接手账目)。无请求级行时开 reserved=0 的账。"""
if not self.ledger_key:
await asyncio.to_thread(auth_system.ledger_open, task_id, self.user.id, 0)
await auth_system.ledger_open(task_id, self.user.id, 0)
return True
return await asyncio.to_thread(auth_system.ledger_rename, self.ledger_key, task_id)
return await auth_system.ledger_rename(self.ledger_key, task_id)
@asynccontextmanager
async def scope(self) -> AsyncGenerator['Billing', None]:
@@ -100,7 +95,7 @@ async def require_billing(
if user is None:
raise AppError('请先登录后再使用解析功能', 401, error_code='unauthorized')
fresh = await asyncio.to_thread(auth_system.get_user_by_id, user.id)
fresh = await auth_system.get_user_by_id(user.id)
if fresh is None:
raise AppError('登录已失效,请重新登录', 401, error_code='unauthorized')
@@ -118,9 +113,9 @@ async def require_billing(
need = 1
key = 'req-' + secrets.token_hex(8)
remaining = await asyncio.to_thread(auth_system.reserve_and_open, fresh['id'], need, key)
remaining = await auth_system.reserve_and_open(fresh['id'], need, key)
if remaining is None:
await asyncio.to_thread(auth_system.add_log, 'fail', fresh['username'], ip,
await auth_system.add_log('fail', fresh['username'], ip,
action or '下载', f'剩余次数不足(本次需 {need} 次)')
raise AppError(f'下载次数已用尽(本次需 {need} 次,当前 {credits} 次),'
'请充值卡密后继续下载', 401, error_code='expired')
+5 -5
View File
@@ -21,10 +21,10 @@ from web.errors import AppError
_SESSION_COOKIE = sess.COOKIE_NAME
def secret_key() -> str:
async def secret_key() -> str:
"""会话签名密钥(密钥来源:data/secret.key)。"""
import auth_system
return auth_system.get_secret_key()
return await auth_system.get_secret_key()
@dataclass
@@ -43,17 +43,17 @@ class SiteUser:
return self.is_admin or self.credits == -1
def _parse_cookie(session_cookie: str | None) -> dict[str, Any] | None:
async def _parse_cookie(session_cookie: str | None) -> dict[str, Any] | None:
# 服务端叠加签发时间校验(SESSION_MAX_AGE):Cookie 自身的 max_age 只约束
# 浏览器,被存下来的旧 Cookie 仍能通过签名校验,故过期必须由服务端把关。
return sess.loads(secret_key(), session_cookie or '', max_age=sess.SESSION_MAX_AGE)
return sess.loads(await secret_key(), session_cookie or '', max_age=sess.SESSION_MAX_AGE)
async def current_user(
session_cookie: Annotated[str | None, Cookie(alias=_SESSION_COOKIE)] = None,
) -> SiteUser | None:
"""当前登录用户(未登录/失效返回 None)。"""
data = _parse_cookie(session_cookie)
data = await _parse_cookie(session_cookie)
uid = sess.get_uid(data)
if not uid:
return None
+30 -11
View File
@@ -6,14 +6,15 @@
- POST/GET /api/v1/global/playlists ↔ Flask /global/playlists(管理员)
- POST/GET /api/v1/global/playlist ↔ Flask /global/playlist(管理员)
"""
import asyncio
import hashlib
import time
from typing import Annotated, Any
from typing import Annotated
from fastapi import APIRouter, Depends, Request
import music_api
from core.state import VIP_CACHE, VIP_PROBE_LOCK, VIP_USER_CACHE
from core.state import VIP_CACHE, VIP_USER_CACHE
from music_api import APIException
from web import ratelimit
from web.deps import SiteUser, current_user, require_admin
@@ -31,8 +32,25 @@ _VIP_NAMES = {
0: '普通用户', 1: '音乐包', 11: '黑胶VIP', 100: '黑胶VIP', 10: '黑胶SVIP', 110: '黑胶SVIP',
}
# 探测单飞:与 core.state 中同一把锁共享,避免并发请求各自串行 8 级探测
_VIP_PROCESS_LOCK: Any = VIP_PROBE_LOCK
# 音质探测单飞锁:probe 一次要打 8 次网易云接口,并发请求各自串行探测会连累
# 服务器账号触发风控,故同一时刻只放一个请求真探测,其余直接拿账号接口的降级结果。
#
# 用 asyncio.Lock:本模块是这把锁**唯一**的消费者(原注释「与 core.state 中同一把
# 锁共享」是 sync 时代残留,当时另有一个线程侧使用者,现已不存在),故从
# core.state 移到这里,也让 core.state 保持「只依赖 threading/collections」。
# 按事件循环隔离:asyncio.Lock 绑定创建它的 loop,应用主 loop 与测试各自的 loop
# 并存,跨 loop 复用会抛 "bound to a different event loop"。
_PROBE_LOCKS: dict[int, asyncio.Lock] = {}
def _probe_lock() -> asyncio.Lock:
"""当前事件循环的探测单飞锁。"""
key = id(asyncio.get_running_loop())
lock = _PROBE_LOCKS.get(key)
if lock is None:
lock = asyncio.Lock()
_PROBE_LOCKS[key] = lock
return lock
def _vip_name(vip_type: int) -> str:
@@ -46,14 +64,15 @@ def _cookies() -> dict[str, str]:
async def _detect_vip(cookies: dict[str, str]) -> tuple[int, str, list[str]]:
"""检测账号会员类型与可解析音质。
单飞锁为 threading.Lock:异步上下文里用 to_thread 有界等待 0.5s,
拿不到锁的请求直接用账号接口给降级结果(不等 8 级探测)。
单飞锁为 asyncio.Lock:有界等待 0.5s,拿不到锁的请求直接用账号接口给
降级结果(不等 8 级探测)。
"""
import asyncio
api = music_api.NeteaseAPI()
acquired = await asyncio.to_thread(_VIP_PROCESS_LOCK.acquire, True, 0.5)
if not acquired:
lock = _probe_lock()
try:
await asyncio.wait_for(lock.acquire(), 0.5)
except TimeoutError:
# 已有请求在探测:不等它(等 8 级探测会拖长响应),直接给降级结果
info = await api.get_account_info(cookies)
if info:
return info.get('vipType', 0), 'account', []
@@ -66,7 +85,7 @@ async def _detect_vip(cookies: dict[str, str]) -> tuple[int, str, list[str]]:
return info.get('vipType', 0), 'account', levels_ok
return probe.get('vip_type', 0), 'probe', levels_ok
finally:
_VIP_PROCESS_LOCK.release()
lock.release()
+10 -8
View File
@@ -11,8 +11,10 @@
鉴权:管理员密码闸(文案"需要管理员权限,请使用管理员账号登录"),
与 auth_system.admin_required 的文案不同,故本模块用本地 admin 闸保持逐字对齐。
说明:QRLoginManager 已异步化(直接 await);QRLoginClient 的 save_cookie/logout/
_get_cookie_info 仍为同步文件操作,经 asyncio.to_thread 执行。
说明:QRLoginManager 已异步化(直接 await)。QRLoginClient 的 save_cookie/logout/
_get_cookie_info 是**微秒级本地文件操作**——读写 1KB 级的 cookie.txt、os.replace、
unlink,全程不碰网络;对它们走 to_thread,线程调度开销比操作本身还大,且并不产生
任何并发收益。故直接同步调用(to_thread 在这里只是给同步 API 套了层异步外壳)。
"""
import asyncio
import logging
@@ -111,12 +113,12 @@ async def admin_qr_check(_admin: AdminUser, request: Request):
cookie = f'MUSIC_U={musu_u};os=pc;appver=8.9.70;'
if cookie_dict.get('__csrf'):
cookie += f"__csrf={cookie_dict['__csrf']};"
saved = await asyncio.to_thread(client.save_cookie, cookie)
saved = client.save_cookie(cookie)
if not saved:
return error('登录成功但 Cookie 保存失败', 500)
_reset_caches()
logger.info('二维码登录成功,Cookie 已自动保存')
info = await asyncio.to_thread(client._get_cookie_info)
info = client._get_cookie_info()
return success({'status': 'success', 'cookie_info': info}, '登录成功,Cookie 已保存')
if code == 801:
return success({'status': 'waiting'}, '等待扫码')
@@ -133,7 +135,7 @@ async def admin_cookie_status(_admin: AdminUser, request: Request):
await ratelimit.guard(request, bucket='free', limit=30, window=60)
try:
client = _admin_client()
info = await asyncio.to_thread(client._get_cookie_info)
info = client._get_cookie_info()
return success(info, '获取 Cookie 状态成功')
except Exception:
return error('服务暂时不可用,请稍后再试', 500)
@@ -145,15 +147,15 @@ async def admin_cookie_delete(_admin: AdminUser, request: Request):
await ratelimit.guard(request, bucket='free', limit=30, window=60)
try:
client = _admin_client()
ok = await asyncio.to_thread(client.logout)
ok = client.logout()
if not ok:
info = await asyncio.to_thread(client._get_cookie_info)
info = client._get_cookie_info()
return error(
f"Cookie 清除失败(文件仍存在:{info.get('file_exists')},"
f"可能被占用或为挂载点,请手动删除)", 500)
_reset_caches()
logger.info('管理员已清除 Cookie')
info = await asyncio.to_thread(client._get_cookie_info)
info = client._get_cookie_info()
return success(info, 'Cookie 已清除')
except Exception:
return error('服务暂时不可用,请稍后再试', 500)
+81 -60
View File
@@ -3,15 +3,20 @@
鉴权:与 auth_system.admin_required 同口径(403 + error_code='forbidden')。
说明:这些端点多为对 auth_system 同步函数(SQLite / SMTP / 文件)的薄封装,
统一经 asyncio.to_thread 执行以不阻塞事件循环;操作者名取当前管理员用户名。
说明:这些端点多为对 auth_system 的薄封装。auth_system 已迁移为**原生异步**,
故直接 await。PBKDF2 校验属 CPU 密集且 CPython 在迭代里**释放 GIL**,是这里唯一
真正值得占用线程的活(唯有 to_thread 下它才是真并行)。本地文件操作的归位口径是
「重活留线程、微秒级直接同步」:备份上传是整库体量,按块流式落盘、仍走 to_thread;
stat / unlink 这类元数据操作,线程调度开销比操作本身还大,直接在事件循环里做完。
操作者名取当前管理员用户名。
"""
import asyncio
import math
import re
import secrets
import time
from typing import Annotated, Any
from pathlib import Path
from typing import Annotated, Any, BinaryIO
from fastapi import APIRouter, Depends, Request
from fastapi.responses import FileResponse
@@ -27,6 +32,29 @@ router = APIRouter()
_RESERVED_USERNAMES = {'admin', 'administrator', 'qfbsh', 'root', 'system', 'moderator', '管理员', '客服'}
_EMAIL_RE = re.compile(r'^[\w.%+-]+@[\w.-]+\.[A-Za-z]{2,}$')
# 备份上传的搬运块大小。上传的是**整库备份**,原实现 `content = await f.read()`
# 会把整库一次性读进内存(本端点对外没有大小上限),等于给上传方一个内存放大器;
# 改成逐块搬运后,内存占用恒为一块。
_UPLOAD_CHUNK = 1 << 20
_SQLITE_MAGIC = b'SQLite format 3'
def _save_upload_stream(src: BinaryIO, dest: Path) -> bytes:
"""把上传流按块写进 dest,并返回文件头(前 16 字节)。
**阻塞文件 IO**:只应由 asyncio.to_thread 调用。文件头在写第一块时就地截获,
省掉「落盘后再把文件重新打开读一次」的那趟回读。
"""
src.seek(0)
header = b''
with dest.open('wb') as out:
while chunk := src.read(_UPLOAD_CHUNK):
if len(header) < 16:
header += chunk[:16 - len(header)]
out.write(chunk)
return header
async def _require_admin(user: Annotated[SiteUser | None, Depends(current_user)]) -> SiteUser:
if user is None or not user.is_admin:
@@ -74,14 +102,14 @@ def _int_or(value: Any, default: int = 0) -> int:
async def _log(admin: SiteUser, action: str, detail: str = '') -> None:
await asyncio.to_thread(auth.add_log, 'success', admin.username, '-', action, detail)
await auth.add_log('success', admin.username, '-', action, detail)
# ============ 用户管理 ============
@router.post('/admin/site/users')
async def users(_admin: AdminUser):
rows = await asyncio.to_thread(auth.list_users)
rows = await auth.list_users()
return success({'users': rows})
@@ -90,8 +118,8 @@ async def user_delete(admin: AdminUser, request: Request):
data = await _request_data(request)
uid = _int_or(data.get('id'))
try:
target = await asyncio.to_thread(auth.get_user_by_id, uid)
ok = await asyncio.to_thread(auth.delete_user, uid)
target = await auth.get_user_by_id(uid)
ok = await auth.delete_user(uid)
if not ok:
return error('用户不存在或为管理员')
await _log(admin, '删除用户', target['username'] if target else str(uid))
@@ -114,12 +142,12 @@ async def user_duration(admin: AdminUser, request: Request):
if count < -1:
return error('次数不能小于 -1(-1 表示不限次数)')
try:
target = await asyncio.to_thread(auth.get_user_by_id, uid)
target = await auth.get_user_by_id(uid)
if not target:
return error('用户不存在')
if target['role'] == 'admin':
return error('管理员不限次数')
await asyncio.to_thread(auth.set_user_credits, uid, count)
await auth.set_user_credits(uid, count)
await _log(admin, '修改用户次数', f'{target["username"]} → {count} 次')
return success(message=f'已修改为 {count} 次')
except Exception as e:
@@ -147,12 +175,11 @@ async def user_create(admin: AdminUser, request: Request):
if len(password) < 8 or len(password) > 128:
return error('密码长度需在 8-128 位之间')
try:
if await asyncio.to_thread(auth.get_user_by_name, username):
if await auth.get_user_by_name(username):
return error('用户名已存在')
if email and await asyncio.to_thread(auth.get_user_by_email, email):
if email and await auth.get_user_by_email(email):
return error('该邮箱已被注册,请更换邮箱')
await asyncio.to_thread(
auth.create_user, username, password, email=email, credits=credits,
await auth.create_user(username, password, email=email, credits=credits,
guard=auth.RegistrationGuard(ip='-', ua='admin创建'))
await _log(admin, '添加用户', f'{username} · {credits} 次')
return success(message=f'用户 {username} 创建成功({credits} 次)')
@@ -168,7 +195,7 @@ async def user_reset_password(admin: AdminUser, request: Request):
if len(newpwd) < 8 or len(newpwd) > 128:
return error('新密码长度需在 8-128 位之间')
try:
target = await asyncio.to_thread(auth.get_user_by_id, uid)
target = await auth.get_user_by_id(uid)
if not target:
return error('用户不存在')
if target['role'] == 'admin':
@@ -179,10 +206,10 @@ async def user_reset_password(admin: AdminUser, request: Request):
target['password_hash'], oldpwd, target['salt'])
if not ok:
return error('原密码错误')
await asyncio.to_thread(auth.reset_password, uid, newpwd)
await auth.reset_password(uid, newpwd)
await _log(admin, '修改自己的密码', '管理员')
return success(message='密码已修改,下次登录请使用新密码(其他设备已全部下线)')
await asyncio.to_thread(auth.reset_password, uid, newpwd)
await auth.reset_password(uid, newpwd)
await _log(admin, '重置密码', target['username'])
return success(message=f'已重置 {target["username"]} 的密码')
except Exception as e:
@@ -194,14 +221,14 @@ async def user_toggle(admin: AdminUser, request: Request):
data = await _request_data(request)
uid = _int_or(data.get('id'))
try:
target = await asyncio.to_thread(auth.get_user_by_id, uid)
target = await auth.get_user_by_id(uid)
if not target:
return error('用户不存在')
if target['role'] == 'admin':
return error('不能对管理员账号执行该操作')
cur = target['status'] if 'status' in target.keys() else 1
new_status = 0 if cur else 1
await asyncio.to_thread(auth.set_user_status, uid, new_status)
await auth.set_user_status(uid, new_status)
await _log(admin, '禁用/启用',
f'{target["username"]} → {"启用" if new_status else "禁用"}')
return success({'status': new_status},
@@ -217,17 +244,17 @@ async def user_ban(admin: AdminUser, request: Request):
action = str(data.get('action') or 'ban')
reason = str(data.get('reason') or '')
try:
row = await asyncio.to_thread(auth.get_user_by_id, uid)
row = await auth.get_user_by_id(uid)
if not row:
return error('用户不存在')
if row['role'] == 'admin':
return error('不能对管理员执行该操作')
if action == 'ban':
if not await asyncio.to_thread(auth.ban_user, uid, reason or '管理员手动封禁'):
if not await auth.ban_user(uid, reason or '管理员手动封禁'):
return error('封禁失败')
await _log(admin, '封禁用户', f'{row["username"]}: {reason or "手动封禁"}')
return success(message=f'已封禁用户「{row["username"]}」,该账号已强制下线')
if not await asyncio.to_thread(auth.unban_user, uid):
if not await auth.unban_user(uid):
return error('解封失败')
await _log(admin, '解封用户', row['username'])
return success(message=f'已解封用户「{row["username"]}」')
@@ -239,7 +266,7 @@ async def user_ban(admin: AdminUser, request: Request):
async def user_cookie(_admin: AdminUser, request: Request):
data = await _request_data(request)
username = str(data.get('username') or '')
user = await asyncio.to_thread(auth.get_user_by_name, username)
user = await auth.get_user_by_name(username)
if not user:
return error('用户不存在', 404)
return success({'username': username, 'cookie': user['netease_cookie'] or ''})
@@ -249,7 +276,7 @@ async def user_cookie(_admin: AdminUser, request: Request):
@router.post('/admin/site/smtp/get')
async def smtp_get(_admin: AdminUser):
cfg = await asyncio.to_thread(auth.get_smtp_config)
cfg = await auth.get_smtp_config()
configured = bool(cfg.get('host') and cfg.get('sender') and cfg.get('password'))
return success({'host': cfg.get('host', ''), 'port': cfg.get('port', ''),
'sender': cfg.get('sender', ''), 'configured': configured})
@@ -269,9 +296,9 @@ async def smtp_save(admin: AdminUser, request: Request):
return error('SMTP 服务器与发件邮箱不能为空')
try:
if not password:
cfg = await asyncio.to_thread(auth.get_smtp_config)
cfg = await auth.get_smtp_config()
password = cfg.get('password', '')
await asyncio.to_thread(auth.save_smtp_config, host, port, sender, password)
await auth.save_smtp_config(host, port, sender, password)
await _log(admin, '保存SMTP配置', f'{host}:{port}')
return success(message='SMTP 配置已保存')
except Exception as e:
@@ -287,12 +314,12 @@ async def smtp_test(admin: AdminUser, request: Request):
if not _EMAIL_RE.match(to):
return error('收件邮箱格式不正确')
try:
await asyncio.to_thread(auth.send_email, to, 'SMTP 测试邮件',
await auth.send_email(to, 'SMTP 测试邮件',
'这是一封来自云村小助手的测试邮件,收到即说明 SMTP 配置成功。')
await _log(admin, 'SMTP测试', to)
return success(message='测试邮件已发送,请查收')
except Exception as e:
await asyncio.to_thread(auth.add_log, 'fail', admin.username, '-', 'SMTP测试', str(e))
await auth.add_log('fail', admin.username, '-', 'SMTP测试', str(e))
return error(f'发送失败: {e!s}', 500)
@@ -303,7 +330,7 @@ async def logs(_admin: AdminUser, request: Request):
data = await _request_data(request)
log_type = 'fail' if str(data.get('type')) == 'fail' else 'success'
page = max(1, _int_or(data.get('page'), 1))
rows, total = await asyncio.to_thread(auth.query_logs, log_type, 10, (page - 1) * 10)
rows, total = await auth.query_logs(log_type, 10, (page - 1) * 10)
return success({'logs': rows, 'type': log_type, 'page': page,
'total': total, 'pages': max(1, (total + 9) // 10)})
@@ -312,7 +339,7 @@ async def logs(_admin: AdminUser, request: Request):
async def logs_clear(admin: AdminUser, request: Request):
data = await _request_data(request)
log_type = 'fail' if str(data.get('type')) == 'fail' else 'success'
deleted = await asyncio.to_thread(auth.clear_logs, log_type)
deleted = await auth.clear_logs(log_type)
await _log(admin, '清除日志', f'{log_type} · {deleted} 条')
return success(message=f'已清除 {"失败" if log_type == "fail" else "成功"}日志 {deleted} 条')
@@ -335,8 +362,7 @@ async def cards_generate(admin: AdminUser, request: Request):
if credits <= 0:
return error('卡密次数必须大于 0')
try:
codes = await asyncio.to_thread(
auth.generate_cards, count, credits, created_by=admin.username,
codes = await auth.generate_cards(count, credits, created_by=admin.username,
prefix=str(data.get('prefix') or 'VIP'), note=str(data.get('note') or ''),
valid_days=valid_days)
await _log(admin, '生成卡密', f'{count} 张 · {credits} 次/张 · 有效{valid_days or "永久"}天')
@@ -351,7 +377,7 @@ async def cards_list(_admin: AdminUser, request: Request):
data = await _request_data(request)
limit = _int_or(data.get('limit'), 100) or 100
offset = _int_or(data.get('offset'), 0)
cards, total = await asyncio.to_thread(auth.list_cards, limit=limit, offset=offset)
cards, total = await auth.list_cards(limit=limit, offset=offset)
return success({'cards': cards, 'total': total})
@@ -363,7 +389,7 @@ async def cards_disable(admin: AdminUser, request: Request):
return error('缺少卡密')
disable = str(data.get('disabled') or '').strip().lower() in ('1', 'true')
try:
await asyncio.to_thread(auth.set_card_disabled, code, disable)
await auth.set_card_disabled(code, disable)
await _log(admin, '禁用卡密' if disable else '恢复卡密', code[:4] + '****')
return success(message=('已禁用 ' if disable else '已恢复 ') + code[:4] + '****')
except ValueError as e:
@@ -379,7 +405,7 @@ async def cards_delete(admin: AdminUser, request: Request):
if not code:
return error('缺少卡密')
try:
await asyncio.to_thread(auth.delete_card, code)
await auth.delete_card(code)
await _log(admin, '删除卡密', code[:4] + '****')
return success(message='已删除 ' + code[:4] + '****')
except ValueError as e:
@@ -392,14 +418,14 @@ async def cards_delete(admin: AdminUser, request: Request):
@router.post('/admin/site/ann/get')
async def ann_get(_admin: AdminUser):
return success(await asyncio.to_thread(auth.get_announcement))
return success(await auth.get_announcement())
@router.post('/admin/site/ann/save')
async def ann_save(admin: AdminUser, request: Request):
data = await _request_data(request)
try:
await asyncio.to_thread(auth.save_announcement, str(data.get('title') or ''),
await auth.save_announcement(str(data.get('title') or ''),
str(data.get('content') or ''), str(data.get('popup_mode') or 'off'))
await _log(admin, '保存公告', str(data.get('title') or ''))
return success(message='公告已保存')
@@ -411,17 +437,17 @@ async def ann_save(admin: AdminUser, request: Request):
@router.post('/admin/site/play_key/get')
async def play_key_get(_admin: AdminUser):
key = await asyncio.to_thread(auth.get_setting, 'playlist_play_key', '')
key = await auth.get_setting('playlist_play_key', '')
if not key:
key = secrets.token_urlsafe(16)
await asyncio.to_thread(auth.set_setting, 'playlist_play_key', key)
await auth.set_setting('playlist_play_key', key)
return success({'key': key})
@router.post('/admin/site/play_key/rotate')
async def play_key_rotate(admin: AdminUser):
key = secrets.token_urlsafe(16)
await asyncio.to_thread(auth.set_setting, 'playlist_play_key', key)
await auth.set_setting('playlist_play_key', key)
await _log(admin, '刷新播放API密钥', '旧密钥已失效')
return success({'key': key}, '密钥已刷新,旧密钥已全部失效')
@@ -431,8 +457,8 @@ async def play_key_rotate(admin: AdminUser):
@router.post('/admin/site/backup/create')
async def backup_create(admin: AdminUser):
try:
name = await asyncio.to_thread(auth.create_backup)
await asyncio.to_thread(auth.set_setting, 'last_backup_ts', str(time.time()))
name = await auth.create_backup()
await auth.set_setting('last_backup_ts', str(time.time()))
await _log(admin, '数据库备份', name)
return success({'name': name}, f'备份完成:{name}')
except Exception as e:
@@ -441,20 +467,17 @@ async def backup_create(admin: AdminUser):
@router.post('/admin/site/backup/list')
async def backup_list(_admin: AdminUser):
backups = await asyncio.to_thread(auth.list_backups)
interval = await asyncio.to_thread(auth.get_setting, 'backup_interval_hours', '0')
def _size() -> int:
return auth.db.DB_PATH.stat().st_size if auth.db.DB_PATH.exists() else 0
return success({'backups': backups, 'interval_hours': interval,
'db_size': await asyncio.to_thread(_size)})
backups = await auth.list_backups()
interval = await auth.get_setting('backup_interval_hours', '0')
# 主库文件的 stat:微秒级元数据操作,线程调度开销比它本身还大,直接同步做
db_size = auth.db.DB_PATH.stat().st_size if auth.db.DB_PATH.exists() else 0
return success({'backups': backups, 'interval_hours': interval, 'db_size': db_size})
@router.get('/admin/site/backup/download')
async def backup_download(_admin: AdminUser, request: Request):
name = request.query_params.get('name') or ''
p = await asyncio.to_thread(auth.backup_path, name)
p = await auth.backup_path(name)
if not p:
return error('备份文件不存在', 404)
resp = FileResponse(str(p), filename=name)
@@ -483,16 +506,14 @@ async def backup_upload(admin: AdminUser, request: Request):
name = f'{base}-{k}.db'
k += 1
dest = auth.db.BACKUP_DIR / name
content = await f.read()
await asyncio.to_thread(dest.write_bytes, content)
with dest.open('rb') as hf:
header = hf.read(16)
if not header.startswith(b'SQLite format 3'):
# 流式落盘:整库体量逐块搬运,文件头随第一块一起截获(不再回读一遍)
header = await asyncio.to_thread(_save_upload_stream, f.file, dest)
if not header.startswith(_SQLITE_MAGIC):
dest.unlink()
return error('不是有效的 SQLite 数据库文件')
await _log(admin, '上传数据库', dest.name)
try:
await asyncio.to_thread(auth.prune_backups, 20, 5)
await auth.prune_backups(20, 5)
except Exception:
pass
return success({'name': dest.name}, '上传成功,可在下方选择该备份进行恢复')
@@ -510,13 +531,13 @@ async def backup_restore(admin: AdminUser, request: Request):
data = await _request_data(request)
name = str(data.get('name') or '')
try:
await asyncio.to_thread(auth.restore_backup, name)
await auth.restore_backup(name)
await _log(admin, '恢复数据库', name)
return success(message='数据库已恢复,请刷新页面')
except ValueError as e:
return error(f'恢复失败: {e!s}', 400)
except Exception as e:
await asyncio.to_thread(auth.add_log, 'fail', admin.username, '-', '恢复数据库', str(e))
await auth.add_log('fail', admin.username, '-', '恢复数据库', str(e))
return error(f'恢复失败: {e!s}', 500)
@@ -525,7 +546,7 @@ async def backup_delete(_admin: AdminUser, request: Request):
data = await _request_data(request)
name = str(data.get('name') or '')
try:
if await asyncio.to_thread(auth.delete_backup, name):
if await auth.delete_backup(name):
return success(message='备份已删除')
return error('备份文件不存在', 404)
except Exception as e:
@@ -544,7 +565,7 @@ async def backup_auto(_admin: AdminUser, request: Request):
if hours < 0:
return error('间隔不能为负')
try:
await asyncio.to_thread(auth.set_setting, 'backup_interval_hours', str(hours))
await auth.set_setting('backup_interval_hours', str(hours))
return success(message=f'定时备份已开启,每 {hours} 小时自动备份' if hours > 0
else '定时备份已关闭')
except Exception as e:
+22 -23
View File
@@ -10,11 +10,10 @@
设计:
- 登录失败锁定与管理员 2FA 票据表**复用 web.auth_state 的模块级状态**,全进程共享同一份
计数/票据,避免「换个框架打就绕过锁定」。
- 密码校验(PBKDF2,10 万次迭代,CPU 密集)与库操作经 asyncio.to_thread 执行,
不阻塞事件循环。
- 密码校验(PBKDF2,10 万次迭代,CPU 密集)已在 auth_system.verify_login 内部经
asyncio.to_thread 卸载;其余 auth_system 调用已迁移为原生异步(aiosqlite)直接 await。
- 会话 Cookie 由 core.session 签发(沿用同一格式与密钥,存量会话继续有效),删 Cookie 亦同口径。
"""
import asyncio
import os
import time
from typing import Annotated
@@ -51,10 +50,10 @@ def _cookie_kwargs() -> dict:
'path': '/', 'secure': secure}
def _login_response(payload: dict, message: str, uid: int, seq: int) -> JSONResponse:
async def _login_response(payload: dict, message: str, uid: int, seq: int) -> JSONResponse:
"""构造带会话 Cookie 的成功响应(沿用同一 Cookie 格式)。"""
resp = JSONResponse(content={'status': 200, 'success': True, 'message': message, 'data': payload})
cookie = cs.dumps(deps.secret_key(), {
cookie = cs.dumps(await deps.secret_key(), {
'site_uid': uid, 'site_uid_seq': seq, 'site_uid_ts': int(time.time()),
})
resp.set_cookie(SESSION_COOKIE, cookie, **_cookie_kwargs())
@@ -109,7 +108,7 @@ async def _resolve_login_identity(username: str) -> tuple:
"""
if '@' not in username:
return None, username
iuser = await asyncio.to_thread(auth.get_user_by_email, username)
iuser = await auth.get_user_by_email(username)
lock_name = iuser['username'] if iuser else (auth._normalize_email(username) or username)
return iuser, lock_name
@@ -124,7 +123,7 @@ async def _allow_login_attempt(lock_name: str, ip: str) -> bool:
if _login_fail_check(lock_name):
return True
if _login_fail_locked_log_once(lock_name):
await asyncio.to_thread(auth.add_log, 'fail', lock_name, ip,
await auth.add_log('fail', lock_name, ip,
'登录', '失败次数过多,临时锁定')
return False
@@ -135,7 +134,7 @@ async def _check_account_status(user, lock_name: str, ip: str):
status = user['status'] if 'status' in keys else 1
if status == 2:
reason = user['ban_reason'] if 'ban_reason' in keys else ''
await asyncio.to_thread(auth.add_log, 'fail', lock_name, ip,
await auth.add_log('fail', lock_name, ip,
'登录', f'封禁账号尝试登录: {reason}')
return error('账号已被封禁' + (f':{reason}' if reason else '') + ',请联系管理员')
if status == 0:
@@ -152,11 +151,11 @@ async def _admin_two_factor_response(user, lock_name: str, ip: str) -> JSONRespo
admin_email = user['email'] if 'email' in user.keys() else ''
if not admin_email:
await asyncio.to_thread(auth.add_log, 'fail', lock_name, ip,
await auth.add_log('fail', lock_name, ip,
'管理员登录', '未绑定邮箱,两步验证不可用')
return error('管理员账号未绑定邮箱,请联系管理员先完成绑定')
ticket = _admin2fa_issue(user)
await asyncio.to_thread(auth.add_log, 'success', lock_name, ip,
await auth.add_log('success', lock_name, ip,
'管理员登录', '密码验证通过,待邮箱验证')
return success({'ticket': ticket, 'email_masked': _admin2fa_mask(admin_email),
'username': user['username'], 'admin_2fa': True},
@@ -173,14 +172,14 @@ async def _user_session_response(user, ip: str) -> JSONResponse:
seq_row = await repo.get_user_by_id(uid)
seq = int((seq_row or {}).get('login_seq') or 1)
banned, _info = await asyncio.to_thread(auth.record_login_location, user, ip)
banned, _info = await auth.record_login_location(user, ip)
if banned:
await db.execute(
'UPDATE users SET login_seq = COALESCE(login_seq, 0) + 1 WHERE id = ?', (uid,))
return error('检测到账号在多个省份异常登录,已自动封禁,请联系管理员')
fresh = await repo.get_user_by_id(uid) or {}
return _login_response({
return await _login_response({
'user': _user_public(fresh),
'duration_text': _format_credits(int(fresh.get('duration_seconds') or 0)),
'redirect': 'admin' if fresh.get('role') == 'admin' else 'user',
@@ -209,7 +208,7 @@ async def login(request: Request):
return error('尝试过于频繁,该账号已临时锁定,请 10 分钟后再试')
verify_name = iuser['username'] if iuser is not None else username
user = await asyncio.to_thread(auth.verify_login, verify_name, password)
user = await auth.verify_login(verify_name, password)
if not user:
from web.auth_state import _login_fail_record
_login_fail_record(lock_name)
@@ -256,7 +255,7 @@ async def admin_login_email(request: Request):
if (not email_input or not v.get('email')
or not _hmac.compare_digest(email_input, v['email'])):
v['attempts'] += 1
await asyncio.to_thread(auth.add_log, 'fail', v['username'], ip, '管理员登录', '绑定邮箱不匹配')
await auth.add_log('fail', v['username'], ip, '管理员登录', '绑定邮箱不匹配')
if v['attempts'] >= _ADMIN_2FA_MAX_ATTEMPTS:
_ADMIN_2FA.pop(ticket, None)
return error('错误次数过多,请重新开始登录')
@@ -266,9 +265,9 @@ async def admin_login_email(request: Request):
if time.time() - last_send < _ADMIN2FA_SEND_GAP:
return error('验证码发送过于频繁,请 1 分钟后再试')
try:
await asyncio.to_thread(auth.send_verify_code, v['email'], 'admin2fa')
await auth.send_verify_code(v['email'], 'admin2fa')
_ADMIN2FA_LAST_SEND[v['username']] = time.time()
await asyncio.to_thread(auth.add_log, 'success', v['username'], ip, '管理员登录',
await auth.add_log('success', v['username'], ip, '管理员登录',
f'绑定邮箱验证通过,验证码已发送({_admin2fa_mask(v["email"])})')
return success(message='验证码已发送至管理员邮箱,请查收')
except (ValueError, RuntimeError) as e:
@@ -293,22 +292,22 @@ async def admin_login_verify(request: Request):
return error('登录会话已过期,请重新开始登录', 401, error_code='admin_2fa_expired')
code = str(data.get('code') or '').strip()
if not await asyncio.to_thread(auth.check_verify_code, v['email'], code, 'admin2fa'):
if not await auth.check_verify_code(v['email'], code, 'admin2fa'):
v['attempts'] += 1
if v['attempts'] >= _ADMIN_2FA_MAX_ATTEMPTS:
_ADMIN_2FA.pop(ticket, None)
return error('错误次数过多,请重新开始登录')
await asyncio.to_thread(auth.add_log, 'fail', v['username'], ip, '管理员登录', '验证码错误')
await auth.add_log('fail', v['username'], ip, '管理员登录', '验证码错误')
return error('验证码错误或已过期')
_ADMIN_2FA.pop(ticket, None) # 一次性
user = await asyncio.to_thread(auth.get_user_by_id, v['user_id'])
user = await auth.get_user_by_id(v['user_id'])
if not user or user['role'] != 'admin':
return error('登录会话已过期,请重新开始登录', 401, error_code='admin_2fa_expired')
keys = user.keys()
status = user['status'] if 'status' in keys else 1
if status == 2:
await asyncio.to_thread(auth.add_log, 'fail', v['username'], ip, '管理员登录', '封禁账号尝试登录')
await auth.add_log('fail', v['username'], ip, '管理员登录', '封禁账号尝试登录')
return error('账号已被封禁,请联系管理员')
if status == 0:
return error('账号已被禁用,请联系管理员')
@@ -320,14 +319,14 @@ async def admin_login_verify(request: Request):
seq_row = await repo.get_user_by_id(uid)
seq = int((seq_row or {}).get('login_seq') or 1)
banned, _info = await asyncio.to_thread(auth.record_login_location, user, ip)
banned, _info = await auth.record_login_location(user, ip)
if banned:
await db.execute('UPDATE users SET login_seq = COALESCE(login_seq, 0) + 1 WHERE id = ?', (uid,))
return error('检测到账号在多个省份异常登录,已自动封禁,请联系管理员')
fresh = await repo.get_user_by_id(uid) or {}
await asyncio.to_thread(auth.add_log, 'success', v['username'], ip, '管理员登录', '两步验证通过')
return _login_response({
await auth.add_log('success', v['username'], ip, '管理员登录', '两步验证通过')
return await _login_response({
'user': _user_public(fresh),
'duration_text': _format_credits(int(fresh.get('duration_seconds') or 0)),
'redirect': 'admin',
+4 -3
View File
@@ -13,7 +13,6 @@
X-Download-Filename(URL 编码,供前端取回原始文件名)
- 文件清理仍由监控线程按 35 分钟保留期统一回收(交付请求不删文件)
"""
import asyncio
import os
from typing import Annotated, Any
from urllib.parse import quote
@@ -67,7 +66,8 @@ async def download_result(user: LoginUser, request: Request):
result = task.get('result') or {}
target = result.get('zip_path') or result.get('file_path')
target_exists = bool(target) and await asyncio.to_thread(os.path.exists, target)
# 微秒级单文件存在性判定,直接同步做(线程调度开销比它本身还大)
target_exists = bool(target) and os.path.exists(target) # noqa: ASYNC240
if status in ('canceled', 'error') and not target_exists:
return error(task.get('msg') or ('任务已取消' if status == 'canceled' else '任务失败'),
200 if status == 'canceled' else 500)
@@ -95,7 +95,8 @@ async def download_batch_zip_result(user: LoginUser, request: Request):
status = task.get('status')
result = task.get('result') or {}
zip_path = result.get('zip_path')
has_deliverable = bool(zip_path) and await asyncio.to_thread(os.path.exists, zip_path)
# 同上:微秒级单文件存在性判定
has_deliverable = bool(zip_path) and os.path.exists(zip_path) # noqa: ASYNC240
if status in ('queued', 'running'):
return error('任务还在进行中,请稍后再试', 202)
if status == 'paused':
+12 -15
View File
@@ -12,7 +12,6 @@
settle/rename 消费过的行天然豁免);任务登记与启动由 web.tasks_registry 承担;
worker 复用 services 层既有实现,账本与交付语义不变。
"""
import asyncio
import logging
import threading
import time
@@ -72,19 +71,17 @@ async def _sync_download_json(downloader, music_id: str, quality: str, cookies:
只负责在成功路径 settle(1, 'delivered')(结算后自动退自然豁免)。
"""
try:
info = await asyncio.to_thread(
downloader.get_music_info, int(music_id), quality, cookies)
info = await downloader.get_music_info(int(music_id), quality, cookies)
except Exception as e:
logger.error('获取音乐信息失败: %s', e)
return error('获取失败:歌曲不存在或暂无版权', 404)
from services._shared import _build_download_base
base = await asyncio.to_thread(
_build_download_base, filename_format,
{'name': info.name, 'artist_string': info.artists})
base = _build_download_base(
filename_format, {'name': info.name, 'artist_string': info.artists})
filename = f'{base}.{info.file_type}'
try:
dl = await asyncio.to_thread(
downloader.download_music_file, int(music_id), quality,
dl = await downloader.download_music_file(
int(music_id), quality,
write_meta=write_meta, meta_fields=meta_fields,
cookies=cookies, music_info=info)
if not dl.success:
@@ -190,7 +187,7 @@ async def download(user: Annotated[SiteUser | None, Depends(current_user)], requ
from services import downloads as services_downloads
from services import tasks as services_tasks
device_id = str(data.get('device_id') or '').strip()[:64]
await asyncio.to_thread(services_tasks._touch_device, device_id)
services_tasks._touch_device(device_id)
task = _single_task({
'music_id': music_id, 'quality': quality,
'filename_format': filename_format, 'write_meta': write_meta,
@@ -273,7 +270,7 @@ async def download_batch_zip(user: Annotated[SiteUser | None, Depends(current_us
device_id = str(data.get('device_id') or '').strip()[:64]
from services import batch as services_batch
from services import tasks as services_tasks
await asyncio.to_thread(services_tasks._touch_device, device_id)
services_tasks._touch_device(device_id)
task = {
'type': 'batch',
'total': len(ids), 'done': 0, 'failed': 0,
@@ -337,7 +334,7 @@ async def podcast_download(user: Annotated[SiteUser | None, Depends(current_user
from music_api import podcast
quality = _podcast_quality(data)
need_zip = str(data.get('zip', 'false')).lower() in ('1', 'true', 'yes')
ref = podcast.extract_podcast_ref(raw)
ref = await podcast.extract_podcast_ref(raw)
if ref['kind'] == 'radio':
return error('这是电台链接,请先解析电台后选择单集下载,或使用「打包整个播客」')
@@ -352,7 +349,7 @@ async def podcast_download(user: Annotated[SiteUser | None, Depends(current_user
device_id = str(data.get('device_id') or '').strip()[:64]
from services import podcast as services_podcast
from services import tasks as services_tasks
await asyncio.to_thread(services_tasks._touch_device, device_id)
services_tasks._touch_device(device_id)
task = {
'type': 'single',
'total': 1, 'done': 0, 'failed': 0,
@@ -403,13 +400,13 @@ async def podcast_batch_zip(user: Annotated[SiteUser | None, Depends(current_use
from music_api import podcast
if raw_radio:
ref = podcast.extract_podcast_ref(raw_radio)
ref = await podcast.extract_podcast_ref(raw_radio)
if ref['kind'] == 'program':
return error('这是单集链接,请使用「下载单集」或直接输入电台链接/ID')
radio_id = ref['id']
radio_name = ''
try:
radio_name = podcast.get_radio_detail(radio_id, cookies).get('name', '')
radio_name = (await podcast.get_radio_detail(radio_id, cookies)).get('name', '')
except APIException:
pass
else:
@@ -423,7 +420,7 @@ async def podcast_batch_zip(user: Annotated[SiteUser | None, Depends(current_use
device_id = str(data.get('device_id') or '').strip()[:64]
from services import podcast as services_podcast
from services import tasks as services_tasks
await asyncio.to_thread(services_tasks._touch_device, device_id)
services_tasks._touch_device(device_id)
task = {
'type': 'batch',
'total': 0, 'done': 0, 'failed': 0,
+5 -14
View File
@@ -1,9 +1,9 @@
# -*- coding: utf-8 -*-
"""首页与静态资源。
"""首页。
- GET / 首页:优先服务 React 构建产物 frontend/dist/index.html;
dist 未构建时回退旧版 templates/index.html(平滑过渡)
- 静态资源由 Starlette StaticFiles 挂载 /assets 与 /static(见 web/app.py)
- GET / 首页:服务 React 构建产物 frontend/dist/index.html
(vite 固定产物文件名,字节级稳定,golden 契约依赖此点)
- 构建产物经 /assets 挂载(见 web/app.py)
"""
from pathlib import Path
@@ -14,20 +14,11 @@ router = APIRouter()
_PROJECT_ROOT = Path(__file__).resolve().parent.parent.parent
_DIST_INDEX = _PROJECT_ROOT / 'frontend' / 'dist' / 'index.html'
_LEGACY_TEMPLATE = _PROJECT_ROOT / 'templates' / 'index.html'
def _render_index() -> str:
"""渲染首页 HTML:React 构建产物优先,缺失时回退旧模板。"""
if _DIST_INDEX.is_file():
"""返回 React 构建产物原样字节(构建产物入库,恒存在)。"""
return _DIST_INDEX.read_text(encoding='utf-8')
import jinja2
env = jinja2.Environment(
loader=jinja2.FileSystemLoader(str(_LEGACY_TEMPLATE.parent)),
autoescape=True,
)
return env.get_template(_LEGACY_TEMPLATE.name).render()
@router.get('/', response_class=HTMLResponse)
+1 -1
View File
@@ -219,7 +219,7 @@ async def _resolve_user_cookies(
if mu and gmu and hmac.compare_digest(mu, gmu):
try:
import auth_system
await asyncio.to_thread(auth_system.add_log, 'fail',
await auth_system.add_log('fail',
user.username if user else '匿名',
ratelimit.client_ip(request), '全局账号冒用拦截',
'浏览器提交的 MUSIC_U 与全局账号一致,已按未登录拒绝')
+8 -14
View File
@@ -7,10 +7,9 @@
- POST/GET /api/v1/podcast/search ↔ Flask /podcast/search
- POST/GET /api/v1/podcast/hot ↔ Flask /podcast/hot
说明:music_api.podcast 是同步模块(底层 core.sync_http),此处以 asyncio.to_thread 包装,
让阻塞的上游调用不占用事件循环——行为不变,且不阻塞其它请求。
说明:music_api.podcast 是原生异步模块(底层 core.http_client),此处直接 await;
解析与直链检查同属一次请求,不再需要线程卸载。
"""
import asyncio
from typing import Annotated
from fastapi import APIRouter, Depends, Request
@@ -51,8 +50,7 @@ async def podcast_parse(_user: LoginUser, request: Request):
cookies = _cookies()
try:
result = await asyncio.to_thread(
podcast.parse_podcast_ref, raw, cookies, page_size=page_size)
result = await podcast.parse_podcast_ref(raw, cookies, page_size=page_size)
except APIException as e:
return error(str(e), 404)
except Exception:
@@ -64,8 +62,7 @@ async def podcast_parse(_user: LoginUser, request: Request):
url_info = None
if song_id:
try:
url_info = await asyncio.to_thread(
podcast.resolve_episode_url, song_id, cookies, quality)
url_info = await podcast.resolve_episode_url(song_id, cookies, quality)
except Exception:
url_info = None
if url_info:
@@ -110,8 +107,7 @@ async def podcast_programs(_user: LoginUser, request: Request):
if not str(radio_id).isdigit():
return error('无效的电台 ID')
try:
result = await asyncio.to_thread(
podcast.get_radio_programs, radio_id, _cookies(),
result = await podcast.get_radio_programs(radio_id, _cookies(),
limit=limit, offset=offset, asc=asc)
return success(result, '获取单集列表成功')
except APIException as e:
@@ -134,8 +130,7 @@ async def podcast_search(_user: LoginUser, request: Request):
if search_type not in ('all', 'radio', 'voice'):
search_type = 'all'
try:
result = await asyncio.to_thread(
podcast.search_podcast, keyword, _cookies(),
result = await podcast.search_podcast(keyword, _cookies(),
limit=limit, offset=offset, search_type=search_type)
return success(result, '播客搜索完成')
except APIException as e:
@@ -153,14 +148,13 @@ async def podcast_hot(_user: LoginUser, request: Request):
offset = max(0, _safe_int(data.get('offset'), 0))
cate_id = data.get('cate_id') or None
try:
radios = await asyncio.to_thread(
podcast.get_hot_radios, _cookies(), limit=limit, offset=offset, cate_id=cate_id)
radios = await podcast.get_hot_radios(_cookies(), limit=limit, offset=offset, cate_id=cate_id)
except APIException as e:
return error(str(e), 404)
except Exception:
return error('服务暂时不可用,请稍后再试', 500)
try:
categories = await asyncio.to_thread(podcast.get_categories)
categories = await podcast.get_categories()
except APIException:
categories = []
# 满页视为可能还有更多
+22 -24
View File
@@ -9,12 +9,11 @@
- POST /api/v1/auth/forgot_password ↔ Flask /auth/forgot_password
说明:
- 同步的 auth_system 调用(SQLite 事务、邮件 SMTP、PBKDF2)经 asyncio.to_thread 执行,
不阻塞事件循环;行为与历史实现完全一致。
- auth_system 已迁移为**原生异步**(aiosqlite / aiosmtplib),此处直接 await,不再经
asyncio.to_thread 桥接;仅 PBKDF2 这类 CPU 密集且释放 GIL 的活仍在内部卸载到线程。
- 发码 IP 限流复用 web.auth_state 的模块级状态 `_SEND_CODE_IP`。
- 失败锁定/验证码消费等安全口径逐字保留。
"""
import asyncio
import os
import sqlite3
import time
@@ -43,10 +42,10 @@ _SESSION_MAX_AGE = cs.SESSION_MAX_AGE
def _login_cookie_response(payload: dict, message: str, uid: int, seq: int,
async def _login_cookie_response(payload: dict, message: str, uid: int, seq: int,
secure: bool) -> JSONResponse:
resp = JSONResponse(content={'status': 200, 'success': True, 'message': message, 'data': payload})
cookie = cs.dumps(deps.secret_key(), {'site_uid': uid, 'site_uid_seq': seq,
cookie = cs.dumps(await deps.secret_key(), {'site_uid': uid, 'site_uid_seq': seq,
'site_uid_ts': int(time.time())})
resp.set_cookie(SESSION_COOKIE, cookie, max_age=_SESSION_MAX_AGE, httponly=True,
samesite='lax', path='/', secure=secure)
@@ -74,7 +73,7 @@ async def send_code(request: Request):
if purpose not in ('register', 'reset'):
return error('验证码用途不正确')
try:
await asyncio.to_thread(auth.send_verify_code, email, purpose)
await auth.send_verify_code(email, purpose)
return success(message='验证码已发送,请查收邮箱')
except (ValueError, RuntimeError) as e:
return error(str(e))
@@ -97,14 +96,14 @@ async def redeem(user: Annotated[SiteUser | None, Depends(current_user)], reques
data = await _request_data(request)
code = str(data.get('code') or '')
try:
credits, remaining = await asyncio.to_thread(auth.redeem_card, code, user.username)
await asyncio.to_thread(auth.add_log, 'success', user.username, ip, '卡密充值',
credits, remaining = await auth.redeem_card(code, user.username)
await auth.add_log('success', user.username, ip, '卡密充值',
f'{code[:4]}**** · +{credits} 次')
return success({'credits': credits, 'duration_seconds': remaining,
'duration_text': auth.format_credits(remaining)},
f'充值成功,+{credits} 次')
except ValueError as e:
await asyncio.to_thread(auth.add_log, 'fail', user.username, ip, '卡密充值', str(e))
await auth.add_log('fail', user.username, ip, '卡密充值', str(e))
return error(str(e))
except Exception:
return error('充值失败,请稍后重试', 500)
@@ -122,16 +121,16 @@ async def forgot_password(request: Request):
if not email:
return error('缺少邮箱')
try:
if not await asyncio.to_thread(auth.check_verify_code, email, code, 'reset'):
if not await auth.check_verify_code(email, code, 'reset'):
return error('邮箱验证码错误或已过期')
user = await asyncio.to_thread(auth.get_user_by_email, email)
user = await auth.get_user_by_email(email)
if not user:
return error('邮箱验证码错误或已过期')
if user['role'] == 'admin':
# 管理员不走自助找回(2FA 保护,统一防枚举文案)
return error('邮箱验证码错误或已过期')
await asyncio.to_thread(auth.reset_password, user['id'], newpwd)
await asyncio.to_thread(auth.add_log, 'success', user['username'],
await auth.reset_password(user['id'], newpwd)
await auth.add_log('success', user['username'],
ratelimit.client_ip(request), '自助找回密码', '邮箱验证码重置')
return success(message='密码已重置,请使用新密码登录(所有设备已下线)')
except Exception:
@@ -162,9 +161,9 @@ async def _reject_registration(message: str, *, email: str, code: str, ttl: int,
验证码在 check_verify_code 里已被 DELETE 原子消费,凡走到建号/查重/防刷
之后才失败的路径都必须还原,否则用户被迫重新收码。
"""
await asyncio.to_thread(auth.restore_verify_code, email, code, ttl, 'register')
await auth.restore_verify_code(email, code, ttl, 'register')
if log_detail:
await asyncio.to_thread(auth.add_log, 'fail', username, ip, '注册', log_detail)
await auth.add_log('fail', username, ip, '注册', log_detail)
return error(message)
@@ -185,10 +184,10 @@ async def _issue_registration_session(user, credits: int, username: str,
fresh = await repo.get_user_by_id(uid) or {}
seq = int(fresh.get('login_seq') or 1)
credits = min(credits, max(0, int(fresh.get('duration_seconds') or 0)))
await asyncio.to_thread(auth.add_log, 'success', username, ip, '注册账号',
await auth.add_log('success', username, ip, '注册账号',
f'赠送 {credits} 次' if credits else '未赠送次数')
secure = os.environ.get('ENABLE_SECURE_COOKIE', '').strip().lower() in ('1', 'true', 'yes')
return _login_cookie_response({
return await _login_cookie_response({
'user': _user_public(fresh),
'duration_text': auth.format_credits(int(fresh.get('duration_seconds') or 0)),
'redirect': 'admin' if fresh.get('role') == 'admin' else 'user',
@@ -217,25 +216,24 @@ async def register(request: Request):
return error(bad_input)
try:
code_ttl = await asyncio.to_thread(auth.check_verify_code, email, code, 'register')
code_ttl = await auth.check_verify_code(email, code, 'register')
if not code_ttl:
return error('邮箱验证码错误或已过期')
rej = {'email': email, 'code': code, 'ttl': code_ttl}
if await asyncio.to_thread(auth.get_user_by_name, username):
if await auth.get_user_by_name(username):
return await _reject_registration('用户名已被注册', **rej)
if email and await asyncio.to_thread(auth.get_user_by_email, email):
if email and await auth.get_user_by_email(email):
return await _reject_registration('该邮箱已被注册,请更换邮箱或联系管理员', **rej)
device_used_flag = bool(device_id) and await asyncio.to_thread(auth.device_used, device_id)
if ip and await asyncio.to_thread(auth.register_ip_exceeded, ip):
device_used_flag = bool(device_id) and await auth.device_used(device_id)
if ip and await auth.register_ip_exceeded(ip):
return await _reject_registration(
'当前网络注册过于频繁,请 24 小时后再试或联系管理员',
username=username, ip=ip, log_detail='同 IP 24h 注册次数超限', **rej)
credits = await _register_credits(device_used_flag)
try:
user = await asyncio.to_thread(
auth.create_user, username, password, email=email, credits=credits,
user = await auth.create_user(username, password, email=email, credits=credits,
guard=auth.RegistrationGuard(
device_id=device_id, ip=ip,
ua=str(request.headers.get('user-agent') or ''),
+2 -3
View File
@@ -12,7 +12,6 @@
这些端点读写 core.state 的共享任务表(与任务执行侧同一批对象),因此
看到的是同一份任务状态。
"""
import asyncio
import time
from typing import Annotated, Any
@@ -173,7 +172,7 @@ async def task_heartbeat(user: LoginUser, request: Request):
device_id = str(data.get('device_id') or '').strip()[:64]
if not device_id:
return error('缺少 device_id 参数')
await asyncio.to_thread(services_tasks._touch_device, device_id)
services_tasks._touch_device(device_id)
resumed = 0
with DOWNLOAD_TASKS_LOCK:
for task in DOWNLOAD_TASKS.values():
@@ -199,7 +198,7 @@ async def task_list(user: LoginUser, request: Request):
device_id = str(data.get('device_id') or '').strip()[:64]
if not device_id:
return error('缺少 device_id 参数')
await asyncio.to_thread(services_tasks._touch_device, device_id)
services_tasks._touch_device(device_id)
my_tasks = []
with DOWNLOAD_TASKS_LOCK:
for tid, task in DOWNLOAD_TASKS.items():
+1 -1
View File
@@ -135,7 +135,7 @@ async def user_qr_check(
try:
if user is not None:
import auth_system
await asyncio.to_thread(auth_system.set_user_netease_cookie, user.username, cookie)
await auth_system.set_user_netease_cookie(user.username, cookie)
except Exception:
pass
logger.info('个人账号扫码登录成功(Cookie 已返回浏览器本地保存)')
+9 -6
View File
@@ -7,7 +7,7 @@
import logging
import re
import uuid
from collections.abc import Callable
from collections.abc import Awaitable, Callable
from typing import Any
from core.config import MAX_ACTIVE_TASKS_PER_USER, MAX_TERMINAL_TASKS, MAX_TOTAL_TASKS
@@ -71,9 +71,12 @@ async def register_download_task(billing: Billing, task: dict[str, Any],
return None
async def start_task(task_id: str, worker: Callable, args: tuple = ()) -> str | None:
"""启动 worker 线程(复用 services 的 _start_task_thread,失败收敛终态并退费)。"""
import asyncio
async def start_task(task_id: str, worker: Callable[..., Awaitable[None]],
args: tuple = ()) -> str | None:
"""启动 worker 协程(复用 services 的 _start_task,失败收敛终态并退费)。
from services._taskcore import _start_task_thread
return await asyncio.to_thread(_start_task_thread, task_id, worker, args)
worker 已是协程函数:直接作为 asyncio 任务挂到当前事件循环(不再起线程),
任务句柄记在任务字典里供监控循环判断「worker 已结束但没人结算」的兜底场景。
"""
from services._taskcore import _start_task
return await _start_task(task_id, worker, args)