Archived
chore(async-infra): 异步基础层 + 工具链 + 清理
异步基础层(core/):
- db.py aiosqlite + WAL:读路径无锁并发、写路径 asyncio.Lock 串行,
busy_timeout 自旋——实现「读写同时进行」(旧实现用 RLock 全读串行)
- http_client.py aiohttp 连接池 + 有界重试(5xx 重试 / 4xx 不重试)+
响应体上限 + 流式接口;按事件循环隔离 session(支持迁移期双 loop)
工具链:
- pyproject.toml ruff(E/F/W/I/B/UP/C4/ASYNC/S/RUF) + pyright(basic) + pytest(asyncio auto)
- requirements-fastapi.txt FastAPI/aiohttp/aiosqlite/aiofiles 等目标栈
- requirements-dev.txt 开发依赖(pytest/ruff/pyright)
清理:删除无用脚本 _csscheck.py 与产物标记 _css_ok.txt/_pkg_ok.txt,同步 .gitignore
验证:pytest 10/10(含「写 200 条同时 4 读协程零异常」并发用例);
ruff All checks passed;pyright 0 errors。
This commit is contained in:
1 parent
41356e89c4
commit
9f56f68b66
16 files changed
+706
-42
No files matched your search
+10
-10
@@ -12,14 +12,14 @@ Flask/FastAPI 迁移期共用,并为后续封装成带锁的单例(State 对
|
||||
依赖仅限标准库 threading/collections——任何下载器/HTTP 相关对象
|
||||
(如批量打包信号量)不放在这里,随对应服务模块搬迁。
|
||||
"""
|
||||
|
||||
import collections
|
||||
import threading
|
||||
from typing import Any, Dict, List
|
||||
|
||||
from typing import Any
|
||||
|
||||
# ============ 短链展开结果缓存 ============
|
||||
# 用户反复解析同一分享链接不再重复请求 163cn.tv(有界防膨胀)
|
||||
SHORTLINK_CACHE: Dict[str, str] = {}
|
||||
SHORTLINK_CACHE: dict[str, str] = {}
|
||||
SHORTLINK_CACHE_LOCK = threading.Lock()
|
||||
|
||||
|
||||
@@ -37,23 +37,23 @@ BATCH_DURATIONS_LOCK = threading.Lock()
|
||||
# ============ 下载任务表 ============
|
||||
# (批量打包 + 单曲下载统一):task_id -> {type, total, done, failed, status, start, msg, cancel, pause, device_id, result, params}
|
||||
# 前端通过 /api/batch_progress 轮询进度,支持取消;完成后文件按 35 分钟保留期清理
|
||||
DOWNLOAD_TASKS: Dict[str, Dict[str, Any]] = {}
|
||||
DOWNLOAD_TASKS: dict[str, dict[str, Any]] = {}
|
||||
DOWNLOAD_TASKS_LOCK = threading.Lock()
|
||||
|
||||
|
||||
# ============ 设备在线心跳 ============
|
||||
# device_id -> 最近心跳时间。心跳超时(PAUSE_AFTER_SECONDS)未上线的设备,其批量任务
|
||||
# 自动暂停让出排队名额;超时超过 CANCEL_AFTER_SECONDS(30 分钟)则自动取消任务并删文件
|
||||
DEVICE_LAST_SEEN: Dict[str, float] = {}
|
||||
DEVICE_LAST_SEEN: dict[str, float] = {}
|
||||
DEVICE_LAST_SEEN_LOCK = threading.Lock()
|
||||
|
||||
|
||||
# ============ 接口请求统计 ============
|
||||
REQUEST_STATS: Dict[str, Dict[str, Any]] = {}
|
||||
REQUEST_STATS: dict[str, dict[str, Any]] = {}
|
||||
STATS_LOCK = threading.Lock()
|
||||
|
||||
# 用户解析/下载操作记录(IP);仅保留最近 2000 条
|
||||
REQ_LOG: List[Dict[str, Any]] = []
|
||||
REQ_LOG: list[dict[str, Any]] = []
|
||||
|
||||
|
||||
# ============ 上游探测缓存 ============
|
||||
@@ -62,10 +62,10 @@ GLOBAL_MUSIC_U_CACHE = {'v': None, 'ts': 0.0}
|
||||
|
||||
# 服务器网易云账号昵称缓存(10 分钟):非管理员可见响应中遮蔽真实昵称
|
||||
SERVER_NICK_CACHE = {'nick': None, 'ts': 0.0}
|
||||
SERVER_NICK_LOCK = threading.Lock() # 第 91 条(P3-9):刷新单飞锁,防并发重复打上游
|
||||
VIP_PROBE_LOCK = threading.Lock() # 第 100 条:音质探测单飞锁(有界等待)
|
||||
SERVER_NICK_LOCK = threading.Lock() # 第 91 条(P3-9):刷新单飞锁,防并发重复打上游
|
||||
VIP_PROBE_LOCK = threading.Lock() # 第 100 条:音质探测单飞锁(有界等待)
|
||||
|
||||
# 全局账号 VIP 探测缓存(5 分钟):probe 一次要打 8 次网易云接口,
|
||||
# 公网下账号栏高频刷新会连累服务器账号触发风控。
|
||||
VIP_CACHE = {'data': None, 'ts': 0.0}
|
||||
VIP_USER_CACHE = {} # user 路径按 cookie 哈希的 5 分钟缓存
|
||||
VIP_USER_CACHE = {} # user 路径按 cookie 哈希的 5 分钟缓存
|
||||
Reference in new issue
Block a user