chore(async-infra): 异步基础层 + 工具链 + 清理

异步基础层(core/):
- db.py   aiosqlite + WAL:读路径无锁并发、写路径 asyncio.Lock 串行,
          busy_timeout 自旋——实现「读写同时进行」(旧实现用 RLock 全读串行)
- http_client.py  aiohttp 连接池 + 有界重试(5xx 重试 / 4xx 不重试)+
          响应体上限 + 流式接口;按事件循环隔离 session(支持迁移期双 loop)

工具链:
- pyproject.toml   ruff(E/F/W/I/B/UP/C4/ASYNC/S/RUF) + pyright(basic) + pytest(asyncio auto)
- requirements-fastapi.txt  FastAPI/aiohttp/aiosqlite/aiofiles 等目标栈
- requirements-dev.txt      开发依赖(pytest/ruff/pyright)

清理:删除无用脚本 _csscheck.py 与产物标记 _css_ok.txt/_pkg_ok.txt,同步 .gitignore

验证:pytest 10/10(含「写 200 条同时 4 读协程零异常」并发用例);
ruff All checks passed;pyright 0 errors。
This commit is contained in:
snowy-yang committed 2026-09-26 00:24:00 +08:00
1 parent 41356e89c4
commit 9f56f68b66
16 files changed
+706 -42

No files matched your search

+137
View File
@@ -0,0 +1,137 @@
# -*- coding: utf-8 -*-
"""异步 SQLite 访问层(aiosqlite + WAL)。
核心目标:**读写同时进行**。
- WAL 模式:读事务与写事务互不阻塞(读快照隔离),这是 SQLite 层并发的根本;
- 读路径:每次操作独立连接、无锁、可并发(`fetch_all`/`fetch_one`/`fetch_val`);
- 写路径:SQLite 全库同一时刻只允许一个写者,用 `asyncio.Lock` 串行化写事务,
但写者不阻塞读者(WAL);
- `busy_timeout`:多进程/多连接下遇到短暂写锁自旋等待而非立刻 SQLITE_BUSY。
与旧实现的差异:旧 `auth_system` 用 `threading.RLock` 把**全部**读写串行化,
读也被写挡住;本层只串行化写,读放行并发,即「读写同时进行」。
"""
import asyncio
import os
from collections.abc import AsyncIterator, Iterable, Sequence
from contextlib import asynccontextmanager
from pathlib import Path
from typing import Any
import aiosqlite
# 站点数据目录(可经环境变量覆盖,便于测试隔离)
DATA_DIR = Path(os.environ.get('SITE_DATA_DIR', 'data'))
DB_PATH = DATA_DIR / 'site.db'
# 写事务串行锁(惰性创建,避免跨事件循环绑定)
_WRITE_LOCK: asyncio.Lock | None = None
def _write_lock() -> asyncio.Lock:
global _WRITE_LOCK
if _WRITE_LOCK is None:
_WRITE_LOCK = asyncio.Lock()
return _WRITE_LOCK
async def _configure(conn: aiosqlite.Connection) -> None:
conn.row_factory = aiosqlite.Row
await conn.execute('PRAGMA journal_mode=WAL') # 读写并发的前提
await conn.execute('PRAGMA busy_timeout=15000') # 遇写锁自旋等待,而非立即报错
await conn.execute('PRAGMA synchronous=NORMAL') # WAL 下兼顾安全与吞吐
await conn.execute('PRAGMA foreign_keys=ON')
async def connect() -> aiosqlite.Connection:
"""新建一个已配置好的连接(调用方负责 close)。"""
DATA_DIR.mkdir(parents=True, exist_ok=True) # noqa: ASYNC240 - 幂等的微秒级一次目录创建,无需线程池
conn = await aiosqlite.connect(str(DB_PATH), timeout=15)
await _configure(conn)
return conn
# ============ 读路径(并发,无锁) ============
async def fetch_all(sql: str, params: Sequence = ()) -> list[aiosqlite.Row]:
conn = await connect()
try:
cur = await conn.execute(sql, params)
return list(await cur.fetchall())
finally:
await conn.close()
async def fetch_one(sql: str, params: Sequence = ()) -> aiosqlite.Row | None:
conn = await connect()
try:
cur = await conn.execute(sql, params)
return await cur.fetchone()
finally:
await conn.close()
async def fetch_val(sql: str, params: Sequence = (), default: Any = None) -> Any:
row = await fetch_one(sql, params)
return row[0] if row is not None else default
# ============ 写路径(串行,不阻塞读) ============
async def execute(sql: str, params: Sequence = ()) -> int:
"""执行单条写语句,返回 lastrowid(INSERT)或 rowcount(UPDATE/DELETE)。"""
async with _write_lock():
conn = await connect()
try:
cur = await conn.execute(sql, params)
await conn.commit()
return cur.lastrowid if cur.lastrowid else cur.rowcount
finally:
await conn.close()
async def execute_many(sql: str, seq: Iterable[Sequence]) -> int:
"""批量写,返回受影响行数。"""
async with _write_lock():
conn = await connect()
try:
cur = await conn.executemany(sql, seq)
await conn.commit()
return cur.rowcount
finally:
await conn.close()
@asynccontextmanager
async def write_transaction() -> AsyncIterator[aiosqlite.Connection]:
"""写事务上下文:多条写语句原子提交(串行锁内)。
用法:
async with write_transaction() as conn:
await conn.execute(...)
await conn.execute(...)
# 退出即 commit;异常则 rollback
"""
async with _write_lock():
conn = await connect()
try:
yield conn
await conn.commit()
except Exception:
await conn.rollback()
raise
finally:
await conn.close()
@asynccontextmanager
async def read_connection() -> AsyncIterator[aiosqlite.Connection]:
"""只读连接上下文(无锁、可并发);需要在一个连接上做多次读时使用。"""
conn = await connect()
try:
yield conn
finally:
await conn.close()