Archived
refactor: auth_system 全面异步化（aiosqlite），拆掉 web/services 层的 to_thread 桥接
拆掉两类垫片的最后一处：auth_system 原为同步 sqlite3 栈，web 层与 services 的
worker 只能靠 asyncio.to_thread 逐个桥接才不阻塞事件循环——每次 DB 访问都白付一次
线程池调度，且「同步调异步」的边界散落在 100+ 个调用点。B1 已把 worker 变成协程，
本批把访问层本身也迁到原生异步，两条线在此合拢。
改动（按子系统）：
- auth_system：db.py 照 core/db.py 现有范式重建为原生异步访问层（aiosqlite + WAL）
——读路径 _q 每次独立连接、无锁、可并发；写路径 _x / write_transaction 用
asyncio.Lock 串行化，写者不阻塞读者。users / cards / credits / ledger / logs /
settings / mail / geo / backup 九模块随之全部 async/await；security.py 是纯计算
（pbkdf2/normalize）不动。__init__ 去掉 _DB_LOCK 与 get_conn 导出。
- web：app / deps / billing / tasks_registry 与 routers/{admin_site,auth,music,
register,user} 直接 await，拆掉包裹的 to_thread。deps.secret_key 改 async，
连带 _parse_cookie 也 async（会话解析要读密钥）。
- services：_taskcore / tasks / downloads / podcast 的账本调用与查询直接 await。
4 处 start_task 调用点（download.py）确认已 await。
设计取舍（为什么这么做，而不是直接包一层）：
- 锁按**事件循环**隔离（_LOOP_LOCKS 以 id(loop) 为键）：asyncio.Lock 绑定创建它的
loop，应用主 loop 与测试各自的 loop 并存，跨 loop 复用会抛 "bound to a different
event loop"。
- WAL 切换用**独立锁** _wal_lock 而非 _write_lock：execute() 已持写锁，再在锁内
connect→_configure 会自锁死（asyncio.Lock 非重入）。切换 + 标记在专用锁内串行并
二次复查，否则并发首连会同时执行 PRAGMA journal_mode=WAL 争排他锁报
database is locked（core/db.py 同款教训，实测复现）。
- force_wal 保留：journal_mode 持久化在库文件头，库文件被删重建时新文件不继承，
必须重设。exists() 判定须在 aiosqlite.connect **之前**（connect 会创建空文件）。
- connect() 用 DB_PATH.parent 兜底建目录，优于 DATA_DIR：生产两者相同，但测试
monkeypatch 只换 DB_PATH 时兜底仍能命中正确目录。
留在 to_thread 上的仅剩**正当的阻塞卸载**，不是同步门面：PBKDF2 的 10 万次迭代
（登录链路每次都要跑）、smtplib 发信、备份文件的 copy2 / os.replace、secret.key
的文件 IO。原有 aria2 / downloader / zip 侧的 to_thread 未动。
不变量逐条保持（迁移只改执行模型，不改语义）：
- ledger_settle 幂等：账本非 open 直接 return 0，重启后重放不重复退款；
- 终态先到先得：_mark_task_terminal 对已 done/error/canceled 的任务直接 return，
堵住 canceled→error 的非法迁移边；
- settle 必须在任务锁**外**调用（ledger_settle 要抢 SQLite 写锁，busy 最长 15s，
持任务锁会让全站进度/提交/取消随库停摆）；
- reserve_and_open 的预扣与开账同事务，消除原两步之间的崩溃丢账窗口。
路径真源红线：DATA_DIR / DB_PATH / BACKUP_DIR / SECRET_KEY_PATH 仍是 auth_system.db
的**模块全局**（用与 core.db 相同的表达式，但**不是**它的别名）。测试只 patch
auth_system.db.*，若改成 core.db 的别名会静默把测试写到生产库。已核 data/site.db
读写前后 sha256 完全一致。
顺带清掉 get_conn：迁移后它只剩「转调 connect()」一层空壳、全仓零消费者（测试
直接开裸连接、备份用 db.connect()），保留等于留一条「拿裸连接自己写」的口子。
测试：19 个文件同步迁移——10 个文件的 web.deps.secret_key 替身改 async；
test_db_conn.py 的多线程首连用例按异步语义拆成 2 条（单 loop gather(16) 覆盖批量
并发首连、WAL 就绪后 8 线程各自 loop 建连覆盖跨 loop）；test_billing_scope.py 去掉
多余的 @pytest.mark.anyio；被 await 且**正常返回**的 monkeypatch 替身一律改 async
（仅 raise 的同步替身仍可用）。
门禁：ruff 全过 / pyright 0-0-0（standard 模式含 tests 与顶层模块）/ pytest
384 项全绿（0 失败 0 error 0 skipped；383→384 因 test_db_conn 拆用例）。
data/site.db 读写前后 sha256 一致（a19f5db…6487），无 -wal/-shm 边车残留。
下一步 B3：删 core/sync_http.py、core/async_bridge.py、core/http_errors.py 及
tests/test_async_bridge.py——三者在生产侧已零消费者（http_errors 仅 sync_http
自用，requests 已非声明依赖）。
This commit is contained in:
snowy-yang committed
1 parent
69a50c2c31
commit
96af56cc27
43 files changed
+1171
-1009
No files matched your search
+67
-97
@@ -1,7 +1,7 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
"""次数卡密生成 / 兑换 / 管理。
|
||||
|
||||
（自 auth_system.py 拆出，行为与原实现一致。）
|
||||
（自 auth_system.py 拆出，行为与原实现一致；DB 访问已迁移为原生异步。）
|
||||
"""
|
||||
|
||||
import secrets
|
||||
@@ -9,16 +9,15 @@ import time
|
||||
from typing import Any
|
||||
|
||||
from .db import (
|
||||
_DB_LOCK,
|
||||
_q,
|
||||
get_conn,
|
||||
write_transaction,
|
||||
)
|
||||
|
||||
# ============ 卡密 ============
|
||||
|
||||
|
||||
def generate_cards(count: int, credits: int, created_by: str = '',
|
||||
prefix: str = '', note: str = '', valid_days: int = 0) -> list[str]:
|
||||
async def generate_cards(count: int, credits: int, created_by: str = '',
|
||||
prefix: str = '', note: str = '', valid_days: int = 0) -> list[str]:
|
||||
"""生成次数卡密（credits = 每张卡可下载次数）
|
||||
prefix: 自定义卡密前缀（默认 VIP-）
|
||||
note: 备注信息
|
||||
@@ -28,29 +27,24 @@ def generate_cards(count: int, credits: int, created_by: str = '',
|
||||
prefix = ''.join(ch for ch in (prefix or 'VIP') if ch.isalnum()).upper()[:10] or 'VIP'
|
||||
expire_at = int(time.time()) + int(valid_days) * 86400 if valid_days and int(valid_days) > 0 else 0
|
||||
codes = []
|
||||
with _DB_LOCK:
|
||||
conn = get_conn()
|
||||
try:
|
||||
for _ in range(count):
|
||||
code = prefix + '-' + secrets.token_hex(5).upper()
|
||||
conn.execute(
|
||||
'INSERT INTO cards (code, minutes, created_by, created_at, note, expire_at) '
|
||||
'VALUES (?, ?, ?, ?, ?, ?)',
|
||||
(code, int(credits), created_by, int(time.time()), note or '', expire_at))
|
||||
codes.append(code)
|
||||
conn.commit()
|
||||
finally:
|
||||
conn.close()
|
||||
async with write_transaction() as conn:
|
||||
for _ in range(count):
|
||||
code = prefix + '-' + secrets.token_hex(5).upper()
|
||||
await conn.execute(
|
||||
'INSERT INTO cards (code, minutes, created_by, created_at, note, expire_at) '
|
||||
'VALUES (?, ?, ?, ?, ?, ?)',
|
||||
(code, int(credits), created_by, int(time.time()), note or '', expire_at))
|
||||
codes.append(code)
|
||||
return codes
|
||||
|
||||
|
||||
def list_cards(limit: int = 100, offset: int = 0) -> tuple[list[dict[str, Any]], int]:
|
||||
async def list_cards(limit: int = 100, offset: int = 0) -> tuple[list[dict[str, Any]], int]:
|
||||
"""卡密列表（支持分页 limit/offset，管理路由透传）
|
||||
返回 (当页卡密列表, 卡密总数)——总数供前端分页控件用，调用方按元组解包"""
|
||||
limit = max(1, min(int(limit), 500))
|
||||
offset = max(0, int(offset))
|
||||
rows = _q('SELECT * FROM cards ORDER BY id DESC LIMIT ? OFFSET ?', (limit, offset))
|
||||
total_row = _q('SELECT COUNT(*) AS c FROM cards')
|
||||
rows = await _q('SELECT * FROM cards ORDER BY id DESC LIMIT ? OFFSET ?', (limit, offset))
|
||||
total_row = await _q('SELECT COUNT(*) AS c FROM cards')
|
||||
total = int(total_row[0]['c']) if total_row else 0
|
||||
now = time.time()
|
||||
cards = []
|
||||
@@ -62,89 +56,65 @@ def list_cards(limit: int = 100, offset: int = 0) -> tuple[list[dict[str, Any]],
|
||||
return cards, total
|
||||
|
||||
|
||||
def set_card_disabled(code: str, disabled: bool) -> None:
|
||||
async def set_card_disabled(code: str, disabled: bool) -> None:
|
||||
"""禁用/恢复未使用卡密（已使用卡密无禁用意义，拒绝操作）"""
|
||||
code = (code or '').strip().upper()
|
||||
with _DB_LOCK:
|
||||
conn = get_conn()
|
||||
try:
|
||||
row = conn.execute('SELECT used_by, disabled FROM cards WHERE code = ?', (code,)).fetchone()
|
||||
if not row:
|
||||
raise ValueError('卡密不存在')
|
||||
if row['used_by']:
|
||||
raise ValueError('卡密已被使用，无需禁用')
|
||||
if bool(row['disabled']) == bool(disabled):
|
||||
raise ValueError('卡密已处于该状态')
|
||||
conn.execute('UPDATE cards SET disabled = ? WHERE code = ?',
|
||||
(1 if disabled else 0, code))
|
||||
conn.commit()
|
||||
except Exception:
|
||||
conn.rollback()
|
||||
raise
|
||||
finally:
|
||||
conn.close()
|
||||
async with write_transaction() as conn:
|
||||
row = await (await conn.execute('SELECT used_by, disabled FROM cards WHERE code = ?', (code,))).fetchone()
|
||||
if not row:
|
||||
raise ValueError('卡密不存在')
|
||||
if row['used_by']:
|
||||
raise ValueError('卡密已被使用，无需禁用')
|
||||
if bool(row['disabled']) == bool(disabled):
|
||||
raise ValueError('卡密已处于该状态')
|
||||
await conn.execute('UPDATE cards SET disabled = ? WHERE code = ?',
|
||||
(1 if disabled else 0, code))
|
||||
|
||||
|
||||
def delete_card(code: str) -> None:
|
||||
async def delete_card(code: str) -> None:
|
||||
"""删除未使用卡密（已使用卡密含兑换凭证：使用者/时间，按 owner 规则不可删）"""
|
||||
code = (code or '').strip().upper()
|
||||
with _DB_LOCK:
|
||||
conn = get_conn()
|
||||
try:
|
||||
row = conn.execute('SELECT used_by FROM cards WHERE code = ?', (code,)).fetchone()
|
||||
if not row:
|
||||
raise ValueError('卡密不存在')
|
||||
if row['used_by']:
|
||||
raise ValueError('已使用的卡密不可删除（保留兑换凭证）')
|
||||
conn.execute('DELETE FROM cards WHERE code = ?', (code,))
|
||||
conn.commit()
|
||||
except Exception:
|
||||
conn.rollback()
|
||||
raise
|
||||
finally:
|
||||
conn.close()
|
||||
async with write_transaction() as conn:
|
||||
row = await (await conn.execute('SELECT used_by FROM cards WHERE code = ?', (code,))).fetchone()
|
||||
if not row:
|
||||
raise ValueError('卡密不存在')
|
||||
if row['used_by']:
|
||||
raise ValueError('已使用的卡密不可删除（保留兑换凭证）')
|
||||
await conn.execute('DELETE FROM cards WHERE code = ?', (code,))
|
||||
|
||||
|
||||
def redeem_card(code: str, username: str) -> tuple[int, int]:
|
||||
async def redeem_card(code: str, username: str) -> tuple[int, int]:
|
||||
"""兑换卡密，返回 (卡密次数, 新剩余次数)（标记已用与加余额同事务，失败整体回滚不吞卡）"""
|
||||
code = (code or '').strip().upper()
|
||||
with _DB_LOCK:
|
||||
conn = get_conn()
|
||||
try:
|
||||
row = conn.execute('SELECT * FROM cards WHERE code = ?', (code,)).fetchone()
|
||||
if not row:
|
||||
raise ValueError('卡密不存在')
|
||||
if row['used_by']:
|
||||
raise ValueError('卡密已被使用')
|
||||
if row['disabled']:
|
||||
raise ValueError('卡密已被禁用，请联系管理员')
|
||||
if row['expire_at'] and row['expire_at'] < time.time():
|
||||
raise ValueError('卡密已过期')
|
||||
# 卡密标记 + 余额增加同事务：任一失败整体回滚，卡密可重试兑换。
|
||||
# WHERE 带 used_by=""/disabled=0 防重标记 + rowcount 校验：命中 0 行
|
||||
# （跨进程并发抢卡）立即回滚报错，绝不出现"未标记已用却加了余额"
|
||||
cur = conn.execute(
|
||||
'UPDATE cards SET used_by = ?, used_at = ? '
|
||||
'WHERE code = ? AND used_by = "" AND disabled = 0',
|
||||
(username, int(time.time()), code))
|
||||
if cur.rowcount != 1:
|
||||
raise ValueError('卡密已被使用')
|
||||
urow = conn.execute('SELECT role, duration_seconds FROM users WHERE username = ?',
|
||||
(username,)).fetchone()
|
||||
if not urow:
|
||||
raise ValueError('用户不存在')
|
||||
minutes = int(row['minutes'])
|
||||
remaining = int(urow['duration_seconds'] or 0)
|
||||
if urow['role'] == 'admin' or remaining == -1:
|
||||
remaining = -1
|
||||
else:
|
||||
remaining = remaining + minutes
|
||||
conn.execute('UPDATE users SET duration_seconds = ? WHERE username = ?',
|
||||
(remaining, username))
|
||||
conn.commit()
|
||||
return minutes, remaining
|
||||
except Exception:
|
||||
conn.rollback()
|
||||
raise
|
||||
finally:
|
||||
conn.close()
|
||||
async with write_transaction() as conn:
|
||||
row = await (await conn.execute('SELECT * FROM cards WHERE code = ?', (code,))).fetchone()
|
||||
if not row:
|
||||
raise ValueError('卡密不存在')
|
||||
if row['used_by']:
|
||||
raise ValueError('卡密已被使用')
|
||||
if row['disabled']:
|
||||
raise ValueError('卡密已被禁用，请联系管理员')
|
||||
if row['expire_at'] and row['expire_at'] < time.time():
|
||||
raise ValueError('卡密已过期')
|
||||
# 卡密标记 + 余额增加同事务：任一失败整体回滚，卡密可重试兑换。
|
||||
# WHERE 带 used_by=""/disabled=0 防重标记 + rowcount 校验：命中 0 行
|
||||
# （跨进程并发抢卡）立即回滚报错，绝不出现"未标记已用却加了余额"
|
||||
cur = await conn.execute(
|
||||
'UPDATE cards SET used_by = ?, used_at = ? '
|
||||
'WHERE code = ? AND used_by = "" AND disabled = 0',
|
||||
(username, int(time.time()), code))
|
||||
if cur.rowcount != 1:
|
||||
raise ValueError('卡密已被使用')
|
||||
urow = await (await conn.execute('SELECT role, duration_seconds FROM users WHERE username = ?',
|
||||
(username,))).fetchone()
|
||||
if not urow:
|
||||
raise ValueError('用户不存在')
|
||||
minutes = int(row['minutes'])
|
||||
remaining = int(urow['duration_seconds'] or 0)
|
||||
if urow['role'] == 'admin' or remaining == -1:
|
||||
remaining = -1
|
||||
else:
|
||||
remaining = remaining + minutes
|
||||
await conn.execute('UPDATE users SET duration_seconds = ? WHERE username = ?',
|
||||
(remaining, username))
|
||||
return minutes, remaining
|
||||
Reference in new issue
Block a user