This repository has been archived on 2026-10-02. You can view files and clone it. You cannot open issues or pull requests or push a commit.
Files
NeteaseUrl/auth_system/users.py
T
snowy-yang 96af56cc27 refactor: auth_system 全面异步化(aiosqlite),拆掉 web/services 层的 to_thread 桥接
拆掉两类垫片的最后一处:auth_system 原为同步 sqlite3 栈,web 层与 services 的
worker 只能靠 asyncio.to_thread 逐个桥接才不阻塞事件循环——每次 DB 访问都白付一次
线程池调度,且「同步调异步」的边界散落在 100+ 个调用点。B1 已把 worker 变成协程,
本批把访问层本身也迁到原生异步,两条线在此合拢。

改动(按子系统):
- auth_system:db.py 照 core/db.py 现有范式重建为原生异步访问层(aiosqlite + WAL)
  ——读路径 _q 每次独立连接、无锁、可并发;写路径 _x / write_transaction 用
  asyncio.Lock 串行化,写者不阻塞读者。users / cards / credits / ledger / logs /
  settings / mail / geo / backup 九模块随之全部 async/await;security.py 是纯计算
  (pbkdf2/normalize)不动。__init__ 去掉 _DB_LOCK 与 get_conn 导出。
- web:app / deps / billing / tasks_registry 与 routers/{admin_site,auth,music,
  register,user} 直接 await,拆掉包裹的 to_thread。deps.secret_key 改 async,
  连带 _parse_cookie 也 async(会话解析要读密钥)。
- services:_taskcore / tasks / downloads / podcast 的账本调用与查询直接 await。
  4 处 start_task 调用点(download.py)确认已 await。

设计取舍(为什么这么做,而不是直接包一层):
- 锁按**事件循环**隔离(_LOOP_LOCKS 以 id(loop) 为键):asyncio.Lock 绑定创建它的
  loop,应用主 loop 与测试各自的 loop 并存,跨 loop 复用会抛 "bound to a different
  event loop"。
- WAL 切换用**独立锁** _wal_lock 而非 _write_lock:execute() 已持写锁,再在锁内
  connect→_configure 会自锁死(asyncio.Lock 非重入)。切换 + 标记在专用锁内串行并
  二次复查,否则并发首连会同时执行 PRAGMA journal_mode=WAL 争排他锁报
  database is locked(core/db.py 同款教训,实测复现)。
- force_wal 保留:journal_mode 持久化在库文件头,库文件被删重建时新文件不继承,
  必须重设。exists() 判定须在 aiosqlite.connect **之前**(connect 会创建空文件)。
- connect() 用 DB_PATH.parent 兜底建目录,优于 DATA_DIR:生产两者相同,但测试
  monkeypatch 只换 DB_PATH 时兜底仍能命中正确目录。

留在 to_thread 上的仅剩**正当的阻塞卸载**,不是同步门面:PBKDF2 的 10 万次迭代
(登录链路每次都要跑)、smtplib 发信、备份文件的 copy2 / os.replace、secret.key
的文件 IO。原有 aria2 / downloader / zip 侧的 to_thread 未动。

不变量逐条保持(迁移只改执行模型,不改语义):
- ledger_settle 幂等:账本非 open 直接 return 0,重启后重放不重复退款;
- 终态先到先得:_mark_task_terminal 对已 done/error/canceled 的任务直接 return,
  堵住 canceled→error 的非法迁移边;
- settle 必须在任务锁**外**调用(ledger_settle 要抢 SQLite 写锁,busy 最长 15s,
  持任务锁会让全站进度/提交/取消随库停摆);
- reserve_and_open 的预扣与开账同事务,消除原两步之间的崩溃丢账窗口。

路径真源红线:DATA_DIR / DB_PATH / BACKUP_DIR / SECRET_KEY_PATH 仍是 auth_system.db
的**模块全局**(用与 core.db 相同的表达式,但**不是**它的别名)。测试只 patch
auth_system.db.*,若改成 core.db 的别名会静默把测试写到生产库。已核 data/site.db
读写前后 sha256 完全一致。
顺带清掉 get_conn:迁移后它只剩「转调 connect()」一层空壳、全仓零消费者(测试
直接开裸连接、备份用 db.connect()),保留等于留一条「拿裸连接自己写」的口子。

测试:19 个文件同步迁移——10 个文件的 web.deps.secret_key 替身改 async;
test_db_conn.py 的多线程首连用例按异步语义拆成 2 条(单 loop gather(16) 覆盖批量
并发首连、WAL 就绪后 8 线程各自 loop 建连覆盖跨 loop);test_billing_scope.py 去掉
多余的 @pytest.mark.anyio;被 await 且**正常返回**的 monkeypatch 替身一律改 async
(仅 raise 的同步替身仍可用)。

门禁:ruff 全过 / pyright 0-0-0(standard 模式含 tests 与顶层模块)/ pytest
384 项全绿(0 失败 0 error 0 skipped;383→384 因 test_db_conn 拆用例)。
data/site.db 读写前后 sha256 一致(a19f5db…6487),无 -wal/-shm 边车残留。

下一步 B3:删 core/sync_http.py、core/async_bridge.py、core/http_errors.py 及
 tests/test_async_bridge.py——三者在生产侧已零消费者(http_errors 仅 sync_http
 自用,requests 已非声明依赖)。
2026-09-28 20:13:19 +08:00

249 lines
10 KiB
Python
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# -*- coding: utf-8 -*-
"""用户增删改查、登录校验与状态。
(自 auth_system.py 拆出,行为与原实现一致;DB 访问已迁移为原生异步。)
"""
import asyncio
import json
import sqlite3
import time
from dataclasses import dataclass
from typing import Any
from .db import (
_q,
_x,
write_transaction,
)
from .security import (
_hash_new,
_normalize_email,
_verify_password,
)
@dataclass(frozen=True)
class RegistrationGuard:
"""注册风控上下文:同 IP 频率闸 + 设备赠送闸。
这些检查必须与建号 INSERT 在**同一写事务内**完成——锁外预检存在
TOCTOU(并发同设备双注册各得一次赠送、并发同 IP 挤过限额),所以它们不是
「可选的附加参数」,而是事务内必须可见的输入。管理建号无风控需求,不传即可。
"""
device_id: str = ''
ip: str = ''
ua: str = ''
ip_limit: int = 0
ip_window: int = 86400
device_once: bool = False
async def create_user(username: str, password: str, email: str = '', credits: int = 0,
role: str = 'user', *,
guard: RegistrationGuard | None = None) -> sqlite3.Row | None:
# 余额只接受 -1(不限)或 >= 0;非法负值一律按 0 处理,禁止出现 -2 这类"假不限"
try:
credits = int(credits)
except (TypeError, ValueError):
credits = 0
if credits < -1:
credits = 0
# 注册入口传 guard=RegistrationGuard(ip_limit=3, device_once=True)——同 IP 24h
# 注册数检查在写事务内与 INSERT 原子完成,堵住「锁外检查 + 并发插入」
# 的 TOCTOU 刷号;管理建号不传 guard(即不限制)。超限抛 ValueError 由调用方转 400/429。
guard = guard or RegistrationGuard()
now = int(time.time())
# PBKDF2 10 万次迭代是 CPU 密集(~50ms):在事务外先算好,避免持写锁时
# 阻塞事件循环(正当的 CPU 卸载,非同步门面)。
phash = await asyncio.to_thread(_hash_new, password)
async with write_transaction() as conn:
if guard.ip_limit > 0 and guard.ip:
cur = await conn.execute(
'SELECT COUNT(*) AS n FROM users WHERE register_ip = ? AND created_at > ?',
(guard.ip, now - guard.ip_window))
row = await cur.fetchone()
n = int(row['n']) if row else 0
if n >= guard.ip_limit:
raise ValueError('当前网络注册过于频繁,请 24 小时后再试或联系管理员')
if guard.device_once and guard.device_id:
# 设备赠送闸并入同一事务——锁外 device_used 预检有 TOCTOU,
# 并发同设备双注册可各得一次赠送;锁内复查命中则本次建号不赠送(账号照常创建)
cur = await conn.execute(
"SELECT COUNT(*) AS n FROM users WHERE trial_device = ? AND role != 'admin'",
(guard.device_id,))
row = await cur.fetchone()
if row and int(row['n']) > 0:
credits = 0
await conn.execute(
'INSERT INTO users (username, password_hash, salt, email, role, duration_seconds, '
'trial_device, register_ip, register_ua, last_active, created_at) '
'VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)',
(username, phash, '', email, role,
-1 if role == 'admin' else int(credits),
guard.device_id, guard.ip, guard.ua, now, now))
return await get_user_by_name(username)
async def get_user_by_name(username: str) -> sqlite3.Row | None:
rows = await _q('SELECT * FROM users WHERE username = ?', (username,))
return rows[0] if rows else None
async def get_user_by_email(email: str) -> sqlite3.Row | None:
"""按邮箱查用户(注册/建号邮箱查重的友好提示;库侧另有唯一索引兜底并发)"""
email = _normalize_email(email)
if not email:
return None
rows = await _q('SELECT * FROM users WHERE email = ? LIMIT 1', (email,))
return rows[0] if rows else None
async def register_ip_exceeded(ip: str, limit: int = 3, window: int = 86400) -> bool:
"""同 IP 24h 内注册数超限(默认 3 次)——防 IP 轮换刷免费赠送次数;
设备指纹可被客户端伪造,IP 维度是第二道闸。家庭 NAT 场景超限可由管理员建号兜底。"""
if not ip:
return False
rows = await _q('SELECT COUNT(*) AS n FROM users WHERE register_ip = ? AND created_at > ?',
(ip, int(time.time() - window)))
return bool(rows and rows[0]['n'] >= limit)
async def get_user_by_id(uid: int) -> sqlite3.Row | None:
rows = await _q('SELECT * FROM users WHERE id = ?', (uid,))
return rows[0] if rows else None
async def verify_login(username: str, password: str) -> sqlite3.Row | None:
"""校验用户名密码。密码正确即返回用户行(含被禁用/封禁账号——
状态语义由登录路由判断:禁用显示"已被禁用"、封禁显示封禁原因。
原实现把禁用账号提前过滤成 None,路由的"账号已被禁用"分支永不可达)"""
user = await get_user_by_name(username)
if user:
# PBKDF2 校验是 CPU 密集(~50ms),卸载到线程避免阻塞事件循环(正当 CPU 卸载)
ok, needs_upgrade = await asyncio.to_thread(
_verify_password, user['password_hash'], password, user['salt'])
if ok:
if needs_upgrade:
# 存量 SHA-256 哈希在登录成功时透明升级为 PBKDF2
new_hash = await asyncio.to_thread(_hash_new, password)
await _x('UPDATE users SET password_hash = ?, salt = \'\' WHERE id = ?',
(new_hash, user['id']))
return user
return None
async def list_users() -> list[dict[str, Any]]:
rows = await _q('SELECT * FROM users ORDER BY id DESC')
return [_user_public(r, full=True) for r in rows]
async def delete_user(uid: int) -> bool:
"""删除用户(卡密 used_by 为用户名字符串无外键——删除时同步
标注"(已注销#id)",管理端卡密列表不再显示悬空的已删用户名)"""
u = await get_user_by_id(uid)
if not u or u['role'] == 'admin':
return False
async with write_transaction() as conn:
name = u['username']
await conn.execute('DELETE FROM users WHERE id = ? AND role != ?', (uid, 'admin'))
await conn.execute("UPDATE cards SET used_by = used_by || ? WHERE used_by = ?",
(f'(已注销#{uid})', name))
return True
async def reset_password(uid: int, new_password: str) -> bool:
"""重置密码(同时递增 login_seq,让所有已签发会话立即失效)"""
new_hash = await asyncio.to_thread(_hash_new, new_password)
await _x('UPDATE users SET password_hash = ?, salt = \'\', '
'login_seq = COALESCE(login_seq, 0) + 1 WHERE id = ?',
(new_hash, uid))
return True
async def set_user_status(uid: int, status: int) -> bool:
"""启用 / 禁用账号(1 启用,0 禁用);管理员账号不可禁用"""
if status not in (0, 1):
return False
row = await _q('SELECT role FROM users WHERE id = ?', (uid,))
if row and row[0]['role'] == 'admin':
return False
return bool(await _x('UPDATE users SET status = ? WHERE id = ?', (status, uid)))
async def ban_user(uid: int, reason: str = '') -> bool:
"""封禁账号(管理员不可封)"""
row = await _q('SELECT role FROM users WHERE id = ?', (uid,))
if not row or row[0]['role'] == 'admin':
return False
return bool(await _x('UPDATE users SET status = 2, ban_reason = ? WHERE id = ?',
(str(reason or '')[:200], uid)))
async def unban_user(uid: int) -> bool:
"""解封账号,恢复正常(清空触发封禁的省份历史,否则下次登录立即再封)"""
return bool(await _x('UPDATE users SET status = 1, ban_reason = \'\', recent_provinces = \'[]\' WHERE id = ?', (uid,)))
async def set_user_credits(uid: int, count: int) -> None:
"""设置用户剩余次数(-1 表示不限)"""
count = -1 if count < 0 else int(count)
await _x('UPDATE users SET duration_seconds = ? WHERE id = ?', (count, uid))
async def set_user_password(uid: int, password: str) -> None:
# 新哈希 + 递增 login_seq(所有旧会话立即失效)
new_hash = await asyncio.to_thread(_hash_new, password)
await _x('UPDATE users SET password_hash = ?, salt = \'\', '
'login_seq = COALESCE(login_seq, 0) + 1 WHERE id = ?',
(new_hash, uid))
async def set_user_netease_cookie(username: str, cookie: str) -> None:
await _x('UPDATE users SET netease_cookie = ? WHERE username = ?', (cookie, username))
def _user_public(r: sqlite3.Row, full: bool = False) -> dict[str, Any]:
d = {
'id': r['id'],
'username': r['username'],
'email': r['email'],
'role': r['role'],
'duration_seconds': r['duration_seconds'],
'created_at': r['created_at'],
'last_login': r['last_login'],
'ann_seen': r['ann_seen'],
}
if full:
d.update({
'status': r['status'] if 'status' in r.keys() else 1,
'trial_device': r['trial_device'],
'register_ip': r['register_ip'],
'register_ua': r['register_ua'],
# 列表不再携带明文网易云 Cookie(前端只用布尔值;原文按需走
# /admin/site/user/cookie 单查接口,避免一次列表请求扩散全部用户凭据)
'has_netease_cookie': bool(r['netease_cookie']),
'ban_reason': r['ban_reason'] if 'ban_reason' in r.keys() else '',
'last_province': _last_province(r),
})
return d
def _last_province(r: sqlite3.Row) -> str:
try:
history = json.loads(r['recent_provinces'] if 'recent_provinces' in r.keys() and r['recent_provinces'] else '[]')
return history[-1].get('p', '') if history else ''
except Exception:
return ''
# ============ 免费体验(设备识别) ============
async def device_used(device_id: str) -> bool:
if not device_id:
return False
rows = await _q("SELECT id FROM users WHERE trial_device = ? AND role != 'admin' LIMIT 1", (device_id,))
return bool(rows)