Archived
拆掉两类垫片的最后一处:auth_system 原为同步 sqlite3 栈,web 层与 services 的
worker 只能靠 asyncio.to_thread 逐个桥接才不阻塞事件循环——每次 DB 访问都白付一次
线程池调度,且「同步调异步」的边界散落在 100+ 个调用点。B1 已把 worker 变成协程,
本批把访问层本身也迁到原生异步,两条线在此合拢。
改动(按子系统):
- auth_system:db.py 照 core/db.py 现有范式重建为原生异步访问层(aiosqlite + WAL)
——读路径 _q 每次独立连接、无锁、可并发;写路径 _x / write_transaction 用
asyncio.Lock 串行化,写者不阻塞读者。users / cards / credits / ledger / logs /
settings / mail / geo / backup 九模块随之全部 async/await;security.py 是纯计算
(pbkdf2/normalize)不动。__init__ 去掉 _DB_LOCK 与 get_conn 导出。
- web:app / deps / billing / tasks_registry 与 routers/{admin_site,auth,music,
register,user} 直接 await,拆掉包裹的 to_thread。deps.secret_key 改 async,
连带 _parse_cookie 也 async(会话解析要读密钥)。
- services:_taskcore / tasks / downloads / podcast 的账本调用与查询直接 await。
4 处 start_task 调用点(download.py)确认已 await。
设计取舍(为什么这么做,而不是直接包一层):
- 锁按**事件循环**隔离(_LOOP_LOCKS 以 id(loop) 为键):asyncio.Lock 绑定创建它的
loop,应用主 loop 与测试各自的 loop 并存,跨 loop 复用会抛 "bound to a different
event loop"。
- WAL 切换用**独立锁** _wal_lock 而非 _write_lock:execute() 已持写锁,再在锁内
connect→_configure 会自锁死(asyncio.Lock 非重入)。切换 + 标记在专用锁内串行并
二次复查,否则并发首连会同时执行 PRAGMA journal_mode=WAL 争排他锁报
database is locked(core/db.py 同款教训,实测复现)。
- force_wal 保留:journal_mode 持久化在库文件头,库文件被删重建时新文件不继承,
必须重设。exists() 判定须在 aiosqlite.connect **之前**(connect 会创建空文件)。
- connect() 用 DB_PATH.parent 兜底建目录,优于 DATA_DIR:生产两者相同,但测试
monkeypatch 只换 DB_PATH 时兜底仍能命中正确目录。
留在 to_thread 上的仅剩**正当的阻塞卸载**,不是同步门面:PBKDF2 的 10 万次迭代
(登录链路每次都要跑)、smtplib 发信、备份文件的 copy2 / os.replace、secret.key
的文件 IO。原有 aria2 / downloader / zip 侧的 to_thread 未动。
不变量逐条保持(迁移只改执行模型,不改语义):
- ledger_settle 幂等:账本非 open 直接 return 0,重启后重放不重复退款;
- 终态先到先得:_mark_task_terminal 对已 done/error/canceled 的任务直接 return,
堵住 canceled→error 的非法迁移边;
- settle 必须在任务锁**外**调用(ledger_settle 要抢 SQLite 写锁,busy 最长 15s,
持任务锁会让全站进度/提交/取消随库停摆);
- reserve_and_open 的预扣与开账同事务,消除原两步之间的崩溃丢账窗口。
路径真源红线:DATA_DIR / DB_PATH / BACKUP_DIR / SECRET_KEY_PATH 仍是 auth_system.db
的**模块全局**(用与 core.db 相同的表达式,但**不是**它的别名)。测试只 patch
auth_system.db.*,若改成 core.db 的别名会静默把测试写到生产库。已核 data/site.db
读写前后 sha256 完全一致。
顺带清掉 get_conn:迁移后它只剩「转调 connect()」一层空壳、全仓零消费者(测试
直接开裸连接、备份用 db.connect()),保留等于留一条「拿裸连接自己写」的口子。
测试:19 个文件同步迁移——10 个文件的 web.deps.secret_key 替身改 async;
test_db_conn.py 的多线程首连用例按异步语义拆成 2 条(单 loop gather(16) 覆盖批量
并发首连、WAL 就绪后 8 线程各自 loop 建连覆盖跨 loop);test_billing_scope.py 去掉
多余的 @pytest.mark.anyio;被 await 且**正常返回**的 monkeypatch 替身一律改 async
(仅 raise 的同步替身仍可用)。
门禁:ruff 全过 / pyright 0-0-0(standard 模式含 tests 与顶层模块)/ pytest
384 项全绿(0 失败 0 error 0 skipped;383→384 因 test_db_conn 拆用例)。
data/site.db 读写前后 sha256 一致(a19f5db…6487),无 -wal/-shm 边车残留。
下一步 B3:删 core/sync_http.py、core/async_bridge.py、core/http_errors.py 及
tests/test_async_bridge.py——三者在生产侧已零消费者(http_errors 仅 sync_http
自用,requests 已非声明依赖)。
418 lines
17 KiB
Python
418 lines
17 KiB
Python
# -*- coding: utf-8 -*-
|
||
"""音乐端点。
|
||
|
||
端点:
|
||
- 免费解析类(需登录,不扣次数):
|
||
POST/GET /api/v1/toplist ↔ Flask /toplist
|
||
POST/GET /api/v1/artist_songs ↔ Flask /artist_songs
|
||
POST/GET /api/v1/comments ↔ Flask /comments
|
||
POST/GET /api/v1/search ↔ Flask /search
|
||
POST/GET /api/v1/album ↔ Flask /album
|
||
POST/GET /api/v1/playlist ↔ Flask /playlist
|
||
POST/GET /api/v1/song ↔ Flask /song(四种 type:url/name/lyric/json)
|
||
|
||
参数获取口径:GET query / POST json / POST form 三路兼容。
|
||
上游调用走 music_api 的**异步**方法(原生并发,无同步桥)。
|
||
"""
|
||
import asyncio
|
||
import hmac
|
||
import logging
|
||
import time
|
||
from dataclasses import dataclass
|
||
from typing import Annotated, Any
|
||
|
||
from fastapi import APIRouter, Depends, Request
|
||
from fastapi.responses import JSONResponse
|
||
|
||
import music_api
|
||
from core import media, repo
|
||
from core.config import VALID_QUALITY_LEVELS
|
||
from core.state import GLOBAL_MUSIC_U_CACHE, SERVER_NICK_CACHE
|
||
from music_api import APIException
|
||
from web import ratelimit
|
||
from web.deps import SiteUser, require_login
|
||
from web.request_utils import cookies as _cookies
|
||
from web.request_utils import extract_id as _extract_id
|
||
from web.request_utils import request_data as _request_data
|
||
from web.request_utils import safe_int as _safe_int
|
||
from web.response import error, success
|
||
from web.serializers import record_req_log as _record_log
|
||
|
||
router = APIRouter()
|
||
|
||
LoginUser = Annotated[SiteUser, Depends(require_login)]
|
||
|
||
logger = logging.getLogger('web.music')
|
||
|
||
_VALID_SONG_TYPES = ['url', 'name', 'lyric', 'json']
|
||
|
||
_SERVER_NICK_LOCK: asyncio.Lock | None = None
|
||
|
||
|
||
def _nick_lock() -> asyncio.Lock:
|
||
global _SERVER_NICK_LOCK
|
||
if _SERVER_NICK_LOCK is None:
|
||
_SERVER_NICK_LOCK = asyncio.Lock()
|
||
return _SERVER_NICK_LOCK
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
async def mask_server_nickname(nick: str, user: SiteUser) -> str:
|
||
"""非管理员响应中把服务器账号昵称替换为站点名。
|
||
|
||
- 管理员本人可见真实昵称;
|
||
- 缓存 10 分钟内复用;上游拿不到时回落站点库里的最后已知昵称继续遮蔽。
|
||
"""
|
||
if not nick or user.is_admin:
|
||
return nick
|
||
import time
|
||
|
||
def _replace(n):
|
||
return nick.replace(n, '云村小助手') if (n and n in nick) else nick
|
||
|
||
if SERVER_NICK_CACHE['nick'] is None or time.time() - SERVER_NICK_CACHE['ts'] > 600:
|
||
async with _nick_lock():
|
||
if SERVER_NICK_CACHE['nick'] is None or time.time() - SERVER_NICK_CACHE['ts'] > 600:
|
||
fetched = ''
|
||
try:
|
||
fetched = (await music_api.NeteaseAPI().get_account_info(_cookies()) or {}).get('nickname', '')
|
||
except Exception:
|
||
fetched = ''
|
||
if fetched:
|
||
SERVER_NICK_CACHE['nick'] = fetched
|
||
SERVER_NICK_CACHE['ts'] = time.time()
|
||
try:
|
||
await repo.set_setting('server_nickname', fetched)
|
||
except Exception:
|
||
pass
|
||
else:
|
||
try:
|
||
persisted = await repo.get_setting('server_nickname', '')
|
||
except Exception:
|
||
persisted = ''
|
||
if persisted:
|
||
SERVER_NICK_CACHE['nick'] = persisted
|
||
SERVER_NICK_CACHE['ts'] = time.time()
|
||
else:
|
||
# 库里也从未有过:缩短重试间隔,本次无遮蔽依据只能放行
|
||
SERVER_NICK_CACHE['ts'] = time.time() - 600 + 30
|
||
return _replace(SERVER_NICK_CACHE['nick'])
|
||
|
||
|
||
@router.api_route('/toplist', methods=['GET', 'POST'])
|
||
async def toplist(_user: LoginUser, request: Request):
|
||
"""官方榜单列表。"""
|
||
await ratelimit.guard(request, bucket='free', limit=30, window=60)
|
||
try:
|
||
result = await music_api.NeteaseAPI().get_toplist()
|
||
return success(result, '获取榜单列表成功')
|
||
except APIException:
|
||
return error('服务暂时不可用,请稍后再试', 500)
|
||
|
||
|
||
@router.api_route('/artist_songs', methods=['GET', 'POST'])
|
||
async def artist_songs(_user: LoginUser, request: Request):
|
||
"""歌手信息与热门歌曲。"""
|
||
await ratelimit.guard(request, bucket='free', limit=30, window=60)
|
||
data = await _request_data(request)
|
||
artist_id = data.get('id') or data.get('artist_id')
|
||
if artist_id in (None, ''):
|
||
return error('缺少 artist_id 参数')
|
||
artist_id = _extract_id(artist_id)
|
||
if not str(artist_id).strip().isdigit():
|
||
return error('无效的歌手ID,请输入数字ID或分享链接')
|
||
try:
|
||
result = await music_api.NeteaseAPI().get_artist_hot_songs(int(artist_id))
|
||
return success(result, '获取歌手歌曲成功')
|
||
except APIException:
|
||
return error('服务暂时不可用,请稍后再试', 500)
|
||
|
||
|
||
@router.api_route('/comments', methods=['GET', 'POST'])
|
||
async def comments(_user: LoginUser, request: Request):
|
||
"""歌曲评论(分页)。"""
|
||
await ratelimit.guard(request, bucket='free', limit=30, window=60)
|
||
data = await _request_data(request)
|
||
song_id = data.get('id') or data.get('song_id')
|
||
limit = max(1, min(_safe_int(data.get('limit'), 20), 100))
|
||
offset = max(0, _safe_int(data.get('offset'), 0))
|
||
if song_id in (None, ''):
|
||
return error('缺少 song_id 参数')
|
||
song_id = _extract_id(song_id)
|
||
if not str(song_id).strip().isdigit():
|
||
return error('无效的歌曲ID,请输入数字ID或分享链接')
|
||
try:
|
||
result = await music_api.NeteaseAPI().get_song_comments(int(song_id), limit, offset)
|
||
return success(result, '获取评论成功')
|
||
except APIException:
|
||
return error('服务暂时不可用,请稍后再试', 500)
|
||
|
||
|
||
@router.api_route('/search', methods=['GET', 'POST'])
|
||
async def search(user: LoginUser, request: Request):
|
||
"""搜索(歌曲/专辑/歌手/歌单),offset 透传上游。"""
|
||
await ratelimit.guard(request, bucket='free', limit=30, window=60)
|
||
data = await _request_data(request)
|
||
keyword = data.get('keyword') or data.get('keywords') or data.get('q')
|
||
limit = max(1, _safe_int(data.get('limit'), 30))
|
||
offset = max(0, _safe_int(data.get('offset'), 0))
|
||
search_type = data.get('type', '1')
|
||
if not keyword:
|
||
return error('缺少 keyword 参数')
|
||
if limit > 100:
|
||
limit = 100
|
||
try:
|
||
stype = int(search_type)
|
||
except (TypeError, ValueError):
|
||
return error('参数格式错误:type 必须为数字')
|
||
try:
|
||
result = await music_api.NeteaseAPI().search_music(
|
||
str(keyword), _cookies(), limit, stype, offset=offset)
|
||
except APIException:
|
||
return error('服务暂时不可用,请稍后再试', 500)
|
||
|
||
# 向后兼容字段 + 服务器昵称遮蔽
|
||
if result:
|
||
for song in result:
|
||
if song.get('type', 'song') == 'song' and 'artists' in song:
|
||
song['artist_string'] = song['artists']
|
||
if song.get('type') == 'playlist':
|
||
if song.get('creator'):
|
||
song['creator'] = await mask_server_nickname(song['creator'], user)
|
||
if song.get('name'):
|
||
song['name'] = await mask_server_nickname(song['name'], user)
|
||
return success(result, '搜索完成')
|
||
|
||
|
||
# ============ 个人/全局账号 Cookie ============
|
||
|
||
def _global_music_u() -> str:
|
||
"""全局账号 MUSIC_U(5 分钟缓存)。"""
|
||
now = time.time()
|
||
if GLOBAL_MUSIC_U_CACHE['v'] is None or now - GLOBAL_MUSIC_U_CACHE['ts'] > 300:
|
||
try:
|
||
ck = _cookies()
|
||
GLOBAL_MUSIC_U_CACHE['v'] = (ck.get('MUSIC_U') or '').strip()
|
||
except Exception:
|
||
GLOBAL_MUSIC_U_CACHE['v'] = ''
|
||
GLOBAL_MUSIC_U_CACHE['ts'] = now
|
||
return GLOBAL_MUSIC_U_CACHE['v']
|
||
|
||
|
||
async def _resolve_user_cookies(
|
||
data: dict[str, Any], user: 'SiteUser | None', request: Request
|
||
) -> dict[str, str]:
|
||
"""解析个人账号 Cookie(浏览器传入的 user_cookie,不落服务器)。
|
||
|
||
全局账号(cookie.txt)零个人身份权限:浏览器提交的 MUSIC_U 若与全局账号一致
|
||
(全局 Cookie 泄露被当个人 Cookie 冒用),一律拒绝并记审计。
|
||
"""
|
||
ck = media.parse_cookie_str(data.get('user_cookie'))
|
||
mu = (ck.get('MUSIC_U') or '').strip()
|
||
gmu = _global_music_u()
|
||
if mu and gmu and hmac.compare_digest(mu, gmu):
|
||
try:
|
||
import auth_system
|
||
await auth_system.add_log('fail',
|
||
user.username if user else '匿名',
|
||
ratelimit.client_ip(request), '全局账号冒用拦截',
|
||
'浏览器提交的 MUSIC_U 与全局账号一致,已按未登录拒绝')
|
||
except Exception:
|
||
pass
|
||
logger.warning('[全局账号冒用拦截] %s 提交的 MUSIC_U 与全局账号一致,已按未登录处理',
|
||
request.url.path)
|
||
return {}
|
||
return ck
|
||
|
||
|
||
|
||
|
||
# ============ 迁移端:专辑 / 歌单 / 歌曲 ============
|
||
|
||
@router.api_route('/album', methods=['GET', 'POST'])
|
||
async def album(_user: LoginUser, request: Request):
|
||
"""专辑详情。"""
|
||
await ratelimit.guard(request, bucket='free', limit=30, window=60)
|
||
data = await _request_data(request)
|
||
album_id = data.get('id')
|
||
if not album_id:
|
||
return error("参数 'album_id' 不能为空", 400)
|
||
album_id = _extract_id(album_id)
|
||
if not str(album_id).strip().isdigit():
|
||
return error('无效的专辑ID,请输入数字ID或分享链接')
|
||
try:
|
||
result = await music_api.NeteaseAPI().get_album_detail(int(album_id), _cookies())
|
||
except APIException:
|
||
return error('服务暂时不可用,请稍后再试', 500)
|
||
return success({'status': 200, 'album': result}, '获取专辑详情成功')
|
||
|
||
|
||
@router.api_route('/playlist', methods=['GET', 'POST'])
|
||
async def playlist(user: LoginUser, request: Request):
|
||
"""歌单详情(非管理员遮蔽服务器账号昵称)。"""
|
||
await ratelimit.guard(request, bucket='free', limit=30, window=60)
|
||
data = await _request_data(request)
|
||
playlist_id = data.get('id')
|
||
if not playlist_id:
|
||
return error("参数 'playlist_id' 不能为空", 400)
|
||
playlist_id = _extract_id(playlist_id)
|
||
if not str(playlist_id).strip().isdigit():
|
||
return error('无效的歌单ID,请输入数字ID或分享链接')
|
||
try:
|
||
result = await music_api.NeteaseAPI().get_playlist_detail(int(playlist_id), _cookies())
|
||
except APIException:
|
||
return error('服务暂时不可用,请稍后再试', 500)
|
||
if isinstance(result, dict):
|
||
if result.get('creator'):
|
||
result['creator'] = await mask_server_nickname(result['creator'], user)
|
||
if result.get('name'):
|
||
result['name'] = await mask_server_nickname(result['name'], user)
|
||
return success({'status': 'success', 'playlist': result}, '获取歌单详情成功')
|
||
|
||
|
||
@dataclass(frozen=True)
|
||
class _SongQuery:
|
||
"""`/song` 的已校验入参。
|
||
|
||
``music_id`` 保持 ``extract_id`` 的原始返回(字符串):它会**原样**进入响应体
|
||
的 ``id`` 字段,golden 快照按字符串比对。转 int 只在上游调用时做(见 ``mid``)。
|
||
"""
|
||
music_id: str
|
||
level: str
|
||
info_type: str
|
||
|
||
@property
|
||
def mid(self) -> int:
|
||
"""上游接口要的整型 ID(响应体仍用原字符串)。"""
|
||
return int(self.music_id)
|
||
|
||
|
||
def _parse_song_query(data: dict[str, Any]) -> _SongQuery | str:
|
||
"""提取并校验 /song 入参:成功给 _SongQuery,失败给**错误消息**。
|
||
|
||
返回消息字符串而非响应对象,是为了让本函数不依赖响应层、可单独断言。
|
||
校验顺序即用户可见的错误优先级(缺参 → ID 非数字 → 音质 → 类型),
|
||
与历史实现逐条一致——改顺序会改变「同时传错时先报哪个」。
|
||
"""
|
||
song_ids = data.get('ids') or data.get('id')
|
||
url = data.get('url')
|
||
if not song_ids and not url:
|
||
return "必须提供 'ids'、'id' 或 'url' 参数"
|
||
music_id = _extract_id(song_ids or url)
|
||
if not music_id.strip().isdigit():
|
||
return '无效的歌曲ID,请输入数字ID或分享链接'
|
||
level = data.get('level', 'lossless')
|
||
if level not in VALID_QUALITY_LEVELS:
|
||
return f"无效的音质参数,支持: {', '.join(VALID_QUALITY_LEVELS)}"
|
||
info_type = data.get('type', 'url')
|
||
if info_type not in _VALID_SONG_TYPES:
|
||
return f"无效的类型参数,支持: {', '.join(_VALID_SONG_TYPES)}"
|
||
return _SongQuery(music_id, level, info_type)
|
||
|
||
|
||
async def _song_url_response(api: Any, query: _SongQuery, cookies: dict[str, str]) -> JSONResponse:
|
||
"""type=url:取直链,附带可读的大小与音质名。"""
|
||
result = await api.get_song_url(query.mid, query.level, cookies)
|
||
if result and result.get('data') and len(result['data']) > 0:
|
||
song_data = result['data'][0] or {}
|
||
if not song_data:
|
||
return error('获取音乐URL失败,可能是版权限制或音质不支持', 404)
|
||
return success({
|
||
'id': song_data.get('id'),
|
||
'url': song_data.get('url'),
|
||
'level': song_data.get('level'),
|
||
'quality_name': media.quality_display_name(song_data.get('level', query.level)),
|
||
'size': song_data.get('size'),
|
||
'size_formatted': media.format_file_size(song_data.get('size', 0)),
|
||
'type': song_data.get('type'),
|
||
'bitrate': song_data.get('br'),
|
||
}, '获取歌曲URL成功')
|
||
return error('获取音乐URL失败,可能是版权限制或音质不支持', 404)
|
||
|
||
|
||
async def _song_json_response(api: Any, query: _SongQuery, cookies: dict[str, str],
|
||
request: Request) -> JSONResponse:
|
||
"""type=json:并行取详情/直链/歌词,歌词失败不阻断整次解析。
|
||
|
||
解析日志只在**本分支**记录——历史实现如此,url/name/lyric 三路都不记。不要
|
||
为了「统一」把它上提到主函数,那会新增其余三路的可观察副作用。
|
||
"""
|
||
song_info, url_info, lyric_info = await asyncio.gather(
|
||
api.get_song_detail(query.mid),
|
||
api.get_song_url(query.mid, query.level, cookies),
|
||
api.get_lyric(query.mid, cookies),
|
||
return_exceptions=True,
|
||
)
|
||
if isinstance(lyric_info, BaseException):
|
||
lyric_info = None
|
||
if isinstance(url_info, BaseException):
|
||
url_info = None
|
||
if isinstance(song_info, BaseException) or not song_info or not song_info.get('songs'):
|
||
return error('未找到歌曲信息', 404)
|
||
|
||
song_data = song_info['songs'][0]
|
||
_al = song_data.get('al') or {}
|
||
_ar_names = ', '.join(filter(None, ((a or {}).get('name') for a in (song_data.get('ar') or []))))
|
||
response_data: dict[str, Any] = {
|
||
'id': query.music_id,
|
||
'name': song_data.get('name') or '',
|
||
'ar_name': _ar_names,
|
||
'al_name': _al.get('name') or '',
|
||
'pic': _al.get('picUrl') or '',
|
||
'level': query.level,
|
||
'lyric': ((lyric_info or {}).get('lrc') or {}).get('lyric') or '',
|
||
'tlyric': ((lyric_info or {}).get('tlyric') or {}).get('lyric') or '',
|
||
'yrc': ((lyric_info or {}).get('yrc') or {}).get('lyric') or '',
|
||
}
|
||
if url_info and url_info.get('data') and len(url_info['data']) > 0:
|
||
url_data = url_info['data'][0]
|
||
response_data.update({
|
||
'url': url_data.get('url', ''),
|
||
'size': media.format_file_size(url_data.get('size', 0)),
|
||
'level': url_data.get('level', query.level),
|
||
})
|
||
else:
|
||
response_data.update({'url': '', 'size': '获取失败'})
|
||
|
||
try:
|
||
_record_log('解析', ratelimit.client_ip(request), response_data.get('name') or query.music_id)
|
||
except Exception:
|
||
pass
|
||
return success(response_data, '获取歌曲信息成功')
|
||
|
||
|
||
@router.api_route('/song', methods=['GET', 'POST'])
|
||
async def song(user: LoginUser, request: Request):
|
||
"""歌曲信息(type: url/name/lyric/json)。"""
|
||
await ratelimit.guard(request, bucket='free', limit=30, window=60)
|
||
data = await _request_data(request)
|
||
|
||
query = _parse_song_query(data)
|
||
if isinstance(query, str):
|
||
return error(query)
|
||
|
||
if data.get('account', 'admin') == 'user':
|
||
cookies = await _resolve_user_cookies(data, user, request)
|
||
if not cookies:
|
||
return error('请先扫码登录个人账号后再使用个人解析', 401)
|
||
else:
|
||
cookies = _cookies()
|
||
|
||
api = music_api.NeteaseAPI()
|
||
try:
|
||
if query.info_type == 'url':
|
||
return await _song_url_response(api, query, cookies)
|
||
if query.info_type == 'name':
|
||
return success(await api.get_song_detail(query.mid), '获取歌曲信息成功')
|
||
if query.info_type == 'lyric':
|
||
return success(await api.get_lyric(query.mid, cookies), '获取歌词成功')
|
||
return await _song_json_response(api, query, cookies, request)
|
||
except APIException:
|
||
return error('服务暂时不可用,请稍后再试', 500)
|