Merge pull request #114 from Lost-MSth/dev

Dev

README.md

@@ -17,42 +17,53 @@ This procedure is mainly used for study and research, and shall not be used for
 
 ## 特性 Features
 
-有以下 We have：
+:x: : 不支持 Not supported
+:warning: : 可能存在问题 / 可能与官方不一样 Possible issues / may differ from official
+:wastebasket: : 不再更新，可能会移除或重构 No longer updated, may be removed or refactored
+:construction: : 建设中 In construction
 
 - 登录、注册 Login and registration
-- 多设备登录 Multi device login
+  - 多设备自动封号 Auto-ban of multiple devices
+  - :warning: 多设备登录 Multi device login
+  - 登录频次限制 Login rate limit
+  - :x: 销号 Destroy account
 - 成绩上传 Score upload
-- PTT
-- 世界排名 Global rank
-- 排名 Rank
+  - 成绩校验 Score check
+  - 成绩排名 Score rank
+- 潜力值机制 Potential
+  - Best 30
+  - :warning: Recent Top 10
+- :warning: 世界排名 Global rank
 - 段位系统 Course system
-- Link Play
+- :warning: Link Play
 - 好友系统 Friends
+  - :x: 好友位提升 Max friend number increase
 - 云端存档 Cloud save
-- 爬梯 Climbing steps
-- 自定义世界模式 Customizable World Mode
-- 自定义歌曲下载 Customizable songs download
-- 单曲和曲包购买（没啥用） Single songs and song packs purchase(useless)
+  - 尝试全剧情、曲目解锁 Try to unlock all the stories and songs
+- 世界模式 World mode
+  - 体力系统 Stamina system
+  - :warning: 普通梯子强化和绳子强化 Normal steps boost & beyond boost
+  - :warning: 角色技能 Character skills
+- 歌曲下载 Songs downloading
+  - :x: 加密下载 Encrypted downloading
+  - 下载校验 Download check
+  - 下载频次限制 Download rate limit
+- 购买系统 Purchase system
+  - 单曲和曲包 Single & Pack
+  - :x: 捆绑包 Bundle
+  - 折扣 Discount
+  - 五周年兑换券 5-th anniversary ticket
+  - :x: Extend 包自动降价 Extend pack automatic price reduction
 - 奖励系统 Present system
 - 兑换码系统 Redeem code system
 - 角色系统 Character system
-- 全剧情解锁 Unlock all the stories
-- 后台查询 Background search
-- 后台自定义信息 Customize some things in the background
-- 成绩校验 Score check
-- 下载校验 Download check
+- 数据记录 Data recording
+  - 用户成绩 Users' scores
+  - 用户每日潜力值 Users' daily potential
+- :wastebasket: 简单的网页管理后台 Simple web admin backend
+- :construction: API
 - 服务器日志 Server log
 
-没有以下 We don't have：
-
-- 服务器安全性保证 Server security assurance
-
-可能有问题 There may be problems：
-
-- Recent 30
-- 一些歌曲的解锁 Some songs' unlocking
-- 同设备多共存登录 Multiple app logins on the same device
-
 ## 说明 Statement
 
 只是很有趣，用处探索中。  
@@ -69,22 +80,28 @@ It is just so interesting. What it can do is under exploration.
 
 只保留最新版本 Only keep the latest version.
 
-> 提醒：更新时请注意保留原先的数据库，以防数据丢失。
->
+> 提醒：更新时请注意保留原先的数据库，以防数据丢失。  
 > Tips: When updating, please keep the original database in case of data loss.
+>
+> 其它小改动请参考各个 commit 信息  
+> Please refer to the commit messages for other minor changes.
 
-### Version 2.11.1
+### Version 2.11.2
 
-- 适用于Arcaea 4.4.0版本 For Arcaea 4.4.0
-- 新搭档 **密特拉·泰尔塞拉**、**不来方斗亚** 已解锁 Unlock the character **Mithra Tercera** and **Toa Kozukata**.
-- 为 **密特拉·泰尔塞拉** 的技能提供支持 Add support for the skill of **Mithra Tercera**.
-- 新增修改搭档的API接口 Add some API endpoints about characters.
+- 适用于 Arcaea 4.4.6 版本 For Arcaea 4.4.6
+- 新搭档 **奈美（暮光）** 已解锁 Unlock the character **Nami (Twilight)**.
+- 新增用户潜力值每日记录功能 Add support for recording users' potential each day.
+- 修复搭档 **光 & 对立（Reunion）** 无法觉醒的问题 Fix a bug that the character **Hikari & Tairitsu (Reunion)** cannot be uncapped. (#100)
+- 添加 `finale/finale_end` 接口尝试修复最终挑战无法解锁结局的问题 Add the `finale/finale_end` endpoint to try to fix the problem that the endings cannot be unlocked correctly in the finale challenge. (#110)
+- 新增获取用户潜力值记录的 API 接口 Add an API endpoint for getting the user's rating records.
 
 ## 运行环境与依赖 Running environment and requirements
 
-- Windows/Linux/Mac OS/Android...
+- Windows / Linux / Mac OS / Android...
 - Python >= 3.6
-- Flask module >= 2.0, Cryptography module >= 3.0.0, limits >= 2.7.0
+  - Flask >= 2.0
+  - Cryptography >= 3.0.0
+  - limits >= 2.7.0
 - Charles, IDA, proxy app... (optional)
 
 <!--
@@ -105,7 +122,7 @@ It is just so interesting. What it can do is under exploration.
 
 ## Q&A
 
-[中文/English](https://github.com/Lost-MSth/Arcaea-server/wiki/Q&A)
+[中文 / English](https://github.com/Lost-MSth/Arcaea-server/wiki/Q&A)
 
 ## 鸣谢 Thanks
 

latest version/api/token.py

@@ -1,6 +1,6 @@
 from base64 import b64decode
 
-from flask import Blueprint, request
+from flask import Blueprint, current_app, request
 
 from core.api_user import APIUser
 from core.error import PostError
@@ -32,6 +32,7 @@ def token_post(data):
     with Connect() as c:
         user = APIUser(c)
         user.login(name, password, request.remote_addr)
+        current_app.logger.info(f'API user `{user.user_id}` log in')
         return success_return({'token': user.api_token, 'user_id': user.user_id})
 
 

latest version/api/users.py

@@ -1,10 +1,12 @@
 from flask import Blueprint, request
 
 from core.api_user import APIUser
+from core.config_manager import Config
 from core.error import InputError, NoAccess, NoData
 from core.score import Potential, UserScoreList
 from core.sql import Connect, Query, Sql
 from core.user import UserChanger, UserInfo, UserRegister
+from core.util import get_today_timestamp
 
 from .api_auth import api_try, request_json_handle, role_required
 from .api_code import error_return, success_return
@@ -191,3 +193,32 @@ def users_user_role_get(user, user_id):
         x = APIUser(c, user_id)
         x.select_role_and_powers()
         return success_return({'user_id': x.user_id, 'role': x.role.role_id, 'powers': [i.power_id for i in x.role.powers]})
+
+
+@bp.route('/<int:user_id>/rating', methods=['GET'])
+@role_required(request, ['select', 'select_me'])
+@request_json_handle(request, optional_keys=['start_timestamp', 'end_timestamp', 'duration'])
+@api_try
+def users_user_rating_get(data, user, user_id):
+    '''查询用户历史rating，`duration`是相对于今天的天数'''
+    # 查别人需要select权限
+    if user_id != user.user_id and not user.role.has_power('select'):
+        return error_return(NoAccess('No permission', api_error_code=-1), 403)
+
+    start_timestamp = data.get('start_timestamp', None)
+    end_timestamp = data.get('end_timestamp', None)
+    duration = data.get('duration', None)
+    sql = '''select time, rating_ptt from user_rating where user_id = ?'''
+    sql_data = [user_id]
+    if start_timestamp is not None and end_timestamp is not None:
+        sql += ''' and time between ? and ?'''
+        sql_data += [start_timestamp, end_timestamp]
+    elif duration is not None:
+        sql += ''' and time between ? and ?'''
+        t = get_today_timestamp()
+        sql_data += [t - duration * 24 * 3600, t]
+
+    with Connect(Config.SQLITE_LOG_DATABASE_PATH) as c:
+        c.execute(sql, sql_data)
+        r = c.fetchall()
+        return success_return({'user_id': user_id, 'data': [{'time': i[0], 'rating_ptt': i[1]} for i in r]})

latest version/core/api_user.py

@@ -127,7 +127,8 @@ def login(self, name: str = None, password: str = None, ip: str = None) -> None:
         if ip is not None:
             self.ip = ip
         if not self.limiter.hit(name):
-            raise RateLimit('Too many login attempts', api_error_code=-205)
+            raise RateLimit(
+                f'Too many login attempts of username {name}', api_error_code=-205)
 
         self.c.execute('''select user_id, password from user where name = :a''', {
                        'a': self.name})
@@ -136,9 +137,9 @@ def login(self, name: str = None, password: str = None, ip: str = None) -> None:
             raise NoData(
                 f'The user `{self.name}` does not exist.', api_error_code=-201, status=401)
         if x[1] == '':
-            raise UserBan(f'The user `{self.name}` is banned.')
+            raise UserBan(f'The user `{x[0]}` is banned.')
         if self.hash_pwd != x[1]:
-            raise NoAccess('The password is incorrect.',
+            raise NoAccess(f'The password of user `{x[0]}` is incorrect.',
                            api_error_code=-201, status=401)
 
         self.user_id = x[0]

latest version/core/bgtask.py

@@ -0,0 +1,50 @@
+from atexit import register
+from concurrent.futures import ThreadPoolExecutor
+
+from .constant import Constant
+from .sql import Connect
+
+
+class BGTask:
+    executor = ThreadPoolExecutor(max_workers=1)
+
+    def __init__(self, func, *args, **kwargs):
+        self.future = self.executor.submit(func, *args, **kwargs)
+
+    def result(self):
+        return self.future.result()
+
+    def cancel(self) -> bool:
+        return self.future.cancel()
+
+    def done(self) -> bool:
+        return self.future.done()
+
+    @staticmethod
+    def shutdown(wait: bool = True):
+        BGTask.executor.shutdown(wait)
+
+
+@register
+def atexit():
+    BGTask.shutdown()
+
+
+def logdb_execute_func(sql, *args, **kwargs):
+    with Connect(Constant.SQLITE_LOG_DATABASE_PATH) as c:
+        c.execute(sql, *args, **kwargs)
+
+
+def logdb_execute_many_func(sql, *args, **kwargs):
+    with Connect(Constant.SQLITE_LOG_DATABASE_PATH) as c:
+        c.executemany(sql, *args, **kwargs)
+
+
+def logdb_execute(sql: str, *args, **kwargs):
+    '''异步执行SQL，日志库写入，注意不会直接返回结果'''
+    return BGTask(logdb_execute_func, sql, *args, **kwargs)
+
+
+def logdb_execute_many(sql: str, *args, **kwargs):
+    '''异步批量执行SQL，日志库写入，注意不会直接返回结果'''
+    return BGTask(logdb_execute_many_func, sql, *args, **kwargs)

latest version/core/character.py

@@ -349,7 +349,7 @@ def character_uncap(self, user=None):
             self.c.execute(
                 '''select amount from user_item where user_id=? and item_id=? and type="core"''', (self.user.user_id, i.item_id))
             y = self.c.fetchone()
-            if not y or i.amount > y[0]:
+            if i.amount > 0 and (not y or i.amount > y[0]):
                 raise ItemNotEnough('The cores are not enough.')
 
         for i in self.uncap_cores:

latest version/core/constant.py

@@ -1,6 +1,7 @@
 from .config_manager import Config
 
-ARCAEA_SERVER_VERSION = 'v2.11.1'
+ARCAEA_SERVER_VERSION = 'v2.11.2'
+ARCAEA_LOG_DATBASE_VERSION = 'v1.1'
 
 
 class Constant:
@@ -101,4 +102,6 @@ class Constant:
     DATABASE_MIGRATE_TABLES = ['user', 'friend', 'best_score', 'recent30', 'user_world', 'item', 'user_item', 'purchase', 'purchase_item', 'user_save',
                                'login', 'present', 'user_present', 'present_item', 'redeem', 'user_redeem', 'redeem_item', 'api_login', 'chart', 'user_course', 'user_char', 'user_role']
 
+    LOG_DATABASE_MIGRATE_TABLES = ['cache', 'user_score', 'user_rating']
+
     UPDATE_WITH_NEW_CHARACTER_DATA = Config.UPDATE_WITH_NEW_CHARACTER_DATA

latest version/core/init.py

@@ -7,11 +7,12 @@
 from traceback import format_exc
 
 from core.config_manager import Config
-from core.constant import ARCAEA_SERVER_VERSION
+from core.constant import ARCAEA_LOG_DATBASE_VERSION, ARCAEA_SERVER_VERSION
 from core.course import Course
 from core.download import DownloadList
 from core.purchase import Purchase
-from core.sql import Connect, DatabaseMigrator, MemoryDatabase
+from core.sql import (Connect, DatabaseMigrator, LogDatabaseMigrator,
+                      MemoryDatabase)
 from core.user import UserRegister
 from core.util import try_rename
 
@@ -208,6 +209,29 @@ def check_update_database(self) -> bool:
                 self.logger.error(
                     f'Failed to new the file {Config.SQLITE_LOG_DATABASE_PATH}')
                 return False
+        else:
+            # 检查更新
+            with Connect(Config.SQLITE_LOG_DATABASE_PATH) as c:
+                try:
+                    x = c.execute(
+                        '''select value from cache where key="version"''').fetchone()
+                except:
+                    x = None
+            if not x or x[0] != ARCAEA_LOG_DATBASE_VERSION:
+                self.logger.warning(
+                    f'Maybe the file `{Config.SQLITE_LOG_DATABASE_PATH}` is an old version.')
+                try:
+                    self.logger.info(
+                        f'Try to update the file `{Config.SQLITE_LOG_DATABASE_PATH}`')
+                    self.update_log_database()
+                    self.logger.info(
+                        f'Success to update the file `{Config.SQLITE_LOG_DATABASE_PATH}`')
+                except Exception as e:
+                    self.logger.error(format_exc())
+                    self.logger.error(
+                        f'Failed to update the file `{Config.SQLITE_LOG_DATABASE_PATH}`')
+                    return False
+
         if not self.check_file(Config.SQLITE_DATABASE_PATH):
             # 新建数据库
             try:
@@ -275,6 +299,12 @@ def update_database(old_path: str, new_path: str = Config.SQLITE_DATABASE_PATH)
             DatabaseMigrator(old_path, new_path).update_database()
             os.remove(old_path)
 
+    @staticmethod
+    def update_log_database(old_path: str = Config.SQLITE_LOG_DATABASE_PATH) -> None:
+        '''直接更新日志数据库'''
+        if os.path.isfile(old_path):
+            LogDatabaseMigrator(old_path).update_database()
+
     def check_song_file(self) -> bool:
         '''检查song有关文件并初始化缓存'''
         f = self.check_folder(Config.SONG_FILE_FOLDER_PATH)

latest version/core/score.py

@@ -2,13 +2,15 @@
 from os import urandom
 from time import time
 
+from .bgtask import BGTask, logdb_execute
+from .config_manager import Config
 from .constant import Constant
 from .course import CoursePlay
 from .error import NoData, StaminaNotEnough
 from .item import ItemCore
 from .song import Chart
 from .sql import Connect, Query, Sql
-from .util import md5
+from .util import get_today_timestamp, md5
 from .world import WorldPlay
 
 
@@ -427,9 +429,20 @@ def update_recent30(self) -> None:
 
     def record_score(self) -> None:
         '''向log数据库记录分数，请注意列名不同'''
-        with Connect(Constant.SQLITE_LOG_DATABASE_PATH) as c2:
-            c2.execute('''insert into user_score values(?,?,?,?,?,?,?,?,?,?,?,?,?)''', (self.user.user_id, self.song.song_id, self.song.difficulty, self.time_played,
-                                                                                        self.score, self.shiny_perfect_count, self.perfect_count, self.near_count, self.miss_count, self.health, self.modifier, self.clear_type, self.rating))
+        logdb_execute('''insert into user_score values(?,?,?,?,?,?,?,?,?,?,?,?,?)''', (self.user.user_id, self.song.song_id, self.song.difficulty, self.time_played,
+                                                                                       self.score, self.shiny_perfect_count, self.perfect_count, self.near_count, self.miss_count, self.health, self.modifier, self.clear_type, self.rating))
+
+    def record_rating_ptt(self, user_rating_ptt: float) -> None:
+        '''向log数据库记录用户ptt变化'''
+        today_timestamp = get_today_timestamp()
+        with Connect(Config.SQLITE_LOG_DATABASE_PATH) as c2:
+            old_ptt = c2.execute('''select rating_ptt from user_rating where user_id=? and time=?''', (
+                self.user.user_id, today_timestamp)).fetchone()
+
+            old_ptt = 0 if old_ptt is None else old_ptt[0]
+            if old_ptt != user_rating_ptt:
+                c2.execute('''insert or replace into user_rating values(?,?,?)''',
+                           (self.user.user_id, today_timestamp, user_rating_ptt))
 
     def upload_score(self) -> None:
         '''上传分数，包括user的recent更新，best更新，recent30更新，世界模式计算'''
@@ -474,7 +487,9 @@ def upload_score(self) -> None:
             self.update_recent30()
 
         # 总PTT更新
-        self.user.rating_ptt = int(self.ptt.value * 100)
+        user_rating_ptt = self.ptt.value
+        self.user.rating_ptt = int(user_rating_ptt * 100)
+        BGTask(self.record_rating_ptt, user_rating_ptt)  # 记录总PTT变换
         self.c.execute('''update user set rating_ptt = :a where user_id = :b''', {
             'a': self.user.rating_ptt, 'b': self.user.user_id})