Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

7.1.1 #696

Merged
merged 8 commits into from
Nov 6, 2024
Merged

7.1.1 #696

merged 8 commits into from
Nov 6, 2024

Conversation

Ghost-chu
Copy link
Collaborator

@Ghost-chu Ghost-chu commented Nov 6, 2024

错误修复

  • 修复流量统计图表在 7.1.0 被破坏的问题 @Ghost-chu @paulzzh
  • 修复 httpd 可能在 IPDB 未下载完成之前就启动的问题 @Ghost-chu
  • 修复 IPDB 下载超时时有时不会切换备用源的问题 @Ghost-chu
  • 修复在非 zh-CN 的系统语言下,界面文本显示异常 @Ghost-chu

Docker

DockerHub: ghostchu/peerbanhelper:v7.1.1
阿里云国内镜像加速: registry.cn-hangzhou.aliyuncs.com/ghostchu/peerbanhelper:v7.1.1

注:有许多小伙伴询问是否可以使用 latest 标签,是可以使用的,只是如果你使用了镜像站,则 latest 标签可能不是最新的版本。


7.1.0 发行注记

Caution

本版包含重要安全修复,不管出于任何理由,您都应该更新到此修复版本。

关键安全性修复

  • 【重要】修复了因错误使用 ORM 框架导致潜在 SQL 注入的问题 @Ghost-chu @paulzzh
    • 恶意攻击者可通过在查询参数中插入 SQL 片段,执行任意 SQL 查询
  • 【重要】修复了登录接口的 POST 登陆方式没有覆盖暴力破解防护的问题 @Ghost-chu
    • 恶意攻击者可能对登录接口发起暴力破解穷举 WebUI Token 以获取 WebUI 访问权限,并间接获取连接的下载器的 WebUI 权限
  • 添加了 robots.txt 并拒绝任何搜索引擎索引并避免在搜索引擎中暴露,但依然可能被 Censys 等网络空间测绘引擎发现,建议使用防火墙保护 @Ghost-chu
    • 如果 PBH 部署在二级目录下,请自行管理 robots.txt
  • 仅在登录阶段传递 Token,避免明文 WebUI Token 泄露

新功能

  • 图表数据现在支持分下载器查看 @Gaojianli @Ghost-chu @paulzzh
  • WebUI 现在支持自定义脚本编辑 @Gaojianli @Ghost-chu
    • 通过编程的方式构建自己的反吸血逻辑
    • 只有在局域网内直接访问 WebUI 才能添加和编辑脚本;通过互联网或者反向代理访问时,仅能查看脚本,不可添加修改编辑
    • 安全警告:自定义脚本可执行任意代码,请仅添加来自可信来源的脚本
  • 【重要】BTN 新增 “脚本规则” 规则类型,PeerBanHelper 现在可接收来自 BTN 服务器下发的脚本以提升基于云的检测防护能力,提高封禁的灵活和精确性 @Ghost-chu
    • 需要手动在 “设置->基础设置->BTN” 开启 “启用脚本执行” 开关,此功能才会生效。请仅在可信 BTN 服务器上启用此功能。
  • WebUI 现在可以进行堆内存转储 @Gaojianli
  • BTN 能力列表页面现在可查看云端规则数量和规则版本号 @Gaojianli @Ghost-chu
  • 其它用户体验改善

错误修复

  • 【重要】优化了 IPMatcher 的CPU和内存占用,解决了困扰已久的规则过多时内存溢出的问题并大幅缩短了匹配 IP 时的 CPU 占用和匹配耗时,现在空载内存仅需要 92MB(GUI) @Ghost-chu @paulzzh
    • 请注意:自 2024/11/06 后,旧版本(< 7.1.0) PBH 可能由于 IP 屏蔽列表的增长而耗尽内存,为了保证正常运行,请升级版本或者更改其最大堆内存
  • 修复了当添加支持完整 PeerID 的下载器(如:BiglyBT/Azureus/Vuze、BitComet 或者 Deluge)时,查看 PeerID 饼图时完全不可读的问题 @Ghost-chu
  • Windows GUI 的打开 WebUI 按钮现在能够自动填充 token 登录 WebUI

Docker

DockerHub: ghostchu/peerbanhelper:v7.1.0
阿里云国内镜像加速: registry.cn-hangzhou.aliyuncs.com/ghostchu/peerbanhelper:v7.1.0

本版已知问题

  • 流量图表可能显示为空

Summary by CodeRabbit

Release Notes

  • New Features

    • Version updated to 7.1.1, enhancing overall application stability.
  • Improvements

    • Streamlined SQL query construction across multiple data access classes, improving code readability and maintainability.
    • Enhanced language file loading process to fill missing translations more effectively.
  • Bug Fixes

    • Adjusted handling of timestamps in various database operations, ensuring accurate data processing.

These updates focus on improving performance and maintainability while ensuring existing functionalities remain intact.

@Ghost-chu Ghost-chu requested a review from a team as a code owner November 6, 2024 14:29
Copy link
Contributor

coderabbitai bot commented Nov 6, 2024

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Walkthrough

The pull request includes updates to several Java classes and the pom.xml file in the PeerBanHelper project. The primary change in pom.xml is a version increment from 7.1.0 to 7.1.1. Various Java classes have been modified to simplify SQL query constructions by removing unnecessary SelectArg wrappers around parameters. Additionally, some methods have had their internal logic adjusted for improved clarity, particularly in handling timestamps and query parameters. Overall, the changes focus on enhancing code readability and maintainability without altering core functionalities.

Changes

File Path Change Summary
pom.xml Version updated from 7.1.0 to 7.1.1.
src/main/java/com/ghostchu/peerbanhelper/PeerBanHelperServer.java Reordered registerHttpServer() method call within start() method to occur before resetKnownDownloaders().
src/main/java/com/ghostchu/peerbanhelper/database/dao/impl/AlertDao.java Removed SelectArg wrappers in deleteOldAlerts and markAsRead methods, simplifying query and update operations.
src/main/java/com/ghostchu/peerbanhelper/database/dao/impl/PeerRecordDao.java Removed SelectArg wrappers in getPendingSubmitPeerRecords and createIfNotExists methods, simplifying SQL queries.
src/main/java/com/ghostchu/peerbanhelper/database/dao/impl/ProgressCheatBlockerPersistDao.java Removed SelectArg wrappers in fetchFromDatabase and cleanupDatabase methods, simplifying query conditions.
src/main/java/com/ghostchu/peerbanhelper/database/dao/impl/TrafficJournalDao.java Removed SelectArg wrappers in timestamp handling in various methods, enhancing SQL query construction.
src/main/java/com/ghostchu/peerbanhelper/ipdb/IPDB.java Modified setupHttpClient to include requestTimeout and changed readTimeout to use a virtual thread factory.
src/main/java/com/ghostchu/peerbanhelper/metric/impl/persist/PersistMetrics.java Updated cleanup method to simplify timestamp handling and indicated virtual thread usage in recordPeerBan.
src/main/java/com/ghostchu/peerbanhelper/module/impl/rule/ActiveMonitoringModule.java Simplified query construction in cleanup method by removing SelectArg wrapper.
src/main/java/com/ghostchu/peerbanhelper/module/impl/webapi/PBHChartController.java Removed SelectArg wrappers in query parameters for various methods, enhancing code clarity.
src/main/java/com/ghostchu/peerbanhelper/module/impl/webapi/PBHMetricsController.java Removed SelectArg wrappers in query parameters for handleBanTrends and handlePeerBans, improving readability.
src/main/java/com/ghostchu/peerbanhelper/module/impl/webapi/PBHTorrentController.java Removed SelectArg wrappers in torrent_id parameters for various methods, simplifying query construction.
src/main/java/com/ghostchu/peerbanhelper/text/TextManager.java Updated load() method for better translation handling and simplified localeEnabled() to always return true.

Possibly related PRs

  • 添加错误自动报告功能 #458: This PR updates the pom.xml file, which is also modified in the main PR for a version bump, indicating a related change in project configuration.
  • 抽离脚本引擎 + 添加 BTN 的脚本支持 #679: This PR introduces a script engine in the PeerBanHelperServer, which may relate to the overall functionality and structure of the application, similar to the changes made in the main PR regarding the server's operation.

Suggested reviewers

  • paulzzh

🐇 In the code, we hop and play,
With timestamps and queries, we clear the way.
No more wrappers, just simple and neat,
Version bumped, our work is complete!
Let’s celebrate with a joyful cheer,
For cleaner code, we hold so dear! 🎉


Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

❤️ Share
🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

@Ghost-chu
Copy link
Collaborator Author

@coderabbitai review

Copy link
Contributor

coderabbitai bot commented Nov 6, 2024

✅ Actions performed

Review triggered.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🧹 Outside diff range and nitpick comments (11)
src/main/java/com/ghostchu/peerbanhelper/database/dao/impl/AlertDao.java (2)

71-71: Consider consistent parameter handling across the method.

While removing SelectArg for the timestamp is appropriate, the method still uses SelectArg for the 'identifier' parameter. For consistency and clarity, consider either:

  1. Using SelectArg for all parameters
  2. Removing SelectArg wrapper for 'identifier' if ORMLite's built-in protection is sufficient

If option 2 is chosen, here's the suggested change:

    public void markAsRead(String identifier) throws SQLException {
        update(updateBuilder()
-               .updateColumnValue("identifier", new SelectArg(identifier))
+               .updateColumnValue("identifier", identifier)
                .updateColumnValue("readAt", new Timestamp(System.currentTimeMillis()))
                .prepare());
    }

Line range hint 1-73: Consider standardizing parameter handling across the DAO.

The class shows inconsistent usage of SelectArg across different methods. Some methods use it for String parameters (identifierAlertExists), while others don't (markAsRead). Consider establishing a consistent pattern for parameter handling across all DAO classes to:

  1. Improve code maintainability
  2. Prevent potential security issues
  3. Make the code more predictable for developers

Consider documenting the chosen approach in the class-level documentation or a central coding guideline document.

src/main/java/com/ghostchu/peerbanhelper/database/dao/impl/PeerRecordDao.java (1)

104-104: Consider maintaining consistent parameter handling patterns.

While using the ID directly is safe since it comes from an entity, the inconsistent use of SelectArg (used for address and downloader but not for torrent_id) could confuse maintainers. Consider either:

  1. Using SelectArg consistently for all parameters
  2. Removing SelectArg for all parameters if the ORM framework provides sufficient protection
  3. Adding a comment explaining the different treatment of parameters
 PeerRecordEntity existing = queryBuilder().where()
         .eq("address", new SelectArg(data.getAddress()))
         .and()
-        .eq("torrent_id", data.getTorrent().getId())
+        // Entity IDs are safe to use directly while user inputs need SelectArg
+        .eq("torrent_id", new SelectArg(data.getTorrent().getId()))
         .and()
         .eq("downloader", new SelectArg(data.getDownloader()))
src/main/java/com/ghostchu/peerbanhelper/database/dao/impl/TrafficJournalDao.java (1)

108-111: Inconsistent parameter handling in query conditions.

The method uses direct value for timestamp but SelectArg for downloader. This inconsistency could be confusing. Consider using the same approach for both parameters - either both with SelectArg or both with proper escaping.

 TrafficJournalEntity existing = queryBuilder().where()
-                .eq("timestamp", data.getTimestamp())
+                .eq("timestamp", new SelectArg(data.getTimestamp()))
                 .and()
                 .eq("downloader", new SelectArg(data.getDownloader())).queryBuilder().queryForFirst();

Or:

 TrafficJournalEntity existing = queryBuilder().where()
                 .eq("timestamp", data.getTimestamp())
                 .and()
-                .eq("downloader", new SelectArg(data.getDownloader())).queryBuilder().queryForFirst();
+                .eq("downloader", MsgUtil.escapeSql(data.getDownloader())).queryBuilder().queryForFirst();
src/main/java/com/ghostchu/peerbanhelper/database/dao/impl/ProgressCheatBlockerPersistDao.java (1)

110-112: Simplify the delete builder chain.

While removing SelectArg is good, the builder pattern usage can be simplified.

Consider simplifying the code:

-        var builder = deleteBuilder();
-        var where = builder.where().lt("lastTimeSeen", timestamp);
-        builder.setWhere(where);
-        return builder.delete();
+        return deleteBuilder()
+                .where()
+                .lt("lastTimeSeen", timestamp)
+                .prepare()
+                .delete();
src/main/java/com/ghostchu/peerbanhelper/metric/impl/persist/PersistMetrics.java (1)

56-58: Consider improving timestamp handling robustness.

While removing the SelectArg wrapper is appropriate, the timestamp handling could be improved:

  1. Consider using UTC consistently instead of system timezone to avoid potential issues if system timezone changes
  2. The timestamp conversion chain can be simplified

Here's a suggested improvement:

-                    new Timestamp(LocalDateTime.now().minusDays(keepDays)
-                            .toInstant(MiscUtil.getSystemZoneOffset())
-                            .toEpochMilli())));
+                    Timestamp.from(LocalDateTime.now(ZoneOffset.UTC)
+                            .minusDays(keepDays)
+                            .toInstant())));
src/main/java/com/ghostchu/peerbanhelper/module/impl/webapi/PBHTorrentController.java (2)

161-162: Simplify the query builder pattern usage.

While the removal of SelectArg is safe here, the query builder usage can be simplified:

-        var queryBuilder = peerRecordDao.queryBuilder().orderBy("lastTimeSeen", false);
-        var queryWhere = queryBuilder.where().eq("torrent_id", t);
-        queryBuilder.setWhere(queryWhere);
+        var queryBuilder = peerRecordDao.queryBuilder()
+                .orderBy("lastTimeSeen", false)
+                .where()
+                .eq("torrent_id", t)
+                .queryBuilder();

Line range hint 1-179: Consider adding API documentation.

The controller would benefit from OpenAPI/Swagger annotations to document:

  • API endpoints and their purposes
  • Request/response models
  • Parameter validations
  • Authentication requirements

This would improve API discoverability and maintainability.

Would you like me to provide an example of how to add OpenAPI annotations to one of the endpoints?

src/main/java/com/ghostchu/peerbanhelper/module/impl/webapi/PBHMetricsController.java (1)

179-182: Consider making the peerId substring length configurable.

The hardcoded substring length of 8 for "peerId" field might need adjustment if the peerId format changes in the future.

Consider extracting this value to a configuration property:

@Component
@IgnoreScan
public class PBHMetricsController extends AbstractFeatureModule {
+    @Value("${peerban.metrics.peerid.substring.length:8}")
+    private int peerIdSubstringLength;
     
     // ... existing code ...
     
     private void handleHistoryNumberAccess(Context ctx) {
         if ("peerId".equalsIgnoreCase(field)) {
-            substringLength = 8;
+            substringLength = peerIdSubstringLength;
         }
     }
}
src/main/java/com/ghostchu/peerbanhelper/text/TextManager.java (1)

Line range hint 293-303: Consider cleaning up commented code in localeEnabled().

The method has been simplified to always return true, but contains commented-out legacy code for regex-based locale filtering. While this change aligns with fixing text display issues for non-Chinese languages by enabling all locales, the commented code should be removed if it's no longer needed.

Apply this diff to clean up the method:

    public boolean localeEnabled(@NotNull String locale, @NotNull List<String> regex) {
        return true;
-//        for (String languagesRegex : regex) {
-//            try {
-//                if (Pattern.matches(CommonUtil.createRegexFromGlob(languagesRegex), locale)) {
-//                    return true;
-//                }
-//            } catch (PatternSyntaxException exception) {
-//                Log.debug("Pattern " + languagesRegex + " invalid, skipping...");
-//            }
-//        }
-//        return false;
    }
src/main/java/com/ghostchu/peerbanhelper/ipdb/IPDB.java (1)

296-299: Consider making timeout values configurable.

The timeout values are currently hardcoded. Consider making them configurable through application properties to allow easier adjustments in different environments.

-                .requestTimeout(Duration.of(2, ChronoUnit.MINUTES))
-                .connectTimeout(Duration.of(15, ChronoUnit.SECONDS))
-                .headersTimeout(Duration.of(15, ChronoUnit.SECONDS))
-                .readTimeout(Duration.of(30, ChronoUnit.SECONDS), Executors.newScheduledThreadPool(1, Thread.ofVirtual().factory()))
+                .requestTimeout(Duration.of(config.getRequestTimeoutMinutes(), ChronoUnit.MINUTES))
+                .connectTimeout(Duration.of(config.getConnectTimeoutSeconds(), ChronoUnit.SECONDS))
+                .headersTimeout(Duration.of(config.getHeadersTimeoutSeconds(), ChronoUnit.SECONDS))
+                .readTimeout(Duration.of(config.getReadTimeoutSeconds(), ChronoUnit.SECONDS),
+                        Executors.newScheduledThreadPool(config.getThreadPoolSize(), Thread.ofVirtual().factory()))
📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL

📥 Commits

Reviewing files that changed from the base of the PR and between 0ff1c09 and d48423d.

📒 Files selected for processing (13)
  • pom.xml (1 hunks)
  • src/main/java/com/ghostchu/peerbanhelper/PeerBanHelperServer.java (1 hunks)
  • src/main/java/com/ghostchu/peerbanhelper/database/dao/impl/AlertDao.java (2 hunks)
  • src/main/java/com/ghostchu/peerbanhelper/database/dao/impl/PeerRecordDao.java (2 hunks)
  • src/main/java/com/ghostchu/peerbanhelper/database/dao/impl/ProgressCheatBlockerPersistDao.java (2 hunks)
  • src/main/java/com/ghostchu/peerbanhelper/database/dao/impl/TrafficJournalDao.java (4 hunks)
  • src/main/java/com/ghostchu/peerbanhelper/ipdb/IPDB.java (2 hunks)
  • src/main/java/com/ghostchu/peerbanhelper/metric/impl/persist/PersistMetrics.java (1 hunks)
  • src/main/java/com/ghostchu/peerbanhelper/module/impl/rule/ActiveMonitoringModule.java (1 hunks)
  • src/main/java/com/ghostchu/peerbanhelper/module/impl/webapi/PBHChartController.java (4 hunks)
  • src/main/java/com/ghostchu/peerbanhelper/module/impl/webapi/PBHMetricsController.java (5 hunks)
  • src/main/java/com/ghostchu/peerbanhelper/module/impl/webapi/PBHTorrentController.java (3 hunks)
  • src/main/java/com/ghostchu/peerbanhelper/text/TextManager.java (1 hunks)
✅ Files skipped from review due to trivial changes (1)
  • pom.xml
🧰 Additional context used
📓 Learnings (3)
src/main/java/com/ghostchu/peerbanhelper/database/dao/impl/ProgressCheatBlockerPersistDao.java (2)
Learnt from: Ghost-chu
PR: PBH-BTN/PeerBanHelper#319
File: src/main/java/com/ghostchu/peerbanhelper/database/dao/impl/ProgressCheatBlockerPersistDao.java:59-59
Timestamp: 2024-10-09T01:34:57.560Z
Learning: In the `ProgressCheatBlockerPersistDao` class, `CopyOnWriteArrayList` is used in the `fetchFromDatabase` method to ensure thread safety when collecting results from a stream.
Learnt from: Ghost-chu
PR: PBH-BTN/PeerBanHelper#319
File: src/main/java/com/ghostchu/peerbanhelper/database/dao/impl/ProgressCheatBlockerPersistDao.java:59-59
Timestamp: 2024-08-15T18:22:54.529Z
Learning: In the `ProgressCheatBlockerPersistDao` class, `CopyOnWriteArrayList` is used in the `fetchFromDatabase` method to ensure thread safety when collecting results from a stream.
src/main/java/com/ghostchu/peerbanhelper/module/impl/webapi/PBHChartController.java (2)
Learnt from: Ghost-chu
PR: PBH-BTN/PeerBanHelper#408
File: src/main/java/com/ghostchu/peerbanhelper/module/impl/webapi/PBHChartController.java:84-89
Timestamp: 2024-08-31T10:59:44.980Z
Learning: Avoid suggesting logger-related improvements in Ghost-chu's code reviews.
Learnt from: Ghost-chu
PR: PBH-BTN/PeerBanHelper#408
File: src/main/java/com/ghostchu/peerbanhelper/module/impl/webapi/PBHChartController.java:84-89
Timestamp: 2024-10-09T01:34:57.560Z
Learning: Avoid suggesting logger-related improvements in Ghost-chu's code reviews.
src/main/java/com/ghostchu/peerbanhelper/module/impl/webapi/PBHMetricsController.java (2)
Learnt from: Ghost-chu
PR: PBH-BTN/PeerBanHelper#408
File: src/main/java/com/ghostchu/peerbanhelper/module/impl/webapi/PBHChartController.java:84-89
Timestamp: 2024-08-31T10:59:44.980Z
Learning: Avoid suggesting logger-related improvements in Ghost-chu's code reviews.
Learnt from: Ghost-chu
PR: PBH-BTN/PeerBanHelper#408
File: src/main/java/com/ghostchu/peerbanhelper/module/impl/webapi/PBHChartController.java:84-89
Timestamp: 2024-10-09T01:34:57.560Z
Learning: Avoid suggesting logger-related improvements in Ghost-chu's code reviews.
🔇 Additional comments (19)
src/main/java/com/ghostchu/peerbanhelper/database/dao/impl/PeerRecordDao.java (1)

46-46: LGTM! Verify input sanitization across the codebase.

The direct use of Timestamp object is safe here. However, since this is part of a broader change removing SelectArg wrappers, let's verify that all timestamp parameters are properly typed.

✅ Verification successful

Timestamp handling is properly typed and safe across the codebase

The verification shows that all timestamp-related operations in the codebase:

  • Use strongly-typed Timestamp parameters in method signatures
  • Directly pass these Timestamp objects to ORM queries
  • No string concatenation or raw string parameters are used in timestamp comparisons

The removal of SelectArg wrapper is safe as all timestamp handling is consistently using proper typing.

🏁 Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Verify that timestamp parameters are properly typed across the codebase
# Look for any string concatenation or raw string parameters in SQL queries with timestamp

# Search for potential string concatenation in SQL queries with timestamp-related fields
rg -g '*.java' -A 3 'where.*["\'].*time|time.*["\'].*where' 

# Search for raw string parameters in timestamp comparisons
ast-grep --pattern 'queryBuilder().where().eq($_, $str)' 

Length of output: 93


Script:

#!/bin/bash
# Search for potential string concatenation in SQL queries with timestamp-related fields
rg -g '*.java' -i "where.*time|time.*where"

# Search for timestamp-related method parameters and their types
ast-grep --pattern 'public $_ $_(Timestamp $_)'

# Search for timestamp comparisons in where clauses
ast-grep --pattern 'where().gt("$_time$_", $_)'
ast-grep --pattern 'where().eq("$_time$_", $_)'
ast-grep --pattern 'where().lt("$_time$_", $_)'

Length of output: 2260

src/main/java/com/ghostchu/peerbanhelper/database/dao/impl/TrafficJournalDao.java (2)

7-7: LGTM: Added MsgUtil import for SQL escaping.

This aligns with the security improvements mentioned in the PR objectives.


77-79: Verify the timestamp handling impact on traffic statistics.

The removal of SelectArg wrappers for timestamps is safe since we're using primitive long values. However, since this change is related to fixing the broken traffic statistics chart mentioned in the PR objectives, we should verify the fix.

✅ Verification successful

Based on the shell script results, I can now generate the final response:

Timestamp handling changes are consistent across the codebase

The removal of SelectArg wrappers is safe and consistent with the timestamp handling throughout the application:

  • Frontend (webui/src/service/charts.ts) correctly sends timestamps as milliseconds using getTime()
  • Backend (PBHChartController.java) properly processes these timestamps for chart data
  • The changes align with the existing timestamp handling patterns in both chart generation and data retrieval
🏁 Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Check for any related timestamp handling changes that might affect the traffic statistics chart
# Look for similar timestamp handling patterns in related files

# Search for similar timestamp handling in chart-related files
rg -l "traffic.*chart|chart.*traffic" | xargs rg -A 5 "timestamp|getTime"

# Search for any test files related to traffic statistics
fd -e java -e kt test.*traffic

Length of output: 6952

src/main/java/com/ghostchu/peerbanhelper/database/dao/impl/ProgressCheatBlockerPersistDao.java (2)

36-36: LGTM! Simplified timestamp comparison is safe.

The removal of SelectArg wrapper is acceptable here since ORMLite's QueryBuilder still handles parameterization internally for Timestamp objects, maintaining SQL injection protection while reducing unnecessary object creation.


Line range hint 52-91: Consider transaction rollback on partial failures in batch operations.

In flushDatabase, SQLExceptions are caught and logged per record, but processing continues. This could lead to inconsistent state if some records fail to update while others succeed.

Consider wrapping the batch operation in a transaction:

     public void flushDatabase(Deque<ProgressCheatBlocker.ClientTaskRecord> records) throws SQLException {
+        getConnectionSource().getReadWriteConnection(getTableName()).setAutoCommit(false);
         callBatchTasks(() -> {
             try {
                 while (!records.isEmpty()) {
                     // ... existing code ...
                 }
+                getConnectionSource().getReadWriteConnection(getTableName()).commit();
                 return null;
+            } catch (SQLException e) {
+                getConnectionSource().getReadWriteConnection(getTableName()).rollback();
+                throw e;
             }
         });
     }

Let's verify the current transaction handling:

src/main/java/com/ghostchu/peerbanhelper/metric/impl/persist/PersistMetrics.java (1)

Line range hint 1-150: Architectural improvements needed.

Several architectural aspects could be enhanced:

  1. The flush() method contains commented-out code that should be removed
  2. Consider adding explicit transaction boundaries for database operations
  3. Virtual thread error handling could be improved to propagate errors to the caller

Let's verify the transaction management pattern across the codebase:

src/main/java/com/ghostchu/peerbanhelper/module/impl/webapi/PBHTorrentController.java (2)

138-138: LGTM! Safe removal of SelectArg wrapper.

The removal of SelectArg wrapper around t.getId() is safe as the ID comes from a trusted database entity. The original torrent query maintains proper parameter binding through the queryByInfoHash method.

Also applies to: 143-143


115-115: LGTM! The SelectArg removal is safe here.

The removal of SelectArg wrapper around result.getId() is safe because the ID comes from a trusted database entity that was already properly queried. The method maintains SQL injection protection for user input (keyword search) using SelectArg.

Let's verify that all user inputs are properly sanitized:

Also applies to: 120-120

✅ Verification successful

The removal of SelectArg wrapper is safe for result.getId()

Based on the codebase analysis:

  1. The result.getId() is an internal ID from a database entity, not user input
  2. All user inputs in the codebase are properly protected with SelectArg wrappers, as evidenced by:
    • User-provided keyword searches using SelectArg for LIKE clauses
    • IP addresses and downloader names consistently wrapped with SelectArg
    • No direct string concatenation in SQL queries found
  3. The pattern of unwrapping SelectArg for internal IDs while maintaining it for user inputs is consistent across the codebase
🏁 Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Verify SQL injection protection for user inputs
# Look for direct string concatenation in SQL queries which might indicate SQL injection vulnerabilities

# Test 1: Look for string concatenation in SQL queries
rg -g '*.java' '(?i)(select|insert|update|delete).*\+.*(\$\{|\{|\'|\"|\+)'

# Test 2: Verify SelectArg usage for user inputs
ast-grep --pattern 'like($_, new SelectArg($$$))'

Length of output: 211


Script:

#!/bin/bash
# Let's fix the syntax and try again with separate commands

# Look for potential SQL injection vulnerabilities through string concatenation
rg -g "*.java" "(?i)(select|insert|update|delete).*\\+.*(\\\$\\{|\\{|\\'|\\\"|\\+)"

# Search for SQL query patterns
rg -g "*.java" "(?i)\\.(eq|like|in)\\s*\\(" -A 2

# Look for SelectArg usage
rg -g "*.java" "new SelectArg"

Length of output: 21168

src/main/java/com/ghostchu/peerbanhelper/module/impl/rule/ActiveMonitoringModule.java (1)

193-193: Verify the safety of removing SelectArg wrapper.

While removing the SelectArg wrapper aligns with the codebase-wide simplification, we need to ensure this doesn't introduce SQL injection vulnerabilities since dataRetentionTime is used directly in the query.

Let's verify the safety of this change:

Additionally, consider adding debug logging before the deletion to track the actual value:

 var where = deleteBuilder.where()
+log.debug("Cleaning up records older than: {}", dataRetentionTime);
         .lt("lastTimeSeen", dataRetentionTime);
✅ Verification successful

Based on the gathered evidence, I can now generate the final response:

Removing SelectArg wrapper is safe in this context

The removal of the SelectArg wrapper is safe because:

  1. dataRetentionTime is a private primitive long field
  2. It's initialized directly from config using getLong() method
  3. The value is validated (if (dataRetentionTime <= 0)) before use
  4. The class extends AbstractFeatureModule and uses ORMLite's DeleteBuilder, which handles primitive parameters safely

The change aligns with the PR's objective of removing unnecessary SelectArg wrappers while maintaining security.

🏁 Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Verify the type and usage of dataRetentionTime
# Test 1: Check if dataRetentionTime is properly initialized as a primitive
ast-grep --pattern 'private long dataRetentionTime;'

# Test 2: Verify all assignments to dataRetentionTime are numeric
rg -A 2 'dataRetentionTime\s*=' 

# Test 3: Check if the ORM framework (likely ORMLite) handles primitive parameters safely
rg -A 5 'deleteBuilder.*where.*lt'

Length of output: 637


Script:

#!/bin/bash
# Let's verify with a more comprehensive analysis

# Check the field declaration and type
ast-grep --pattern 'class ActiveMonitoringModule {
  $$$
  private $_ dataRetentionTime;
  $$$
}'

# Check the DeleteBuilder usage context
rg -B 5 -A 5 'deleteBuilder.*where.*lt'

# Check if there are any SQL injection related protections in the codebase
rg -l 'SelectArg|QueryBuilder|DeleteBuilder'

# Check the cleanup method implementation
ast-grep --pattern 'cleanup() {
  $$$
}'

Length of output: 1282


Script:

#!/bin/bash
# Let's focus on the specific file and its ORM usage

# Check the full context of the cleanup method in ActiveMonitoringModule
rg -B 10 -A 10 'cleanup\(\).*{' src/main/java/com/ghostchu/peerbanhelper/module/impl/rule/ActiveMonitoringModule.java

# Check how DeleteBuilder is used in other DAOs for comparison
rg -A 3 'DeleteBuilder.*where' src/main/java/com/ghostchu/peerbanhelper/database/dao/

# Check if dataRetentionTime is validated or sanitized before use
rg -B 5 'dataRetentionTime' src/main/java/com/ghostchu/peerbanhelper/module/impl/rule/ActiveMonitoringModule.java

# Check the parent class/interface for any SQL safety mechanisms
rg -l 'class.*ActiveMonitoringModule.*extends|implements'

Length of output: 7219


Script:

#!/bin/bash
# Let's try one final verification with fixed regex

# Check the cleanup method context with fixed regex
rg -B 10 -A 10 'cleanup\(' src/main/java/com/ghostchu/peerbanhelper/module/impl/rule/ActiveMonitoringModule.java

# Check if this class extends AbstractFeatureModule which might have SQL safety
rg -A 5 'class ActiveMonitoringModule extends' src/main/java/com/ghostchu/peerbanhelper/module/impl/rule/ActiveMonitoringModule.java

# Check how DeleteBuilder is used in the specific DAO class
rg -A 3 'DeleteBuilder' src/main/java/com/ghostchu/peerbanhelper/database/dao/impl/PeerRecordDao.java

# Check the AbstractPBHDao for any SQL safety mechanisms
rg -A 5 'protected.*DeleteBuilder' src/main/java/com/ghostchu/peerbanhelper/database/dao/AbstractPBHDao.java

Length of output: 1686

src/main/java/com/ghostchu/peerbanhelper/module/impl/webapi/PBHMetricsController.java (3)

63-65: LGTM! Query construction simplified and stream formatting improved.

The changes simplify the query construction by removing unnecessary SelectArg wrappers around timestamp parameters while maintaining the same functionality. The stream operation formatting improvements enhance code readability.

Also applies to: 76-78


197-199: Query simplification looks good, but please clarify the hardcoded zero.

The removal of SelectArg wrappers simplifies the code, but there's a hardcoded zero in the UniversalFieldDateResult constructor that needs explanation.

Let's check the usage of UniversalFieldDateResult:

Also applies to: 206-206

✅ Verification successful

Let me gather more information about the UniversalFieldDateResult class and its usage.


The hardcoded zero is correct - it's used for trend data where percentage isn't relevant

Looking at the codebase, I can see that UniversalFieldDateResult is a record class with three fields: timestamp, count, and percent. In PBHMetricsController, the zero is intentionally used for the percent field since this endpoint only needs the timestamp and count for trend visualization, unlike other usages in HistoryDao where percentage calculations are relevant.

🏁 Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Check how UniversalFieldDateResult is used elsewhere

# Test: Look for other instances of UniversalFieldDateResult construction
rg "new.*UniversalFieldDateResult.*" --type java

# Test: Look for the class definition
ast-grep --pattern 'class UniversalFieldDateResult {
  $$$
}'

Length of output: 547


Script:

#!/bin/bash
# Find the definition and usage of UniversalFieldDateResult
rg -A 10 "class.*UniversalFieldDateResult" --type java

# Find all references to UniversalFieldDateResult
rg "UniversalFieldDateResult" --type java -A 5

Length of output: 3617


90-91: Performance optimization for first-screen requests looks good.

The optimization to redirect "banAt" field queries with "day" type to handlePeerBans should improve first-screen load times.

Let's verify the performance impact of this optimization:

✅ Verification successful

Performance optimization for first-screen requests is properly implemented

The code changes show a well-implemented optimization where:

  • The handlePeerBans method uses a more efficient query by selecting only necessary columns ("id", "banAt")
  • It directly processes daily ban trends without the overhead of the generic date handling logic
  • The optimization is specifically targeted for the common first-screen use case (banAt + day type)

The implementation aligns with the performance goals and doesn't introduce any potential issues.

🏁 Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Check for any performance-related tests or benchmarks

# Test: Look for performance tests
rg -l "Performance|Benchmark|Test.*banAt.*day" --type java

# Test: Look for related configuration or documentation
rg -l "first.*screen|performance.*optimization" --type-add config:*.{properties,yml,yaml,xml} --type config

Length of output: 441


Script:

#!/bin/bash
# Let's verify the optimization by checking:
# 1. The implementation of handlePeerBans method
# 2. The overall flow in the controller
# 3. Any related performance configurations

# Check the implementation details
ast-grep --pattern 'handlePeerBans($$$) {
  $$$
}'

# Look for the complete method implementation
rg -A 20 "handleHistoryDateAccess.*\{" --type java

# Check for any performance-related configurations in properties files
fd -e properties -x cat {}

Length of output: 2834


Script:

#!/bin/bash
# Let's check the handlePeerBans implementation to understand the optimization
rg -A 30 "handlePeerBans.*\{" --type java

# Also check for any comments or documentation about this optimization
rg -B 5 -A 10 "首屏|first.*screen|performance" --type java

Length of output: 9925

src/main/java/com/ghostchu/peerbanhelper/module/impl/webapi/PBHChartController.java (3)

111-113: LGTM! Improved query readability while maintaining security.

The removal of unnecessary SelectArg wrappers for timestamp comparisons while retaining it for the user-provided downloader parameter is a good improvement. This maintains security for user inputs while making the code more readable.

Also applies to: 117-119, 122-122


166-168: LGTM! Consistent query improvements.

The changes maintain consistency with other query improvements in the file, removing unnecessary SelectArg wrappers from timestamp comparisons. The added spacing in the iterator implementation improves readability.

Also applies to: 173-175, 188-188


89-91: Verify traffic statistics chart functionality.

The code formatting improvement looks good. Since this change is related to fixing the broken traffic statistics chart mentioned in the PR objectives, please ensure that the chart now displays correctly.

✅ Verification successful

Let me analyze the new context from the shell scripts. The output reveals:

  1. No test files were found for traffic-related functionality
  2. Recent changes show the addition of TrafficJournalDao.java and TrafficJournalEntity.java files
  3. The TrafficJournalDao.getDayOffsetData() method (which is called in the review snippet) has proper implementation for handling traffic data with timezone fixes

Let me verify one more aspect about the timezone handling:


Based on the shell script results, I can now generate a final response. The code shows proper timezone handling with:

  1. A centralized MiscUtil.getSystemZoneOffset() utility
  2. Consistent timezone fixing implementation in both TrafficJournalDao and PBHChartController
  3. Proper usage of the timezone fixes in the traffic data processing chain

Traffic statistics chart implementation verified

The code changes are well-implemented with proper timezone handling and traffic data processing. The getDayOffsetData method correctly:

  • Handles both overall and specific downloader traffic data
  • Applies timezone fixes consistently
  • Calculates proper traffic offsets between days
🏁 Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Check for any related test files or recent changes to traffic-related code
# Look for test files
echo "Searching for related test files..."
fd -e java -e groovy -e kotlin "Traffic.*Test|.*TrafficTest" 

# Look for recent changes to traffic-related files
echo "Checking recent changes to traffic-related files..."
git log -p --since="1 month ago" -- "**/Traffic*.java" "**/traffic*.java"

Length of output: 7626


Script:

#!/bin/bash
# Check for timezone-related code and utilities
echo "Checking timezone utilities..."
rg -A 5 "getSystemZoneOffset|fixTimezone" 

# Check for any related configuration files
echo "Checking configuration files..."
fd -e properties -e yml -e yaml | xargs rg -l "timezone|time.zone"

Length of output: 7333

src/main/java/com/ghostchu/peerbanhelper/text/TextManager.java (1)

Line range hint 166-178: Verify the impact of removing locale filtering.

The removal of locale filtering logic (by making localeEnabled() always return true) and the addition of a second fillMissing() call appear to be related changes aimed at fixing text display issues. However, we should verify that:

  1. All locales are properly loaded and displayed
  2. The fallback mechanism works correctly for missing translations
  3. There are no performance impacts from loading all locales

Let's check the locale-related files and their structure:

✅ Verification successful

Let me analyze the locale-related findings and generate a final response.

Locale handling changes are safe and properly implemented

The removal of locale filtering is safe because:

  1. The codebase has a robust fallback mechanism:

    • Default fallback to en_us is enforced if no languages are enabled
    • Missing translations are filled from messages_fallback.yml
    • The fillMissing() call ensures completeness of translations
  2. Performance impact is minimal:

    • Only 3 locale files exist: en_us, zh_cn, and messages_fallback.yml
    • Locales are loaded once at startup and cached in memory
    • The ConcurrentHashMap in LanguageFilesManager ensures thread-safe access
  3. All locales are properly handled:

    • Locale codes are normalized (lowercase, underscore-separated)
    • Browser language preferences are respected via Accept-Language header
    • System default locale is considered as fallback
🏁 Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Verify locale files and their structure

# Check available locale files
echo "Available locale files:"
fd -e yml -e yaml . src/main/resources/lang/

# Check for potential performance impact by counting total locales
echo -e "\nTotal number of locale files:"
fd -e yml -e yaml . src/main/resources/lang/ | wc -l

# Check for any hardcoded locale references that might conflict
echo -e "\nChecking for hardcoded locale references:"
rg -i "locale|language" --type java

Length of output: 51166

src/main/java/com/ghostchu/peerbanhelper/ipdb/IPDB.java (2)

46-46: LGTM!

The added import for Executors is necessary for the virtual thread pool configuration.


296-299: LGTM! Timeout configurations address the IPDB download issues.

The changes properly address the timeout issues mentioned in the PR objectives:

  • 2-minute request timeout allows sufficient time for IPDB downloads
  • Virtual thread pool improves performance for read timeout operations
  • Explicit timeout values for connect and headers operations
src/main/java/com/ghostchu/peerbanhelper/PeerBanHelperServer.java (1)

190-190: LGTM: Improved service startup order.

Moving registerHttpServer() before resetKnownDownloaders() aligns with the PR objectives to ensure proper service initialization order, particularly for IPDB downloads.

Let's verify there are no dependencies on this order by checking for any HTTP-related calls in resetKnownDownloaders:

✅ Verification successful

Moving HTTP server initialization earlier is safe and beneficial

The code inspection reveals:

  • resetKnownDownloaders() only performs login and ban list resets on downloaders, with no HTTP dependencies
  • registerHttpServer() initializes the web server that's needed for IPDB functionality
  • Moving HTTP server initialization earlier ensures the web service is available before any downloader operations
🏁 Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Check for HTTP-related calls in resetKnownDownloaders and related methods
# Test: Look for HTTP/network calls that might be affected by the reordering
ast-grep --pattern $'class PeerBanHelperServer {
  $$$
  resetKnownDownloaders() {
    $$$
  }
  $$$
}'

Length of output: 104


Script:

#!/bin/bash
# Let's try a different approach using ripgrep to find the resetKnownDownloaders method
rg -U "resetKnownDownloaders.*?\{.*?\}" -A 10

Length of output: 47


Script:

#!/bin/bash
# Also check for any HTTP client or server initialization dependencies
rg "registerHttpServer.*?\{.*?\}" -U -A 10

Length of output: 44


Script:

#!/bin/bash
# Look for any direct file references to understand the method's purpose
rg "resetKnownDownloaders" --type java

Length of output: 345


Script:

#!/bin/bash
# Let's try a simpler pattern to find the method content
rg -U "private void resetKnownDownloaders\(\).*?\{" -A 20

Length of output: 2096


Script:

#!/bin/bash
# And look for the registerHttpServer method
rg -U "private void registerHttpServer\(\).*?\{" -A 20

Length of output: 2503


Script:

#!/bin/bash
# Also check for any references to HTTP or server in the class
rg "http|server" -i src/main/java/com/ghostchu/peerbanhelper/PeerBanHelperServer.java

Length of output: 1405

@Ghost-chu Ghost-chu merged commit 7a312a2 into release Nov 6, 2024
15 checks passed
@coderabbitai coderabbitai bot mentioned this pull request Nov 11, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants