Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(database): 支持多种数据库类型 & 安全性增强 #3462

Closed
wants to merge 46 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
46 commits
Select commit Hold shift + click to select a range
1a434ab
feat(database): support multiply database type & secure enhance
May 25, 2024
0419384
feat(config): change default config
May 25, 2024
55dd933
fix(oracle): oracle xml bug
May 31, 2024
026a774
fix(datasize): modify database data field size to bigger for varchar,…
May 31, 2024
1878aa7
fix(method): inject possible method args
Jun 18, 2024
9128f17
feat(dm/kingbase): adapt for dm/kingbase database
Jun 21, 2024
fbdf8ce
style(login): adjust login page css style
Jun 21, 2024
013bc38
feat(platform): separate database platform & login page style & log p…
Jul 5, 2024
afc06c2
docs(gitignore): update git ignore rules
Jul 5, 2024
b89276b
docs(readme): add multiply platform database doc
Jul 5, 2024
340a0ad
docs(readme): update multiply platform database doc
Jul 5, 2024
0379ab3
Merge branch 'xuxueli:master' into master
Ice2Faith Aug 27, 2024
a8cdc60
fix(log): log sql bug & concurrent control for scheduler
Aug 27, 2024
45601dc
feat(sm-crypto): sm-crypto dependency update
Aug 28, 2024
e06969b
feat(compiler): compiler with parameters options
Aug 28, 2024
364296b
feat(dockerfile): support docker file
Sep 26, 2024
d2aa63a
refactor(pageable): refactor pageable implements logic
Sep 26, 2024
7889f01
Merge branch 'refs/heads/xuxueli-master'
Nov 20, 2024
584d773
feat(upgrade): upgrade with official
Nov 20, 2024
f352510
feat(password): password update with confirm
Nov 20, 2024
8b77b01
feat(upgrade): sync multiply code
Nov 21, 2024
beff927
feat(readme): update README.multiply.md
Nov 21, 2024
c3e35db
Merge remote-tracking branch 'official/master' into tmp/official
Dec 11, 2024
68ffe61
feat(official): sync official
Dec 11, 2024
43ef0e4
feat(jobinfo): jobinfo list page add newest_trigger_time & newest_log…
Dec 12, 2024
87a6a1a
feat(jobgroup): jobgroup title length limit change
Dec 12, 2024
78cd945
feat(jobinfo): jobinfo add remark column on detail page
Dec 12, 2024
20bd8bf
feat(jobinfo): jobinfo add remark column on detail page
Dec 12, 2024
f596d02
feat(email-config): email support online edit config
Dec 13, 2024
0902d16
fix(token): token verify for empty token string
Dec 13, 2024
7283f92
feat(login): login cookie auto refresh
Dec 19, 2024
a6f746f
feat(login): login cookie auto refresh
Dec 19, 2024
bb98f7e
feat(login): login cookie auto refresh
Dec 19, 2024
4eaae7f
feat(login): login cookie auto refresh
Dec 19, 2024
8721508
Merge remote-tracking branch 'origin/master'
Dec 19, 2024
5a53876
fix(token): token expire and refresh control
Dec 20, 2024
3fc1cac
feat(starter): add springboot starter
Dec 20, 2024
d22af33
Merge branch 'official' into merge
Jan 11, 2025
d343737
feat(official): sync official features
Jan 11, 2025
5bc11c8
feat(official): sync official features
Jan 11, 2025
cf0f567
doc(readme): update readme
Jan 11, 2025
11ff896
doc(readme): update readme
Jan 11, 2025
935fc96
fix(test): test code
Jan 11, 2025
8e6b137
Merge branch 'master' into merge
Jan 11, 2025
a903cf9
feat(official): sync official features
Jan 11, 2025
8388972
feat(official): sync official features
Jan 11, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 0 additions & 23 deletions .github/workflows/maven.yml

This file was deleted.

7 changes: 7 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,7 +1,14 @@
.idea
.classpath
.project
logs
*.log
*.iml
target/
.DS_Store
.gitattributes
logs
*.log
*.bak
*.tmp
~.*
4 changes: 4 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,10 @@ Now, it's already open source, and many companies use it in production environme

XXL-JOB是一个分布式任务调度平台,其核心设计目标是开发迅速、学习简单、轻量级、易扩展。现已开放源代码并接入多家公司线上产品线,开箱即用。

## fork introduce
- please view README.multiply.md

[README.multiply.md](README.multiply.md)

## Sponsor
XXL-JOB is an open source and free project, with its ongoing development made possible entirely by the support of these awesome backers.
Expand Down
202 changes: 202 additions & 0 deletions README.multiply.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,202 @@
# multiply platform database supports

## project
- this project is a branch from https://github.com/xuxueli/xxl-job .
- and this project was forked
- it changes include in pull request https://github.com/xuxueli/xxl-job/pull/3462
```shell
feat(database): support multiply database type & secure enhance #3462
```
- aim to adapt oracle/gbase xxl-job-admin.
- those change not effect xxl-job-core,.
- below config database platform to switch target platform
```shell
# database platform, mysql|oracle|postgre|gbase|h2|dm|kingbase|sqlserver
xxl.job.database.platform.type=mysql
```
- actually, mysql grammar adapted to gbase, or other, so that, other database could be use mysql platform.

- 这个项目是 https://github.com/xuxueli/xxl-job 的一个分支。
- 这是一个fork出来的项目
- 它的变更都包含在PR中 https://github.com/xuxueli/xxl-job/pull/3462
```shell
feat(database): support multiply database type & secure enhance #3462
```
- 目标是适配 oracle/gbase 版本的 xxl-job-admin
- 这些改变不影响 xxl-job-core
- 下面的配置数据库平台为目标平台
```shell
# database platform, mysql|oracle|postgre|gbase|h2|dm|kingbase|sqlserver
xxl.job.database.platform.type=mysql
```
- 实际上,mysql 的语法适用于 gbase ,或者其他兼容的数据库,所以其他数据库也可以使用 mysql 平台

## notice
- 2025-01-11
- 同步最新的3.0.0-SNAPSHOT版本,为最后一个同步官方的springboot2版本
- 官方在此之后升级为springboot3+jdk17
- 后续将会在springboot3分支中保持与官方版本同步,同时将会协同地将官方修改也同步到springboot2(master)版本中
- 不过分支命名可能会变更:master->springboot2, springboot3->master

## Branch Merge Rule
- [master] fork from official master and adapt multiply database
- [springboot3] adapt to springboot3 & jakarta & jdk17 base on master
- [multiply/master] partial upgrade package for [master]
- [multiply/springboot3] partial upgrade package for [springboot3]
- merge rule : source merge to target [source] -> [target]
- all rule is :
- [master] -> [springboot3]
- [master] -> [multiply/master]
- [springboot3] -> [multiply/springboot3]

- [master] 从官方fork其master分支进行多数据库适配
- [springboot3] 基于master分支适配 springboot3 & jakarta & jdk17
- [multiply/master] 是 [master]分支的分包打包适配
- [multiply/springboot3] 是 [springboot3]分支的分包打包适配
- merge 规则 : 源分支合并到目标分支 [source] -> [target]
- 所有规则如下 :
- [master] -> [springboot3]
- [master] -> [multiply/master]
- [springboot3] -> [multiply/springboot3]

## features
- support database type list
- mysql(official)
- oracle
- postgre
- gbase(南大通用)
- h2
- dm(dameng/达梦)
- kingbase(人大金仓)
- sqlserver(>=2012)
- support auto run init sql
- finish
- secure protect
- password payload transfer encrypt by sm2
- cookie token encrypt by sm2
- sm2 support by com.antherd/sm-crypto-0.3.0
- because of it support java and js environment
- password database store encode by BCryptPasswordEncoder instead of MD5
- this change was not adapt official
- will cause that already exists user cannot login
- so that, suggest new project to use it
- but you can direct modify database table to cover password
- sql is : update table xxl_job_user set password='$2a$10$rElzP.wCmjyjsVIIqoP4fe8u1qH3otIxiG4UhYs9A3Ivsrm1LrpOu' where 1=1;
- this update will set all user password to : 123456
- and next, to change password in web page
- api invoke
- support use api invoke admin by http header name "token"
- got token value by /login response header "token"
- response such as:
- token: expire1716627232689.073d3d163...
- it split by '.' as two part
- first part is expire1716627232689 mean this token will expire at 1716627232689 milliseconds timestamp
- second part is token body
- and then, request with this header to invoke api
- request token is "expire1716627232689.073d3d163..." or "073d3d163...", all right
- how to login ?
- like this:
- static/js/login.1.js
- to search "submitHandler" or "generateKeyPairHex"

- 支持的数据库库类型
- mysql(官方)
- oracle
- postgre
- gbase(南大通用)
- h2
- dm(达梦数据库)
- kingbase(人大金仓)
- sqlserver(>=2012)
- 支持自动运行初始化脚本
- 已完成
- 安全保护
- 密码的报文传输使用SM2加密
- Cookie的token使用SM2加密
- SM2 由com.antherd/sm-crypto-0.3.0提供
- 因为他支持java和js环境
- 数据库保存的密码使用BCryptPasswordEncoder代替MD5
- 这个改变不兼容官方
- 将导致已经存在的用户无法登录
- 所以,建议新项目使用
- 但是,你也可以直接修改数据库表覆盖密码
- SQL是:update table xxl_job_user set password='$2a$10$rElzP.wCmjyjsVIIqoP4fe8u1qH3otIxiG4UhYs9A3Ivsrm1LrpOu' where 1=1;
- 这个更新将会设置所有用户的密码为:123456
- 然后,去网页中更新密码
- api 调用
- 支持调用admin的接口,通过在http请求头中添加 "token"
- token 在 /login 接口响应头 "token" 中
- 响应头类似:
- token: expire1716627232689.073d3d163...
- 它被 '.' 分为两部分
- 第一部分是 expire1716627232689 表示这个 token 将会在 1716627232689 毫秒时间戳过期
- 第二部分是 token 体
- 接下来,请求时携带这个请求头调用
- 请求的 token 是 "expire1716627232689.073d3d163..." 或者 "073d3d163..." 都对
- 怎么登录呢?
- 像这样:
- static/js/login.1.js
- 搜索 "submitHandler" 或者 "generateKeyPairHex"

## usage
- 1. download the package
- xxl-job-admin-all.tar.gz
```shell
wget https://gitee.com/ice2faith/xxl-job/releases/download/v3.0.0-SNAPSHOT-springboot3/xxl-job-admin-all.tar.gz
```
- 2. release it
```shell
tar -xzvf xxl-job-admin-all.tar.gz
```
- 3. cd to package
```shell
cd xxl-job-admin-all/xxl-job-admin
```
- 4. modify the config file
```shell
vi resources/application-dev.properties
```
- 5. run it
```shell
./jarctrl.sh restart
```
- or direct run jar
```shell
java -jar xxl-job-admin.jar
```
- 6. view log
```shell
./jarctrl.sh log
```

## previews

- login page
- 登录页

![img_login.png](./doc/images/img_login.png)

- index page
- 首页

![img_index.png](./doc/images/img_index.png)

- encrypt transport
- 加密传输

![img_crypto.png](./doc/images/img_crypto.png)

- log page enhance
- 日志页增强

![img_log_page.png](./doc/images/img_log_page.png)

- executor page
- 执行器页面

![img_registry.png](./doc/images/img_registry.png)

- task page
- 任务页面

![img_task.png](./doc/images/img_task.png)
Loading