Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support read data from mysql when upload data #574

Open
wants to merge 628 commits into
base: main
Choose a base branch
from

Conversation

FancyXun
Copy link

@FancyXun FancyXun commented Jul 19, 2024

Follow this issue: FederatedAI/FATE#5669

Support uploading data from mysql:

Change the 'file' in upload_guest_mysql.json like bellow:

{
"file":"mysql://root:[email protected]:13312/fl/BREAST_HETERO_GUEST",
"head": true,
"partitions": 16,
"extend_sid": true,
"meta": {
"delimiter": ",",
"label_name": "y",
"match_id_name": "id"
},
"namespace": "experiment",
"name": "breast_hetero_guest_mysql"
}

Upload data
flow data upload -c upload_guest_mysql.json

The three formats of the file variable in this JSON are as follows:

mysql://user:password@host_ip:host_port/db/table
file:///path/to/local_file.csv
/path/to/local_file.csv

zhihuiwan and others added 30 commits December 7, 2023 21:08
Signed-off-by: zhihuiwan <[email protected]>
Signed-off-by: zhihuiwan <[email protected]>
Signed-off-by: zhihuiwan <[email protected]>
Signed-off-by: zhihuiwan <[email protected]>
Signed-off-by: zhihuiwan <[email protected]>
Signed-off-by: zhihuiwan <[email protected]>
Signed-off-by: zhihuiwan <[email protected]>
Signed-off-by: zhihuiwan <[email protected]>
Signed-off-by: zhihuiwan <[email protected]>
Signed-off-by: zhihuiwan <[email protected]>
Signed-off-by: zhihuiwan <[email protected]>
Signed-off-by: zhihuiwan <[email protected]>
Signed-off-by: zhihuiwan <[email protected]>
Signed-off-by: zhihuiwan <[email protected]>
Signed-off-by: zhihuiwan <[email protected]>
Signed-off-by: zhihuiwan <[email protected]>
Signed-off-by: zhihuiwan <[email protected]>
Signed-off-by: chengtcc <[email protected]>
Signed-off-by: zhihuiwan <[email protected]>
Signed-off-by: zhihuiwan <[email protected]>
Signed-off-by: zhihuiwan <[email protected]>
Signed-off-by: zhihuiwan <[email protected]>
Signed-off-by: zhihuiwan <[email protected]>
Signed-off-by: zhihuiwan <[email protected]>
Signed-off-by: zhihuiwan <[email protected]>
Signed-off-by: zhihuiwan <[email protected]>
Signed-off-by: zhihuiwan <[email protected]>
Signed-off-by: zhihuiwan <[email protected]>
Signed-off-by: zhihuiwan <[email protected]>
Signed-off-by: zhihuiwan <[email protected]>
Signed-off-by: zhihuiwan <[email protected]>
@FancyXun
Copy link
Author

@sagewe
@dylan-fan

@sagewe
Copy link
Contributor

sagewe commented Jul 19, 2024

首先很感谢你的贡献,社区对DCO有强制要求,请依照 Details 进行修复

有空看看 @zhihuiwan

@sagewe sagewe requested a review from zhihuiwan July 19, 2024 02:48
@FancyXun
Copy link
Author

首先很感谢你的贡献,社区对DCO有强制要求,请依照 Details 进行修复

有空看看 @zhihuiwan

Done!

@FancyXun
Copy link
Author

补充一下:这样做的好处就是,能兼容现有的upload.json,只需要根据里面file的前缀判断是用读取mysql或者本地文件,json不需要新增内容。 @sagewe

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants