Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature][doc][Connector-V2][Fake] Add FakeSource connector documentation #5255

Merged
merged 14 commits into from
Oct 20, 2023

Conversation

zhilinli123
Copy link
Contributor

@zhilinli123 zhilinli123 commented Aug 9, 2023

Purpose of this pull request

subTask:#4572

Check list

zhilinli123 added a commit to zhilinli123/incubator-seatunnel that referenced this pull request Aug 11, 2023
@zhilinli123
Copy link
Contributor Author

zhilinli123 commented Aug 19, 2023

@hailin0 @liugddx @Hisoka-X PTAL

{
kind = DELETE
fields = [1, "A_1", 100]
source {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I remember FakeSource can produce data randomly, could you also add a demo case use that feature? Please.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

@zhilinli123
Copy link
Contributor Author

zhilinli123 commented Sep 13, 2023

PTAL: @Hisoka-X @EricJoy2048 @liugddx

@zhilinli123 zhilinli123 reopened this Sep 18, 2023
# Conflicts:
#	docs/en/connector-v2/source/FakeSource.md
@Hisoka-X
Copy link
Member

Since there have been many changes to the Fake connector recently, it is best for us to merge this PR to the dev branch as soon as possible. Could you update this PR to date? @zhilinli123 cc @ruanwenjun

# Conflicts:
#	docs/en/connector-v2/source/FakeSource.md
@zhilinli123
Copy link
Contributor Author

Since there have been many changes to the Fake connector recently, it is best for us to merge this PR to the dev branch as soon as possible. Could you update this PR to date? @zhilinli123 cc @ruanwenjun

I made some changes. Thank you for your hard work

| double.min | double | no | 0 |
| double.max | double | no | 0x1.fffffffffffffP+1023 |
| double.template | list | no | - |
| table-names | list | no | - |
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you missed this.

| Name | Type | Required | Default | Description |
|---------------------|----------|----------|-------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| schema | config | yes | - | Define Schema information |
| fields | config | yes | - | Define Field information |
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where is fields?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Uploading cd0ba9dc-0ce2-4c58-86ca-e2785bd38458.jpeg…

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

### schema [config]

#### fields [Config]

The schema of fake data that you want to generate

#### Examples

here

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a sub config of schema. Not for FakeSource root config. Please refer https://github.com/apache/seatunnel/blob/dev/docs/en/concept/schema-feature.md

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

| double.fake.mode | string | no | range | The fake mode of generating float data, support `range` and `template`, default `range`,if use configured it to `template`, user should also configured `double.template` option |
| double.min | double | no | 0 | The min value of double data that connector generated |
| double.max | double | no | 0x1.fffffffffffffP+1023 | The max value of double data that connector generated |
| double.template | list | no | - | The template list of double type that connector generated, if user configured it, connector will randomly select an item from the template list |
| table-names | list | no | - |
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

description for table-names?

| double.min | double | no | 0 | The min value of double data that connector generated |
| double.max | double | no | 0x1.fffffffffffffP+1023 | The max value of double data that connector generated |
| double.template | list | no | - | The template list of double type that connector generated, if user configured it, connector will randomly select an item from the template list |
| table-names | list | no | - | The Supports configuration of multiple data sources for example: ["test.table1", "test.table2"] |
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Check the old one https://github.com/apache/seatunnel/pull/5255/files#diff-368d0e6bf6b91efe5c90ec4eaa163de7332b3e6e64797a0fc6b27dd3d0711326L266. You losted many important information in the new one description.

Comment on lines -268 to -271
The table list that connector generated, used to simulate multi-table scenarios.

This option will override the `table` option in the `schema` option.
For example, if you configure the `table-names` option as follows, the connector will generate data for the `test.table1` and `test.table2` tables, the `database.schema.table` will be drop.
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Hisoka-X here?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks review,complete modification

@hailin0 hailin0 merged commit f81f9f6 into apache:dev Oct 20, 2023
7 checks passed
@zhilinli123 zhilinli123 deleted the feature-doc-fakeSource branch October 24, 2023 09:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants