ci: Use the correct Python executables for extracting plugin metadata #1883

edgarrmondragon · 2024-11-05T18:02:23Z

feat: Allow passing a custom Python exectuable to extract-sdk-metadata-to-s3 hub-utils#190

netlify · 2024-11-05T18:02:41Z

✅ Deploy Preview for meltano-hub ready!

Name	Link
🔨 Latest commit	`b4d6fcb`
🔍 Latest deploy log	https://app.netlify.com/sites/meltano-hub/deploys/672a75f745bd4900084f0ce1
😎 Deploy Preview	https://deploy-preview-1883--meltano-hub.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

Co-authored-by: Edgar Ramírez Mondragón <[email protected]> Co-authored-by: edgarrmondragon <[email protected]> Co-authored-by: Edgar Ramírez Mondragón <[email protected]>

github-actions · 2024-11-05T19:46:36Z

Testing plugin tap-apple-search-ads (ticketswap variant):

Starting test job...
...
Job completed.

Auto-generated README.md

`tap-apple-search-ads`

AppleSearchAds tap class.

Built with the Meltano Singer SDK.

Capabilities

catalog
state
discover
about
stream-maps
schema-flattening
batch

Settings

Setting	Required	Default	Description
client_id	True	None	The client id to authenticate against the API service
client_secret	True	None	The client secret to authenticate against the API service
org_id	True	None	The organisation id in your apple search ads.
start_date	False	None	Start date for reporting streams, format in YYYY-MM-DD.
end_date	False	2024-11-05	End date for reporting streams, format in YYYY-MM-DD.
report_granularity	False	None	The granularity of reporting streams. One of HOURLY, DAILY, WEEKLY, MONTHLY.
stream_maps	False	None	Config object for stream maps capability. For more information check out Stream Maps.
stream_map_config	False	None	User-defined config values to be used within map expressions.
faker_config	False	None	Config for the `Faker` instance variable `fake` used within map expressions. Only applicable if the plugin specifies `faker` as an addtional dependency (through the `singer-sdk` `faker` extra or directly).
faker_config.seed	False	None	Value to seed the Faker generator for deterministic output: https://faker.readthedocs.io/en/master/#seeding-the-generator
faker_config.locale	False	None	One or more LCID locale strings to produce localized output for: https://faker.readthedocs.io/en/master/#localization
flattening_enabled	False	None	'True' to enable schema flattening and automatically expand nested properties.
flattening_max_depth	False	None	The max depth to flatten schemas.
batch_config	False	None
batch_config.encoding	False	None	Specifies the format and compression of the batch files.
batch_config.encoding.format	False	None	Format to use for batch files.
batch_config.encoding.compression	False	None	Compression format to use for batch files.
batch_config.storage	False	None	Defines the storage layer to use when writing batch files
batch_config.storage.root	False	None	Root path to use when writing batch files.
batch_config.storage.prefix	False	None	Prefix to use when writing batch files.

A full list of supported settings and capabilities is available by running: tap-apple-search-ads --about

Version info

tap-apple-search-ads v[could not be detected], Meltano SDK v0.39.1

Usage info

melty-bot % tap-apple-search-ads --help
Usage: tap-apple-search-ads [OPTIONS]

  Execute the Singer tap.

Options:
  --version                 Display the package version.
  --about                   Display package metadata and settings.
  --format [json|markdown]  Specify output style for --about
  --config TEXT             Configuration file location or 'ENV' to use
                            environment variables.
  --discover                Run the tap in discovery mode.
  --test TEXT               Use --test to sync a single record for each
                            stream. Use --test=schema to test schema output
                            without syncing records.
  --catalog PATH            Use a Singer catalog file with the tap.
  --state PATH              Use a bookmarks file for incremental replication.
  --help                    Show this message and exit.

Detected capabilities

✅ 'discover'

✅ 'catalog'

✅ 'state'

✅ 'about'

JSON Metadata

{
  "name": "tap-apple-search-ads",
  "description": "AppleSearchAds tap class.",
  "version": "[could not be detected]",
  "sdk_version": "0.39.1",
  "supported_python_versions": null,
  "capabilities": [
    "catalog",
    "state",
    "discover",
    "about",
    "stream-maps",
    "schema-flattening",
    "batch"
  ],
  "settings": {
    "type": "object",
    "properties": {
      "client_id": {
        "type": [
          "string"
        ],
        "description": "The client id to authenticate against the API service",
        "secret": true,
        "writeOnly": true
      },
      "client_secret": {
        "type": [
          "string"
        ],
        "description": "The client secret to authenticate against the API service",
        "secret": true,
        "writeOnly": true
      },
      "org_id": {
        "type": [
          "string"
        ],
        "description": "The organisation id in your apple search ads."
      },
      "start_date": {
        "type": [
          "string",
          "null"
        ],
        "format": "date",
        "description": "Start date for reporting streams, format in YYYY-MM-DD."
      },
      "end_date": {
        "type": [
          "string",
          "null"
        ],
        "format": "date",
        "default": "2024-11-05",
        "description": "End date for reporting streams, format in YYYY-MM-DD."
      },
      "report_granularity": {
        "type": [
          "string",
          "null"
        ],
        "description": "The granularity of reporting streams. One of HOURLY, DAILY, WEEKLY, MONTHLY.",
        "enum": [
          "HOURLY",
          "DAILY",
          "WEEKLY",
          "MONTHLY"
        ]
      },
      "stream_maps": {
        "type": [
          "object",
          "null"
        ],
        "properties": {},
        "description": "Config object for stream maps capability. For more information check out [Stream Maps](https://sdk.meltano.com/en/latest/stream_maps.html)."
      },
      "stream_map_config": {
        "type": [
          "object",
          "null"
        ],
        "properties": {},
        "description": "User-defined config values to be used within map expressions."
      },
      "faker_config": {
        "type": [
          "object",
          "null"
        ],
        "properties": {
          "seed": {
            "oneOf": [
              {
                "type": [
                  "number"
                ]
              },
              {
                "type": [
                  "string"
                ]
              },
              {
                "type": [
                  "boolean"
                ]
              },
              {
                "type": "null"
              }
            ],
            "description": "Value to seed the Faker generator for deterministic output: https://faker.readthedocs.io/en/master/#seeding-the-generator"
          },
          "locale": {
            "oneOf": [
              {
                "type": [
                  "string"
                ]
              },
              {
                "type": "array",
                "items": {
                  "type": [
                    "string"
                  ]
                }
              },
              {
                "type": "null"
              }
            ],
            "description": "One or more LCID locale strings to produce localized output for: https://faker.readthedocs.io/en/master/#localization"
          }
        },
        "description": "Config for the [`Faker`](https://faker.readthedocs.io/en/master/) instance variable `fake` used within map expressions. Only applicable if the plugin specifies `faker` as an addtional dependency (through the `singer-sdk` `faker` extra or directly)."
      },
      "flattening_enabled": {
        "type": [
          "boolean",
          "null"
        ],
        "description": "'True' to enable schema flattening and automatically expand nested properties."
      },
      "flattening_max_depth": {
        "type": [
          "integer",
          "null"
        ],
        "description": "The max depth to flatten schemas."
      },
      "batch_config": {
        "type": [
          "object",
          "null"
        ],
        "properties": {
          "encoding": {
            "type": [
              "object",
              "null"
            ],
            "properties": {
              "format": {
                "type": [
                  "string",
                  "null"
                ],
                "description": "Format to use for batch files.",
                "enum": [
                  "jsonl",
                  "parquet"
                ]
              },
              "compression": {
                "type": [
                  "string",
                  "null"
                ],
                "description": "Compression format to use for batch files.",
                "enum": [
                  "gzip",
                  "none"
                ]
              }
            },
            "description": "Specifies the format and compression of the batch files."
          },
          "storage": {
            "type": [
              "object",
              "null"
            ],
            "properties": {
              "root": {
                "type": [
                  "string",
                  "null"
                ],
                "description": "Root path to use when writing batch files."
              },
              "prefix": {
                "type": [
                  "string",
                  "null"
                ],
                "description": "Prefix to use when writing batch files."
              }
            },
            "description": "Defines the storage layer to use when writing batch files"
          }
        }
      }
    },
    "required": [
      "client_id",
      "client_secret",
      "org_id"
    ]
  }
}

Discovered streams

campaign_granular_reports
campaign_reports
campaigns

github-actions · 2024-11-05T19:46:36Z

Testing plugin tap-aircall (ticketswap variant):

Starting test job...
...
Job completed.

Auto-generated README.md

`tap-aircall`

Aircall tap class.

Built with the Meltano Singer SDK.

Capabilities

catalog
state
discover
about
stream-maps
schema-flattening
batch

Settings

Setting	Required	Default	Description
api_key	True	None	The key to authenticate against the API service
api_token	True	None	The token to authenticate against the API service
start_date	False	None	The earliest record date to sync
stream_maps	False	None	Config object for stream maps capability. For more information check out Stream Maps.
stream_map_config	False	None	User-defined config values to be used within map expressions.
faker_config	False	None	Config for the `Faker` instance variable `fake` used within map expressions. Only applicable if the plugin specifies `faker` as an addtional dependency (through the `singer-sdk` `faker` extra or directly).
faker_config.seed	False	None	Value to seed the Faker generator for deterministic output: https://faker.readthedocs.io/en/master/#seeding-the-generator
faker_config.locale	False	None	One or more LCID locale strings to produce localized output for: https://faker.readthedocs.io/en/master/#localization
flattening_enabled	False	None	'True' to enable schema flattening and automatically expand nested properties.
flattening_max_depth	False	None	The max depth to flatten schemas.
batch_config	False	None
batch_config.encoding	False	None	Specifies the format and compression of the batch files.
batch_config.encoding.format	False	None	Format to use for batch files.
batch_config.encoding.compression	False	None	Compression format to use for batch files.
batch_config.storage	False	None	Defines the storage layer to use when writing batch files
batch_config.storage.root	False	None	Root path to use when writing batch files.
batch_config.storage.prefix	False	None	Prefix to use when writing batch files.

A full list of supported settings and capabilities is available by running: tap-aircall --about

Version info

tap-aircall v[could not be detected], Meltano SDK v0.36.1

Usage info

melty-bot % tap-aircall --help
Usage: tap-aircall [OPTIONS]

  Execute the Singer tap.

Options:
  --version                 Display the package version.
  --about                   Display package metadata and settings.
  --format [json|markdown]  Specify output style for --about
  --config TEXT             Configuration file location or 'ENV' to use
                            environment variables.
  --discover                Run the tap in discovery mode.
  --test TEXT               Use --test to sync a single record for each
                            stream. Use --test=schema to test schema output
                            without syncing records.
  --catalog PATH            Use a Singer catalog file with the tap.
  --state PATH              Use a bookmarks file for incremental replication.
  --help                    Show this message and exit.

Detected capabilities

✅ 'discover'

✅ 'catalog'

✅ 'state'

✅ 'about'

JSON Metadata

{
  "name": "tap-aircall",
  "description": "Aircall tap class.",
  "version": "[could not be detected]",
  "sdk_version": "0.36.1",
  "supported_python_versions": null,
  "capabilities": [
    "catalog",
    "state",
    "discover",
    "about",
    "stream-maps",
    "schema-flattening",
    "batch"
  ],
  "settings": {
    "type": "object",
    "properties": {
      "api_key": {
        "type": [
          "string"
        ],
        "description": "The key to authenticate against the API service",
        "secret": true,
        "writeOnly": true
      },
      "api_token": {
        "type": [
          "string"
        ],
        "description": "The token to authenticate against the API service",
        "secret": true,
        "writeOnly": true
      },
      "start_date": {
        "type": [
          "string",
          "null"
        ],
        "format": "date-time",
        "description": "The earliest record date to sync"
      },
      "stream_maps": {
        "type": [
          "object",
          "null"
        ],
        "properties": {},
        "description": "Config object for stream maps capability. For more information check out [Stream Maps](https://sdk.meltano.com/en/latest/stream_maps.html)."
      },
      "stream_map_config": {
        "type": [
          "object",
          "null"
        ],
        "properties": {},
        "description": "User-defined config values to be used within map expressions."
      },
      "faker_config": {
        "type": [
          "object",
          "null"
        ],
        "properties": {
          "seed": {
            "oneOf": [
              {
                "type": [
                  "number"
                ]
              },
              {
                "type": [
                  "string"
                ]
              },
              {
                "type": [
                  "boolean"
                ]
              },
              {
                "type": "null"
              }
            ],
            "description": "Value to seed the Faker generator for deterministic output: https://faker.readthedocs.io/en/master/#seeding-the-generator"
          },
          "locale": {
            "oneOf": [
              {
                "type": [
                  "string"
                ]
              },
              {
                "type": "array",
                "items": {
                  "type": [
                    "string"
                  ]
                }
              },
              {
                "type": "null"
              }
            ],
            "description": "One or more LCID locale strings to produce localized output for: https://faker.readthedocs.io/en/master/#localization"
          }
        },
        "description": "Config for the [`Faker`](https://faker.readthedocs.io/en/master/) instance variable `fake` used within map expressions. Only applicable if the plugin specifies `faker` as an addtional dependency (through the `singer-sdk` `faker` extra or directly)."
      },
      "flattening_enabled": {
        "type": [
          "boolean",
          "null"
        ],
        "description": "'True' to enable schema flattening and automatically expand nested properties."
      },
      "flattening_max_depth": {
        "type": [
          "integer",
          "null"
        ],
        "description": "The max depth to flatten schemas."
      },
      "batch_config": {
        "type": [
          "object",
          "null"
        ],
        "properties": {
          "encoding": {
            "type": [
              "object",
              "null"
            ],
            "properties": {
              "format": {
                "type": [
                  "string",
                  "null"
                ],
                "description": "Format to use for batch files.",
                "enum": [
                  "jsonl",
                  "parquet"
                ]
              },
              "compression": {
                "type": [
                  "string",
                  "null"
                ],
                "description": "Compression format to use for batch files.",
                "enum": [
                  "gzip",
                  "none"
                ]
              }
            },
            "description": "Specifies the format and compression of the batch files."
          },
          "storage": {
            "type": [
              "object",
              "null"
            ],
            "properties": {
              "root": {
                "type": [
                  "string",
                  "null"
                ],
                "description": "Root path to use when writing batch files."
              },
              "prefix": {
                "type": [
                  "string",
                  "null"
                ],
                "description": "Prefix to use when writing batch files."
              }
            },
            "description": "Defines the storage layer to use when writing batch files"
          }
        }
      }
    },
    "required": [
      "api_key",
      "api_token"
    ]
  }
}

Discovered streams

calls
contacts
numbers
tags
users

github-actions · 2024-11-05T19:46:40Z

Testing plugin tap-dbt (meltanolabs variant):

Starting test job...
...
Job completed.

Auto-generated README.md

`tap-dbt`

Singer tap for the dbt Cloud API.

Built with the Meltano Singer SDK.

Capabilities

catalog
state
discover
about
stream-maps
schema-flattening
batch

Supported Python Versions

3.8
3.9
3.10
3.11
3.12

Settings

Setting	Required	Default	Description
api_key	True	None	API key for the dbt Cloud API
account_ids	True	None	dbt Cloud account IDs
base_url	False	https://cloud.getdbt.com/api/v2	Base URL for the dbt Cloud API
user_agent	False	tap-dbt/0.12.3.post1 Singer tap for the dbt Cloud API.	User-Agent to make requests with
page_size	True	5000	Page size to use in limit= url parameter
stream_maps	False	None	Config object for stream maps capability. For more information check out Stream Maps.
stream_map_config	False	None	User-defined config values to be used within map expressions.
faker_config	False	None	Config for the `Faker` instance variable `fake` used within map expressions. Only applicable if the plugin specifies `faker` as an addtional dependency (through the `singer-sdk` `faker` extra or directly).
faker_config.seed	False	None	Value to seed the Faker generator for deterministic output: https://faker.readthedocs.io/en/master/#seeding-the-generator
faker_config.locale	False	None	One or more LCID locale strings to produce localized output for: https://faker.readthedocs.io/en/master/#localization
flattening_enabled	False	None	'True' to enable schema flattening and automatically expand nested properties.
flattening_max_depth	False	None	The max depth to flatten schemas.
batch_config	False	None
batch_config.encoding	False	None	Specifies the format and compression of the batch files.
batch_config.encoding.format	False	None	Format to use for batch files.
batch_config.encoding.compression	False	None	Compression format to use for batch files.
batch_config.storage	False	None	Defines the storage layer to use when writing batch files
batch_config.storage.root	False	None	Root path to use when writing batch files.
batch_config.storage.prefix	False	None	Prefix to use when writing batch files.

A full list of supported settings and capabilities is available by running: tap-dbt --about

Version info

tap-dbt v0.12.3.post1, Meltano SDK v0.40.0

Usage info

melty-bot % tap-dbt --help
Usage: tap-dbt [OPTIONS]

  Execute the Singer tap.

Options:
  --version                 Display the package version.
  --about                   Display package metadata and settings.
  --format [json|markdown]  Specify output style for --about
  --config TEXT             Configuration file location or 'ENV' to use
                            environment variables.
  --discover                Run the tap in discovery mode.
  --test TEXT               Use --test to sync a single record for each
                            stream. Use --test=schema to test schema output
                            without syncing records.
  --catalog PATH            Use a Singer catalog file with the tap.
  --state PATH              Use a bookmarks file for incremental replication.
  --help                    Show this message and exit.

Detected capabilities

✅ 'discover'

✅ 'catalog'

✅ 'state'

✅ 'about'

JSON Metadata

{
  "name": "tap-dbt",
  "description": "Singer tap for the dbt Cloud API.",
  "version": "0.12.3.post1",
  "sdk_version": "0.40.0",
  "supported_python_versions": [
    "3.8",
    "3.9",
    "3.10",
    "3.11",
    "3.12"
  ],
  "capabilities": [
    "catalog",
    "state",
    "discover",
    "about",
    "stream-maps",
    "schema-flattening",
    "batch"
  ],
  "settings": {
    "type": "object",
    "properties": {
      "api_key": {
        "type": [
          "string"
        ],
        "description": "API key for the dbt Cloud API"
      },
      "account_ids": {
        "type": "array",
        "items": {
          "type": [
            "string"
          ]
        },
        "description": "dbt Cloud account IDs"
      },
      "base_url": {
        "type": [
          "string",
          "null"
        ],
        "default": "https://cloud.getdbt.com/api/v2",
        "description": "Base URL for the dbt Cloud API"
      },
      "user_agent": {
        "type": [
          "string",
          "null"
        ],
        "default": "tap-dbt/0.12.3.post1 Singer tap for the dbt Cloud API.",
        "description": "User-Agent to make requests with"
      },
      "page_size": {
        "type": [
          "integer"
        ],
        "default": 5000,
        "description": "Page size to use in limit= url parameter"
      },
      "stream_maps": {
        "type": [
          "object",
          "null"
        ],
        "properties": {},
        "description": "Config object for stream maps capability. For more information check out [Stream Maps](https://sdk.meltano.com/en/latest/stream_maps.html)."
      },
      "stream_map_config": {
        "type": [
          "object",
          "null"
        ],
        "properties": {},
        "description": "User-defined config values to be used within map expressions."
      },
      "faker_config": {
        "type": [
          "object",
          "null"
        ],
        "properties": {
          "seed": {
            "oneOf": [
              {
                "type": [
                  "number"
                ]
              },
              {
                "type": [
                  "string"
                ]
              },
              {
                "type": [
                  "boolean"
                ]
              },
              {
                "type": "null"
              }
            ],
            "description": "Value to seed the Faker generator for deterministic output: https://faker.readthedocs.io/en/master/#seeding-the-generator"
          },
          "locale": {
            "oneOf": [
              {
                "type": [
                  "string"
                ]
              },
              {
                "type": "array",
                "items": {
                  "type": [
                    "string"
                  ]
                }
              },
              {
                "type": "null"
              }
            ],
            "description": "One or more LCID locale strings to produce localized output for: https://faker.readthedocs.io/en/master/#localization"
          }
        },
        "description": "Config for the [`Faker`](https://faker.readthedocs.io/en/master/) instance variable `fake` used within map expressions. Only applicable if the plugin specifies `faker` as an addtional dependency (through the `singer-sdk` `faker` extra or directly)."
      },
      "flattening_enabled": {
        "type": [
          "boolean",
          "null"
        ],
        "description": "'True' to enable schema flattening and automatically expand nested properties."
      },
      "flattening_max_depth": {
        "type": [
          "integer",
          "null"
        ],
        "description": "The max depth to flatten schemas."
      },
      "batch_config": {
        "type": [
          "object",
          "null"
        ],
        "properties": {
          "encoding": {
            "type": [
              "object",
              "null"
            ],
            "properties": {
              "format": {
                "type": [
                  "string",
                  "null"
                ],
                "description": "Format to use for batch files.",
                "enum": [
                  "jsonl",
                  "parquet"
                ]
              },
              "compression": {
                "type": [
                  "string",
                  "null"
                ],
                "description": "Compression format to use for batch files.",
                "enum": [
                  "gzip",
                  "none"
                ]
              }
            },
            "description": "Specifies the format and compression of the batch files."
          },
          "storage": {
            "type": [
              "object",
              "null"
            ],
            "properties": {
              "root": {
                "type": [
                  "string",
                  "null"
                ],
                "description": "Root path to use when writing batch files."
              },
              "prefix": {
                "type": [
                  "string",
                  "null"
                ],
                "description": "Prefix to use when writing batch files."
              }
            },
            "description": "Defines the storage layer to use when writing batch files"
          }
        }
      }
    },
    "required": [
      "api_key",
      "account_ids",
      "page_size"
    ]
  }
}

Discovered streams

accounts
connections
environments
jobs
projects
repositories
runs
users

github-actions · 2024-11-05T19:46:40Z

Testing plugin tap-circle-ci (meltanolabs variant):

Starting test job...
...
Job completed.

Auto-generated README.md

`tap-circle-ci`

Singer tap for the CircleCI API.

Built with the Meltano Singer SDK.

Capabilities

catalog
state
discover
about
stream-maps
schema-flattening
batch

Supported Python Versions

3.8
3.9
3.10
3.11
3.12

Settings

Setting	Required	Default	Description
token	True	None	Personal API Token you have generated that can be used to access the CircleCI API
org_slug	True	None	Organization slug in the form vcs-slug/org-name. Example: org-slug=gh/CircleCI-Public
base_url	False	https://circleci.com/api/v2	The API base URL to use for requests. Default, https://circleci.com/api/v2.
user_agent	False	tap-circle-ci/0.3.3 Singer tap for the CircleCI API.	User-Agent header
stream_maps	False	None	Config object for stream maps capability. For more information check out Stream Maps.
stream_map_config	False	None	User-defined config values to be used within map expressions.
faker_config	False	None	Config for the `Faker` instance variable `fake` used within map expressions. Only applicable if the plugin specifies `faker` as an addtional dependency (through the `singer-sdk` `faker` extra or directly).
faker_config.seed	False	None	Value to seed the Faker generator for deterministic output: https://faker.readthedocs.io/en/master/#seeding-the-generator
faker_config.locale	False	None	One or more LCID locale strings to produce localized output for: https://faker.readthedocs.io/en/master/#localization
flattening_enabled	False	None	'True' to enable schema flattening and automatically expand nested properties.
flattening_max_depth	False	None	The max depth to flatten schemas.
batch_config	False	None
batch_config.encoding	False	None	Specifies the format and compression of the batch files.
batch_config.encoding.format	False	None	Format to use for batch files.
batch_config.encoding.compression	False	None	Compression format to use for batch files.
batch_config.storage	False	None	Defines the storage layer to use when writing batch files
batch_config.storage.root	False	None	Root path to use when writing batch files.
batch_config.storage.prefix	False	None	Prefix to use when writing batch files.

A full list of supported settings and capabilities is available by running: tap-circle-ci --about

Version info

tap-circle-ci v0.3.3, Meltano SDK v0.41.0

Usage info

melty-bot % tap-circle-ci --help
Usage: tap-circle-ci [OPTIONS]

  Execute the Singer tap.

Options:
  --version                 Display the package version.
  --about                   Display package metadata and settings.
  --format [json|markdown]  Specify output style for --about
  --config TEXT             Configuration file location or 'ENV' to use
                            environment variables.
  --discover                Run the tap in discovery mode.
  --test TEXT               Use --test to sync a single record for each
                            stream. Use --test=schema to test schema output
                            without syncing records.
  --catalog PATH            Use a Singer catalog file with the tap.
  --state PATH              Use a bookmarks file for incremental replication.
  --help                    Show this message and exit.

Detected capabilities

✅ 'discover'

✅ 'catalog'

✅ 'state'

✅ 'about'

JSON Metadata

{
  "name": "tap-circle-ci",
  "description": "Singer tap for the CircleCI API.",
  "version": "0.3.3",
  "sdk_version": "0.41.0",
  "supported_python_versions": [
    "3.8",
    "3.9",
    "3.10",
    "3.11",
    "3.12"
  ],
  "capabilities": [
    "catalog",
    "state",
    "discover",
    "about",
    "stream-maps",
    "schema-flattening",
    "batch"
  ],
  "settings": {
    "type": "object",
    "properties": {
      "token": {
        "type": [
          "string"
        ],
        "description": "Personal API Token you have generated that can be used to access the CircleCI API"
      },
      "org_slug": {
        "type": [
          "string"
        ],
        "description": "Organization slug in the form vcs-slug/org-name. Example: org-slug=gh/CircleCI-Public"
      },
      "base_url": {
        "type": [
          "string",
          "null"
        ],
        "default": "https://circleci.com/api/v2",
        "description": "The API base URL to use for requests. Default, https://circleci.com/api/v2."
      },
      "user_agent": {
        "type": [
          "string",
          "null"
        ],
        "default": "tap-circle-ci/0.3.3 Singer tap for the CircleCI API.",
        "description": "User-Agent header"
      },
      "stream_maps": {
        "type": [
          "object",
          "null"
        ],
        "properties": {},
        "description": "Config object for stream maps capability. For more information check out [Stream Maps](https://sdk.meltano.com/en/latest/stream_maps.html)."
      },
      "stream_map_config": {
        "type": [
          "object",
          "null"
        ],
        "properties": {},
        "description": "User-defined config values to be used within map expressions."
      },
      "faker_config": {
        "type": [
          "object",
          "null"
        ],
        "properties": {
          "seed": {
            "oneOf": [
              {
                "type": [
                  "number"
                ]
              },
              {
                "type": [
                  "string"
                ]
              },
              {
                "type": [
                  "boolean"
                ]
              },
              {
                "type": "null"
              }
            ],
            "description": "Value to seed the Faker generator for deterministic output: https://faker.readthedocs.io/en/master/#seeding-the-generator"
          },
          "locale": {
            "oneOf": [
              {
                "type": [
                  "string"
                ]
              },
              {
                "type": "array",
                "items": {
                  "type": [
                    "string"
                  ]
                }
              },
              {
                "type": "null"
              }
            ],
            "description": "One or more LCID locale strings to produce localized output for: https://faker.readthedocs.io/en/master/#localization"
          }
        },
        "description": "Config for the [`Faker`](https://faker.readthedocs.io/en/master/) instance variable `fake` used within map expressions. Only applicable if the plugin specifies `faker` as an addtional dependency (through the `singer-sdk` `faker` extra or directly)."
      },
      "flattening_enabled": {
        "type": [
          "boolean",
          "null"
        ],
        "description": "'True' to enable schema flattening and automatically expand nested properties."
      },
      "flattening_max_depth": {
        "type": [
          "integer",
          "null"
        ],
        "description": "The max depth to flatten schemas."
      },
      "batch_config": {
        "type": [
          "object",
          "null"
        ],
        "properties": {
          "encoding": {
            "type": [
              "object",
              "null"
            ],
            "properties": {
              "format": {
                "type": [
                  "string",
                  "null"
                ],
                "description": "Format to use for batch files.",
                "enum": [
                  "jsonl",
                  "parquet"
                ]
              },
              "compression": {
                "type": [
                  "string",
                  "null"
                ],
                "description": "Compression format to use for batch files.",
                "enum": [
                  "gzip",
                  "none"
                ]
              }
            },
            "description": "Specifies the format and compression of the batch files."
          },
          "storage": {
            "type": [
              "object",
              "null"
            ],
            "properties": {
              "root": {
                "type": [
                  "string",
                  "null"
                ],
                "description": "Root path to use when writing batch files."
              },
              "prefix": {
                "type": [
                  "string",
                  "null"
                ],
                "description": "Prefix to use when writing batch files."
              }
            },
            "description": "Defines the storage layer to use when writing batch files"
          }
        }
      }
    },
    "required": [
      "token",
      "org_slug"
    ]
  }
}

Discovered streams

jobs
pipelines
workflows

github-actions · 2024-11-05T19:46:49Z

Testing plugin tap-mysql (meltanolabs variant):

Starting test job...
...
Job completed.

Auto-generated README.md

`tap-mysql`

Singer tap for MySQL.

Built with the Meltano Singer SDK.

Capabilities

catalog
state
discover
about
stream-maps
schema-flattening
batch

Settings

Setting	Required	Default	Description
host	False	None	Hostname for mysql instance. Note if sqlalchemy_url is set this will be ignored.
port	False	3306	The port on which mysql is awaiting connection. Note if sqlalchemy_url is set this will be ignored.
user	False	None	User name used to authenticate. Note if sqlalchemy_url is set this will be ignored.
password	False	None	Password used to authenticate. Note if sqlalchemy_url is set this will be ignored.
database	False	None	Database name. Note if sqlalchemy_url is set this will be ignored.
sqlalchemy_options	False	None	sqlalchemy_url options (also called the query), to connect to PlanetScale you must turn on SSL see PlanetScale information below. Note if sqlalchemy_url is set this will be ignored.
sqlalchemy_url	False	None	Example pymysql://[username]:[password]@localhost:3306/[db_name][?options] see https://docs.sqlalchemy.org/en/20/dialects/mysql.html#module-sqlalchemy.dialects.mysql.pymysql for more information
filter_schemas	False	None	If an array of schema names is provided, the tap will only process the specified MySQL schemas and ignore others. If left blank, the tap automatically determines ALL available MySQL schemas.
is_vitess	False	None	By default we'll check if the database is a Vitess instance. If you would rather not automatically check, set this to `False`. See Vitess/PlanetScale documentation below for more information.
ssh_tunnel	False	None	SSH Tunnel Configuration, this is a json object
stream_maps	False	None	Config object for stream maps capability. For more information check out Stream Maps.
stream_map_config	False	None	User-defined config values to be used within map expressions.
faker_config	False	None	Config for the `Faker` instance variable `fake` used within map expressions. Only applicable if the plugin specifies `faker` as an addtional dependency (through the `singer-sdk` `faker` extra or directly).
flattening_enabled	False	None	'True' to enable schema flattening and automatically expand nested properties.
flattening_max_depth	False	None	The max depth to flatten schemas.
batch_config	False	None

A full list of supported settings and capabilities is available by running: tap-mysql --about

Version info

tap-mysql v[could not be detected], Meltano SDK v0.35.2

Usage info

melty-bot % tap-mysql --help
Usage: tap-mysql [OPTIONS]

  Execute the Singer tap.

Options:
  --version                 Display the package version.
  --about                   Display package metadata and settings.
  --format [json|markdown]  Specify output style for --about
  --config TEXT             Configuration file location or 'ENV' to use
                            environment variables.
  --discover                Run the tap in discovery mode.
  --test TEXT               Use --test to sync a single record for each
                            stream. Use --test=schema to test schema output
                            without syncing records.
  --catalog PATH            Use a Singer catalog file with the tap.
  --state PATH              Use a bookmarks file for incremental replication.
  --help                    Show this message and exit.

Detected capabilities

✅ 'discover'

✅ 'catalog'

✅ 'state'

✅ 'about'

JSON Metadata

{
  "name": "tap-mysql",
  "description": "Singer tap for MySQL.",
  "version": "[could not be detected]",
  "sdk_version": "0.35.2",
  "supported_python_versions": null,
  "capabilities": [
    "catalog",
    "state",
    "discover",
    "about",
    "stream-maps",
    "schema-flattening",
    "batch"
  ],
  "settings": {
    "type": "object",
    "properties": {
      "host": {
        "type": [
          "string",
          "null"
        ],
        "description": "Hostname for mysql instance. Note if sqlalchemy_url is set this will be ignored."
      },
      "port": {
        "type": [
          "integer",
          "null"
        ],
        "default": 3306,
        "description": "The port on which mysql is awaiting connection. Note if sqlalchemy_url is set this will be ignored."
      },
      "user": {
        "type": [
          "string",
          "null"
        ],
        "description": "User name used to authenticate. Note if sqlalchemy_url is set this will be ignored."
      },
      "password": {
        "type": [
          "string",
          "null"
        ],
        "description": "Password used to authenticate. Note if sqlalchemy_url is set this will be ignored.",
        "secret": true,
        "writeOnly": true
      },
      "database": {
        "type": [
          "string",
          "null"
        ],
        "description": "Database name. Note if sqlalchemy_url is set this will be ignored."
      },
      "sqlalchemy_options": {
        "type": [
          "object",
          "null"
        ],
        "properties": {},
        "additionalProperties": {
          "type": [
            "string"
          ]
        },
        "description": "sqlalchemy_url options (also called the query), to connect to PlanetScale you must turn on SSL see PlanetScale information below. Note if sqlalchemy_url is set this will be ignored."
      },
      "sqlalchemy_url": {
        "type": [
          "string",
          "null"
        ],
        "description": "Example pymysql://[username]:[password]@localhost:3306/[db_name][?options] see https://docs.sqlalchemy.org/en/20/dialects/mysql.html#module-sqlalchemy.dialects.mysql.pymysql for more information",
        "secret": true,
        "writeOnly": true
      },
      "filter_schemas": {
        "type": [
          "array",
          "null"
        ],
        "items": {
          "type": [
            "string"
          ]
        },
        "description": "If an array of schema names is provided, the tap will only process the specified MySQL schemas and ignore others. If left blank, the tap automatically determines ALL available MySQL schemas."
      },
      "is_vitess": {
        "type": [
          "boolean",
          "null"
        ],
        "description": "By default we'll check if the database is a Vitess instance. If you would rather not automatically check, set this to `False`. See Vitess/PlanetScale documentation below for more information."
      },
      "ssh_tunnel": {
        "type": [
          "object",
          "null"
        ],
        "properties": {
          "enable": {
            "type": [
              "boolean"
            ],
            "default": false,
            "description": "Enable an ssh tunnel (also known as bastion host), see the other ssh_tunnel.* properties for more details"
          },
          "host": {
            "type": [
              "string"
            ],
            "description": "Host of the bastion host, this is the host we'll connect to via ssh"
          },
          "username": {
            "type": [
              "string"
            ],
            "description": "Username to connect to bastion host"
          },
          "port": {
            "type": [
              "integer"
            ],
            "default": 22,
            "description": "Port to connect to bastion host"
          },
          "private_key": {
            "type": [
              "string"
            ],
            "description": "Private Key for authentication to the bastion host",
            "secret": true,
            "writeOnly": true
          },
          "private_key_password": {
            "type": [
              "string",
              "null"
            ],
            "description": "Private Key Password, leave None if no password is set",
            "secret": true,
            "writeOnly": true
          }
        },
        "required": [
          "enable",
          "host",
          "username",
          "port",
          "private_key"
        ],
        "description": "SSH Tunnel Configuration, this is a json object"
      },
      "stream_maps": {
        "type": [
          "object",
          "null"
        ],
        "properties": {},
        "description": "Config object for stream maps capability. For more information check out [Stream Maps](https://sdk.meltano.com/en/latest/stream_maps.html)."
      },
      "stream_map_config": {
        "type": [
          "object",
          "null"
        ],
        "properties": {},
        "description": "User-defined config values to be used within map expressions."
      },
      "faker_config": {
        "type": [
          "object",
          "null"
        ],
        "properties": {
          "seed": {
            "oneOf": [
              {
                "type": [
                  "number"
                ]
              },
              {
                "type": [
                  "string"
                ]
              },
              {
                "type": [
                  "boolean"
                ]
              },
              {
                "type": "null"
              }
            ],
            "description": "Value to seed the Faker generator for deterministic output: https://faker.readthedocs.io/en/master/#seeding-the-generator"
          },
          "locale": {
            "oneOf": [
              {
                "type": [
                  "string"
                ]
              },
              {
                "type": "array",
                "items": {
                  "type": [
                    "string"
                  ]
                }
              },
              {
                "type": "null"
              }
            ],
            "description": "One or more LCID locale strings to produce localized output for: https://faker.readthedocs.io/en/master/#localization"
          }
        },
        "description": "Config for the [`Faker`](https://faker.readthedocs.io/en/master/) instance variable `fake` used within map expressions. Only applicable if the plugin specifies `faker` as an addtional dependency (through the `singer-sdk` `faker` extra or directly)."
      },
      "flattening_enabled": {
        "type": [
          "boolean",
          "null"
        ],
        "description": "'True' to enable schema flattening and automatically expand nested properties."
      },
      "flattening_max_depth": {
        "type": [
          "integer",
          "null"
        ],
        "description": "The max depth to flatten schemas."
      },
      "batch_config": {
        "type": [
          "object",
          "null"
        ],
        "properties": {
          "encoding": {
            "type": [
              "object",
              "null"
            ],
            "properties": {
              "format": {
                "type": [
                  "string",
                  "null"
                ],
                "description": "Format to use for batch files.",
                "enum": [
                  "jsonl",
                  "parquet"
                ]
              },
              "compression": {
                "type": [
                  "string",
                  "null"
                ],
                "description": "Compression format to use for batch files.",
                "enum": [
                  "gzip",
                  "none"
                ]
              }
            },
            "description": "Specifies the format and compression of the batch files."
          },
          "storage": {
            "type": [
              "object",
              "null"
            ],
            "properties": {
              "root": {
                "type": [
                  "string",
                  "null"
                ],
                "description": "Root path to use when writing batch files."
              },
              "prefix": {
                "type": [
                  "string",
                  "null"
                ],
                "description": "Prefix to use when writing batch files."
              }
            },
            "description": "Defines the storage layer to use when writing batch files"
          }
        }
      }
    }
  }
}

edgarrmondragon temporarily deployed to preview November 5, 2024 18:02 — with GitHub Actions Inactive

edgarrmondragon mentioned this pull request Nov 5, 2024

feat: Allow passing a custom Python exectuable to extract-sdk-metadata-to-s3 meltano/hub-utils#190

Merged

edgarrmondragon temporarily deployed to preview November 5, 2024 18:03 — with GitHub Actions Inactive

edgarrmondragon had a problem deploying to production November 5, 2024 18:04 — with GitHub Actions Failure

edgarrmondragon temporarily deployed to preview November 5, 2024 19:29 — with GitHub Actions Inactive

edgarrmondragon had a problem deploying to preview November 5, 2024 19:29 — with GitHub Actions Failure

edgarrmondragon temporarily deployed to preview November 5, 2024 19:29 — with GitHub Actions Inactive

edgarrmondragon temporarily deployed to preview November 5, 2024 19:36 — with GitHub Actions Inactive

ci: Use the correct Python executables for extracting plugin metadata

d2800c2

edgarrmondragon force-pushed the edgarrmondragon/fix/use-correct-python-executables branch from 380d646 to d2800c2 Compare November 5, 2024 19:45

edgarrmondragon temporarily deployed to preview November 5, 2024 19:45 — with GitHub Actions Inactive

chore: [Hub Bot] Refresh metadata 2024-11-05 (#1884)

b4d6fcb

Co-authored-by: Edgar Ramírez Mondragón <[email protected]> Co-authored-by: edgarrmondragon <[email protected]> Co-authored-by: Edgar Ramírez Mondragón <[email protected]>

edgarrmondragon temporarily deployed to preview November 5, 2024 19:46 — with GitHub Actions Inactive

edgarrmondragon marked this pull request as ready for review November 5, 2024 19:46

edgarrmondragon requested review from tayloramurphy, pnadolny13 and cjohnhanson as code owners November 5, 2024 19:46

edgarrmondragon temporarily deployed to preview November 5, 2024 19:46 — with GitHub Actions Inactive

edgarrmondragon merged commit e6ce9fe into main Nov 5, 2024
18 checks passed

edgarrmondragon deleted the edgarrmondragon/fix/use-correct-python-executables branch November 5, 2024 19:48

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ci: Use the correct Python executables for extracting plugin metadata #1883

ci: Use the correct Python executables for extracting plugin metadata #1883

edgarrmondragon commented Nov 5, 2024 •

edited

Loading

netlify bot commented Nov 5, 2024 •

edited

Loading

github-actions bot commented Nov 5, 2024 •

edited

Loading

`tap-apple-search-ads`

Capabilities

Settings

github-actions bot commented Nov 5, 2024 •

edited

Loading

`tap-aircall`

Capabilities

Settings

github-actions bot commented Nov 5, 2024 •

edited

Loading

`tap-dbt`

Capabilities

Supported Python Versions

Settings

github-actions bot commented Nov 5, 2024 •

edited

Loading

`tap-circle-ci`

Capabilities

Supported Python Versions

Settings

github-actions bot commented Nov 5, 2024 •

edited

Loading

`tap-mysql`

Capabilities

Settings

ci: Use the correct Python executables for extracting plugin metadata #1883

ci: Use the correct Python executables for extracting plugin metadata #1883

Conversation

edgarrmondragon commented Nov 5, 2024 • edited Loading

netlify bot commented Nov 5, 2024 • edited Loading

✅ Deploy Preview for meltano-hub ready!

github-actions bot commented Nov 5, 2024 • edited Loading

tap-apple-search-ads

Capabilities

Settings

github-actions bot commented Nov 5, 2024 • edited Loading

tap-aircall

Capabilities

Settings

github-actions bot commented Nov 5, 2024 • edited Loading

tap-dbt

Capabilities

Supported Python Versions

Settings

github-actions bot commented Nov 5, 2024 • edited Loading

tap-circle-ci

Capabilities

Supported Python Versions

Settings

github-actions bot commented Nov 5, 2024 • edited Loading

tap-mysql

Capabilities

Settings

edgarrmondragon commented Nov 5, 2024 •

edited

Loading

netlify bot commented Nov 5, 2024 •

edited

Loading

github-actions bot commented Nov 5, 2024 •

edited

Loading

`tap-apple-search-ads`

github-actions bot commented Nov 5, 2024 •

edited

Loading

`tap-aircall`

github-actions bot commented Nov 5, 2024 •

edited

Loading

`tap-dbt`

github-actions bot commented Nov 5, 2024 •

edited

Loading

`tap-circle-ci`

github-actions bot commented Nov 5, 2024 •

edited

Loading

`tap-mysql`