List plugins to report their status

[iainelder]

Run steampipe plugin list after installing any plugin. It has two use cases:

• Reports configuration errors to the user
• Realize side effects such as generation of information_schema metadata

I made a new GitHub repo called steampipe-csv-init with repros for the described behavior as custom workflows using production version v1.5.2. Below I link to an example workflow run for each use case.

If you need unit tests, please provide guidance.

Report configuration errors

For the repro see run 7 of job report-errors.

If you set up the CSV plugin without a path config:

      - name: Set up Steampipe
        uses: turbot/steampipe-action-setup@v1
        with:
          plugin-connections: |
            connection "csv" {
              plugin = "csv"
            }

Then the new command reports plugin failed to start.

+-----------+---------+-------------+
| Installed | Version | Connections |
+-----------+---------+-------------+
|           |         |             |
+-----------+---------+-------------+
+--------------------------------------------+-------------+------------------------+
| Failed                                     | Connections | Reason                 |
+--------------------------------------------+-------------+------------------------+
| hub.steampipe.io/plugins/turbot/csv@latest | csv         | plugin failed to start |
+--------------------------------------------+-------------+------------------------+

When the you add the correct path config:

      - name: Set up Steampipe
        uses: turbot/steampipe-action-setup@v1
        with:
          plugin-connections: |
            connection "csv" {
              plugin = "csv"
              paths = [ "*.csv" ]
            }

Then the list command reports that the plugin is correctly installed.

+--------------------------------------------+---------+-------------+
| Installed                                  | Version | Connections |
+--------------------------------------------+---------+-------------+
| hub.steampipe.io/plugins/turbot/csv@latest | 1.0.0   | csv         |
+--------------------------------------------+---------+-------------+

A future PR could act on the output of this query to make the workflow fail with an error. Details of why the plugin failed to start may be found in the plugin log file.

Initalizes information_schema metadata

For the repro see run 7 of job init-information-schema.

A Steampipe user may use the information_schema metadata to the state of Steampipe's database in a GitHub workflow.

For example, a user may run this query to show which CSV files Steampipes exposes as tables.

      - name: List tables in CSV schema
        run: steampipe query "SELECT table_name FROM information_schema.tables WHERE table_schema = 'csv';"

The user has placed a CSV called acme.csv.

Before running the new command, the csv metadata is empty.

+------------+
| table_name |
+------------+
+------------+

After running the new command, the csv metadata shows the acme table.

+------------+
| table_name |
+------------+
| acme       |
+------------+

[cbruno10]

Thanks @iainelder for raising this PR! I ran the checks and seems like the dist/ files need updates. Can you please run npm run prepare (I believe this is the command) to re-build those files?

@pskrbasu For the 2 goals @iainelder raised above, is steampipe plugin list the best command to run to accomplish both?

[iainelder]

seems like the dist/ files need updates. Can you please run npm run prepare (I believe this is the command) to re-build those files?

Done.

[pskrbasu]

@iainelder @cbruno10

Reports configuration errors to the user

Running steampipe plugin list after configuring the connections is a good way to catch failures. I agree with that.

Realize side effects such as generation of information_schema metadata

Running steampipe plugin list in this case does seem to solve the issue(this is merely a workaround), but it looks like a bug in steampipe that the information_schema returns nothing in the first place. Weirdly, the information_schema does not have the newly created connection details yet, but the connection is ready to be used/queried.
I have raised turbot/steampipe#4456 to track and fix this issue.

[iainelder]

@pskrbasu , thank you for the feedback, and thank you for raising the information_schema issue with the core Steampipe project.

What are the next steps for this PR?

[cbruno10]

@iainelder Please see requested changes, thanks!

[iainelder]

@cbruno10 commented in a resolved thread:

Also, could we combine the lines in this if statement with the one above, since they check for the same thing?

I was about to argue for keeping the if blocks separate, even though they check the same condition, because in theory the actions of each branch are independent. You don't need to install before list, and you don't need to list before install.

They are not really independent actions, though. In practice there is no reason to list the plugins when none are installed.

You've persuaded me to fuse the blocks together.

[iainelder]

@cbruno10 , I made changes to address all your feedback. How does it look now?

[cbruno10]

Thanks @iainelder for the quick changes! I'm out of office for the rest of this week, but will merge in this change and release a new action version early next week.

[cbruno10]

Thanks again @iainelder !