How To Deploy A Connector

Athena Federated Queries require that you have a source specific connector deployed as a Lambda function in your account. These connectors can either be custom connectors you've written using the Athena Query Federation SDK, or they can be a community built connector. The Amazon Athena team has built and published >10 connectors to AWS Serverless Application Repository. In this tutorial we will learn how to deploy a pre-built connector using Serverless Application Repository without the need to write any code.

The following instructions will walk you through connecting Athena to Cloudwatch Logs and then running an Athena query to search through a Log Group in Cloudwatch Logs.

All instructions should be carried out in us-east-1 while these features are in 'preview'.

1. Navigate to the S3 Console and create a new bucket in us-east-1. We will use this bucket when configuring our connector.
2. Navigate to Serverless Application Repository.
3. Click on 'Available Applications' in the left hand navigation area.
4. Check the box labeled "Show apps that create custom IAM Roles or resource policies". We do this because Amazon Athena connectors are encouraged to create scoped down IAM Roles that limit which resources the connector can interact with.
5. Search for "AthenaCloudwatchConnector" (no quotes). If you are doing this as part of a workshop, we are working with the Serverless Application Repository team to correct an issue which incorrectly identifies the sudden, very similar searches, as a DDoS and results in a "Too Many Requests" error. If you encounter this error, wait a moment and retry your search or use this direct link to the AthenaCloudwatchConnector
6. Complete the deployment configuration by specifying:
   • a spill bucket that our connector can use as scratch space while processing a query. (use the S3 bucket you created earlier)
   • a catalog name (all lowercase) that will be used as the name of the lambda function the connector deploys. We'll also use this name in the SQL when querying our connector.
   • fill in any other missing fields.
7. Check the box acknowledging that deploying the connector will create custom IAM roles and then click Deploy. The deployment will take 2 - 3 minutes.
8. Once your connector has successfully deployed, you can navigate to the Amazon Athena console in us-east-1 and create a new workgroup by clicking on the 'Workgroup' tab above the query editor. You need to create a workgroup called "AmazonAthenaPreviewFunctionality" (no quotes). This is a special workgroup name which is used to enable Athena preview functionality such as federation, UDFs, and Athena ML.
9. Be sure to switch to your new workgroup if you haven't already. You can do this by clicking on the 'Workgroup" tab above the Athena query editor.
10. You are now ready to run your first federated query. You can use the below query to get all the logs from a specific Log Group in Cloudwatch. In this particular example we are querying the logs generated by our connector. Be sure to replace <CATALOG_NAME> with the name of your Lambda function.

select * from "lambda:<CATALOG_NAME>"."/aws/lambda/<CATALOG_NAME>".all_log_streams limit 100