Skip to content
Anthony Virtuoso edited this page Nov 26, 2019 · 13 revisions

Amazon Athena Query Federation

The Amazon Athena Query Federation SDK allows you to customize Amazon Athena with your own code. This enables you to integrate with new data sources, proprietary data formats, or build in new user defined functions. Initially these customizations will be limited to the parts of a query that occur during a TableScan operation but will eventually be expanded to include other parts of the query execution plan using the same easy to understand interface.

We've written integrations with more than 20 databases, storage formats, and live APIs in order to refine this interface and balance flexibility with ease of use. We hope that making this SDK and initial set of connectors Open Source will allow us to continue to improve the experience and performance of Athena Query Federation.

This functionality is currently in Public Preview while customers provide us feedback on usability, ease of using the service or building new connectors. We do not recommend that you use these connectors in production or use this preview to make assumptions about the performance of Athena’s Federation features. As we receive more feedback, we will make improvements to the preview and lift raise limits associated with query/connector performance, APIs, SDKs, and user experience. The best way to understand the performance of Athena Data Source Connectors is to run a benchmark when they become generally available (GA) or review our performance guidance.

Useful Pages

  1. AWS Docs: Athena Query Federation
  2. AWS Docs: Athena UDFs
  3. Quick Start Guide
  4. Available Connectors
  5. Federation Features
  6. How To Build A Connector or UDF
  7. Gathering diagnostic info for support
  8. Frequently Asked Questions
  9. Common Problems
  10. Installation Pre-requisites
  11. Known Limitations & Open Issues
  12. Predicate Pushdown How-To
  13. Java Doc

Helpful Videos

  1. Insert Quick Start Video
  2. Insert Workshop Video
  3. Insert DocumentDB Video
  4. Insert Athena Talk Video
  5. Insert Analytics Talk Video

We Are Hiring!

One of our goals in making this project Open Source was to empower our customers to customize/extend/contribute. If you'd like the opportunity to contribute even more, we have openings for Software Engineers, Managers, and Solutions Architects in Seattle and NYC. Here are just a few of available opportunities to work with our team and shape this and other projects.