🎉 New Source Gmail

[elattma]

Tell us about the new connector you’d like to have

• Which source and which destination?
• Source gmail
• Do you need a specific version of the underlying data source e.g: you specifically need support for an older version of the API or DB?
• No

Describe the context around this new connector

• Why do you need this integration? How does your team intend to use the data? This helps us understand the use case.
• To read data from gmail. We intend to index the text content for later use.
• How often do you want to run syncs?
• Daily or weekly I think is fine.
• If this is an API source connector, which entities/endpoints do you need supported?
• N/A
• If the connector is for a paid service, can we name you as a mutual user when we subscribe for an account? Which company should we name?
• Yes

Describe the alternative you are considering or using

What are you considering doing if you don’t have this integration through Airbyte?
Building out the data pipeline myself.

Are you willing to submit a PR?

Yes!! I'm more than happy to :)

[KrishnaPG]

Came across this while looking for GMail connector for AirByte. Wondering if this has been made available and how to use this functionality.

[natikgadzhi]

I don't think we have this yet.

[aaronsteers]

Confirmed. As of today, I think you would need to create a workaround.

This can be done with IFTTT or with an "autoforward" handling, either on a Gmail filter or on the email box itself. It's pretty hacky, but in the past I've created an auto-forward rule for emails, sending to a Slack or Teams channel - since both of these allow you to get an email address for a given channel. Then Airbyte would (in theory at leasT) be able to pull from those channels.

Not the ideal situation at all, but this can work for some use cases.

I'll "+1" the desire to have a dedicated Gmail source, fwiw. Not a critical connector perhaps, but would be a nice addition - especially for AI/LLM use cases.

[natikgadzhi]

Okay so if Claude is not hallucinating too much (questionable), here's a snippet, just as an example, of how one could hypothetically do Gmail API to fetch the list of emails and their bodies:

import requests
import base64

ACCESS_TOKEN = "YOUR_ACCESS_TOKEN"
HEADERS = {"Authorization": f"Bearer {ACCESS_TOKEN}"}

# List emails
response = requests.get(
    "https://gmail.googleapis.com/gmail/v1/users/me/messages",
    headers=HEADERS,
    params={"labelIds": "INBOX", "maxResults": 10}
)
messages = response.json().get("messages", [])

for message in messages:
    # Get email content
    msg_id = message["id"]
    response = requests.get(
        f"https://gmail.googleapis.com/gmail/v1/users/me/messages/{msg_id}",
        headers=HEADERS,
        params={"format": "full"}
    )
    email_data = response.json()
    
    # Process email body
    if "payload" in email_data and "body" in email_data["payload"]:
        body_data = email_data["payload"]["body"].get("data")
        if body_data:
            decoded_body = base64.urlsafe_b64decode(body_data).decode("utf-8")
            print(f"Email Body: {decoded_body}")
    
    # Process attachments
    if "payload" in email_data and "parts" in email_data["payload"]:
        for part in email_data["payload"]["parts"]:
            if "filename" in part and part["filename"]:
                attachment_id = part["body"]["attachmentId"]
                response = requests.get(
                    f"https://gmail.googleapis.com/gmail/v1/users/me/messages/{msg_id}/attachments/{attachment_id}",
                    headers=HEADERS
                )
                attachment_data = response.json()["data"]
                decoded_attachment = base64.urlsafe_b64decode(attachment_data)
                
                with open(part["filename"], "wb") as f:
                    f.write(decoded_attachment)
                print(f"Saved attachment: {part['filename']}")

How to do this in Airbyte

If one were to build this in Builder, I would suggest:

1. It's possible ;)
2. Make a stream for "message_metadata" and fetch the list of messages there.
3. Make a stream "message_content" and make it a child stream, use "message_metadata" as parent. This should grab you the message content.

There are funny things with how to support threading, and how to fetch multiple labels, and what to do with file attachments, but that would be a decent start.