Skip to content

Enable the use of SQLite data sources from remote locations #39

Open
@d33bs

Description

@d33bs

What happens?

Hello - thank you so much for the work on sqlite_scanner, it's amazing to use this through DuckDB!

Would it be possible to use DuckDB to query SQLite files from remote locations? For example, it'd be stellar to query a SQLite file stored on AWS S3 using DuckDB queries. It seems like this could involve efforts from the httpfs extension. That said, please don't hesitate to let me know if it'd be more appropriate to add an issue elsewhere!

To Reproduce

Below is a Pythonic demonstration of what one might be able to do if this issue were addressed (building on the example sqlite_scan queries within the docs):

import duckdb

duckdb.connect().execute(
    """
    /* install and load sqlite plugin for duckdb */
    INSTALL sqlite_scanner;
    LOAD sqlite_scanner;
    
    /* perform query on film table */
    SELECT * FROM sqlite_scan('s3://bucket-name/sakila.db', 'film');
    """
)

OS:

MacOS

SQLite Version:

3.37.0

DuckDB Version:

0.7.1

DuckDB Client:

Python

Full Name:

Dave Bunten

Affiliation:

University of Colorado

Have you tried this on the latest master branch?

  • I agree

Have you tried the steps to reproduce? Do they include all relevant data and configuration? Does the issue you report still appear there?

  • I agree

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions