Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PHPORM-186 GridFS adapter for Filesystem #2985

Merged
merged 8 commits into from
Jun 4, 2024
Merged
Show file tree
Hide file tree
Changes from 6 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,10 @@
# Changelog
All notable changes to this project will be documented in this file.

## [4.5.0] - upcoming

* Add GridFS integration for Laravel File Storage by @GromNaN in [#2984](https://github.com/mongodb/laravel-mongodb/pull/2985)

## [4.4.0] - 2024-05-31

* Support collection name prefix by @GromNaN in [#2930](https://github.com/mongodb/laravel-mongodb/pull/2930)
Expand Down
3 changes: 3 additions & 0 deletions composer.json
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,8 @@
},
"require-dev": {
"mongodb/builder": "^0.2",
"league/flysystem-gridfs": "^3.28",
"league/flysystem-read-only": "^3.0",
jmikola marked this conversation as resolved.
Show resolved Hide resolved
"phpunit/phpunit": "^10.3",
"orchestra/testbench": "^8.0|^9.0",
"mockery/mockery": "^1.4.4",
Expand All @@ -45,6 +47,7 @@
"illuminate/bus": "< 10.37.2"
},
"suggest": {
"league/flysystem-gridfs": "Filesystem storage in MongoDB with GridFS",
"mongodb/builder": "Provides a fluent aggregation builder for MongoDB pipelines"
},
"minimum-stability": "dev",
Expand Down
148 changes: 148 additions & 0 deletions docs/filesystems.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,148 @@
.. _laravel-filesystems:

==================
GridFS Filesystems
==================

.. facet::
:name: genre
:values: tutorial

.. meta::
:keywords: php framework, gridfs, code example

Overview
--------

You can use the
`GridFS Adapter for Flysystem <https://flysystem.thephpleague.com/docs/adapter/gridfs/>`__
to store large files in MongoDB.
GridFS lets you store files of unlimited size in the same database as your data.


Configuration
-------------

Before using the GridFS driver, install the Flysystem GridFS package through the
Composer package manager by running the following command:

.. code-block:: bash

composer require league/flysystem-gridfs

Configure `Laravel File Storage <https://laravel.com/docs/{+laravel-docs-version+}/filesystem>`__
to use the ``gridfs`` driver in ``config/filesystems.php``:

.. code-block:: php

'disks' => [
'gridfs' => [
'driver' => 'gridfs',
'connection' => 'mongodb',
],
],

GromNaN marked this conversation as resolved.
Show resolved Hide resolved
You can configure the disk the following settings in ``config/filesystems.php``:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I: I think you can delete "the disk"

Suggested change
You can configure the disk the following settings in ``config/filesystems.php``:
You can configure the following settings in ``config/filesystems.php``:


.. list-table::
:header-rows: 1
:widths: 25 75

* - Setting
- Description

* - ``driver``
- **Required**. Specifies the filesystem driver to use. Must be ``gridfs`` for MongoDB.

* - ``connection``
- The database connection used to store jobs. It must be a ``mongodb`` connection. The driver uses the default connection if a connection is not specified.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

S: remove the article to match the other descriptions

Suggested change
- The database connection used to store jobs. It must be a ``mongodb`` connection. The driver uses the default connection if a connection is not specified.
- Database connection used to store jobs. It must be a ``mongodb`` connection. The driver uses the default connection if a connection is not specified.


* - ``database``
- Name of the MongoDB database for the GridFS bucket. The driver uses the database of the connection if a database is not specified.

* - ``bucket``
- Name or instance of the GridFS bucket. A database can contain multiple buckets identified by their name. Defaults to ``fs``.

* - ``prefix``
- Specifies a prefix for the name of the files that are stored in the bucket. Using a distinct bucket is recommended
in order to store the files in a different collection.

* - ``read-only``
- If ``true``, writing to the GridFS bucket is disabled. Write operations will return ``false`` or throw exceptions
depending on the configuration of ``throw``. Defaults to ``false``.

* - ``throw``
- If ``true``, exceptions are thrown when an operation cannot be performed. If ``false``,
operations return ``true`` on success and ``false`` on error. Defaults to ``false``.

You can also use a factory or a service name to create an instance of ``MongoDB\GridFS\Bucket``.
In this case, the options ``connection`` and ``database`` are ignored:

.. code-block:: php

use Illuminate\Foundation\Application;
use MongoDB\GridFS\Bucket;
jmikola marked this conversation as resolved.
Show resolved Hide resolved

'disks' => [
'gridfs' => [
'driver' => 'gridfs',
'bucket' => static function (Application $app): Bucket {
return $app['db']->connection('mongodb')
->getMongoDB()
->selectGridFSBucket([
'bucketName' => 'avatars',
'chunkSizeBytes' => 261120,
]);
},
],
],

Usage
-----

A benefit of using Laravel Filesystem is that it provides a common interface
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

S: Reword so this sentence is a little less subjective

Suggested change
A benefit of using Laravel Filesystem is that it provides a common interface
Laravel Filesystem provides a common interface

for all the supported file systems. You can use the ``gridfs`` disk in the same
way as the ``local`` disk.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

S: introduce the code

Suggested change
The following example writes a file to the ``gridfs`` disk, then reads the file:

.. code-block:: php

$disk = Storage::disk('gridfs');

// Write the file "hello.txt" into GridFS
$disk->put('hello.txt', 'Hello World!');

// Read the file
echo $disk->get('hello.txt'); // Hello World!

To learn more Laravel File Storage, see
`Laravel File Storage <https://laravel.com/docs/{+laravel-docs-version+}/filesystem>`__
in the Laravel documentation.

Versioning
----------

In GridFS, file names are metadata in file documents identified by unique
MongoDB ObjectID. If multiple documents share the same file name, they are
jmikola marked this conversation as resolved.
Show resolved Hide resolved
considered "revisions" and further distinguished by creation timestamps.

The Laravel MongoDB integration uses the GridFS Flysystem adapter. It interacts
with file revisions in the following ways:

GromNaN marked this conversation as resolved.
Show resolved Hide resolved
- Reading a file reads the last revision of this file name
- Writing a file creates a new revision for this file name
- Renaming a file renames all the revisions of this file name
- Deleting a file deletes all the revisions of this file name

The GridFS Adapter for Flysystem does not provide access to a specific revision
of a filename. You must use the
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

S: I'd stay consistent with using either "filename" or "file name"

Suggested change
of a filename. You must use the
of a file name. You must use the

`GridFS API <https://www.mongodb.com/docs/php-library/current/tutorial/gridfs/>`__
if you need to work with revisions, as shown in the following code:

.. code-block:: php

// Create a bucket service from the MongoDB connection
/** @var \MongoDB\GridFS\Bucket $bucket */
$bucket = $app['db']->connection('mongodb')->getMongoDB()->selectGridFSBucket();

// Download the last but one version of a file
$bucket->openDownloadStreamByName('hello.txt', ['revision' => -2])
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since PathPrefixer doesn't apply here, would users need to write /hello.txt, or do the prefixes only kick in when a path is used? I expect this could be a pain point for users, that may warrant a note.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't use the leading / in the PHP GridFS documentation. I don't know what to write in a note.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's no prefixing done in PHPLIB, so the filename is used exactly as provided.

My question here is: would hello.txt here refer to the same file written by Flysystem, or would you need to use /hello.txt? It's not clear to me if Flysystem adds a prefix to every file name or just those that appear to use paths (i.e. already contain a / somewhere in the middle of the string).

Copy link
Member

@jmikola jmikola Jun 4, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps this is answered by #2985 (comment). In that case, I suppose the note would be to remind users that prefixing is entirely handled by Flysystem. So if they're using a non-empty prefix option they'll need to ensure that filenames are prefixed manually when using PHPLIB directly.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Flysystem always adds the prefix to the file name stored in MongoDB.

67 changes: 67 additions & 0 deletions src/MongoDBServiceProvider.php
Original file line number Diff line number Diff line change
Expand Up @@ -4,15 +4,28 @@

namespace MongoDB\Laravel;

use Closure;
use Illuminate\Cache\CacheManager;
use Illuminate\Cache\Repository;
use Illuminate\Filesystem\FilesystemAdapter;
use Illuminate\Filesystem\FilesystemManager;
use Illuminate\Foundation\Application;
use Illuminate\Support\ServiceProvider;
use InvalidArgumentException;
use League\Flysystem\Filesystem;
use League\Flysystem\GridFS\GridFSAdapter;
use League\Flysystem\ReadOnly\ReadOnlyFilesystemAdapter;
use MongoDB\GridFS\Bucket;
use MongoDB\Laravel\Cache\MongoStore;
use MongoDB\Laravel\Eloquent\Model;
use MongoDB\Laravel\Queue\MongoConnector;
use RuntimeException;

use function assert;
use function class_exists;
use function get_debug_type;
use function is_string;
use function sprintf;

class MongoDBServiceProvider extends ServiceProvider
{
Expand Down Expand Up @@ -66,5 +79,59 @@ public function register()
return new MongoConnector($this->app['db']);
});
});

$this->registerFlysystemAdapter();
}

private function registerFlysystemAdapter(): void
{
// GridFS adapter for filesystem
$this->app->resolving('filesystem', static function (FilesystemManager $filesystemManager) {
$filesystemManager->extend('gridfs', static function (Application $app, array $config) {
if (! class_exists(GridFSAdapter::class)) {
throw new RuntimeException('GridFS adapter for Flysystem is missing. Try running "composer require league/flysystem-gridfs"');
}

$bucket = $config['bucket'] ?? null;

if ($bucket instanceof Closure) {
// Get the bucket from a factory function
$bucket = $bucket($app, $config);
jmikola marked this conversation as resolved.
Show resolved Hide resolved
} elseif (is_string($bucket) && $app->has($bucket)) {
// Get the bucket from a service
$bucket = $app->get($bucket);
} elseif (is_string($bucket) || $bucket === null) {
// Get the bucket from the database connection
$connection = $app['db']->connection($config['connection']);
if (! $connection instanceof Connection) {
throw new InvalidArgumentException(sprintf('The database connection "%s" does not use the "mongodb" driver.', $config['connection'] ?? $app['config']['database.default']));
}

$bucket = $connection->getMongoClient()
->selectDatabase($config['database'] ?? $connection->getDatabaseName())
->selectGridFSBucket(['bucketName' => $config['bucket'] ?? 'fs', 'disableMD5' => true]);
}

if (! $bucket instanceof Bucket) {
throw new InvalidArgumentException(sprintf('Unexpected value for GridFS "bucket" configuration. Expecting "%s". Got "%s"', Bucket::class, get_debug_type($bucket)));
}

$adapter = new GridFSAdapter($bucket, $config['prefix'] ?? '');

/** @see FilesystemManager::createFlysystem() */
if ($config['read-only'] ?? false) {
if (! class_exists(ReadOnlyFilesystemAdapter::class)) {
throw new RuntimeException('Read-only Adapter for Flysystem is missing. Try running "composer require league/flysystem-read-only"');
}

$adapter = new ReadOnlyFilesystemAdapter($adapter);
}

/** Prevent using backslash on Windows in {@see FilesystemAdapter::__construct()} */
$config['directory_separator'] = '/';

return new FilesystemAdapter(new Filesystem($adapter, $config), $adapter, $config);
});
});
}
}
Loading
Loading