Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use Symfony Messenger to handle ItemImport messages #134

Merged
merged 24 commits into from
Aug 29, 2022
Merged
Show file tree
Hide file tree
Changes from 20 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 17 additions & 17 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -379,49 +379,49 @@ This plugin will also import product associations. It's a zero configuration imp

### Launch data import from CLI

To actually import data you must first create queue items with the **enqueue command** and then you can import them with the **consume command**.
To actually import data you must first create queue items with the **import command** and then you can import them with the **consume command**.

#### Schedule Akeneo PIM import button

This button allows you to queue a product directly from the admin index page. By adding the product to the queue, it will be updated by the Akeneo PIM at the next consume run.

![Schedule Akeneo PIM import button](schedule-akeneo-import-button.png)

#### Enqueue command
#### Import command

To create queue items you can use the `webgriffe:akeneo:enqueue` console command:
To create queue items you can use the `webgriffe:akeneo:import` console command:

```bash
bin/console webgriffe:akeneo:enqueue --since="2020-01-30"
bin/console webgriffe:akeneo:import --since="2020-01-30"
```

This will enqueue all Akeneo entities updated after the provided date.
This will import all Akeneo entities updated after the provided date.

You can also use a "since file" where to read the since date:

```bash
echo "2020-01-30" > var/storage/akeneo-sincefile.txt
bin/console webgriffe:akeneo:enqueue --since-file="var/storage/akeneo-sincefile.txt"
bin/console webgriffe:akeneo:import --since-file="var/storage/akeneo-sincefile.txt"
```

When run with the since file, the enqueue command will write the current date/time to the since file after the enqueueing process is terminated. This is useful when you put the enqueue command in cron:
When run with the since file, the import command will write the current date/time to the since file after the importing process is terminated. This is useful when you put the import command in cron:

```
* * * * * /usr/bin/php /path/to/sylius/bin/console -e prod -q webgriffe:akeneo:enqueue --since-file=/path/to/sylius/var/storage/akeneo-enqueue-sincefile.txt
* * * * * /usr/bin/php /path/to/sylius/bin/console -e prod -q webgriffe:akeneo:import --since-file=/path/to/sylius/var/storage/akeneo-import-sincefile.txt
```

This way the enqueue command is run repeatedly enqueuing only producs modified since the last command execution.
This way the import command is run repeatedly importing only products modified since the last command execution.

You can also enqueue items only for specific importers:
You can also import items only for specific importers:

```bash
bin/console webgriffe:akeneno:enqueue --importer="Product" --importer="MyImporter" --since="2020-01-30"
bin/console webgriffe:akeneno:import --importer="Product" --importer="MyImporter" --since="2020-01-30"
```

You can also enqueue items regardless of their last update date:
You can also import items regardless of their last update date:

```bash
bin/console webgriffe:akeneno:enqueue --all
bin/console webgriffe:akeneno:import --all
```

#### Consume command
Expand Down Expand Up @@ -474,20 +474,20 @@ It could be useful to add also this command to your scheduler to run automatical
To make all importers and other plugin features work automatically the following is the suggested crontab:

```
0 * * * * /path/to/sylius/bin/console -e prod -q webgriffe:akeneo:enqueue --all --importer="AttributeOptions"
* * * * * /path/to/sylius/bin/console -e prod -q webgriffe:akeneo:enqueue --since-file=/path/to/sylius/var/storage/akeneo-enqueue-sincefile.txt --importer="Product" --importer="ProductAssociations"
0 * * * * /path/to/sylius/bin/console -e prod -q webgriffe:akeneo:import --all --importer="AttributeOptions"
* * * * * /path/to/sylius/bin/console -e prod -q webgriffe:akeneo:import --since-file=/path/to/sylius/var/storage/akeneo-import-sincefile.txt --importer="Product" --importer="ProductAssociations"
* * * * * /path/to/sylius/bin/console -e prod -q webgriffe:akeneo:consume
0 0 * * * /path/to/sylius/bin/console -e prod -q webgriffe:akeneo:cleanup-queue
0 */6 * * * /path/to/sylius/bin/console -e prod -q webgriffe:akeneo:reconcile
```

This will:
* Enqueue the update of all attribute options every hour
* Import the update of all attribute options every hour
* Import, every minute, all products that have been modified since the last execution, along with their associations
* Clean the imported items queue older than 10 days, every day at midnight
* Reconcile Akeneo deleted products every 6 hours

Enqueue, Consume and Reconcile commands uses a [lock mechanism](https://symfony.com/doc/current/console/lockable_trait.html) which prevents running them if another instance of the same command is already running.
Import, Consume and Reconcile commands uses a [lock mechanism](https://symfony.com/doc/current/console/lockable_trait.html) which prevents running them if another instance of the same command is already running.

## Architecture & customization

Expand Down
2 changes: 1 addition & 1 deletion UPGRADE-2.0.md
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,7 @@ Removed all deprecations of the v1.x releases.
##### Removed
- [BC] Property Webgriffe\SyliusAkeneoPlugin\DependencyInjection\WebgriffeSyliusAkeneoExtension::$valueHandlersTypesDefinitions was removed
- [BC] Removed the service `webgriffe_sylius_akeneo_plugin.repository.cleanable_queue_item`, use the `webgriffe_sylius_akeneo.repository.cleanable_queue_item` instead.
- [BC] Removed the service `webgriffe_sylius_akeneo_plugin.controller.product_enqueue_controller`, use the `webgriffe_sylius_akeneo.controller.product_enqueue_controller` instead.
- [BC] Removed the service `webgriffe_sylius_akeneo_plugin.controller.product_enqueue_controller`, use the `webgriffe_sylius_akeneo.controller.product_import_controller` instead.
- [BC] Removed the resource `webgriffe_sylius_akeneo_plugin.queue_item` use the `webgriffe_sylius_akeneo.queue_item` instead.

### Test changes
Expand Down
39 changes: 0 additions & 39 deletions features/cleaning_queue.feature

This file was deleted.

52 changes: 0 additions & 52 deletions features/enqueuing_generic_items.feature

This file was deleted.

64 changes: 0 additions & 64 deletions features/enqueuing_products.feature

This file was deleted.

15 changes: 0 additions & 15 deletions features/enqueuing_products_associations.feature

This file was deleted.

63 changes: 63 additions & 0 deletions features/importing_generic_items.feature
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
@importing_generic_items
Feature: Importing items
In order to import data from Akeneo
As an Administrator
I want to import items from the Akeneo PIM

Background:
Given I am logged in as an administrator
And the store has a product association type "Pack" with a code "PACK"

@cli @ui
Scenario: Importing items when no item is modified since the given date
When I import items for all importers modified since date "2020-01-20 01:00:00"
And I browse products
Then I should see 0 products in the list

@cli @ui
Scenario: Importing items without a since date
When I import items for all importers with no since date
Then I should be notified that a since date is required
When I browse products
Then I should see 0 products in the list

@cli @ui
Scenario: Importing items with an invalid since date
When I import items for all importers with invalid since date
Then I should be notified that the since date must be a valid date
When I browse products
Then I should see 0 products in the list

@cli @ui
Scenario: Importing items with a since date specified from a not existent file
When I import items with since date specified from a not existent file
Then I should be notified that the since date file does not exists
When I browse products
Then I should see 0 products in the list

@cli @ui
Scenario: Importing all items regardless last modified date
Given there is a product "1314976" updated at "2022-06-15" on Akeneo
And there is a product "10597353" updated at "2022-07-23" on Akeneo
And there is a product "11164822" updated at "2022-08-01" on Akeneo
When I import all items for all importers
And I browse products
Then I should see 3 products in the list
And the product with code "11164822" should have an association "Pack" with product "10597353"

@cli @ui
Scenario: Importing all items for one importer only
Given there is a product "1314976" updated at "2022-06-15" on Akeneo
And there is a product "10597353" updated at "2022-07-23" on Akeneo
And there is a product "11164822" updated at "2022-08-01" on Akeneo
When I import all items for the "Product" importer
And I browse products
Then I should see 3 products in the list
And the product with code "11164822" should not have an association "Pack" with product "10597353"

@cli @ui
Scenario: Enqueuing all items for a not existent importer
When I import all items for a not existent importer
Then I should be notified that the importer does not exists
When I browse products
Then I should see 0 products in the list
Original file line number Diff line number Diff line change
Expand Up @@ -10,8 +10,8 @@ Feature: Importing product associations from queue
And the store has a product "10627329"
And the store has a product "upsell-product-1" with code "upsell-product-1"
And the store has a product "upsell-product-2" with code "upsell-product-2"
And there is one product associations to import with identifier "10627329" in the Akeneo queue
And the store has a product association type "Upsell" with a code "UPSELL"
When I import all items in queue
And there is a product "10627329" on Akeneo
When I import all from Akeneo
Then the product "10627329" should be associated to product "upsell-product-1" for association with code "UPSELL"
And the product "10627329" should be associated to product "upsell-product-2" for association with code "UPSELL"
Loading