Unable to change propagation method - Parent Element not found

[yannkost]

Bug Description

I'm trying to change the propagation method of a neo field to "Only save blocks to the site they were created in". When the queye Job is "Applying new propagation method to Neo blocks", following error occurs: Parent Element not found.

Note: The error occurs on a child block of a parent neoblock.

An interessing point, when debugging in the craft-cms/queue/jobs/AppliyNewPropagationMethod.php file, method processItem:

// Append the clone to the source's parent
$parentId = $item::find()
    ->site('*')
    ->ancestorOf($item->id)
    ->ancestorDist(1)
    ->unique()
    ->status(null)
    ->drafts(null)
    ->provisionalDrafts(null)
    ->select(['elements.id'])
    ->scalar();

$foundParent = $parentId;
$newElementSiteIdTest = $newElement->siteId;

if ($parentId !== false) {
    // If we've cloned the parent, use the clone's ID instead

    if (isset($this->duplicatedElementIds[$parentId][$newElement->siteId])) {
        $parentId = $this->duplicatedElementIds[$parentId][$newElement->siteId]; 
//--------------------------------------------------------------------------------------------------------
// ---> Here the id get's changed of the parent Element, but no structure Element with this parentId exists in the structureelement table
//--------------------------------------------------------------------------------------------------------
    }

    $structuresService->append($item->structureId, $newElement, $parentId, Structures::MODE_INSERT);
//--------------------------------------------------------------------------------------------------------
// ---> The append fails, as there is not row with the specified parentId
//--------------------------------------------------------------------------------------------------------

For the parent all entries in structurelements seem to exist (there are 13 rows for 13 sites):

For some reason, the way the parentId for child blocks is decided seems erronous? Is there some kinda help to debug this partuclar problem?

Craft-CMS version: 4.7.1
Neo version: 4.0.3

Before trying to change the progapation method, I tried to clean up erronous fileds with gc/run, resaving neoblocks with the cli, removing drafts and all revisions, but no changes.

Steps to reproduce

Expected behaviour

No response

Neo version

4.0.3

Craft CMS version

4.7.1

What is the affected Neo field's propagation method?

Save blocks to all sites the owner element is saved in

Does this issue involve templating, and if so, is eager-loading used?

This is not a templating issue

[ttempleton]

If possible, could you please send a database backup and your composer files to [email protected] and let us know which field this is happening with (if the Craft project contains more than one Neo field), and we'll have a look at it.

[yannkost]

I just wanted to ask, if you have gotten the email with the database dump? I haven't gotten a confirmation on this side. :) (It's the weekend, so no worries :) )

On my side, I'm still working on the problem, making some discoveries in the database. A question arises: Does every neoblock have an entry in the table neoblockstructures?

As I realized, the following query give me avout 8k of the 42k neoblocks that seems to be missing such an entry.

Select * from neoblocks n
where n.id in (Select elementId from structureelements s)
and n.id not in(Select ownerId from neoblockstructures n2);

Generating this entry does not seem to hard, still, I would have to know, if the entry has to be present for each site or not. (Or just for the main site, as this is the current saving strategy I'm trying to change.

[ttempleton]

@yannkost Sorry, yes, I have received the email, thanks for sending it. Unfortunately I wasn't able to download the database backup, could you please resend that?

Regarding your other question - the neoblockstructures table doesn't contain Neo block data directly (the ownerId refers to the entry/category/etc. where the Neo field is being used). You should get the same number of results after deleting the last line of the query. That said, I am a bit concerned if there are blocks that don't have data in structureelements (unless the blocks were deleted, or possibly affected by a bug like #840).

[yannkost]

@ttempleton oh, so the ownerId is the entryId where the block is being used, not the neoblock id, got it. I think all neoblocks have an structureelements line in the database.

I just sent the database with another service, could you tell me, if this is working for you?

[yannkost]

As general information:

Select * from neoblocks -> 42349 rows

Select * from neoblocks n
where n.id in (Select elementId from structureelements s) -> 8019 rows

[ttempleton]

Confirming I have now received the database backup - thanks!

[yannkost]

For what I can see, the child blocks seem to want to append to the right parentId, but the structureId is wrong.

[ttempleton]

Yep I believe you are correct. Craft's ApplyNewPropagationMethod is using the structureId of the element being duplicated, presumably because in any case where inbuilt Craft elements belong to a structure, elements from different sites will still belong to the same structure. We will need to override that somehow.

[yannkost]

I'm going with following code for now:

$parentStructureId = null;
if($item instanceof Block){
    $structeElements = (new Query())
        ->select([])
        ->from(['structureelements'])
        ->where(['elementId' => $parentId ])
        ->all();
    if(sizeof($structeElements) == 1){
        $parentStructureId = $structeElements[0]['structureId'];
        if($parentStructureId !== $item->structureId){
            $structuresService->append($parentStructureId, $newElement, $parentId, Structures::MODE_INSERT);
        }else{
            $structuresService->append($item->structureId, $newElement, $parentId, Structures::MODE_INSERT);
        }
    }else{
        $structuresService->append($item->structureId, $newElement, $parentId, Structures::MODE_INSERT);
    }
}else {
    $structuresService->append($item->structureId, $newElement, $parentId, Structures::MODE_INSERT);
} 

[yannkost]

 try {
    $structuresService->append($item->structureId, $newElement, $parentId, Structures::MODE_INSERT);
} catch (\Exception $e) {
    if ($item instanceof Block) {
        $parentStructureId = null;
        $structeElements = (new Query())
            ->select([])
            ->from(['structureelements'])
            ->where(['elementId' => $parentId])
            ->all();
        if (sizeof($structeElements) == 1) {
            $parentStructureId = $structeElements[0]['structureId'];
            if ($parentStructureId !== $item->structureId) {
                $structuresService->append($parentStructureId, $newElement, $parentId, Structures::MODE_INSERT);
            } else {
                throw new \yii\base\Exception("this is a nope!");
            }
        }
    }
}

I now also tried this, but when used, the entries on the other sites loses the field in the backend....

[ttempleton]

I've just pushed a fix for this that involves:

• copying ApplyNewPropagationMethod::processItem() into a new ApplyNeoPropagationMethod class so we can remove the structureId stuff causing the error;
• storing the correct structure data and saving the block structures at the end of applying the propagation method change, which should avoid the issue you experienced in your last comment, where the blocks were lost (this was happening because the block structure creation was happening after the first batch job of applying the propagation method change).

This fix looks to me to be working with your database; I'd like to do some more testing with my usual testing environment, and then I'll have a new release tagged tomorrow.

[yannkost]

@ttempleton sounds awesome, can you tell me, how I can get this fix? I will report back, if this is working on my end.

[ttempleton]

If you'd like to try it out, you can do so by setting your composer.json file's spicyweb/craft-neo requirement to 4.x-dev as 4.0.3 and running composer update.

[yannkost]

@ttempleton Worked like a charm! Thanks you and if you have a "buymeacoffee" just tell me. This is a big save for me!

[ttempleton]

Thanks for confirming that @yannkost - I am also happy with further testing on my usual testing environment, and have released Neo 4.0.5 including that fix.