Inconsistent behavior of both explorers

[KristofKlein]

Hi,

one of "those useless but important" tickets:
We see quite some inconsistency when using the two explorer. E.g. we have users that see "Lost" Objects that are in use (even the new consumption report lists them) and the objects/files are still in the repo and present in the folder.

another strange inconsistency we see is user A sees objects as "manually assigned" while user B does not. for the same repo. So for the same objects we get once "manual" and once "all good and assigned via Ninja".

Once in a while we also see that the range explorer signals "no consumption" while opened in a bigger workspace context ( so for some repos/folders it works, for others it does not). Once the dedicated app folder is opened directly/alone it works correct .... (which in fact could be related to the two problems above....)

Frontend and Backend are hosted by us, and both are "up2date".
Verbose is active for Ninja, but I can not see any issue there.

Getting new numbers does work fine as well...

[vjekob]

This may have to do with two things:

• App pools
• How and when users are merging their work into mainline

Regarding app pools: I documented it a little bit, but it probably requires a big fat disclaimer - assignment explorer (for the time being) cannot work with app pools. Apps that "own" the object will see them as normal objects; other apps will see them as lost.

Regarding merging, imagine Developer A creating an object ID assignment using Ninja. Until he/she merges that into mainline and makes it available to everyone, everyone else will see it as lost. To overcome this behavior would require a massive overhaul on the back end, and also substantially slowing Ninja down. I plan on adding features that help identifying whether an object is possibly used by another developer in another branch, vs. if it's actually probably just a "lost" object.

Now, why some people see objects as manually assigned, I would say it may have to do with how Ninja syncs the consumption. I would need some repro steps. I am not sure what you mean when you say "frontend" being hosted by you - do you build your own internal copy of Ninja and distribute it internally in your team as VSIX?

[KristofKlein]

Upsi, did I write frontend? :D first day after a stressful vacation ;) Sorry for that. of course only the backend!

merging: to understand this:
B: Checks Ninja Explorer: all peachy
A: lets get me a new ID from Ninja (50001)
B: Checks Ninja Explorer again: "upsi 50001 is lost!"
A: commits all his code (what is mainline in this context? like main branch? or has AL Ninja a "mainline")
B: Sees changes, pulls them: Lost Object is no more lost?

Puuuh, I know documentation sucks, but this kind of infos what those explorer look out for and when and what would be really helpful to read somewhere and also to explain this to someone ;) maybe start with what "lost" means: lost to whom? Ninja? the current developers branch?

Well repo steps I can hardly deliver. But in fact it was nothing special as such. People opened their workspaces and got confused by the new explorer and what it was telling them.
We do use VS Code workspaces with like 10 - 14 Apps ( while one is still our "monster" with like 3000+ al files)

And as I tried to express the biggest confusion came with objects in our develop or main branch - been their for quite a while suddenly shown up as manually assigned...for some, but not all developers... others saying why is there no consumption shown in our main app that definitely was synced and is used with Ninja...

Apart from that we do not use app pools.

Hope this questions don't drive you nuts and if I missed to read the documentation carefully: I am sorry :)

[vjekob]

Don't worry, it doesn't drive me nuts 😉

"Mainline" is whatever you call your main branch (main, master, trunk, ...). What you described is expected behavior. Developer who assigns an object will see it as a valid assignment, whereas all others will see it as lost, until that object is integrated into mainline (i.e. the pull request is merged). There is not much that I can meaningfully do to make this work different.

There is always of course a solution, but to make this kind of information immediate to all developers would require me to take apart the back end completely and make it a lot slower, and that's not likely to happen.

The only way for Ninja to know which object is truly lost, and which is in fact simply living in another branch somewhere else, is to record information about which branch each object is consumed.

Imagine this hypothetical scenario:

• A assigns new ID in branch "dev-a"
• Back end stores information that this ID is now consumed, and stores branch information ("dev-a")
• Anyone developing on other branches doesn't see this object as lost

This is what you would like to have.

However - these are all possible:

• You merge object to mainline - Ninja needs to know! But how does it know? Your pipeline must feed that information to Ninja, and suddenly Ninja is no longer a simple VS Code extension but a full-blown DevOps system!
• You delete your branch locally - Ninja needs to know because now this object is lost. It's somewhat easy to detect this situation for as long as you are actually running Ninja. But if you delete your branch using Git while Ninja is not running, Ninja cannot know. Your back end will never properly see this object as either consumed or lost, because it will keep thinking that it's assigned on branch "dev-a" and has no clue that this branch is not available anymore
• You delete your branch remotely - Ninja again needs to know. Again, some DevOps/GitHub trigger would need to synch this information, and we are again talking about a full-blown DevOps system.
• What about stale branches? I know a lot of dev teams that never delete old branches after merging. There could be a lot of mess and Ninja thinking that objects are out there which in fact should be treated as lost.

There is possibly more.

Long story short - to make Ninja keep an accurate track of all this is a massive undertaking, and I am just not going to do anything about it at this moment. "Lost" objects are not necessarily truly lost, as the documentation says (and as Ninja tells you when you try to unassign them). That information should be treated as a hint, not as an authoritative statement about whether an ID is used or not. Without Ninja you have no visibility into this. With Ninja you at least can see which objects are potentially consumed.

As you can see, this is no simple problem and I don't think I am going to invest any effort into this - there are so many more useful features I can work on.

On another hand, can you give me repro steps for manually assigned objects? You said that sometimes some developers see objects as manually assigned, which aren't really manually assigned. I'd like more info about that.

[KristofKlein]

Thanks for this long answer. Know you are busy somewhere else :)

I agree in what you write and I think I also understand it. I was not about to get any overhaul of Ninja, I think for me the main focus of it is the fact to "hand out the next available number". And that is what it is doing just right. Those Explorers must have come with the need to "understand Ninja better" and to give "inside" into the backend. So those are "nice to have for me". But I don't like the fact about them producing questions and irritations.

So I definitely don't want to change the numbering feature of the ninja and they way it does that.

Coming back to the observations:

• I checkout a branch, Ninja loads, and tells me about lost objects which in fact are still there:
  

• a user reported to me that he made a table extension used AL ninja to get the ID and received immediately the warning that the ID is manual:
  

Give me some more time to align a bit internal with the developer what and how they did things. As I said it also seems to make a difference if you open via a workspace or just a folder... and as you wrote branches are of course a major topic, so I will need to make sure we are all talking about the same and not some "old dangling never updated" branches.

I will follow up once I know more!

And thanks again for taking the time to explain all this!

[vjekob]

Thanks for your detailed feedback. I hope you understand I am taking it seriously 🙂 And thanks for wanting to make this a better product 🙏

I will look into the branch checkout issue where you get a lot of objects reported as lost, I believe I should be able to easily replicate that.

Regarding the report your developer gave about an object being reported as manual while he/she swears it was actually assigned by Ninja, I know for a fact that developers sometimes select Microsoft's suggestion (second in the list) rather than Ninja's suggestion (first in the list) - I have both seen developers do that, and I can see it in telemetry (only 56% of suggested numbers are actually accepted; I do understand that developers will often ask Ninja for a number, but then not assign anything, but I don't think that this is what happens in 44% of cases).

So, please look into this issue a bit more and let me know, especially if you have reliable repeat steps.

Keep in mind that I want this tool to be useful and as non-intrusive and non-distracting as possible.

[KristofKlein]

Yes - those "this happened to me issues" are painful (to follow up). But I must say I still believe him, as the number he used is from a "later object id range". so if he would have used ALs suggestion it would have been some 504xxx something. So just the number would make me believe he used the ninja... But as I said, I will need some time to collect all those cases and see if this is something I could reproduce....

[vjekob]

And I believe you, too! (for the record)

So, please follow up with some repro steps and I'll look into it. If I have time, I'll try to figure out something by analyzing the code to see if I put a bug or two in there...

[KristofKlein]

there we go, He just showed me.

creates a new file, uses the tcodeunit snipped, gets an Ninja ID (selects from Ninja) gives an object name, starts programming some basic stuff, hits save: error shows up: I have asked him to run with verbose and you can clearly see that Ninja did the request getNext and also returns the ID in question

[KristofKlein]

Now I have several developers reporting the same :/ some also report it magically disappears after a while. Also got one "I didn't know what it was about, but I pressed yes [guess he meant the code action to manually reserve/assign the ID]". And that did work - while I am a bit puzzled about that statement: how can that be? What is the backend doing if an ID is manually reserved/assigned (if this is the right term?) while it is actually already in pick'd state? Can several persons at the same time do reassign an object and will they all get an ok (of crouse not, right? Right?) ? Obviously nothing we want to try out in our productive environment :D

[vjekob]

There is no "yes" button when storing a manually assigned ID, so I can't really say what that developer did, in response to what problem, and what exactly "did work". Without exact steps that I can actually match to what the product does, I can't really help.

Ninja does ask you to confirm, and there is a "Yes" button when unassigning a "lost" object ID. If this is what the developer did, then please ask that developer why he says "I didn't know what it was about" and then presses "Yes", while there is a "Learn more" button right next to it?

Now, about the behavior when two developers see the same manually assigned object ID and then attempt to store it in the back end at the same time. The first who does that will succeed. The second will get an error message. You can't store the same ID twice, don't worry 😉

[vjekob]

BTW - I am still waiting on actual repro steps, actual error reports, so far I only get apocryphal stories of "once this happened to me" without actual indicators of anything.

Screenshots, would help, for example...

[KristofKlein]

Hi,

so, there is nothing really special about the fact that causes the UI to complain that an AL Ninja picked number is not assigned with Ninja.

as I wrote and also placed a screenshot with the verbose from Ninja (#59 (comment)).

They open the project, wait until VS Code is ready, create a new document, create a new codeunit using the "tcodeunit" snipped, Intellisense the ID , pick from Ninja, continue adding a name, some code. save. And with the save the warning shows up. (in the screenshot that is errorlense that places the warning behind the line).
And that's it.

I do have - until now - 3 developers that see this behavior.

(About this yes/no "didn't know what I was doing, but pressed the button anyway" ... What do you think why I opened the issue with : one of "those useless but important" tickets - I'm just the man in the middle - that is the quality of feedback that i receive, I try to reconstruct the cases as good as I can ... so please be patient with me)

[KristofKlein]

latest finding:

Assignment Explorer shows the same ID twice (not only this XML Port but also Tables, Codeuntis, Pages, etc) under "Conflicts...

[KristofKlein]

So once in a while another developer joins my desk and asks about the same. What we have observed is that the message about "an ID that was assigned by Ninja gets marked as not assigned by Ninja" disappears after some time... Brings me back to the mechanics in the back: as the number gets reserved, it should be in the backend. no doubt here. But what about the frontend? when is it fetching details about picked numbers? Is it caching? Can I force a refresh? What also would help the developer is, that, once they think they did something wrong, and try to add the potential not assigned number to the backend via the code action, instead of getting an error, it tells them it was them that added it... maybe a timestamp when it was added...
As we run the backend on our own, maybe I made something wrong on the deployment?! So if you could think of something obvious that could cause all this hickups - just drop a comment :) [and a small remark her: I wanted to use your service...but mgmt...]