Initial design proposal of simultaneous flow editing by multiple clients #66
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,121 @@ | ||
| --- | ||
| state: draft | ||
| --- | ||
|
|
||
| # Simultaneous Flow Editing | ||
|
|
||
| ## Summary | ||
|
|
||
| With current Node-RED, multiple clients can connect to the same runtime and can edit the same flow independently. However, editing results by multiple clients are not synchronized, so conflicts of flows occur during deployment of the flow. There is a feature to support merging flow conflicts referring node property values, but it is difficult for end users to merge node properties in proper way. | ||
|
|
||
|  | ||
|
|
||
|  | ||
|
|
||
| Therefore, this design document proposes a new Node-RED editor feature that supports multiple clients to edit the same flow simultaneously. | ||
|
|
||
| ## Authors | ||
|
|
||
| - Hiroyasu Nishiyama | ||
|
|
||
| ## Proposal | ||
|
|
||
| Here, we consider the following two methods as candidates for simultaneous editing support of a Node-RED flow: | ||
|
|
||
| 1. Synchronize editing actions among clients, | ||
|
|
||
| 2. Allow locking a tab by a client to make single client can only edit the tab. | ||
|
|
||
| ### Method1: Edit Action Synchronization | ||
|
|
||
| #### 1-a: Requirements | ||
|
|
||
| - The flow edit result of a client is immediately reflected to the flow edited on workspaces of other clients. | ||
|
|
||
| #### 1-b: Issues | ||
|
|
||
| - How to reflect flow editing state among clients? | ||
|
|
||
| - How to handle other kind of state changes of Node-RED including editor and runtime (e.g. node installation, execution result, etc.)? | ||
|
|
||
| #### 1-c: Implementation | ||
|
|
||
| - Extends the edit history information currently used for *undo* / *redo* of flow editing operations, and sends editing action information to other clients via the runtime. | ||
|
There was a problem hiding this comment. I think I have a branch of code locally that started looking at how to generate JSON Patch fragments from the history component. That seemed to be the most sensible way to approach this - as it should be possible to apply patches in a well defined order (and also has some well-known algorithms for resolving out-of-order merges). There was a problem hiding this comment. My assumption was to serialize the editing actions by communicating with runtime APIs. There was a problem hiding this comment. JSON Patch is one possible format - but the key is the history component would be the central place to know that a change has happened, and that change should be communicated to other editors. When the other editors receive the details of that event, they need to apply them to the local configuration - translating the event from whatever format it is in back into whatever modification to internal state in the editor is needed. There was a problem hiding this comment. I agree. My assumption is that the runtime manages the edit histories to keep the order consistent for all clients. There was a problem hiding this comment. Shifting the edit history to the runtime is a major change to how the editor works. We'd need to see a lot more detail on how you think that would work. Resolving concurrent edit state between multiple parties is a very complex task - there are countless research papers on the topic. This is the part that I'm most interested in - we cannot underestimate the difficulty in getting it right. For example, how do you resolve conflicting changes to the same resource. How do you manage ordering of changes. How do you manage undoing a change. But this still remains an academic conversation untill we under stand how we handle the deploy button issue. There was a problem hiding this comment. Thanks for your comment. I have updated the design document. Regarding the issue of the deploy button, I would like to suggest a method that all users to save the fix and then deploy it. |
||
|
|
||
| - The client that received the edit action reproduces the flow edit result by executing the received edit action on the client browser. | ||
|
|
||
| - In order to this properly work, ordering of edit actions are required to guarantee the order in which the edit operations are applied. | ||
|
|
||
| - Serialization of actions are provided as a feature of the runtime. | ||
|
|
||
| Though, there are performance concerns about this with large number of clients or slow network. | ||
|
|
||
| - Changes other than the flow should also be treated as editing actions to be sent. However, it is necessary not to execute anything that involves a run-time operation. | ||
|
|
||
|  | ||
|
|
||
| - We may need to synchronize client state at some point by sending refresh request to all client. | ||
|
|
||
| - Whether to use sync mode can be specified in `settings.js`. | ||
|
There was a problem hiding this comment. Yes - this would have to be an opt-in feature. I also wonder whether this should be provided by a plugin rather than part of the core. It would require some additional APIs to be exposed in core for sure, but it may be a cleaner approach - particular for those who embed Node-RED and don't want this type of functionality at all. There was a problem hiding this comment. Yes. Various event extensions will be required in Node-RED side, but I think it can be implemented as a plugin. |
||
|
|
||
| ### Method2: Locking Tab | ||
|
|
||
| #### 2-a: Requirements | ||
|
|
||
| Lock the flow edit target of client A in order to suppress or notify the edit by another client B. | ||
|
There was a problem hiding this comment. Being able to lock tabs comes up in other contexts. For example, restricting who can edit certain flows based on user name/role. So I think there is value in exploring this approach as I think it will help other use cases. There are plenty of difficulties around managing the locked state - and how you handle the case when someone wanders off for a long weekend, forgetting they have left a tab locked. And you do describe some possible ways to manage that. There was a problem hiding this comment. As you wrote, the ability to limit flow editing may be useful in team development. |
||
|
|
||
| #### 2-b: Issues | ||
|
|
||
| - What to lock to edit? | ||
|
|
||
| - What restrictions are placed on locked targets? | ||
|
|
||
| #### 2-c: Implementation | ||
|
|
||
| - Allow each client to be able to get exclusive editing rights (*lock*) for each tab. | ||
|
|
||
| - Place a button to lock the tab on the settings panel of each tab. | ||
|
|
||
|  | ||
|
|
||
| The lock button expected to have the following states: | ||
|
|
||
| | Tab State | Button Label | Description | | ||
| | --------- | ------------ | ----------------------------- | | ||
| | Free | Lock | Tab is not locked | | ||
| | Owned | Unlock | Tab is locked by this client | | ||
| | Locked | Locked | Tab is locked by other client | | ||
|
|
||
|  | ||
|
|
||
|
|
||
|
|
||
| - Tab lock state is managed by the following manner: | ||
|
|
||
| - If a tab is locked by another client, the tab cannot be edited (**read only**). | ||
| The tab lock request is sent to the runtime, which manages and arbitrates the state and broadcast the result to other clients. | ||
|
|
||
| - If a client that holds a lock loses connection to the runtime, the runtime unlocks the tabs that the client is locking after a period of time. At this point, the runtime also notifies this to other clients. | ||
|
|
||
| - If the disconnect client reconnects, then | ||
|
|
||
| - If the tab has not changed, regain the lock. | ||
|
|
||
| - If the tab has changed, perform the conventional node merge process | ||
| (*the same issues of flow conflict resolution remain in this case*) | ||
|
|
||
| - To make each tab editable by separate clients, it is desirable to divide the flow file into tab-by-tabs. | ||
|
|
||
| - Whether to use lock mode can be specified in `settings.js`. | ||
|
|
||
| ## Discussion | ||
|
|
||
| - The problem with method 2 is that conflicts may occur when locks are not acquired and a flow is edited by multiple clients. So, there still exist the same issues with current Node-RED implementation when conflicts occur. | ||
|
|
||
| - Compared with method 2, method 1 has to serialize and broadcast the edit actions to all connected clients. This may cause a performance concern when the number of clients increases. | ||
|
|
||
| However, this method seems to be more intuitive for users. So, I would like to recommend this method as a means for simultaneous flow editing. | ||
knolleary marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
|
||
| ## History | ||
|
|
||
| - 2022-01-30 - Initial proposal | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It is easy to compare this to the user experience on Google Docs. However this is one very important difference.
In Google Docs, you are all editing the document on the server. No one has to hit save - the changes are 'live'.
In Node-RED, nothing is saved in the runtime until someone presses the Deploy button.
That difference is very important as it raises a number of questions over what happens when a user hits Deploy.
They will end up deploying their work and whatever the other user was in the middle of doing - with no guarantee the other user was ready to deploy their work. They could completely break things. This is one of the reasons I haven't tackled this yet - despite a few early experiments. And it is why the current concurrent editing model is based around when a user Deploys their work.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
As I wrote above, within a small team, I think it's appropriate to deal with this by an arrangement between members.
Our users have similar issues with team development on current Node-RED and they are covering the issue by operations.
However, there may be some support features that need to be improved, such as a feature for helping participating users agree on deploy.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
My worry is the feature makes the collaboration issues harder - because there is nothing built-in to help co-ordinate deploys. User's could get into a real mess if this feature allows User A to deploy User B's half-finished work. Even if it relies on co-ordination outside of Node-RED, accidents still happen.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I understand such concerns. I think that one guard is that this feature is opt-in feature. So, we can expect team members understand the behavior using it.
As for the support function for this, for example, notification to the user who is editing and authorization from them may be useful.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think the design needs to carefully describe what the user workflows are so we can see what type of interaction is possible and what issues come up.