Add memory to copilot

[ryderwishart]

Every cell can have a 'history' or memory.

If you go to a cell, we can pull up similar memories, so we can see a history of edits made on similar content, and then try to programmatically apply those to the cell you are on.

We can also abstract memories, creating more generalized 'rules' for the translation/editing process based on LLM-synthesized memories.

E.g., if the user fixes punctuation in 10 different ways, we can create a more general memory of how to fix punctuation including all 10 ways in one memory. This is probably more valuable for edits that can't be applied programmatically.

If you are looking at a cell, perhaps it makes sense to have the copilot suggest for you 'patterns' based on memories of previous edits.

Example feature to aim for

If you fix the proper nouns in one book, you want to be able to jump to another book and have the LLM suggest correcting the proper nouns there as well.

[ryderwishart]

From @BenjaminScholtens

export enum EditType {
    USER_EDIT = "user-edit",
    LLM_EDIT = "llm-edit",
    LLM_GENERATION = "llm-generation",
}

export type CustomNotebook = vscode.NotebookCellData & {
    language: string;
    metadata: vscode.NotebookCellData["metadata"] & {
        edits?: {
            cellValue: string;
            timestamp: number;
            type: EditType;
        }[];
    };
};

above is my proposal for how we track changes in the cells of the codex editor. Basically every time a user clicks the "generate draft" button it will save an edit with the type LLM_GENERATION. If a user makes their own edits to the cell and click save it will save an edit entry with type USER_EDIT. If a user uses some sort of llm edit tool it will save with that. We could add new EditTypes whenever. And yes if you follow my logic we will always have a duplicate of the current value. Cell.value will be "[current value 123]" and Cell.metadata.edits[lastIndex] will be "[current value 123]".

example in action

    {
      "kind": 2,
      "value": "<span>This is a test</span>\n",
      "languageId": "scripture",
      "outputs": [],
      "metadata": {
        "type": "text",
        "id": "MAT 1:1",
        "data": {},
        "edits": [
          {
            "cellValue": "<span>This is a test</span>\n",
            "timestamp": 1727201112868,
            "type": "user-edit"
          }
        ]
      }
    },

[ryderwishart]

I am thinking we should

• update LLM to be just AI - or else be more specific about LLM vs. S2S? Not critical really.
• add a USER_AND_AI_GENERATION option for when AI generation was triggered and user edited
• add a USER_AND_AI_EDIT option for when AI and user edits both took place

The idea here is to be able to track how often AI suggestions are the final step.

[ryderwishart]

Nevermind @BenjaminScholtens actually set it up to save an edit step every time you create an LLM generation, and then subsequent user edits will be saved as an edit step separately.