fix: issue_comments linting added issue_comments:edited, created and … #1

sshivaditya2019 · 2024-08-27T04:27:43Z

Resolves #32

Created webhook listener for issue_comments:created and issue_comments:edited.
Uses OpenAI embeddings.
Embeddings are stored in Supabase DB
New OpenAI adapter can be expanded in future to support text generation and question answering using RAG.
issue_comments:deleted, deletes the comment from the DB
Expects theComment ID , Issue Body in the payload.
OpenAI Key, Supabase Key and URL have to be added in the envs.
When deployed a new table has to be created with the associated supabase account with the following schema. For Vector type, the pgvector extension must be enabled in the postgres extensions.

issue_comment
id: int8
issuebody: Text,
commentbody: Text,
embedding: Vector(3072)

…deleted

sshivaditya2019 · 2024-08-27T04:30:03Z

@0x4007

github-actions · 2024-08-27T04:32:36Z

Unlisted binaries (2)

Filename	binaries
package.json	`supabase` `cross-env-shell`

Unused types (4)

Filename	types
src/types/database.ts	`Tables` `TablesInsert` `TablesUpdate` `Enums`

.vscode/settings.json

package.json

src/adapters/openai/helpers/embedding.ts

src/main.ts

src/plugin.ts

src/types/env.ts

tests/main.test.ts

0x4007 · 2024-08-27T04:35:40Z

Fix Resolves #32 by placing the full URL of the issue in place of the number.

package.json

gentlementlegen · 2024-08-28T06:24:07Z

@sshivaditya2019 It seems that you are storing the vectors within Supabase but I see no Supabase folder for setup. Also, typings should be used so Supabase offers auto-complete. Please fix the unit tests as well.

sshivaditya2019 · 2024-08-28T14:14:27Z

@sshivaditya2019 It seems that you are storing the vectors within Supabase but I see no Supabase folder for setup. Also, typings should be used so Supabase offers auto-complete. Please fix the unit tests as well.

Fixed added schema for table. Unit tests work now. Schema typing should be available in the project. Embedding type now expects a vector array of length 3072

wrangler.toml

Co-authored-by: Mentlegen <[email protected]>

gentlementlegen · 2024-08-28T14:35:03Z

@sshivaditya2019 Thanks a lot for the changes, appreciated. Since you seem to use Supabase, I assume that we should spin up a new instance specially for this project because this data would not make much sense in our current database, @0x4007 please confirm.

As such could you please add a migration script that would:

run migrations
update database types

when code is pushed to the main branch? Here is an example to get you started: https://github.com/ubiquibot/user-activity-watcher/blob/development/.github/workflows/database.yml#L35
It would be also nice to have code generation commands within the package.json, I don't think I saw them. Here is an example: https://github.com/ubiquibot/user-activity-watcher/blob/development/package.json#L21

0x4007 · 2024-08-28T14:35:55Z

Sure new database is fine to keep things simple.

sshivaditya2019 · 2024-08-28T15:09:53Z

@sshivaditya2019 Thanks a lot for the changes, appreciated. Since you seem to use Supabase, I assume that we should spin up a new instance specially for this project because this data would not make much sense in our current database, @0x4007 please confirm.

As such could you please add a migration script that would:

run migrations

update database types

when code is pushed to the main branch? Here is an example to get you started: https://github.com/ubiquibot/user-activity-watcher/blob/development/.github/workflows/database.yml#L35 It would be also nice to have code generation commands within the package.json, I don't think I saw them. Here is an example: https://github.com/ubiquibot/user-activity-watcher/blob/development/package.json#L21

Added Github Workflow, and fixed the package.json.

src/adapters/openai/helpers/embedding.ts

tests/__mocks__/adapter.ts

gentlementlegen · 2024-08-28T16:14:49Z

Please fix the Knip issues, I will try to set up this on my repo and give it a run.

sshivaditya2019 · 2024-08-28T16:23:50Z

Knip should be fixed in the latest commit.

src/types/database.ts

gentlementlegen

I had a horrendous time trying this locally and had to rely on a GitHub deployment to test it. A few major changes to consider:

remove the commands section from the manifest otherwise the plugin will never run
inside compute.yml rename the content properly (plugin-name)
in compute.yml add the GitHub secrets properly or the run will crash (I assume this will run as an Action because it's a long running task)
update README.md properly to help future collaborators

Related run: https://github.com/Meniole/issue-comment-embeddings/actions/runs/10612546023/job/29414425736#step:5:8

package.json

manifest.json

sshivaditya2019 · 2024-08-29T13:35:55Z

I had a horrendous time trying this locally and had to rely on a GitHub deployment to test it. A few major changes to consider:

remove the commands section from the manifest otherwise the plugin will never run

inside compute.yml rename the content properly (plugin-name)

in compute.yml add the GitHub secrets properly or the run will crash (I assume this will run as an Action because it's a long running task)

update README.md properly to help future collaborators

Related run: https://github.com/Meniole/issue-comment-embeddings/actions/runs/10612546023/job/29414425736#step:5:8

I expected this to run on worker, since action takes too long for each message.
Its working for me as expected locally, might be some issue in .ubiquibot-config.yml. Could you please let me know what issue you were facing?
Updated theREADME.md

.github/.ubiquibot-config.yml

package.json

wrangler.toml

tests/__mocks__/helpers.ts

gentlementlegen · 2024-08-30T07:31:27Z

src/main.ts

@@ -10,7 +10,6 @@ import { plugin } from "./plugin";
 */
 export async function run() {
  const payload = github.context.payload.inputs;
-
  const env = Value.Decode(envSchema, payload.env);


This will always crash on run:
https://github.com/Meniole/issue-comment-embeddings/actions/runs/10628609152/job/29463778679#step:5:45

The environment is populated either from GitHub or the Worker. Since this runs as an action, it should be loaded directly within process.env, which you can add inside payload.env if you want. In the case of a worker, it is send within the payload by Cloudflare.

But, as mentioned before, I expect this to run in a Cloudflare worker environment. The env variables are sent within the payload.

I think running as worker is fine.

gentlementlegen

Some more changes might be needed to be able to run this one, but getting closer (c.f. my last comment).

Testing actions is tedious. I would advise you to create your own organization (also works on your personal account but usage restrictions are way lower) and set up the bot so you can try some runs and make sure it works as intended.

sshivaditya2019 · 2024-08-30T08:27:32Z

Could you please confirm if the .ubiquibot-config.yml file isn't working for you when you try using it? I tested it again by creating a new issue and adding comments, and everything worked fine on my end.

Is there any particular reason you keep trying to set it up in Github Action environment. The plugin could be made for a Cloudflare Worker or Standalone Github action right ? I don't want this to run in Github Action environment and is intended to be setup on Worker env.

sshivaditya2019 · 2024-08-30T08:42:16Z

Screen.Recording.2024-08-30.at.4.39.35.AM.mov

0x4007 · 2024-08-31T03:21:14Z

Very lovely QA testing video thank you.

0x4007

This pull looks great. I just have some cosmetic changes

0x4007 · 2024-08-30T03:27:05Z

.github/workflows/compute.yml

@@ -1,4 +1,4 @@
-name: "the name of the plugin"
+name: "@ubiquibot/issue-comment-embeddings"


Suggested change

name: "@ubiquibot/issue-comment-embeddings"

name: "@ubiquity-os/comment-vector-embeddings"

Shouldn't this eventually cover all comments?

Ubiquibot is deprecated

ideally this plugin should be generalized to allow us to pass in anything to generate embeddings. Maybe we can generate a unique ID based on where it's coming in from (GitHub, Telegram) and the project it's posted to (repository/issue, or group chat id) etc

The purpose of this is to serve as the foundation of the system's awareness across all organization operations.

0x4007 · 2024-08-31T03:22:45Z

.github/.ubiquibot-config.yml

+    id: test-app
+    uses:
+      - plugin: https://ubiquibot-issue-comment-embeddings.sshivaditya.workers.dev
+        runsOn: [ "issue_comment.created", "issue_comment.edited", "issue_comment.deleted" ]


Can't we do "issue_comment" to be concise? @gentlementlegen

Not with the current implementation because it wouldn't match the event and wouldn't be triggered.

This file should be removed, otherwise no plugin will run inside of this repo.

0x4007 · 2024-08-31T03:24:50Z

manifest.json

-      "description": "Command 1 with an argument."
-    }
-  }
+  "name": "@ubiquibot/issue-comment-embeddings",


ubiquity-os

0x4007 · 2024-08-31T03:25:14Z

manifest.json

-  }
+  "name": "@ubiquibot/issue-comment-embeddings",
+  "description": "Issue comment plugin for Ubiquibot. It enables the storage, updating, and deletion of issue comment embeddings.",
+  "ubiquity:listeners": ["issue_comment.created", "issue_comment.edited", "issue_comment.deleted"]


Possibly issue_comment only

0x4007 · 2024-08-31T03:27:01Z

src/main.ts

@@ -10,7 +10,6 @@ import { plugin } from "./plugin";
 */
 export async function run() {
  const payload = github.context.payload.inputs;
-
  const env = Value.Decode(envSchema, payload.env);


I think running as worker is fine.

0x4007 · 2024-08-31T03:29:37Z

wrangler.toml

@@ -1,4 +1,4 @@
-name = "your-plugin-name"
+name = "ubiquibot-issue-comment-embeddings"


UbiquiBot term is deprecated in favor of UbiquityOS (or when lowercase, ubiquity-os)

gentlementlegen · 2024-08-31T10:54:39Z

@0x4007 My concern about having this as a worker is the following:

we don't need to handle commands so I don't see the need for instant feedback
workers are limited to 50 network calls and this plugin will most likely hit the DB a lot (each supabase action is a network call)
it's a long running process

But I will test it as a worker and see if it works.

sshivaditya2019 · 2024-08-31T11:12:07Z

@0x4007 My concern about having this as a worker is the following:

we don't need to handle commands so I don't see the need for instant feedback

workers are limited to 50 network calls and this plugin will most likely hit the DB a lot (each supabase action is a network call)

it's a long running process

But I will test it as a worker and see if it works.

If there are multiple comments simultaneously they might create a lot of action.
I am not sure about this but we would loose a lot of build minutes for this.
As far as I know, it's 50 outbound sub-requests per request. This could be avoided if we use Cloudflare D1.
Yeah, it's supposed to be a long process running it again and again doesn't make sense

0x4007 · 2024-08-31T11:31:31Z

GitHub actions are totally free for us. Workers can be costly.

It is only a problem to do workers if we have to spend money

sshivaditya2019 · 2024-08-31T11:48:36Z

GitHub actions are totally free for us. Workers can be costly.

It is only a problem to do workers if we have to spend money

I think there should be a third option apart from workers and actions. Features like this are better suited for a standalone web server.
That being said total average execution time for this worker is 1ms CPU time and the free plan includes 30 Million ms CPU time.

gentlementlegen · 2024-08-31T11:56:20Z

Tested thsi on my Cloudflare, works fine, thanks for all the bug fixes. In the end switching this plugin from Worker to Action should be as simple as changing a URL to an Action path.

Please remove .ubiquibot-config.yml and good to go.

sshivaditya2019 · 2024-08-31T13:11:23Z

@gentlementlegen Removed .ubiquibot-config.yml. Changed @ubiquibot to ubiquity-os

gentlementlegen

Good with me. @0x4007 if you can create the Supabase instance and set the secrets, I do not have permission to create new instances on the company's account, thanks.

0x4007 · 2024-08-31T14:19:35Z

I think create and we can transfer later? I don't have a lot of time on my computer these days due to the upcoming conference.

Keyrxng · 2024-08-31T14:35:45Z

This is great stuff gg @sshivaditya2019

@gentlementlegen @0x4007 I think it might be a good idea as another issue to improve the db structure so that conversations are grouped together easily. This way when working with the embeddings it's easier to get all of the context specific to a certain task or PR review discussion etc. For instance telegram chat embeddings would be grouped with the issue it relates to in order to give a fuller picture when the embeddings are used.

I guess using the task body is possible but we tend to avoid using arbitrary strings for situations like this

0x4007 · 2024-09-08T12:17:17Z

src/types/context.ts

 /**
 * Update `manifest.json` with any events you want to support like so:
 *
 * ubiquity:listeners: ["issue_comment.created", ...]
 */
-export type SupportedEventsU = "issue_comment.created";
+export type SupportedEventsU = "issue_comment.created" | "issue_comment.deleted" | "issue_comment.edited";


@Keyrxng can't this type be dynamically generated by importing the manifest.json file?

Unfortunately not without trickery because string arrays type as string and inference goes out of the window.

The easiest way to do it would be to convert them into keys inside the manifest as opposed to a string array this way we could easily extract string literal types from the object.

fix: issue_comments linting added issue_comments:edited, created and …

9c0de23

…deleted

sshivaditya2019 marked this pull request as ready for review August 27, 2024 04:29

0x4007 reviewed Aug 27, 2024

View reviewed changes

fix: test and linting

a4ee41e

gentlementlegen reviewed Aug 27, 2024

View reviewed changes

package.json Outdated Show resolved Hide resolved

fix: plugin name

d91b991

sshivaditya2019 requested a review from gentlementlegen August 27, 2024 16:21

fix: wrangler name

f890071

sshivaditya2019 requested a review from 0x4007 August 27, 2024 18:05

sshivaditya added 2 commits August 28, 2024 02:29

feat: modified tests

97e267f

fix: fixed tests missing supabase files

0e870ac

gentlementlegen reviewed Aug 28, 2024

View reviewed changes

wrangler.toml Outdated Show resolved Hide resolved

Update wrangler.toml

3b25774

Co-authored-by: Mentlegen <[email protected]>

fix: github workflow, types package.json, env examples

16786d7

gentlementlegen reviewed Aug 28, 2024

View reviewed changes

src/adapters/openai/helpers/embedding.ts Show resolved Hide resolved

gentlementlegen reviewed Aug 28, 2024

View reviewed changes

tests/__mocks__/adapter.ts Show resolved Hide resolved

fix: knip workflow

f325310

fix: cspell fix

736bea6

gentlementlegen reviewed Aug 28, 2024

View reviewed changes

src/types/database.ts Outdated Show resolved Hide resolved

gentlementlegen reviewed Aug 29, 2024

View reviewed changes

package.json Outdated Show resolved Hide resolved

manifest.json Outdated Show resolved Hide resolved

sshivaditya added 2 commits August 29, 2024 08:22

fix: manifest.json, compute.yml

21409d5

fix: readme.md

2fec447

fix: readme.md

9c5fbfe

gentlementlegen approved these changes Aug 30, 2024

View reviewed changes

.github/.ubiquibot-config.yml Outdated Show resolved Hide resolved

package.json Outdated Show resolved Hide resolved

package.json Outdated Show resolved Hide resolved

wrangler.toml Outdated Show resolved Hide resolved

tests/__mocks__/helpers.ts Outdated Show resolved Hide resolved

sshivaditya added 2 commits August 29, 2024 22:36

fix: remove config.yml and wrangler.toml namespace entries

127cc22

fix: package.json

806c6c0

gentlementlegen self-requested a review August 30, 2024 04:06

gentlementlegen reviewed Aug 30, 2024

View reviewed changes

gentlementlegen requested changes Aug 30, 2024

View reviewed changes

sshivaditya added 3 commits August 30, 2024 03:54

fix: added config.yml

d12c522

fix: added config.yml

221d34d

fix: added config.yml

c0f784b

0x4007 approved these changes Aug 31, 2024

View reviewed changes

fix: removed config.yml changed name

744e08c

gentlementlegen approved these changes Aug 31, 2024

View reviewed changes

0x4007 merged commit c6dd05e into ubiquity-os-marketplace:development Aug 31, 2024
2 checks passed

ubiquity-os bot mentioned this pull request Aug 31, 2024

issues_comment.created vector embeddings ubiquity-os/plugins-wishlist#32

Closed

0x4007 reviewed Sep 8, 2024

View reviewed changes

		@@ -1,4 +1,4 @@
		name: "the name of the plugin"
		name: "@ubiquibot/issue-comment-embeddings"

	name: "@ubiquibot/issue-comment-embeddings"
	name: "@ubiquity-os/comment-vector-embeddings"

		@@ -1,4 +1,4 @@
		name = "your-plugin-name"
		name = "ubiquibot-issue-comment-embeddings"

fix: issue_comments linting added issue_comments:edited, created and … #1

fix: issue_comments linting added issue_comments:edited, created and … #1

Conversation

sshivaditya2019 commented Aug 27, 2024 • edited Loading

sshivaditya2019 commented Aug 27, 2024

github-actions bot commented Aug 27, 2024 • edited Loading

Unlisted binaries (2)

Unused types (4)

0x4007 commented Aug 27, 2024 • edited Loading

gentlementlegen commented Aug 28, 2024 • edited Loading

sshivaditya2019 commented Aug 28, 2024

gentlementlegen commented Aug 28, 2024

0x4007 commented Aug 28, 2024

sshivaditya2019 commented Aug 28, 2024

gentlementlegen commented Aug 28, 2024

sshivaditya2019 commented Aug 28, 2024

gentlementlegen left a comment • edited Loading

Choose a reason for hiding this comment

sshivaditya2019 commented Aug 29, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

gentlementlegen left a comment • edited Loading

Choose a reason for hiding this comment

sshivaditya2019 commented Aug 30, 2024 • edited Loading

sshivaditya2019 commented Aug 30, 2024

0x4007 commented Aug 31, 2024

0x4007 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

gentlementlegen commented Aug 31, 2024

sshivaditya2019 commented Aug 31, 2024 • edited Loading

0x4007 commented Aug 31, 2024

sshivaditya2019 commented Aug 31, 2024 • edited Loading

gentlementlegen commented Aug 31, 2024

sshivaditya2019 commented Aug 31, 2024

gentlementlegen left a comment

Choose a reason for hiding this comment

0x4007 commented Aug 31, 2024 • edited Loading

Keyrxng commented Aug 31, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sshivaditya2019 commented Aug 27, 2024 •

edited

Loading

github-actions bot commented Aug 27, 2024 •

edited

Loading

0x4007 commented Aug 27, 2024 •

edited

Loading

gentlementlegen commented Aug 28, 2024 •

edited

Loading

gentlementlegen left a comment •

edited

Loading

gentlementlegen left a comment •

edited

Loading

sshivaditya2019 commented Aug 30, 2024 •

edited

Loading

sshivaditya2019 commented Aug 31, 2024 •

edited

Loading

sshivaditya2019 commented Aug 31, 2024 •

edited

Loading

0x4007 commented Aug 31, 2024 •

edited

Loading