-
Notifications
You must be signed in to change notification settings - Fork 5
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: add example types to manifest #68
base: development
Are you sure you want to change the base?
Conversation
Any QA we can demo with? |
export const exampleCommandExecutionSchema = T.Object({ | ||
commandInvokation: T.String({ minLength: 1 }), | ||
parameters: T.Optional(T.Record(T.String(), T.Any())), | ||
expectedOutput: T.String({ minLength: 1 }), | ||
}); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
spelling mistake commandInvocation
is commandInvocation
supposed to be the comment? example: @UbiquityOS can you change my address to 0x00
Parameters are the expected way to infer parameters from the comment, right?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
{
"examples": [
{
"commandInvocation": "@UbiquityOS what is my wallet address?",
"parameters": {
"repositoryOwner": "repoOwnerUsername",
"repositoryName": "example-repo",
"issueNumber": 42,
"author": "user1"
},
"expectedOutput": "{function: 'command-query', parameters: {\"username\": \"user1\"}}"
}
]
}
Yes,commandInvocation
is supposed to be a commend, the parameters
, are values that are given to the LLM, and the expectedOutput
is what the LLM should output, in this case it would be a tool call.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Oh, I thought the expectedOutput
was supposed to be the output from the plugin, and parameters
are the tool call generated by the LLM.
Maybe we could change the name from parameters
to issueContext
, githubContext
or just context
, and expectedOutput
to expectedToolCall
, what do you think?
Is it important for expectedOutput
to be a string instead of an object? Because it's easier to write an object than to deal with escape characters and it's more readable.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Oh, I thought the
expectedOutput
was supposed to be the output from the plugin, andparameters
are the tool call generated by the LLM. Maybe we could change the name fromparameters
toissueContext
,githubContext
or justcontext
, andexpectedOutput
toexpectedToolCall
, what do you think?
That sounds good. These terms would function more as guiding examples, which could be included in the prompt to assist with generating the tool calls.
Is it important for
expectedOutput
to be a string instead of an object? Because it's easier to write an object than to deal with escape characters and it's more readable.
No, it can definitely be an object instead
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We follow the manifest standard so any property name that does not conform should be prefixed with ubiquity:
I am not sure how to QA this? Should I make some sample manifests? |
It would be ideal if you set up a kernel and we could interact with it with this upgraded behavior. Have you done this before? |
We had a discussion with @gentlementlegen where we both agreed that following the standard so strictly is unnecessary for our use case as these are not browser extensions or PWAs. We are already breaking the standard because |
Its not hard to make the prefix. I don't see why not conforming to standards was the conclusion you guys came to, its not right. |
Because we can but I do not really see the point as we can conform to our own standards, we are not making a manifest for a web app, it just makes names in the manifest longer for no reason so far. |
Following this logic there's little reason to follow any standards. It doesn't make sense why you decide this to be an exception especially if the trade off is fixable with a find and replace. |
Resolves #242