Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

AI function calling, state summary, and spatial references #329

Open
wants to merge 17 commits into
base: main
Choose a base branch
from
Open

AI function calling, state summary, and spatial references #329

wants to merge 17 commits into from

Conversation

benptc
Copy link
Contributor

@benptc benptc commented Jun 6, 2024

Adds SpatialApplicationAPI and LanguageInterface to spatialDraw tool.

  • Provides an addLine API that AI can use to draw a line.
  • Performs the coordinate system conversions to say "draw a line from origin to cursor position" in world coordinates and it shows up at the intended location (in the tool's local coordinates).
  • Attempts to use gen-AI to parse the drawings coordinate list and describe its contents as visual shapes.

Part of the ptcrealitylab/vuforia-spatial-toolbox-userinterface#725 set of PRs.

benptc and others added 17 commits April 12, 2024 14:51
@benptc
spatialDraw tool uses SpatialApplicationAPI.js
44f1149
@benptc
Merge branch 'main' into iframe-service-orchestrator
5708270
@benptc
updates the draw tool with an implementation for apiDrawLine (program…
b308d1e 
…matically drawing from start to end point)
@benptc
Merge branch 'main' into iframe-service-orchestrator
26fa9b2
@benptc
properly converts GPT-drawing coordinates to tool coordinate system
b4bf26f
@benptc
adds LanguageInterface support to spatialDraw tool
66d4f4c
@hobinjk-ptc
Add spatialSensor tool
b1cd199
@hobinjk-ptc
Allow moving spatial sensors while analytics is open
31c08b5
@benptc
comment out summarizedState from spatialDraw tool, for now
41bf827
@hobinjk-ptc
Add language interface to sensor
6d9f152
@benptc
Merge branch 'spatialSensor' into ai-function-calling
3655181
@benptc
Merge branch 'main' into ai-function-calling
b4678c0
@benptc
adds drawMultipointPath to drawing tool
f7efe30
@benptc
Merge branch 'main' into ai-function-calling
bec2e90
@benptc
remove spatialSensor from core-addon
bbd8b7e
@benptc
Merge branch 'main' into ai-function-calling
40fbc01
@benptc
drawMultipointPath draws straight edges instead of curves
ffb2bb0
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@benptc @hobinjk-ptc