Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable document model in sycamore.query + query-ui improvements #884

Open
wants to merge 19 commits into
base: main
Choose a base branch
from

Conversation

baitsguy
Copy link
Contributor

@baitsguy baitsguy commented Oct 8, 2024

A few changes all over to support doc model and usability for sycamore query.

  1. Use document model in sycamore.query by enabling reconstruct_document in db scans and use_elements in llm map ops
  2. Allow custom properties in extract_entity when use_elements is True
  3. Write LogicalPlan in SycamoreExecutor when trace_dir provided
  4. Query UI updates to support local mode, view query plan in TraceViewer. Local mode will only work for query plans that are runnable on local_mode

note: The queries have only been manually tested so it's possible we'll have to make more changes. Might need UI changes if we want to show/highlight elements within a doc that passed a filter/opensearch query

@baitsguy baitsguy requested a review from mdwelsh October 8, 2024 21:48
@baitsguy baitsguy changed the title Enable document model in sycamore.query Enable document model in sycamore.query + QueryUI improvements Oct 8, 2024
@baitsguy baitsguy changed the title Enable document model in sycamore.query + QueryUI improvements Enable document model in sycamore.query + query-ui improvements Oct 8, 2024
@baitsguy baitsguy marked this pull request as ready for review October 8, 2024 22:22
apps/query-ui/queryui/Sycamore_Query.py Outdated Show resolved Hide resolved
apps/query-ui/queryui/main.py Show resolved Hide resolved
apps/query-ui/queryui/test_configuration.py Show resolved Hide resolved
lib/sycamore/sycamore/query/execution/sycamore_executor.py Outdated Show resolved Hide resolved
apps/query-ui/queryui/util.py Outdated Show resolved Hide resolved
lib/sycamore/sycamore/query/execution/sycamore_executor.py Outdated Show resolved Hide resolved
lib/sycamore/sycamore/query/execution/sycamore_operator.py Outdated Show resolved Hide resolved
lib/sycamore/sycamore/query/execution/sycamore_operator.py Outdated Show resolved Hide resolved
lib/sycamore/sycamore/query/execution/sycamore_operator.py Outdated Show resolved Hide resolved
lib/sycamore/sycamore/query/execution/sycamore_operator.py Outdated Show resolved Hide resolved
@baitsguy baitsguy requested a review from mdwelsh October 9, 2024 01:57
apps/query-ui/queryui/Sycamore_Query.py Outdated Show resolved Hide resolved
apps/query-ui/queryui/main.py Outdated Show resolved Hide resolved
apps/query-ui/queryui/test_configuration.py Show resolved Hide resolved
lib/sycamore/sycamore/query/logical_plan.py Outdated Show resolved Hide resolved
lib/sycamore/sycamore/query/execution/sycamore_executor.py Outdated Show resolved Hide resolved
@baitsguy baitsguy requested a review from mdwelsh October 9, 2024 06:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants