Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Export to HuggingFace. Support glaive-function-calling-v2 in the demo, clusters, and via sharegpt. #1113

Merged
merged 6 commits into from
Jan 23, 2024

Conversation

nsthorat
Copy link
Contributor

@nsthorat nsthorat commented Jan 22, 2024

Add Dataset.to_huggingface which returns a HuggingFace dataset. Users can then publish it explicitly.

To get the glaive-function-calling-v2 dataset to work we had to:

  • Write a notebook to map() the data in lilac over to the sharegpt format.

Here is the output dataset: https://huggingface.co/datasets/lilacai/glaive-function-calling-v2-sharegpt

Demos:

Other:

  • Allow this ./run_server_dev.sh ./demo_data

Input:
image

Output:
image

Copy link

Check out this pull request on  ReviewNB

See visual diffs & provide feedback on Jupyter Notebooks.


Powered by ReviewNB

@nsthorat nsthorat changed the title [not ready] Support glaive-function-calling-v2 in the demo, clusters, and via sharegpt. [not ready] Export to HuggingFace. Support glaive-function-calling-v2 in the demo, clusters, and via sharegpt. Jan 22, 2024
@nsthorat nsthorat changed the title [not ready] Export to HuggingFace. Support glaive-function-calling-v2 in the demo, clusters, and via sharegpt. Export to HuggingFace. Support glaive-function-calling-v2 in the demo, clusters, and via sharegpt. Jan 22, 2024
@nsthorat nsthorat requested review from dsmilkov and brilee January 22, 2024 23:36
lilac/data/dataset_duckdb.py Outdated Show resolved Hide resolved
@dsmilkov
Copy link
Collaborator

niceee!

lilac/db_manager.py Show resolved Hide resolved
run_server_dev.sh Show resolved Hide resolved
@nsthorat nsthorat merged commit 3e8f292 into main Jan 23, 2024
4 checks passed
@nsthorat nsthorat deleted the nik-demo2 branch January 23, 2024 16:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants