Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Price Explorer Tab #3663

Open
wants to merge 3 commits into
base: main
Choose a base branch
from
Open
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
72 changes: 51 additions & 21 deletions fastchat/serve/gradio_web_server_multi.py
Original file line number Diff line number Diff line change
Expand Up @@ -53,31 +53,61 @@
def build_visualizer():
visualizer_markdown = """
# 🔍 Arena Visualizer
This tool provides an interactive way to explore how people are using Chatbot Arena.
Using *[topic clustering](https://github.com/MaartenGr/BERTopic)*, we organized user-submitted prompts from Arena battles into broad and specific categories.
Dive in to uncover insights about the distribution and themes of these prompts!
Arena visualizer provides interactive tools to explore and draw insights from our leaderboard data.
"""
gr.Markdown(visualizer_markdown, elem_id="visualizer_markdown")
expandText = "👇 Expand to see detailed instructions on how to use the visualizer"
with gr.Accordion(expandText, open=False):
instructions = """
- Hover Over Segments: View the category name, the number of prompts, and their percentage.
- *On mobile devices*: Tap instead of hover.
- Click to Explore:
- Click on a main category to see its subcategories.
- Click on subcategories to see example prompts in the sidebar.
- Undo and Reset: Click the center of the chart to return to the top level.

Visualizer is created using Arena battle data collected from 2024/6 to 2024/8.
"""
gr.Markdown(instructions)

frame = """
<iframe class="visualizer" width="100%"
src="https://storage.googleapis.com/public-arena-no-cors/index.html">
with gr.Tabs():
with gr.Tab("Topic Explorer", id=0):
topic_markdown = """
This tool provides an interactive way to explore how people are using Chatbot Arena.
Using *[topic clustering](https://github.com/MaartenGr/BERTopic)*, we organized user-submitted prompts from Arena battles into broad and specific categories.
Dive in to uncover insights about the distribution and themes of these prompts! """
gr.Markdown(topic_markdown)
expandText = (
"👇 Expand to see detailed instructions on how to use the visualizer"
)
with gr.Accordion(expandText, open=False):
instructions = """
- Hover Over Segments: View the category name, the number of prompts, and their percentage.
- *On mobile devices*: Tap instead of hover.
- Click to Explore:
- Click on a main category to see its subcategories.
- Click on subcategories to see example prompts in the sidebar.
- Undo and Reset: Click the center of the chart to return to the top level.

Visualizer is created using Arena battle data collected from 2024/6 to 2024/8.
"""
gr.Markdown(instructions)

frame = """
<iframe class="visualizer" width="100%"
src="https://storage.googleapis.com/public-arena-no-cors/index.html">
</iframe>
"""
gr.HTML(frame)
with gr.Tab("Price Analysis", id=1):
price_markdown = """
This scatterplot displays a selection of the arena's models, showing their arena score plotted against their cost.
"""
gr.Markdown(price_markdown)
expandText = (
"👇 Expand to see detailed instructions on how to use the scatterplot"
)
with gr.Accordion(expandText, open=False):
instructions = """
- Hover Over Points: View the model's arena score, cost, organization, and license.
- Click on Points: Click on a point to visit the website where the pricing details were sourced.
- Use the Legend: Click an organization name on the right to display its models. To compare models, click multiple organization names.

The prices were obtained either from the model's organization's website or through the [together.ai](https://www.together.ai/) pricing model. The scatterplot was last updated using Arena battle data from 2024/12. We only displayed models that have public pricing/parameter information.
"""
gr.Markdown(instructions)
frame = """
<iframe width="100%" scrolling="no" style="height: 800px; border: 1px solid lightgrey; border-radius: 10px;"
src="https://storage.googleapis.com/public-arena-no-cors/scatterplot.html">
</iframe>
"""
gr.HTML(frame)
gr.HTML(frame)


def load_demo(context: Context, request: gr.Request):
Expand Down
Loading