Store the image moderation and text moderation logs #3478

BabyChouSr · 2024-08-15T04:58:58Z

Right now, we don't store the text moderation and image moderation info when it can be very helpful.

…m-sys#3413) Co-authored-by: Wei-Lin Chiang <[email protected]> Co-authored-by: Wei-Lin Chiang <[email protected]> Co-authored-by: simon-mo <[email protected]>

infwinston

quick first pass. overall looks good to me!

infwinston · 2024-08-16T05:39:54Z

fastchat/serve/moderation/moderator.py

@@ -0,0 +1,167 @@
+import datetime


Super awesome to see we implement this abstraction!

fastchat/serve/gradio_block_arena_anony.py

infwinston

thanks @BabyChouSr left some comments

fastchat/serve/gradio_block_arena_named.py

infwinston · 2024-08-20T05:23:03Z

fastchat/serve/gradio_block_arena_named.py

-def flash_buttons():
+def flash_buttons(dont_show_vote_buttons: bool = False):
+    if dont_show_vote_buttons:
+        yield [no_change_btn] * 4 + [enable_btn] * 2


this actually ends up breaking the ui - need to keep yield + return pattern

fastchat/serve/gradio_block_arena_vision_anony.py

infwinston · 2024-08-20T05:30:19Z

fastchat/serve/gradio_block_arena_vision_anony.py

+            + [disable_btn] * 4
+            + [no_change_btn] * 3


why x4 + x3 vs x7 before

fastchat/serve/gradio_block_arena_vision_named.py

fastchat/serve/gradio_web_server.py

…ion-log

infwinston

Thanks @BabyChouSr ! this is super awesome. took a pass and left some comments. I think the main discussion item is what data should we log? maybe we don't need to log the entire moderation output which will take lots of space

infwinston · 2024-08-27T04:18:01Z

fastchat/serve/gradio_block_arena_anony.py

@@ -342,18 +352,45 @@ def bot_response_multi(
    request: gr.Request,
 ):
    logger.info(f"bot_response_multi (anony). ip: {get_ip(request)}")
+    states = [state0, state1]
+
+    if states[0] is None or states[0].skip_next:


maybe we can use this variable

states[0].content_moderator.text_flag

infwinston · 2024-10-11T21:33:53Z

fastchat/serve/gradio_web_server.py

@@ -151,6 +154,7 @@ def dict(self):
            {
                "conv_id": self.conv_id,
                "model_name": self.model_name,
+                "moderation": self.content_moderator.conv_moderation_responses,


I'm worried this would make our logs too huge... do we need to store the complete result or just moderation: True | False

the log increased probably 3x larger from

{"tstamp": 1728682280.9245, "type": "chat", "model": "chatgpt-4o-latest", "gen_params": {"temperature": 0.7, "top_p": 0.7, "max_new_tokens": 1024}, "start": 1728682277.5957, "finish": 1728682280.9245, "state": {"template_name": "gpt-4-turbo-2024-04-09", "system_message": "You are ChatGPT, a large language model trained by OpenAI, based on the GPT-4 architecture.\nKnowledge cutoff: 2023-11\nCurrent date: 2024-10-11\n\nImage input capabilities: Enabled\nPersonality: v2", "roles": ["user", "assistant"], "messages": [["user", "heyy"], ["assistant", "Hey! How\u2019s it going?"]], "offset": 0, "conv_id": "79aac635c1924185ad5e2d2c07d626a7", "model_name": "chatgpt-4o-latest"}

to

{"tstamp": 1728682280.9245, "type": "chat", "model": "chatgpt-4o-latest", "gen_params": {"temperature": 0.7, "top_p": 0.7, "max_new_tokens": 1024}, "start": 1728682277.5957, "finish": 1728682280.9245, "state": {"template_name": "gpt-4-turbo-2024-04-09", "system_message": "You are ChatGPT, a large language model trained by OpenAI, based on the GPT-4 architecture.\nKnowledge cutoff: 2023-11\nCurrent date: 2024-10-11\n\nImage input capabilities: Enabled\nPersonality: v2", "roles": ["user", "assistant"], "messages": [["user", "heyy"], ["assistant", "Hey! How\u2019s it going?"]], "offset": 0, "conv_id": "79aac635c1924185ad5e2d2c07d626a7", "model_name": "chatgpt-4o-latest", "moderation": [{"text_moderation": {"response": {"harassment": 1.1623426871665288e-05, "harassment_threatening": 4.7560683924530167e-07, "hate": 3.7377474200184224e-06, "hate_threatening": 3.018905303520114e-08, "illicit": null, "illicit_violent": null, "self_harm": 5.3270043281372637e-05, "self_harm_instructions": 3.869533247780055e-05, "self_harm_intent": 9.72322523011826e-05, "sexual": 0.0004021718923468143, "sexual_minors": 3.3023070500348695e-06, "violence": 1.4770392908758367e-06, "violence_graphic": 2.4935059173003538e-06, "self-harm": 5.3270043281372637e-05, "sexual/minors": 3.3023070500348695e-06, "hate/threatening": 3.018905303520114e-08, "violence/graphic": 2.4935059173003538e-06, "self-harm/intent": 9.72322523011826e-05, "self-harm/instructions": 3.869533247780055e-05, "harassment/threatening": 4.7560683924530167e-07}, "flagged": false}, "nsfw_moderation": {"flagged": false}, "csam_moderation": {"flagged": false}}, {"text_moderation": {"response": {"harassment": 3.410908902878873e-05, "harassment_threatening": 3.6979138258175226e-06, "hate": 2.1581441615126096e-05, "hate_threatening": 3.484581512225304e-08, "illicit": null, "illicit_violent": null, "self_harm": 4.098457338841399e-06, "self_harm_instructions": 5.236282163423311e-07, "self_harm_intent": 9.65906565397745e-07, "sexual": 0.0002804335963446647, "sexual_minors": 7.323227464439697e-07, "violence": 5.0677666877163574e-05, "violence_graphic": 9.602602403901983e-06, "self-harm": 4.098457338841399e-06, "sexual/minors": 7.323227464439697e-07, "hate/threatening": 3.484581512225304e-08, "violence/graphic": 9.602602403901983e-06, "self-harm/intent": 9.65906565397745e-07, "self-harm/instructions": 5.236282163423311e-07, "harassment/threatening": 3.6979138258175226e-06}, "flagged": false}, "nsfw_moderation": {"flagged": false}, "csam_moderation": {"flagged": false}}]}, "ip": "76.102.1.74"}

infwinston · 2024-10-15T08:57:14Z

fastchat/serve/gradio_block_arena_vision_anony.py

+    "gpt-4o-2024-05-13": 4,
+    "gpt-4-turbo-2024-04-09": 4,
+    "claude-3-haiku-20240307": 4,
+    "claude-3-sonnet-20240229": 4,
+    "claude-3-5-sonnet-20240620": 4,
+    "claude-3-opus-20240229": 4,
+    "gemini-1.5-flash-api-0514": 4,
+    "gemini-1.5-pro-api-0514": 4,
+    "llava-v1.6-34b": 4,
+    "reka-core-20240501": 4,
+    "reka-flash-preview-20240611": 4,
+    "reka-flash": 4,


let's remove them and keep the dict empty? (people may not have access to these models)

infwinston · 2024-10-15T08:57:59Z

fastchat/serve/gradio_block_arena_named.py

@@ -301,14 +346,19 @@ def bot_response_multi(
            break


-def flash_buttons():
+def flash_buttons(show_vote_buttons: bool = True):


sorry could you say more what's this for?

infwinston · 2024-10-15T09:00:38Z

fastchat/serve/gradio_block_arena_named.py

@@ -175,19 +178,27 @@ def add_text(
                no_change_btn,
            ]
            * 6
+            + [True]


maybe use a variable say show_vote_button? I see below we have a few more [True] [False]

infwinston · 2024-10-15T09:07:45Z

fastchat/serve/monitor/monitor.py

@@ -884,6 +884,7 @@ def build_leaderboard_tab(
    elo_results_file,
    leaderboard_table_file,
    arena_hard_leaderboard,
+    vision=True,


do we need this?

infwinston · 2024-10-15T09:09:26Z

fastchat/serve/gradio_web_server.py

+        top_p,
+        max_new_tokens,
+        request,
+    )
    get_remote_logger().log(data)


I think this might cause bug for

get_remote_logger().log(data)

? as data is not constructed anymore

infwinston · 2024-10-15T09:14:03Z

fastchat/serve/gradio_block_arena_anony.py

 )

 logger = build_logger("gradio_web_server_multi", "gradio_web_server_multi.log")

 num_sides = 2
 enable_moderation = False
+use_remote_storage = False


why do we need this be global variable? also should it be globally False?

infwinston and others added 8 commits July 5, 2024 00:17

update

a71e3c6

Use Reka Python SDK and add script for benchmarking and add send_btn (l…

68023e1

…m-sys#3413) Co-authored-by: Wei-Lin Chiang <[email protected]> Co-authored-by: Wei-Lin Chiang <[email protected]> Co-authored-by: simon-mo <[email protected]>

Store text and image moderation logs

cb4da0d

Update moderation

605add3

Run formatter

4492299

Show vote button

2723660

Fix pylint

51f9a0d

Fix pylint

38a1360

BabyChouSr marked this pull request as ready for review August 16, 2024 05:14

BabyChouSr requested a review from infwinston August 16, 2024 05:23

infwinston reviewed Aug 16, 2024

View reviewed changes

BabyChouSr added 2 commits August 16, 2024 06:35

Save bad images

e10d11b

Address comments

5159d3b

infwinston reviewed Aug 20, 2024

View reviewed changes

BabyChouSr and others added 12 commits August 27, 2024 03:07

Save moderation info per turn

dba425f

Change states

d289be9

Clean up

7911ecd

Get rid of previous moderation response

1527aac

Rename

36c67da

Enable vision arena across all tabs (lm-sys#3483)

1ccbe8b

Merge branch 'main' into moderation-log

b11f710

Merge branch 'main' into moderation-log

571f39e

Merge remote-tracking branch 'fastchat/operation-202407' into moderat…

3555d01

…ion-log

Format

fe45c6f

Merge with unified vision arena

a2200e4

Fix edge case

c90b8fc

BabyChouSr changed the base branch from operation-202407 to main October 6, 2024 03:36

BabyChouSr added 3 commits October 8, 2024 00:55

Merge

807b66f

Merge

24ce7b7

Merge

a25bd4d

BabyChouSr added 2 commits October 8, 2024 01:19

Fix

c6c284e

Format

87b6390

BabyChouSr requested a review from infwinston October 8, 2024 01:20

infwinston reviewed Oct 15, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Store the image moderation and text moderation logs #3478

Store the image moderation and text moderation logs #3478

BabyChouSr commented Aug 15, 2024

infwinston left a comment

infwinston Aug 16, 2024 •

edited

Loading

infwinston left a comment

infwinston Aug 20, 2024

BabyChouSr Aug 27, 2024

infwinston Aug 20, 2024

infwinston left a comment •

edited

Loading

infwinston Aug 27, 2024

infwinston Oct 11, 2024

infwinston Oct 15, 2024 •

edited

Loading

infwinston Oct 15, 2024

infwinston Oct 15, 2024

infwinston Oct 15, 2024

infwinston Oct 15, 2024 •

edited

Loading

infwinston Oct 15, 2024 •

edited

Loading

Store the image moderation and text moderation logs #3478

Are you sure you want to change the base?

Store the image moderation and text moderation logs #3478

Conversation

BabyChouSr commented Aug 15, 2024

infwinston left a comment

Choose a reason for hiding this comment

infwinston Aug 16, 2024 • edited Loading

Choose a reason for hiding this comment

infwinston left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

infwinston left a comment • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

infwinston Oct 15, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

infwinston Oct 15, 2024 • edited Loading

Choose a reason for hiding this comment

infwinston Oct 15, 2024 • edited Loading

Choose a reason for hiding this comment

infwinston Aug 16, 2024 •

edited

Loading

infwinston left a comment •

edited

Loading

infwinston Oct 15, 2024 •

edited

Loading

infwinston Oct 15, 2024 •

edited

Loading

infwinston Oct 15, 2024 •

edited

Loading