Chat: Error: POST /v1/chat/completions 401 - with ssl

[gruenie]

2.25.0

Intel(R) Xeon(R) CPU E3-1220 v6, 32GB RAM, Ubuntu 24.04.1, no GPU

**I have a set of localAI (AIO, docker compose) with portainer, open WebUI. In the normal use (http) all works fine so far, also when I use the chat through open WebUI.
But when I add ssl to the system, then I get problems with the chat-function. After a few prompts (anyway with which model) I get an "Error: POST /v1/chat/completions 401". After that I can't continue the chat (see screenshot).

I tried it with "nginx proxy manager" and also with "traefik" with the same results. I'm using a real wildcard-certificate, which works without any problems or error-messages also through the proxies.
I'm also using an API_KEY.**

see above

It should work both with and without SSL.
Since the problem only appears to occur with the chat function (at least text-to-image works in both cases), I suspect that it is a small bug or that you have to change settings somewhere.

2025-02-08T08:46:39.249882617Z
2025-02-08T08:46:39.249990226Z 8:46AM DBG Sending chunk: {"created":1739004394,"object":"chat.completion.chunk","id":"a08e9e59-ba0c-44b1-b4e4-9439d0fc13aa","model":"gpt-4","choices":[{"index":0,"finish_reason":"","delta":{"content":"e"}}],"usage":{"prompt_tokens":0,"completion_tokens":0,"total_tokens":0}}
2025-02-08T08:46:39.249998114Z
2025-02-08T08:46:39.250141977Z 8:46AM DBG Sending chunk: {"created":1739004394,"object":"chat.completion.chunk","id":"a08e9e59-ba0c-44b1-b4e4-9439d0fc13aa","model":"gpt-4","choices":[{"index":0,"finish_reason":"","delta":{"content":"e"}}],"usage":{"prompt_tokens":0,"completion_tokens":0,"total_tokens":0}}
2025-02-08T08:46:39.250149200Z
2025-02-08T08:46:39.250273967Z 8:46AM DBG Sending chunk: {"created":1739004394,"object":"chat.completion.chunk","id":"a08e9e59-ba0c-44b1-b4e4-9439d0fc13aa","model":"gpt-4","choices":[{"index":0,"finish_reason":"","delta":{"content":"d"}}],"usage":{"prompt_tokens":0,"completion_tokens":0,"total_tokens":0}}
2025-02-08T08:46:39.250281176Z
2025-02-08T08:46:39.456462553Z 8:46AM DBG Sending chunk: {"created":1739004394,"object":"chat.completion.chunk","id":"a08e9e59-ba0c-44b1-b4e4-9439d0fc13aa","model":"gpt-4","choices":[{"index":0,"finish_reason":"","delta":{"content":" "}}],"usage":{"prompt_tokens":0,"completion_tokens":0,"total_tokens":0}}
2025-02-08T08:46:39.456484809Z
2025-02-08T08:46:39.456722631Z 8:46AM DBG Sending chunk: {"created":1739004394,"object":"chat.completion.chunk","id":"a08e9e59-ba0c-44b1-b4e4-9439d0fc13aa","model":"gpt-4","choices":[{"index":0,"finish_reason":"","delta":{"content":"a"}}],"usage":{"prompt_tokens":0,"completion_tokens":0,"total_tokens":0}}
2025-02-08T08:46:39.456731356Z
2025-02-08T08:46:39.456824308Z 8:46AM DBG Sending chunk: {"created":1739004394,"object":"chat.completion.chunk","id":"a08e9e59-ba0c-44b1-b4e4-9439d0fc13aa","model":"gpt-4","choices":[{"index":0,"finish_reason":"","delta":{"content":"s"}}],"usage":{"prompt_tokens":0,"completion_tokens":0,"total_tokens":0}}
2025-02-08T08:46:39.456834802Z
2025-02-08T08:46:39.456912889Z 8:46AM DBG Sending chunk: {"created":1739004394,"object":"chat.completion.chunk","id":"a08e9e59-ba0c-44b1-b4e4-9439d0fc13aa","model":"gpt-4","choices":[{"index":0,"finish_reason":"","delta":{"content":"s"}}],"usage":{"prompt_tokens":0,"completion_tokens":0,"total_tokens":0}}
2025-02-08T08:46:39.456935284Z
2025-02-08T08:46:39.457113558Z 8:46AM DBG Sending chunk: {"created":1739004394,"object":"chat.completion.chunk","id":"a08e9e59-ba0c-44b1-b4e4-9439d0fc13aa","model":"gpt-4","choices":[{"index":0,"finish_reason":"","delta":{"content":"i"}}],"usage":{"prompt_tokens":0,"completion_tokens":0,"total_tokens":0}}
2025-02-08T08:46:39.457130923Z
2025-02-08T08:46:39.457861658Z 8:46AM DBG Sending chunk: {"created":1739004394,"object":"chat.completion.chunk","id":"a08e9e59-ba0c-44b1-b4e4-9439d0fc13aa","model":"gpt-4","choices":[{"index":0,"finish_reason":"","delta":{"content":"s"}}],"usage":{"prompt_tokens":0,"completion_tokens":0,"total_tokens":0}}
2025-02-08T08:46:39.457882329Z
2025-02-08T08:46:39.457885229Z 8:46AM DBG Sending chunk: {"created":1739004394,"object":"chat.completion.chunk","id":"a08e9e59-ba0c-44b1-b4e4-9439d0fc13aa","model":"gpt-4","choices":[{"index":0,"finish_reason":"","delta":{"content":"t"}}],"usage":{"prompt_tokens":0,"completion_tokens":0,"total_tokens":0}}
2025-02-08T08:46:39.457888648Z
2025-02-08T08:46:39.457890807Z 8:46AM DBG Sending chunk: {"created":1739004394,"object":"chat.completion.chunk","id":"a08e9e59-ba0c-44b1-b4e4-9439d0fc13aa","model":"gpt-4","choices":[{"index":0,"finish_reason":"","delta":{"content":"a"}}],"usage":{"prompt_tokens":0,"completion_tokens":0,"total_tokens":0}}
2025-02-08T08:46:39.457893888Z
2025-02-08T08:46:39.457895986Z 8:46AM DBG Sending chunk: {"created":1739004394,"object":"chat.completion.chunk","id":"a08e9e59-ba0c-44b1-b4e4-9439d0fc13aa","model":"gpt-4","choices":[{"index":0,"finish_reason":"","delta":{"content":"n"}}],"usage":{"prompt_tokens":0,"completion_tokens":0,"total_tokens":0}}
2025-02-08T08:46:39.457899000Z
2025-02-08T08:46:39.457901180Z 8:46AM DBG Sending chunk: {"created":1739004394,"object":"chat.completion.chunk","id":"a08e9e59-ba0c-44b1-b4e4-9439d0fc13aa","model":"gpt-4","choices":[{"index":0,"finish_reason":"","delta":{"content":"c"}}],"usage":{"prompt_tokens":0,"completion_tokens":0,"total_tokens":0}}
2025-02-08T08:46:39.457904253Z
2025-02-08T08:46:39.457906441Z 8:46AM DBG Sending chunk: {"created":1739004394,"object":"chat.completion.chunk","id":"a08e9e59-ba0c-44b1-b4e4-9439d0fc13aa","model":"gpt-4","choices":[{"index":0,"finish_reason":"","delta":{"content":"e"}}],"usage":{"prompt_tokens":0,"completion_tokens":0,"total_tokens":0}}
2025-02-08T08:46:39.457909504Z
2025-02-08T08:46:39.662625159Z 8:46AM DBG Sending chunk: {"created":1739004394,"object":"chat.completion.chunk","id":"a08e9e59-ba0c-44b1-b4e4-9439d0fc13aa","model":"gpt-4","choices":[{"index":0,"finish_reason":"","delta":{"content":","}}],"usage":{"prompt_tokens":0,"completion_tokens":0,"total_tokens":0}}
2025-02-08T08:46:39.662649381Z
2025-02-08T08:46:39.870289684Z 8:46AM DBG Sending chunk: {"created":1739004394,"object":"chat.completion.chunk","id":"a08e9e59-ba0c-44b1-b4e4-9439d0fc13aa","model":"gpt-4","choices":[{"index":0,"finish_reason":"","delta":{"content":" "}}],"usage":{"prompt_tokens":0,"completion_tokens":0,"total_tokens":0}}
2025-02-08T08:46:39.870314781Z
2025-02-08T08:46:39.870319602Z 8:46AM DBG Sending chunk: {"created":1739004394,"object":"chat.completion.chunk","id":"a08e9e59-ba0c-44b1-b4e4-9439d0fc13aa","model":"gpt-4","choices":[{"index":0,"finish_reason":"","delta":{"content":"f"}}],"usage":{"prompt_tokens":0,"completion_tokens":0,"total_tokens":0}}
2025-02-08T08:46:39.870323578Z
2025-02-08T08:46:39.870325809Z 8:46AM DBG Sending chunk: {"created":1739004394,"object":"chat.completion.chunk","id":"a08e9e59-ba0c-44b1-b4e4-9439d0fc13aa","model":"gpt-4","choices":[{"index":0,"finish_reason":"","delta":{"content":"e"}}],"usage":{"prompt_tokens":0,"completion_tokens":0,"total_tokens":0}}
2025-02-08T08:46:39.870339258Z
2025-02-08T08:46:39.870470901Z 8:46AM DBG Sending chunk: {"created":1739004394,"object":"chat.completion.chunk","id":"a08e9e59-ba0c-44b1-b4e4-9439d0fc13aa","model":"gpt-4","choices":[{"index":0,"finish_reason":"","delta":{"content":"e"}}],"usage":{"prompt_tokens":0,"completion_tokens":0,"total_tokens":0}}
2025-02-08T08:46:39.870480967Z
2025-02-08T08:46:39.870784738Z 8:46AM DBG Sending chunk: {"created":1739004394,"object":"chat.completion.chunk","id":"a08e9e59-ba0c-44b1-b4e4-9439d0fc13aa","model":"gpt-4","choices":[{"index":0,"finish_reason":"","delta":{"content":"l"}}],"usage":{"prompt_tokens":0,"completion_tokens":0,"total_tokens":0}}
2025-02-08T08:46:39.984598589Z
2025-02-08T08:46:40.088960193Z 8:46AM DBG Sending chunk: {"created":1739004394,"object":"chat.completion.chunk","id":"a08e9e59-ba0c-44b1-b4e4-9439d0fc13aa","model":"gpt-4","choices":[{"index":0,"finish_reason":"","delta":{"content":" "}}],"usage":{"prompt_tokens":0,"completion_tokens":0,"total_tokens":0}}
2025-02-08T08:46:40.088985805Z
2025-02-08T08:46:40.089118775Z 8:46AM DBG Sending chunk: {"created":1739004394,"object":"chat.completion.chunk","id":"a08e9e59-ba0c-44b1-b4e4-9439d0fc13aa","model":"gpt-4","choices":[{"index":0,"finish_reason":"","delta":{"content":"f"}}],"usage":{"prompt_tokens":0,"completion_tokens":0,"total_tokens":0}}
2025-02-08T08:46:40.089134984Z
2025-02-08T08:46:40.089296965Z 8:46AM DBG Sending chunk: {"created":1739004394,"object":"chat.completion.chunk","id":"a08e9e59-ba0c-44b1-b4e4-9439d0fc13aa","model":"gpt-4","choices":[{"index":0,"finish_reason":"","delta":{"content":"r"}}],"usage":{"prompt_tokens":0,"completion_tokens":0,"total_tokens":0}}
2025-02-08T08:46:40.089311956Z
2025-02-08T08:46:40.089315430Z 8:46AM DBG Sending chunk: {"created":1739004394,"object":"chat.completion.chunk","id":"a08e9e59-ba0c-44b1-b4e4-9439d0fc13aa","model":"gpt-4","choices":[{"index":0,"finish_reason":"","delta":{"content":"e"}}],"usage":{"prompt_tokens":0,"completion_tokens":0,"total_tokens":0}}
2025-02-08T08:46:40.089319616Z
2025-02-08T08:46:40.089434486Z 8:46AM DBG Sending chunk: {"created":1739004394,"object":"chat.completion.chunk","id":"a08e9e59-ba0c-44b1-b4e4-9439d0fc13aa","model":"gpt-4","choices":[{"index":0,"finish_reason":"","delta":{"content":"e"}}],"usage":{"prompt_tokens":0,"completion_tokens":0,"total_tokens":0}}
2025-02-08T08:46:40.089443632Z
2025-02-08T08:46:40.310646427Z 8:46AM DBG Sending chunk: {"created":1739004394,"object":"chat.completion.chunk","id":"a08e9e59-ba0c-44b1-b4e4-9439d0fc13aa","model":"gpt-4","choices":[{"index":0,"finish_reason":"","delta":{"content":" "}}],"usage":{"prompt_tokens":0,"completion_tokens":0,"total_tokens":0}}
2025-02-08T08:46:40.310675695Z
2025-02-08T08:46:40.310908467Z 8:46AM DBG Sending chunk: {"created":1739004394,"object":"chat.completion.chunk","id":"a08e9e59-ba0c-44b1-b4e4-9439d0fc13aa","model":"gpt-4","choices":[{"index":0,"finish_reason":"","delta":{"content":"t"}}],"usage":{"prompt_tokens":0,"completion_tokens":0,"total_tokens":0}}
2025-02-08T08:46:40.310930984Z
2025-02-08T08:46:40.311026039Z 8:46AM DBG Sending chunk: {"created":1739004394,"object":"chat.completion.chunk","id":"a08e9e59-ba0c-44b1-b4e4-9439d0fc13aa","model":"gpt-4","choices":[{"index":0,"finish_reason":"","delta":{"content":"o"}}],"usage":{"prompt_tokens":0,"completion_tokens":0,"total_tokens":0}}
2025-02-08T08:46:40.311034102Z
2025-02-08T08:46:40.517102023Z 8:46AM DBG Sending chunk: {"created":1739004394,"object":"chat.completion.chunk","id":"a08e9e59-ba0c-44b1-b4e4-9439d0fc13aa","model":"gpt-4","choices":[{"index":0,"finish_reason":"","delta":{"content":" "}}],"usage":{"prompt_tokens":0,"completion_tokens":0,"total_tokens":0}}
2025-02-08T08:46:40.517123391Z
2025-02-08T08:46:40.517126375Z 8:46AM DBG Sending chunk: {"created":1739004394,"object":"chat.completion.chunk","id":"a08e9e59-ba0c-44b1-b4e4-9439d0fc13aa","model":"gpt-4","choices":[{"index":0,"finish_reason":"","delta":{"content":"a"}}],"usage":{"prompt_tokens":0,"completion_tokens":0,"total_tokens":0}}
2025-02-08T08:46:40.517130029Z
2025-02-08T08:46:40.517132292Z 8:46AM DBG Sending chunk: {"created":1739004394,"object":"chat.completion.chunk","id":"a08e9e59-ba0c-44b1-b4e4-9439d0fc13aa","model":"gpt-4","choices":[{"index":0,"finish_reason":"","delta":{"content":"s"}}],"usage":{"prompt_tokens":0,"completion_tokens":0,"total_tokens":0}}
2025-02-08T08:46:40.517135521Z
2025-02-08T08:46:40.517137706Z 8:46AM DBG Sending chunk: {"created":1739004394,"object":"chat.completion.chunk","id":"a08e9e59-ba0c-44b1-b4e4-9439d0fc13aa","model":"gpt-4","choices":[{"index":0,"finish_reason":"","delta":{"content":"k"}}],"usage":{"prompt_tokens":0,"completion_tokens":0,"total_tokens":0}}
2025-02-08T08:46:40.517141018Z
2025-02-08T08:46:40.732151043Z 8:46AM DBG Sending chunk: {"created":1739004394,"object":"chat.completion.chunk","id":"a08e9e59-ba0c-44b1-b4e4-9439d0fc13aa","model":"gpt-4","choices":[{"index":0,"finish_reason":"","delta":{"content":"."}}],"usage":{"prompt_tokens":0,"completion_tokens":0,"total_tokens":0}}
2025-02-08T08:46:40.732178545Z
2025-02-08T08:46:40.939685498Z 8:46AM DBG Sending chunk: {"created":1739004394,"object":"chat.completion.chunk","id":"a08e9e59-ba0c-44b1-b4e4-9439d0fc13aa","model":"gpt-4","choices":[{"index":0,"finish_reason":"","delta":{"content":""}}],"usage":{"prompt_tokens":0,"completion_tokens":0,"total_tokens":0}}
2025-02-08T08:46:40.939703195Z
2025-02-08T08:46:40.939909074Z 8:46AM DBG Sending chunk: {"created":1739004394,"object":"chat.completion.chunk","id":"a08e9e59-ba0c-44b1-b4e4-9439d0fc13aa","model":"gpt-4","choices":[{"index":0,"finish_reason":"","delta":{"content":""}}],"usage":{"prompt_tokens":10,"completion_tokens":24,"total_tokens":34}}
2025-02-08T08:46:40.939926690Z
2025-02-08T08:46:40.940065250Z 8:46AM DBG GRPC(gpt-4-127.0.0.1:44291): stdout {"timestamp":1739004400,"level":"INFO","function":"print_timings","line":327,"message":"prompt eval time = 1047.75 ms / 10 tokens ( 104.77 ms per token, 9.54 tokens per second)","slot_id":0,"task_id":93,"t_prompt_processing":1047.748,"num_prompt_tokens_processed":10,"t_token":104.7748,"n_tokens_second":9.54427973138579}
2025-02-08T08:46:40.940639687Z 8:46AM DBG GRPC(gpt-4-127.0.0.1:44291): stdout {"timestamp":1739004400,"level":"INFO","function":"print_timings","line":341,"message":"generation eval time = 4921.64 ms / 24 runs ( 205.07 ms per token, 4.88 tokens per second)","slot_id":0,"task_id":93,"t_token_generation":4921.642,"n_decoded":24,"t_token":205.06841666666665,"n_tokens_second":4.876421324427905}
2025-02-08T08:46:40.940774889Z 8:46AM DBG GRPC(gpt-4-127.0.0.1:44291): stdout {"timestamp":1739004400,"level":"INFO","function":"print_timings","line":351,"message":" total time = 5969.39 ms","slot_id":0,"task_id":93,"t_prompt_processing":1047.748,"t_token_generation":4921.642,"t_total":5969.389999999999}
2025-02-08T08:46:40.940784330Z 8:46AM DBG GRPC(gpt-4-127.0.0.1:44291): stdout {"timestamp":1739004400,"level":"INFO","function":"update_slots","line":1634,"message":"slot released","slot_id":0,"task_id":93,"n_ctx":8192,"n_past":33,"n_system_tokens":0,"n_cache_tokens":34,"truncated":false}
2025-02-08T08:46:58.178315656Z 8:46AM WRN Client error ip=172.19.0.2 latency="52.943µs" method=POST status=401 url=/v1/chat/completions
2025-02-08T08:47:23.712593694Z 8:47AM INF Success ip=127.0.0.1 latency="9.977µs" method=GET status=200 url=/readyz
2025-02-08T08:48:23.769213235Z 8:48AM INF Success ip=127.0.0.1 latency="9.803µs" method=GET status=200 url=/readyz

Additional context

[Hello-World-Traveler]

Same issue but with GPU (localai/localai:latest-aio-gpu-nvidia-cuda-12 LocalAI Version v2.26.0)

All I see is 9:17PM WRN Client error ip= latency="132.021µs" method=POST status=401 url=/v1/chat/completions
models: falcon3-10b-instruct, llava-v1.6-7b-mmproj-f16

Unauthorized error?

DBG No configuration file found at /tmp/localai/upload/uploadedFiles.json

9:16PM DBG No configuration file found at /tmp/localai/config/assistants.json

9:16PM DBG No configuration file found at /tmp/localai/config/assistantsFile.json

[gruenie]

After experimenting a bit more, it seems that the problem described only occurs in the combination of SSL and setting the API_KEY.
With activated API_KEY and SSL (via Traefik) the error message appears. Without API_KEY there is no problem with or without SSL.
So it seems that authentication with SSL is somehow going wrong.

[Hello-World-Traveler]

After experimenting a bit more, it seems that the problem described only occurs in the combination of SSL and setting the API_KEY. With activated API_KEY and SSL (via Traefik) the error message appears. Without API_KEY there is no problem with or without SSL. So it seems that authentication with SSL is somehow going wrong.

Removed API key, no proxy and no SSL, no difference for me. Models are hit and miss.

[jganseman]

Experiencing the same problem, using Caddy. The caddy logs (docker) don't show anything out of the ordinary. The LocalAI logs (also docker) show:

9:26AM INF Success ip=192.168.0.1 latency="523.191µs" method=GET status=200 url=/chat/smollm2-1.7b-instruct
9:26AM WRN Client error ip=192.168.0.1 latency="70.674µs" method=POST status=401 url=/v1/chat/completions
9:26AM INF Success ip=192.168.0.1 latency="406.431µs" method=GET status=200 url=/chat/smollm2-1.7b-instruct
9:26AM INF Success ip=192.168.0.1 latency="475.06µs" method=POST status=200 url=/v1/chat/completions

The WRN indicates the Error: POST /v1/chat/completions 401 event. Sometimes after a reload it does suddenly work, for 1 question, but then it fails on a followup question. I guess only about 10% of requests comes through OK.

Any other logs I could take a look at?

Update: working locally on HTTP IP addresss only and bypassing Caddy / SSL, the problem seems to persist. From my end it looks like the issue seems to stem from API_KEY functionality rather than SSL?

[Hello-World-Traveler]

I used to have the API key set now when I hit an error, I restart and it works. the models that didn't work before now work.
I don't know if it's the API or restart for the back-end to reload.