Do not pick gpt-4o as default model provider when it is not available as supported model in the provider model list.

[sangee2004]

gptscript version - v0.0.0-dev-faaee653-dirty

Steps to reproduce the problem:

1. Created a new chatgpt account on Free Trial
2. Execute any gptscript using this account's apikey

Script execution fails with following error since gpt-4o is used as the default provider in this case:

gptscript  --disable-cache ~/gptscript/examples/helloworld.gpt
11:01:46 started  [main]
11:01:46 sent     [main]
         content  [1] content | Waiting for model response...2024/07/29 11:01:46 error, status code: 404, message: The model `gpt-4o` does not exist or you do not have access to it.

gptscript --list-models does not list gpt-4o as expected in this case.

gptscript  --list-models                                      
babbage-002
dall-e-2
dall-e-3
davinci-002
gpt-3.5-turbo
gpt-3.5-turbo-0125
gpt-3.5-turbo-1106
gpt-3.5-turbo-16k
gpt-3.5-turbo-instruct
gpt-3.5-turbo-instruct-0914
gpt-4o-mini
gpt-4o-mini-2024-07-18
text-embedding-3-large
text-embedding-3-small
text-embedding-ada-002
tts-1
tts-1-1106
tts-1-hd
tts-1-hd-1106
whisper-1

Expected Behavior:
It should be possible for gptscript to not use gpt-4o in this case and may be prompt the user to pick a default provider that can be used as the default provider for all gptscript executions in this case.

[drpebcak]

I think failure is the right response here, but maybe a better error can be provided since this is the default.

Scripts are not really transferable between models, and especially if they don't have gpt-4o they are unlikely to have gpt-4... so they would have to use gpt-3.5 or worse. That would usually not work as expected.

[Erudition]

I came here from the getting started guide, and followed all instructions including getting an OpenAPI key, when I hit this error:

run encountered an error: error, status code: 404, message: The model gpt-4o does not exist or you do not have access to it. with error output:

The documentation never mentioned the model used, or that I would have to get access to it, or that I need to follow additional steps to make it available. Showstopper.

[drpebcak]

The only thing you need to do to get access to it is have a paid OpenAI api account, unless you have an org that has done previous configuration to limit what models are available. It is available by default to all functional paid accounts.

OpenAI does not have a free tier, so if you don't have a paid account, you won't have access to the api unless you've received credits in some other way.

[Erudition]

have a paid OpenAI api account

This was the unmentioned requirement, yes.

OpenAI does not have a free tier

Not a tier, but models - IIRC some models didn't require credits when I signed up. I guess that's gone now.

[drpebcak]

Aside from very old stuff like gpt-3-sandbox, all the gpt models have required credits in recent times as far as I am aware. OpenAI used to do a small free credit grant to new accounts that would expire after a few months, but they don't even do that anymore.

While we can certainly update the 'getting started' doc to include the phrase 'paid OpenAI API key,' I think there's a certain expectation that the payment terms for OpenAI are out of scope for our documentation. For example you don't see mentions of having to pay for an AWS account in the Terraform docs. A quick spot check of some other popular projects that use OpenAI's api don't mention paying either.

As it relates to the original issue being reported here though, we might be able to wrap this with a more helpful error. OpenAI makes this weird by delisting gpt-4o entirely but leaving other models you still can't use without paying...
We can probably safely assume that if someone is using the OpenAI provider with gpt-4o and they get a 404 back, we should instead show a helpful message about making sure their key is set up properly with an account that has access to the model they are trying to use.