CodeGate as an "Operating system" for AI

[kantord]

What is this idea about

This idea is based on a comparison of OpenRouter and CodeGate. There are some similarities between OpenRouter and CodeGate (such as being able to route requests between different models), but OpenRouter is running in the cloud and seems to be a bit more focused on developing apps, whereas CodeGate is more useful for end users.

The reason I am using "Operating system" as an analogy is because there are some features that are analogous, and positioning it as an operating system could both drive the creation of new ideas as well as make it simpler to explain CodeGate's features to users.

Some analogies:

Operating system	CodeGate
Installing applications	"Installing" models
Installing peripherals	Connecting apps to CodeGate
Choosing default App for file types	Choosing default model for different files
Collecting error logs	Collecting security logs
Measuring performance	Measuring costs
Connecting different apps through files	Connecting AI plugins with models through our pipeline
App store/package manager	List of available models through providers
File manager	Custom Prompt template manager
Antivirus	Dependency insights
Encrypted messaging	Secret leak prevention

How do I imagine this idea playing out in practice?

I would imagine that:

• We would be aiming to replace OpenRouter for "end users" that is, people who use AI for code generation and similar features. This mean leaning into cost savings features, reliability features and heuristics for choosing models. This could help us position CodeGate as an "essential tool" for managing different AI applications in complex scenarios
• We would think about ways to use this analogy to explain CodeGate to users
• We would be leaning into the analogy of it being an operating system more, and use this as inspiration for more feature ideas. This could include ideas such as understanding CodeGate integrations as "drivers" for peripherals and providing an interface to install them. Or having a "plugin store" for different plugins that integrate into CodeGate's pipeline feature

[lukehinds]

I am not sure about this, as said in slack, the real reason people use openrouter is because they want to connect to multiple service providers but only have one account to open and add credits too. when they do so, openerouter crowd sources buying the tokens, so you get them a bit cheaper then you would going direct to anthropic etc. Its value is really 'we are cheaper then going direct to openai / anthropic, by using lots of users to bulk buy at high volume'

[kantord]

I am not sure about this, as said in slack, the real reason people use openrouter is because they want to connect to multiple service providers but only have one account to open and add credits too. when they do so, openerouter crowd sources buying the tokens, so you get them a bit cheaper then you would going direct to anthropic etc. Its value is really 'we are cheaper then going direct to openai / anthropic, by using lots of users to bulk buy at high volume'

You are right, the aspect of crowd sourcing the tokens is an aspect that I didn't consider in this idea. Although - I am not sure of the business aspect here - it might as well be something we might do too 🤔

Seems like some AI apps (like Cursor) are trying to become effectively a reseller of the tokens by charging users directly. In the case of cursor, some models are covered in a monthly fee, and many other AI tools charge a flat monthly free. So a "single subscription", as in, a single flat monthly fee that supports various different AI applications at the same time might be attractive to users who are already paying 2-3 subscriptions?

[lukehinds]

Lets keep this issue open for now, it raises an interesting concept. "open"router is a very confusing name, when I first came across it I really thought it was an OSS project, but its a pretty decent revenue generating business (albeit they OSS quite a bit of their stuff)