Standardizing AI in WordPress

[felixarntz]

While I've been working on AI Services as an infrastructure for using various LLMs in WordPress, in a plugin like this it can only go so far. Therefore I'm opening this issue for discussion on some alternative paths forward. How can we get to standardization of AI in WordPress in a way that's best aligned with the project's philosophies but also unlocks it for a large chunk of the WordPress ecosystem?

I'm going to start by outlining some considerations and ideas based on them, looking forward to getting other people's thoughts and discussing from there. Apologies for the wall of text, but I want to make sure I provide enough context for what I'm thinking, to kickstart a good discussion.

General considerations

Some general considerations about this plugin specifically and a potential future with AI in WordPress Core:

• While the API from this plugin could potentially be merged into Core more or less as-is, the proprietary and (typically) commercial nature of AI services may not be a good fit for WordPress Core given its philosophies.
  • Potentially the API could be merged without any actual service implementations being merged, with all of those being plugins. This would avoid WordPress Core being opinionated in any way about specific services to support.
  • The downside is that the pure API itself could not be used for actual AI features, so WordPress Core could not receive any AI features unless one of the plugins that implement support for an AI service would be installed and active.
• While a WordPress Core merge could certainly be explored, we also need to consider the "before WordPress Core" state where we currently are. A quick thought would be that this should become a canonical plugin. And while I agree that this would be a great step, it doesn't necessarily satisfy all the needs of the WordPress ecosystem.
• For example, even though plugin dependencies are now supported in WordPress Core, this is not feasible to use for every plugin:
  • Some plugins may support older versions of WordPress.
  • Some plugins may not want to formally depend on another plugin in a way that the user sees it and has to be concerned with managing that dependency.
• Other plugins that want to leverage the APIs that the AI Services plugin provides without having it as an actual plugin dependency can currently not really use it. The AI Services plugin can only be used as a formal dependency, or as an "optional" dependency where a certain feature of another plugin only works if AI Services is installed and active.
• To make the unified AI APIs available for those other plugins, another implementation avenue would be needed, for instance a library that could e.g. be bundled via Composer.
• An additional thought when thinking about libraries is the broader PHP ecosystem: Since there is no popular tool available in the PHP ecosystem that provides a common API abstraction for LLMs (at least not that I'm aware of), it could be worth exploring even a WordPress-agnostic library.

Potential plans towards AI standardization

Based on these considerations, let's consider two possible options:

• Either an AI API abstraction layer like the one from the AI Services plugin is soon feasible to merge into WordPress Core.
• Or the consensus is that it needs more time for the (young!) AI ecosystem to evolve, and AI should remain plugin territory for now.

For the first option, the AI Services plugin could effectively become a feature plugin. For the second option though, it only satisfies the needs for part of the WordPress plugin ecosystem, as mentioned before, so we should consider the idea of breaking certain parts of the foundational API into separate libraries. Last but not least, the ideal approach would be one where the relevant APIs could in the future still be merged into WordPress Core in a way that plugins built on top of the AI Services plugin or the (potential future) libraries could easily transition to the WordPress Core built-in solution.

Here's an idea for the restructuring of the AI Services plugin into libraries:

• The most foundational piece could be a WordPress-agnostic PHP library (Composer package) that provides an API abstraction for LLMs.
  • Much of the relevant API surface in the plugin is already WordPress-agnostic, so it would not be a terribly huge lift to make it entirely WordPress-agnostic and aligned with patterns common among other PHP projects.
• Built on top of that PHP library, there could be another PHP library specific to WordPress (another Composer package): It would "translate" the PHP generic API abstraction layer to be more aligned with "the WordPress way", but more importantly it would add WordPress-specific functionality on top, such as the REST API endpoints which allow using the LLMs client-side in JavaScript, which is equally, if not even more important for WordPress AI use-cases. The library could also centrally handle the storage and possibly even administrative UI for API credentials.
  • The WordPress-specific PHP library could be used by any plugin, and multiple plugins could bundle it and coexist without any conflicts. This could be achieved by namespace-prefixing the third-party code (already a recommended pattern for bundling any third-party code in WordPress plugins) and the REST API endpoints being exposed under a plugin-specific namespace.
  • In other words, each plugin would rely on its own bundled versions and its own REST API endpoints. That of course does not sound great from a code reusability perspective, but this is the reality of the WordPress ecosystem today.
• In addition to the WordPress-specific PHP library, there should be a WordPress-specific JavaScript library (NPM package) that contains the JavaScript client implementation of the AI API abstraction, covering both on-device AI as well as the server-side registered LLMs - the latter via the REST API endpoints from the WordPress-specific PHP library.
  • In other words, the WordPress PHP library would be a requirement to use the WordPress JavaScript library - or alternatively at least the REST API endpoint client logic of the WordPress JavaScript library could only be used by a plugin if it was also using the WordPress PHP library.
• Despite these libraries potentially being bundled into multiple plugins that are active at the same time, certain aspects could still be centrally handled and only loaded once (e.g. a central admin screen for API credentials), to avoid UI fragmentation.
  • Alternatively, only the storage mechanism is shared and plugins would implement the UI themselves in their own administrative area. This would still avoid the need to paste the same API key in multiple places, as they would be synced due to the shared storage mechanism.
• The AI Services plugin could still exist as an easy way to use all of these, and it could be used in the same way it is today, with its own features on top such as its "AI playground" (an admin screen to manually send AI prompts and explore/compare outputs, which I'm currently working on).

Examples

The above explanation is probably best clarified with examples. To summarize, we'd have the following components:

• 🦿 AI abstraction PHP library
• 🇼 WordPress-specific AI PHP library
• 📱 WordPress-specific AI JavaScript library
• 🤖 AI Services WordPress plugin

The dependency tree between these components would be as follows:

• 🤖 AI Services WordPress plugin
  • 🇼 WordPress-specific AI PHP library
    • 🦿 AI abstraction PHP library
  • 📱 WordPress-specific AI JavaScript library

A WordPress plugin that wants to make use of the APIs without depending on the AI Services plugin could use the same dependencies:

• ⚙️ Some third-party plugin
  • 🇼 WordPress-specific AI PHP library
    • 🦿 AI abstraction PHP library
  • 📱 WordPress-specific AI JavaScript library

Plugins that don't mind having another plugin as dependency could rely on AI Services as they can today.

And last but not least, even projects outside of the WordPress ecosystem could benefit from the API abstraction, depending purely on the WordPress-agnostic PHP library:

• Some other PHP CMS, PHP extension, or PHP CLI tool
  • 🦿 AI abstraction PHP library

Interoperability with potential AI abstraction in WordPress Core

The above proposal would lead to minimum disruption, should such the API eventually be merged into WordPress Core:

• WordPress plugins using the WordPress-specific PHP API could transition from the library's namespace to WordPress Core's classes, and then remove the bundled PHP library.
• WordPress plugins using the WordPress-specific JS API could transition to its similar implementation in WordPress Core, then remove the bundled JS and PHP libraries.
• The REST API endpoints themselves would simply transition from each plugin's specific namespace to a similar but unified implementation under WordPress Core's REST API namespace.

---

With all this as a starting point - let's discuss, I'd love to get your thoughts.

[joehoyle]

I'd be in favor of more iteration and innovation via a plugin than thinking about Core merging at this point. I'm not totally up to speed on the project, but from what I've seen so far I think is great progress to abstracting LLMs for low-level APIs, but there's still a lot of opportunity in the CMS agents side of things (e.g. Drupal's current approach / AI architecture).

I'd still like to see more architectural advancement to see the potential scope and future of this project, to understand if it's where the "How does WordPress move towards a first class AI CMS". I don't think this project may ever include any direct AI implementations, and that would be fine, but I do feel there's a lot of work to do in scoping out APIs and concepts to essentially scaffold the CMS at large into services, agents, RAG and more, and still allow for a budding ecosystem of user experiences to be built on top of that.

So, I'm wondering if the goal is to standardize AI APIs (OpenAI, etc) into a standard code library (much the same as any LLM-agnostic framework out there), or is the project more concerned with our the CMS can be deeply integrated with AI (which is where we're more getting into agents, and deeper AI architecture).?

[Jameswlepage]

I’m basically in agreement with everything you mentioned in the OP, Felix.

I love an atomic approach, where there are many smaller packages working together inside of a bigger system.

Building this out as modular composer packages; a lower level PHP implementation, consisting of multiple ‘adapters’ for each major AI provider, a WordPress library, and finally a plugin housing all of this would get my vote.

PHP desperately needs something like the AI SDK by Vercel; mainly that Unified Provider API. I could see this work benefiting the entire industry because it just doesn’t exist. When we were at WPAI, we kept bumping into this issue. We had a Plugin and Backend Service (Laravel), and started thinking about building our own called ElleLLM (Greg mentioned this in his issue).

On the WordPress side of things, standardization would be immediately helpful for plugin developers trying to wrestle with multiple providers, API keys (BYOK or not), and streaming. A composer package + NPM for client side / FSE stuff would be immediately helpful on this front and ideally have some contribution from multiple teams & ppl to accelerate it to maturity.

For consumers and as a preview, surfacing everything in a Plugin that’s treated almost as an experimental canonical, like Performance Lab would also be smart IMO.

I think AI is changing too quickly for any implementation to be considered for core ATM. There’s also the ‘3rd party provider’ question - basically needed for any valuable implementation as of Dec 2024, but goes again the mentality of Core.

The landscape is changing, and it’s quickly becoming more feasible to run models on-server, on-device etc etc.

Having that plugin as a showcase and grounds for POCs/experimentation, while adopting the modular library approach allows us to quickly keep up with industry changes, build in parallel to the WP project, and eventually take a step back, and consider what (if anything) should make it to core.

What are next steps :)?

[felixarntz]

@joehoyle Some inline respones:

I'd be in favor of more iteration and innovation via a plugin than thinking about Core merging at this point. I'm not totally up to speed on the project, but from what I've seen so far I think is great progress to abstracting LLMs for low-level APIs, but there's still a lot of opportunity in the CMS agents side of things (e.g. Drupal's current approach / AI architecture).

Yeah, that's where my head is at as well. I mention a Core merge as potential future opportunity, but agree it is too early given the rapidly evolving AI landscape. For instance, with the emergence of on-device AI maybe long-term WordPress Core would be able to have built-in support for such options, thus making usage of commercial AI cloud services optional - they would probably still be relevant, but at least WordPress Core would be able to provide more than just a technical foundation if it could use e.g. browser built-in AI.

So, I'm wondering if the goal is to standardize AI APIs (OpenAI, etc) into a standard code library (much the same as any LLM-agnostic framework out there), or is the project more concerned with our the CMS can be deeply integrated with AI (which is where we're more getting into agents, and deeper AI architecture).?

It's mostly about the first. With this effort, I'd like to build an AI API abstraction framework (abstracted so that it works both with REST APIs from cloud services as well as e.g. browser native APIs). So this would be a foundation for using AI, without being opinionated on how it should be used or more deeply integrated. That would be something that could be built on top of this framework, but out of scope for the framework itself.

[felixarntz]

@Jameswlepage Thank you, these additional thoughts are a helpful confirmation of the proposed direction.

I think AI is changing too quickly for any implementation to be considered for core ATM. There’s also the ‘3rd party provider’ question - basically needed for any valuable implementation as of Dec 2024, but goes again the mentality of Core.

100%, I also replied that in my previous response - the more on-device or on-server AI solutions are available and capable, the more realistic IMO it would be to propose some kind of AI support directly in WordPress Core. But it's too early for now for sure.

Having that plugin as a showcase and grounds for POCs/experimentation, while adopting the modular library approach allows us to quickly keep up with industry changes, build in parallel to the WP project, and eventually take a step back, and consider what (if anything) should make it to core.

Yeah, I think a key aspect here is to design these API abstractions in a way that ideally they could later be implemented in a similar way in WordPress Core, and to minimize the potential future cost of transitioning from the early APIs to the WordPress Core APIs. I think with the modular approach and the WordPress-specific wrapper PHP package we would be well set up for that.

What are next steps :)?

I think we should start with the WordPress-agnostic PHP package, as that is the lowest-level piece. We can develop that maybe in a new open-source GitHub org, just so that it doesn't live in my personal account and anyone can more easily get involved. I think we can discuss the architecture in an issue - and last but not least we'll need to think about a good name for the package.

I'll set something up, so we can really kick this off in the new year.