Quarkus SQL-based RAG recipe

[jmartisk]

To test this out, add this to $HOME/.local/share/containers/podman-desktop/extensions-storage/redhat.ai-lab/user-catalog.json:

{                                                                                                                                                                                                                                         
      "id": "chatbot-java-quarkus-rag",
      "description": "This is a Quarkus-based chat demo application with RAG.",
      "name": "Java-based ChatBot (Quarkus) with RAG",
      "repository": "https://github.com/jmartisk/ai-lab-recipes",
      "ref": "quarkus-rag",
      "icon": "natural-language-processing",
      "categories": ["natural-language-processing"],
      "basedir": "recipes/natural_language_processing/chatbot-java-quarkus-rag",
      "readme": "README TBD",
      "recommended": [
        "hf.instructlab.granite-7b-lab-GGUF",
        "hf.instructlab.merlinite-7b-lab-GGUF",
        "hf.lmstudio-community.granite-3.0-8b-instruct-GGUF"
      ],
      "backend": "llama-cpp"
    }

One thing I'm not sure about is how to properly declare that the Quarkus app pod has to wait for the PostgreSQL pod to start. I've added a 10-second sleep (see the Containerfile) to the startup now, but... that's a very nasty and probably unreliable workaround.

CC @geoand, @maxandersen, @cescoffier

[slemeur]

Very cool thanks @jmartisk !
cc @jeffmaury

[rhatdan]

Waiting on @jmartisk or @jeffmaury

[jmartisk]

Is anything required from me at this point? I assume this is waiting for more reviewers' eyes. From my PoV this is ready atm.

[jeffmaury]

The recipes should following the cloud native pattern so should implement retry if the targeted service is not available or handle propertly failures so is this required ?

[jmartisk]

Unfortunately, Quarkus itself does not support retries like this (it relies on it being restarted if the database isn't ready yet) and just refuses to boot if it can't connect :( I know this is a terrible solution, but I don't have any better idea.

[jmartisk]

Or is it possible to declare somewhere (in ai-lab.yaml) that the Quarkus pod should be restarted until it boots properly?

[jeffmaury]

No because the design has been made so that there is no synchronisation between the containers that are started. There is no way to delay the connection to the database ?

[jmartisk]

@yrodiere could you advise if there is a way to let Quarkus boot even when the database is not available yet, and let Hibernate execute its database generation strategy more lazily? I checked the configuration properties and nothing came to my mind.

[yrodiere]

There's no built-in way to let db generation execute lazily -- among other reasons because that could be very dangerous (you don't want that to happen twice).

Also, booting with an offline DB is technically not supported yet (it's WIP: quarkusio/quarkus#43396), but if you're not using anything exotic it'll probably work anyway.

The way to go would be to disable DB generation/validation, and execute it explicitly through custom code relying on org.hibernate.SessionFactory#getSchemaManager .

The problem will be to detect when you should initialize the DB... Maybe Agroal provides a "first connection" event that could be used for that, I don't know. That's probably something that should be filed as a new feature, as a follow-up to 43396 -- but it's not limited to Hibernate ORM, you'd have the same problem with Flyway schema management (which, by the way, is what you should be using in prod).

[jmartisk]

Ok, in any case, none of this looks like something we want to do in this recipe that should be kept as simple as possible. IMO, AI lab should provide a way to declare dependencies between pods. Without it, I have no better idea than a dumb sleep command

[jmartisk]

Hmm, or maybe, add some piece of a shell script to wait for the database port to be open before calling /usr/local/s2i/run?? Would that be ok?