Skip to content

Commit

Permalink
Add --engine to allow overriding engine from command line
Browse files Browse the repository at this point in the history
Make description of tool more consistentin readme, man page and
help information.

Better explain environment variables

Signed-off-by: Daniel J Walsh <[email protected]>
  • Loading branch information
rhatdan committed Oct 15, 2024
1 parent 230b6c7 commit 728baf3
Show file tree
Hide file tree
Showing 4 changed files with 74 additions and 29 deletions.
17 changes: 11 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,12 +5,17 @@
The RamaLama project's goal is to make working with AI boring
through the use of OCI containers.

On first run RamaLama inspects your system for GPU support, falling back to CPU
support if no GPUs are present. It then uses container engines like Podman or
Docker to pull the appropriate OCI image with all of the software necessary to
run an AI Model for your systems setup. This eliminates the need for the user
to configure the system for AI themselves. After the initialization, RamaLama
will run the AI Models within a container based on the OCI image.
RamaLama tool facilitates local management and serving of AI Models.

On first run RamaLama inspects your system for GPU support, falling back to CPU support if no GPUs are present.

RamaLama uses container engines like Podman or Docker to pull the appropriate OCI image with all of the software necessary to run an AI Model for your systems setup.

Running in containers eliminates the need for users to configure the host system for AI. After the initialization, RamaLama runs the AI Models within a container based on the OCI image.

RamaLama then pulls AI Models from model registires. Starting a chatbot or a rest API service from a simple single command. Models are treated similarly to how Podman and Docker treat container images.

When both Podman and Docker are installed, RamaLama defaults to Podman, The `RAMALAMA_CONTAINER_ENGINE=docker` environment variable can override this behaviour. When neather are installed RamaLama will attempt to run the model with software on the local system.

RamaLama supports multiple AI model registries types called transports.
Supported transports:
Expand Down
25 changes: 13 additions & 12 deletions docs/ramalama.1.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,20 +9,17 @@ ramalama - Simple management tool for working with AI Models
## DESCRIPTION
RamaLama : The goal of RamaLama is to make AI boring.

On first run RamaLama inspects your system for GPU support, falling back to CPU
support if no GPUs are present. RamaLama uses container engines like Podman or
Docker to pull the appropriate OCI image with all of the software necessary to
run an AI Model for your systems setup. This eliminates the need for the user
to configure the system for AI themselves. After the initialization, RamaLama
will run the AI Models within a container based on the OCI image.
RamaLama tool facilitates local management and serving of AI Models.

RamaLama first pulls AI Models from model registires. It then start a chatbot
or a service as a rest API from a simple single command. Models are treated
similarly to the way that Podman or Docker treat container images.
On first run RamaLama inspects your system for GPU support, falling back to CPU support if no GPUs are present.

If you have both Podman and Docker installed, RamaLama defaults to Podman, use
the `RAMALAMA_CONTAINER_ENGINE=docker` environment variable to override this
behaviour.
RamaLama uses container engines like Podman or Docker to pull the appropriate OCI image with all of the software necessary to run an AI Model for your systems setup.

Running in containers eliminates the need for users to configure the host system for AI. After the initialization, RamaLama runs the AI Models within a container based on the OCI image.

RamaLama then pulls AI Models from model registires. Starting a chatbot or a rest API service from a simple single command. Models are treated similarly to how Podman and Docker treat container images.

When both Podman and Docker are installed, RamaLama defaults to Podman, The `RAMALAMA_CONTAINER_ENGINE=docker` environment variable can override this behaviour. When neather are installed RamaLama will attempt to run the model with software on the local system.

RamaLama supports multiple AI model registries types called transports. Supported transports:

Expand Down Expand Up @@ -82,6 +79,10 @@ use environment variable "RAMALAMA_IN_CONTAINER=false" to change default.
#### **--dryrun**
show container runtime command without executing it (default: False)

#### **--engine**
run RamaLama using the specified container engine.
use environment variable RAMALAMA_CONTAINER_ENGINE to modify the default behaviour.

#### **--help**, **-h**
show this help message and exit

Expand Down
58 changes: 49 additions & 9 deletions ramalama/cli.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,3 @@
from argparse import ArgumentParser
from pathlib import Path
import argparse
import glob
Expand Down Expand Up @@ -68,11 +67,42 @@ def use_container():
return True


class ArgumentParserWithDefaults(argparse.ArgumentParser):
def add_argument(self, *args, help=None, default=None, **kwargs):
if help == "==SUPPRESS==":
return
if help is not None:
kwargs['help'] = help
if default is not None and args[0] != '-h':
kwargs['default'] = default
if help is not None:
kwargs['help'] += ' (Default: {})'.format(default)
super().add_argument(*args, **kwargs)


def init_cli():
parser = ArgumentParser(
description = """\
RamaLama tool facilitates local management and serving of AI Models.
On first run RamaLama inspects your system for GPU support, falling back to CPU support if no GPUs are present.
RamaLama uses container engines like Podman or Docker to pull the appropriate OCI image with all of the software \
necessary to run an AI Model for your systems setup.
Running in containers eliminates the need for users to configure the host system for AI. After the initialization, \
RamaLama runs the AI Models within a container based on the OCI image.
RamaLama then pulls AI Models from model registires. Starting a chatbot or a rest API service from a simple single \
command. Models are treated similarly to how Podman and Docker treat container images.
When both Podman and Docker are installed, RamaLama defaults to Podman, The `RAMALAMA_CONTAINER_ENGINE=docker` \
environment variable can override this behaviour. When neather are installed RamaLama will attempt to run the model \
with software on the local system.
"""
parser = ArgumentParserWithDefaults(
prog="ramalama",
formatter_class=argparse.ArgumentDefaultsHelpFormatter,
description="Simple management tool for working with AI Models.",
description=description,
formatter_class=argparse.RawTextHelpFormatter,
)
parser.add_argument("--store", default=get_store(), help="store AI Models in the specified directory")
parser.add_argument(
Expand All @@ -84,7 +114,16 @@ def init_cli():
dest="container",
default=use_container(),
action="store_true",
help="run RamaLama in the default container",
help="""run RamaLama in the default container.
The RAMALAMA_IN_CONTAINER environment variable modifies default behaviour.""",
)
parser.add_argument(
"--engine",
dest="engine",
default=container_manager(),
action="store_true",
help="""run RamaLama using the specified container engine.
The RAMALAMA_CONTAINER_ENGINE environment variable modifies default behaviour.""",
)
parser.add_argument(
"--image",
Expand All @@ -96,7 +135,8 @@ def init_cli():
dest="container",
default=False,
action="store_false",
help="do not run RamaLama in the default container",
help="""do not run RamaLama in the default container.
The RAMALAMA_IN_CONTAINER environment variable modifies default behaviour.""",
)
parser.add_argument(
"--runtime",
Expand Down Expand Up @@ -240,7 +280,7 @@ def containers_parser(subparsers):


def _list_containers(args):
conman = container_manager()
conman = args.engine
if conman == "":
raise IndexError("no container manager (Podman, Docker) found")

Expand Down Expand Up @@ -435,7 +475,7 @@ def stop_parser(subparsers):
def _stop_container(args, name):
if not name:
raise IndexError("must specify a container name")
conman = container_manager()
conman = args.engine
if conman == "":
raise IndexError("no container manager (Podman, Docker) found")

Expand Down Expand Up @@ -533,7 +573,7 @@ def run_container(args):
if in_container():
return False

conman = container_manager()
conman = args.engine
if conman == "":
return False

Expand Down
3 changes: 1 addition & 2 deletions ramalama/model.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
import os
import sys
from ramalama.common import container_manager, exec_cmd, default_image, in_container
from ramalama.common import exec_cmd, default_image, in_container


file_not_found = """\
Expand All @@ -21,7 +21,6 @@ class Model:
"""Model super class"""

model = ""
conman = container_manager()
type = "Model"
common_params = ["-c", "2048"]

Expand Down

0 comments on commit 728baf3

Please sign in to comment.