Support local model with inference-engine mlx

[OKHand-Zy]

Enhancement: support local and custom models #165

This is a modified version of my existing code, although the code quality may not be ideal. It supports running local path models using mlx for both CLI and ChatAPI. (For now, you still need to manually place the model into the ~/.cache/exo directory before use. I'm working on automating this, but my unfamiliarity with grpc requires further research.)

I would greatly appreciate any suggestions on how to improve or optimize the code for better results.

Changes:

• Added a "How to use local models" section to the README.
• Implemented init_exo_env to configure local model cards and the local model store.
• Added bypass logic for local models using if...else statements.

[blindcrone]

I like the idea here, but think rather than rely on a folder structure this should use config files or command line arguments to specify paths to model implementations and populate things like the model card list at runtime. I'm considering refactoring the inference engine to take model implementations by default and use the shard downloader as one of a few possible routes to get weights, and I think automatically instantiating and parsing a default directory structure for this purpose creates a lot of potential for issues down the line

[OKHand-Zy]

@blindcrone
Recently, while implementing the HTTP functionality for the local model, I realized what you meant. I've switched to using aiohttp to establish an HTTP service on each node. When needed, I'll check which node has the necessary data and use an internal network to download it in chunks (similar to how exo does it). Afterward, I'll rely on the inference_engine in the command to use the model, instead of configuring it through a config file. I'm wondering if this aligns with your thoughts? If there's a better approach, I'm open to suggestions.

[OKHand-Zy]

New Updates:

• Automatic broadcasting of local model downloads is now supported.
• Added --stored-model-ip and --stored-model-port arguments.
• CLI and ChatAPI now support running local models.
• Added a 'storedhost' directory within the download directory for local model code.
• Added instructions on how to run local models to the README.

Note: This feature only supports the --inference-engine mlx option.