Skip to content

Commit

Permalink
environ.get
Browse files Browse the repository at this point in the history
  • Loading branch information
outtanames committed Feb 1, 2024
1 parent 65b36a0 commit c1c7c28
Show file tree
Hide file tree
Showing 4 changed files with 17 additions and 6 deletions.
Binary file added docs/assets/nos_catalog_t4_complete.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file removed docs/assets/nos_profile_list.png
Binary file not shown.
21 changes: 16 additions & 5 deletions docs/cli/profile.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,28 +4,39 @@
memory footprint and utilization on CPU/GPU.
Currently, the nos profiler itself runs natively in the execution environment (i.e. outside
the NOS server), so you'll need to install both the `server` and `test` dependencies alongside
your existing NOS installation
your existing NOS installation.

```bash
pip install torch-nos[server,test]
```

## NOS Profile Commands

### all
Profile all models

If you have the time, you can construct a profiling catalog on your machine in its entirety with:
```bash
nos profile all
```
### rebuild-catalog
Generate a fresh catalog with
```bash
nos profile rebuild-catalog
```

### model
You can also profile specific models with
```bash
nos profile method -m openai/clip-vit-large-patch14
nos profile model -m openai/clip-vit-large-patch14
```

### method
Or an entire method type with
```bash
nos profile method -m encode_image
```
to benchmark e.g. all image embedding models.

### list
Dump the nos profiling catalog with `nos profile list`

![NOS Profile List](../assets/nos_profile_list.png)
![NOS Profile List](../assets/nos_catalog_t4_complete.png)
2 changes: 1 addition & 1 deletion nos/common/spec.py
Original file line number Diff line number Diff line change
Expand Up @@ -444,7 +444,7 @@ def load_profile_catalog(self) -> "ModelSpecMetadataCatalog":
catalog_path = NOS_PROFILE_CATALOG_PATH

# Check if we have NOS_PROFILE_CATALOG_PATH_OVERRIDE in the environment
if "NOS_PROFILE_CATALOG_PATH_OVERRIDE" in os.environ:
if os.environ.get("NOS_PROFILE_CATALOG_PATH_OVERRIDE"):
catalog_path = Path(os.environ["NOS_PROFILE_CATALOG_PATH_OVERRIDE"])
logger.debug(f"Using custom profile catalog [path={catalog_path}]")

Expand Down

0 comments on commit c1c7c28

Please sign in to comment.