|
| 1 | +Contribution |
| 2 | +============ |
| 3 | + |
| 4 | +## Contributing to Intel® Extension for PyTorch\* |
| 5 | + |
| 6 | +Thank you for your interest in contributing to Intel® Extension for PyTorch\*! Before you begin writing code, it is important that you share your intention to contribute with the team, based on the type of contribution: |
| 7 | + |
| 8 | +1. You want to propose a new feature and implement it. |
| 9 | + - Post about your intended feature in an [issue](https://github.com/intel/intel-extension-for-pytorch/issues), and we shall discuss the design and implementation. Once we agree that the plan looks good, go ahead and implement it. |
| 10 | +2. You want to implement a feature or bug-fix for an outstanding issue. |
| 11 | + - Search for your issue in the [issue list](https://github.com/intel/intel-extension-for-pytorch/issues). |
| 12 | + - Pick an issue and comment that you'd like to work on the feature or bug-fix. |
| 13 | + - If you need more context on a particular issue, please ask and we shall provide. |
| 14 | + |
| 15 | +Once you implement and test your feature or bug-fix, please submit a Pull Request to https://github.com/intel/intel-extension-for-pytorch. |
| 16 | + |
| 17 | +## Developing Intel® Extension for PyTorch\* |
| 18 | + |
| 19 | +A full set of instructions on installing Intel® Extension for PyTorch\* from source is here: |
| 20 | +https://github.com/intel/intel-extension-for-pytorch#install-extension-by-compiling-from-source |
| 21 | + |
| 22 | +To develop on your machine, here are some tips: |
| 23 | + |
| 24 | +1. Uninstall all existing Intel® Extension for PyTorch\* installs. You may need to run `pip uninstall intel_extension_for_pytorch` multiple times. You'll know `intel_extension_for_pytorch` is fully uninstalled when you see `WARNING: Skipping intel_extension_for_pytorch as it is not installed`. (You should only have to `pip uninstall` a few times, but you can always `uninstall` with `timeout` or in a loop if you're feeling lazy.) |
| 25 | + |
| 26 | +```bash |
| 27 | +yes | pip uninstall intel_extension_for_pytorch |
| 28 | +``` |
| 29 | + |
| 30 | +2. Clone a copy of Intel® Extension for PyTorch\* from source: |
| 31 | + |
| 32 | +```bash |
| 33 | +git clone https://github.com/intel/intel-extension-for-pytorch.git |
| 34 | +cd intel-extension-for-pytorch |
| 35 | +``` |
| 36 | + |
| 37 | +2.1. If you already have Intel® Extension for PyTorch\* from source, update it: |
| 38 | + |
| 39 | +```bash |
| 40 | +git pull --rebase |
| 41 | +git submodule sync --recursive |
| 42 | +git submodule update --init --recursive --jobs 0 |
| 43 | +``` |
| 44 | + |
| 45 | +If you want to have no-op incremental rebuilds (which are fast), see the section below titled "Make no-op build fast." |
| 46 | + |
| 47 | +3. Install Intel® Extension for PyTorch\* in `develop` mode: |
| 48 | + |
| 49 | +The change you have to make is to replace |
| 50 | + |
| 51 | +```bash |
| 52 | +python setup.py install |
| 53 | +``` |
| 54 | + |
| 55 | +with |
| 56 | + |
| 57 | +```bash |
| 58 | +python setup.py develop |
| 59 | +``` |
| 60 | + |
| 61 | +This mode will symlink the Python files from the current local source tree into the Python install. Hence, if you modify a Python file, you do not need to reinstall PyTorch again and again. This is especially useful if you are only changing Python files. |
| 62 | + |
| 63 | +For example: |
| 64 | +- Install local Intel® Extension for PyTorch\* in `develop` mode |
| 65 | +- modify your Python file `intel_extension_for_pytorch/__init__.py` (for example) |
| 66 | +- test functionality |
| 67 | +- modify your Python file `intel_extension_for_pytorch/__init__.py` |
| 68 | +- test functionality |
| 69 | +- modify your Python file `intel_extension_for_pytorch/__init__.py` |
| 70 | +- test functionality |
| 71 | + |
| 72 | +You do not need to repeatedly install after modifying Python files (`.py`). However, you would need to reinstall if you modify Python interface (`.pyi`, `.pyi.in`) or non-Python files (`.cpp`, `.cc`, `.cu`, `.h`, ...). |
| 73 | + |
| 74 | +In case you want to reinstall, make sure that you uninstall Intel® Extension for PyTorch\* first by running `pip uninstall intel_extension_for_pytorch` until you see `WARNING: Skipping intel_extension_for_pytorch as it is not installed`; next run `python setup.py clean`. After that, you can install in `develop` mode again. |
| 75 | + |
| 76 | +### Tips and Debugging |
| 77 | + |
| 78 | +* A prerequisite to installing Intel® Extension for PyTorch\* is CMake. We recommend installing it with [Homebrew](https://brew.sh/) with `brew install cmake` if you are developing on MacOS or Linux system. |
| 79 | +* Our `setup.py` requires Python >= 3.6 |
| 80 | +* If you run into errors when running `python setup.py develop`, here are some debugging steps: |
| 81 | + 1. Run `printf '#include <stdio.h>\nint main() { printf("Hello World");}'|clang -x c -; ./a.out` to make sure your CMake works and can compile this simple Hello World program without errors. |
| 82 | + 2. Nuke your `build` directory. The `setup.py` script compiles binaries into the `build` folder and caches many details along the way, which saves time the next time you build. If you're running into issues, you can always `rm -rf build` from the toplevel `pytorch` directory and start over. |
| 83 | + 3. If you have made edits to the Intel® Extension for PyTorch\* repo, commit any change you'd like to keep and clean the repo with the following commands (note that clean _really_ removes all untracked files and changes.): |
| 84 | + ```bash |
| 85 | + git submodule deinit -f . |
| 86 | + git clean -xdf |
| 87 | + python setup.py clean |
| 88 | + git submodule update --init --recursive --jobs 0 # very important to sync the submodules |
| 89 | + python setup.py develop # then try running the command again |
| 90 | + ``` |
| 91 | + 4. The main step within `python setup.py develop` is running `make` from the `build` directory. If you want to experiment with some environment variables, you can pass them into the command: |
| 92 | + ```bash |
| 93 | + ENV_KEY1=ENV_VAL1[, ENV_KEY2=ENV_VAL2]* python setup.py develop |
| 94 | + ``` |
| 95 | + |
| 96 | +## Unit testing |
| 97 | + |
| 98 | +### Python Unit Testing |
| 99 | + |
| 100 | +All PyTorch test suites are located in the `test` folder and start with `test_`. Run individual test suites using the command `python test/cpu/FILENAME.py`, where `FILENAME` represents the file containing the test suite you wish to run. |
| 101 | + |
| 102 | +For example, to run all the TorchScript JIT tests (located at `test/cpu/test_jit.py`), you would run: |
| 103 | + |
| 104 | +```bash |
| 105 | +python test/cpu/test_jit.py |
| 106 | +``` |
| 107 | + |
| 108 | +You can narrow down what you're testing even further by specifying the name of an individual test with `TESTCLASSNAME.TESTNAME`. Here, `TESTNAME` is the name of the test you want to run, and `TESTCLASSNAME` is the name of the class in which it is defined. |
| 109 | + |
| 110 | +Going off the above example, let's say you want to run `test_Sequential`, which is defined as part of the `TestJit` class in `test/cpu/test_jit.py`. Your command would be: |
| 111 | + |
| 112 | +```bash |
| 113 | +python test/test_jit.py TestJit.test_Sequential |
| 114 | +``` |
| 115 | + |
| 116 | +The `expecttest` and `hypothesis` libraries must be installed to run the tests. `mypy` is an optional dependency, and `pytest` may help run tests more selectively. All these packages can be installed with `conda` or `pip`. |
| 117 | + |
| 118 | +### Better local unit tests with `pytest` |
| 119 | + |
| 120 | +We don't officially support `pytest`, but it works well with our `unittest` tests and offers a number of useful features for local developing. Install it via `pip install pytest`. |
| 121 | + |
| 122 | +If you want to just run tests that contain a specific substring, you can use the `-k` flag: |
| 123 | + |
| 124 | +```bash |
| 125 | +pytest test/cpu/test_nn.py -k Loss -v |
| 126 | +``` |
| 127 | + |
| 128 | +The above is an example of testing a change to all Loss functions: this command runs tests such as `TestNN.test_BCELoss` and `TestNN.test_MSELoss` and can be useful to save keystrokes. |
| 129 | + |
| 130 | +### Local linting |
| 131 | + |
| 132 | +You can run the same linting steps that are used in CI locally via `make`: |
| 133 | + |
| 134 | +```bash |
| 135 | +# Lint all files |
| 136 | +make lint -j 6 # run lint (using 6 parallel jobs) |
| 137 | + |
| 138 | +# Lint only the files you have changed |
| 139 | +make quicklint -j 6 |
| 140 | +``` |
| 141 | + |
| 142 | +These jobs may require extra dependencies that aren't dependencies of Intel® Extension for PyTorch\* itself, so you can install them via this command, which you should only have to run once: |
| 143 | + |
| 144 | +```bash |
| 145 | +make setup_lint |
| 146 | +``` |
| 147 | + |
| 148 | +To run a specific linting step, use one of these targets or see the [`Makefile`](Makefile) for a complete list of options. |
| 149 | + |
| 150 | +```bash |
| 151 | +# Check for tabs, trailing newlines, etc. |
| 152 | +make quick_checks |
| 153 | + |
| 154 | +make flake8 |
| 155 | + |
| 156 | +make mypy |
| 157 | + |
| 158 | +make cmakelint |
| 159 | + |
| 160 | +make clang-tidy |
| 161 | +``` |
| 162 | + |
| 163 | +To run a lint only on changes, add the `CHANGED_ONLY` option: |
| 164 | + |
| 165 | +```bash |
| 166 | +make <name of lint> CHANGED_ONLY=--changed-only |
| 167 | +``` |
| 168 | + |
| 169 | +### C++ Unit Testing |
| 170 | + |
| 171 | +Intel® Extension for PyTorch\* offers tests located in the `test/cpp` folder. These tests are written in C++ and use the Google Test testing framework. After compiling Intel® Extension for PyTorch\* from source, the test runner binaries will be written to the `build/bin` folder. The command to run one of these tests is `./build/bin/FILENAME --gtest_filter=TESTSUITE.TESTNAME`, where `TESTNAME` is the name of the test you'd like to run and `TESTSUITE` is the suite that test is defined in. |
| 172 | + |
| 173 | +For example, if you wanted to run the test `MayContainAlias`, which is part of the test suite `ContainerAliasingTest` in the file `test/cpp/jit/test_alias_analysis.cpp`, the command would be: |
| 174 | + |
| 175 | +```bash |
| 176 | +./build/bin/test_jit --gtest_filter=ContainerAliasingTest.MayContainAlias |
| 177 | +``` |
| 178 | + |
| 179 | +## Writing documentation |
| 180 | + |
| 181 | +So you want to write some documentation and don't know where to start? |
| 182 | + |
| 183 | +Intel® Extension for PyTorch\* uses [Google style](http://sphinxcontrib-napoleon.readthedocs.io/en/latest/example_google.html) for formatting docstrings. Length of line inside docstrings block must be limited to 80 characters to fit into Jupyter documentation popups. |
| 184 | + |
| 185 | +### Building documentation |
| 186 | + |
| 187 | +To build the documentation: |
| 188 | + |
| 189 | +1. Build and install Intel® Extension for PyTorch\* |
| 190 | + |
| 191 | +2. Install the prerequisites |
| 192 | + |
| 193 | +```bash |
| 194 | +cd docs |
| 195 | +pip install -r requirements.txt |
| 196 | +``` |
| 197 | + |
| 198 | +3. Generate the documentation HTML files. The generated files will be in `docs/_build/html`. |
| 199 | + |
| 200 | +```bash |
| 201 | +make clean |
| 202 | +make html |
| 203 | +``` |
| 204 | + |
| 205 | +#### Tips |
| 206 | + |
| 207 | +The `.rst` source files live in [docs/tutorials](https://github.com/intel/intel-extension-for-pytorch/tree/master/docs/tutorials). Some of the `.rst` files pull in docstrings from Intel® Extension for PyTorch\* Python code (for example, via the `autofunction` or `autoclass` directives). To vastly shorten doc build times, it is helpful to remove the files you are not working on, only keeping the base `index.rst` file and the files you are editing. The Sphinx build will produce missing file warnings but will still complete. |
| 208 | + |
0 commit comments