An open Hawaiian indigenous plant identification and aggregation application. Developed for the 2018 Hawaii Annual Code Challenge. Learn more about the challenge and our app at devpost.
- Overview
- Quick Start Application
- Complete Application Setup
- Data Collection
- Machine Learning
- Results
- Usage
- Sponsors
To get started, first read the introductory text under Quick Start Application. Or read the Data Collection, Machine Learning, and Results sections of this README.
When ready for more information about each step of the process, visit any of the following:
Our app was built for the iPhone 6 running on iOS version 12. However, deploying the app to an iPhone device is a long process, especially if your Mac is not up-to-date with the latest OS or Xcode version. Therefore, we recommend these quick start steps for getting up-and-running quickly on the browser. Note, however, that the main feature of the app, plant identification, will not work because the native iOS camera cannot run on the browser.
If you would rather follow the complete instructions and deploy to an iOS device, follow the steps under Complete Application Setup.
- Clone this repository by running the following command in a terminal:
git clone [email protected]:HACC2018/ohia.ai.git. - Change into the ohia.ai project directory:
cd ohia.ai. - When running anything related to the client or server, we require Python 2.7 to be installed, since that is the version that the tfjs-node package supports. Switch your terminal to use Python 2.7 (more information here).
- Switch your terminal to use Node 10.12.0 (more information here).
- From the project root directory, run
npm install. Among other items, you should see "Downloading libtensorflow" and "Building TensorFlow Node.js bindings" run without error in your output. - Create a
.envfile in the project root by copying the template in.env.sample. - Source your environment by running:
source .env. - You should now see these variables in your environment by running
env. - Add the machine learning model weight files from our Google Drive by first downloading and unzipping
weights.tar.gzusing the commandtar xzf weights.tar.gz, followed by replacing the/server/models/mobilenetv1-1.00folder with the respective unzipped folder. - Run the server:
npm run dev-server. You should see the following output:▶ npm run dev-server > [email protected] dev-server /path/to/ohia.ai > nodemon server/index.js [nodemon] 1.18.4 [nodemon] to restart at any time, enter `rs` [nodemon] watching: *.* [nodemon] starting `node server/index.js` 2018-11-07 07:25:31.386112: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.2 AVX AVX2 FMA Server for ohia.ai listening on port 3000
- Open another terminal window and switch to Python 2.7 and Node 10.12.0.
- Source the environment:
source .env. - Install Vue CLI:
npm install -g @vue/cli. - Install Vue CLI addon:
npm install -g @vue/cli-init. - Install Quasar CLI:
npm install -g quasar-cli. - From the project root directory, install client dependencies:
npm run install:client. - Run
npm run devto view the app in a browser. You should see the following output:▶ npm run install:client > [email protected] install:client ~/ohia.ai > cd client && npm install > [email protected] install ~/ohia.ai/client/node_modules/fsevents > node install [fsevents] Success: "~/ohia.ai/client/node_modules/fsevents/lib/binding/Release/node-v64-darwin-x64/fse.node" already installed Pass --update-binary to reinstall or --build-from-source to recompile added 1333 packages from 612 contributors and audited 16152 packages in 23.649s found 0 vulnerabilities (python2) - Visit http://localhost:8080 to view the app.
- The app is best viewed by enabling the Chrome device toolbar and setting the device to iPhone 6/7/8.
- Clone this repository by running the following command in a terminal:
git clone [email protected]:HACC2018/ohia.ai.git. - Change into the ohia.ai project directory:
cd ohia.ai. - When running anything related to the client or server, we require Python 2.7 to be installed, since that is the version that the tfjs-node package supports. Switch your terminal to use Python 2.7 (more information here).
- Switch your terminal to use Node 10.12.0 (more information here).
- Follow the instructions on the API Server README.
- Follow the instructions on the App Client README.
Machine learning involves learning from data, thus data collection is an integral part of our application. For more details see ohia.ai/data_collection. View the scraped data in our Google Drive, within the images folder, and our plant metadata within the plant_meta folder.
Behind the scenes, ohia.ai is powered by machine learning and AI. Our team utilizes modern deep learning techniques and large open source datasets to achieve highly accurate classification on a wide range of flora found throughout the Hawaiian Islands. For more details see ohia.ai/machine_learning.
| Plant Name | Top 1 Accuracy | Top 3 Accuracy | Top 5 Accuracy | Image Count |
|---|---|---|---|---|
| Abutilon | 80.55% | 83.33% | 88.88% | 374 |
| Achyranthes | 53.12% | 65.62% | 78.12% | 258 |
| Aleurites | 46.66% | 80.00% | 86.66% | 176 |
| Ananas | 86.00% | 90.00% | 96.00% | 612 |
| Anthurium | 73.91% | 97.10% | 97.10% | 758 |
| Artocarpus | 73.07% | 88.46% | 88.46% | 303 |
| Bonamia | 66.66% | 66.66% | 66.66% | 44 |
| Bougainvillea | 96.36% | 100.00% | 100.00% | 663 |
| Canavalia | 62.50% | 79.16% | 83.33% | 212 |
| Chrysodracon | 63.63% | 81.81% | 81.81% | 276 |
| Cocos | 83.87% | 90.32% | 93.54% | 311 |
| Colubrina | 80.00% | 100.00% | 100.00% | 67 |
| Cordia | 36.00% | 64.00% | 72.00% | 209 |
| Cordyline | 82.75% | 100.00% | 100.00% | 335 |
| Cortaderia | 50.00% | 100.00% | 100.00% | 39 |
| Cyperus | 78.57% | 91.83% | 94.89% | 871 |
| Delairea | 71.42% | 100.00% | 100.00% | 111 |
| Dioscorea | 57.14% | 80.00% | 85.71% | 261 |
| Gouania | 66.66% | 66.66% | 66.66% | 31 |
| Heliconia | 87.71% | 94.73% | 98.24% | 622 |
| Hibiscus | 77.82% | 91.63% | 95.39% | 2,151 |
| Ischaemum | 50.00% | 75.00% | 75.00% | 41 |
| Isodendrion | 00.00% | 50.00% | 50.00% | 20 |
| Marsilea | 75.00% | 100.00% | 100.00% | 51 |
| Mezoneuron | 100.00% | 100.00% | 100.00% | 18 |
| Nothocestrum | 37.50% | 50.00% | 75.00% | 64 |
| Officinale | 66.00% | 82.00% | 88.00% | 556 |
| Panicum | 58.06% | 90.32% | 98.38% | 580 |
| Pennisetum | 83.33% | 100.00% | 100.00% | 162 |
| Peucedanum | 00.00% | 50.00% | 50.00% | 31 |
| Plumeria | 79.20% | 89.10% | 91.08% | 741 |
| Portulaca | 61.84% | 82.89% | 82.89% | 623 |
| Pritchardia | 84.61% | 96.15% | 98.07% | 464 |
| Pseudognaphalium | 66.66% | 85.18% | 94.44% | 420 |
| Rubus | 84.56% | 93.95% | 97.31% | 1,113 |
| Scaevola | 70.66% | 84.00% | 90.66% | 689 |
| Schenkia | 12.50% | 50.00% | 62.50% | 48 |
| Senecio | 80.80% | 92.92% | 94.94% | 854 |
| Sesbania | 60.00% | 80.00% | 80.00% | 104 |
| Tetramolopium | 65.00% | 90.00% | 95.00% | 146 |
| Tibouchina | 33.33% | 100.00% | 100.00% | 26 |
| Vigna | 50.00% | 68.75% | 75.00% | 101 |
