Move djl-serving project from djl repo

Author: frankfliu

.gitignore

@@ -0,0 +1,27 @@
+.gradle
+.DS_Store
+.idea
+*.iml
+build
+libs
+*.gz
+*.params
+*.zip
+*.jar
+*.so
+*.dylib
+*.dll
+*.class
+
+# Eclipse
+.settings
+.project
+.classpath
+bin/
+
+#node
+node_modules/
+**/static/dist/
+
+#vscode
+.vscode

CONTRIBUTING.md

@@ -23,25 +23,26 @@ reported the issue. Please try to include as much information as you can. Detail
 ## Contributing via Pull Requests
 Contributions via pull requests are much appreciated. Before sending us a pull request, please ensure that:
 
-1. You are working against the latest source on the *main* branch.
+1. You are working against the latest source on the *master* branch.
 2. You check existing open, and recently merged, pull requests to make sure someone else hasn't addressed the problem already.
-3. You open an issue to discuss any significant work - we would hate for your time to be wasted.
+3. You open an issue to discuss any significant work - we would hate for your time to be wasted. If there is an existing issue your are interested in working on, you should comment on it so others don't start working on the same issue.
 
 To send us a pull request, please:
 
 1. Fork the repository.
 2. Modify the source; please focus on the specific change you are contributing. If you also reformat all the code, it will be hard for us to focus on your change.
-3. Ensure local tests pass.
-4. Commit to your fork using clear commit messages.
-5. Send us a pull request, answering any default questions in the pull request interface.
-6. Pay attention to any automated CI failures reported in the pull request, and stay involved in the conversation.
+3. Look at the [contributor documentation](docs/development/README.md), especially the docs in bold, for help setting up your development environment and information about various conventions.
+4. Ensure local tests pass.
+5. Commit to your fork using clear commit messages.
+6. Send us a pull request, answering any default questions in the pull request interface.
+7. Pay attention to any automated CI failures reported in the pull request, and stay involved in the conversation.
 
 GitHub provides additional document on [forking a repository](https://help.github.com/articles/fork-a-repo/) and
 [creating a pull request](https://help.github.com/articles/creating-a-pull-request/).
 
 
 ## Finding contributions to work on
-Looking at the existing issues is a great way to find something to contribute on. As our projects, by default, use the default GitHub issue labels (enhancement/bug/duplicate/help wanted/invalid/question/wontfix), looking at any 'help wanted' issues is a great place to start.
+Looking at the existing issues is a great way to find something to contribute on. As our projects, by default, use the default GitHub issue labels (enhancement/bug/duplicate/help wanted/invalid/question/wontfix), looking at any 'Call for Contribution' or 'good first issue' issues is a great place to start.
 
 
 ## Code of Conduct
@@ -57,3 +58,5 @@ If you discover a potential security issue in this project we ask that you notif
 ## Licensing
 
 See the [LICENSE](LICENSE) file for our project's licensing. We will ask you to confirm the licensing of your contribution.
+
+We may ask you to sign a [Contributor License Agreement (CLA)](http://en.wikipedia.org/wiki/Contributor_License_Agreement) for larger changes.

README.md

@@ -1,17 +1,154 @@
-## My Project
+# DJL Serving
 
-TODO: Fill this README out!
+## Overview
 
-Be sure to:
+DJL serving is a high performance universal model serving solution. You can use djl-serving serve the
+following models out of the box:
 
-* Change the title in this README
-* Edit your repository description on GitHub
+- PyTorch TorchScript model
+- TensorFlow SavedModel bundle
+- Apache MXNet model
 
-## Security
+You can install extra extensions to enable the following models:
 
-See [CONTRIBUTING](CONTRIBUTING.md#security-issue-notifications) for more information.
+- ONNX model
+- PaddlePaddle model
+- TFLite model
+- Neo DLR (TVM) model
+- XGBoost model
+- Sentencepiece model
+- fastText/BlazingText model
 
-## License
+## Architecture
 
-This project is licensed under the Apache-2.0 License.
+DJL serving is built on top of [Deep Java Library](https://djl.ai). You can visit
+[DJL github repository](https://github.com/deepjavalibrary/djl) to learn more about DJL. 
 
+![Architecture Diagram](https://resources.djl.ai/images/djl-serving/architecture.png)
+
+## Key features
+
+- Performance, DJL serving running multithreading inference in a single JVM. Our benchmark shows
+DJL serving has higher throughput than most of C++ based model server on the market.
+- Ease of use, DJL serving can serve most of the models out of box.
+- Easy to extend, DJL serving plugins make it easy for user to add their own extensions.
+- Auto-scale, DJL serving automatically scales up/down worker threads based on the load.
+- Dynamic batching, DJL serving supports dynamic batching to increase throughput.
+- Model versioning, DJL allows user to load different version of a model on a single endpoint.
+- Multi-engine support, DJL allows user to serve models from different engines at the same time.
+
+## Installation
+
+For macOS
+
+```
+brew cask install djl-serving
+
+# Start djl-serving as service:
+brew services start djl-serving
+
+# Stop djl-serving service
+brew services stop djl-serving
+```
+
+For Ubuntu
+
+```
+curl -O https://publish.djl.ai/djl-serving/djl-serving_0.12.0-1_all.deb
+sudo dpkg -i djl-serving_0.12.0-1_all.deb
+```
+
+For Windows
+
+We are considering to create a `chocolatey` package for Windows. For the time being, you can 
+download djl-serving zip file from [here](https://publish.djl.ai/djl-serving/serving-0.12.0.zip).
+
+### Docker
+
+You can also use docker to run DJL Serving:
+
+```
+docker run -itd -p 8080:8080 deepjavalibrary/djl-serving
+```
+
+## Run DJL Serving
+
+Use the following command to start model server locally:
+
+```sh
+djl-serving
+```
+
+The model server will be listening on port 8080. You can also load a model for serving on start up:
+
+```sh
+djl-serving -m "https://resources.djl.ai/demo/mxnet/resnet18_v1.zip"
+```
+
+Open another terminal, and type the following command to test the inference REST API:
+
+```sh
+curl -O https://resources.djl.ai/images/kitten.jpg
+curl -X POST http://localhost:8080/predictions/resnet18_v1 -T kitten.jpg
+
+or:
+
+curl -X POST http://localhost:8080/predictions/resnet18_v1 -F "[email protected]"
+
+[
+  {
+    "className": "n02123045 tabby, tabby cat",
+    "probability": 0.4838452935218811
+  },
+  {
+    "className": "n02123159 tiger cat",
+    "probability": 0.20599420368671417
+  },
+  {
+    "className": "n02124075 Egyptian cat",
+    "probability": 0.18810515105724335
+  },
+  {
+    "className": "n02123394 Persian cat",
+    "probability": 0.06411745399236679
+  },
+  {
+    "className": "n02127052 lynx, catamount",
+    "probability": 0.010215568356215954
+  }
+]
+```
+
+For more command line options:
+
+```sh
+djl-serving --help
+usage: djl-serving [OPTIONS]
+ -f,--config-file <CONFIG-FILE>    Path to the configuration properties file.
+ -h,--help                         Print this help.
+ -m,--models <MODELS>              Models to be loaded at startup.
+ -s,--model-store <MODELS-STORE>   Model store location where models can be loaded.
+```
+
+## REST API
+
+DJL Serving use RESTful API for both inference and management calls.
+
+When DJL Serving startup, it starts two web services:
+* [Inference API](docs/inference_api.md)
+* [Management API](docs/management_api.md)
+
+By default, DJL Serving listening on 8080 port and only accessible from localhost.
+Please see [DJL Serving Configuration](docs/configuration.md) for how to enable access from remote host.
+
+# Plugin management
+
+DJL Serving supports plugins, user can implement their own plugins to enrich DJL Serving features.
+See [DJL Plugin Management](docs/plugin_management.md) for how to install plugins to DJL Serving.
+
+## Logging
+you can set the logging level on the command-line adding a parameter for the JVM
+
+```sh
+-Dai.djl.logging.level={FATAL|ERROR|WARN|INFO|DEBUG|TRACE}
+```

build.gradle

@@ -0,0 +1,116 @@
+plugins {
+    id "com.github.spotbugs" version "4.7.1" apply false
+}
+
+defaultTasks 'build'
+
+def djl_version = System.getenv("DJL_VERSION")
+djl_version = (djl_version == null) ? "0.13.0-SNAPSHOT" : djl_version
+
+allprojects {
+    group 'ai.djl'
+    version "${djl_version}"
+
+    repositories {
+        mavenCentral()
+        maven {
+            url 'https://oss.sonatype.org/content/repositories/snapshots/'
+        }
+    }
+
+    apply plugin: 'idea'
+    idea {
+        module {
+            outputDir = file('build/classes/java/main')
+            testOutputDir = file('build/classes/java/test')
+            // inheritOutputDirs = true
+        }
+    }
+}
+
+def javaProjects() {
+    return subprojects.findAll { new File(it.projectDir, "src/main").exists() }
+}
+
+configure(javaProjects()) {
+    apply plugin: 'java-library'
+    sourceCompatibility = 1.8
+    targetCompatibility = 1.8
+    compileJava.options.encoding = "UTF-8"
+    compileTestJava.options.encoding = "UTF-8"
+    if (JavaVersion.current() != JavaVersion.VERSION_1_8) {
+        compileJava.options.compilerArgs.addAll(["--release", "8"])
+    }
+
+    apply plugin: 'eclipse'
+
+    eclipse {
+        jdt.file.withProperties { props ->
+            props.setProperty "org.eclipse.jdt.core.circularClasspath", "warning"
+        }
+        classpath {
+            sourceSets.test.java {
+                srcDirs = ["src/test/java"]
+                exclude "**/package-info.java"
+            }
+        }
+    }
+
+    apply from: file("${rootProject.projectDir}/tools/gradle/java-formatter.gradle")
+    apply from: file("${rootProject.projectDir}/tools/gradle/check.gradle")
+
+    test {
+        // tensorflow mobilenet and resnet require more cpu memory
+        maxHeapSize = "4096m"
+        doFirst {
+            if (JavaVersion.current() != JavaVersion.VERSION_1_8) {
+                jvmArgs = [
+                        '--add-opens', "java.base/jdk.internal.loader=ALL-UNNAMED"
+                ]
+            }
+        }
+
+        useTestNG() {
+            // suiteXmlFiles << new File(rootDir, "testng.xml") //This is how to add custom testng.xml
+        }
+
+        testLogging {
+            showStandardStreams = true
+            events "passed", "skipped", "failed", "standardOut", "standardError"
+        }
+
+        doFirst {
+            systemProperties System.getProperties()
+            systemProperties.remove("user.dir")
+            // systemProperty "ai.djl.logging.level", "debug"
+            systemProperty "org.slf4j.simpleLogger.defaultLogLevel", "debug"
+            systemProperty "org.slf4j.simpleLogger.log.org.mortbay.log", "warn"
+            systemProperty "disableProgressBar", "true"
+            systemProperty "nightly", System.getProperty("nightly", "false")
+            if (gradle.startParameter.offline) {
+                systemProperty "offline", "true"
+            }
+            // This is used to avoid overriding on default engine for modules:
+            // mxnet-engine, mxnet-model-zoo, api (MockEngine), basicdataset, fasttext, etc
+            if (project.name != "integration" && project.name != "examples") {
+                systemProperties.remove("ai.djl.default_engine")
+            }
+        }
+    }
+
+    compileJava {
+        options.compilerArgs << "-proc:none" << "-Xlint:all,-options,-static" << "-Werror"
+    }
+
+    compileTestJava {
+        options.compilerArgs << "-proc:none" << "-Xlint:all,-options,-static" << "-Werror"
+    }
+
+    jar {
+        manifest {
+            attributes("Automatic-Module-Name": "ai.djl.${project.name.replace('-', '_')}")
+        }
+    }
+}
+
+apply from: file("${rootProject.projectDir}/tools/gradle/jacoco.gradle")