updated_readme

README.md

@@ -1 +1,63 @@
-# cancer-dl
+# cancer-dl
+
+# Chest Cancer Detection Using Deep Learning with MLOps
+
+This project focuses on detecting chest cancer from medical images using deep learning techniques. It integrates MLOps practices for robust and reproducible model development, deployment, and monitoring.
+
+## Project Overview
+
+- **Data Source**: Chest cancer images from Kaggle.
+- **Model**: VGG16 with a transfer learning approach.
+- **Processes**:
+  - **Data Ingestion**: Automated data ingestion pipeline.
+  - **Training**: Initial training of a base model and modification of the model for improved performance.
+  - **Evaluation**: Rigorous evaluation of model performance using various metrics.
+  - **Prediction**: Generation of predictions using the trained model.
+
+## MLOps Tools and Techniques
+
+- **MLflow**: Used for tracking experiments and hyperparameter tuning.
+  ![Hyperparameter Tuning with MLflow](static\mlflow.png)
+
+- **DAGsHub**: Visualized data pipeline for streamlined data processing and management.
+  ![Data Pipeline Display with DAGsHub](static\dagshub.png)
+
+- **DVC (Data Version Control)**: Implemented for data versioning to ensure reproducibility.
+
+- **Flask Application**: Built a web app to provide an interface for users to upload images and receive predictions.
+  ![Flask App Interface](static\flask_app.png)
+
+- **Docker**: Dockerized the application for consistent deployment across environments.
+
+- **GitHub Actions**: Implemented CI/CD pipelines for continuous integration and deployment.
+
+- **AWS**: Deployed the application on AWS for scalable access.
+
+## Getting Started
+
+To get started with this project, follow these steps:
+
+1. Clone the repository:  
+   ```bash
+   git clone https://github.com/SathvikNayak123/cancer-dl.git
+   ```
+2. Install the necessary dependencies:
+    ```bash
+    pip install -r requirements.txt
+    ```
+3. Initialized DVC:
+    ```bash
+    dvc init
+    ```
+4. Run app
+    ```bash
+    python app.py
+    ```
+
+## Deployment
+
+The model and application are deployed on AWS, ensuring scalability and high availability. 
+The deployment process is automated using Docker and GitHub Actions for seamless updates and maintenance.
+
+## P.S
+For details about project and workflow and commands for AWS deployment, refer **worklow.md**

app.py

@@ -1,35 +1,36 @@
 from flask import Flask, request, jsonify, render_template
 import os
 from flask_cors import CORS, cross_origin
-from cnnClassifier.pipeline.predict import PredictionPipeline
 from cnnClassifier.utils.common import decodeImage
+from cnnClassifier.pipeline.predict import PredictionPipeline
 
 
-
-os.environ['LANG'] = 'en_US.UTF-8'
-os.environ['LC_ALL'] = 'en_US.UTF-8'
+os.putenv('LANG', 'en_US.UTF-8')
+os.putenv('LC_ALL', 'en_US.UTF-8')
 
 app = Flask(__name__)
 CORS(app)
 
+
 class ClientApp:
     def __init__(self):
         self.filename = "inputImage.jpg"
         self.classifier = PredictionPipeline(self.filename)
 
-clApp = ClientApp()
 
 @app.route("/", methods=['GET'])
 @cross_origin()
 def home():
     return render_template('index.html')
 
-@app.route("/train", methods=['GET', 'POST'])
+@app.route("/train", methods=['GET','POST'])
 @cross_origin()
 def trainRoute():
-    os.system("dvc repro")
+    os.system("python main.py")
+    # os.system("dvc repro")
     return "Training done successfully!"
 
+
 @app.route("/predict", methods=['POST'])
 @cross_origin()
 def predictRoute():
@@ -39,5 +40,7 @@ def predictRoute():
     return jsonify(result)
 
 
+
 if __name__ == "__main__":
-    app.run(host='0.0.0.0', port=8080)
+    clApp = ClientApp()
+    app.run(host='0.0.0.0', port=8080) #for AWS

dvc.lock

@@ -36,13 +36,13 @@ stages:
         - 224
         - 3
         INCLUDE_TOP: false
-        LEARNING_RATE: 0.01
+        LEARNING_RATE: 0.001
         WEIGHTS: imagenet
     outs:
     - path: artifacts/prepare_base_model
       hash: md5
-      md5: cfd2c0def201f160979a863fcdaf5c58.dir
-      size: 118054560
+      md5: 7159acba305ef31d358d9ed3f146f526.dir
+      size: 118053368
       nfiles: 2
   training:
     cmd: python src/cnnClassifier/pipeline/stage03_training.py
@@ -54,8 +54,8 @@ stages:
       nfiles: 485
     - path: artifacts/prepare_base_model
       hash: md5
-      md5: cfd2c0def201f160979a863fcdaf5c58.dir
-      size: 118054560
+      md5: 7159acba305ef31d358d9ed3f146f526.dir
+      size: 118053368
       nfiles: 2
     - path: config/config.yaml
       hash: md5
@@ -69,16 +69,16 @@ stages:
       params.yaml:
         AUGMENTATION: true
         BATCH_SIZE: 16
-        EPOCHS: 1
+        EPOCHS: 20
         IMAGE_SIZE:
         - 224
         - 224
         - 3
     outs:
     - path: artifacts/training/model.h5
       hash: md5
-      md5: c99218865b913c4c55239c5dc49b6748
-      size: 59337520
+      md5: 1a16e1f5ef6f389a3358067108ece334
+      size: 59135136
   evaluation:
     cmd: python src/cnnClassifier/pipeline/stage04_evalaution.py
     deps:
@@ -89,8 +89,8 @@ stages:
       nfiles: 485
     - path: artifacts/training/model.h5
       hash: md5
-      md5: c99218865b913c4c55239c5dc49b6748
-      size: 59337520
+      md5: 1a16e1f5ef6f389a3358067108ece334
+      size: 59135136
     - path: config/config.yaml
       hash: md5
       md5: e0b119309ec4caf177de98dd90dc661a
@@ -109,5 +109,5 @@ stages:
     outs:
     - path: scores.json
       hash: md5
-      md5: 8a87fa0c813c47fc86132f2a38e62141
-      size: 72
+      md5: dc37e3ce7631dc1307717b6f5220eedb
+      size: 73

params.yaml

@@ -2,7 +2,7 @@ AUGMENTATION: True
 IMAGE_SIZE: [224, 224, 3] # as per VGG 16 model
 BATCH_SIZE: 16
 INCLUDE_TOP: False
-EPOCHS: 1
+EPOCHS: 20
 CLASSES: 2
 WEIGHTS: imagenet
-LEARNING_RATE: 0.01
+LEARNING_RATE: 0.001

requirements.txt

@@ -1,17 +1,17 @@
-tensorflow==2.12.0
+tensorflow
 pandas 
 gdown
 dvc
-mlflow==2.2.2
+mlflow
 dagshub
 notebook
 numpy
 matplotlib
 seaborn
-python-box==6.0.2
+python-box
 pyYAML
 tqdm
-ensure==1.0.2
+ensure
 joblib
 types-PyYAML
 scipy

scores.json

@@ -1,4 +1,4 @@
 {
-    "loss": 9.625835418701172,
-    "accuracy": 0.6736111044883728
+    "loss": 0.0715596154332161,
+    "accuracy": 0.9791666865348816
 }

templates/index.html

@@ -5,8 +5,6 @@
     <meta name="viewport" content="width=device-width, initial-scale=1.0">
     <meta http-equiv="X-UA-Compatible" content="ie=edge">
     <title>cnncls</title>
-	<link rel="shortcut icon"
-        href="data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAACAAAAAgCAYAAABzenr0AAAIq0lEQVRYw6WWa3BV1RXH/2vv88zNi5AHgTyIISSIEAgPeUwUrFDFx9hqx9F2dDp2dGy105m2fFG/tLXtdPqYim0dWkundbSjBtEWqIoVCWhIkKcJGpKQBxCSkJt7b3If55x99uoHkCk2QqTr29lz9t6//d/rv9ciTDHOjGucPefL/uFkxZYPjGkh3JLZ+VxKAnzsNPfOKfTiD6+RYW15Vo8ledw15ZTWpSv90DmYptYT6RmH+8W6D7pFY9LHqlNxWZhSpmsJ6UJqeAFP5NvKn1sU+jOy1a7qwmDbhqXZbXPL3cHiCPFVAQRaY097qrBpv/pq68nwof64vSielFYAA0wEYg1NAnRhCQIAUrCERqEdpufP0ocWloW/vW+1/a/6KichSUwdIOGF4rWW5LIdR8On3zturz7nmQ4jBCkTwtBwLfZybT/tGgGICQnPQDwwnUDBCbUAICBIYVaOSt40T7+2upaeumdlpDff+d/tjMkAmpqHl/91n/XnloFInR8aJBjIdkJ/drF/srooaK4oct5zrcwn84oy0Bpo7beRyOia8VR4S+ewccOpmFmWUoY8E5eRVw4F93UN61JBqe8r1keMzygxKYAg+FqzrzSTIX2eMy04vqY23LL2Wt305Yb8PtcS2hCRS+YklW6LJ9XL2/an5rZ0+t94v9u4fyBmV3qelD2juHFwTN0eahwDoK8IcPfqwkO90eSjAfmbiiL+6Jo62njn9TnHqovti5O19ggAmEwGgAt3rAB0HO1LPrV1f2rn3i5s6hziupvnixcbr7W22FLoKSdhc0cCKqSFrqnHrqt0B1zHADJRIxjurNIHNheZTt71ysida7DfHQx1HKR5GwatWQ1dmLkskCSQ9BS93jZR3z3or26oyX3xtkXO2FXZEABYJSgYOFgd9O77DvU332acPTJDeEmXdGCQNNgz3FSYVzFCBXU7ac6XXnDm39Em3MLgqt6BvtEAjkmU4xJnmRLaT5DX/vpiffyNXxt9uxuFFxXEDGaCIA0Cg4UAswENU6vS+rNc/8Amp279piBSmXpx92jJjOk2r1+cM3w+na/ggkO9qqy9N/nNiiLzFV+pT7yPmq7lIy9tsgZ2r4TKkGCACWChQZqhSUAoDVAIFiys060zg0zqCd+Pj223HnnntYPiuews7UmRehRA/2UBWns86zdv6m/v/djduHJ2avZt1SNPZp3c9Qs5sG+lVB4xM0IBCCYY4acPEAFEIAYMHQLEsEaPZac77B+ULlhnnxqvqe7vlxW2GfywtSe9cfk1bvoSx/33x1uH/eUHTvK3PBZBZbHVJU/vvSns27dWqCQhJBAYhmYI1gBpgBiEEAADFIKgQXx+zBrqqK4ffK7+9oXB01kmj7/fJb7+zpHUqkx4qREuAkRTWuzppHUDMVlYW6KaNywSm40T/1xrxbpdwfrikzvVMNQEGUOtjY/PP/j+8kq9YyBh5rf1yXtHxtmYFOB0lEs05Fcs6HB2fvjqishR04z1NYgwgCZx/pRfAEILApLRWe7Ih5XlhfRSboS9U1Gx4lj3eOGkAMcHtXN23MxzHSNRkGsd4lhfEatkGUiDWINF+IUUUASQn8qSfqK2sc7oLIqIc6MTsuTjITE5wFhCwVeAY4dBUT4lOfSIOSQCQNAXFOApA0hmEGkwJE3PMZKmIC/DFg1N2DQpQH4EyoTyVaBdFYSFWkYSEEYcVxmCBNi0tHYK0od7fHsizYYl/OCa6f7kSZjw6JwlVMeEL7KHY8EKt6w+auQUnQERmL7o9gRNEtrOjgnLPNxxSs0fS3NxrqP6GmuNzKQADTVWZmZB2ByEhAN9xoam7jLHn1HfrIyckGGAWYAvl4R83ifEBDDATNBOyb4uu7G/d8x4MOOzU1Wgd5dOt3onBVgyU/CaOrVrTkHqzIkRc+X2g/yYqrzlDT1j8SkR4oIKn58DLABNBCYGE6AjhT5Xrd3+bl9J3rm4v7Qy3x9YWY2maa68pEZc0jk+/+yPR8+Mhm7PEN9gCu1eU1PxfLk9mtSJvjUyEzUufxPi/Okl4Nn5mqtuft1c8sAvnYKZQzoMhubNCFvW1Wft/NOmpz+/H6iYbgfvtKd/B2Qyc0rk0QVVkUE5896/6CBeHnQ0PWLE+l0BBfCFGsB8XhRxoa5pjcAp0GH5mnftueuflLOWjSwhAV/rlwmwTCHUVZXjINqdq9q3PY72rQ+KaNccI3OOQgLADMkMkAEtbei8sqhfeUOTOf/un9rVjb0ks/7/tvzTCEc7pH/u5AJ14t8bZLR7HY20Z8uJPmi7AFy8UMusvBbUrH8bZct2WSWLMlP3y+fEu0cTeUMxb2lNRd7+JbOtiYsgoUecGMjhD34l9dEXgNIVoJt/xqJ4/oRhuhcl7h1jau/LZFsSmXUL3OALAWz/MFn8apv60aE+3DOvNPzDXSuynlm30B4pcKYm2LaDSpyNpjdsbdPfLcnRb97f6Dx7a73rTVq0PjtwNuY5P2lKPvGPY+5DY55p9I4FG0cmguUDI+r3bx1Nvb2qxkhlu9akGw+Oh7Kt06t6oy1+z94u89GeqF0xPStcKmSQfO948o83zouEVwRQSquMr1oKHP9rsbQsTQTC2tPjrO8eCa4vyw3fXFXnH35+d7KlKNvrLs8L4YeE5i4hxuOofWxzbG1/3L6ze8SZG/OEBATGA84digeLdx5I05SvIKPCOZt2TKz4Wyt/7+SItTilTMGkIRnItjTnOxzPc9RYxAqhoBFLW5RIysK4LyNplkSKQUKjNCdMN5TTltsb1M8fvilnYMoAIWs7lQnVM2+lKgaG1MY93eKu3qhVHPimYCERXqyOJgQziAESDCUACR/TbOVdVxp8tH6B3LyqLuvvN9aZiauyYcgaJ4ZDd+u+sYYDXfLu/jHnjrMxv3o4bZIiFxACxApCK5gywMx8mqia7rfUlmDr2gWRHbcutk5lmZdvJP4Dl9dVkYcTQYMAAAAASUVORK5CYII=" />
     <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/4.0.0/css/bootstrap.min.css" integrity="sha384-Gn5384xqQ1aoWXA+058RXPxPg6fy4IWvTNh0E263XmFcJlSAwiGgFAW/dAiS6JXm" crossorigin="anonymous">
 	<style>
 		body{background-color: #eff2f9;}
@@ -108,6 +106,17 @@ <h3 class="text-center py-4">Chest Cancer Classification</h3>
 								<button type="button" class="btn btn-primary col-md-5 col-xs-5 ml-3 mr-4" id="uload">Upload</button>
 								<button id="send" type="button" class="btn btn-success col-md-5 col-xs-5">Predict</button>
 							</div>
+
+
+
+
+
+
+							<!-- change url value  -->
+
+
+
+
 							<input type="hidden" class="form-control mr-2" id="url" placeholder="Enter REST Api url..." value="../predict"/>
 							<input name="upload" type="file" id="fileinput" style="position:absolute;top:-500px;"/><br/>
 						</form>
@@ -124,9 +133,6 @@ <h5 class="mb-2"><center>Prediction Results</center></h5>
 		</section>
 	</div>
 
-	<img class="logo" src="data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAIkAAAAlCAMAAABBJDpuAAAAVFBMVEX/fgD/iQA7NzIyMjMnNYInNYEnNYInNYEzMzInNYQnNYETJzsnNYIwN1wxMjNoTis4O1YrNm0xOF47PE9DP0dEQEVpTVkAJJ4nNYEOLJHJbiAzMzOmmmHRAAAAFnRSTlOh/vu+986oen9fP2UqEzQkFAIDAAEEILVBJQAAA3FJREFUWMPNmO2WoyAMQG0VBARtdVRs3/89NwkgaF1rd2aOm18tIrnkgwSz7n+R7LcV1ELkeS7E7VySRlyfQXJhTiO55c+liP4cEuH1X8E7gaV+S3Jv2zYZ/Vr+/Sdx2vP6doc/fSPcf/GOpLSFVXFUW8t+giRP47TOd60yk1hbxIhSP0ACKPWrvw7YBIT/lcQY84G3Wk17ugWQfg5UsQJpjflKSLQi7xTWyk0SXTJ4VnB5D1yccxmp4Z+mX5KGNS+SPXVNll0ulyxL0wYmloBB6zL/MpCoAgjAJiUM6w0SboMwPUdRogvNKcNM3smFdRvEcJJkKcflQa+X0pNIfBGW0zKqjyQtw4VLKQlI7ZLgdrRNSZDjMQx2GB4pCkxkCrfGWUTJ0Egan6muCHgJCUwtnCk0mrJ9Q8K5LUqtdSfynkAedhynaRqngVDaXLiJEAzkFkM71IuIlWRYtSQpbXSZDvQ7JPCei9brs+oaABkn74LRIkpFaVxGnzjvIxTYREOoEwn5r01JTDRTcO87Eu3zNUeTPKZpjrIRrNIDCjzALbPk7LJMIokOEQu2K7yKQAKvFDF9pVe0R+LGb7h1MMkw2ijjA/1zhSfSphvEAChXJEhAqwYStIJWXnTpvbdH4hTUwSQ2JRmQpILDLYbB7J41SYiLcNozu5Z3JMo7RxBJahI7WSRByCMkqJt9hySESbVJcvmARJGFExJmdBSl2oM2+T4JLWfgaGFptizlPUm1FydHSdyZ7UnS4ySph2kS0kJrkvr5NJg7dnrJnfxgxPozRPGtkyPNOxtaiDvbIAF9L+5B5zTI2BwlwXmM+U3zReav3o0F8oUEFFZ42McDZYomOUziFmezgWyo251Uc1nyKJol7URCEvIYUEbHYaEGQi9C9egwCdVfHwiaajfjKMUcHcRacCylhWJbJDfq47EEDhYqoIWjHnwDXZPpPiChPQelLU+Cbm4w50FmYgkvFwpiWwCC/Um/SLcNElOC9RVj6qWpCiEMvRUK4zLpdKmPw8rVldy/qxjn60wLrVLWJIPLiZJRr3fsvnOHPvare+lB93tb37T20D82jqOt+zNuXhuNvNhp7X+PpHb3rvV9R5xjE2SpyCN9ffQO+GtWIZr5Xnytz4gTMIRYfys47/uJEXny/aQ/95tSUwmU+uRvSp/IHzQqZYpYrQmxAAAAAElFTkSuQmCC" />
-
-
 <div id="loading"><div class="loader"></div></div>
 <script src="https://ajax.googleapis.com/ajax/libs/jquery/3.4.1/jquery.min.js"></script>
 <script src="https://cdnjs.cloudflare.com/ajax/libs/popper.js/1.12.9/umd/popper.min.js" integrity="sha384-ApNbgh9B+Y1QKtv3Rn7W3mgPxhU9K/ScQsAP7hUibX39j7fakFPskvXusvfa0b4Q" crossorigin="anonymous"></script>