Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Chat with corp. text content #36

Open
wants to merge 4 commits into
base: main
Choose a base branch
from
Open

Chat with corp. text content #36

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
eaea4e2
Created using Colaboratory
molgoo Aug 15, 2023
18a80f9
Update app.py
molgoo Aug 15, 2023
f7de90c
sss
molgoo Aug 16, 2023
739382e
test
molgoo Aug 16, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 0 additions & 2 deletions .env.example
View file
Open in desktop

This file was deleted.

259 changes: 259 additions & 0 deletions Making_the_Most_of_your_Colab_Subscription.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,259 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {
"id": "view-in-github",
"colab_type": "text"
},
"source": [
"<a href=\"https://colab.research.google.com/github/molgoo/ask-multiple-pdfs/blob/main/Making_the_Most_of_your_Colab_Subscription.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "SKQ4bH7qMGrA"
},
"source": [
"# Making the Most of your Colab Subscription\n",
"\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "QMMqmdiYMkvi"
},
"source": [
"## Faster GPUs\n",
"\n",
"Users who have purchased one of Colab's paid plans have access to premium GPUs. You can upgrade your notebook's GPU settings in `Runtime > Change runtime type` in the menu to enable Premium accelerator. Subject to availability, selecting a premium GPU may grant you access to a V100 or A100 Nvidia GPU.\n",
"\n",
"The free of charge version of Colab grants access to Nvidia's T4 GPUs subject to quota restrictions and availability.\n",
"\n",
"You can see what GPU you've been assigned at any time by executing the following cell. If the execution result of running the code cell below is \"Not connected to a GPU\", you can change the runtime by going to `Runtime > Change runtime type` in the menu to enable a GPU accelerator, and then re-execute the code cell.\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "23TOba33L4qf"
},
"outputs": [],
"source": [
"gpu_info = !nvidia-smi\n",
"gpu_info = '\\n'.join(gpu_info)\n",
"if gpu_info.find('failed') >= 0:\n",
" print('Not connected to a GPU')\n",
"else:\n",
" print(gpu_info)"
]
},
{
"cell_type": "code",
"source": [
"from google.colab import drive\n",
"drive.mount('/content/drive')"
],
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "flUoqVkOVxJr",
"outputId": "9262468b-8025-4781-8b68-2b25de35b704"
},
"execution_count": 4,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"Mounted at /content/drive\n"
]
}
]
},
{
"cell_type": "code",
"source": [
"!git clone . https://github.com/molgoo/ask-multiple-pdfs"
],
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "boGOeQKSXWJH",
"outputId": "a177c19b-df5e-4b28-ddef-6ff6fa5e0593"
},
"execution_count": 9,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"fatal: repository '.' does not exist\n"
]
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "Sa-IrJS1aRVJ"
},
"source": [
"In order to use a GPU with your notebook, select the `Runtime > Change runtime type` menu, and then set the hardware accelerator dropdown to GPU."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "65MSuHKqNeBZ"
},
"source": [
"## More memory\n",
"\n",
"Users who have purchased one of Colab's paid plans have access to high-memory VMs when they are available.\n",
"\n",
"\n",
"\n",
"You can see how much memory you have available at any time by running the following code cell. If the execution result of running the code cell below is \"Not using a high-RAM runtime\", then you can enable a high-RAM runtime via `Runtime > Change runtime type` in the menu. Then select High-RAM in the Runtime shape dropdown. After, re-execute the code cell.\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "V1G82GuO-tez"
},
"outputs": [],
"source": [
"from psutil import virtual_memory\n",
"ram_gb = virtual_memory().total / 1e9\n",
"print('Your runtime has {:.1f} gigabytes of available RAM\\n'.format(ram_gb))\n",
"\n",
"if ram_gb < 20:\n",
" print('Not using a high-RAM runtime')\n",
"else:\n",
" print('You are using a high-RAM runtime!')"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "BJW8Qi-pPpep"
},
"source": [
"## Longer runtimes\n",
"\n",
"All Colab runtimes are reset after some period of time (which is faster if the runtime isn't executing code). Colab Pro and Pro+ users have access to longer runtimes than those who use Colab free of charge.\n",
"\n",
"## Background execution\n",
"\n",
"Colab Pro+ users have access to background execution, where notebooks will continue executing even after you've closed a browser tab. This is always enabled in Pro+ runtimes as long as you have compute units available.\n",
"\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "uLlTRcMM_h0k"
},
"source": [
"## Relaxing resource limits in Colab Pro\n",
"\n",
"Your resources are not unlimited in Colab. To make the most of Colab, avoid using resources when you don't need them. For example, only use a GPU when required and close Colab tabs when finished.\n",
"\n",
"\n",
"\n",
"If you encounter limitations, you can relax those limitations by purchasing more compute units via Pay As You Go. Anyone can purchase compute units via [Pay As You Go](https://colab.research.google.com/signup); no subscription is required."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "mm8FzEidvPs6"
},
"source": [
"## Send us feedback!\n",
"\n",
"If you have any feedback for us, please let us know. The best way to send feedback is by using the Help > 'Send feedback...' menu. If you encounter usage limits in Colab Pro consider subscribing to Pro+.\n",
"\n",
"If you encounter errors or other issues with billing (payments) for Colab Pro, Pro+, or Pay As You Go, please email [[email protected]](mailto:[email protected])."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "qB3bdLe8jkAa"
},
"source": [
"## More Resources\n",
"\n",
"### Working with Notebooks in Colab\n",
"- [Overview of Colaboratory](/notebooks/basic_features_overview.ipynb)\n",
"- [Guide to Markdown](/notebooks/markdown_guide.ipynb)\n",
"- [Importing libraries and installing dependencies](/notebooks/snippets/importing_libraries.ipynb)\n",
"- [Saving and loading notebooks in GitHub](https://colab.research.google.com/github/googlecolab/colabtools/blob/master/notebooks/colab-github-demo.ipynb)\n",
"- [Interactive forms](/notebooks/forms.ipynb)\n",
"- [Interactive widgets](/notebooks/widgets.ipynb)\n",
"\n",
"<a name=\"working-with-data\"></a>\n",
"### Working with Data\n",
"- [Loading data: Drive, Sheets, and Google Cloud Storage](/notebooks/io.ipynb)\n",
"- [Charts: visualizing data](/notebooks/charts.ipynb)\n",
"- [Getting started with BigQuery](/notebooks/bigquery.ipynb)\n",
"\n",
"### Machine Learning Crash Course\n",
"These are a few of the notebooks from Google's online Machine Learning course. See the [full course website](https://developers.google.com/machine-learning/crash-course/) for more.\n",
"- [Intro to Pandas DataFrame](https://colab.research.google.com/github/google/eng-edu/blob/main/ml/cc/exercises/pandas_dataframe_ultraquick_tutorial.ipynb)\n",
"- [Linear regression with tf.keras using synthetic data](https://colab.research.google.com/github/google/eng-edu/blob/main/ml/cc/exercises/linear_regression_with_synthetic_data.ipynb)\n",
"\n",
"\n",
"<a name=\"using-accelerated-hardware\"></a>\n",
"### Using Accelerated Hardware\n",
"- [TensorFlow with GPUs](/notebooks/gpu.ipynb)\n",
"- [TensorFlow with TPUs](/notebooks/tpu.ipynb)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "RFm2S0Gijqo8"
},
"source": [
"<a name=\"machine-learning-examples\"></a>\n",
"\n",
"## Machine Learning Examples\n",
"\n",
"To see end-to-end examples of the interactive machine learning analyses that Colaboratory makes possible, check out these tutorials using models from [TensorFlow Hub](https://tfhub.dev).\n",
"\n",
"A few featured examples:\n",
"\n",
"- [Retraining an Image Classifier](https://tensorflow.org/hub/tutorials/tf2_image_retraining): Build a Keras model on top of a pre-trained image classifier to distinguish flowers.\n",
"- [Text Classification](https://tensorflow.org/hub/tutorials/tf2_text_classification): Classify IMDB movie reviews as either *positive* or *negative*.\n",
"- [Style Transfer](https://tensorflow.org/hub/tutorials/tf2_arbitrary_image_stylization): Use deep learning to transfer style between images.\n",
"- [Multilingual Universal Sentence Encoder Q&A](https://tensorflow.org/hub/tutorials/retrieval_with_tf_hub_universal_encoder_qa): Use a machine learning model to answer questions from the SQuAD dataset.\n",
"- [Video Interpolation](https://tensorflow.org/hub/tutorials/tweening_conv3d): Predict what happened in a video between the first and the last frame.\n"
]
}
],
"metadata": {
"colab": {
"name": "Making the Most of your Colab Subscription",
"provenance": [],
"history_visible": true,
"gpuType": "V100",
"include_colab_link": true
},
"kernelspec": {
"display_name": "Python 3",
"name": "python3"
},
"accelerator": "GPU"
},
"nbformat": 4,
"nbformat_minor": 0
}
16 changes: 14 additions & 2 deletions app.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,16 @@
# import ipdb
# !git clone https://github.com/alejandro-ao/ask-multiple-pdfs
# export PATH="/home/codespace/.local/lib/python3.10/site-packages/bin:$PATH"
# !pip install -r ask-multiple-pdfs/requirements.txt
# !streamlit run app.py (how to rin this app)
# %load app.py in order to modify
import os
os.environ['OPENAI_API_KEY'] = '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'
#with open('app.py', 'r+') as f: The next 3 lines load it to a colab cell
# code_content = f.read()
#print(code_content)
import langchain
langchain.verbose = False
import streamlit as st
from dotenv import load_dotenv
from PyPDF2 import PdfReader
Expand All @@ -9,7 +22,7 @@
from langchain.chains import ConversationalRetrievalChain
from htmlTemplates import css, bot_template, user_template
from langchain.llms import HuggingFaceHub

#OPENAI_API_KEY="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"
def get_pdf_text(pdf_docs):
text = ""
for pdf in pdf_docs:
Expand Down Expand Up @@ -99,6 +112,5 @@ def main():
st.session_state.conversation = get_conversation_chain(
vectorstore)


if __name__ == '__main__':
main()