pip3 install streamlit import sys import hashlib import pickle import streamlit as st from dotenv import load_dotenv from PyPDF2 import PdfReader from streamlit_extras.add_vertical_space import add_vertical_space from langchain.text_splitter import RecursiveCharacterTextSplitter from langchain.embeddings.openai import OpenAIEmbeddings from langchain.vectorstores import FAISS from langchain.llms import OpenAI from langchain.chains.questions_answering import load_qa_chain from langchain.callback import get_openai_callback def compute_hash(file): # Create a hash object hasher = hashlib.sha256() # Read the file in chunks to avoid using too much memory chunk_size = 8192 for chunk in iter(lambda: file.read(chunk_size), b''): hasher.update(chunk) # Return the hexadecimal digest of the hash return hasher.hexdigest() def main(): st.header("Smartdoc") load_dotenv() # Retrieve the API key from environment variable api_key = os.getenv('YOUR_ENV_VARIABLE') # Replace 'YOUR_ENV_VARIABLE' with your actual variable name # Upload PDFs pdf = st.file_uploader("Upload your PDF", type="pdf") st.write(pdf.name) if pdf is not None: pdf_reader = PdfReader(pdf) text = "" for page in pdf_reader.pages: text += page.extract_text() text_splitter = RecursiveCharacterTextSplitter( chunk_size=1000, chunk_overlap=200, length_function=len ) chunks = text_splitter.split_text(text=text) # Embeddings embeddings = OpenAIEmbeddings(model_name="your_model_name", api_key=api_key) # Specify the model name here # Compute the hash of the PDF content pdf_hash = compute_hash(pdf) # Use the hash as part of the filename for the embeddings file vector_store_path = f"{pdf.name}_{pdf_hash}.pkl" if os.path.exists(vector_store_path): # If the embeddings file already exists, load it with open(vector_store_path, "rb") as f: VectorStore = pickle.load(f) st.write('Embeddings loaded from the Disk') else: # If the embeddings file does not exist, calculate the embeddings VectorStore = FAISS.from_texts(chunks, embedding=embeddings) with open(vector_store_path, "wb") as f: pickle.dump(VectorStore, f) st.write('Embeddings Computation Completed') # Accept user questions/query query = st.text_input("Ask questions about your PDF file:") st.write(query) if query: docs = VectorStore.similarity_search(query=query, k=3) # Provide the correct method name here if docs: # Check if docs is not empty llm = OpenAI(model_name="your_model_name", api_key=api_key) # Specify the model name here chain = load_qa_chain(llm=llm, chain_type="stuff") # Correct function name with get_openai_callback() as cb: response = chain.run(input_documents=docs, question=query) print(cb) st.write(response if response is not None else "No relevant information found.") else: st.write("No relevant documents found.") st.write(docs) if __name__ == "__main__": main()
Write, Run & Share Python code online using OneCompiler's Python online compiler for free. It's one of the robust, feature-rich online compilers for python language, supporting both the versions which are Python 3 and Python 2.7. Getting started with the OneCompiler's Python editor is easy and fast. The editor shows sample boilerplate code when you choose language as Python or Python2 and start coding.
OneCompiler's python online editor supports stdin and users can give inputs to programs using the STDIN textbox under the I/O tab. Following is a sample python program which takes name as input and print your name with hello.
import sys
name = sys.stdin.readline()
print("Hello "+ name)
Python is a very popular general-purpose programming language which was created by Guido van Rossum, and released in 1991. It is very popular for web development and you can build almost anything like mobile apps, web apps, tools, data analytics, machine learning etc. It is designed to be simple and easy like english language. It's is highly productive and efficient making it a very popular language.
When ever you want to perform a set of operations based on a condition IF-ELSE is used.
if conditional-expression
#code
elif conditional-expression
#code
else:
#code
Indentation is very important in Python, make sure the indentation is followed correctly
For loop is used to iterate over arrays(list, tuple, set, dictionary) or strings.
mylist=("Iphone","Pixel","Samsung")
for i in mylist:
print(i)
While is also used to iterate a set of statements based on a condition. Usually while is preferred when number of iterations are not known in advance.
while condition
#code
There are four types of collections in Python.
List is a collection which is ordered and can be changed. Lists are specified in square brackets.
mylist=["iPhone","Pixel","Samsung"]
print(mylist)
Tuple is a collection which is ordered and can not be changed. Tuples are specified in round brackets.
myTuple=("iPhone","Pixel","Samsung")
print(myTuple)
Below throws an error if you assign another value to tuple again.
myTuple=("iPhone","Pixel","Samsung")
print(myTuple)
myTuple[1]="onePlus"
print(myTuple)
Set is a collection which is unordered and unindexed. Sets are specified in curly brackets.
myset = {"iPhone","Pixel","Samsung"}
print(myset)
Dictionary is a collection of key value pairs which is unordered, can be changed, and indexed. They are written in curly brackets with key - value pairs.
mydict = {
"brand" :"iPhone",
"model": "iPhone 11"
}
print(mydict)
Following are the libraries supported by OneCompiler's Python compiler
Name | Description |
---|---|
NumPy | NumPy python library helps users to work on arrays with ease |
SciPy | SciPy is a scientific computation library which depends on NumPy for convenient and fast N-dimensional array manipulation |
SKLearn/Scikit-learn | Scikit-learn or Scikit-learn is the most useful library for machine learning in Python |
Pandas | Pandas is the most efficient Python library for data manipulation and analysis |
DOcplex | DOcplex is IBM Decision Optimization CPLEX Modeling for Python, is a library composed of Mathematical Programming Modeling and Constraint Programming Modeling |