How to Feed Website Data into LLM using Langchain

In this post I will try to get you to run your LLM locally and feed your own data into it. To be able to do this you need to be running your own Jupyter Notebook. To see how you can do that please follow this link.

How to Run Jupyter Notebook on MacOS

Set Langchain

Lets get our Langchain working.

1. Head to https://www.langchain.com/ to create an account.
2. Now click +New Project
3. Just give a name description. You dont need to give a dataset.
4. OUTPUT: Project Name
5. Click on API keys from bottom of left sidebar
6. OUTPUT: Create an API Key and save.

Set OpenAI

1. Head to https://openai.com/ Click login and then head to sign up. (stupid design)
2. Fill required info and complete sign up.
3. Choose API
4. On left choose API key. Verify your phone and create your API Key
5. OUTPUT: API Key

For starter you will have 5$ grant. That will be more than enough for you to test your app. But if you want you can increase the limit and credit.

Set Configuration

Open your Jupyter Notebook choose a folder to put your code in and start typing! First we gonna need some packages:

#Install required packages.
%pip install langchain
%pip install beautifulsoup4
%pip install langchain-openai
%pip install faiss-cpu

Afterwards lets set our configuration. Keep the outputs we prepared on before steps ready for this step.

#Add configuration for APIs and connecting to Langchain. 
import os
os.environ["LANGCHAIN_TRACING_V2"] = "true"
os.environ["LANGCHAIN_ENDPOINT"]="<https://api.smith.langchain.com>"
os.environ["LANGCHAIN_PROJECT"]="<put_langchain_project_name>"
os.environ["LANGCHAIN_API_KEY"] = "<your_langchain_api_key>"
os.environ["OPENAI_API_KEY"] = "<your_openai_key>"

Start Coding

Now lets load required base libraries.

#Install required libraries.
from langchain_openai import ChatOpenAI
from langchain_openai import OpenAIEmbeddings
from langchain_community.vectorstores import FAISS
from langchain_core.prompts import ChatPromptTemplate
from langchain.chains.combine_documents import create_stuff_documents_chain
from langchain.chains import create_retrieval_chain

If you are going to feed data from link we need to use WebBaseLoader. This will load from data into loader. And then feed that into docs variable.

Afterwards we need to create a variable to split text into consumable sizes for our LLM. This is one of the basic things you need to understand for how LLM works. Its probably way over my head to explain this but basically you need to let LLM consume data in small chunks. So that it uses less compute power.

Next we create a variable called documents and let text splitter do its thing on docs variable where we loaded the sites data.

from langchain_community.document_loaders import WebBaseLoader
from langchain.text_splitter import RecursiveCharacterTextSplitter

#Loader used for setting up context.
loader = WebBaseLoader("<link>")
docs = loader.load()

#We parse through data here. We divide it into chunks
text_splitter = RecursiveCharacterTextSplitter()
documents = text_splitter.split_documents(docs)

Now lets create our llm variable and load OpenAI llm. For testing purposes I am gonna use gpt 3.5 turbo. Since its 10-100x more cheaper than GPT 4.

Then create the embeddings variable.

Now the fun part we gonna give context to our LLM. LLMs follow context strictly. Here you need to give your specific instructions on how it should answer to questions. You can make it a great writer or you can limit its ability to hallucinate by only answering whats in the content provided.

And lets load the data we prepared in the last step into the retrieval. And voile we are ready.

#Designate LLM model and load it into variable LLM
llm = ChatOpenAI(temperature=0, model="gpt-3.5-turbo")
embeddings = OpenAIEmbeddings()

#Prompt
prompt = ChatPromptTemplate.from_template("""Answer the following question based only on the provided context:
<context>
{context}
</context>
Question: {input}""")
document_chain = create_stuff_documents_chain(llm, prompt)

#Feed the data we prepared earlier into retrieval.
vector = FAISS.from_documents(documents, embeddings)
retriever = vector.as_retriever()
retrieval_chain = create_retrieval_chain(retriever, document_chain)

Finally give your input and print answer into screen. Lets run it!

#And invoke retrieval to work through our data by asking question.
response = retrieval_chain.invoke({"input": "Summarize the content"})
print(response["answer"])

Congratulations you did it. Now you have a LLM working on your local fed by your own data. Reach out to me for more help or with tips on how i can improve this post.