So far, we've explored the LangChain modules and how to use them (refer to the earlier blog post on LangChain Modules here). In this section, we'll focus on the LangChain Indexes and Agent module and also walk through the process of creating and launching a web application that everyone can access. To make things easier, we'll be using Databutton, an all-in-one online workspace to build and deploy web apps, integrated with Streamlit, a Python web- development framework known for its support in building interactive web applications.
In simpler terms, LangChain Agents are tools that enable Large Language Models (LLMs) to perform various actions, such as accessing Google search, executing Python calculations, or making SQL queries, thereby empowering LLMs to make informed decisions and interact with users by using tools and observing their outputs. The official documentation of LangChain describes Agents as:
" …there is an agent which has access to a suite of tools. Depending on the user input, the agent can then decide which, if any, of these tools to call…
In building agents, there are several abstractions involved. The Agent abstraction contains the application logic, receiving user input and previous steps to return either an AgentAction (tool and input) or AgentFinish (completion information). Agent covers another aspect, called Tools, which represents the actions agents can take, while Toolkits group tools for specific use cases (e.g., SQL querying). Lastly, the Agent Executor manages the iterative execution of the agent with the available tools. Thus, in this section, we will briefly explore such abstractions while using the Agent functionality to integrate tools and primarily focus on building a real-world easily deployable web application.
This module provides utility functions for structuring documents using indexes and allowing LLMs to interact with them effectively. We will focus on one of the most commonly used retrieval systems, where indexes are used to find the most relevant documents based on a user's query. Additionally, LangChain supports various index and retrieval types, with a focus on vector databases for unstructured data. We will explore this component in detail as it can be leveraged in a wide number of real-world applications.
Image 1 Langchain workflow by Author
Workflow of a question & answer generation interface using Retrieval index, where we leverage all types of Indexes which LangChain provides. Indexes are primarily of four types, namely : Document Loaders, Text Splitters, VectorStores, and Retrievers. Briefly, (a) the documents fetched from any datasource is split into chunks using text splitter modules (b) Embeddings are created (c)Stored over a vector store index ( vector databases such as chromadb / pinecone / weaviate, etc ) (d) Queries from the user is retrieved via retrieval QA chain
from langchain.document_loaders import WikipediaLoader
docs = WikipediaLoader(query='LangChain', load_max_docs=2).load()
docs[0].metadata
docs[0].page_content[:400]
CharacterTextSplitter
is used to split the loaded documents into smaller chunks for further processing.from langchain.text_splitter import CharacterTextSplitter
text_splitter = CharacterTextSplitter(chunk_size=4000, chunk_overlap=0)
texts = text_splitter.split_documents(docs)
OpenAIEmbeddings
the module is then employed to generate embeddings for the text chunks.from langchain.embeddings import OpenAIEmbeddings
embeddings = OpenAIEmbeddings(openai_api_key=OPENAI_API_KEY)
from langchain.vectorstores import Chroma
db = Chroma.from_documents(texts, embeddings)
retriever = db.as_retriever()
from langchain.chains import RetrievalQA
from langchain.llms import OpenAI
Qa = RetrievalQA.from_chain_type(llm=OpenAI(openai_api_key=OPENAI_API_KEY), chain_type="stuff", retriever=retriever)
At this stage, we can easily seek answers from the stored indexed data. For instance,
query = "What is LangChain?"
qa.run(query)
LangChain is a framework designed to simplify the creation of applications using large language models (LLMs).
query = "When was LangChain founded?"
qa.run(query)
LangChain was founded in October 2022.
query = "When was LangChain founded?"
qa.run(query)
LangChain was founded in October 2022.
query = "Who is the founder?"
qa.run(query)
The founder of LangChain is Harrison Chase.
The Q&A functionality implemented using the retrieval chain provides reasonable answers to most of our queries. Different types of indexes provided by LangChain, can be leveraged for various real-world use cases involving data structuring and retrieval.
Moving forward, we will delve into the next section, where we will focus on the final component called the "Agent." During this section, we will not only gain a hands-on understanding of its usage but also build and deploy a web app using an online workspace called Databutton.
To begin using Databutton, all that is required is to sign up through their official website. Once logged in, we can either create a blank template app from scratch or choose from the pre-existing templates provided by Databutton.
Image by Author | Screen grasp showing on how to start working with a new blank app
Once the blank app is created, we generate our online workspace consisting of several features for building and deploying the app. We can immediately begin writing our code within the online editor. The only requirement at this stage is to include the necessary packages or dependencies that our app requires.
Image by Author | Screen grasp showing the different components available within the Databutton App's online workspace.
Databutton's workspace initialization includes some essential packages by default. However, for our specific app, we need to add two additional packages - openai and langchain. This can be easily accomplished within the "configuration" workspace of Databutton.
Image by Author | Screen grasp of the configuration options within Databutton's online workspace.
Here we can add the additional packages which we need for working with our app. The workspace is generated with few pre-installed dependencies.
Now that we have a basic understanding of Agents and their abstraction methods, let's put them to use, alongside incorporating some basic Streamlit syntax for the front end.
# Modules to Import
import streamlit as st
import sys
import io
import re
from typing import Callable, Any
from langchain.agents.tools import Tool
from langchain.agents import initialize_agent
from langchain.llms import OpenAI
from langchain.chains import LLMChain
from langchain import LLMMathChain
from langchain import PromptTemplate
st.title()
syntax and also enables the user to enter their OpenAI API key using the st.text_input()
widget.# Set the title of the app
st.title("LangChain `Agent` Module Usage Demo App")
# Get the OpenAI API key from the user
OPENAI_API_KEY = st.text_input(
"Enter your OpenAI API Key to get started", type="password"
)
As we discussed in the previous sections, we need to define a template for the prompt that incorporates a placeholder for the user's query.
# Define a template for the prompt
template = """You are a friendly and polite AI Chat Assistant.
You must try to provide accurate but concise answers.
If you don't know the answer, just say "I don't know."
Question: {query}
Answer:
"""
# Create a prompt template object with the template
prompt = PromptTemplate(template=template, input_variables=["query"])
Next, we implement a conditional loop. If the user has provided an OpenAI API key, we proceed with the flow of the app. The user is asked to enter their query using the st.text_input() widget.
# Check if the user has entered an OpenAI API key
if OPENAI_API_KEY:
# Get the user's query
query = st.text_input("Ask me anything")
Once the user has the correct API keys inserted, from this point onward, we will proceed with the implementation of LangChain modules. Some of these modules may be new to us, while others may have already been covered in our previous sections.
Next, we create instances of the OpenAI language model, OpenAI, the LLMMathChain for maths-related queries, and the LLMChain for general-purpose queries.
# Check if the user has entered a query
if query:
# Create an instance of the OpenAI language model
llm = OpenAI(temperature=0, openai_api_key=OPENAI_API_KEY)
# Create an instance of the LLMMathChain for math-related queries
llm_math_chain = LLMMathChain.from_llm(llm=llm, verbose=True)
# Create an instance of the LLMChain for general-purpose queries
llm_chain = LLMChain(llm=llm, prompt=prompt)
Following that, we create a list of tools that the agent will utilize. Each tool comprises a name, a corresponding function to handle the query, and a brief description.
# Create a list of tools for the agent
tools = [
Tool(
name="Search",
func=llm_chain.run,
description="Useful for when you need to answer general purpose questions",
),
Tool(
name="Calculator",
func=llm_math_chain.run,
description="Useful for when you need to answer questions about math",
),
]
Further, we need to initialize a zero-shot agent with the tools and other parameters. This agent employs the ReAct framework to determine which tool to utilize based solely on the description associated with each tool. It is essential to provide a description of each tool.
# Initialize the zero-shot agent with the tools and parameters
zero_shot_agent = initialize_agent(
agent="zero-shot-react-description",
tools=tools,
llm=llm,
verbose=True,
max_iterations=3,
)
And now finally, we can easily call the zero-shot agent with the user's query using the run(query) method.
# st.write(zero_shot_agent.run(query))
However, this would only yield the final outcome of the result within our Streamlit UI, without providing access to the underlying LangChain thought process (i.e. the verbose ) that we typically observe in a Notebook environment. This information is crucial to understand which tools our agent is opting for based on the user query. To address this, a helper function called capture_and_display_output was created.
# Helper function to dump LangChain Verbose / Though Process
# Function to capture and display the output of a function
def capture_and_display_output(func: Callable[..., Any], args, **kwargs) -> Any:
# Redirect stdout to a string buffer
original_stdout = sys.stdout
sys.stdout = output_catcher = io.StringIO()
# Call the function and capture the response
response = func(args, *kwargs)
# Restore the original stdout and get the captured output
sys.stdout = original_stdout
output_text = output_catcher.getvalue()
# Clean the output text by removing escape sequences
cleaned_text = re.sub(r"\x1b\[([0-9]{1,2}(;[0-9]{1,2})?)?[m|K]", "", output_text)
# Split the cleaned text into lines and concatenate them with line breaks
lines = cleaned_text.split("\n")
concatenated_string = "\n".join([s if s else "\n" for s in lines])
# Display the captured output in an expander
with st.expander("Thoughts", expanded=True):
st.write(concatenated_string)
This function allows users to monitor the actions undertaken by the agent. Consequently, the response from the agent is displayed within the UI.
# Call the zero-shot agent with the user's query and capture the output
response = capture_and_display_output(zero_shot_agent.run, query)
Image by Author | Screen grasp of the app in local deployment displays the entire verbose or rather the thought process
The app can now be easily deployed by clicking the "Deploy" button on the top left-hand side. The deployed app will provide us with a unique URL that can be shared with everyone!
Image by Author | Screen grasp of the Databutton online workspace showing the Deploy options.
Yay! We have successfully built and deployed a LangChain-based web app from scratch. Here's the link to the app ! The app also consists of a view code page , which can be accessed via this link.
To test the web app, we will employ two different types of prompts. One will be a general question that can be answered by any LLMs, while the other will be a maths-related question. Our hypothesis is that the LangChain agents will intelligently determine which agents to execute and provide the most appropriate response. Let's proceed with the testing to validate our assumption.
Image by Author | Screen grasped from the deployed web app.
Two different prompts were used to validate our assumptions. Based on the thought process ( displayed in the UI under the thoughts expander ), we can easily interpret which Tool has been chosen by the Agent. (Left) Usage of LLMMath chain incorporating Tool (Right) Usage of a simple LLM Chain incorporating Tool.
To summarise, we have not only explored various aspects of working with LangChain and LLMs but have also successfully built and deployed a web app powered by LangChain. This demonstrates the versatility and capabilities of LangChain in enabling the development of powerful applications.
Avratanu Biswas, Ph.D. Student ( Biophysics ), Educator, and Content Creator, (Data Science, ML & AI ).