How to work with LangChain Python modules

This article is the second part of a series of articles, please refer to Part 1 for learning how to Get to grips with LangChain framework and how to utilize it for building LLM-powered Apps

Introduction

In this section, we dive into the practical usage of LangChain modules. Building upon the previous overview of LangChain components, we will work within a Python environment to gain hands-on coding experience. However, it is important to note that this overview is not a substitute for the official documentation, and it is recommended to refer to the documentation for a more comprehensive understanding.

Choosing the Right Python Environment

When working with Python, Jupyter Notebook and Google Colab are popular choices for quickly getting started in the Python environment. Additionally, Visual Studio Code (VSCode) Atom, PyCharm, or Sublime Text integrated with a conda environment are also excellent options. While many of these can be used, Google Colab is used here for its convenience in quick testing and code sharing. Find the code link here.

Prerequisites

Before we begin, make sure to install the necessary Python libraries. Use the pip command within a notebook cell to install them.

Installing LangChain: In order to install the "LangChain" library, which is essential for this section, you can conveniently use the following command:

!pip install langchain

Regular Updates: Personally, I would recommend taking advantage of LangChain’s frequent releases by frequently upgrading the packages. Use the following command for this purpose:

!pip install langchain  - -  upgrade

Integrating LangChain with LLMs: Previously, we discussed how the LangChain library facilitates interaction with Large Language Models (LLMs) provided by platforms such as OpenAI, Cohere, or HuggingFace. To integrate LangChain with these models, we need to follow these steps:

Obtain API Keys: In this tutorial, we will use OpenAI. We need to sign up; to easily access the API keys for the various endpoints which Open AI provides. The key must be confidential. You can obtain the API via this link.
Install Python Package: Install the required Python package associated with your chosen LLM provider. For OpenAI language models, execute the command:

!pip install openai

Configuring the API Key for OpenAI: To initialize the API key for the OpenAI library, we will use the getpass Python Library. Alternatively, you can set the API key as an environment variable.

# Importing the library
OPENAI_API_KEY = getpass.getpass()
import getpass
# In order to double check
# print(OPENAI_API_KEY) # not recommended

Running the above lines of code will create a secure text input widget where we can enter the API key, obtained for accessing OpenAI LLMs endpoints. After hitting enter, the inputted value will be stored as the assigned variable OPENAI_API_KEY, allowing it to be used for subsequent operations throughout our notebook.

We will explore different LangChain modules in the section below:

Prompt Template

We need to import the necessary module, PromptTemplate, from the langchain library. A multi-line string variable named template is created - representing the structure of the prompt and containing placeholders for the context, question, and answer which are the crucial aspects of any prompt template.

how-to-work-with-langchain-python-modules-img-0

Image by Author | Key components of a prompt template is shown in the figure.

A PromptTemplate the object is instantiated using the template variable. The input_variables parameter is provided with a list containing the variable names used in the template, in this case, only the query.:

from langchain import PromptTemplate

template = """ You are a Scientific Chat Assistant.
Your job is to answer scientific facts and evidence, in a bullet point wise.

Context: Scientific evidence is necessary to validate claims, establish credibility,
and make informed decisions based on objective and rigorous investigation.

Question: {query}

Answer:
"""

prompt = PromptTemplate(template=template, input_variables=["query"])

The generated prompt structure can be further utilized to dynamically fill in the question placeholder and obtain responses within the specified template format. Let's print our entire prompt!

print(prompt)
lc_kwargs={'template': ' You are an Scientific Chat Assistant.\nYour job is to reply scientific facts and evidence in a bullet point wise.\n\nContext: Scientific evidence is necessary to validate claims, establish credibility, \nand make informed decisions based on objective and rigorous investigation.\n\nQuestion: {query}\n\nAnswer: \n', 'input_variables': ['query']} input_variables=['query'] output_parser=None partial_variables={} template=' You are an Scientific Chat Assistant.\nYour job is to reply scientific facts and evidence in a bullet point wise.\n\nContext: Scientific evidence is necessary to validate claims, establish credibility, \nand make informed decisions based on objective and rigorous investigation.\n\nQuestion: {query}\n\nAnswer: \n' template_format='f-string' validate_template=True

Chains

The LangChain documentation covers various types of LLM chains, which can be effectively categorized into two main groups: Generic chains and Utility chains.

how-to-work-with-langchain-python-modules-img-1

Image 2: Chains

Chains can be broadly classified into Generic Chains and Utility Chains. (a) Generic chains are designed to provide general-purpose language capabilities, such as generating text, answering questions, and engaging in natural language conversations by leveraging LLMs. On the other contrary, (b) Utility Chains: are specialized to perform specific tasks or provide targeted functionalities. These chains are fine-tuned and optimized for specific use cases. Note, although Index-related chains can be classified into a sub-group, here we keep such chains under the banner of utility chains. They are often considered to be very useful while working with Vector databases.

Since this is the very first time we are running the LLM chain, we will walk through the code in detail.
We need to import the OpenAI LLM module from langchain.llms and the LLMChain module from langchain Python package.
Then, an instance of the OpenAI LLM is created, using the arguments such as temperature (affects the randomness of the generated responses), openai_api_key (the API key for OpenAI which we just assigned before), model (the specific OpenAI language model to be used - other models are available here), and streaming.
Note the verbose argument is pretty useful to understand the abstraction that LangChain provides under the hood, while executing our query.
Next, an instance of LLMChain is created, providing the prompt (the previously defined prompt template) and the LLM (the OpenAI LLM instance).
The query or question is defined as the variable query.
Finally, the llm_chain.run(query) line executes the LLMChain with the specified query, generating the response based on the defined prompt and the OpenAI LLM:

# Importing the OpenAI LLM module
from langchain.llms import OpenAI
# Importing the LLMChain module
from langchain import LLMChain

# Creating an instance of the OpenAI LLM
llm = OpenAI(temperature=0.9, openai_api_key=OPENAI_API_KEY, model="text-davinci-003", streaming=True)

# Creating an instance of the LLMChain with the provided prompt and OpenAI LLM
llm_chain = LLMChain(prompt=prompt,llm=llm, verbose=True)

# Defining the query or question to be asked
query = "What is photosynthesis?"

# Running the LLMChain with the specified query
print(llm_chain.run(query))

Let's have a look at the response that is generated after running the chain with and without verbose,

a) with verbose = True;

Prompt after formatting:

You are an Scientific Chat Assistant. Your job is to reply scientific facts and evidence in a bullet point wise.

Context: Scientific evidence is necessary to validate claims, establish credibility, and make informed decisions based on objective and rigorous investigation. Question: What is photosynthesis?

Answer:

> Finished chain.

• Photosynthesis is the process used by plants, algae and certain bacteria to convert light energy from the sun into chemical energy in the form of sugars.

• Photosynthesis occurs in two stages: the light reactions and the Calvin cycle. • During the light reactions, light energy is converted into ATP and NADPH molecules.

• During the Calvin cycle, ATP and NADPH molecules are used to convert carbon dioxide into sugar molecules.

b ) with verbose = False;

• Photosynthesis is a process used by plants and other organisms to convert light energy, normally from the sun, into chemical energy which can later be released to fuel the organisms' activities.

• During photosynthesis, light energy is converted into chemical energy and stored in sugars.

• Photosynthesis occurs in two stages: light reactions and the Calvin cycle. The light reactions trap light energy and convert it into chemical energy in the form of the energy-storage molecule ATP. The Calvin cycle uses ATP and other molecules to create glucose.

Seems like our general-purpose LLMChain has done a pretty decent job and given a reasonable output by leveraging the LLM.

Now let's move onto the utility chain and understand it, using a simple code snippet:

from langchain import OpenAI
from langchain import LLMMathChain
llm = OpenAI(temperature=0.9,openai_api_key= OPENAI_API_KEY)
# Using the LLMMath Chain / LLM defined in Prompt Template section
llm_math = LLMMathChain.from_llm(llm = llm, verbose = True)
question = "What is 4 times 5"
llm_math.run(question)
# You know what the response would be 🎈

Here the utility chain serves a specific function, i.e. to solve a fundamental maths question using the LLMMathChain. It's crucial to look at the prompt used under the hood for such chains. However , in addition, a few more notable utility chains are there as well,

BashChain: A utility chain designed to execute Bash commands and scripts.
SQLDatabaseChain: This utility chain enables interaction with SQL databases
SummarizationChain: The SummarizationChain is designed specifically for text summarization tasks.

Such utility chains, along with other available chains in the LangChain framework, provide specialized functionalities and ready-to-use tools that can be utilized to expedite and enhance various aspects of the language processing pipeline.

Memory

Until now, we have seen, each incoming query or input to the LLMs or to its subsequent chain is treated as an independent interaction, meaning it is "stateless" (in simpler terms, information IN, information OUT). This can be considered as one of the major drawbacks, as it hinders the ability to provide a seamless and natural conversational experience for users who are seeking reasonable responses further on. To overcome this limitation and enable better context retention, LangChain offers a broad spectrum of memory components that are extremely helpful.

how-to-work-with-langchain-python-modules-img-2

Image by Author | The various types of Memory modules that LangChain provides.

By utilizing the memory components supported, it becomes possible to remember the context of the conversation, making it more coherent and intuitive. These memory components allow for the storage and retrieval of information, enabling the LLMs to have a sense of continuity. This means they can refer back to previous relevant contexts, which greatly enhances the conversational experience for users. A typical example of such memory-based interaction is the very popular chatbot - ChatGPT, which remembers the context of our conversations.

Let's have a look at how we can leverage such a possibility using LangChain:

from langchain.llms import OpenAI
from langchain.chains import ConversationChain
from langchain.memory import ConversationBufferMemory

llm = OpenAI(temperature=0, openai_api_key= OPENAI_API_KEY)
conversation = ConversationChain(
llm=llm,
verbose=True,
memory = ConversationBufferMemory()
)

In the above code, we have initialized an instance of the ConversationChain class, configuring it with the OpenAI language model, enabling verbose mode for detailed output, and utilizing a ConversationBufferMemory for memory management during conversations. Now, let's begin our conversation,

conversation.predict(input="Hi there!I'm Avra")

Prompt after formatting:

The following is a friendly conversation between a human and an AI. The AI is talkative and provides lots of specific details from its context. If the AI does not know the answer to a question, it truthfully says it does not know.

Current conversation:

Human: Hi there! I'm Avra

AI:

> Finished chain.

' Hi, Avra! It's nice to meet you. My name is AI. What can I do for you today?

Let's add a few more contexts to the chain, so that later we can test the context memory of the chain.

conversation.predict(input="I'm interested in soccer and building AI web apps.")

Prompt after formatting:

Current conversation:

Human: Hi there!I'm Avra

AI: Hi Avra! It's nice to meet you. My name is AI. What can I do for you today?

Human: I'm interested in soccer and building AI web apps.

AI:

> Finished chain.

' That's great! Soccer is a great sport and AI web apps are a great way to explore the possibilities of artificial intelligence. Do you have any specific questions about either of those topics?

Now, we make a query, which requires the chain to trace back to its memory storage and provide a reasonable response based on it.

conversation.predict(input="Who am I and what's my interest ?")

Prompt after formatting:

The following is a friendly conversation between a human and an AI. The AI is talkative and provides lots of specific details from its context. If the AI does not know the answer to a question, it truthfully says it does not know.

Current conversation:

Human: Hi there!I'm Avra

AI: Hi Avra! It's nice to meet you. My name is AI. What can I do for you today?

Human: I'm interested in soccer and building AI web apps.

AI: That's great! Soccer is a great sport and AI web apps are a great way to explore the possibilities of artificial intelligence. Do you have any specific questions about either of those topics?

Human: Who am I and what's my interest ?

AI:

> Finished chain.

' That's a difficult question to answer. I don't have enough information to answer that question. However, based on what you've told me, it seems like you are Avra and your interests are soccer and building AI web apps.

The above response highlights the significance of the ConversationBufferMemory chain in retaining the context of the conversation. It would be worthwhile to try out the above example without a buffer memory to get a clear perspective of the importance of the memory module. Additionally, LangChain provides several memory modules that can enhance our understanding of memory management in different ways, to handle conversational contexts.

Moving forward, we will delve into the next section, where we will focus on the final two components called the “Indexes” and the "Agent." During this section, we will not only gain a hands-on understanding of its usage but also build and deploy a web app using an online workspace called Databutton.