ad
ad

Chat with Dataframe - Streamlit Chatbot using Langchian Agent and Ollama | Generative AI | LLM

Science & Technology

Introduction

Introduction

Hello everyone, I am Saran and in this video, let's discuss how we can build a Streamlit chatbot where the user can upload a CSV or an Excel file and then chat with a data frame. For this chat, we will be using the open-source model "Gamma" through the Ollama library, which allows us to host our models on our local machine. We can also use other open-source models like Lambda 2, Lambda 3, and so on. The objective is consistent across these models.

Project Setup

Installing Required Libraries

First, we need to install the required libraries. These libraries are listed in the requirements.txt file:

  • wama to access the model
  • langchain and langchain-community to access the LLMs and agents
  • langchain-experimental for agents specifically
  • pandas and openpyxl for data frame operations
  • streamlit for building the user interface

To install these libraries, use the following pip command:

pip install -r requirements.txt

Project Directory Setup

Create a project directory named DF_chat_reenv and within it, create a virtual environment:

mkdir DF_chat_reenv
cd DF_chat_reenv
python -m venv venv
source venv/bin/activate

Ensure your requirements.txt file is within this directory, then run the pip install command:

pip install -r requirements.txt

Building the Streamlit Application

Creating Project Structure

  1. Create a directory named SRC.
  2. Inside SRC, create a Python file named main.py.

Importing Libraries

Within main.py, import the necessary libraries:

import pandas as pd
import streamlit as st
from langchain.agents import agent
from langchain.experimental.agents import create_pandas_dataframe_agent
from langchain.ollama import chatollama

Streamlit Configuration

Configure the Streamlit page:

st.set_page_config(
    page_title="DF Chat",
    page_icon="?",
    layout="centered"
)

Reading File Function

Create a helper function to read CSV and Excel files:

def read_data(file):
    if file.name.endswith(".csv"):
        return pd.read_csv(file)
    else:
        return pd.read_excel(file)

Setting Up Streamlit Widgets

Create a file upload widget and display the data frame:

uploaded_file = st.file_uploader("Choose a file", type=["csv", "xlsx", "xls"])

if uploaded_file:
    st.session_state.df = read_data(uploaded_file)
    st.write("Data Frame Preview:")
    st.dataframe(st.session_state.df.head())

Chat Functionality

Initializing Chat History

Initialize chat history in the session state:

if 'chat_history' not in st.session_state:
    st.session_state.chat_history = []

Displaying Chat History

Create a loop to display previous chat messages:

for message in st.session_state.chat_history:
    with st.chat_message(message["role"]):
        st.markdown(message["content"])

Creating Input Box

Create a chat input box for user queries:

user_prompt = st.chat_input("Ask LLM")

if user_prompt:
    with st.chat_message("user"):
        st.markdown(user_prompt)
    
    st.session_state.chat_history.append(("role": "user", "content": user_prompt))

Setting Up the LLM

Load the LLM using the following code:

llm = chatollama.ChatOllama(
    model="gamma-2b",
    temperature=0.7
)

Creating Data Frame Agent

Create a data frame agent:

pandas_df_agent = create_pandas_dataframe_agent(
    llm=llm,
    dataframe=st.session_state.df,
    verbose=True
)

Invoke the agent and update session state:

response = pandas_df_agent.invoke(user_prompt)
assistant_response = response['output']

st.session_state.chat_history.append((
    "role": "assistant",
    "content": assistant_response
))

with st.chat_message("assistant"):
    st.markdown(assistant_response)

Running the Application

To run the application, use the following command within your terminal:

streamlit run SRC/main.py

Introduction

  • Streamlit
  • Chatbot
  • Dataframe
  • CSV
  • Excel
  • Langchain
  • Agent
  • Ollama
  • LLM
  • Chat History
  • Dataframe Agent

Introduction

Q: What libraries are required for building this application?

A: You need wama, langchain, langchain-community, langchain-experimental, pandas, openpyxl, and streamlit.

Q: How do I install the necessary libraries?

A: Create a requirements.txt file listing all libraries and use the command pip install -r requirements.txt.

Q: How can I create a project structure?

A: Create a directory, add a virtual environment, and place your main.py in an SRC subdirectory.

Q: What types of files can be uploaded for data analysis?

A: You can upload CSV and Excel files.

Q: How does the chat function maintain a session history?

A: Chat history is maintained within the Streamlit session state to preserve previous conversation context.

Q: How do I initialize the LLM and DataFrame Agent?

A: Use the chatollama library to initialize the LLM and create_pandas_dataframe_agent for the DataFrame Agent.

Q: How do I run the Streamlit application?

A: Use the command streamlit run SRC/main.py within your terminal.