LangChain

Introduction

• LangChain is essentially a library of abstractions for Python and Javascript, representing common steps and concepts necessary to work with language models. These modular components—like functions and object classes—serve as the building blocks of generative AI programs.

• Components can be “chained” together to create applications for minimizing the amount of code and fine understanding required to execute complex NLP tasks.

• Importing language models into LangChain is easy, provided you have an API key. The LLM class is designed to provide a standard interface for all models.

Chain

• any two runnables can be “chained” together into sequences. The output of the previous runnable’s .invoke() call is passed as input to the next runnable. This can be done using the .pipe() method.

import { ChatOpenAI } from "@langchain/openai";
import { StringOutputParser } from "@langchain/core/output_parsers";
import { ChatPromptTemplate } from "@langchain/core/prompts";

const model = new ChatOpenAI({
  model: "gpt-4o-mini",
  temperature: 0
});


const prompt = ChatPromptTemplate.fromTemplate("tell me a joke about {topic}");

const chain = prompt.pipe(model).pipe(new StringOutputParser());
await chain.invoke({ topic: "bears" });

import { RunnableSequence } from "@langchain/core/runnables";

const composedChainWithLambda = RunnableSequence.from([
  chain,
  (input) => ({ joke: input }),
  analysisPrompt,
  model,
  new StringOutputParser(),
]);

await composedChainWithLambda.invoke({ topic: "beets" });

Prompt Templates

• To format user input into a format that can be passed to a language model.

import { FewShotPromptTemplate } from "@langchain/core/prompts";

const examples = [
  {
    question: "Who lived longer, Muhammad Ali or Alan Turing?",
    answer: `
  Are follow up questions needed here: Yes.
  Follow up: How old was Muhammad Ali when he died?
  Intermediate answer: Muhammad Ali was 74 years old when he died.
  Follow up: How old was Alan Turing when he died?
  Intermediate answer: Alan Turing was 41 years old when he died.
  So the final answer is: Muhammad Ali
  `,
  },
];
const prompt = new FewShotPromptTemplate({
  examples,
  examplePrompt,
  suffix: "Question: {input}",
  inputVariables: ["input"],
});

const formatted = await prompt.format({
  input: "Who was the father of Mary Ball Washington?",
});
console.log(formatted.toString());
// Question: Who lived longer, Muhammad Ali or Alan Turing?

//  Are follow up questions needed here: Yes.
//  Follow up: How old was Muhammad Ali when he died?
//  Intermediate answer: Muhammad Ali was 74 years old when he died.
//  Follow up: How old was Alan Turing when he died?
//  Intermediate answer: Alan Turing was 41 years old when he died.
 // So the final answer is: Muhammad Ali
 // Question: Who was the father of Mary Ball Washington?
// Question: Who was the father of Mary Ball Washington?

Example Selector

• To select the correct few shot examples to pass to the prompt.

import { BaseExampleSelector } from "@langchain/core/example_selectors";
import { Example } from "@langchain/core/prompts";

class CustomExampleSelector extends BaseExampleSelector {
  private examples: Example[];

  constructor(examples: Example[]) {
    super();
    this.examples = examples;
  }

  async addExample(example: Example): Promise<void | string> {
    this.examples.push(example);
    return;
  }

  async selectExamples(inputVariables: Example): Promise<Example[]> {
    // This assumes knowledge that part of the input will be a 'text' key
    const newWord = inputVariables.input;
    const newWordLength = newWord.length;

    // Initialize variables to store the best match and its length difference
    let bestMatch: Example | null = null;
    let smallestDiff = Infinity;

    // Iterate through each example
    for (const example of this.examples) {
      // Calculate the length difference with the first word of the example
      const currentDiff = Math.abs(example.input.length - newWordLength);

      // Update the best match if the current one is closer in length
      if (currentDiff < smallestDiff) {
        smallestDiff = currentDiff;
        bestMatch = example;
      }
    }

    return bestMatch ? [bestMatch] : [];
  }
}

const examples = [
  { input: "hi", output: "ciao" },
  { input: "bye", output: "arrivaderci" },
  { input: "soccer", output: "calcio" },
];

const exampleSelector = new CustomExampleSelector(examples);
await exampleSelector.selectExamples({ input: "okay" });
// [ { input: "bye", output: "arrivaderci" } ]

Chat Models

• Integrate with different engines models that take message in and output a message

import { initChatModel } from "langchain/chat_models/universal";

// Returns a @langchain/openai ChatOpenAI instance.
const gpt4o = await initChatModel("gpt-4o", {
  modelProvider: "openai",
  temperature: 0,
});
// Returns a @langchain/anthropic ChatAnthropic instance.
const claudeOpus = await initChatModel("claude-3-opus-20240229", {
  modelProvider: "anthropic",
  temperature: 0,
});
// Returns a @langchain/google-vertexai ChatVertexAI instance.
const gemini15 = await initChatModel("gemini-1.5-pro", {
  modelProvider: "google-vertexai",
  temperature: 0,
});

// Since all model integrations implement the ChatModel interface, you can use them in the same way.
console.log(`GPT-4o: ${(await gpt4o.invoke("what's your name")).content}\n`);
console.log(
  `Claude Opus: ${(await claudeOpus.invoke("what's your name")).content}\n`
);
console.log(
  `Gemini 1.5: ${(await gemini15.invoke("what's your name")).content}\n`
);

/*
GPT-4o: I'm an AI language model created by OpenAI, and I don't have a personal name. You can call me Assistant or any other name you prefer! How can I help you today?

Claude Opus: My name is Claude. It's nice to meet you!

Gemini 1.5: I don't have a name. I am a large language model, and I am not a person. I am a computer program that can generate text, translate languages, write different kinds of creative content, and answer your questions in an informative way.
*/

Message

• It is the input and output of chat models. we can use the utiltity functions to control the output

import {
  AIMessage,
  HumanMessage,
  SystemMessage,
  trimMessages,
} from "@langchain/core/messages";
import { ChatOpenAI } from "@langchain/openai";

const messages = [
  new SystemMessage("you're a good assistant, you always respond with a joke."),
  new HumanMessage("i wonder why it's called langchain"),
  new AIMessage(
    'Well, I guess they thought "WordRope" and "SentenceString" just didn\'t have the same ring to it!'
  ),
  new HumanMessage("and who is harrison chasing anyways"),
  new AIMessage(
    "Hmmm let me think.\n\nWhy, he's probably chasing after the last cup of coffee in the office!"
  ),
  new HumanMessage("what do you call a speechless parrot"),
];

const trimmed = await trimMessages(messages, {
  maxTokens: 45,
  strategy: "last",
  tokenCounter: new ChatOpenAI({ modelName: "gpt-4" }),
});

console.log(
  trimmed
    .map((x) =>
      JSON.stringify(
        {
          role: x._getType(),
          content: x.content,
        },
        null,
        2
      )
    )
    .join("\n\n")
);
// {
//  "role": "human",
//  "content": "and who is harrison chasing anyways"
// }

// {
//  "role": "ai",
//  "content": "Hmmm let me think.\n\nWhy, he's probably chasing after the last cup of coffee in the office!"
// }

// {
//  "role": "human",
//  "content": "what do you call a speechless parrot"
// }

Output Parser

• It is for taking the output of an LLM and parsing into more structured format.

import { StringOutputParser } from "@langchain/core/output_parsers";
import { HumanMessage, SystemMessage } from "@langchain/core/messages";

const messages = [
  new SystemMessage("Translate the following from English into Italian"),
  new HumanMessage("hi!"),
];

const parser = new StringOutputParser();
const result = await model.invoke(messages);
await parser.invoke(result);
// "ciao"

Document Loader

• It is responsible for loading documents from a variety of sources.

• Sample CSV

id,text
1,This is a sentence.
2,This is another sentence.

import { CSVLoader } from "@langchain/community/document_loaders/fs/csv";

const loader = new CSVLoader("src/document_loaders/example_data/example.csv");

const docs = await loader.load();
/*
[
  Document {
    "metadata": {
      "line": 1,
      "source": "src/document_loaders/example_data/example.csv",
    },
    "pageContent": "id: 1
text: This is a sentence.",
  },
  Document {
    "metadata": {
      "line": 2,
      "source": "src/document_loaders/example_data/example.csv",
    },
    "pageContent": "id: 2
text: This is another sentence.",
  },
]
*/

Text Splitter

• To split the large text into chunks

// Load the web source from internet
const pTagSelector = "p";
const loader = new CheerioWebBaseLoader(
  "https://lilianweng.github.io/posts/2023-06-23-agent/",
  {
    selector: pTagSelector,
  }
);

const docs = await loader.load();

// split our documents into chunks of 1000 characters 
// with 200 characters of overlap between chunks. 
const textSplitter = new RecursiveCharacterTextSplitter({
  chunkSize: 1000,
  chunkOverlap: 200,
});
const allSplits = await textSplitter.splitDocuments(docs)
console.log(allSplits.length); // 28
console.log(allSplits[0].pageContent.length); // 996
console.log(allSplits[10].metadata);
// {
//  source: "https://lilianweng.github.io/posts/2023-06-23-agent/",
//  loc: { lines: { from: 1, to: 1 } }
// }

Embedding models

• Take a text as a input , and convert it into vectors

import { OpenAIEmbeddings } from "@langchain/openai";

const embeddings = new OpenAIEmbeddings();
const res = await embeddings.embedQuery("Hello world");
/*
[
   -0.004845875,   0.004899438,  -0.016358767,  -0.024475135, -0.017341806,
    0.012571548,  -0.019156644,   0.009036391,  -0.010227379, -0.026945334,
    0.022861943,   0.010321903,  -0.023479493, -0.0066544134,  0.007977734,
   0.0026371893,   0.025206111,  -0.012048521,   0.012943339,  0.013094575,
   -0.010580265,  -0.003509951,   0.004070787,   0.008639394, -0.020631202,
  ... 1511 more items
]
*/

Vector Store

• To store and search over unstructured data is to embed it and store the resulting embedding vectors, and then at query time to embed the unstructured query and retrieve the embedding vectors that are 'most similar' to the embedded query.

import { MemoryVectorStore } from "langchain/vectorstores/memory";
import { OpenAIEmbeddings } from "@langchain/openai";
import { TextLoader } from "langchain/document_loaders/fs/text";

// Create docs with a loader
const loader = new TextLoader("src/document_loaders/example_data/example.txt");
const docs = await loader.load();

// Load the docs into the vector store
const vectorStore = await MemoryVectorStore.fromDocuments(
  docs,
  new OpenAIEmbeddings()
);

// Search for the most similar document
const resultOne = await vectorStore.similaritySearch("hello world", 1);

console.log(resultOne);

/*
  [
    Document {
      pageContent: "Hello world",
      metadata: { id: 2 }
    }
  ]
*/

Retrievers

• For taking a query and returning relevant documents, can be used with vector store for similarity search

import * as fs from "node:fs";

import { OpenAIEmbeddings, ChatOpenAI } from "@langchain/openai";
import { RecursiveCharacterTextSplitter } from "@langchain/textsplitters";
import { MemoryVectorStore } from "langchain/vectorstores/memory";
import {
  RunnablePassthrough,
  RunnableSequence,
} from "@langchain/core/runnables";
import { StringOutputParser } from "@langchain/core/output_parsers";
import { ChatPromptTemplate } from "@langchain/core/prompts";
import type { Document } from "@langchain/core/documents";

const formatDocumentsAsString = (documents: Document[]) => {
  return documents.map((document) => document.pageContent).join("\n\n");
};

// Initialize the LLM to use to answer the question.
const model = new ChatOpenAI({
  model: "gpt-4o",
});
const text = fs.readFileSync("state_of_the_union.txt", "utf8");
const textSplitter = new RecursiveCharacterTextSplitter({ chunkSize: 1000 });
const docs = await textSplitter.createDocuments([text]);
// Create a vector store from the documents.
const vectorStore = await MemoryVectorStore.fromDocuments(
  docs,
  new OpenAIEmbeddings()
);

// Initialize a retriever wrapper around the vector store
const vectorStoreRetriever = vectorStore.asRetriever();

// Create a system & human prompt for the chat model
const SYSTEM_TEMPLATE = `Use the following pieces of context to answer the question at the end.
If you don't know the answer, just say that you don't know, don't try to make up an answer.
----------------
{context}`;

const prompt = ChatPromptTemplate.fromMessages([
  ["system", SYSTEM_TEMPLATE],
  ["human", "{question}"],
]);

const chain = RunnableSequence.from([
  {
    context: vectorStoreRetriever.pipe(formatDocumentsAsString),
    question: new RunnablePassthrough(),
  },
  prompt,
  model,
  new StringOutputParser(),
]);

const answer = await chain.invoke(
  "What did the president say about Justice Breyer?"
);

console.log({ answer });

/*
  {
    answer: 'The president honored Justice Stephen Breyer by recognizing his dedication to serving the country as an Army veteran, Constitutional scholar, and retiring Justice of the United States Supreme Court. He thanked Justice Breyer for his service.'
  }
*/

Tool

• Tool contain a description of the tool (to pass to the language model) as well as the implementation of the function to call.

• Tool calling allows a chat model to respond to a given prompt by “calling a tool”. The model only generates the arguments to a tool, and actually running the tool (or not) is up to the user.

import { tool } from "@langchain/core/tools";
import { z } from "zod";

/**
 * Note that the descriptions here are crucial, as they will be passed along
 * to the model along with the class name.
 */
const calculatorSchema = z.object({
  operation: z
    .enum(["add", "subtract", "multiply", "divide"])
    .describe("The type of operation to execute."),
  number1: z.number().describe("The first number to operate on."),
  number2: z.number().describe("The second number to operate on."),
});

const calculatorTool = tool(
  async ({ operation, number1, number2 }) => {
    // Functions must return strings
    if (operation === "add") {
      return `${number1 + number2}`;
    } else if (operation === "subtract") {
      return `${number1 - number2}`;
    } else if (operation === "multiply") {
      return `${number1 * number2}`;
    } else if (operation === "divide") {
      return `${number1 / number2}`;
    } else {
      throw new Error("Invalid operation.");
    }
  },
  {
    name: "calculator",
    description: "Can perform mathematical operations.",
    schema: calculatorSchema,
  }
);

const llmWithTools = llm.bindTools([calculatorTool]);
const res = await llmWithTools.invoke("What is 3 * 12? Also, what is 11 + 49?");

console.log(res.tool_calls);

/* 
[
  {
    name: 'calculator',
    args: { operation: 'multiply', number1: 3, number2: 12 },
    type: 'tool_call',
    id: 'call_01lvdk2COLV2hTjRUNAX8XWH'
  },
  {
    name: 'calculator',
    args: { operation: 'add', number1: 11, number2: 49 },
    type: 'tool_call',
    id: 'call_fB0vo8VC2HRojZcj120xIBxM'
  }
]
*/

LangGraph

• It is a library for building stateful, multi-actor applications with LLMs, used to create agent and multi-agent workflows.

• Cycles and Branching: Implement loops and conditionals in your apps.

• Persistence: Automatically save state after each step in the graph. Pause and resume the graph execution at any point to support error recovery, human-in-the-loop workflows, time travel and more.

• Human-in-the-Loop: Interrupt graph execution to approve or edit next action planned by the agent.

• Streaming Support: Stream outputs as they are produced by each node (including token streaming).

• Integration with LangChain: LangGraph integrates seamlessly with LangChain and LangSmith (but does not require them).

import { AIMessage, BaseMessage, HumanMessage } from "@langchain/core/messages";
import { tool } from "@langchain/core/tools";
import { z } from "zod";
import { ChatAnthropic } from "@langchain/anthropic";
import { StateGraph, StateGraphArgs } from "@langchain/langgraph";
import { MemorySaver } from "@langchain/langgraph";
import { ToolNode } from "@langchain/langgraph/prebuilt";

// Define the state interface
interface AgentState {
  messages: BaseMessage[];
}

// Define the graph state
const graphState: StateGraphArgs<AgentState>["channels"] = {
  messages: {
    reducer: (x: BaseMessage[], y: BaseMessage[]) => x.concat(y),
  },
};

// Define the tools for the agent to use
const weatherTool = tool(async ({ query }) => {
  // This is a placeholder for the actual implementation
  if (query.toLowerCase().includes("sf") || query.toLowerCase().includes("san francisco")) {
    return "It's 60 degrees and foggy."
  }
  return "It's 90 degrees and sunny."
}, {
  name: "weather",
  description:
    "Call to get the current weather for a location.",
  schema: z.object({
    query: z.string().describe("The query to use in your search."),
  }),
});

const tools = [weatherTool];
const toolNode = new ToolNode<AgentState>(tools);

// Define the model and bine with tool
const model = new ChatAnthropic({
  model: "claude-3-5-sonnet-20240620",
  temperature: 0,
}).bindTools(tools);

// Define the function that determines whether to continue or not
function shouldContinue(state: AgentState) {
  const messages = state.messages;
  const lastMessage = messages[messages.length - 1] as AIMessage;

  // If the LLM makes a tool call, then we route to the "tools" node
  // and execute the tool implementation logic
  if (lastMessage.tool_calls?.length) {
    return "tools";
  }
  // Otherwise, we stop (reply to the user)
  return "__end__";
}

// Define the function that calls the model
async function callModel(state: AgentState) {
  const messages = state.messages;
  const response = await model.invoke(messages);

  // We return a list, because this will get added to the existing list
  return { messages: [response] };
}

// Define a new graph
// Define the node with key and implementation
// Define the starting point 
// Define the conditional logic
const workflow = new StateGraph<AgentState>({ channels: graphState })
  .addNode("agent", callModel)
  .addNode("tools", toolNode)
  .addEdge("__start__", "agent")
  .addConditionalEdges("agent", shouldContinue)
  .addEdge("tools", "agent");

// Initialize memory to persist state between graph runs
const checkpointer = new MemorySaver();

// Finally, we compile it!
// This compiles it into a LangChain Runnable.
// Note that we're (optionally) passing the memory when compiling the graph
const app = workflow.compile({ checkpointer });

// Use the Runnable
const finalState = await app.invoke(
  { messages: [new HumanMessage("what is the weather in sf")] },
  { configurable: { thread_id: "42" } }
);

console.log(finalState.messages[finalState.messages.length - 1].content);

/*
Based on the information I received, the current weather in San Francisco is:

Temperature: 60 degrees Fahrenheit
Conditions: Foggy

San Francisco is known for its foggy weather, especially during certain times of the year. The moderate temperature of 60°F (about 15.5°C) is quite typical for the city, which generally has mild weather year-round due to its coastal location.

Is there anything else you'd like to know about the weather in San Francisco or any other location?
*/