# Chunking

## Introduction

* A critical component of RAG is the **chunking process**, where large documents are divided into smaller, more manageable pieces called “**chunks**.”

Chunking serves multiple purposes in RAG:

* **Efficiency**: Smaller chunks reduce computational overhead during retrieval.
* **Relevance**: Precise chunks increase the likelihood of retrieving relevant information.
* **Context Preservation**: Proper chunking maintains the integrity of the information, ensuring coherent responses.

However, inappropriate chunking can lead to:

* **Loss of Context**: Breaking information at arbitrary points can disrupt meaning.
* **Redundancy**: Overlapping chunks may introduce repetitive information.
* **Inconsistency**: Variable chunk sizes can complicate retrieval and indexing.

## Strategies

### Fixed Length

<figure><img src="https://1374779285-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-MFW3x2aqEO8GF2kr3VU%2Fuploads%2F9QigcOBJnO3z1g4WFOKz%2Fimage.png?alt=media&#x26;token=7c37356e-ce36-4216-a9dc-d390f84b6043" alt=""><figcaption></figcaption></figure>

**Advantages**:

* **Simplicity**: Easy to implement without complex algorithms.
* **Uniformity**: Produces consistent chunk sizes, simplifying indexing.

**Challenges**:

* **Context Loss**: May split sentences or ideas, leading to incomplete information.
* **Relevance Issues**: Critical information might span multiple chunks, reducing retrieval effectiveness.

```javascript
import { CharacterTextSplitter } from "@langchain/textsplitters";
const textSplitter = new CharacterTextSplitter({
  chunkSize: 100,
  chunkOverlap: 0,
});
const texts = await textSplitter.splitText(document);
```

### Text-structured / Recursive based <a href="#text-structured-based" id="text-structured-based"></a>

<figure><img src="https://1374779285-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-MFW3x2aqEO8GF2kr3VU%2Fuploads%2FwOh9d5nMhIvLYG3qtjz2%2Fimage.png?alt=media&#x26;token=28d4ddcb-6612-4739-aad0-d1ad25cc350c" alt=""><figcaption></figcaption></figure>

* **Richer Context**: Provides more information than sentence-based chunks.
* **Logical Division**: Aligns with the natural structure of the text, split by the character that defined

**Challenges**:

* **Inconsistent Sizes**: Paragraph lengths can vary widely.
* **Token Limits**: Large paragraphs may exceed token limitations of the model.

```javascript
import { RecursiveCharacterTextSplitter } from "langchain/text_splitter";
import { Document } from "@langchain/core/documents";

const text = `Some other considerations include:

- Do you deploy your backend and frontend together, or separately?
- Do you deploy your backend co-located with your database, or separately?

**Production Support:** As you move your LangChains into production, we'd love to offer more hands-on support.
Fill out [this form](https://airtable.com/appwQzlErAS2qiP0L/shrGtGaVBVAz7NcV2) to share more about what you're building, and our team will get in touch.

## Deployment Options

See below for a list of deployment options for your LangChain app. If you don't see your preferred option, please get in touch and we can add it to this list.`;

const splitter = new RecursiveCharacterTextSplitter({
  chunkSize: 50,
  chunkOverlap: 1,
  separators: ["|", "##", ">", "-"],
});

const docOutput = await splitter.splitDocuments([
  new Document({ pageContent: text }),
]);

console.log(docOutput.slice(0, 3));

```

```
[
  Document {
    pageContent: "Some other considerations include:",
    metadata: { loc: { lines: { from: 1, to: 1 } } }
  },
  Document {
    pageContent: "- Do you deploy your backend and frontend together",
    metadata: { loc: { lines: { from: 3, to: 3 } } }
  },
  Document {
    pageContent: "r, or separately?",
    metadata: { loc: { lines: { from: 3, to: 3 } } }
  }
]
```

```javascript
const markdownText = `
# 🦜️🔗 LangChain

⚡ Building applications with LLMs through composability ⚡

## Quick Install

\`\`\`bash
# Hopefully this code block isn't split
pip install langchain
\`\`\`

As an open-source project in a rapidly developing field, we are extremely open to contributions.
`;

const mdSplitter = RecursiveCharacterTextSplitter.fromLanguage("markdown", {
  chunkSize: 60,
  chunkOverlap: 0,
});
const mdDocs = await mdSplitter.createDocuments([markdownText]);

mdDocs;
```

````
[
  Document {
    pageContent: "# 🦜️🔗 LangChain",
    metadata: { loc: { lines: { from: 2, to: 2 } } }
  },
  Document {
    pageContent: "⚡ Building applications with LLMs through composability ⚡",
    metadata: { loc: { lines: { from: 4, to: 4 } } }
  },
  Document {
    pageContent: "## Quick Install",
    metadata: { loc: { lines: { from: 6, to: 6 } } }
  },
  Document {
    pageContent: "```bash\n# Hopefully this code block isn't split",
    metadata: { loc: { lines: { from: 8, to: 9 } } }
  },
  Document {
    pageContent: "pip install langchain",
    metadata: { loc: { lines: { from: 10, to: 10 } } }
  },
  Document {
    pageContent: "```",
    metadata: { loc: { lines: { from: 11, to: 11 } } }
  },
  Document {
    pageContent: "As an open-source project in a rapidly developing field, we",
    metadata: { loc: { lines: { from: 13, to: 13 } } }
  },
  Document {
    pageContent: "are extremely open to contributions.",
    metadata: { loc: { lines: { from: 13, to: 13 } } }
  }
]
````

### Semantic Chunking <a href="#id-5ea1" id="id-5ea1"></a>

<figure><img src="https://1374779285-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-MFW3x2aqEO8GF2kr3VU%2Fuploads%2FvYX5pzpVLuww0ea98zEM%2Fimage.png?alt=media&#x26;token=db6c8ce2-46a7-4e1c-86b0-9d1ac33375ed" alt=""><figcaption></figcaption></figure>

**How it works**: Utilizes embeddings or machine learning models to split text based on semantic meaning, ensuring each chunk is cohesive in topic or idea.

**Best for**: Complex queries requiring deep understanding, such as technical manuals or academic papers.

**Advantages**:

* **Contextual Relevance**: Chunks are meaningfully grouped, improving retrieval accuracy.
* **Flexibility**: Adapts to the text’s inherent structure and content.

**Challenges**:

* **Complexity**: Requires advanced NLP models and computational resources.
* **Processing Time**: Semantic analysis can be time-consuming.

```python
from langchain_experimental.text_splitter import SemanticChunker
from langchain_openai.embeddings import OpenAIEmbeddings

text_splitter = SemanticChunker(OpenAIEmbeddings())
docs = text_splitter.create_documents([state_of_the_union])
print(docs[0].page_content)
```

## References

{% embed url="<https://python.langchain.com/docs/concepts/text_splitters/#semantic-meaning-based>" %}

{% embed url="<https://masteringllm.medium.com/11-chunking-strategies-for-rag-simplified-visualized-df0dbec8e373>" %}
