# Building a HybridRAG System for Financial Document Analysis: An End-to-End flow * **Introduction** Financial document analysis presents unique challenges for Large Language Models (LLMs). Domain-specific terminology, complex relationships, and the sheer volume of data demand a sophisticated approach. Hybrid Retrieval-Augmented Generation (HybridRAG) offers a powerful solution by combining the strengths of VectorRAG and GraphRAG. This blog post provides a comprehensive guide to building a HybridRAG system, covering everything from data ingestion to deployment, and leveraging cutting-edge technologies like Cortex Search, Cortex Native Embedding Model, Amazon Bedrock, and AWS Neptune. **1\. Understanding the HybridRAG Advantage** * Briefly recap the limitations of traditional VectorRAG and GraphRAG when applied to financial documents (summarize points from the whitepaper introduction). * **Hybrid Approach Benefits:** Explain how HybridRAG overcomes these limitations by: * Providing richer context through both semantic similarity (VectorRAG) and structured relationships (GraphRAG). * Improving accuracy and reducing hallucination by grounding responses in verifiable data. * Enabling more nuanced insights by capturing complex connections between financial entities and metrics. **2\. Architecture Overview: A High-Level View** * Present a simplified architecture diagram illustrating the key components of the HybridRAG system: * Data Ingestion (S3) * Document Chunking * Cortex Embedding Generation * Cortex Search (Vector Database) * Bedrock LLM for Knowledge Graph Construction * AWS Neptune (Graph Database) * Hybrid Retrieval and Fusion Logic * Bedrock LLM for Response Generation * User Interface (e.g., Streamlit) * **Key Technologies:** Briefly introduce each technology and its role in the architecture. **3\. Step-by-Step Implementation Guide** This section will walk the reader through the process of building the HybridRAG system. It will be divided into sub-sections, each covering a specific stage of the implementation. * **3.1. Setting Up the Infrastructure** * **AWS Account and Permissions:** Provide a checklist of AWS resources needed and the necessary IAM permissions (S3, SageMaker (if using it for custom model training), Neptune, Lambda, Bedrock access). * **Cortex Setup:** * Explain how to deploy Cortex Search and the Cortex Native Embedding Model. * Provide instructions on obtaining API keys and configuring access. * Link to Cortex documentation for detailed setup instructions. * **Bedrock Access:** Explain how to request access to different LLMs within Amazon Bedrock (Anthropic Claude, AI21 Labs Jurassic-2, etc.). * **Snowflake Setup:** If integrating with Snowflake, provide instructions on creating a Snowflake account, setting up a warehouse, and creating a database. * **3.2. Data Ingestion and Preprocessing** * **Loading Financial Documents into S3:** Explain how to load financial documents (e.g., earnings reports) into an S3 bucket. * **Code Example:** ```python import boto3 s3 = boto3.client('s3') bucket_name = 'your-bucket-name' file_path = 'path/to/your/document.pdf' key = 'financial_documents/amazon_earnings_2024.pdf' s3.upload_file(file_path, bucket_name, key) print(f"File uploaded to s3://{bucket_name}/{key}") ``` * **Document Chunking:** Discuss different chunking strategies (fixed-size, semantic chunking). * **Code Example (Conceptual):** ```python # Simplified chunking example def chunk_document(text, chunk_size=512): chunks = [text[i:i + chunk_size] for i in range(0, len(text), chunk_size)] return chunks ``` * **3.3. Building the VectorRAG Component** * **Generating Embeddings with Cortex Native Embedding Model:** * Provide a detailed code example for calling the Cortex API to generate embeddings for each chunk. ```python import requests import json CORTEX_API_KEY = "YOUR_CORTEX_API_KEY" CORTEX_EMBEDDING_ENDPOINT = "YOUR_CORTEX_EMBEDDING_ENDPOINT" def get_cortex_embedding(text): headers = { "Authorization": f"Bearer {CORTEX_API_KEY}", "Content-Type": "application/json" } data = {"text": text} response = requests.post(CORTEX_EMBEDDING_ENDPOINT, headers=headers, data=json.dumps(data)) response.raise_for_status() return response.json()["embedding"] # Example usage text_chunk = "Amazon's AWS revenue increased by 12% in Q1 2024." embedding = get_cortex_embedding(text_chunk) print(f"Embedding: {embedding[:10]}...") # Print first 10 values ``` * **Storing Embeddings in Cortex Search:** * Provide a code example for indexing the embeddings in Cortex Search, along with relevant metadata (document ID, chunk number, etc.). ```python CORTEX_SEARCH_API_KEY = "YOUR_CORTEX_SEARCH_API_KEY" CORTEX_SEARCH_INDEX_ENDPOINT = "YOUR_CORTEX_SEARCH_INDEX_ENDPOINT" def add_to_cortex_search(document_id, embedding, metadata={}): headers = { "Authorization": f"Bearer {CORTEX_SEARCH_API_KEY}", "Content-Type": "application/json" } data = { "document_id": document_id, "embedding": embedding, "metadata": metadata } response = requests.post(CORTEX_SEARCH_INDEX_ENDPOINT, headers=headers, data=json.dumps(data)) response.raise_for_status() return response.json() # Example document_id = "amazon_earnings_2024_chunk_1" metadata = {"source": "Amazon 2024 Earnings Report", "type": "Financial Report"} result = add_to_cortex_search(document_id, embedding, metadata) if result.get("success"): print(f"Successfully indexed document {document_id} in Cortex Search") else: print(f"Error indexing: {result}") ``` * **3.4. Building the GraphRAG Component** * **Knowledge Graph Construction with Bedrock LLMs:** * Explain how to use prompt engineering with Bedrock LLMs (e.g., Anthropic Claude) to extract entities and relationships from the financial documents. Refer to the two-tiered LLM chain mentioned in the "paste.txt" document. ```python import boto3 import json BEDROCK_REGION = "your-aws-region" BEDROCK_MODEL_ID = "anthropic.claude-v2" # Or another suitable Bedrock model bedrock = boto3.client(service_name='bedrock-runtime', region_name=BEDROCK_REGION) def extract_triplets_with_bedrock(text_chunk): prompt = f""" You are an expert in financial document analysis. Extract the key entities and relationships from the following text chunk. Return the result as a list of (subject, predicate, object) triplets. Text Chunk: {text_chunk} Triplets: """ body = json.dumps({ "prompt": prompt, "max_tokens_to_sample": 500, # Adjust as needed "temperature": 0.0, # For deterministic extraction "top_p": 1 # Use top_p sampling }) modelId = BEDROCK_MODEL_ID accept = 'application/json' contentType = 'application/json' response = bedrock.invoke_model(body=body, modelId=modelId, accept=accept, contentType=contentType) response_body = json.loads(response.get('body').read()) # Parse the LLM's response to extract the triplets. This will depend on the output format of the LLM. triplets = parse_triplets_from_llm_response(response_body['completion']) return triplets # Example text_chunk = "Amazon's revenue increased by 15% to $574.8 billion in 2024, with AWS contributing significantly to the growth." triplets = extract_triplets_with_bedrock(text_chunk) print(f"Extracted Triplets: {triplets}") # Important: Implement the parse_triplets_from_llm_response function # This function needs to take the LLM's text output and convert it into a list of tuples. ``` * Discuss prompt engineering techniques to improve the accuracy of entity and relationship extraction. * **Loading Knowledge Graph into AWS Neptune:** * Provide a code example for loading the extracted triplets into AWS Neptune. ```python import boto3 from gremlin_python import traversal from gremlin_python.driver.driver_remote_connection import DriverRemoteConnection NEPTUNE_ENDPOINT = "YOUR_NEPTUNE_ENDPOINT" # e.g., "wss://..." NEPTUNE_PORT = 8182 def load_triplets_to_neptune(triplets): try: conn = DriverRemoteConnection(f'wss://{NEPTUNE_ENDPOINT}:{NEPTUNE_PORT}/gremlin', 'g') g = traversal().withRemote(conn) for subject, predicate, object in triplets: # Add vertices and edges g.addV(subject).property('name', subject).next() g.addV(object).property('name', object).next() g.V(subject).addE(predicate).to(g.V(object)).next() conn.close() print("Triplets loaded successfully into Neptune.") except Exception as e: print(f"Error loading triplets into Neptune: {e}") # Example usage: sample_triplets = [ ("Amazon", "revenue", "$574.8 billion"), ("Amazon", "growth", "15%"), ("AWS", "contributes_to", "Amazon revenue") ] load_triplets_to_neptune(sample_triplets) ``` * **3.5. Hybrid Retrieval and Fusion** * **Querying Cortex Search:** Show how to query Cortex Search with a user's question to retrieve relevant text chunks. ```python # (Code snippet from previous Cortex Search section, adapted for querying) ``` * **Querying AWS Neptune:** Demonstrate how to query AWS Neptune to retrieve relevant entities and relationships based on the user's question. Provide example Gremlin queries. ```python # Example Gremlin query to find all companies related to "AWS" gremlin_query = "g.V('AWS').inE('related_to').outV().valueMap()" # Execute the Gremlin query using boto3 and the Neptune API (similar to the load_triplets example) ``` * **Fusion Strategies:** Explain and demonstrate how to combine the results from Cortex Search and AWS Neptune. Options include: * **Ranking/Scoring:** Combine the relevance scores from both retrieval methods. * **Filtering:** Use the GraphRAG results to filter the VectorRAG results (e.g., only keep results that are connected to a specific entity in the KG). * **Enrichment:** Use the GraphRAG results to add more information to the VectorRAG results (e.g., append related entities and relationships to the text chunks). * **3.6. Response Generation with Bedrock LLMs** * Explain how to format the combined context from Cortex Search and AWS Neptune and pass it to a Bedrock LLM for response generation. ```python # (Code snippet from previous Bedrock LLM interaction section, adapted for HybridRAG context) ``` * Discuss prompt engineering strategies to ensure the LLM effectively uses the combined context. * Example Prompts: * "Based on this text from Cortex Search: \[text\_chunks\] and this knowledge graph information from Neptune: \[entities\_and\_relationships\], answer the question: \[user\_question\]" * "You are a financial expert. Use the following information to answer the question: \[user\_question\]. Relevant text: \[text\_chunks\]. Related entities and relationships: \[entities\_and\_relationships\]" **4\. Deployment and Scaling** * **Deploying the HybridRAG System as an AWS Lambda Function:** Explain how to package the code and deploy it as a Lambda function. * **API Gateway Integration:** Discuss the use of API Gateway to create an API endpoint for the Lambda function. * **Scaling Considerations:** * Scaling Cortex Search: Discuss the scalability options for Cortex Search (e.g., increasing the number of replicas). * Scaling AWS Neptune: Discuss the scalability options for AWS Neptune (e.g., increasing instance size, adding read replicas). * Scaling Bedrock: Highlight Bedrock's ability to handle concurrent requests.