AWS BedRock - Boto3 Demo - LLAMA2 Model

AWS BedRock - Boto3 Demo - LLAMA2 Model

Meta has released an accessible version of Llama, a family of large language models utilizing transformer architecture. Llama, trained on publicly available data, demonstrates proficiency in various natural language processing tasks like translation and summarization. Key features include coherent and contextually relevant text generation, achieved through attention mechanisms and masked language modeling during pre-training. Llama 2, an optimized dialogue variant, is tailored for commercial and research use in English, specifically in chat-based applications. It showcases advanced capabilities in text generation and chat optimization, providing a versatile tool for technical applications like chatbots and virtual assistants.

Previous Blog on this Learning Series

Blog 1:

Blog 2:

Blog 3:

Blog 4:

Blog 5:

Blog 6:

Github Link - Notebook

Environment Setup

I am using vscode local environment with AWS Credential configured.

Install Latest Python

! python --version
Python 3.11.5

Upgrade pip

! pip install --upgrade pip

Install latest boto3,awscli, boto3-core

! pip install --no-build-isolation --force-reinstall \
    "boto3>=1.33.6" \
    "awscli>=1.31.6" \

Load the Library

import json
import os
import sys

import boto3
import botocore

bedrock = boto3.client(service_name="bedrock")
bedrock_runtime = boto3.client(service_name="bedrock-runtime")

META - Llama2 Chat Model

Set the Prompt

llama2_70b_code_prompt = "[INST]You are an economist with access to lots of data[/INST]\nWrite an article about why gold price is going up in india"

Configure the Model configuration

body = json.dumps({
    "max_gen_len": 256,
    "top_p": 0.9, # Top P is a text generation technique, sampling from the most probable tokens in a distribution.
    "temperature": 0.5 #Temperature controls randomness; higher values increase diversity, lower values boost predictability.

Invoke the Model

response = bedrock_runtime.invoke_model(
    # You can replace above model with meta.llama2-13b-chat-v1 also

Parse the response for Text Completion

# print (response)
response_body = json.loads(response.get('body').read())
# response_body
parse_text = response_body['generation']
text = parse_text[parse_text.index('\n')+1:]
llama2_output = text.strip()

Text Completion

India has been seeing a steady increase in gold prices over the past few years, and this trend is expected to continue in the near future. There are several factors that contribute to this upward trend, and as an economist with access to lots of data, I will explore some of the key reasons behind the rising gold prices in India.\n\nFirst and foremost, the increasing demand for gold in India is a major driver of its rising price. Gold has always been a prized possession in Indian culture, and it is considered a safe haven investment, especially during times of economic uncertainty. With the global economy facing headwinds, investors are increasingly turning to gold as a hedge against inflation and market volatility. This increased demand for gold has led to a rise in its price.\n\nAnother significant factor contributing to the rising gold prices in India is the country's growing wealth and prosperity. As the Indian economy grows, so does the purchasing power of its citizens. This has led to an increase in the demand for luxury goods, including gold. In fact, India is now the world's second-largest consumer of gold, after China. The rising demand for gold