It started with a frustrating error: ThrottlingException. I was building my portfolio's RAG chatbot using AWS Bedrock, but I kept hitting rate limits. That roadblock became the catalyst for a major upgrade. Today, I want to share how I migrated to Google Gemini 1.5 Flash and took my chatbot from a simple Q&A bot to a fully agentic system that can control the UI.
Watch the agent navigate the site and toggle themes in real-time.
The Problem: AWS Constraints
My initial implementation used AWS Bedrock (Titan Embeddings + Mistral). While powerful, it had limitations for a personal portfolio:
- Cost & Quotas: Hitting throttling limits on critical demo days.
- Latency: Cold starts and embedding generation times were noticeable.
The Solution: Gemini 1.5 Flash
I switched to Google's Gemini 1.5 Flash. It's incredibly fast, has a massive context window (1M tokens), and most importantly, it supports Tool Calling out of the box.
Scaling to Gemini 2.5 & 3072D Embeddings
As of late 2025, I've further upgraded the system to Gemini 2.5 Flash, ensuring even higher stability and performance. I also migrated the vector database to use gemini-embedding-001 with full 3072-dimensional embeddings (up from 1024D). This massively improved retrieval meaningfulness.
Smarter UI/UX:
- Greeting Detection: The bot now detects "hi" or "hello" and skips the RAG search entirely to save resources and keep the UI clean.
- Compact Sources: References are now displayed as sleek, tag-like chips that expand on demand, keeping the focus on the conversation.
- Higher Precision: With 3072D vectors, I increased the similarity threshold to 0.6 for generic queries, virtually eliminating irrelevant hallucinations.
- Build & Performance: I removed legacy AWS Bedrock dependencies to fix build timeouts and reduce bundle size.
- Robust Streaming: Solved "Streaming failed" errors by optimizing the Gemini 2.5 integration for better fault tolerance.
Building "Agentic" Features
I didn't just want a chatbot that talks; I wanted one that does things. I implemented three core tools:
1. Navigation Tool
Instead of explaining where my projects are, the AI just takes you there.
{
name: 'navigate_to_page',
description: 'Navigates the user to a specific page...',
parametersJsonSchema: {
type: 'object',
properties: {
path: { type: 'string' }
}
}
}2. Theme Control
Users can say "Make it dark" or "Toggle theme", and the AI executes a toggle_theme tool. This connects the NLP layer directly to my Next.js next-themes provider.
3. Smart Blog Search
I removed hardcoded URLs from the system prompt. Now, I use a search_blogs tool backed by a client-side fuzzy search registry.
Next-Level: Context Awareness
The coolest feature? "Chat with this Article".
I implemented a global ChatContext that tracks which blog post you're reading. When you click "Ask AI", it injects the article title into the system prompt, prioritizing RAG chunks from that specific page. It feels less like a generic bot and more like a helpful reading assistant.
Deep Dive: Under the Hood
For the developers reading this, here is how the magic happens. The core of the "Agentic" behavior lies in how we handle the streaming response from Gemini.
1. The Streaming Generator
Unlike a standard chatbot that just yields text, my streamGeminiContent generator in lib/gemini.ts separates Tool Calls (actions) from Text (responses). This allows the frontend to intercept an action like toggle_theme and execute it immediately.
// lib/gemini.ts
export async function* streamGeminiContent(systemPrompt: string, userMessage: string) {
const response = await genai.models.generateContentStream({
model: 'gemini-2.5-flash',
contents: [
{ role: 'user', parts: [{ text: `${systemPrompt}\n\nUser Question: ${userMessage}` }] },
],
config: { tools: tools } // Tools defined earlier
});
for await (const chunk of response) {
// 1. Check for Function Calls (Actions)
const functionCalls = (chunk as any).functionCalls;
if (functionCalls) {
for (const call of functionCalls) {
yield { action: call }; // Yield action for frontend to execute
}
}
// 2. Check for Text Content (Dialogue)
const text = (chunk as any).text;
if (text) {
yield { text };
}
}
}2. RAG Logic: Cosine Similarity
To retrieve the right context, I calculate the Cosine Similarity between the user's query embedding and my blog chunks. This happens in app/api/chat/route.ts.
// app/api/chat/route.ts
function cosineSimilarity(vecA: number[], vecB: number[]): number {
let dotProduct = 0;
let normA = 0;
let normB = 0;
for (let i = 0; i < vecA.length; i++) {
dotProduct += vecA[i] * vecB[i];
normA += vecA[i] * vecA[i];
normB += vecB[i] * vecB[i];
}
return dotProduct / (Math.sqrt(normA) * Math.sqrt(normB));
}
// ... inside the route handler ...
const similarities = embeddings.map(chunk => ({
...chunk,
similarity: cosineSimilarity(userEmbedding, chunk.embedding)
}));
// Filter for high relevance only
const topChunks = similarities
.filter(chunk => chunk.similarity > 0.05)
.sort((a, b) => b.similarity - a.similarity)
.slice(0, 6);Performance Wins
Switching from AWS Bedrock to Gemini wasn't just about features; the performance and developer experience improvements were tangible.
Zero Cold Starts
Serverless functions using the Gemini API respond instantly. AWS Bedrock occasionally had connection setup latency on cold lambdas.
Massive Context
Gemini's 1M token window means I don't have to be aggressive with truncating context. I can feed entire blog posts into the prompt safely.
Better Developer Exp
The @google/genai SDK is modern, typed, and intuitive. The AWS SDK v3 is powerful but extremely verbose for simple inference tasks.
Cost Efficiency
Gemini 1.5 Flash is incredibly cost-effective for the high volume of "casual" chat interactions a portfolio site receives.
What's Next?
I'm planning to expand the agent's capabilities even further—maybe letting it scroll the page for you or summarize specific sections.
If you're building RAG systems, I highly recommend looking into "Agentic" patterns. Giving your LLM tools to control the UI changes the entire user experience.