Skip to content
10 min read

Teaching Gemini to Paint: How I Built NLP-Powered Theme Switching

A deep dive into how I engineered a chat interface that lets users switch website themes using natural language commands like 'Make it dark' or 'I want a hacking vibe'.

AI EngineeringGeminiNext.jsUXNatural Language Processing

Originally published in Patil Protocol, the newsletter by Aniket Patil.

"Make it dark."

It sounds simple, right? But making a website respond to that command—not by clicking a button, but by understanding the intent and executing code—is where the magic of modern AI Engineering lies.

I wanted an Agent that could control the environment it inhabited. If a user says "This is too bright," the Agent should dim the lights, not just apologize.

Watch the Agent toggle themes in real-time.

The Vision: Beyond Buttons

Traditional theme switchers are boring. You click a button. The colors change. That's it.

I wanted something more expressive. What if someone could say "give me that matrix look" and the AI would understand that means green, dark, and techy? What about "I want something warm" mapping to an orange palette?

This isn't just about themes—it's about teaching AI to be a silent operator that controls the UI without being asked for permission at every step.

The Foundation: CSS Tokens

Before AI could paint, I needed a canvas that accepted paint. My early experimentation revealed a critical flaw: hardcoded colors.

I spent days migrating the entire site from utility classes like text-gray-900 to semantic tokens like text-foreground. This was the unglamorous but essential plumbing work.

:root {
  /* Light theme colors */
  --bg-light: #FAFAFA;
  --text-primary-light: #1E293B;

  /* Dark theme colors */
  --bg-dark: #1A1A2E;
  --text-primary-dark: #F8FAFC;

  /* Brand colors - Digital Blue */
  --primary: #3B82F6;
  --accent: #06B6D4;
}

/* Additional palettes override these */
[data-palette="nature-green"] {
  --primary: #65A30D;
  --accent: #4ADE80;
  --bg-dark: #1F2E28;
}

With four distinct palettes (Digital Blue, Warm Tech, Nature Green, Solar Yellow) each with light and dark variants, the AI now had 8 possible combinations to work with.

The Tool Call Breakthrough

This was the moment everything clicked. I defined a change_theme tool in Gemini's system prompt that could independently set mode, palette, or both:

// lib/gemini.ts - Actual implementation
{
  name: 'change_theme',
  description: 'Universal theme control. Sets mode (light/dark) and/or color palette.',
  parametersJsonSchema: {
    type: 'object',
    properties: {
      mode: {
        type: 'string',
        enum: ['light', 'dark'],
      },
      palette: {
        type: 'string',
        description: 'Palettes: "digital-blue", "warm-tech", "nature-green", "solar-yellow"',
        enum: ['digital-blue', 'warm-tech', 'nature-green', 'solar-yellow'],
      },
    },
  },
}

The magic is in the separation. When a user says "make it dark but keep the colors," the AI calls:

change_theme({ mode: 'dark' })

But "give me that matrix vibe" triggers:

change_theme({ mode: 'dark', palette: 'nature-green' })

The Action Handler: Client-Side Execution

The frontend receives the tool call from the stream and executes it immediately. Here's the actual handler from my codebase:

// hooks/use-chat-core.ts
const handleAction = useCallback((action: any) => {
  switch (action.name) {
    case 'change_theme':
      // Can set mode, palette, or both
      if (action.args.mode) {
        setTheme(action.args.mode);  // next-themes
      }
      if (action.args.palette) {
        setPalette(action.args.palette);  // PaletteContext
      }
      break;

    case 'toggle_mode':
      // Quick toggle without changing palette
      setTheme(resolvedTheme === 'dark' ? 'light' : 'dark');
      break;

    case 'cycle_palette':
      // User says "next theme" — cycles through palettes
      cyclePalette();
      break;
  }
}, [setTheme, setPalette, cyclePalette, resolvedTheme]);

The key insight: I separated mode (light/dark via next-themes) from palette (color scheme via PaletteContext). This orthogonality lets users say "I want warm colors but keep it bright" and get exactly what they asked for.

The PaletteContext Architecture

The PaletteContext is a React context that manages color palettes independently of light/dark mode. It persists the user's choice to localStorage and applies the palette by setting a data-palette attribute on the document:

// context/PaletteContext.tsx
export const PALETTES = ['digital-blue', 'warm-tech', 'nature-green', 'solar-yellow'] as const;
export type Palette = (typeof PALETTES)[number];

const STORAGE_KEY = 'portfolio-palette';

export function PaletteProvider({ children }: { children: ReactNode }) {
  const [palette, setPaletteState] = useState<Palette>('warm-tech');

  // Load from localStorage on mount
  useEffect(() => {
    const stored = localStorage.getItem(STORAGE_KEY);
    if (stored && PALETTES.includes(stored as Palette)) {
      setPaletteState(stored as Palette);
    }
  }, []);

  // Apply palette to DOM and persist
  useEffect(() => {
    document.documentElement.setAttribute('data-palette', palette);
    localStorage.setItem(STORAGE_KEY, palette);
  }, [palette]);

  const cyclePalette = useCallback(() => {
    const currentIndex = PALETTES.indexOf(palette);
    const nextIndex = (currentIndex + 1) % PALETTES.length;
    setPaletteState(PALETTES[nextIndex]);
  }, [palette]);

  return (
    <PaletteContext.Provider value={{ palette, setPalette, cyclePalette }}>
      {children}
    </PaletteContext.Provider>
  );
}

The Streaming Architecture

Gemini's generateContentStream yields chunks that can contain either text or function calls. My streamGeminiContent generator separates these:

// lib/gemini.ts - Streaming with tool call extraction
export async function* streamGeminiContent(systemPrompt: string, userMessage: string) {
  const response = await genai.models.generateContentStream({
    model: 'gemini-2.5-flash',
    config: {
      systemInstruction: { parts: [{ text: systemPrompt }] },
      tools: tools,  // Theme tools, navigation, etc.
      temperature: 0.7,
    },
    contents: [{ role: 'user', parts: [{ text: userMessage }] }],
  });

  for await (const chunk of response) {
    // Extract function calls
    const functionCalls = chunk.functionCalls;
    if (functionCalls && functionCalls.length > 0) {
      for (const call of functionCalls) {
        yield { action: call };  // Frontend catches this
      }
    }

    // Extract text content
    if (chunk.text) {
      yield { text: chunk.text };
    }
  }
}

The frontend then buffers these actions until the stream completes, ensuring the conversational confirmation ("Okay, switching to dark mode!") appears before the theme actually changes.

The UX Nightmare: Premature Actions

A fascinating bug emerged during testing. The AI was too fast.

If a user asked "Show me your projects," the AI would trigger the navigation tool immediately. The page would redirect before the AI could finish saying "Here are my projects..."

The Problem: Actions were executing during the stream, not after.

The Fix: Response Buffering. I rewrote the client-side hook to buffer the tool execution. We now wait for the text stream to fully complete before firing the action:

// hooks/use-chat-core.ts - Buffered action execution
let pendingAction: any = null;

// During streaming
if (parsed.action) {
  pendingAction = parsed.action;  // Store, don't execute
}

// After stream completes
if (pendingAction) {
  // Small delay so user sees the completion state
  setTimeout(() => handleAction(pendingAction), 800);
}

The Conversational Confirmation Pattern

Initially, I tried a "silent operator" approach where the AI would just execute actions without saying anything. But it felt robotic and unintuitive—users didn't know if the command worked until they saw the change.

The solution was Conversational Confirmation. When the AI executes a tool, it also gives a friendly acknowledgment: "Okay, switching to dark mode now!" or "Sure, let me give you that nature green look!"

// System prompt instruction
5. **THEME**: Use change_theme, toggle_mode, or cycle_palette.
   **ALWAYS give a friendly confirmation.** Examples:
   - "Okay, switching to dark mode now!" (then call toggle_mode)
   - "Sure, let me give you that nature green look!" (then call change_theme)
   - "Absolutely! Changing to warm colors." (then call change_theme)
   Do NOT be silent. Be conversational and confirm what you're doing.

This creates a natural, human-like interaction. The AI feels like a helpful assistant rather than a cold command executor.

Critical Lessons

State Sync is Hard

The ChatContext shares state between the floating chat and the full /chat page. Without it, theme changes in one wouldn't reflect in the other.

Latency Matters

I chose Gemini 2.5 Flash over Pro specifically for sub-500ms tool execution. For "snap" feedback, speed beats reasoning depth.

Orthogonal Controls

Separating mode and palette into independent parameters gives exponential flexibility with linear complexity.

Lazy Load Heavy Components

The FloatingChat component was too heavy for initial bundle. Using next/dynamic to lazy load it improved LCP significantly.

Try It Yourself

This journey from hardcoded CSS to an NLP-controlled environment has been one of the most rewarding parts of building this portfolio. It turns the website from a static document into a dynamic application that collaborates with the user.

Right now, try saying:

  • "Make it dark"
  • "I want warm colors"
  • "Give me that matrix vibe"
  • "Switch to light mode with blue theme"

Use the chat button in the bottom-right corner, or go to /chat.

The future isn't just about clicking buttons; it's about asking for what you want and having the interface reshape itself around you.