The chatbot on my portfolio was fast, but it was stateless from a user's point of view. If someone refreshed or switched devices, the conversation was gone. I wanted to solve two things at the same time: durable chat history and production-grade visibility into what was happening in the backend. Here's how I did it.
What I Wanted to Fix
I focused on three practical goals:
- Persist conversations for signed-in users across sessions.
- Restore the latest session across devices without manual export.
- Track the full request lifecycle with structured cloud logs and error reporting.
Chat Persistence with Firestore
I moved session persistence to Firestore using a user-linked chat session model. Each session stores messages and update timestamps, and the API restores the most recent session for authenticated users. This gave me predictable cross-device continuity without forcing users to manually export anything.
Session Schema Design
Each chat session is stored as a Firestore document keyed by a UUID. The client generates a sessionId per browser and persists it in localStorage. Here's the document structure:
// Firestore: chatSessions/{sessionId}
{
sessionId: "uuid-v4", // Document ID
userId: "[email protected]", // null for anonymous users
createdAt: Timestamp, // Set on first write only
updatedAt: Timestamp, // Updated after every reply
messages: [
{ role: "user", content: "What do you do?", timestamp: "ISO-8601" },
{ role: "assistant", content: "I'm a Full Stack...", timestamp: "ISO-8601" }
]
}Sessions are never deleted. When a user clicks "Clear Chat", a new UUID is generated client-side — the old session stays preserved as a historical record.
The Save Pattern: Timeout-Guarded Upsert
Saving to Firestore happens server-side inside the streaming response handler. I use Promise.race with a timeout to prevent the stream from hanging if Firestore is slow:
// Save session to Firestore (timeout-guarded)
const FIRESTORE_SAVE_TIMEOUT_MS = 7000;
if (sessionId && fullResponse) {
const sessionMessages = [
...previousMessages,
{ role: 'user', content: message, timestamp: new Date().toISOString() },
{ role: 'assistant', content: fullResponse, timestamp: new Date().toISOString() },
];
const sessionRef = firestore.collection('chatSessions').doc(sessionId);
try {
await Promise.race([
(async () => {
// Upsert with merge — creates or updates
await sessionRef.set(
{
sessionId,
userId: userId ?? null,
messages: sessionMessages,
updatedAt: FieldValue.serverTimestamp(),
},
{ merge: true }
);
// Set createdAt only on first write
const doc = await sessionRef.get();
if (!doc.data()?.createdAt) {
await sessionRef.update({ createdAt: FieldValue.serverTimestamp() });
}
})(),
new Promise((_, reject) =>
setTimeout(
() => reject(new Error('Firestore save timeout')),
FIRESTORE_SAVE_TIMEOUT_MS
)
),
]);
} catch (err) {
// Report to Cloud Error Reporting (non-blocking)
reportCloudError({
service: 'chat-api',
message: `Firestore save failed: ${err.message}`,
context: { sessionId, userId },
});
}
}Key design choices here: merge: true gives upsert semantics so I don't need to check if the doc exists, and FieldValue.serverTimestamp() ensures timestamps come from Firestore's server clock, not the client.
Firestore Client: REST Mode for Serverless
One important detail — I initialize the Firestore client with preferRest: true. In serverless environments like Vercel, long-lived gRPC channels can cause DNS stalls and connection issues. REST mode avoids this:
import { Firestore } from '@google-cloud/firestore';
export const firestore = new Firestore({
projectId: process.env.GCP_PROJECT_ID,
// Serverless-friendly: avoid long-lived gRPC channel issues
preferRest: true,
credentials: {
client_email: process.env.GCP_CLIENT_EMAIL,
private_key: process.env.GCP_PRIVATE_KEY.replace(/\\n/g, '\n'),
},
});Client Restore Priority
I adjusted the restore order for logged-in users: try server history first, then fall back to local cache only if server data is empty or unavailable:
// Client-side restore priority 1. User signs in → fetch latest session from Firestore by userId 2. If server returns messages → use them (source of truth) 3. If server is empty or fails → fall back to localStorage cache 4. Anonymous users → always use localStorage only
That small ordering change fixed several confusing edge cases where stale local data would override newer server conversations.
Observability with Cloud Logging and Error Reporting
The second piece was observability. I wanted operational answers quickly: who asked, what they asked, what happened during streaming, and where failures occurred.
Structured Event Logging
I built a lightweight wrapper around the Cloud Logging REST API. Every log entry carries structured JSON payloads — not just text strings. This makes logs filterable and queryable in GCP Logs Explorer:
// lib/gcp-observability.ts
type Severity = 'DEFAULT' | 'INFO' | 'WARNING' | 'ERROR' | 'CRITICAL';
interface CloudLogPayload {
logId: string;
severity?: Severity;
message: string;
jsonPayload?: Record<string, unknown>;
}
export async function writeCloudLog({
logId,
severity = 'INFO',
message,
jsonPayload,
}: CloudLogPayload): Promise<void> {
const projectId = process.env.GCP_PROJECT_ID;
if (!projectId) return;
const accessToken = await getGcpAccessToken();
if (!accessToken) return;
const entry = jsonPayload
? { severity, jsonPayload: { message, ...jsonPayload } }
: { severity, textPayload: message };
await fetch('https://logging.googleapis.com/v2/entries:write', {
method: 'POST',
headers: {
Authorization: `Bearer ${accessToken}`,
'Content-Type': 'application/json',
},
body: JSON.stringify({
logName: `projects/${projectId}/logs/${encodeURIComponent(logId)}`,
resource: { type: 'global', labels: { project_id: projectId } },
entries: [entry],
}),
});
}Dual-Write Error Reporting
For errors, I don't just log — I report to both Cloud Error Reporting API and Cloud Logging in a single call. Error Reporting gives me auto-grouping and alerting, and the correlated log entry gives me context:
export async function reportCloudError({
message,
service,
version = '1.0.0',
context,
}: ErrorReportPayload): Promise<void> {
const projectId = process.env.GCP_PROJECT_ID;
if (!projectId) return;
// 1. Report to Error Reporting API (auto-groups similar errors)
await gcpPost(
`https://errorreporting.googleapis.com/v1beta1/projects/${projectId}/events:report`,
{ message, serviceContext: { service, version } }
);
// 2. Also emit a structured Cloud Log entry
await writeCloudLog({
logId: `${service}-errors`,
severity: 'ERROR',
message,
jsonPayload: context,
});
}What Gets Logged
I log at clear lifecycle stages so I can trace any request end-to-end:
- Request accepted — user message, session ID, IP, previous message count.
- Stream completed — chunk count, total streaming time, first chunk latency, response preview.
- Session save success — session ID, total messages saved.
- Session save failure — error message, session context (via Error Reporting).
// Example: logging stream completion with timing metadata
await logChatEvent('[Chat API] Stream completed', {
route: '/api/chat',
userId: userId ?? 'anonymous',
sessionId,
chunkCount,
totalStreamTimeMs: Number(totalStreamTime),
firstChunkLatencyMs: firstChunkTime
? Number((firstChunkTime - geminiStreamStart).toFixed(2))
: null,
assistantResponsePreview: fullResponse.slice(0, 600),
});This made troubleshooting significantly faster, especially for serverless edge cases where logs can disappear if not handled carefully.
Identity Without Static Keys
For cloud authentication, I used Workload Identity Federation (WIF) with OIDC instead of long-lived JSON keys. The key outcome: no static secret file in the deployment path for log/error write access.
How WIF Works in This Setup
Instead of downloading a GCP service account JSON key and shipping it with the deployment, I use Vercel's native OIDC provider:
- Vercel issues an OIDC token — automatically available via
@vercel/oidc. - Token exchanged via Google STS — validated against a Workload Identity Pool.
- Service Account impersonation — STS returns a short-lived access token.
- API calls made with ephemeral token — no static keys ever touch the deployment.
The Auth Implementation
Here's the actual auth flow using google-auth-library and Vercel's OIDC:
// lib/gcp-wif-auth.ts
import { getVercelOidcToken } from '@vercel/oidc';
import { ExternalAccountClient } from 'google-auth-library';
export async function getGcpAccessToken(): Promise<string | null> {
const authClient = ExternalAccountClient.fromJSON({
type: 'external_account',
audience: `//iam.googleapis.com/projects/${projectNumber}/locations/global/workloadIdentityPools/${poolId}/providers/${providerId}`,
subject_token_type: 'urn:ietf:params:oauth:token-type:jwt',
token_url: 'https://sts.googleapis.com/v1/token',
service_account_impersonation_url:
`https://iamcredentials.googleapis.com/v1/projects/-/serviceAccounts/${serviceAccountEmail}:generateAccessToken`,
subject_token_supplier: {
getSubjectToken: async () => getVercelOidcToken(),
},
});
if (!authClient) return null;
const result = await authClient.getAccessToken();
return typeof result === 'string' ? result : result?.token ?? null;
}All sensitive values (projectNumber, poolId, providerId,serviceAccountEmail) live in environment variables and are never committed to source control.
Architecture at a Glance
Here's how the four pieces fit together:
Firestore Persistence
Session Storage
Sessions saved with merge: true upsert, timeout-guarded writes, and FieldValue.serverTimestamp() for consistent ordering.
Cloud Logging
Structured Events
JSON payloads via Cloud Logging REST API. Every lifecycle stage logged with timing metadata and response previews.
Error Reporting
Dual-Write
Writes to both Error Reporting API (auto-grouping + alerts) and Cloud Logging (correlated log entries) for every failure.
WIF Authentication
Zero Static Keys
Vercel OIDC → Google STS → short-lived service account tokens. Credentials never touch the deployment.
Production Lessons
Indexes Matter
Firestore requires composite indexes for queries combining filters and ordering. If you're querying sessions by userId and ordering by updatedAt, that index must exist. Missing indexes silently return empty results — not an error, just nothing.
Serverless Timing Matters
In serverless functions, the runtime can terminate once the response stream closes. If your Firestore write is purely fire-and-forget (no await), it may never complete. I wrap critical writes in await Promise.race() with a timeout so the write gets a fair chance but doesn't hold the response hostage.
Transport Choices Matter
The Firestore client defaults to gRPC. In serverless environments, gRPC channels can hit DNS stalls or load-balancer pick issues on cold starts. Setting preferRest: true switches to HTTP/1.1, which is stateless and much more reliable in short-lived function contexts.
Local Cache Is a Fallback
For authenticated users, the server session is the source of truth. Local cache is only used when the server is unreachable or returns no data. Getting this priority wrong leads to stale conversations overwriting fresh ones.
Final Outcome
The chat experience now behaves like a product, not a demo. Users get stable session continuity, and I get reliable observability for both happy-path traffic and failures. That combination made the system much easier to trust and iterate on.