Building an AI-Powered WhatsApp Notification Router with Python and Llama 3.3: A Development Journey
Introduction
Managing high volumes of incoming messages across multiple communication channels often introduces significant operational friction. When operating in fast-paced environments, filtering critical updates from casual chatter manually becomes a bottleneck. To solve this problem, I built an automated notification routing system using Python and Large Language Models (LLMs) to categorize, prioritize, and route communications contextually.
Motivation and Technical Stack
As communication channels expand, message fatigue becomes a common engineering and productivity hurdle. The objective of this project was to construct a deterministic routing pipeline capable of analyzing incoming communication payloads and mapping them to appropriate destination parameters based on semantic context.
Why Python Was Chosen
Python provides an extensive ecosystem for data manipulation and API integration. Its rich library support—ranging from data analysis tools like pandas to robust HTTP clients and official machine learning SDKs—makes it an efficient language for orchestrating backend workflows and interacting with external LLM endpoints.
Why Groq Llama 3.3 Was Used
The project was initially prototyped using Gemini models. However, due to service deprecation policies and shifting API endpoints, the backend was successfully migrated to Groq, utilizing the high-performance Llama 3.3 architecture. Groq's specialized inference infrastructure provides reliable execution speeds and accurate natural language comprehension suitable for JSON-structured output generation.
Architecture Overview
The system is structured as a multi-stage pipeline where data flows from structured datasets through preprocessing, LLM evaluation, and final validation. Below is the conceptual workflow of the notification routing engine:
[ Data Sources (CSV / Metadata) ]
│
▼
[ Retrieval Engine ]
│
▼
[ Prompt Builder ]
│
▼
[ Router Agent ] ◄──► [ Groq LLM Client ]
│
(Valid JSON)
│
▼
[ Output Generator ]
The core components of the architecture interact sequentially to process inputs and produce verified routing decisions.
Challenges Faced During Development
Building and stabilizing the pipeline involved addressing several practical engineering challenges:
- Gemini Model Deprecation: Adapting to sudden endpoint shifts by refactoring the integration layer for standard SDK compatibility.
- Groq Rate Limits: Managing request throttling through exponential backoff strategies and batching requests.
- Prompt Engineering: Refining instructions to ensure models adhere strictly to requested schemas.
- JSON Parsing & Output Validation: Handling malformed LLM outputs, markdown code block wrappers, and schema mismatches through defensive parsing code.
Python Code Example
Here is a realistic implementation of the Router Agent using the official Groq SDK and standard JSON handling:
import os
import json
from groq import Groq
def router_agent(notification_text, context_data):
client = Groq(api_key=os.environ.get("GROQ_API_KEY"))
prompt = f"""
Analyze the following notification against the provided context data and return a valid JSON object with keys "destination" and "confidence".
Notification: {notification_text}
Context: {json.dumps(context_data)}
"""
completion = client.chat.completions.create(
model="llama-3.3-70b-versatile",
messages=[
{"role": "system", "content": "You are a strict JSON routing assistant. Output only valid JSON."},
{"role": "user", "content": prompt}
],
temperature=0.1,
response_format={"type": "json_object"}
)
raw_response = completion.choices[0].message.content
try:
parsed_decision = json.loads(raw_response)
return parsed_decision
except json.JSONDecodeError:
# Fallback handling for malformed responses
return {"destination": "unrouted", "confidence": 0.0}
Gemini vs. Groq Comparison
| Feature | Gemini (Initial Prototype) | Groq Llama 3.3 (Migration Target) |
|---|---|---|
| API Status | Subject to deprecation/version changes | Stable, actively supported endpoint |
| Inference Infrastructure | Standard cloud execution | Specialized LPU hardware accelerators |
| Response Latency | Variable | High throughput and low latency |
Project Architecture Components
| Component | Description |
|---|---|
| Retrieval Engine | Retrieves contextual information from datasets such as messages.csv, users.csv, groups.csv, business_accounts.csv, message_history.csv, notification summaries, image metadata, and voice metadata. |
| Prompt Builder | Constructs structured system and user prompts combining notification payloads with contextual record attributes. |
| Router Agent | Sends the constructed prompt to the LLM and validates the JSON response before writing the routing decision. |
| LLM Client | Interfaces directly with the official Groq SDK for chat completions. |
| Output Generator | Compiles routing logs and writes verified outputs to persistent storage formats like CSV. |
Performance Observations and Lessons Learned
Testing the pipeline with historical dataset extracts demonstrated that enforcing strict JSON response formats significantly improves downstream automation reliability. Key takeaways from the build process include the necessity of treating LLM outputs as untrusted inputs, implementing robust fallback mechanisms for rate limits, and maintaining modular client separation to ease future model migrations.
Future Improvements
To scale the application for enterprise requirements, future iterations will explore:
- Multi-Agent Routing: Delegating specialized categorization tasks to cooperating agent nodes.
- Vector Search: Integrating embedding databases for semantic context lookup over massive message histories.
- Prompt Injection Protection: Sanitizing incoming text payloads to guard against malicious instruction overrides.
- Confidence Calibration: Implementing threshold-based routing rules where low-confidence decisions require human verification.
- Human Feedback Loops: Logging corrections to fine-tune future classification rules.
Frequently Asked Questions
The primary goal is to automate the categorization and routing of incoming messages based on structured contextual data and semantic analysis.
No. The Retrieval Engine retrieves contextual information from static or synchronized datasets such as messages.csv, users.csv, groups.csv, business_accounts.csv, message_history.csv, notification summaries, image metadata, and voice metadata.
The Router Agent sends the constructed prompt to the LLM and validates the JSON response before writing the routing decision.
The project was initially built using Gemini but later migrated to Groq because of model availability and API changes.
The application uses the official Python Groq SDK rather than custom or deprecated client wrappers.
Rate limits are managed via exception handling, retry decorators, and request pacing to prevent threshold breaches.
The output generator currently compiles decisions into structured CSV files for downstream auditing and review.
Defensive try-except blocks catch decoding errors and reroute or log fallback fallback values to prevent pipeline crashes.
Conclusion
Transitioning this notification routing pipeline from a prototype into a functional architecture highlighted the importance of API resilience, strict schema validation, and defensive programming. By leveraging modern inference engines like Groq alongside structured Python workflows, developers can build dependable automation pipelines capable of handling real-world data constraints.