Chapter 13

Books → Learning Ai From Scratch → Introduction → Chapter 13

Chapter 13

LLM Application Patterns

Chapter 13: LLM Application Patterns

Large Language Models are not useful only because they can chat. Their real power appears when they are placed inside application patterns: repeatable designs for solving business problems using prompts, retrieval, tools, workflow rules, APIs, databases, and user interfaces. This chapter explains the most common LLM application patterns in a practical way so that you can identify which pattern fits a real project.

A pattern is a reusable solution style. For example, a chatbot pattern is useful when a user needs conversation. A classification pattern is useful when an incoming message must be assigned to a category. A report generation pattern is useful when structured data must be converted into readable business language. Once you learn these patterns, you can design many AI applications confidently.

Learning Objectives

Understand the major types of LLM applications used in real projects.
Know when to use chatbot, document Q&A, summarization, extraction, classification, and agent-style workflow patterns.
Learn the basic architecture, input, output, example prompt, example response, risks, and best practices for every major pattern.
Understand how these patterns can be combined to build enterprise AI applications.
Prepare for later chapters on agents, evaluation, monitoring, guardrails, cost, and production readiness.

Chapter Outcome

After completing this chapter, you should be able to look at a business requirement and decide whether it needs a chatbot, RAG-based document Q&A, summarizer, classifier, extractor, SQL generator, email assistant, workflow automation assistant, or multi-modal AI application. You should also be able to draw a simple architecture and write a first version of the prompt for each pattern.

13.1 What Is an LLM Application Pattern?

An LLM application pattern is a standard way of using a language model to solve a particular type of task. The LLM is only one part of the application. A real application also needs input handling, authentication, prompt templates, data retrieval, validation, output formatting, monitoring, and sometimes human approval.

For example, suppose a company wants an AI system that reads customer emails and decides whether the issue is related to billing, technical support, cancellation, refund, or sales. This is not mainly a chatbot problem. It is a classification problem. The best design is usually an LLM classification pattern, possibly supported by examples, business rules, and confidence scoring.

Suppose another company wants employees to ask questions such as, "How many paid leaves can I carry forward?" The answer must come from the company HR policy. This is not only a general chat problem. It is a document Q&A or RAG pattern because the model must retrieve relevant policy content before answering.

Simple way to think about patterns

Requirement type	Useful pattern
User wants conversation	Chatbot
User asks questions from documents	Document Q&A / RAG
Long text must become short	Summarization
Input must be assigned to a label	Classification
Specific fields must be pulled from text	Extraction
Numbers must be converted into business language	Report generation
User wants help analyzing data	Data analysis assistant
User wants code help	Code assistant
User wants database answers from natural language	SQL generation
User wants email drafting or reply help	Email assistant
Customer issue must be handled automatically	Customer support automation
Ticket must go to the correct team	Ticket routing
Company knowledge must be searchable by AI	Knowledge base assistant
AI must perform multi-step tasks	Workflow automation
AI must handle text plus images/audio/video	Multi-modal AI application

13.2 Common Building Blocks Across Patterns

Although each pattern solves a different problem, many of them use the same building blocks. Understanding these blocks helps you design any LLM application more easily.

User / System Input

Application UI or API

Prompt Template + Business Rules

+---- Optional Retrieval from Documents / Database / Tools

LLM Call

Output Parser / Validator / Guardrails

Final Response / Action / Workflow Update

Logs, Feedback, Evaluation, Monitoring

The core idea is simple: collect the input, prepare the model instruction, optionally retrieve supporting data, call the LLM, validate the answer, and return the result to the user or downstream system.

Design questions before choosing a pattern

Is the user asking for conversation, an answer, a summary, a category, a field extraction, a report, or an action?
Does the answer require private company data or only general knowledge?
Should the output be free text, JSON, table, email, SQL, code, or a workflow action?
Is there a risk if the model gives a wrong answer?
Does the application need human approval before taking action?
How will the quality be measured?

13.3 Chatbot Pattern

Problem Statement

A chatbot allows users to interact with an AI system through conversation. The user can ask follow-up questions, clarify requirements, and receive answers in a natural language format. Chatbots are useful for learning assistants, support assistants, HR assistants, sales assistants, and internal productivity tools.

Architecture

User Chat Window

Conversation API

Session History + User Profile + Prompt Template

LLM

Safety Check + Response Formatter

Chat Response

Input and Output

Input	Output
User message, previous conversation history, user role, optional business context.	Natural language answer, clarification question, instruction, or formatted response.

Example Prompt

You are a helpful support assistant. Answer the user clearly and ask a clarifying question only when required. User question: "I cannot login after password reset. What should I do?"

Example Response

Please try clearing your browser cache and then login again with the new password. If the issue continues, check whether your account is locked. I can also help you prepare a support ticket with the error message and your user ID.

Risks

The chatbot may answer outside its knowledge boundary.
It may sound confident even when the answer is uncertain.
Long conversation history can cause confusion or high token cost.
Users may share sensitive data in chat.

Best Practices

Define the chatbot role clearly in the system prompt.
Keep conversation history summarized when it becomes long.
Use RAG when the chatbot must answer from company documents.
Add guardrails for privacy, unsafe content, and restricted topics.
Log conversations for quality review while protecting sensitive data.

13.4 Document Q&A Pattern

Problem Statement

Document Q&A allows users to ask questions from PDFs, Word files, web pages, policy documents, manuals, contracts, standard operating procedures, and knowledge articles. This pattern usually uses RAG because the LLM must retrieve the most relevant document chunks before answering.

Architecture

Documents -> Chunking -> Embeddings -> Vector Database

User Question -> Query Embedding -> Retriever -> Relevant Chunks

Prompt + Context

LLM

Answer with Sources

Input and Output

Input	Output
User question and a document collection already indexed in a vector database.	Answer grounded in retrieved document context, often with source references.

Example Prompt

Answer the question only using the provided policy context. If the answer is not available, say that the policy does not provide enough information. Question: "How many casual leaves can an employee take in a year?" Context: {retrieved_policy_chunks}

Example Response

According to the provided HR policy context, employees are eligible for 12 casual leaves per calendar year. The policy also says unused casual leave cannot be carried forward.

Risks

Wrong chunk retrieval can produce wrong answers.
Old or duplicate documents may confuse the model.
Source citation can be misleading if not mapped correctly.
Very large documents need good chunking and metadata.

Best Practices

Use clean document ingestion and metadata.
Choose chunk size carefully based on document type.
Use source citations for trust.
Add fallback when no relevant context is found.
Evaluate retrieval quality separately from answer quality.

13.5 Summarization Pattern

Problem Statement

Summarization converts long content into shorter, useful content. It is used for meeting notes, long reports, emails, legal documents, call transcripts, medical discharge summaries, customer complaints, and research papers.

Architecture

Long Text / Document

Pre-processing and Chunking

Chunk-level Summaries

Summary Merger

Final Summary + Key Points + Action Items

Input and Output

Input	Output
Long text, document, transcript, report, or multiple files.	Short summary, executive summary, bullet summary, action items, risks, or decision notes.

Example Prompt

Summarize the following customer call transcript for a support manager. Include issue, customer sentiment, promised action, and next step. Transcript: {transcript}

Example Response

Issue: The customer is unable to access the billing portal. Sentiment: Frustrated but cooperative. Promised action: Support will reset the account session and share a new login link. Next step: Follow up within 24 hours and confirm login success.

Risks

Important details may be missed.
The summary may over-compress complex content.
The model may add interpretation not present in the source.
Long documents can exceed context limits.

Best Practices

Specify the summary type and audience.
Ask for structured sections.
Use map-reduce summarization for long documents.
Ask the model not to add unsupported facts.
Keep original source available for audit.

13.6 Classification Pattern

Problem Statement

Classification assigns input text to one or more predefined categories. It is useful for ticket routing, sentiment detection, email triage, fraud alert grouping, document type detection, risk classification, and intent recognition.

Architecture

Incoming Text

Prompt with Label Definitions + Examples

LLM Classifier

JSON Output: label, confidence, reason

Workflow Rule / Dashboard / Queue

Input and Output

Input	Output
Text to classify and list of allowed labels with definitions.	Category, confidence score, short reason, and optionally next action.

Example Prompt

Classify this IT ticket into one of: Network, Hardware, Database, Security, HR Payroll, Application Support. Return JSON only. Ticket: "I clicked a suspicious email link and now my laptop is behaving strangely."

Example Response

{"category":"Security","confidence":"high","reason":"The ticket mentions a suspicious email link and possible compromised device behavior."}

Risks

Labels may overlap.
The model may invent labels if not restricted.
Confidence may not be calibrated.
Regulated decisions should not rely only on LLM classification.

Best Practices

Provide allowed labels and definitions.
Return structured JSON.
Add examples for confusing labels.
Use confidence thresholds and human review.
Track accuracy with a labeled test set.

13.7 Extraction Pattern

Problem Statement

Extraction pulls specific fields from unstructured or semi-structured text. It is used to extract invoice details, resume skills, contract dates, patient discharge details, customer names, complaint IDs, order numbers, and policy clauses.

Architecture

Source Text / Document

Extraction Prompt + Schema

LLM

JSON Parser + Validation

Database / API / Review Screen

Input and Output

Input	Output
Document text and required output schema.	Structured JSON, table, CSV row, or database-ready object.

Example Prompt

Extract invoice_number, vendor_name, invoice_date, total_amount, currency, and payment_due_date from this invoice text. Return JSON only. Text: {invoice_text}

Example Response

{"invoice_number":"INV-2026-1045","vendor_name":"ABC Services Pvt Ltd","invoice_date":"2026-06-10","total_amount":45000,"currency":"INR","payment_due_date":"2026-07-10"}

Risks

OCR errors can cause wrong extraction.
Missing values may be guessed.
Different document formats may need different prompts.
JSON can break if not validated.

Best Practices

Use strict output schema.
Tell the model to return null for missing values.
Validate extracted fields using business rules.
Use human review for low-confidence extraction.
Keep source text and extracted data linked for traceability.

13.8 Report Generation Pattern

Problem Statement

Report generation converts data, metrics, and findings into readable business reports. It is used in finance, sales, project management, operations, risk management, and executive dashboards.

Architecture

Structured Data / Metrics

Data Aggregation + Business Rules

Report Prompt Template

LLM

Formatted Report: Summary, Trends, Risks, Recommendations

Input and Output

Input	Output
Metrics, tables, KPI values, comments, time period, and audience type.	Executive summary, detailed report, risk analysis, recommendation section, or email-ready report.

Example Prompt

Create a weekly project status report for senior management using these metrics: completed_tasks=42, delayed_tasks=6, open_risks=3, budget_used=72%, sprint_goal_status="at risk". Include summary, risks, and recommended actions.

Example Response

This week the project completed 42 tasks, but the sprint goal is at risk due to 6 delayed tasks and 3 open risks. Budget consumption is at 72%, which is acceptable but should be monitored. Recommended actions: review delayed tasks, assign owners to risks, and re-check sprint scope.

Risks

The model may create unsupported explanations.
Poor input data leads to poor reports.
Reports may sound good but hide uncertainty.
Numeric mistakes can occur if calculations are left to the model.

Best Practices

Calculate numbers outside the LLM.
Pass verified metrics to the model.
Ask the model to explain only from provided data.
Use templates for consistent report sections.
Review important reports before publishing.

13.9 Data Analysis Assistant Pattern

Problem Statement

A data analysis assistant helps users understand datasets, ask questions about metrics, generate charts, identify anomalies, explain trends, and create insights. In production, the LLM should not freely guess from data. It should use tools such as SQL, Python, BI APIs, or governed semantic layers.

Architecture

User Question

Intent Understanding

+--> Data Catalog / Schema Lookup

+--> SQL or Python Tool

Verified Result Table / Chart Data

LLM Explanation

Insight + Caveats + Next Questions

Input and Output

Input	Output
Natural language question, dataset schema, business definitions, and allowed analysis tools.	Insight, table, chart description, trend explanation, anomaly explanation, or next-step recommendation.

Example Prompt

The following sales data has already been calculated by SQL. Explain the trend for a business user and mention possible reasons without claiming certainty. Data: {monthly_sales_table}

Example Response

Sales increased steadily from January to March, dropped slightly in April, and recovered in May. The recovery may indicate improved demand or campaign impact, but campaign and market data should be checked before making a final conclusion.

Risks

The model may calculate incorrectly.
It may confuse column meanings.
It may overstate correlation as causation.
Sensitive data access must be controlled.

Best Practices

Use governed data access.
Let SQL/Python perform calculations.
Give the model data definitions.
Ask for caveats and assumptions.
Log generated queries and results.

13.10 Code Assistant Pattern

Problem Statement

A code assistant helps developers write, explain, debug, refactor, document, and test code. It can improve productivity, but it must be used carefully because generated code may be insecure, outdated, or incompatible with the project environment.

Architecture

Developer Request

Project Context + Language + Framework + Error Logs

LLM Code Assistant

Generated Code / Explanation / Test Cases

Developer Review + Unit Tests + Security Scan

Input and Output

Input	Output
Requirement, existing code, error message, stack trace, language, framework, and constraints.	Code snippet, explanation, debugging steps, test case, refactoring plan, or documentation.

Example Prompt

Explain why this Python function fails and provide a corrected version. Keep the answer beginner-friendly. Code: {code_snippet}

Example Response

The function fails because it tries to add a string and an integer. Convert the input to an integer before addition, and handle invalid input using try/except.

Risks

Generated code may not run.
It may use old APIs.
It may introduce security vulnerabilities.
It may ignore project-specific conventions.

Best Practices

Always run and test generated code.
Provide exact language, version, framework, and constraints.
Ask for explanation, not only code.
Use code review and security scanning.
Avoid pasting secrets or proprietary code unless the environment is approved.

13.11 SQL Generation Pattern

Problem Statement

SQL generation allows users to ask questions in natural language and receive SQL queries or data answers. It is useful for analytics, reporting, business intelligence, and self-service data access. This pattern should be designed with strong safeguards because SQL can access sensitive data or produce expensive queries.

Architecture

Natural Language Question

Schema + Business Definitions + Security Rules

LLM Generates SQL

SQL Validator / Permission Check / Cost Guard

Database Execution

Result Explanation

Input and Output

Input	Output
User question, database schema, table descriptions, column definitions, and access rules.	SQL query, result table, explanation, or error clarification.

Example Prompt

Generate SQL for BigQuery. Use only the provided schema. Question: "Show total sales by month for 2026." Schema: sales(order_id, order_date, amount, customer_id). Return SQL only.

Example Response

SELECT FORMAT_DATE("%Y-%m", order_date) AS sales_month, SUM(amount) AS total_sales FROM sales WHERE EXTRACT(YEAR FROM order_date) = 2026 GROUP BY sales_month ORDER BY sales_month;

Risks

SQL may scan too much data.
The model may use non-existing columns.
Wrong joins can produce wrong numbers.
Unauthorized data exposure can occur.

Best Practices

Provide only allowed schemas.
Validate SQL before execution.
Use read-only database accounts.
Apply row-level and column-level security.
Set query cost limits and require approval for risky queries.

13.12 Email Assistant Pattern

Problem Statement

An email assistant drafts, rewrites, summarizes, classifies, and responds to emails. It can save time in sales, HR, support, operations, project management, and customer communication.

Architecture

Email Thread / User Instruction

Context Builder: recipient, tone, purpose, constraints

LLM Draft Generator

Policy Check + Human Review

Draft / Send / Archive / Follow-up Reminder

Input and Output

Input	Output
Email thread, desired tone, objective, recipient type, and action requested.	Draft email, reply, subject line, summary, or follow-up note.

Example Prompt

Draft a polite email to a customer explaining that the refund is approved and will be processed within 5 business days. Keep it professional and concise.

Example Response

Subject: Refund Approval Update

Dear Customer,

Your refund request has been approved. The amount will be processed within 5 business days and credited to your original payment method. Thank you for your patience.

Regards,

Support Team

Risks

The assistant may send incorrect commitments.
Sensitive information may be included accidentally.
Tone may not match company policy.
Automatic sending without review can be risky.

Best Practices

Prefer draft mode before sending.
Use approved tone and policy templates.
Check recipient, attachments, and sensitive content.
Use human approval for external emails.
Log email generation and final edits.

13.13 Customer Support Automation Pattern

Problem Statement

Customer support automation combines multiple patterns: classification, summarization, RAG, response drafting, workflow action, and escalation. It helps reduce response time and improve support consistency.

Architecture

Customer Message

Classify Intent + Priority + Sentiment

+--> Retrieve Knowledge Articles / Policies

Draft Response + Suggested Action

Confidence Check

+--> Auto Reply if Safe

+--> Human Agent if Risky

Ticket Update + Monitoring

Input and Output

Input	Output
Customer message, account metadata, product information, knowledge base, and support policies.	Suggested response, ticket category, priority, knowledge article, resolution step, or escalation.

Example Prompt

A customer wrote: "My payment was deducted but order is not showing." Classify issue, suggest priority, draft response, and mention required backend check.

Example Response

Category: Payment/Order Sync. Priority: High. Draft: We are sorry for the inconvenience. Please share your transaction ID so we can verify the payment and order status. Backend check required: payment gateway confirmation and order creation log.

Risks

Wrong auto-reply can upset customers.
Refund or legal commitments may be made incorrectly.
Complex cases may need human support.
Model may miss account-specific facts.

Best Practices

Use confidence thresholds.
Keep humans in the loop for refunds, legal, or angry customers.
Retrieve approved knowledge articles.
Track resolution accuracy and customer satisfaction.
Avoid making promises the system cannot verify.

13.14 Ticket Routing Pattern

Problem Statement

Ticket routing sends incoming issues to the correct team or queue. It is widely used in IT service management, HR helpdesks, banking operations, telecom support, and enterprise shared services.

Architecture

Incoming Ticket

Clean Text + Extract Entities

Classify Category + Urgency + Product

Routing Rules

Assigned Team + SLA + Suggested First Response

Input and Output

Input	Output
Ticket title, description, user department, product, attachment metadata, and historical ticket examples.	Assigned team, category, priority, SLA, and explanation.

Example Prompt

Route this ticket to the correct team. Teams: Network, Hardware, Database, Security, Application Support, HR Payroll. Ticket: "Oracle server connection timeout from reporting app." Return JSON.

Example Response

{"team":"Database","priority":"medium","reason":"The issue mentions Oracle server connection timeout, which is usually handled by the database team. Application support may be involved if DB is healthy."}

Risks

Some tickets need multiple teams.
Ambiguous descriptions can cause wrong routing.
Historical data may contain old team names.
Bad routing increases SLA breaches.

Best Practices

Use label definitions and examples.
Allow secondary team suggestions.
Use confidence score and fallback queue.
Continuously retrain or update examples from resolved tickets.
Measure routing accuracy and reassignment rate.

13.15 Knowledge Base Assistant Pattern

Problem Statement

A knowledge base assistant helps employees or customers find accurate answers from approved articles, FAQs, manuals, policies, and troubleshooting guides. It is similar to document Q&A but often includes article ranking, feedback, and content improvement suggestions.

Architecture

Knowledge Articles / FAQs / Manuals

Indexing: metadata + embeddings + keyword search

User Question

Hybrid Retrieval + Re-ranking

LLM Answer + Article Links

Feedback: helpful/not helpful + gap detection

Input and Output

Input	Output
User question, knowledge base content, metadata, article quality signals, and access permissions.	Answer, article links, step-by-step guide, or escalation suggestion.

Example Prompt

Using the retrieved knowledge base articles, answer the user question and include the most relevant article title. Question: "How do I reset VPN access after password change?" Context: {articles}

Example Response

After a password change, disconnect the VPN client, sign in with the new password, and approve MFA. If login still fails, clear saved credentials and retry. Relevant article: "VPN Login After Password Reset".

Risks

Outdated articles can produce outdated answers.
Multiple articles may conflict.
Access control errors can expose restricted information.
No-answer scenarios may be handled poorly.

Best Practices

Keep article metadata updated.
Use hybrid search for better recall.
Show article links and source titles.
Use feedback to identify poor articles.
Respect role-based access control.

13.16 Workflow Automation Pattern

Problem Statement

Workflow automation uses an LLM to understand instructions, decide next steps, call tools, update systems, create tasks, send drafts, or trigger approvals. This pattern is powerful but must be designed with clear permissions and safety checks.

Architecture

User Request

Intent + Required Action Detection

Plan Steps

+--> Tool/API Calls

+--> Human Approval for Sensitive Actions

Execute Workflow

Confirm Result + Log Action

Input and Output

Input	Output
User request, available tools, permissions, business rules, and system state.	Completed workflow, draft action, task creation, system update, or approval request.

Example Prompt

The user says: "Create a follow-up task for the payment issue and assign it to the billing team." Identify the workflow steps and return a safe action plan.

Example Response

Action plan: 1) Create support task titled "Payment issue follow-up". 2) Assign to Billing Team. 3) Link it to the original ticket. 4) Set due date according to SLA. 5) Notify the user after task creation.

Risks

Wrong tool calls can change real systems.
The model may misunderstand user intent.
Permissions may be too broad.
Automation without approval can cause business damage.

Best Practices

Use least-privilege tool permissions.
Ask for confirmation before sensitive actions.
Validate tool inputs.
Keep detailed audit logs.
Use deterministic workflow rules where possible.

13.17 Multi-modal AI Application Pattern

Problem Statement

Multi-modal applications process more than one type of input: text, images, audio, video, documents, tables, and sometimes sensor data. They are useful for medical imaging support, insurance claim inspection, product catalog search, education, accessibility, manufacturing quality checks, and visual document understanding.

Architecture

Text / Image / Audio / Video Input

Modality-specific Pre-processing

+--> OCR / Speech-to-Text / Image Understanding / Frame Extraction

Multi-modal Model or LLM + Tools

Structured Output / Explanation / Recommendation

Input and Output

Input	Output
Text plus image, scanned document, audio transcript, video frames, or mixed content.	Description, classification, extracted fields, answer, recommendation, or generated content.

Example Prompt

Analyze this product damage image and customer description. Identify visible damage, classify severity, and suggest next support action. Description: "The package arrived with a broken corner."

Example Response

Visible issue: corner damage is reported and may be packaging-related. Severity: medium, pending image verification. Suggested action: request order ID, confirm if product is affected, and initiate replacement workflow if damage is confirmed.

Risks

Images may be unclear or misleading.
Medical/legal decisions require expert review.
Privacy risk is high for images and audio.
Multi-modal models may still hallucinate details.

Best Practices

Use high-quality inputs.
Ask the model to mention uncertainty.
Use human review for high-risk decisions.
Mask sensitive information when possible.
Store media securely with access control.

13.18 How Patterns Are Combined in Real Projects

In real systems, one application often uses multiple LLM patterns together. A customer support solution may classify the ticket, summarize the message, retrieve a knowledge article, draft a response, update the ticket, and route the case to a team. A document assistant may combine document Q&A, summarization, extraction, and report generation. A data assistant may combine SQL generation, data analysis, report generation, and email drafting.

Example: Enterprise Support Copilot

Customer Message

+--> Classification: billing / technical / refund / complaint

+--> Extraction: order number, product, issue date

+--> RAG: retrieve policy and troubleshooting article

+--> Summarization: create short case summary

+--> Response Drafting: prepare customer reply

+--> Workflow Automation: create task / escalate / update CRM

Human Review or Auto Resolution

The most important design skill is not memorizing patterns separately, but knowing how to combine them safely. Start with the smallest useful pattern, test it, then add more capabilities step by step.

13.19 Pattern Selection Decision Guide

Question	Recommended pattern
Do users need a conversational interface?	Chatbot
Must answers come from documents?	Document Q&A / Knowledge Base Assistant
Is the input too long and needs reduction?	Summarization
Do you need to assign a category?	Classification
Do you need fields in JSON/table format?	Extraction
Do you need narrative from metrics?	Report Generation
Do users ask business questions from data?	Data Analysis Assistant / SQL Generation
Do developers need code help?	Code Assistant
Do support cases need end-to-end handling?	Customer Support Automation
Do tasks need system actions?	Workflow Automation / Agents
Do inputs include images, audio, or video?	Multi-modal AI Application

13.20 Practical Enterprise Example: AI Helpdesk Assistant

Let us design a practical AI helpdesk assistant for an IT department. Employees submit support tickets in natural language. The AI assistant should classify the ticket, summarize the issue, route it to the correct team, retrieve a helpful article, and draft the first response.

Architecture

Employee Ticket

Text Cleaner and PII Masker

+--> Classification Pattern: team and category

+--> Summarization Pattern: short issue summary

+--> Extraction Pattern: system name, error code, urgency

+--> Knowledge Base Assistant: relevant article

+--> Email/Response Assistant: draft reply

Ticket Update + Human Agent Review

Example Input

Subject: VPN not working after password reset

Description: I changed my password this morning. Now VPN is not accepting the new password. I need access urgently for client work.

Expected AI Output

Category: Network / VPN

Priority: High

Summary: User cannot login to VPN after password reset and needs urgent access.

Extracted entity: VPN, password reset, client work urgency

Suggested article: VPN login after password change

Draft response: Please clear saved VPN credentials, restart the VPN client, and login using the new password. If MFA is enabled, approve the login prompt. If the problem continues, we will reset your VPN profile.

Why multiple patterns are useful here

Classification alone only gives the team. Summarization helps the support agent quickly understand the issue. Extraction captures system and urgency. Knowledge base retrieval gives a reliable troubleshooting step. Response drafting saves time. Together, these patterns create a useful business application.

13.21 Pattern-Level Production Checklist

Define the exact task: chat, answer, summarize, classify, extract, generate, analyze, route, or automate.
Specify allowed inputs and expected outputs.
Use structured output such as JSON when downstream systems need to consume the result.
Keep prompts version-controlled like application code.
Use examples for difficult or ambiguous cases.
Separate model reasoning from final user-visible response.
Validate all outputs before using them in production workflows.
Use confidence thresholds and human review for risky decisions.
Do not let the LLM directly modify critical systems without approval.
Track accuracy, latency, cost, and user feedback.
Protect personal, financial, medical, and confidential data.
Create a fallback path when the model cannot answer safely.

13.22 Common Mistakes Across LLM Application Patterns

Mistake	Why it is a problem	Better approach
Using a chatbot for every problem	Many tasks need classification, extraction, or retrieval instead of open conversation.	Choose the pattern based on the output required.
No output validation	The model may return invalid JSON, wrong labels, or unsupported answers.	Use parsers, schemas, rules, and tests.
No source grounding	Answers can hallucinate when they require private knowledge.	Use RAG or approved knowledge base retrieval.
Letting the LLM do calculations	LLMs can make arithmetic and aggregation mistakes.	Use SQL/Python for calculations, LLM for explanation.
No human review for risky actions	Wrong automation can affect customers or systems.	Add approval gates for high-impact actions.
Ignoring cost and latency	Long prompts and unnecessary context increase cost.	Optimize prompt, context, model choice, and caching.

13.23 Mini Project: Build a Pattern Selector

A useful exercise is to build a simple AI pattern selector. The user enters a business requirement, and the system recommends the best LLM pattern with an explanation.

Example requirements

"Read HR policy and answer employee questions." -> Document Q&A / RAG.
"Assign incoming support tickets to teams." -> Classification / Ticket Routing.
"Extract invoice amount and vendor name from PDFs." -> Extraction.
"Create weekly project status from Jira metrics." -> Report Generation.
"Answer sales questions from database." -> SQL Generation / Data Analysis Assistant.
"Draft replies to customer complaints." -> Email Assistant / Customer Support Automation.

Pseudo-code

def select_llm_pattern(requirement):

if mentions_documents_or_policy(requirement):

return "Document Q&A / RAG"

if asks_for_category_or_team(requirement):

return "Classification / Ticket Routing"

if asks_to_extract_fields(requirement):

return "Extraction"

if asks_for_shorter_version(requirement):

return "Summarization"

if asks_to_query_database(requirement):

return "SQL Generation / Data Analysis Assistant"

if asks_to_perform_actions(requirement):

return "Workflow Automation"

return "Chatbot or General Assistant"

Chapter Summary

This chapter explained the most important LLM application patterns used in practical AI systems. A chatbot is useful for conversation, but it is not the answer to every problem. Document Q&A uses retrieval to answer from trusted documents. Summarization reduces long content. Classification assigns labels. Extraction converts unstructured text into structured data. Report generation converts metrics into business language. Data analysis assistants use tools and verified results. Code assistants help developers but require review. SQL generation enables natural language access to databases but needs strong security. Email assistants improve communication. Customer support automation and ticket routing combine several patterns. Knowledge base assistants use approved content. Workflow automation performs multi-step actions with tools and approvals. Multi-modal applications handle text, images, audio, and video.

The key lesson is that LLM applications should be designed as systems, not just prompts. Each pattern has input, architecture, output, risks, and best practices. A good AI developer chooses the correct pattern, grounds the model when needed, validates the output, protects data, monitors quality, and adds human review where risk is high.

Key Terms

Term	Meaning
LLM application pattern	A reusable design for solving a specific type of task using a language model.
Chatbot	A conversational AI interface that responds to user messages.
Document Q&A	A pattern where users ask questions and the system answers from documents.
Summarization	The process of converting long content into shorter useful content.
Classification	Assigning text to one or more predefined categories.
Extraction	Pulling specific structured fields from text or documents.
Report generation	Creating readable business reports from data or metrics.
SQL generation	Creating database queries from natural language questions.
Knowledge base assistant	An AI assistant that answers using approved knowledge articles.
Workflow automation	Using AI to understand instructions and trigger actions through tools or APIs.
Multi-modal AI	AI that can process multiple input types such as text, images, audio, and video.

Practice Exercises

Write five business requirements and identify the best LLM application pattern for each.
Design a classification prompt for routing IT tickets to Network, Hardware, Database, Security, HR Payroll, and Application Support.
Create an extraction schema for invoice processing. Include invoice number, date, vendor, amount, tax, and due date.
Write a prompt that summarizes a meeting transcript into decisions, action items, owners, and deadlines.
Draw a simple architecture for a knowledge base assistant for your company or college.
Compare chatbot and document Q&A. Explain when chatbot alone is not enough.
Design a customer support automation flow that uses classification, RAG, response drafting, and human review.
Write three risks of SQL generation and three controls to reduce those risks.
Create a mini design for a data analysis assistant that uses SQL for calculation and LLM for explanation.
Choose one pattern and write a complete example with problem statement, input, prompt, output, risk, and best practice.

Appendix: Quick Reference Table

Pattern	Best for	Typical output	Needs retrieval?	Needs human review?
Chatbot	Conversation	Natural response	Sometimes	Sometimes
Document Q&A	Answers from documents	Grounded answer + source	Yes	For sensitive topics
Summarization	Long content reduction	Summary/action items	No, unless sources are external	Sometimes
Classification	Labels and routing	Category + confidence	Sometimes	For low confidence
Extraction	Structured fields	JSON/table	No, but validation needed	For critical fields
Report generation	Business narrative	Report	Sometimes	Often
Data analysis assistant	Data insights	Explanation/chart/table	Uses data tools	Sometimes
Code assistant	Developer help	Code/explanation/tests	Sometimes	Yes, code review
SQL generation	Database questions	SQL/result explanation	Schema retrieval	For risky queries
Email assistant	Drafting replies	Email draft	Sometimes	Usually before sending
Support automation	Ticket handling	Reply/action/escalation	Often	For risky cases
Workflow automation	Multi-step actions	System action	Sometimes	Yes for sensitive actions
Multi-modal AI	Text + image/audio/video	Description/extraction/action	Sometimes	For high-risk domains