Dynamic Prompts

Static vs Dynamic Prompts

Runtime Context Injection

from datetime import date, datetime

def build_system_prompt(user: dict, task_meta: dict) -> str:
    lines = [
        f"You are an assistant for {user['name']} ({user['role']}).",
        f"Today is {date.today().isoformat()}, {datetime.now().strftime('%H:%M')} UTC.",
        f"Task type: {task_meta['type']}.",
    ]
    if task_meta.get("urgency") == "high":
        lines.append("This is urgent — be concise and prioritise speed.")
    return "\n".join(lines)

Dynamic Few-Shot Example Selection

Select the most relevant examples from a large library using semantic similarity:

from sentence_transformers import SentenceTransformer, util
import json

model = SentenceTransformer("all-MiniLM-L6-v2")

with open("prompts/examples.json") as f:
    examples = json.load(f)

example_embeddings = model.encode([e["input"] for e in examples])

def select_examples(query: str, k: int = 3) -> list:
    query_emb = model.encode(query)
    scores = util.cos_sim(query_emb, example_embeddings)[0]
    top_k = scores.topk(k).indices.tolist()
    return [examples[i] for i in top_k]

Persona-Based Prompt Adaptation

PERSONAS = {
    "beginner": "Explain step by step. Avoid jargon. Include examples.",
    "expert": "Be concise. Use technical terminology. Skip basics.",
    "executive": "Focus on business impact. Use bullet points. Max 100 words.",
}

def adapt_prompt(base_prompt: str, persona: str) -> str:
    style = PERSONAS.get(persona, PERSONAS["beginner"])
    return f"{base_prompt}\n\nCommunication style: {style}"

Prompt Length Budgeting

import tiktoken

def trim_to_budget(text: str, max_tokens: int, model: str = "gpt-4o") -> str:
    enc = tiktoken.encoding_for_model(model)
    tokens = enc.encode(text)
    if len(tokens) <= max_tokens:
        return text
    return enc.decode(tokens[:max_tokens])

Retrieval-Augmented Prompts

def build_rag_prompt(question: str, retrieved_chunks: list[str]) -> str:
    context = "\n\n---\n\n".join(retrieved_chunks[:5])
    return f"""Answer the question using only the provided context.
If the answer is not in the context, say "I don't know."

Context:
{context}

Question: {question}"""

Caching Dynamic Prompts

Dynamic prompts can be expensive to rebuild on every request — especially when they involve database lookups, API calls, or embedding searches. Apply caching at the component level, not the full prompt.

import hashlib, json
from functools import lru_cache
from datetime import datetime

# Cache static persona strings (pure function, safe to cache forever)
@lru_cache(maxsize=32)
def get_persona_block(persona: str) -> str:
    return PERSONAS.get(persona, PERSONAS["beginner"])

# Cache retrieved examples keyed on query hash (TTL ~1 hour)
_example_cache: dict[str, tuple[float, list]] = {}

def cached_examples(query: str, k: int = 3) -> list:
    key = hashlib.sha256(query.encode()).hexdigest()[:16]
    now = datetime.utcnow().timestamp()
    if key in _example_cache:
        ts, result = _example_cache[key]
        if now - ts < 3600:
            return result
    result = select_examples(query, k)
    _example_cache[key] = (now, result)
    return result

For prompts that share the same static prefix (e.g. a long system prompt), use OpenAI Prompt Caching — prefix tokens are cached automatically for repeated requests with the same leading content, cutting cost by up to 50%.

Error Handling and Fallbacks

Dynamic prompt construction can fail at runtime — missing user records, empty retrieval results, encoding errors. Always define a safe fallback path.

def build_prompt_safe(user_id: str, task: dict) -> str:
    try:
        user = db.get_user(user_id)          # may raise
        examples = cached_examples(task["query"])  # may return []
        return build_full_prompt(user, task, examples)
    except Exception as exc:
        logger.warning("Prompt build failed: %s — using fallback", exc)
        return build_fallback_prompt(task)  # minimal safe prompt

def build_fallback_prompt(task: dict) -> str:
    return (
        "You are a helpful assistant.\n"
        f"Task: {task.get('query', 'Help the user.')}"
    )

Best Practices

• Inject only what is needed. Trim long context chunks to their relevant portion before inserting — every extra token raises cost and reduces focus.
• Version your templates. Store dynamic prompt templates in a YAML promptbook with version tags so you can roll back when a prompt change degrades quality.
• Log both prompt and response. When debugging unexpected outputs, the full constructed prompt is invaluable — log it (redacting PII) at the debug level.
• Test with the broadest persona. Persona-adaptive prompts often break on edge personas. Include a "stranger" or "unknown" persona in your test suite.
• Set a hard token budget. Always pass max_tokens to limit runaway generation and reduce cost spikes from unexpectedly large context windows.