Week 5 Lab — Writing Prompts Well

Goal

Apply the four core techniques from today's lecture to the same function, evaluated against a specification you write first. The specification step is the most important part — do not skip it.

Working style: Individual work, pair submission · Total time: ~60 minutes

Overview

You work on your own through Step 0 and the four techniques, then pair up to compare outputs, pick one docstring to approve, write one shared paragraph, and submit together. You write the prompts, the model writes the docstring, and you judge it.

Timeline

The Function

This is a different function from the one in the lecture demonstrations.

def process_records(records, threshold):
    result = []
    seen = {}
    for r in records:
        key = r.get('id')
        val = r.get('score', 0)
        if key in seen:
            continue
        seen[key] = True
        if val >= threshold:
            result.append({'id': key, 'score': val, 'status': 'pass'})
        else:
            result.append({'id': key, 'score': val, 'status': 'fail'})
    return result

Read this carefully before writing anything. Understand what it does, what each parameter means, what it returns, and what might surprise a new engineer reading this for the first time.

Your Task and What You Produce

Task: get the model to write a docstring for process_records that meets your own specification. You write the prompts, the model writes the docstring, and your job is to judge it.

You produce:

• Your specification (Step 0)
• Four docstrings, one per technique, each with the prompt that produced it and your analysis
• At the end, your pair's approved docstring: the one you would approve as code steward (you may edit it, and note what you changed), with a shared paragraph explaining the choice

Which docstring counts for each technique: for zero-shot and few-shot, the model's output; for chain-of-thought, the docstring after the reasoning steps; for self-correction, the revised docstring.

Step 0 — Write Your Specification (5 minutes)

Before you write any prompt, write a specification for what a good docstring for this function should contain. Use these five fields:

• What the function does: one sentence describing the overall behaviour.
• Parameters: for each parameter, its name, expected type, and what it represents.
• Returns: what the function returns, including the structure of each item and what each field means.
• Edge cases: at least two behaviours a new engineer might not anticipate from reading the code.
• Format: which docstring style (Google, NumPy, plain prose) and any length constraint.

Keep this visible throughout the lab. Every output you evaluate gets checked against it — not against a general sense of quality.

Part 1 — Four Techniques (35 minutes)

Run each technique on the function above. For each one: the prompt verbatim, the full output, and a short analysis.

Use whichever model and interface you have access to — Claude.ai, ChatGPT, Gemini, or the API. The lab works on any interface.

Technique 1: Zero-shot

Write a prompt asking the model to write a docstring for this function. Write it naturally — no examples, no reasoning scaffold.

Record:

• Your prompt (verbatim)
• The full model output
• Analysis (2–3 sentences): which fields from your specification did this satisfy? Which did it miss or handle inaccurately? Name the field and describe the gap specifically.

Technique 2: Few-shot

Use the three (function, docstring) examples below. Copy all three into your prompt, in order, and then add your own task for process_records.

Each example follows the same format: a Google-style docstring with Args, Returns, and an extra Edge cases section. A Raises section appears only where the code raises an exception explicitly. Read all three before you write your prompt — notice what they include and at what level of detail, since the model will learn from them, so whatever they show is what it is likely to copy.

Example 1

Function:
def group_by_key(records, key):
    groups = {}
    for r in records:
        value = r.get(key)
        if value not in groups:
            groups[value] = []
        groups[value].append(r)
    return groups

Docstring:
"""
Group a list of dictionaries by the value stored under a given key.

Args:
    records (list[dict]): The records to group.
    key (str): The key whose value determines each record's group.

Returns:
    dict: A mapping from each distinct value of ``key`` to the list of
        records that have that value. Groups appear in order of first
        occurrence, and records keep their original order within a group.

Edge cases:
    - A record without ``key`` is placed in the group ``None``, together
      with any record whose value for ``key`` is explicitly ``None``.
    - The records are not copied, so changing a returned record also
      changes the input list.
    - An empty ``records`` list returns an empty dict.
"""

Example 2

Function:
def normalize_scores(scores, lo=0.0, hi=1.0):
    if not scores:
        return []
    smallest = min(scores)
    largest = max(scores)
    if smallest == largest:
        return [lo] * len(scores)
    span = largest - smallest
    return [lo + (s - smallest) / span * (hi - lo) for s in scores]

Docstring:
"""
Rescale a list of numbers linearly so they fall between ``lo`` and ``hi``.

Args:
    scores (list[float]): The numbers to rescale.
    lo (float): The value assigned to the smallest score. Defaults to 0.0.
    hi (float): The value assigned to the largest score. Defaults to 1.0.

Returns:
    list[float]: A new list, in the same order as ``scores``, where the
        smallest score becomes ``lo`` and the largest becomes ``hi``.

Edge cases:
    - An empty list returns an empty list.
    - If every score is the same, there is no range to scale, so every
      element of the result is ``lo`` (not the midpoint).
    - ``lo`` and ``hi`` are not validated: if ``lo`` is greater than
      ``hi``, the ordering of the scores is reversed.
"""

Example 3

Function:
import time

def call_with_retry(func, attempts=3, delay=1.0):
    if attempts < 1:
        raise ValueError("attempts must be at least 1")
    for i in range(attempts):
        try:
            return func()
        except Exception:
            if i == attempts - 1:
                raise
            time.sleep(delay)

Docstring:
"""
Call a function, retrying if it raises an exception.

Args:
    func (Callable[[], Any]): The function to call. It takes no arguments;
        use ``functools.partial`` or a lambda to supply them.
    attempts (int): The maximum number of calls to make. Defaults to 3.
    delay (float): Seconds to wait between attempts. Defaults to 1.0.

Returns:
    Any: Whatever ``func`` returns on its first successful call.

Raises:
    ValueError: If ``attempts`` is less than 1.
    Exception: The exception raised by the final attempt, if every
        attempt fails.

Edge cases:
    - Every ``Exception`` subclass is retried, including errors that can
      never succeed on retry (such as a bad argument), so those still wait
      ``delay`` seconds between attempts.
    - There is no wait after the final failed attempt; the last exception
      is re-raised immediately.
    - ``KeyboardInterrupt`` and ``SystemExit`` are not caught and stop the
      retry loop at once.
"""

Record:

• Your full prompt including examples (verbatim — include the examples)
• The full model output
• Analysis (2–3 sentences): what changed compared to zero-shot? Did format improve? Did the examples help with the edge case fields from your specification? Did anything get worse?

Technique 3: Chain-of-thought

Write a prompt that asks the model to reason explicitly before writing the docstring. Use the explicit CoT structure from the lecture: instruct the model to first describe what the function does, then identify the parameters, then identify the return value, then identify at least one non-obvious edge case, and then write the docstring.

Record:

• Your prompt (verbatim)
• The full model output including the reasoning steps
• Analysis (2–3 sentences): did the reasoning steps improve the docstring? Point to a specific place where the reasoning led to a better or worse result than zero-shot. Did the model identify the edge cases from your specification?

Technique 4: Self-correction

Generate an initial docstring using any prompt. Then write a critique prompt that evaluates the output against your specification using specific criteria — at least one criterion per field in your specification. Then run the revision.

Record:

• Your generation prompt (verbatim)
• The initial output
• Your critique prompt (verbatim) — must contain at least three specific criteria, each checkable as pass/fail
• The critique output
• The revised output
• Analysis (2–3 sentences): did the critique identify a genuine problem from your specification? Did the revision address it? Did the critique approve anything your specification would have failed?

Part 2 — Pair Discussion, Approved Docstring, and Exit Ticket (10 minutes)

Pair with the person next to you. Spend five minutes comparing your four docstrings and discussing, then five minutes picking one and writing one paragraph together.

Discussion: if you were the code steward reviewing a PR that added a docstring to this function, which of your docstrings would you approve? Which would you send back for revision, and with what comment?

Approved docstring (one per pair): pick the one docstring your pair would approve, from either partner's four. You may edit it before approving; if you do, note what you changed and why.

Shared paragraph (one per pair): 4–6 sentences explaining your choice. Which technique produced the docstring you would most trust in production, and why? Your reasoning must reference your specifications and must address cost — if this were a system generating thousands of docstrings, does your answer change?

Submission

One document per pair (PDF, Word, or plain text, or paste into the E3 text field), containing:

1. Both partners' names and student IDs, at the top
2. Each partner's specification (the five Step 0 fields), labelled with the partner's name
3. Each partner's four technique records: that partner's own prompt verbatim, full output, the docstring produced, and 2–3 sentence analysis for each technique
4. Your pair's approved docstring (edits noted) and the one shared exit-ticket paragraph explaining the choice

Graded on: completion. Both partners receive the credit, and each partner's section should show that partner's own work. We encourage you to write detailed analyses that reference your actual specification and actual outputs — the more specific you are about what changed and why, the more useful the lab will be for Assignment 1.

Notes

The function is intentionally non-trivial. process_records has several behaviours worth documenting. It deduplicates by id, keeping only the first occurrence. It uses 0 as the score when the score key is missing. Each returned item is a dictionary with id, score, and status. It raises no error for records without an id — instead it treats them all as having id None, so only the first such record is kept and the rest are silently dropped as duplicates. A good docstring addresses at least two of these.

Connection to Assignment 1: Assignment 1 Part 1 uses the same structure — five techniques on a code generation task (you add step-back), prompt verbatim, full output, analysis. Students who write specific, specification-driven analyses here write better Assignment 1 submissions. The habit of checking against a written specification rather than a general sense of quality is exactly what the assignment grades.

← Back to Calendar