# Data Scientist Interview Questions — Python

Source: Codeayan (https://codeayan.com)
Canonical: https://codeayan.com/get-hired/data-scientist/python
Questions: 53
Last updated: 2026-08-21
Licence: free to read. Please cite Codeayan when quoting.

---

## 1. Explain mutable versus immutable types. Why does the distinction matter the moment you pass something into a function?

*Easy · Very Common*

**Short answer.** Immutable objects such as int, str and tuple cannot be changed after creation; lists, dicts and sets can. Python passes references, so a function receiving a mutable object can modify the caller's data in place. Rebinding the parameter name affects nothing outside the function.

Two functions that look almost identical behave completely differently.

```
def add_topping(order, topping):
    order.append(topping)     # mutates caller's list

def replace_order(order, new_items):
    order = new_items         # rebinds local name only

pizza = ["cheese"]
add_topping(pizza, "olives")
print(pizza)                  # ['cheese', 'olives']

replace_order(pizza, ["paneer"])
print(pizza)                  # ['cheese', 'olives']  ← unchanged
```

The parameter name is a local label pointing at the same object the caller passed. `append` reaches through that label and changes the object itself, which the caller sees. Assignment moves the label to a different object and leaves the original alone.

An immutable object cannot be reached through in that way at all, which is why passing an int or a string is always safe. `s.upper()` returns a new string; it does not modify `s`.

The failure this causes in real code is quiet. A helper that normalises a config dictionary and mutates it in place looks fine in isolation, and then a caller that expected its own dictionary untouched starts seeing modified values three functions away. When a function modifies its argument, either name it so that is obvious, or copy the input at the top and return the new object.

One thing worth knowing for the follow-up: immutability is shallow. A tuple cannot be reassigned, but if it holds a list, that list is still mutable.

```
row = ("BLR", ["delayed"])
row[1].append("cancelled")    # works
```

That also means such a tuple is not hashable and cannot be used as a dictionary key.

**Likely follow-ups**

- If Python passes references, why doesn't reassigning the parameter inside the function change the caller's variable?
- Which built-in types are immutable, and is a tuple always safe to share?
- How would you write a function that clearly signals it modifies its argument?

---

## 2. Why is a plain Python loop over ten lakh numbers slow, and what do you do instead?

*Medium · Very Common*

**Short answer.** Every iteration of a Python loop costs interpreter overhead: bytecode dispatch, type checking and object boxing on each element. Vectorised libraries push the loop into compiled C over contiguous typed memory, so the same arithmetic runs one to two orders of magnitude faster.

The loop body is not the cost. The machinery around it is.

For each element, CPython dispatches bytecode, checks the operand types at runtime, allocates a new object for the result, and updates reference counts. A Python integer is not four bytes in a row; it is an object with a header, scattered on the heap. Ten lakh iterations pay all of that ten lakh times.

Compiled libraries store the same data as a contiguous block of typed memory and run the loop in C, checking the type once for the whole array rather than once per element. The arithmetic was never the bottleneck; the per-element bookkeeping was.

```
# per-element interpreter overhead
scaled = []
for r in rainfall_mm:
    scaled.append(r * 0.0393)

# one call, loop runs in C
scaled = rainfall_arr * 0.0393
```

A list comprehension is faster than the explicit loop, because it avoids repeated attribute lookup on `append` and has a tighter bytecode path, but it is still an interpreted loop over boxed objects. Do not present it as vectorisation.

Two things worth saying in an interview.

Vectorisation only applies when the operation is uniform across elements. Anything genuinely sequential, where element *n* depends on the result for *n-1*, cannot be expressed that way, and the answer is then Numba, Cython, or accepting the loop.

And the trade-off is memory. A vectorised operation typically allocates a full output array, so a chained expression over a large array can allocate several intermediates. A generator loop uses almost nothing. On data that fits comfortably, take the speed; near the memory ceiling, chunk it.

Below a few thousand elements, the difference is microseconds and not worth restructuring code for. Measure before rewriting.

**Likely follow-ups**

- If the operation can't be vectorised, what are your remaining options?
- Does a list comprehension count as vectorisation?
- At what data size does the loop actually stop mattering?

---

## 3. When would you use a tuple instead of a list? What does the immutability actually buy you?

*Easy · Very Common*

**Short answer.** Use a list for a homogeneous, growing collection and a tuple for a fixed record whose positions mean different things. Immutability buys you hashability, so tuples work as dictionary keys and set members, and it signals to a reader that the structure will not change.

The convention most teams follow is about meaning, not performance. A list holds many of the same thing and its length varies. A tuple holds a fixed set of fields where position carries meaning.

```
sensor_ids = ["T-01", "T-02", "T-03"]      # list: many of one thing
reading = ("T-01", 34.2, "2026-08-19")     # tuple: one record
```

The concrete capability immutability gives you is hashability. A tuple of immutable values can be a dictionary key or a set member; a list cannot.

```
daily_max = {}
daily_max[("T-01", "2026-08-19")] = 41.7   # fine

daily_max[["T-01", "2026-08-19"]] = 41.7
# TypeError: unhashable type: 'list'
```

That single property is why tuples turn up as composite keys in caches, lookup tables and grouping logic all over real codebases.

The performance argument is usually overstated. Tuples are marginally smaller and construct slightly faster, and unless you are creating crores of them in a hot loop, that difference will not be what makes your job slow. Do not lead with it in an interview; lead with intent and hashability.

Once a tuple has more than three fields, positional access becomes a liability. `reading[2]` tells a reader nothing. `NamedTuple` or a frozen dataclass keeps the immutability and adds names:

```
from typing import NamedTuple

class Reading(NamedTuple):
    sensor_id: str
    celsius: float
    taken_on: str
```

Still hashable, still immutable, and `r.celsius` survives someone reordering the fields.

**Likely follow-ups**

- Is a tuple always faster than a list, and does that difference matter in your code?
- What would you use instead of a tuple when the fields have names?
- Can you use a list as a dictionary key, and why not?

---

## 4. How do you read a value out of a dictionary when the key might not be there?

*Easy · Very Common*

**Short answer.** Square brackets raise KeyError on a missing key. `.get(key)` returns None instead, and `.get(key, default)` returns whatever default you supply. Use brackets when a missing key is a bug you want to hear about, and `.get()` when absence is expected and has a sensible fallback.

```
settings = {"retries": 3, "timeout": None}

settings["region"]              # KeyError: 'region'
settings.get("region")          # None
settings.get("region", "apac")  # 'apac'
```

Choosing between them is a decision about whether absence is normal. Reading a required configuration key with `.get()` and a silent default means a typo in the config file produces a working program with wrong behaviour, which is far harder to debug than a `KeyError` on startup. Reading an optional field from a parsed API response with brackets means one missing attribute crashes the whole batch.

The trap in that example is `timeout`. Both a missing key and a key set to `None` make `.get()` return `None`, so you cannot tell them apart:

```
settings.get("timeout")        # None — but the key exists
settings.get("nonsense")       # None — key doesn't exist

"timeout" in settings          # True
```

When the difference matters, test with `in` rather than inferring from the returned value. This bites when `None` is a legitimate stored value meaning “no limit” and your code treats it as “not configured” and substitutes a default.

One more thing people get wrong: `.get()` never inserts. If you want to read-or-create in one step, that is `.setdefault()`.

```
by_city = {}
by_city.setdefault("Pune", []).append("order-118")
```

For accumulating into a dict repeatedly, `collections.defaultdict(list)` reads better and avoids repeating the default on every line.

**Likely follow-ups**

- What's the difference between .get() returning None and the key existing with a value of None?
- When would you reach for defaultdict instead?
- What does .setdefault() do that .get() doesn't?

---

## 5. You copied a list of lists and changing the copy also changed the original. Explain what happened and how you'd fix it.

*Medium · Very Common*

**Short answer.** A shallow copy creates a new outer container holding references to the same inner objects, so mutating an inner object is visible through both copies. `copy.deepcopy` recursively copies everything, giving full independence at the cost of time and memory.

```
import copy

template = [["mon", 0], ["tue", 0]]

shallow = copy.copy(template)      # or template[:] or list(template)
shallow[0][1] = 9
print(template)     # [['mon', 9], ['tue', 0]]  ← changed

deep = copy.deepcopy(template)
deep[1][1] = 5
print(template)     # [['mon', 9], ['tue', 0]]  ← safe
```

The shallow copy did create a new outer list. What it did not do is create new inner lists, so `shallow[0]` and `template[0]` are the same object, and mutating through either label is visible through both.

This turns up most often with a default structure reused across iterations. A roster template copied per employee, where every employee ends up sharing one shift list, produces a bug where updating one person’s schedule silently updates everyone’s. It looks like a database problem until you find the copy.

Deep copy is not free. It walks the entire object graph and reconstructs it, which on a large nested structure inside a loop is genuinely slow. Two alternatives worth mentioning. Rebuild the structure explicitly with a comprehension when you know its shape:

```
fresh = [[day, count] for day, count in template]
```

Or avoid the problem by making the inner elements immutable, since a tuple cannot be mutated through a shared reference at all.

Two details for the follow-ups. `deepcopy` handles cyclic references correctly by keeping a memo of objects it has already copied, so it does not recurse forever. And a class can control what happens by defining `__deepcopy__`, which matters for objects holding a database connection or a file handle that should not be duplicated.

**Likely follow-ups**

- What does deepcopy do when the structure contains a cycle?
- Would this problem arise if the inner elements were tuples of strings?
- How expensive is deepcopy on a large nested structure, and what would you do instead?

---

## 6. Explain the GIL. What does it stop you doing, and what does it not?

*Hard · Very Common*

**Short answer.** CPython's global interpreter lock allows only one thread to execute Python bytecode at a time, so threads give no speedup on CPU-bound work. It is released during I/O waits and inside many C extensions, which is why threads still help for network and disk work.

The lock protects CPython’s internal state, principally reference counts. Without it, two threads incrementing the same object’s refcount concurrently would corrupt it, and making every refcount operation atomic would slow down single-threaded code, which is the overwhelming majority of Python code.

The consequence is blunt. Four threads doing arithmetic on four cores do not run four times faster; they take turns holding the lock and finish in roughly the time one thread would take, plus switching overhead.

Three things people get wrong.

**It is not the language.** It is a CPython implementation detail. Jython and IronPython have no GIL, and PyPy has one for the same reasons CPython does. Saying “Python has a GIL” is loose; saying “CPython has one” is correct and interviewers notice.

**It is released during I/O.** A thread waiting on a socket or a disk read gives up the lock, so other threads run. That is why threading genuinely helps for a hundred concurrent API calls despite the GIL.

**C extensions can release it.** Long-running numeric operations in numpy, scipy and similar libraries release the GIL around the compiled loop, so those genuinely use multiple cores while Python-level code cannot. A workload that spends its time inside such calls is much less GIL-bound than it first appears.

The escape routes for CPU-bound Python are processes, which each have their own interpreter and lock, or pushing the hot loop into compiled code through Cython or Numba.

One current detail worth stating precisely: Python 3.13 introduced an optional free-threaded build with the GIL disabled, following PEP 703. It is opt-in, not the default interpreter, carries a single-threaded performance cost, and the ecosystem is still catching up. Mentioning it accurately, with those caveats, reads better than either ignoring it or claiming the GIL is gone.

**Likely follow-ups**

- If the GIL exists, why do threads still help with a hundred HTTP calls?
- Where does numpy fit into this?
- Is the GIL part of the Python language or of CPython?

---

## 7. Explain `*args` and `**kwargs`. What are they doing on the way in, and on the way out?

*Easy · Very Common*

**Short answer.** In a function signature, `*args` collects extra positional arguments into a tuple and `**kwargs` collects extra keyword arguments into a dict. At a call site, the same symbols do the reverse: they unpack a sequence into positional arguments and a mapping into keyword arguments.

Same syntax, two directions. In the definition it packs; at the call it unpacks.

```
def log_event(event, *tags, **fields):
    print(event, tags, fields)

log_event("upload", "batch", "s3", size=940, retries=2)
# upload ('batch', 's3') {'size': 940, 'retries': 2}

payload = {"size": 940, "retries": 2}
log_event("upload", "batch", **payload)   # unpacking
```

The names are convention only; `*a` and `**kw` work identically. Stick with `args` and `kwargs` because everyone reading your code expects them.

Where these earn their place is a wrapper that must forward whatever it receives without knowing the signature it is wrapping. A retry helper, a timing wrapper, a decorator: all of them accept `*args, **kwargs` and pass them straight through, so they work with any function.

```
def with_retry(fn, *args, **kwargs):
    for attempt in range(3):
        try:
            return fn(*args, **kwargs)
        except TimeoutError:
            continue
    raise
```

Two things worth knowing beyond the basics.

A bare `*` forces everything after it to be keyword-only, which is a good way to stop callers passing a run of unlabelled booleans:

```
def export(path, *, overwrite=False, compress=True):
    ...
export("out.csv", overwrite=True)   # positional not allowed
```

And the cost of `**kwargs` is that it destroys the signature. Callers get no autocomplete, typos become silent extra dict entries rather than errors, and type checkers cannot help. Use it where forwarding is genuinely the point, not to avoid deciding what parameters a function takes.

**Likely follow-ups**

- What does a bare `*` in a signature mean?
- If a caller passes the same argument positionally and by keyword, what happens?
- Why does a decorator almost always accept both of these?

---

## 8. What's wrong with writing `def process(records, seen=[])`? Show me what actually happens.

*Medium · Very Common*

**Short answer.** The default is evaluated once, when the function is defined, not on each call. A default list or dict is therefore shared by every call that omits the argument, and mutations accumulate across calls. Use None as the default and create the container inside the function body.

```
def collect(row, batch=[]):
    batch.append(row)
    return batch

collect("r1")    # ['r1']
collect("r2")    # ['r1', 'r2']  ← not ['r2']
collect("r3")    # ['r1', 'r2', 'r3']
```

The list is created once, at `def` time, and stored on the function object itself. You can see it directly:

```
collect.__defaults__      # (['r1', 'r2', 'r3'],)
```

Every call that omits the argument gets that same list, so state leaks between calls that have nothing to do with each other.

The correct form is a sentinel:

```
def collect(row, batch=None):
    if batch is None:
        batch = []
    batch.append(row)
    return batch
```

`is None` rather than `if not batch`, because an empty list passed deliberately by the caller is falsy and would be silently replaced.

In production this rarely announces itself. A parsing helper with `errors={}` accumulates every error from every file processed since the service started, and memory grows steadily while each individual call looks correct. In a long-running worker that is a slow leak nobody attributes to the function.

The same evaluate-once rule catches other defaults. `def report(as_of=datetime.now())` freezes the timestamp at import time, so a service running for a week reports the same date forever. Pass `None` and compute inside.

Immutable defaults are safe, since there is nothing to mutate: `def f(n=0, label="")` behaves as everyone expects.

There is one deliberate use of the behaviour, worth knowing but not worth writing: attaching a mutable default as a cheap per-function cache. `functools.lru_cache` does the same job without surprising the next reader.

**Likely follow-ups**

- Is `def f(x=[])` ever the behaviour you actually want?
- What would you use as a default for a dict parameter?
- Does the same problem apply to a default of datetime.now()?

---

## 9. Sort these records by department and then by joining date, newest first. Why does the order you apply the sorts matter?

*Easy · Very Common*

**Short answer.** Pass a function to `key=` to extract what you want to sort on, and `reverse=True` to flip the order. Python's sort is stable, meaning equal elements keep their existing relative order, so you can sort by a secondary field first and then by the primary field.

```
staff = [
    {"name": "Riya", "dept": "ops", "joined": "2021-04-01"},
    {"name": "Arun", "dept": "eng", "joined": "2023-07-15"},
    {"name": "Meera", "dept": "ops", "joined": "2019-01-09"},
]

by_dept = sorted(staff, key=lambda r: (r["dept"], r["joined"]))
```

A tuple key sorts by the first element and uses the second to break ties, which handles most multi-field cases in one call.

Mixed directions are where stability earns its keep. You cannot put `reverse=True` on just one element of a tuple key, so you sort twice, least significant first:

```
rows = sorted(staff, key=lambda r: r["joined"], reverse=True)
rows = sorted(rows, key=lambda r: r["dept"])
```

Because Python’s sort is stable, the second sort preserves the date ordering within each department. Reverse the order of those two statements and the result is wrong. For numeric fields there is a shortcut: negate the value in the key, so `key=lambda r: (r["dept"], -r["salary"])`.

`operator.itemgetter` is often faster and clearer than a lambda for straightforward field access:

```
from operator import itemgetter
sorted(staff, key=itemgetter("dept", "joined"))
```

Two things to keep straight. `sorted()` returns a new list and works on any iterable; `.sort()` mutates a list in place and returns `None`. Writing `rows = rows.sort()` silently sets `rows` to `None`, and the failure surfaces later as a confusing `TypeError`.

And the key function is called exactly once per element, then the results are compared. So an expensive key such as a date parse costs n calls rather than n log n, which is the reason `key=` replaced the old comparison-function style.

**Likely follow-ups**

- How would you sort ascending on one field and descending on another in a single pass?
- What's the difference between `sorted()` and `.sort()`?
- Why is `key=` preferred over the old `cmp` style?

---

## 10. Write me a comprehension that builds a dict from these records. Then show me a nested one and tell me where you'd stop.

*Easy · Very Common*

**Short answer.** The same syntax builds lists, dicts and sets: brackets give a list, braces with a colon give a dict, braces without give a set. Nested loops read in the same order you would write them as statements. Beyond two loops or one condition, a plain for loop is easier to follow.

```
books = [("Ghachar Ghochar", 2013), ("Em and the Big Hoom", 2012)]

titles  = [t for t, _ in books]                # list
by_year = {t: y for t, y in books}             # dict
years   = {y for _, y in books}                # set
```

Adding a condition filters, and a conditional expression before the `for` transforms:

```
recent = [t for t, y in books if y >= 2013]
tags = ["new" if y >= 2013 else "old" for _, y in books]
```

Those two `if`s sit in different positions and do different jobs. Filtering goes after the `for`; choosing between two values goes before it. Getting them confused is a common syntax error.

Nesting reads in statement order, outer loop first:

```
shelves = [["a1", "a2"], ["b1"]]
flat = [code for shelf in shelves for code in shelf]
```

Read it as `for shelf in shelves:` then `for code in shelf:`, which is exactly the order you would write the loops. People frequently expect the reverse and get an error about an undefined name.

Where to stop is a judgement worth voicing. Two loops with a condition is around the limit before a comprehension stops being readable, and a reader having to decode it costs more than the two extra lines a loop would take. There is no rule here, only the next person reading it.

One memory point. Brackets build the entire list in memory before returning. Parentheses produce a generator expression instead, which yields lazily and is what you want when feeding a large sequence into `sum` or a loop:

```
total = sum(y for _, y in books)     # no intermediate list
```

Comprehensions have their own scope from Python 3 onwards, so the loop variable does not leak out.

**Likely follow-ups**

- In a nested comprehension, which for loop is the outer one?
- What does a comprehension with parentheses instead of brackets give you?
- Does the loop variable leak into the surrounding scope?

---

## 11. What is a generator, and what do you give up by using one instead of returning a list?

*Medium · Very Common*

**Short answer.** A function containing `yield` returns a generator: calling it runs no code, and each iteration executes until the next yield and pauses there, keeping local state. You gain constant memory over huge sequences. You give up length, indexing, and the ability to iterate more than once.

```
def parse_log(path):
    with open(path) as f:
        for line in f:
            if line.startswith("ERROR"):
                yield line.strip()

for entry in parse_log("app.log"):   # one line in memory
    handle(entry)
```

The list version builds every matching line before the caller sees the first one. On a 40 GB log file that is the difference between a job that runs and one that gets killed by the memory limit.

Calling `parse_log("app.log")` executes nothing. It returns a generator object. The body runs only when something iterates it, advancing to the next `yield` and freezing there with all locals intact.

What you give up matters as much as what you gain.

```
gen = parse_log("app.log")
len(gen)          # TypeError
gen[0]            # TypeError
list(gen)         # works, but defeats the purpose

for x in gen: ... # consumes it
for x in gen: ... # yields nothing, silently
```

That last one is the failure that reaches production. A generator passed to two functions works for the first and gives the second an empty sequence, with no error and no warning. The downstream count comes out as zero and someone spends an afternoon on it. If two passes are needed, materialise once with `list()` and accept the memory cost, or call the generator function again.

One subtlety with the file example: the `with` block stays open while the generator is alive, and if the caller abandons it partway, the file is closed when the generator is garbage collected rather than at a predictable point.

`yield from` delegates to another iterable, which flattens nested generators without an explicit loop and correctly forwards values sent into the generator.

**Likely follow-ups**

- What happens if you iterate the same generator twice?
- How would you get the length of what a generator will produce?
- What does `yield from` do that a loop with yield doesn't?

---

## 12. Walk me through try, except, else and finally. What goes in each one, and why not put everything in the try?

*Medium · Very Common*

**Short answer.** `try` holds the code that might fail, `except` handles a specific failure, `else` runs only if no exception occurred, and `finally` runs either way for cleanup. Keeping the try block narrow matters, because a wide one catches errors from lines you never intended to protect.

```
try:
    conn = connect(dsn)
except ConnectionError as exc:
    log.error("could not reach vendor API: %s", exc)
    raise
else:
    payload = conn.fetch_invoices()   # only if connect worked
finally:
    conn.close() if conn else None
```

The `else` block is the part most people never use, and it is what keeps the `try` narrow. Put `fetch_invoices()` inside the `try` and a `ConnectionError` raised deep inside the fetch gets caught by a handler written for the connection step, and your log says the wrong thing about the wrong operation.

`finally` runs whether or not an exception occurred, and whether or not the block returned. That makes it the right place for releasing a lock, closing a handle, or restoring a setting. In most cases a `with` block expresses the same intent more clearly, since the context manager owns the cleanup.

Three things that separate a working answer from a careful one.

**Catch narrowly.** A bare `except:` catches `KeyboardInterrupt` and `SystemExit` as well, so your job cannot be stopped cleanly. `except Exception` is better and still too broad as a default, since a typo raising `NameError` is swallowed and reported as a vendor failure.

**Never swallow silently.** `except Exception: pass` is how a nightly pipeline reports success while writing nothing. If you catch it, either handle it meaningfully or log it and re-raise.

**Preserve the chain.** When you wrap an error in your own exception type, `raise LoadError("vendor feed failed") from exc` keeps the original traceback attached. Without `from`, the underlying cause is harder to find in the logs.

One sharp edge worth knowing: a `return` inside `finally` discards any exception currently propagating, so an error disappears with no trace.

**Likely follow-ups**

- What's wrong with `except Exception` as your only handler?
- If finally contains a return, what happens to an exception in flight?
- When would you use `raise ... from` rather than a bare raise?

---

## 13. What does `with` actually guarantee, and how would you write a context manager yourself?

*Medium · Very Common*

**Short answer.** `with` guarantees that setup runs before the block and cleanup runs after it, whether the block finishes normally, returns, or raises. A context manager is any object with `__enter__` and `__exit__`; the `__exit__` method receives the exception details if one occurred.

The guarantee is the point. A `try`/`finally` gives you the same thing, and `with` moves the cleanup logic into the object that owns the resource, so every caller gets it right without remembering.

```
class Stopwatch:
    def __init__(self, label):
        self.label = label

    def __enter__(self):
        self.start = time.perf_counter()
        return self          # bound to `as`

    def __exit__(self, exc_type, exc, tb):
        elapsed = time.perf_counter() - self.start
        log.info("%s took %.2fs", self.label, elapsed)
        return False         # do not suppress

with Stopwatch("model export") as sw:
    export_weights()
```

`__exit__` receives three arguments describing any exception that escaped the block, all `None` if the block finished cleanly. Returning a truthy value from `__exit__` suppresses that exception, which is occasionally what you want and much more often a mistake. Return `False`, or nothing at all, unless you deliberately mean to swallow it.

Whatever `__enter__` returns is what `as` binds. Returning `self` is the common choice; returning `None` is fine when the block does not need a handle.

For most cases the generator form is shorter and reads better:

```
from contextlib import contextmanager

@contextmanager
def temp_setting(obj, **overrides):
    saved = {k: getattr(obj, k) for k in overrides}
    for k, v in overrides.items():
        setattr(obj, k, v)
    try:
        yield obj
    finally:
        for k, v in saved.items():
            setattr(obj, k, v)
```

The `try`/`finally` around the `yield` is essential. Without it, an exception in the body propagates out of the `yield` and the restoration never runs, which is exactly the guarantee you were trying to provide.

One gotcha: opening several resources with a single `with` and commas is fine, but if the second one fails during setup, the first is still cleaned up correctly. Nesting manually with plain assignments does not give you that.

**Likely follow-ups**

- If the body raises, does your cleanup still run, and how does the manager know?
- How would you write one so that it suppresses a specific exception?
- What does `contextlib.contextmanager` do to a generator?

---

## 14. Explain `self`, and what the difference is between an attribute set in `__init__` and one set on the class body.

*Easy · Very Common*

**Short answer.** `self` is the instance, passed automatically as the first argument to every method. An attribute assigned in `__init__` belongs to that instance alone; one assigned in the class body is shared by every instance, and Python falls back to it when the instance has no attribute of that name.

```
class Turbine:
    site = "Jaisalmer"      # class attribute, shared
    log = []                # class attribute, shared and mutable

    def __init__(self, tag):
        self.tag = tag      # instance attribute
```

Lookup goes instance first, then class. So `t.site` finds nothing on the instance and falls back to the class, and every turbine reports the same site. That is useful for genuine constants and defaults.

Assignment does not follow the same path. Writing `t.site = "Bhuj"` creates an instance attribute that shadows the class one for that object only; the class attribute is untouched and every other instance still sees Jaisalmer. Reading falls back, writing does not.

The mutable class attribute is the real trap:

```
a, b = Turbine("T1"), Turbine("T2")
a.log.append("fault")
b.log              # ['fault']
```

`a.log.append(...)` never assigns, so it never creates an instance attribute. It reaches through to the shared list and mutates it. Two objects that should be independent now share state, and in a long-running service that list also grows without bound. Anything mutable belongs in `__init__` as `self.log = []`.

`self` is not a keyword, just a convention. Python passes the instance as the first positional argument to any method called on it, so `t.spin()` is `Turbine.spin(t)`. Rename it and the code works and every reviewer will object.

One detail for the follow-up: `__init__` initialises an object that already exists and must return `None`. The object is created by `__new__`, which you rarely touch outside of immutable subclasses and singletons.

**Likely follow-ups**

- What happens when you assign to an attribute that only exists on the class?
- Why is a mutable class attribute a problem?
- What does `__init__` return, and what actually creates the object?

---

## 15. Why does every Python project tell you to create a virtual environment first?

*Easy · Very Common*

**Short answer.** A virtual environment gives each project its own isolated site-packages directory and its own installed versions. Without one, every project shares the system interpreter's packages, so upgrading a library for one project silently changes behaviour in every other project on the machine.

The problem is a single shared namespace. One project needs an older release of a plotting library that a colleague’s project cannot use, and there is exactly one place to install it.

```
python -m venv .venv
source .venv/bin/activate      # Windows: .venvScriptsactivate
pip install -r requirements.txt
```

`venv` is in the standard library from Python 3.3 onwards, so nothing needs installing first. Activating it puts that environment’s `python` and `pip` at the front of your PATH, so installs land inside the project directory rather than in the system interpreter.

Two failures make this concrete.

Installing into the system Python on Linux can break OS tooling that depends on specific package versions. Recent versions of Debian and Ubuntu now refuse a bare `pip install` outside a virtual environment for exactly this reason, and the error message confuses people who have never used one.

The subtler one is drift. Code runs on your machine and fails in the pipeline, because your machine picked up a library version installed eighteen months ago for something else. Nobody can reproduce it and nobody can explain it. A per-project environment plus a pinned requirements file removes the whole category.

Pin what you install. `pip freeze > requirements.txt` captures exact versions including transitive dependencies, which is what you want for reproducing a deployment. A hand-written file listing only your direct dependencies with ranges is better for a library others will install. Tools like Poetry and uv manage both sides with a lock file, and are worth naming if the interviewer asks what you use.

One practical note: keep `.venv/` out of version control and commit the requirements file instead. The environment is derived; the specification is the artefact.

**Likely follow-ups**

- Two projects need different versions of the same library — what does that look like without a venv?
- What's the difference between `pip freeze` and a requirements file you write by hand?
- Where does a container fit in if you already have virtual environments?

---

## 16. Write me a decorator from scratch and explain what the `@` symbol is actually doing.

*Hard · Very Common*

**Short answer.** `@decorator` above a function is shorthand for reassigning the name: the function is passed to the decorator and the name is rebound to whatever comes back. A decorator is any callable that takes a function and returns a replacement, usually a wrapper closing over the original.

The syntax hides an assignment. These two are identical:

```
@audit
def issue_refund(ticket_id, amount):
    ...

def issue_refund(ticket_id, amount):
    ...
issue_refund = audit(issue_refund)
```

That second form is the whole concept. `audit` receives the original function object and returns something else, and the name now points at the replacement.

Writing one:

```
import functools

def audit(fn):
    @functools.wraps(fn)
    def wrapper(*args, **kwargs):
        log.info("calling %s", fn.__name__)
        result = fn(*args, **kwargs)
        log.info("%s returned", fn.__name__)
        return result
    return wrapper
```

Three details carry the weight. `*args, **kwargs` means the wrapper forwards anything, so one decorator works on functions with any signature. `return result` is easy to omit and turns every decorated function into one that returns None, which is a bug that passes import and fails at runtime. And `functools.wraps` copies the original’s `__name__`, `__doc__` and `__wrapped__` onto the wrapper, without which tracebacks, help output and anything doing introspection all report `wrapper`.

A decorator taking arguments needs one more layer, because `@retry(times=3)` calls `retry(times=3)` first and applies the result as the decorator:

```
def retry(times):
    def decorator(fn):
        @functools.wraps(fn)
        def wrapper(*args, **kwargs):
            for _ in range(times - 1):
                try:
                    return fn(*args, **kwargs)
                except TimeoutError:
                    continue
            return fn(*args, **kwargs)
        return wrapper
    return decorator
```

Three nested functions is the standard shape and the part people fumble under pressure. Practise writing it.

Two things worth saying. Decorators run at import time, when the `def` is executed, so an expensive decorator body costs you on every import. And methods work unchanged: `self` arrives as the first element of `args`, since by then it is an ordinary positional argument.

**Likely follow-ups**

- Your decorated function's `__name__` now says `wrapper` — how do you fix that?
- How would you write a decorator that takes an argument, like `@retry(times=3)`?
- Can you decorate a method the same way, and what happens to `self`?

---

## 17. You've written two versions of a function and want to know which is faster. How do you measure it properly?

*Medium · Common*

**Short answer.** Use `timeit`, which runs the code many times and disables garbage collection during the run. A single `time.time()` measurement is dominated by noise from other processes, cache state and the interpreter warming up. Benchmark on realistic input sizes, not toy ones.

```
import timeit

setup = "from ranking import score_v1, score_v2, sample"
t1 = timeit.timeit("score_v1(sample)", setup=setup, number=200)
t2 = timeit.timeit("score_v2(sample)", setup=setup, number=200)
print(f"v1 {t1:.3f}s  v2 {t2:.3f}s  ratio {t1/t2:.2f}x")
```

In a notebook, `%timeit` does the same thing with automatic repeat selection and reports the spread.

Three things make a benchmark honest.

**Repeat enough.** A single run of a fast function measures scheduler noise more than your code. `timeit` loops and reports the total, which averages the jitter out.

**Report the minimum across repeats, not the mean.** External interference can only ever make a run slower, never faster, so the fastest observed run is the closest estimate of the true cost. `timeit.repeat` gives you several totals to take the minimum from.

**Keep setup out of the measurement.** Building the input inside the timed statement measures the construction, and if both versions build it, the real difference gets diluted into noise.

The failure that produces wrong conclusions: measuring on a 100-element list. Fixed costs such as function call overhead dominate at that size, and the version that wins there frequently loses at ten lakh elements where allocation behaviour matters. Benchmark at the size you actually run.

Cache effects also mislead. If the first version populated a cache or warmed the CPU’s data cache, the second looks faster for reasons unrelated to the code. Run them in both orders and check the ordering does not change the answer.

For I/O-bound work, `timeit` is the wrong tool entirely, since you would be measuring the network. Profile the wall-clock time across many runs and look at the distribution, because the tail matters more than the average there.

**Likely follow-ups**

- Why does timeit report the minimum rather than the mean?
- Your second version is faster only because it ran second — how would that happen?
- What would you measure instead if the function is I/O bound?

---

## 18. You need to remove duplicates from a large collection and then check membership repeatedly. Why a set, and is the O(1) lookup claim really true?

*Easy · Common*

**Short answer.** A set stores elements in a hash table, so membership testing is average O(1) regardless of size, while the same test on a list is O(n). Deduplication is a single constructor call. The costs are that sets hold only hashable objects and do not preserve a meaningful order.

The difference shows up as soon as the collection grows.

```
blocked = set(load_blocked_pincodes())    # 90,000 entries

if pincode in blocked:                    # average O(1)
    reject(pincode)
```

With a list, `in` walks the elements one by one until it finds a match, so checking one lakh incoming pincodes against a 90,000-entry list is roughly nine billion comparisons. Against a set it is one lakh hash lookups.

Set operations replace loops you would otherwise write by hand:

```
served = {"560001", "560034", "560076"}
requested = {"560034", "411001"}

requested & served      # {'560034'}   both
requested - served      # {'411001'}   requested but not served
requested | served      # union
requested ^ served      # in one but not both
```

Three caveats worth raising before the interviewer does.

The O(1) is average case. Hash collisions degrade lookups, and in pathological cases where many elements hash to the same bucket, behaviour approaches linear. For ordinary strings and integers this is not something you will meet, but the honest phrasing is “average O(1)”, not “always O(1)”.

Only hashable objects can go in. A set of lists raises `TypeError`; a set of tuples is fine.

Order is not preserved in any way you should rely on. `set(names)` deduplicates and returns the items in hash order, which will surprise anyone expecting the original sequence. If order matters, `list(dict.fromkeys(names))` deduplicates while keeping first occurrence, since dicts preserve insertion order from Python 3.7 onwards.

**Likely follow-ups**

- What happens to the ordering of your data when you pass it through a set?
- How would you deduplicate while keeping the first occurrence of each item?
- Under what circumstances does that O(1) lookup degrade?

---

## 19. Build me a preprocessing pipeline that streams records through several transformation steps. What does that buy you?

*Hard · Common*

**Short answer.** Write each stage as a generator taking an iterable and yielding transformed items. Chaining them means one record moves through the whole chain at a time, so memory stays flat and each stage is independently testable with a small list.

```
def load(paths):
    for p in paths:
        with open(p) as f:
            for line in f:
                yield json.loads(line)

def drop_incomplete(docs):
    for d in docs:
        if d.get("abstract") and d.get("year"):
            yield d

def tokenise(docs):
    for d in docs:
        d["tokens"] = d["abstract"].lower().split()
        yield d

def to_features(docs, vocab):
    for d in docs:
        yield {t: d["tokens"].count(t) for t in vocab}

pipeline = to_features(tokenise(drop_incomplete(load(paths))), vocab)
```

Constructing `pipeline` runs nothing. Only when something iterates it does one record get pulled through all four stages, so peak memory is one record plus the vocabulary.

Three properties make this worth the structure.

Each stage is a function over an iterable, so testing it needs no files: `list(drop_incomplete([{"abstract": "x"}]))`. That is far easier than testing a single function that does all four things.

Stages compose in any order and a new one slots in without touching the others.

And filtering early is free. `drop_incomplete` sitting before `tokenise` means discarded records never get tokenised, and the saving is automatic rather than something you had to arrange.

Two things that break the model.

Calling `list()` anywhere in the chain materialises everything, and you are back to holding the dataset. That includes an innocent-looking `len()`, which forces the same thing.

And any stage needing global knowledge cannot be lazy. Building the vocabulary, computing a mean for standardisation, or sorting all require a full pass. The usual answer is two passes: one to compute the statistic, one to stream the transformation using it. Fitting a scaler on the streaming pass would use future records to transform earlier ones, which is leakage.

For counting drops, keep a counter in an outer scope and increment inside the stage, since generator locals survive across yields.

**Likely follow-ups**

- One stage needs to see the whole dataset — where does that break the design?
- How would you count how many records each stage dropped?
- Where would you put a progress indicator in this chain?

---

## 20. Walk me through slicing. What does `data[::-1]` do, and does slicing give me the original object or a copy?

*Easy · Common*

**Short answer.** `sequence[start:stop:step]` takes from start up to but not including stop. Negative indices count from the end, and a negative step walks backwards, so `[::-1]` reverses. Slicing a list returns a new list, so modifying the slice does not affect the original.

```
lanes = ["A", "B", "C", "D", "E", "F"]

lanes[1:4]      # ['B', 'C', 'D']   stop is exclusive
lanes[:3]       # ['A', 'B', 'C']
lanes[-2:]      # ['E', 'F']        last two
lanes[::2]      # ['A', 'C', 'E']   every second
lanes[::-1]     # ['F', 'E', ...]   reversed
```

The exclusive stop is what makes `lanes[:3] + lanes[3:]` reconstruct the original with no overlap and no gap, which is the reason for the convention.

Slicing produces a new list. That is why `copy = original[:]` is a common idiom for making a copy:

```
toll_lanes = lanes[:]
toll_lanes.append("G")
print(len(lanes))     # 6 — unaffected
```

The copy is shallow, though. The new list is independent, and the objects inside it are the same objects. With a list of lists, mutating an inner list is visible through both.

Slices behave differently from indexes at the boundaries, which is a genuine gotcha:

```
lanes[10]       # IndexError
lanes[3:99]     # ['D', 'E', 'F'] — no error
lanes[8:12]     # [] — no error
```

An out-of-range slice clamps silently. That is convenient in a paging loop and dangerous when it hides an off-by-one, since a wrong index gives you an empty list rather than an exception.

Two practical notes. `[::-1]` reverses a copy while `list.reverse()` reverses in place, so pick by whether you need the original. And on a very large list, slicing copies every element, so taking `big[:-1]` inside a loop is quietly quadratic; `itertools.islice` iterates without copying.

**Likely follow-ups**

- Your slice copy still shares the inner objects — when does that matter?
- Why does an out-of-range slice not raise IndexError when an out-of-range index does?
- How would you slice every third element starting from the end?

---

## 21. A helper function is called repeatedly with the same arguments and takes two seconds each time. What would you do?

*Medium · Common*

**Short answer.** Decorate it with `functools.lru_cache`, which stores results keyed on the arguments and returns the stored value on a repeat call. The arguments must be hashable, the function must be pure, and an unbounded cache grows without limit in a long-running process.

```
from functools import lru_cache

@lru_cache(maxsize=512)
def embed_phrase(phrase: str) -> tuple[float, ...]:
    return tuple(model.encode(phrase))

embed_phrase.cache_info()
# CacheInfo(hits=8140, misses=512, maxsize=512, currsize=512)
```

`cache_info()` is the first thing to check after adding one. A hit rate near zero means the arguments vary more than you assumed and you are paying the caching overhead for nothing.

Three conditions have to hold.

**Arguments must be hashable.** A list or a dict raises `TypeError: unhashable type`. Convert to a tuple or a frozenset at the call site, which is why the return above is a tuple rather than a list.

**The function must be pure.** Same input, same output, no side effects. Caching a function that reads a file or the current time returns the first result forever, and the bug looks like stale data with no obvious cause.

**The return value must be treated as immutable.** The cache stores the object, not a copy, so if the caller mutates a returned list, every subsequent cache hit gets the mutated version. Returning tuples avoids the whole problem.

`maxsize` is the setting that matters in production. `lru_cache(maxsize=None)`, and its shorthand `functools.cache` from Python 3.9, never evict, so a service caching on a wide key space grows until it is killed. Bound it unless the key space is genuinely small and fixed.

Two extras. The cache is per process, so eight worker processes hold eight separate caches with no sharing. And decorating a method caches on `self` too, which keeps the instance alive for as long as the entry exists; that is a real memory leak in a long-lived process, and `cached_property` or an explicit instance-level cache is the safer pattern there.

**Likely follow-ups**

- What happens if you decorate a function that takes a list?
- Why does maxsize=None worry you in a long-running process?
- How would you check whether the cache is actually being hit?

---

## 22. Show me where you'd reach for itertools rather than writing the loop yourself.

*Medium · Common*

**Short answer.** `itertools` provides lazy building blocks: `chain` to join iterables, `islice` to take a slice without materialising, `product` and `combinations` for parameter sweeps, and `groupby` for consecutive runs. Everything returns an iterator, so it is consumed once and holds nothing in memory.

The functions that come up most in analysis work:

```
from itertools import chain, islice, product, combinations

# treat several files as one stream
records = chain(load(a), load(b), load(c))

# first 500 without building the rest
head = list(islice(records, 500))

# every hyperparameter combination
grid = product([0.01, 0.1], [16, 64, 256], ["adam", "sgd"])
for lr, batch, opt in grid:
    run_trial(lr, batch, opt)

# every unordered pair of features
for a, b in combinations(feature_names, 2):
    check_correlation(a, b)
```

`product` replaces three nested loops and reads better. `combinations` gives unordered pairs without repetition, so `(a, b)` appears and `(b, a)` does not, which is what you want for a symmetric measure like correlation. `permutations` gives both orders and is what you want when order carries meaning.

Two things about laziness.

`chain` does not concatenate. It yields from each source in turn, so joining a hundred large files costs nothing in memory.

`islice` slices any iterable, including an infinite generator, and unlike `[:500]` it does not require a sequence. It also cannot take negative indices, since it never knows where the end is.

The gotcha everyone meets once: these return iterators, consumed after a single pass.

```
grid = product([1, 2], [3, 4])
len(list(grid))    # 4
len(list(grid))    # 0  ← exhausted
```

A parameter grid iterated in a loop and then printed for the log comes out empty, with no error. Wrap in `list()` if you need it twice.

Two more worth naming. `itertools.groupby` groups only consecutive equal keys, so sort first. And `count` with `islice` generates a bounded run from an unbounded source, which is the usual way to sample from something infinite.

**Likely follow-ups**

- What's the difference between combinations and permutations here?
- Your itertools object gave results the first time and nothing the second — why?
- How would you take just the first thousand items from an infinite generator?

---

## 23. What's the difference between `is` and `==`? And why does comparing small numbers with `is` seem to work?

*Medium · Common*

**Short answer.** `==` asks whether two objects have equal value; `is` asks whether they are the same object in memory. They give the same answer for small integers and short strings because CPython caches those objects, which misleads people into using `is` for value comparison. Use `is` only for None and other singletons.

```
a = 256
b = 256
a == b      # True
a is b      # True

x = 1000
y = 1000
x == y      # True
x is y      # False
```

Nothing changed about the language between those two blocks. CPython pre-creates integer objects from -5 to 256 and reuses them, so both names point at the identical cached object. At 1000 there is no cache, so two separate objects exist with equal value.

Short strings behave similarly through interning, and the rules for when interning happens vary by how the string was created and by CPython version. Both behaviours are implementation details of CPython, not guarantees of Python, and code that depends on them can break on another interpreter or another release.

The rule that follows: use `==` for value, and `is` only where identity is genuinely the question. In practice that means the singletons.

```
if config is None:          # correct
if config == None:          # works, but non-idiomatic
```

`is None` is preferred for a real reason beyond style. `==` calls the left operand’s `__eq__`, and a class can define that method to return anything it likes, including True when compared with None. `is` cannot be overridden, so it always answers the actual question.

The failure this causes in production is a comparison that passes every test on small values and fails on real data. A validation checking `status_code is 200` works throughout development and breaks when someone loads the code path where the value is parsed from JSON. It reads as a mysterious intermittent bug, and static analysis tools now flag it precisely because it is so common.

**Likely follow-ups**

- Which one would you use to test for None, and why that one?
- Two variables both holding the string "batch" — will `is` return True?
- What method does `==` actually call under the hood?

---

## 24. What counts as falsy in Python? And why is `if x:` not the same test as `if x is not None:`?

*Easy · Common*

**Short answer.** Empty containers, zero, empty strings, None and False are all falsy. So `if x:` is false for an empty list and for zero, not only for None. When you mean "the value was not supplied", test `x is not None`, or a legitimate zero gets treated as missing.

The falsy set is short and worth knowing exactly: `None`, `False`, zero of any numeric type, empty sequences and mappings (`""`, `[]`, `()`, `{}`, `set()`), and any object whose class defines `__bool__` or `__len__` to say so. Everything else is truthy.

The bug this creates is specific and common:

```
def apply_discount(order_total, override=None):
    if not override:
        override = DEFAULT_DISCOUNT
    return order_total * (1 - override)

apply_discount(2000, override=0)   # applies DEFAULT, not 0
```

A caller who explicitly asked for zero discount gets the default instead, because `0` is falsy. Nobody notices until a customer is charged a discount that was meant to be waived. The fix is to test for the thing you actually mean:

```
if override is None:
    override = DEFAULT_DISCOUNT
```

The same shape appears with empty strings, where a deliberately blank field is replaced by a placeholder, and with empty lists, where “query returned no rows” is confused with “query was never run”.

That said, `if items:` is idiomatic and correct when you genuinely mean “is this container non-empty”. Prefer it over `if len(items) > 0:`, which says the same thing more loudly. The judgement is about whether zero and empty are meaningful values in your domain.

One related behaviour to have ready. `or` and `and` return one of their operands, not a boolean:

```
name = user_input or "guest"      # 'guest' if user_input is falsy
```

Convenient, and it carries the same zero-and-empty-string trap.

**Likely follow-ups**

- A function returns an empty list on success — how would you write the caller's check?
- What does `or` return, and how is that different from returning True?
- How would you make your own class falsy when it holds no items?

---

## 25. Threading, multiprocessing or asyncio — how do you decide which one a workload needs?

*Hard · Common*

**Short answer.** Classify the bottleneck first. Waiting on network or disk means threads or asyncio. Burning CPU in Python means processes. Asyncio scales to thousands of concurrent waits more cheaply than threads but requires the whole call path to be async, which threads do not.

One question decides it: while this code runs, is the CPU busy or waiting?

Threads
Processes
asyncio

Suits
I/O-bound
CPU-bound
I/O-bound, high concurrency

Parallel Python bytecode
no
yes
no

Memory per worker
small
full interpreter
tiny

Data sharing
shared memory
pickled
shared, one thread

Code changes
minimal
must be picklable
async all the way down

For simulating five hundred parameter settings, each running a numeric fit for thirty seconds, processes are the answer. The work is Python-level computation and only separate interpreters get separate cores.

For fetching two thousand documents, asyncio or threads. Both overlap the waiting. Threads need no rewrite of the calling code, which matters when the client library is synchronous. Asyncio scales further because a coroutine costs far less than a thread’s stack, but it demands the whole path be async, and one blocking call inside a coroutine stalls everything.

For a mixed workload, which is the common real case, layer them. Fetch concurrently with threads or asyncio, then hand the CPU-heavy parsing to a process pool. Trying to do both with one mechanism means one half of the job is on the wrong tool.

Two costs worth naming. A hundred processes means a hundred interpreters and a hundred copies of whatever each loads, which on a machine with 16 GB is often the binding constraint rather than cores. And anything crossing a process boundary is pickled, so sending a large object to a worker that computes briefly can cost more than doing the work inline.

Only threads and asyncio share Python objects directly. That is convenient and it is also why threads need locks around mutable shared state, which processes do not.

**Likely follow-ups**

- Your workload does both — downloads then heavy parsing. How would you structure that?
- What's the memory cost of each approach at a hundred workers?
- Which of these can share a Python object between workers?

---

## 26. Run this CPU-heavy function across all cores with multiprocessing. What does it cost you?

*Hard · Common*

**Short answer.** `multiprocessing.Pool` distributes work across separate interpreters, each with its own GIL, so Python-level computation genuinely runs in parallel. The costs are process startup, pickling arguments and results across the boundary, and memory for a full interpreter per worker.

```
from multiprocessing import Pool

def fit_one(seed):
    return run_simulation(seed, n_steps=50_000)

if __name__ == "__main__":
    with Pool(processes=8) as pool:
        results = pool.map(fit_one, range(200))
```

The main guard is not optional. On Windows and on macOS with the spawn start method, each child re-imports the main module, and without the guard each child runs the pool creation again, spawning more children until the machine gives up.

Three costs shape when this is worth it.

**Startup.** Spawning eight interpreters takes a noticeable fraction of a second each, and with spawn each child re-imports your modules. For work measured in milliseconds per item, startup dominates and the parallel version is slower.

**Serialisation.** Arguments and return values are pickled. A worker that takes a small seed and returns a small result is ideal. One that receives a large array, computes for fifty milliseconds and returns another large array spends most of its time in pickle.

**Memory.** Each worker is a full interpreter with its own copy of imported modules and any data it loads. Eight workers each loading a 2 GB lookup table is 16 GB.

The picklability constraint catches people. Lambdas, locally defined functions, open file handles and database connections all fail. The worker function must be importable at module level, and each worker builds its own connections.

Use `imap_unordered` when results can be consumed as they finish, which keeps memory flat over a long run instead of collecting everything. `chunksize` matters too: with many small tasks, the default sends them one at a time and the IPC overhead swamps the work.

For a large read-only table, initialise it once per worker with the pool’s `initializer` rather than passing it with every task.

**Likely follow-ups**

- Your worker function fails to pickle — what's usually in it?
- Why does the same script hang on Windows without a main guard?
- How would you send a large read-only lookup table to every worker?

---

## 27. Type hints don't do anything at runtime. So what's the point of adding them?

*Medium · Common*

**Short answer.** They are checked by tools, not by the interpreter. A type checker such as mypy reads them and flags mismatches before you run anything. They also document intent precisely, drive editor autocomplete, and are consumed at runtime by libraries such as pydantic and dataclasses.

```
from pathlib import Path

def load_trials(
    path: Path,
    min_score: float = 0.0,
) -> list[dict[str, float]]:
    ...
```

Python stores those annotations and ignores them. The function will happily return a string:

```
def count() -> int:
    return "seven"      # runs fine, mypy flags it
```

So the value comes entirely from tooling and from readers. Three concrete returns.

**Errors before runtime.** A function annotated `-> dict | None` whose caller indexes the result directly is a `TypeError` waiting for the one input that yields None. mypy finds it in seconds; a test suite finds it only if that case is covered.

**Precision that comments cannot give.** `records: list[dict[str, float]]` tells the next reader the exact shape. A docstring saying “takes records” does not, and drifts silently when the code changes.

**Editor support.** Autocomplete and rename-refactoring work far better with annotations, which is a daily saving rather than an occasional one.

Some libraries do read them at runtime. `dataclasses` builds `__init__` from the annotations, and pydantic validates and coerces incoming data against them, which is why it is common at API and config boundaries.

Two practical points for research code. Annotate function signatures and skip local variables; the signatures carry nearly all the value at a fraction of the effort. And `from __future__ import annotations` lets you write `list[dict]` and `X | None` on older versions, since the built-in generics landed in 3.9 and the `|` union syntax in 3.10.

Adopt gradually. mypy checks what is annotated and leaves the rest alone, so a large notebook-grown codebase can be typed one module at a time rather than all at once.

**Likely follow-ups**

- If they're not enforced, can a function annotated -> int return a string?
- Where do hints actually get used at runtime by a library?
- How would you type a function that takes either a path or an open file?

---

## 28. Python can't find a variable inside a function. Where does it look, and in what order?

*Medium · Common*

**Short answer.** Python resolves a name through four scopes in order: Local, Enclosing function, Global (module level) and Builtins. The first match wins. Assigning to a name anywhere in a function makes it local for that whole function unless you declare it `global` or `nonlocal`.

LEGB is the lookup order, and the part that catches people is not the lookup but the assignment rule.

```
tax_rate = 0.18

def price_with_tax(base):
    print(tax_rate)          # UnboundLocalError
    tax_rate = 0.05
    return base * (1 + tax_rate)
```

The error appears on a line that only reads the variable. Python scans the whole function body at compile time, sees an assignment to `tax_rate`, and marks it local for the entire function, including lines above the assignment. At the moment of the `print` it is local and not yet bound.

`global` says the name refers to the module-level variable, and `nonlocal` says it refers to the nearest enclosing function’s variable:

```
def make_counter():
    count = 0
    def record():
        nonlocal count       # without this: UnboundLocalError
        count += 1
        return count
    return record
```

`count += 1` is an assignment, so the same rule applies inside the inner function. `nonlocal` points it back out one level. `global` would not work here, since `count` is not at module level.

Two things worth saying in an interview. Only functions, classes, modules and comprehensions create scopes. `for`, `if` and `with` do not, so a loop variable is still visible after the loop ends, which surprises people coming from C-family languages.

And `global` in production code is usually a smell. Module-level mutable state makes functions untestable and behaviour order-dependent, and in a multi-threaded worker it becomes a race condition. Pass the value in and return the result instead.

**Likely follow-ups**

- You assign to a name inside a function that also exists at module level — what does Python decide?
- When would you reach for `nonlocal` rather than `global`?
- Does a for loop or an if block create a new scope?

---

## 29. You need a small record type with a few fields. dict, namedtuple or dataclass?

*Medium · Common*

**Short answer.** A dataclass gives named typed fields, a generated `__init__`, `__repr__` and `__eq__`, and mutability by default. A NamedTuple is immutable, hashable and unpacks like a tuple. A plain dict has no fixed shape, so a typo creates a new key instead of raising.

```
from dataclasses import dataclass, field

@dataclass
class TrialResult:
    trial_id: str
    accuracy: float
    params: dict[str, float]
    notes: list[str] = field(default_factory=list)

r = TrialResult("t-04", 0.913, {"lr": 0.01})
r.accuracy          # attribute access, not r["accuracy"]
```

The generated `__repr__` is worth the decorator on its own. Printing this shows every field with its value, where a plain class shows an object address.

The comparison that matters:

dict
NamedTuple
dataclass

Fixed fields
no
yes
yes

Typo behaviour
silent new key
AttributeError
AttributeError

Mutable
yes
no
yes, unless frozen

Hashable
no
yes
only if frozen

Methods
no
yes
yes

The typo row is the practical argument. `result["acuracy"] = 0.9` on a dict creates a second key and the original stays untouched, so a metric silently stops updating. On a dataclass it raises immediately.

`field(default_factory=list)` is required for mutable defaults. Writing `notes: list[str] = []` raises a `ValueError` at class definition time, because a single list would be shared across every instance. The dataclass machinery refuses rather than letting you create that bug.

`@dataclass(frozen=True)` makes instances immutable and generates a `__hash__`, so they can go in sets and be used as dict keys. For a record you never intend to modify, that is the safer default.

Choose a dict for genuinely dynamic data such as a parsed JSON payload of unknown shape. Choose a NamedTuple when it must be a tuple, for unpacking or for interoperating with tuple-based code. Choose a dataclass for anything else, and reach for pydantic when the data comes from outside and needs validating.

**Likely follow-ups**

- What does frozen=True give you beyond immutability?
- Why can't you write a list as a field default?
- Which of these would you send over a process boundary?

---

## 30. Build me a list of functions in a loop, each one multiplying by its loop index. Then tell me why they all do the same thing.

*Hard · Common*

**Short answer.** A closure captures the variable, not the value it had at definition time. Every function built in the loop refers to the same loop variable, which holds its final value once the loop ends. Bind the value with a default argument or `functools.partial` to fix it.

```
multipliers = []
for factor in [2, 5, 10]:
    multipliers.append(lambda x: x * factor)

[m(3) for m in multipliers]     # [30, 30, 30] ← not [6, 15, 30]
```

Nothing is captured at the moment each lambda is created. Each one holds a reference to the enclosing variable `factor`, and looks it up when called. By the time you call them, the loop has finished and `factor` is 10.

This is late binding, and it is the correct behaviour for closures generally. A closure that captured values would be useless for the counter pattern, where the whole point is that the inner function sees updates to the outer variable.

Two ways to bind the value at definition time. A default argument, which is evaluated once when the lambda is defined:

```
for factor in [2, 5, 10]:
    multipliers.append(lambda x, f=factor: x * f)
```

Or `functools.partial`, which stores the argument explicitly:

```
from functools import partial
from operator import mul

multipliers = [partial(mul, f) for f in [2, 5, 10]]
```

Where this bites in real work is callbacks. Registering a handler per queue inside a loop, or building a list of retry functions each bound to a different endpoint, and every one of them ends up pointing at the last item. The symptom is that all your workers process the same partition, which looks like a configuration problem rather than a language one.

A comprehension has exactly the same issue, since it also builds the functions before any of them run. And each closure holds its variables in a cell object, visible as `fn.__closure__`, which is worth knowing if an interviewer pushes on the mechanism.

**Likely follow-ups**

- Does the same problem show up with a list comprehension building lambdas?
- How would `functools.partial` solve this, and is it clearer?
- What does a closure actually hold — the value or the variable?

---

## 31. When would you use `map` and `filter` with a lambda, and when would you write a comprehension instead?

*Easy · Common*

**Short answer.** A comprehension is usually clearer for transforming or filtering a collection, and it reads left to right. `map` and `filter` earn their place when you already have a named function to pass, since `map(int, values)` needs no lambda at all. Avoid `map` with a lambda.

Compare the same operation three ways on a list of ticket prices:

```
prices = ["450", "1200", "300"]

list(map(int, prices))                    # clean
list(map(lambda p: int(p) * 1.05, prices))
[int(p) * 1.05 for p in prices]           # clearer
```

The first is good code. `map` with an existing callable is compact and says exactly what it does. The second is where readability drops: a lambda inside a map means reading right to left through two layers to work out what happens to each item. The comprehension states the transformation first and the source second, which is how people read.

Filtering follows the same pattern:

```
list(filter(lambda p: int(p) > 400, prices))
[p for p in prices if int(p) > 400]
```

Combining both in one comprehension is natural and needs no nesting:

```
[int(p) * 1.05 for p in prices if int(p) > 400]
```

Two limits on lambdas that are worth stating. A lambda holds one expression, so no statements, no assignments, no try blocks. A conditional expression is allowed because it is an expression: `lambda p: "high" if p > 1000 else "low"`.

And a lambda assigned to a name gains nothing over `def`, while losing a useful name in tracebacks. `charge = lambda x: x * 1.05` shows up as `<lambda>` when it raises, and PEP 8 recommends `def` there.

One behavioural difference worth knowing: in Python 3, `map` and `filter` return lazy iterators, not lists. They consume nothing until iterated and are exhausted after one pass, so a `map` object you iterate twice gives you results the first time and nothing the second.

**Likely follow-ups**

- `map` returns a lazy object in Python 3 — when does that matter?
- Can a lambda contain an if statement, or an assignment?
- How would you write a map and filter together as one comprehension?

---

## 32. Your model works in a notebook and an engineer has to deploy it. What has to change before you hand it over?

*Hard · Common*

**Short answer.** Extract the logic into importable functions with explicit inputs and outputs, remove hardcoded paths and hidden state, pin the dependencies, and add a small test on the transformation logic. The handover artefact is a package with a defined entry point, not a notebook.

The notebook and the deployed system differ in four ways, and each one needs work.

**Hidden state becomes explicit arguments.** A cell using `df_train` defined eleven cells earlier works only because that variable is still in memory. Wrap it in a function taking what it needs and returning what it produces. That single change makes the code testable, reusable and reviewable.

```
def build_features(readings: list[dict],
                   vocab: dict[str, int]) -> list[list[float]]:
    ...
```

**Paths and constants come from configuration.** `/home/ayan/Downloads/aug_export.csv` and a threshold typed into a cell both have to become parameters read from arguments or a settings object.

**Order becomes guaranteed.** Cells can run in any order, and the engineer’s environment will run top to bottom exactly once. Restart the kernel and run everything before you hand anything over; if it fails, the notebook never described what produced your results.

**Dependencies get pinned.** An exported environment file, so the same versions install on the other side.

The technical risk to raise unprompted is training-serving skew. Preprocessing written twice, once in your notebook and once in the serving code, will drift, and the model then sees features it was not trained on. Ship one implementation that both paths import. If a scaler or an encoder was fitted on training data, its fitted state has to be serialised and shipped with the model, not refitted at serving time.

Hand over more than code: the training data version or query, the fitted artefacts, the metrics you measured and on which split, and the expected input schema with types and ranges. An engineer who cannot tell whether the model is behaving correctly cannot operate it.

**Likely follow-ups**

- The preprocessing lives in four scattered cells — what would you do with it first?
- How do you make sure the features at serving time match training?
- What would you hand over besides the code?

---

## 33. What's the difference between an iterable and an iterator? Which methods does each one need?

*Medium · Common*

**Short answer.** An iterable defines `__iter__` and can produce an iterator. An iterator defines both `__iter__` and `__next__`, holds the position, and raises StopIteration when exhausted. A list is an iterable that hands out a fresh iterator each time, which is why you can loop over it repeatedly.

A `for` loop is doing three things behind the scenes: call `iter()` on the object, call `next()` repeatedly, and stop when `StopIteration` is raised.

```
racks = ["R1", "R2"]
it = iter(racks)          # list -> list_iterator
next(it)                  # 'R1'
next(it)                  # 'R2'
next(it)                  # StopIteration
```

The list itself has no position. Each `iter()` call creates a new iterator with its own cursor, which is why two nested loops over the same list work independently.

An iterator returns itself from `__iter__`, which is what lets you pass one to a `for` loop directly. It also means iterating it consumes it. A file object is its own iterator, so a second loop over an already-read file yields nothing at all, with no error. That is the same silent-empty failure a consumed generator produces.

Writing one by hand:

```
class Countdown:
    def __init__(self, start):
        self.n = start

    def __iter__(self):
        return self

    def __next__(self):
        if self.n

**Likely follow-ups**

- Why can you loop over a list twice but not over a file object twice?
- What does `iter()` do when a class only defines `__getitem__`?
- Where does StopIteration go in a for loop?

---

## 34. You need the index alongside each item, and you need to walk two lists together. Show me both, and tell me what zip does when the lists differ in length.

*Easy · Common*

**Short answer.** `enumerate(seq)` yields index-item pairs and takes a `start` argument. `zip(a, b)` yields tuples pairing corresponding elements and stops at the shortest input, silently dropping the tail of the longer one. Pass `strict=True` in Python 3.10 or later to make a length mismatch raise.

```
crops = ["wheat", "gram", "mustard"]

for i, crop in enumerate(crops, start=1):
    print(f"Plot {i}: {crop}")
```

The `start` argument saves the `i + 1` that otherwise appears everywhere in report-numbering code.

Zip pairs elements positionally:

```
plots = ["P1", "P2", "P3"]
yields = [22.5, 19.1]

list(zip(plots, yields))
# [('P1', 22.5), ('P2', 19.1)]  ← P3 vanishes
```

No error, no warning. The third plot is gone from the output, and if this is feeding a report, the total is short by one row and nothing indicates why. It is a genuinely common source of quietly wrong results when two sequences come from different sources and one is missing a record.

Two ways to guard against it. From Python 3.10, `zip(plots, yields, strict=True)` raises `ValueError` on a mismatch, which is what you want whenever equal length is an invariant. Before 3.10, assert the lengths, or use `itertools.zip_longest` when the shorter sequence should be padded:

```
from itertools import zip_longest
list(zip_longest(plots, yields, fillvalue=None))
```

Combining both is idiomatic and reads well:

```
for i, (plot, y) in enumerate(zip(plots, yields), start=1):
    ...
```

Note the parentheses around the inner pair. Without them the unpacking fails.

Two more things worth knowing. `zip(*rows)` transposes, turning rows into columns, and it is the standard way to unzip a list of pairs back into two tuples. And in Python 3, `zip` and `enumerate` return lazy iterators, so wrap in `list()` if you need to look at the result more than once.

**Likely follow-ups**

- How would you make zip raise instead of truncating?
- What does `zip(*rows)` do?
- Can you unzip back into separate sequences, and what do you get?

---

## 35. You've run a t-test in scipy and got two numbers back. What do they mean, and what would you check before trusting them?

*Medium · Common*

**Short answer.** The first value is the test statistic and the second is the p-value, the probability of seeing a difference this large if the null hypothesis were true. Check the test's assumptions, report an effect size alongside it, and know that scipy will compute the result regardless of whether those assumptions hold.

```
from scipy import stats

stat, p = stats.ttest_ind(group_a, group_b, equal_var=False)
print(f"t = {stat:.3f}, p = {p:.4f}")
```

`equal_var=False` gives Welch’s t-test, which does not assume the two groups have equal variance. It is the safer default and loses very little power when the variances happen to be equal, so most practitioners now use it routinely.

What scipy will not do is stop you. Feed it two skewed distributions, or paired samples through an independent test, or data with wild outliers, and it returns a number with the same confidence as always. The function has no way to know your design.

So the checks come first. Are the observations independent, or is the same user in both groups? Is the sample large enough that the central limit theorem covers the non-normality? Are there outliers driving the difference, which for a t-test on the mean can flip the conclusion on their own?

Two things about interpreting the p-value.

It is not the probability the null is true, and it is not the probability the effect is real. It is the probability of data at least this extreme under the null. That distinction is what interviewers are usually listening for.

And with a large sample, a trivially small difference becomes statistically significant. At two lakh observations a 0.3% lift can produce p = 0.001 and be worth nothing commercially. Always report the effect size and a confidence interval next to the p-value, because those describe the magnitude that the p-value does not.

Multiple testing is the other trap. Running twenty comparisons at the 0.05 level gives roughly a 64% chance of at least one false positive. Apply a correction, or say clearly that the analysis was exploratory.

The non-parametric alternative is `stats.mannwhitneyu` when normality is genuinely doubtful.

**Likely follow-ups**

- Your p-value is 0.03 with a sample of two lakh — how excited should you be?
- What does equal_var=False change?
- You ran twenty tests and one came back significant — what would you do?

---

## 36. You set a random seed and the results still change between runs. What could be causing that?

*Easy · Common*

**Short answer.** There is more than one generator. Python's `random`, numpy's, and any framework's each have separate state, so seeding one leaves the others free. Beyond that, thread scheduling, set and dict iteration over hash-randomised objects, and GPU non-determinism all introduce variation a seed cannot control.

```
import random, os
import numpy as np

SEED = 20260819
random.seed(SEED)
np.random.seed(SEED)
os.environ["PYTHONHASHSEED"] = str(SEED)
```

Seeding one and assuming the rest follow is the usual cause. A shuffle from `random` and a sample from numpy draw from entirely separate state.

The `PYTHONHASHSEED` line has a catch worth knowing: setting it inside the script is too late, because hash randomisation is fixed when the interpreter starts. It has to be in the environment before launch. Where it matters is any code iterating over a set of strings, since the order varies between processes and a downstream sample then differs.

Three other sources a seed does not touch.

Concurrency. Threads or processes completing in a different order change the order results are combined, and floating-point addition is not associative, so the totals differ in the last digits. Seeding each worker separately and combining deterministically is the fix.

GPU operations. Many are non-deterministic by default because of atomic accumulation order. Frameworks expose a deterministic mode, usually at a performance cost.

Library defaults. Scikit-learn estimators taking `random_state` use the global numpy state when you leave it as None, which makes the result depend on how many random draws happened earlier in the session.

The pattern that survives review is to stop using global state. Create an explicit generator and pass it where it is needed:

```
rng = np.random.default_rng(SEED)
idx = rng.choice(len(records), size=500, replace=False)
```

`default_rng` is the current numpy API and is preferred over `np.random.seed`, because the object is passed explicitly and no unrelated code can consume from it. Log the seed with every run, or you cannot reproduce even the runs that were deterministic.

**Likely follow-ups**

- Which libraries have their own separate generators?
- Why does adding threads break reproducibility even with a seed?
- What would you record so a run can be reproduced a year later?

---

## 37. What's actually wrong with writing `except:` with nothing after it?

*Medium · Common*

**Short answer.** A bare `except:` catches everything derived from BaseException, including KeyboardInterrupt and SystemExit, so your process cannot be interrupted or shut down cleanly. It also swallows genuine bugs such as typos raising NameError, turning them into whatever your handler pretends went wrong.

```
while True:
    try:
        job = queue.pop()
        proces_job(job)          # typo
    except:
        log.warning("job failed, retrying")
```

Two things are broken here and neither is visible. The typo raises `NameError` on every single iteration, and the handler reports it as a job failure, so the log fills with retry messages about a queue that is working perfectly. And pressing Ctrl-C raises `KeyboardInterrupt`, which the bare except catches, so the loop keeps running and the only way out is to kill the process.

The exception hierarchy is why. `KeyboardInterrupt`, `SystemExit` and `GeneratorExit` inherit from `BaseException` directly, not from `Exception`, precisely so that ordinary handlers do not intercept them. A bare `except:` sits at the `BaseException` level and undoes that design.

```
except Exception:        # excludes Ctrl-C and sys.exit
except (TimeoutError, ConnectionError):   # better still
```

Catch what you can actually do something about. If the handler’s response is “retry the network call”, then catch the network errors, and let a `KeyError` from malformed data propagate to somewhere that knows what to do with it.

There are two legitimate uses of a broad catch. A top-level supervisor loop in a long-running worker, where the job must not die because one item failed, and a plugin boundary where you cannot know what user code raises. In both cases the handler must log the full traceback and re-raise or record the failure:

```
except Exception:
    log.exception("job %s failed", job.id)   # includes traceback
    failed.append(job.id)
```

`log.exception` rather than `log.warning` is the detail that matters. Without the traceback you have a message and no way to find the line.

Interviewers ask this because a swallowed exception is the hardest class of bug to debug: the program reports success while doing nothing.

**Likely follow-ups**

- So what's the narrowest handler you'd write for a network call that might time out?
- If you must catch broadly, what do you do inside the handler?
- What's the difference between `except Exception` and `except BaseException`?

---

## 38. A colleague reruns your analysis six months later and gets different numbers. The code hasn't changed. What did?

*Medium · Common*

**Short answer.** The installed packages. An unpinned requirements file installs whatever the latest compatible releases are on the day, so a default parameter or an algorithm changed underneath the code. Pin exact versions in a lock file, including transitive dependencies, and record the Python version too.

`pip install scikit-learn` in March and again in September gives two different versions. Library defaults change between releases, algorithms get reimplemented, and a solver’s default tolerance shifts. Your code is identical and the numbers are not.

Two files with different jobs:

```
# requirements.in — what you asked for
scikit-learn>=1.4
scipy
```

```
# requirements.txt — what actually got installed
scikit-learn==1.4.2
scipy==1.13.0
joblib==1.4.2
threadpoolctl==3.5.0
numpy==1.26.4
```

The second is generated, by `pip-compile` from pip-tools or by `pip freeze`, and it includes the transitive dependencies. Those matter as much as the direct ones: a change in a numerical backend changes results even when every package you named stayed the same.

Pin the Python version as well. Behaviour differs between minor releases, and an environment built on 3.11 is not the same as one on 3.12.

Two limits worth stating honestly.

A lock file pins Python packages and not the system libraries beneath them. BLAS implementations and CPU instruction sets affect floating-point results in the last digits, which is usually irrelevant and occasionally is not. A container image pins the whole stack including the OS libraries, which is why it is the stronger answer for anything that must reproduce exactly.

And environment reproducibility does nothing about the data. If the source table has been corrected or extended since March, an identical environment gives a different answer for entirely legitimate reasons. Record the data version, the query, and the row count alongside the environment.

Conda users get the same with an exported environment file, which additionally captures non-Python dependencies. `uv` and Poetry both produce lock files with the same guarantee and better resolution behaviour.

**Likely follow-ups**

- What's the difference between the file you write by hand and the one you generate?
- Would a container solve this completely?
- How would you capture the data version alongside the environment?

---

## 39. How would you make a notebook that anyone can run top to bottom and get your numbers?

*Medium · Common*

**Short answer.** Restart the kernel and run all cells before trusting anything, because out-of-order execution means the displayed output may not match the code. Beyond that: no absolute paths, no manually edited variables, seeds set at the top, and dependencies pinned.

The defining problem is hidden state. A variable defined in a cell you later edited still exists in memory, so a notebook can display correct output that its own code cannot reproduce. Restart-and-run-all is the only test that matters, and it should be habitual before sharing, not something you do once at the end.

Beyond that, five things.

**Paths.** `/Users/ayan/Desktop/exp3.csv` works nowhere else. Read a root from an environment variable or a config cell at the top.

**Seeds at the top, in one place.** Set them in the first executable cell so anyone can see what they are and change them deliberately.

**No cells that must be skipped.** A cell that downloads 40 GB and is normally commented out is a landmine. Either guard it with a flag or move it out of the notebook.

**Nothing edited by hand.** A value typed over a computed result is invisible in the code and gone the moment the cell reruns.

**Pinned dependencies alongside.** The environment file lives next to the notebook, not in someone’s memory.

Even after all that, a colleague can get different numbers if the underlying data has changed. Record the data version, the query and the row count in a cell so a mismatch is visible rather than mysterious.

For review, `nbstripout` or `jupytext` help enormously. The raw `.ipynb` is JSON containing outputs and execution counts, so a diff is unreadable and merge conflicts are common. Stripping outputs before committing, or pairing the notebook with a `.py` representation, makes the change reviewable.

The structural fix is to keep heavy logic in imported modules and let the notebook be a thin surface over it. Less code in the notebook means less that can be run out of order.

**Likely follow-ups**

- Restart-and-run-all passes but a colleague still gets different output — what's left?
- How would you review a notebook diff in a pull request?
- Where should the heavy computation live if not in the notebook?

---

## 40. Someone sends you a pickle file to load. Any concerns?

*Medium · Common*

**Short answer.** Unpickling executes code, so loading a file from an untrusted source can run anything on your machine. Pickles are also tied to the class definitions and library versions present when they were written, so a loaded object can fail or behave incorrectly after an upgrade.

Pickle is not a data format that a reader parses. It is a small instruction stream that the unpickler executes, and part of that instruction set can import a module and call it. A crafted pickle can run an arbitrary command as your user, and it happens during `pickle.load`, before you have inspected anything.

```
import pickle
obj = pickle.load(open("model_from_slack.pkl", "rb"))
# code has already run by this point
```

There is no safe mode, no sandbox flag, and no validation you can do first. The only mitigation is not loading pickles you did not produce, or loading them somewhere disposable.

The second problem bites even with your own files. A pickle stores a reference to the class by module path, not the class definition itself. Load it into an environment where that module has moved, the class has gained a field, or the library has been upgraded, and you get an `AttributeError`, a `ModuleNotFoundError`, or worse, an object that reconstructs with the wrong attributes and behaves oddly rather than failing.

That makes pickle poor for anything meant to last. A model pickled with one scikit-learn version and loaded under another is explicitly not supported, and the failure can be silent.

What to use instead depends on the payload. For data, use a format with a real parser: JSON, Parquet, or CSV. For configuration, TOML or YAML. For numeric arrays, `numpy.save`. For a trained model that must survive an upgrade, an interchange format such as ONNX, or code that rebuilds the model from saved parameters.

Where pickle remains reasonable: short-lived caching inside one process or one pipeline run, in an environment you control, where the file is regenerated rather than archived. Record the library versions with it, because you will need them.

**Likely follow-ups**

- What would you use instead for data you receive from outside?
- Why does a pickle break after you upgrade a library?
- Is there any way to make loading a pickle safe?

---

## 41. You've been handed a 30 GB CSV export and 8 GB of RAM. How do you process it?

*Medium · Common*

**Short answer.** Iterate the file object directly, which yields one line at a time and holds only that line in memory. Avoid `.read()` and `.readlines()`, both of which pull the whole file in. For fixed-size chunks of binary data, read in a loop with an explicit chunk size.

```
# loads 30 GB into RAM, then dies
rows = open("exports/claims.csv").readlines()

# streams, one line at a time
with open("exports/claims.csv") as f:
    header = next(f)
    for line in f:
        process(line)
```

A file object is its own iterator, so the `for` loop reads a buffered chunk from disk and hands you one line per iteration. Memory stays flat regardless of file size. That `next(f)` for the header works because the same iterator is consumed by both.

For CSV specifically, `csv.reader` wraps the same streaming behaviour and handles quoted fields containing newlines, which naive line splitting gets wrong. Reach for it rather than `line.split(",")` on anything that came out of a spreadsheet.

Binary or fixed-size reading needs an explicit loop:

```
with open(path, "rb") as f:
    while chunk := f.read(1024 * 1024):
        digest.update(chunk)
```

The walrus operator needs Python 3.8 or later; before that it is a `while True` with a break.

Two things that decide whether streaming actually works for your problem.

Aggregates are fine if they are incremental. Running totals, counts, min and max all work in one pass with constant memory. A median or an exact distinct count needs the whole dataset, so either accept the memory, spill to a temporary file, or use an approximate algorithm.

The failure mode people hit anyway is accumulating inside the loop. Streaming the file and appending every parsed row to a list uses the same memory as reading it all at once, just more slowly. Write results out as you go, or aggregate into something bounded.

If the work per line is genuinely CPU-heavy, `multiprocessing` over line ranges helps; if it is I/O bound, it mostly will not, since one disk is the constraint.

**Likely follow-ups**

- What if you need a total that depends on the whole file?
- How would you handle a record that spans multiple lines?
- Where would you parallelise this, and what stops you?

---

## 42. What's the difference between `__str__` and `__repr__`? Which one shows up when you print a list of your objects?

*Medium · Common*

**Short answer.** `__repr__` is for developers and should be unambiguous; `__str__` is for end users and should be readable. Printing a container calls `__repr__` on each element, not `__str__`, so define `__repr__` first. `__str__` falls back to `__repr__` when it is missing.

```
class Route:
    def __init__(self, code, stops):
        self.code = code
        self.stops = stops

    def __repr__(self):
        return f"Route({self.code!r}, stops={self.stops})"

    def __str__(self):
        return f"{self.code} ({self.stops} stops)"

r = Route("KA-17", 12)
print(r)         # KA-17 (12 stops)
print([r])       # [Route('KA-17', stops=12)]
r                # Route('KA-17', stops=12)  in the REPL
```

The list case is the one people get wrong. Containers always call `__repr__` on their elements, so a class with only `__str__` defined prints beautifully on its own and shows `<__main__.Route object at 0x7f...>` inside a list, in a debugger, and in a log line that formats a collection. If you write only one of the two, write `__repr__`.

The `!r` in the f-string calls `repr()` on the field, which keeps quotes around strings so the output stays unambiguous.

`__eq__` changes comparison from identity to whatever you define:

```
def __eq__(self, other):
        if not isinstance(other, Route):
            return NotImplemented
        return self.code == other.code
```

Returning `NotImplemented` for unknown types lets Python try the reflected operation instead of forcing a wrong answer.

The consequence people forget: defining `__eq__` sets `__hash__` to None, so instances become unhashable and cannot go in a set or be used as dict keys. If you need both, define `__hash__` over the same fields, and only when those fields are immutable. A hashable object whose hash changes after insertion becomes unfindable in the dict it is sitting in.

`@dataclass` generates `__init__`, `__repr__` and `__eq__` from annotations, and `frozen=True` adds a `__hash__`. For plain data-carrying classes it is usually the better starting point.

**Likely follow-ups**

- If you define `__eq__`, what else do you have to define and why?
- What would you use `__hash__` for after that?
- How does a dataclass change this whole picture?

---

## 43. Why does everyone write `if __name__ == "__main__":` at the bottom of a script?

*Easy · Common*

**Short answer.** Python sets `__name__` to `"__main__"` in the file being run directly, and to the module's name when it is imported. The guard runs your script logic only in the first case, so importing the module for its functions does not execute the whole job as a side effect.

```
# ingest.py
def load_readings(path):
    ...

def main():
    load_readings("data/august.csv")
    publish_summary()

if __name__ == "__main__":
    main()
```

Run `python ingest.py` and `__name__` is `"__main__"`, so `main()` fires. Write `from ingest import load_readings` in a notebook or another module, and `__name__` is `"ingest"`, so the guard is false and only the definitions are created.

Without it, every import runs the entire job. Someone imports one helper function and the full August ingestion kicks off, writing to the production summary table. It is not a subtle failure, and it is one people meet exactly once.

Testing is the everyday version of the same problem. A test module importing your script to test one function triggers the whole pipeline before any test runs, so the suite is slow and has side effects nobody asked for.

Three related points.

Anything at module level runs on import, guard or not. A database connection or an expensive model load written at the top of the file executes for every importer, so those belong inside `main()` or a function that is called deliberately.

`multiprocessing` on Windows and on macOS with the spawn start method re-imports the main module in each child process. Without the guard, each child runs your script again, which spawns more children, and the process count grows until something gives.

Keep `main()` thin and the guard thinner. Argument parsing goes inside `main()` rather than at module level, so importing the module does not attempt to read `sys.argv` from whatever process happens to be running.

**Likely follow-ups**

- What breaks specifically when multiprocessing imports your module on Windows?
- Is the guard needed in a file that will only ever be imported?
- Where would you put the argument parsing relative to the guard?

---

## 44. Your feature engineering script gets killed by the OOM killer. How do you find what is consuming the memory?

*Medium · Occasional*

**Short answer.** Use `tracemalloc` to attribute allocations to the lines that made them, and compare snapshots taken at different points to see growth. For line-by-line attribution of an entire function, `memory_profiler` reports the increment each statement causes.

`tracemalloc` is in the standard library and needs no install:

```
import tracemalloc

tracemalloc.start()
features = build_features(records)
snapshot = tracemalloc.take_snapshot()

for stat in snapshot.statistics("lineno")[:10]:
    print(stat)

current, peak = tracemalloc.get_traced_memory()
print(f"peak {peak / 1e6:.0f} MB")
```

The statistics name the file and line where each block of memory was allocated, sorted by size. That usually ends the investigation in one run, because a script that dies at 12 GB has one or two lines responsible.

`get_traced_memory` returning the peak matters more than the current figure. A script can free everything and still have been killed at a moment when three copies of the data existed at once, and a snapshot taken afterwards shows nothing wrong.

For a per-line view of one function, `memory_profiler` with its `@profile` decorator prints the increment each statement caused, run under `python -m memory_profiler script.py`. It is slower than `tracemalloc` and easier to read for a single hot function.

Two traps worth knowing.

`sys.getsizeof` measures only the container. A list of one lakh strings reports about 800 KB because it counts the pointers, not the strings they point at. It is useful for a single object and misleading for anything nested.

And a process that peaked at 12 GB may keep a high RSS even after freeing the objects, because the allocator does not always return memory to the OS. So `tracemalloc` can correctly show the objects gone while `top` still shows a large process.

Once you know the line, the fix is usually structural: stream instead of materialising, process in chunks, delete intermediates you no longer need, or switch a list of dicts to a more compact representation.

**Likely follow-ups**

- sys.getsizeof on a list of strings under-reports badly — why?
- How would you measure peak usage rather than usage at one moment?
- What would you change once you know which line is responsible?

---

## 45. When would you use functools.partial instead of just writing a lambda?

*Medium · Occasional*

**Short answer.** `partial` fixes some arguments of a function and returns a new callable taking the rest. It beats a lambda when the result must be picklable, when you want the underlying function inspectable, and when building callables in a loop, because it binds values immediately rather than capturing the variable.

```
from functools import partial

def resize_crop(image, width, height, interpolation):
    ...

thumb = partial(resize_crop, width=128, height=128,
                interpolation="bilinear")

thumb(img)                        # uses the fixed values
thumb(img, interpolation="cubic") # override at call time
```

The same thing with a lambda works, and three differences decide between them.

**Picklability.** A lambda cannot be pickled, so it cannot be sent to a `ProcessPoolExecutor` or a multiprocessing pool. A `partial` of a module-level function can. This alone settles most cases in parallel preprocessing code:

```
with ProcessPoolExecutor() as pool:
    pool.map(partial(resize_crop, width=128, height=128,
                     interpolation="bilinear"), images)
```

**Late binding.** Building callables in a loop with a lambda captures the variable, so all of them see the final value. `partial` stores the argument at construction, so each one keeps its own:

```
scalers = [partial(rescale, factor=f) for f in factors]
```

**Introspection.** `thumb.func`, `thumb.args` and `thumb.keywords` expose what was bound, which a lambda hides entirely. That matters when something logs the callable or a debugger shows it, since a lambda reports only `<lambda>`.

Two limitations. Positional arguments bind left to right, so fixing the third positional argument means using a keyword instead. And a partial of a partial works but reads badly; one level is usually the limit before a named function is clearer.

Where a lambda still wins: a throwaway expression in a `key=` or `filter`, where none of the above matters and the lambda is shorter.

**Likely follow-ups**

- What happens to keyword arguments you didn't fix — can the caller still pass them?
- Why does partial survive pickling when a lambda doesn't?
- How would you fix an argument that isn't the first one?

---

## 46. Your experiment code is a thousand lines of notebook cells. Would restructuring it as classes actually help?

*Medium · Occasional*

**Short answer.** Classes help when several pieces of code share the same state and the same lifecycle: a configuration, a fitted transformer, a run directory. They help less when the work is a sequence of transformations, where plain functions taking data and returning data are easier to test and reuse.

The honest answer is that most experiment code needs functions, not classes, and the useful classes are few and specific.

Where a class earns its place is around state with a lifecycle. Something that is configured, then fitted, then applied, and must remember what it learned:

```
class ScoreCalibrator:
    def __init__(self, method: str = "isotonic"):
        self.method = method
        self._fitted = None

    def fit(self, raw_scores, outcomes):
        self._fitted = _fit_curve(raw_scores, outcomes,
                                  self.method)
        return self

    def apply(self, raw_scores):
        if self._fitted is None:
            raise RuntimeError("call fit() first")
        return _apply_curve(self._fitted, raw_scores)
```

The class exists because `apply` needs what `fit` learned, and passing that around by hand between notebook cells is exactly how a calibration fitted on the test split ends up applied to itself.

That guard clause is the other benefit. A class can enforce its own ordering, where loose cells cannot enforce anything.

Where classes do not help: a chain of transformations. `clean(raw)` then `featurise(clean)` then `score(features)` are pure functions, trivially testable with a small input, and wrapping them in a class with `self.data` mutated at each step makes them harder to test and impossible to reuse in a different order.

A class holding only data with no behaviour is a dataclass, and one with a single method is a function with extra syntax.

The refactor to propose in an interview is the hybrid. Move the logic into modules, keep the notebook as a thin surface that imports and calls it:

```
from experiments.calibration import ScoreCalibrator
```

The logic becomes testable and version-controllable, the notebook keeps the plots next to the code, and the reproducibility problem of out-of-order cells shrinks to whatever is left in the notebook.

**Likely follow-ups**

- What would you keep in the notebook after this refactor?
- How does a config object fit alongside the classes?
- When is a class the wrong answer and a plain function right?

---

## 47. You want every model wrapper in the codebase to expose the same three methods. How would you enforce that?

*Hard · Occasional*

**Short answer.** Subclass `abc.ABC` and mark the required methods with `@abstractmethod`. Python then refuses to instantiate any subclass that has not implemented all of them, raising TypeError. It is an instantiation-time check, not a compile-time one, and it says nothing about signatures.

```
from abc import ABC, abstractmethod

class Forecaster(ABC):
    @abstractmethod
    def fit(self, history): ...

    @abstractmethod
    def predict(self, horizon): ...

    def describe(self):            # concrete, inherited
        return type(self).__name__

class NaiveSeasonal(Forecaster):
    def fit(self, history):
        self.last_cycle = history[-52:]
        return self
    # predict not implemented

NaiveSeasonal()
# TypeError: Can't instantiate abstract class NaiveSeasonal
# with abstract method predict
```

The failure happens when someone tries to create an instance, not when the module is imported and not when the class is defined. So an incomplete subclass sitting unused in the codebase raises nothing until it is used, which is worth knowing when you expect the error earlier.

An ABC mixes required and provided behaviour freely. `describe` above is ordinary and every subclass gets it, which is the difference between an ABC and a bare protocol.

Two limits worth stating in an interview.

It checks names, not signatures. A subclass defining `predict(self)` with no horizon argument satisfies the ABC completely and fails at call time. A type checker catches that; the ABC does not.

And Python does not need an ABC for polymorphism. Duck typing means any object with `fit` and `predict` works wherever one is expected, which is exactly how scikit-learn estimators interoperate without inheriting from anything. The ABC buys you an early, loud failure and a single place documenting the contract, at the cost of forcing inheritance.

`typing.Protocol`, from Python 3.8, is the middle ground: a static checker verifies structural conformance with no inheritance required, and nothing is enforced at runtime.

`isinstance` still works normally against an ABC, which is the usual reason to keep one when a factory has to dispatch on type.

**Likely follow-ups**

- When does the error actually fire — at import, or when someone instantiates?
- What does duck typing give you that an ABC does not?
- How would you check the type of an argument without naming every subclass?

---

## 48. Two computed values should be equal and `==` says they are not. What's happening, and how do you compare them?

*Medium · Occasional*

**Short answer.** Floating point stores binary approximations, so arithmetic accumulates tiny errors and two mathematically equal results can differ in the last bits. Compare with `math.isclose(a, b)`, which applies a relative tolerance, rather than testing exact equality.

```
sum([0.1] * 10) == 1.0        # False
sum([0.1] * 10)               # 0.9999999999999999
```

Nothing went wrong. 0.1 has no exact binary representation, so each addition carries a rounding error and ten of them accumulate into a visible difference.

```
import math
math.isclose(sum([0.1] * 10), 1.0)    # True
```

`math.isclose` uses a relative tolerance of 1e-09 by default, meaning the values must agree to about nine significant figures. Relative is the right default because the acceptable gap scales with magnitude: a difference of 0.001 is nothing between two values around a crore and enormous between two values around 0.002.

The exception is comparing against zero. Relative tolerance is meaningless there, since any tolerance times zero is zero, so `isclose(1e-15, 0.0)` returns False. Supply an absolute tolerance:

```
math.isclose(residual, 0.0, abs_tol=1e-12)
```

That is the case people get wrong most often, and it turns up whenever you check whether a gradient or a residual has converged.

Two related points for a data science context.

Addition is not associative in floating point, so summing a large array in a different order gives a slightly different total. Parallel reductions and chunked sums therefore produce results that differ in the last digits between runs, which is not a bug and does surprise people comparing outputs across machines.

And accumulated error grows with the number of operations, so a running total over a crore values drifts more than one over a hundred. Compensated summation algorithms exist for when that matters.

For money, floats are the wrong type entirely and `Decimal` is the answer, but that is a different problem from tolerance comparison.

**Likely follow-ups**

- What's the difference between the relative and absolute tolerance in isclose?
- How would you compare against zero specifically?
- Why can summing the same numbers in a different order change the total?

---

## 49. Your calculation returns inf and then everything downstream becomes NaN. Walk me through what happened.

*Medium · Occasional*

**Short answer.** Python ints grow arbitrarily large, but floats follow IEEE 754 and overflow to `inf` past about 1.8e308. Any arithmetic involving inf or NaN produces NaN, which then propagates silently through every subsequent operation without raising anything.

Python’s integers have no fixed width, so `2 ** 5000` is exact. Floats do not share that property:

```
import math

1.8e308 * 10          # inf
math.exp(800)         # OverflowError
1e308 * 10 - 1e308 * 10   # nan
float("inf") - float("inf")   # nan
0.0 / 0.0             # ZeroDivisionError in Python
```

Note the inconsistency: dividing floats by zero raises in pure Python, while the same operation inside numpy produces `inf` with a warning. That difference alone explains why the same formula behaves differently depending on whether the input is a list or an array.

The propagation is what makes this dangerous. Once one NaN exists, every mean, sum and comparison touching it produces NaN, and nothing raises. A metric reported as NaN at the end of a two-hour training run gives you no indication of which step produced it.

Two behaviours worth knowing precisely.

`nan == nan` is False, by IEEE definition, since NaN represents an undefined result and two undefined results are not comparable. So you cannot test for it with equality; use `math.isnan(x)`.

And NaN in a sort corrupts the ordering, because every comparison against it returns False, so the sorted output is not sorted in any meaningful way.

The usual sources in modelling code: the log of zero or a negative, `exp` of a large positive, division where the denominator can be zero, and subtracting two large near-equal quantities. The last one destroys precision even when it does not overflow.

The practical response is to check at boundaries rather than hunt afterwards. Assert that intermediate arrays are finite after each stage, which converts a silent NaN into a loud failure at the step that caused it. Working in log space, and using stabilised formulations such as log-sum-exp, avoids the overflow in the first place.

**Likely follow-ups**

- Why does NaN == NaN return False?
- Where would a Python int overflow, given it has arbitrary precision?
- How would you find the first NaN in a long chain of computations?

---

## 50. Write a sampler that gives the same sample when someone reruns it next quarter, after more rows have been added.

*Medium · Occasional*

**Short answer.** Hash a stable identifier and select rows whose hash falls below a threshold. That is reproducible without a seed, and it is stable as data grows, because a new record either qualifies or does not without displacing anyone already selected.

The seeded approach is reproducible only against unchanged data:

```
rng = np.random.default_rng(20260819)
sample = rng.choice(applications, size=5000, replace=False)
```

Add ten thousand applications and rerun with the same seed, and you get a different sample, because the population changed. That is correct behaviour and usually not what you wanted.

Hashing a stable key gives you both properties:

```
import hashlib

def in_sample(key: str, pct: int, salt: str = "audit-2026") -> bool:
    digest = hashlib.md5(f"{salt}:{key}".encode()).hexdigest()
    return int(digest[:8], 16) % 100 < pct

sample = [a for a in applications if in_sample(a["app_id"], 5)]
```

Every application’s membership depends only on its own ID, so the sample is reproducible with no seed to record and stable as the population grows. Change the salt and you get an independent sample of the same size, which is how you draw a second non-overlapping cohort later.

Two things to be deliberate about.

Do not use Python’s built-in `hash()` for this. It is randomised per process for strings, so the same key gives different results in different runs. `hashlib` is deterministic across processes and versions.

And sampling rows is not sampling entities. Selecting 5% of loan applications over-represents applicants with many applications, so any per-applicant statistic from that sample is biased. Hash the applicant ID instead, and take all their applications.

For stratified sampling, apply the hash within each stratum and take the same percentage from each, which preserves the population mix. Equal-size strata need a different analysis afterwards, with weights.

Whatever you choose, persist the sampled identifiers alongside the analysis. Six months later, “5% hashed on app_id with salt audit-2026” is a reproducible description, and the stored list is the proof.

**Likely follow-ups**

- What breaks if you sample by row position instead of by key?
- How would you keep the class proportions while sampling?
- Where would you record what was sampled, and why bother?

---

## 51. When would you define your own exception class rather than raising ValueError?

*Medium · Occasional*

**Short answer.** Define your own when callers need to distinguish your failure from everything else and handle it specifically. A shared base class for your library lets callers catch all of it in one clause. Built-ins are fine when the meaning genuinely matches, such as a bad argument value.

The test is whether a caller would want to catch it separately. If the answer is no, `ValueError` with a clear message is enough and adding a class is noise.

```
class FeedError(Exception):
    """Base for everything this module raises."""

class SchemaMismatch(FeedError):
    def __init__(self, expected, found):
        self.expected = expected
        self.found = found
        super().__init__(
            f"expected {expected} columns, found {found}"
        )

class FeedUnavailable(FeedError):
    pass
```

The base class is the part that earns its keep. A caller who does not care which specific thing went wrong writes `except FeedError:` once, and keeps working when you add a fourth subclass next month. Without a shared base, every caller maintains a growing tuple of your exception names.

Carrying structured attributes matters more than the message. A retry layer needs to know whether the failure is transient, and parsing that out of a string is fragile:

```
except SchemaMismatch as exc:
    if exc.found > exc.expected:
        drop_extra_columns()
```

Two things worth mentioning in an interview. Inherit from a built-in when the semantics genuinely match and callers may already be catching it: a custom lookup failure inheriting from `KeyError` keeps existing code working. Do not inherit from `BaseException`; that is reserved for things that should escape ordinary handlers.

And when wrapping a lower-level error, preserve the chain:

```
except ConnectionError as exc:
    raise FeedUnavailable("vendor feed down") from exc
```

`from exc` attaches the original traceback. Without it, the log shows your exception and nothing about the socket error that caused it, which is exactly the detail you need at 2 AM.

**Likely follow-ups**

- What would you put in the exception besides a message?
- Where would you inherit from something other than Exception?
- How do you keep the original error visible when you wrap it in your own?

---

## 52. You've trained a model and need to save it for the serving team. How do you do that, and what do you ship with it?

*Medium · Occasional*

**Short answer.** `joblib.dump` writes the fitted object and handles large numeric arrays more efficiently than plain pickle. It uses pickle underneath, so the same trust and version-compatibility caveats apply. Ship the library versions, the preprocessing, and a small set of known inputs and expected outputs.

```
import joblib, sklearn, sys

artefact = {
    "model": fitted_pipeline,
    "sklearn_version": sklearn.__version__,
    "python_version": sys.version.split()[0],
    "trained_on": "2026-08-19",
    "feature_order": feature_names,
    "metrics": {"auc": 0.874, "n_train": 412_000},
}
joblib.dump(artefact, "churn_v7.joblib", compress=3)
```

Saving a dict rather than the bare estimator is the point. The model alone tells the serving team nothing about what it expects or whether their environment matches yours.

`joblib` is preferred over plain pickle here because it stores large numeric arrays more efficiently, particularly for estimators holding big coefficient matrices. It is a wrapper over pickle, not a replacement, so unpickling still executes code and version coupling still applies. Never load a joblib file from an untrusted source.

Serialise the whole pipeline, not just the final estimator. If the scaler and encoder were fitted on training data, their fitted state must travel with the model, or serving refits them on different data and the features no longer match what the model learned. That is the most common way a model that validated well performs poorly in production.

`feature_order` matters more than it looks. Most estimators take a positional array, so a serving path that assembles features in a different order produces confident nonsense with no error at all.

Two operational habits. A version mismatch warning on load is not noise; scikit-learn says explicitly that cross-version loading is unsupported, and the object may work while behaving differently. Match the version or retrain.

And ship a handful of input rows with their expected predictions. The serving team runs them after loading, and a mismatch tells them immediately that something in the environment or the feature assembly is wrong, before any traffic reaches it.

**Likely follow-ups**

- The serving team gets a warning about version mismatch — do you ignore it?
- Why joblib rather than plain pickle for this?
- How would you verify the loaded model is the one you trained?

---

## 53. What notebook magics do you actually use, and what would you reach for to find a slow cell?

*Easy · Occasional*

**Short answer.** `%timeit` for a single expression and `%%timeit` for a whole cell, `%%time` for one untimed run, `%prun` for a function-level profile, and `%debug` to open a post-mortem debugger after an exception. `%autoreload` picks up edits to imported modules without restarting.

The distinction to have ready: a single `%` is a line magic applying to the rest of that line, and `%%` is a cell magic applying to the entire cell and must be the first line of it.

```
%timeit compute_ndcg(scores, labels)      # repeats, reports spread
%%time                                     # one run, wall and CPU
build_index(corpus)
```

`%timeit` loops and reports the best of several runs, which is the honest measurement. `%time` runs once and is what you want when the operation is slow enough that repeating it is impractical.

For finding where the time goes inside a call:

```
%prun -l 15 train_ranker(features, labels)
```

That gives the cProfile output inline, sorted by cumulative time, without leaving the notebook. `%lprun` from line_profiler goes further and reports per line, after `%load_ext line_profiler`.

`%debug` is the one people underuse. Run it in a fresh cell straight after a traceback and it drops you into a post-mortem debugger at the frame where the exception occurred, with all the locals intact. You can inspect the variable that was None without rerunning a twenty-minute cell.

`%autoreload` matters once your logic lives in imported modules:

```
%load_ext autoreload
%autoreload 2
```

Edits to imported files take effect on the next call with no kernel restart. It has real limits: it does not reliably pick up new class attributes, changes to decorators, or module-level constants already bound, so when behaviour looks stale, restart before debugging further.

Two others worth naming. `%who` and `%whos` list the variables currently in memory, which is a quick way to see the hidden state accumulated in a long session. And `!` runs a shell command, so `!pip list | grep scikit` checks the environment without leaving the notebook.

**Likely follow-ups**

- What's the difference between the single-percent and double-percent forms?
- Why would %autoreload not pick up your change?
- Where does %debug get you after a traceback?

---

More Data Scientist sets: https://codeayan.com/get-hired/data-scientist
All interview prep: https://codeayan.com/get-hired