As asked
Write a function to detect whether a given string comparison function is constant-time by measuring its execution time across multiple inputs. How would you statistically analyze the results to determine if a timing oracle exists?
Sample answer outline
A strong answer implements high-resolution timing around the comparison function, runs hundreds of trials per input pair (same prefix length, different suffix), groups measurements by prefix length, and applies a t-test or Mann-Whitney U test to detect statistically significant timing differences between groups. The candidate should discuss measurement noise from CPU caches, scheduler jitter, and why many trials are required, and explain that a p-value below 0.01 would indicate a timing oracle.
Reference implementation (python)
import time
import statistics
def measure_comparison(fn, a, b, trials=500):
times = []
for _ in range(trials):
start = time.perf_counter_ns()
fn(a, b)
times.append(time.perf_counter_ns() - start)
return times
def unsafe_compare(a, b):
return a == b
times_match = measure_comparison(unsafe_compare, "secret", "secret")
times_miss = measure_comparison(unsafe_compare, "secret", "xxxxxx")
print(statistics.mean(times_match), statistics.mean(times_miss))
# TODO: apply t-test to determine if difference is significantExpect these follow-ups
- How does the Welch t-test differ from the Mann-Whitney U test for detecting timing differences, and when would you choose one over the other given non-normal measurement distributions?