AI Fundamentals Interview Guide: Metrics, Data Quality, and Production Judgment

AI fundamentals interviews for engineering roles emphasize judgment under imperfect data. Interviewers present skewed datasets, leaking features, and metric choices that optimize the wrong objective—then ask how you would detect and fix them before shipping.

Classification metrics translate directly to product decisions. Discuss when accuracy misleads, how threshold tuning affects precision and recall for fraud or support routing, and why ROC-AUC can hide poor performance on rare classes that matter most.

Data leakage appears subtly in temporal splits, duplicate users across train and test, and future information embedded in features. Candidates describe validation strategies—time-based splits, group splits—and how leakage manifests as mysteriously good offline metrics with production failure.

Bias-variance and overfitting conversations should connect to regularization, early stopping, and monitoring generalization gaps in live traffic. Strong answers bridge classical ML intuition to modern LLM eval slices and drift detection.