Download Machine Learning Project Structuring: Certificate Practice Exam and more Exams Technology in PDF only on Docsity!
Certificate Practice Exam
Question 1. What is the primary purpose of defining a single-number evaluation metric in a machine learning project? A) To simplify code implementation B) To accelerate model comparison and iteration C) To reduce the size of the dataset D) To eliminate the need for a validation set Answer: B Explanation: A single-number metric provides a quick, consistent way to compare different models, enabling faster iteration cycles. Question 2. In the context of ML project metrics, what is a guardrail metric? A) A metric used only during training B) A satisficing metric that must meet a minimum threshold C) The primary metric to be maximized D) A metric that measures computational cost Answer: B Explanation: Guardrail metrics are satisficing metrics that ensure the model meets essential safety or performance constraints before optimizing the main metric. Question 3. Which of the following best describes “optimizing” metrics? A) Metrics that must stay above a fixed value B) Metrics that the model aims to maximize or minimize C) Metrics used only for hyperparameter tuning D) Metrics that are ignored during model selection Answer: B
Certificate Practice Exam
Explanation: Optimizing metrics are the primary objectives that the training process seeks to improve, such as accuracy or loss. Question 4. When should you consider changing your dev and test sets? A) When the model reaches 100% training accuracy B) When the current splits no longer reflect the real‑world data distribution C) When the dataset size doubles D) When the model’s inference time exceeds a threshold Answer: B Explanation: If the dev/test sets stop representing the production environment, they no longer provide reliable performance estimates. Question 5. What is the typical proportion of a dataset allocated to the dev set for a large dataset (≥ 1 M examples)? A) 0.1% B) 1% C) 5% D) 20% Answer: B Explanation: For very large datasets, a small fraction (around 1%) is sufficient to obtain reliable validation statistics. Question 6. Human‑Level Performance (HLP) is used as a proxy for which concept? A) Training loss B) Bayes error C) Overfitting
Certificate Practice Exam
C) Reduce model capacity or increase regularization D) Change the optimizer Answer: B Explanation: Large bias indicates underfitting; increasing model capacity or reducing regularization can help reduce bias. Question 10. What does orthogonalization refer to in model development? A) Training two models on the same data B) Adjusting a model to improve one metric without harming another C) Using orthogonal vectors in feature space D) Ensuring training and test sets are orthogonal Answer: B Explanation: Orthogonalization means improving a specific aspect (e.g., training performance) while keeping other aspects (e.g., generalization) unchanged. Question 11. During error analysis, why is it useful to calculate the ROI for fixing a specific error category? A) To estimate the monetary cost of labeling data B) To prioritize fixes that will yield the biggest performance gain per effort C) To determine the number of epochs needed for training D) To decide the learning rate schedule Answer: B Explanation: ROI quantifies expected performance improvement versus effort, helping focus on the most impactful error categories. Question 12. If the dev set contains mislabeled examples, what is the most appropriate immediate action?
Certificate Practice Exam
A) Retrain the model from scratch B) Remove the mislabeled examples from the dev set C) Increase the learning rate D) Add more layers to the network Answer: B Explanation: Mislabels in the dev set corrupt performance estimates; correcting them restores reliable validation. Question 13. Which learning paradigm is most suitable when you have a large labeled dataset for a single task? A) Transfer learning B) Multi‑task learning C) End‑to‑end deep learning D) Reinforcement learning Answer: C Explanation: End‑to‑end deep learning can learn directly from raw data when abundant labeled examples are available for a single task. Question 14. When is transfer learning likely to provide the greatest benefit? A) When the source and target tasks are unrelated B) When the target task has abundant labeled data C) When the source task has learned useful low‑level features similar to the target D) When the model architecture is a decision tree Answer: C Explanation: Transfer learning leverages previously learned representations that are relevant to the new task, especially when data is scarce.
Certificate Practice Exam
D) To increase the size of the training data Answer: B Explanation: The training‑dev set, sampled from the training distribution, helps differentiate high variance (model issue) from distribution mismatch. Question 18. Which of the following is a common technique to address training‑dev distribution mismatch? A) Adding dropout layers B) Manually adjusting the training data to resemble the dev distribution C) Reducing batch size to 1 D) Using a higher learning rate Answer: B Explanation: Aligning the training data distribution with the target (dev/test) distribution reduces mismatch. Question 19. When should you consider using data synthesis techniques? A) When you have excess labeled data B) When the model is overfitting C) When you need to augment under‑represented scenarios in the training set D) When the learning rate is too low Answer: C Explanation: Synthetic data can fill gaps in the training distribution, especially for rare but important cases. Question 20. Which of the following best describes the “first‑system‑quickly” approach? A) Building a complex pipeline before any evaluation
Certificate Practice Exam
B) Deploying a minimal viable model to obtain early feedback, then iterating based on error analysis C) Training the final model for the maximum number of epochs from the start D) Skipping validation to save time Answer: B Explanation: Starting with a simple baseline allows rapid identification of major error sources for focused improvements. Question 21. If a model’s dev error is dominated by a small set of high‑impact error categories, what is the most effective next step? A) Increase the number of hidden layers B) Collect more data for the dominant error categories C) Lower the learning rate D) Switch to a different optimizer Answer: B Explanation: Targeted data collection or labeling for the most problematic categories can dramatically reduce overall error. Question 22. What does a high “avoidable variance” indicate about a model? A) The model is too simple B) The model is overfitting the training data C) The model has insufficient training data or regularization D) The model’s architecture is unsuitable for the task Answer: C Explanation: High variance suggests the model fits training data well but fails to generalize, often due to limited data or insufficient regularization.
Certificate Practice Exam
Explanation: Early layers capture low‑level patterns (edges, textures) that are generally transferable; freezing them preserves learned representations and speeds up training. Question 26. Which of the following indicates that a model is suffering from high bias? A) Training accuracy 99%, dev accuracy 60% B) Training accuracy 60%, dev accuracy 58% C) Training loss continues to increase after many epochs D) Model predictions are highly variable across runs Answer: B Explanation: Low training accuracy close to dev accuracy suggests underfitting, a hallmark of high bias. Question 27. If a model’s performance on the test set is significantly worse than on the dev set, what is the most likely cause? A) Overfitting to the dev set (hyperparameter over‑tuning) B) Insufficient training epochs C) Too many training examples D) Incorrect loss function Answer: A Explanation: Excessive tuning on the dev set can lead to a model that does not generalize to unseen test data. Question 28. What is the main benefit of using a “dev‑test” split instead of a single validation set? A) Allows for parallel training on both sets B) Provides an unbiased final evaluation after model selection C) Reduces the need for data preprocessing
Certificate Practice Exam
D) Guarantees higher training accuracy Answer: B Explanation: The test set remains untouched during model development, giving an unbiased estimate of real‑world performance. Question 29. Which of the following is a typical sign that your dataset suffers from label noise? A) Training loss plateaus at zero B) Model achieves perfect accuracy on training but poor accuracy on dev C) Random fluctuations in loss despite a stable model architecture D) High variance in predictions for identical inputs Answer: C Explanation: Label noise introduces randomness that prevents loss from stabilizing even when the model structure is appropriate. Question 30. When performing error analysis, why is it important to quantify the potential performance gain from fixing each error category? A) To determine the exact number of epochs needed for training B) To allocate resources efficiently toward the most impactful fixes C) To decide which optimizer to use D) To compute the model’s memory footprint Answer: B Explanation: Estimating the gain helps prioritize work that yields the largest improvement per effort. Question 31. What does the term “data leakage” refer to in the context of ML project splits?
Certificate Practice Exam
Question 34. What is a common sign that a model would benefit from a larger capacity (e.g., more layers or units)? A) Training loss fails to decrease after many epochs B) Training loss rapidly reaches zero while dev loss remains high C) Model training takes longer than expected D) Model has high inference latency Answer: A Explanation: If the model cannot reduce training loss, it is likely under‑parameterized, indicating a need for greater capacity. Question 35. Why might you choose an end‑to‑end deep learning approach over a modular pipeline? A) When you have limited computational resources B) When you possess a massive labeled dataset and want the model to learn feature extraction automatically C) When interpretability of intermediate steps is crucial D) When the problem is easily solved with linear regression Answer: B Explanation: End‑to‑end models excel when abundant data allows the network to learn all necessary transformations internally. Question 36. Which of the following is a potential drawback of using a very large guardrail metric threshold (e.g., extremely low latency requirement)? A) It may force the model to be overly complex B) It could limit the choice of architectures, reducing achievable performance on the main metric
Certificate Practice Exam
C) It guarantees higher accuracy D) It eliminates the need for a dev set Answer: B Explanation: Strict guardrails can constrain model design, potentially sacrificing performance on the primary objective. Question 37. When analyzing variance, what does a small gap between training and training‑dev errors suggest? A) High bias B) Low variance, but possible data mismatch C) Overfitting to the training set D) Incorrect loss function Answer: B Explanation: If training‑dev (same distribution as training) error is close to training error, variance is low; a larger dev‑test gap indicates distribution mismatch. Question 38. If the dev set is much smaller than the training set, what risk increases? A) Underfitting B) Overfitting to the dev set during hyperparameter tuning C) Data leakage D) Gradient explosion Answer: B Explanation: A tiny dev set provides noisy estimates, leading developers to over‑tune to its idiosyncrasies. Question 39. Which of the following is an example of a “hard” error category during error analysis?
Certificate Practice Exam
Explanation: Covariate shift occurs when P(X) changes between training and test, but P(Y|X) remains unchanged. Question 42. When using transfer learning, why might you fine‑tune the later layers of the pretrained network? A) Later layers capture task‑specific features that need adaptation to the new domain B) Later layers contain generic edge detectors C) Fine‑tuning later layers reduces training time D) Later layers are always linear Answer: A Explanation: The deeper layers encode higher‑level concepts that are more specific to the original task; adapting them helps the model align with the target task. Question 43. Which of the following best describes “early stopping” as a regularization technique? A) Stopping training when the learning rate reaches zero B) Halting training when dev loss stops improving for a set number of epochs to prevent overfitting C) Ending training after a fixed number of epochs regardless of performance D) Stopping training when training loss reaches zero Answer: B Explanation: Early stopping monitors dev performance and stops training before the model begins to overfit. Question 44. If a model’s performance improves when you add more training data that is similar to the dev set, what does this suggest? A) The model suffers from high bias
Certificate Practice Exam
B) The model suffers from high variance or data mismatch C) The optimizer is misconfigured D) The loss function is inappropriate Answer: B Explanation: Adding data that better matches the dev distribution reduces variance or addresses mismatch, leading to improved performance. Question 45. Which of the following is a reason to keep the test set completely untouched until final evaluation? A) To use it for hyperparameter tuning B) To ensure an unbiased estimate of real‑world performance C) To increase training speed D) To reduce overfitting on the training set Answer: B Explanation: The test set must remain unseen during development to provide an unbiased performance metric. Question 46. When might you prefer a pipeline architecture (multiple components) over an end‑to‑end model? A) When you have a small amount of labeled data and want to leverage domain‑specific preprocessing B) When you want the model to learn all features automatically C) When you aim for the highest possible inference speed D) When you have unlimited computational resources Answer: A Explanation: Pipelines allow incorporation of handcrafted features or external knowledge, which can be valuable when data is limited.
Certificate Practice Exam
Answer: B Explanation: Adjusting a model to improve one metric (training loss) at the expense of another (dev loss) shows a lack of orthogonalization. Question 50. When using multi‑task learning, how can you detect negative transfer? A) All tasks improve simultaneously B) One or more tasks degrade in performance compared to single‑task baselines C) Training loss becomes zero for all tasks D) Model size decreases Answer: B Explanation: Negative transfer occurs when sharing representations harms performance on one or more tasks relative to training them separately. Question 51. What is the primary reason to set a minimum size for the test set (e.g., at least 5 % of total data)? A) To ensure the test set can be used for training later B) To obtain statistically reliable performance estimates C) To speed up model training D) To reduce overfitting on the test set Answer: B Explanation: A sufficiently large test set reduces variance in performance estimates, making them trustworthy. Question 52. If a model’s inference latency exceeds the guardrail threshold, which of the following is the most direct remedy? A) Increase the batch size during training B) Replace the model with a smaller architecture or apply model compression techniques
Certificate Practice Exam
C) Add more layers to improve accuracy D) Collect more training data Answer: B Explanation: Reducing model size or applying pruning/quantization directly lowers inference time. Question 53. Which of the following is a typical sign of label imbalance affecting model performance? A) The model predicts the majority class for most inputs B) Training loss rapidly diverges C) The model achieves perfect accuracy on the minority class D) The optimizer fails to converge Answer: A Explanation: Imbalanced labels often cause the model to bias toward the majority class, leading to poor recall on minority classes. Question 54. When should you consider using a “validation‑in‑the‑loop” approach (i.e., evaluating on dev after each epoch)? A) When training time is negligible B) When you need to monitor for overfitting and apply early stopping C) When you have a single massive dataset with no splits D) When you are only interested in training loss Answer: B Explanation: Frequent dev evaluation enables detection of overfitting and triggers early stopping. Question 55. What does the term “Bayes error” refer to?
