Docsity
Docsity
Prepare for your exams
Prepare for your exams

Study with the several resources on Docsity

Earn points to download
Earn points to download

Earn points by helping other students or get them with a premium plan

Guidelines and tips
Guidelines and tips
Sell on Docsity
Sell on Docsity
Docsity AI
Docsity AI
Log inSign up

Prepare for your exams

Study with the several resources on Docsity

Find documents

Prepare for your exams with the study notes shared by other students like you on Docsity

Search for your university

Find the specific documents for your university's exams

Docsity AINEW

Summarize your documents, ask them questions, convert them into quizzes and concept maps

Explore questions

Clear up your doubts by reading the answers to questions asked by your fellow students

Earn points to download

Earn points by helping other students or get them with a premium plan

Share documents
download point icon20 Points

For each uploaded document

Answer questions
download point icon5 Points

For each given answer (max 1 per day)

All the ways to get free points
Get points immediately

Choose a premium plan with all the points you need

Study Opportunities

Choose your next study program

Get in touch with the best universities in the world. Search through thousands of universities and official partners

Community

Ask the community

Ask the community for help and clear up your study doubts

Free resources

Our save-the-student-ebooks!

Download our free guides on studying techniques, anxiety management strategies, and thesis advice from Docsity tutors

ISYE 6501 - Midterm 2 Study Guide, Study Guides, Projects, Research of Operational Research

Cambridge Technical InstituteOperational Research

ISYE 6501 - Midterm 2 Study Guide

Typology: Study Guides, Projects, Research

2025/2026

Available from 05/03/2026

mariebless0
mariebless0 šŸ‡ŗšŸ‡ø

3.9

(7)

3.2K documents

1 / 22

Toggle sidebar

This page cannot be seen from the preview

Don't miss anything!

bg1
ISYE 6501 - Midterm 2 Study Guide
when might overfitting occurs
- when the # of factors are close to or larger than the # of data points
causing the model to potentially fit too closely to random effects
Why are simple models better than complex ones
- less data is required; less chance of insignificant factors and easier to
interpret
what is forward selection
- we select the best new factor and see if it's good enough (R^2, AIC, or p-
value) add it to our model and fit the model with the current set of factors.
Then at the end we remove factors that are lower than a certain threshold
what is backward elimination
- we start with all factors and find the worst on a supplied threshold (p =
0.15). If it is worse we remove it and start the process over. We do that until
we have the number of factors that we want and then we move the factors
lower than a second threshold (p =
.05) and fit the model with all set of factors
what is stepwise regression
- it is a combination of forward selection and backward elimination. We can
either start with all factors or no factors and at each step we remove or
add a factor. As we go through the procedure after adding each new factor
and at the end we eliminate right away factors that no longer appear.
what type of algorithms are stepwise selection?
- Greedy algorithms - at each step they take one thing that looks best
what is LASSO
- a variable selection method where the coefficients are determined by both
minimizing the squared error and the sum of their absolute value not being
over a certain threshold
How do you choose t in LASSO
- use the lasso approach with different values of t and see which gives the
best trade off
why do we have to scale the data for LASSO
-if we don't the measure of the data will artificially affect how big the
coefficients need to be
1
pf3
pf4
pf5
pf8
pf9
pfa
pfd
pfe
pff
pf12
pf13
pf14
pf15
pf16

Partial preview of the text

Download ISYE 6501 - Midterm 2 Study Guide and more Study Guides, Projects, Research Operational Research in PDF only on Docsity!

ISYE 6501 - Midterm 2 Study Guide

when might overfitting occurs

  • when the # of factors are close to or larger than the # of data points causing the model to potentially fit too closely to random effects Why are simple models better than complex ones
  • less data is required; less chance of insignificant factors and easier to interpret what is forward selection
  • we select the best new factor and see if it's good enough (R^2, AIC, or p- value) add it to our model and fit the model with the current set of factors. Then at the end we remove factors that are lower than a certain threshold what is backward elimination
  • we start with all factors and find the worst on a supplied threshold (p = 0.15). If it is worse we remove it and start the process over. We do that until we have the number of factors that we want and then we move the factors lower than a second threshold (p = .05) and fit the model with all set of factors what is stepwise regression
  • it is a combination of forward selection and backward elimination. We can either start with all factors or no factors and at each step we remove or add a factor. As we go through the procedure after adding each new factor and at the end we eliminate right away factors that no longer appear. what type of algorithms are stepwise selection?
  • Greedy algorithms - at each step they take one thing that looks best what is LASSO
  • a variable selection method where the coefficients are determined by both minimizing the squared error and the sum of their absolute value not being over a certain threshold How do you choose t in LASSO
  • use the lasso approach with different values of t and see which gives the best trade off why do we have to scale the data for LASSO -if we don't the measure of the data will artificially affect how big the coefficients need to be

What is elastic net?

  • A variable selection method that works by minimizing the squared error and constraining the combination of absolute values of coefficients and their squares
  • When you can collect data quickly. When the data is representative and the amount of data is small compared to the whole population

Do you have to decide the sample size ahead of time for A/B tests

  • no, and we can run the hypothesis test anytime we want What is full factorial design
  • you test every combination and then use ANOVA to determine importance of each factor What is fractional factorial design
  • when you test a subset of the entire set of combinations What is a balanced design?
  • You test each choice the same # of times and each pair of choices the same # of times When is regression effective work well to determine important factors?
  • If there aren't significant interactions between the factors what is exploration?
  • focusing on getting more information; in this case, to determine with more certainty which ad is really the best what is exploitation
  • we're focused on getting immediate value; in this example, to show the add that seems to be doing best so far, because it seems to be most likely to be clicked. what is the multi-armed bandit approach and how does it balance exploration and exploitation.
  • We start with no info and have an equal probability of selecting each alternative. After performing some tests, we've gotten more information, so we can update the probabilities of each one being best and start assigning new tests according to those probabilities. We keep testing multiple alternatives; so, we're still doing exploration. But we make it more likely to pick the best ones so we're also doing exploitation What are some of the parameters in the multi-armed bandit approach
  • number of tests between recalculating probabilities; how to update the probabilities; and how to pick an alternative to test based on probabilities and/or expected values. For updating we can use Bayesian updates or estimate from the observed distribution What are common reasons that data sets are missing values?
    • a person accidentally types in the wrong value
  • a person did not want to reveal the true value
  • an automated system did not work correctly to record the value What are some examples of why there might be bias in missing data
  • Heart transplants: If there's a variable "date of death" it will be missing for patients still living and thus the missing data will naturally include more successful transplant cases What are three ways of dealing with missing data that don't require imputation
  • discard the data, use categorical variables to indicate missing data, estimate missing values What are the pros and cons of throwing away missing data
  • Pros: not potentially introducing errors; easy to implement Cons: don't want to lose to many data points; potential for censored or biased missing data What is the categorical variable approach
  • If the data is categorical, we just add another category "missing". With quantitative variables you include interactions variables between the categorical variable and other variables. Why wouldn't you want to fill in missing quantitative variabes with 0
  • It can lead to problems if some types of data points are more likely than others to have missing data. The coefficients of the other variables might be pulled in one direction or another to try to account for the missing data What are the advantages and disadvantages of imputing missing data with the mean, median (numeric) or mode (categorical) - Advantage: hedge against being too wrong and easy to compute Disadvantage: it can be biased imputation. Example people with high income less likely to answer survey and thus the mean/median will underestimate the missing value What are the advantages and disadvantages of using regression for imputation - It reduces or eliminates the problem of bias. Also gives better values for missing data Disadvantages: we have to build, validate and test a whole other model just to fill in the missing data and then we have to do it all over again to get the answer we want. Also we are using the same data twice: once for imputation and a second time to fit the model How does adding variability to a regression imputation compare to one without - without: more accurate on average but has less accurate variability with: it's less accurate on average but has more accurate variability

When should you not use imputation? - When more than 5% of the data is moving per factor

first and more later on What do q-q plots help visual - if two data sets follow the same distribution.

why are q-q plots sometimes better than statistical tests - sometimes the statistical test will lead us in the wrong direction because most points might match but may be bad matches at the ends what is the memoryless property - it doesn't matter what's happened in the past, all that matters is where we are now If the data fits exponential distribution is it memoryless? - Yes If a data is memoryless is it exponential - yes Which distributions are memoryless - poisson and exponential Can a distribution not be memoryless and still be exponential - no what are deterministic simulations - same inputs give the same outputs what are stochastic simulations? - when there is randomness what are continuous-time simulations? - When changes happen continuously. Example: chemical processes, propagations What are discrete-event simulatIons - changes happen at discrete time points. Example: call center simulations someone calls worker finishes talking to someone. what are the elements of simulation model? - entities, modules, actions, resources, decisions point, and statistical tracking what are entities - things that move through the simulation (bags, people, etc) what are modules - parts of process (queues, storage, etc) what are replications - number of runs of a simulation Why is it important to validate a simulation by comparing to real data as much as possible? - If the simulation isn't a good reflection of reality, then any insights we gain from studying the simulation might not be applicable in reality what do prescriptive simulations answer - what-if questions what's an example of heuristic optimization - what's the best buffer size to have at each step in the process

quality of a set of values for the variables, which we're trying to maximize or minimize what is the solution - values for each variable what is a feasible solution - variable values that satisfy all constraints

what is an optimal solution - feasible solution with the best objective value What do binary variables do - They allow for more-complex models what is the objective function in linear regression - trying to minimize the squared error what are the statistical variables and constants in linear regression - the data; the coefficients what are the variables and constants in optimization model for linear regression - the data is the constant and the coefficients are the variables What is the objective function in logistic regression - to minimize the prediction error What are the variables in logistic regression - the coefficients in soft and hard classification SVMs what are the variables - the coefficients what is the constraint in hard classification for SVMS - each observation has to be on the right side of the line What is the objective function for hard classification - maximize the margin for soft classification what is the objective function - minimize classification error and maximize the margin what is the objective function for a time series model - minimize prediction error what are the variables in k-means clustering - coordinate of cluster centers and if a point is part of certain cluster What are the constraints in k-means clustering - each data point is assigned to a cluster What is the objective function in k-means - minimize total distance from data points to their cluster centers What are the order of fastest to slowest optimization problems - linear programs, convex quadratic programs, convex programs, integer programs, general non-convex programs are convex optimization problems guaranteed to find optimal solution - yes

what is a linear program? - f(x) is a linear function; constraint set X is defined by linear equations and inequalities what is convex quadratic program - f(x) is a convex quadratic function. Minimize f(x) or Maximize -f(x). constraint set X is defined by linear equations and inequalities what is constraint set X defined by in linear programs - linear equations and inequalities what is a convex optimization problem - objective f(x) is concave (if maximizing) or convex (if minimizing). Constraint set X is a convex set what is constraint set X in a convex optimization program - a convex set what is a integer program - linear program plus some (or all) variables restricted to take only integer values; variables could be binary (either 0 or

what are the basic steps to solve an optimization problem - 1) Initialization: pick values for all the variables (they may be simple, bad and not satisfy all of the constraints) 2.) find an improving direction t and make a change in that direction of some amount called the step size (theta) 3.) repeat using the the old solution plus the improving direction times the step size What should you do if your problem is too hard - Use a heuristic: rule of thumb process. It is ually gives good solutions what are some common network models? - shortest path model - find quickest/shortest route from one place to another; assignment model - which worker gets which job to maximize workforce efficiency; maximum flow model - how much oil can flow through complex network of pipes what type of models are network models - linear program what are the constraints in network models - flow into node = flow out of node; flow on arc between min and max allowable what do we get without constraints in a network model if data is all integers - an optimal solution where all the variables have integer values True or false: Requiring some variables in a linear program to take integer values can make it take a lot longer to solve. - True Adding integer variables moves the model from a linear program, which usually solves very quickly, to an integer program, which sometimes takes a long time to solve.

Do optimization implicitly assume that we know all of the values of the input data exactly? - Optimization models treat all of the data as known exactly

what is a non-zero sum game - it's possible for the total benefit for everyone to be higher or lower depending on the decisions made. what is a zero sum game - whatever one side gets, the other side looses what is stable equilibrium - Neither station has incentive to change what is pure strategy - just one choice (gas station) What is mixed strategy - randomize decisions according to probabilities (rock, paper, scissors); If your opponent knows you're going to pick rock she'll pick paper and beat you every time. Instead, a randomized or mixed strategy, where you randomly pick each one with probability 1/3 is your best bet. what is perfect information - know all about everyone else's situation what is imperfect information - Neither one really knows the others profit margins exactly. And in still other situation, some people have more information than others, so it's not symmetric what do descriptive, predictive and prescriptive models assume - that the systems do not react what is the idea of deep learning - to train a system to react to whatever patterns our human brain is reacting to without knowing what it's reacting to. what are some areas where deep learning is useful - recognizing speech and writing, natural language processing, and image recognition. what are the down sides of neural networks? - often do't give the best results. They require lots of data to rain and it's often hard to chose and tune the learning algorithm so the weights don't change too slowly but also don't change so quickly that they jump all over the place. What are the three levels of neurons - input level, hidden level, and output level What is the process of neural networks? - * Each input neuron accepts a single piece of information

  • for example, if we are trying to solve the CAPTCHA problem we might divide the picture of a digit into pixels and every pixel's status between fully white and fully black goes to its own input neuron
  • As inputs come in, they're passed to the first level of hidden neurons. Each simulated neuron calculates a weighted value of those inputs and sends the result to neurons at the next level. Those next level neurons do the same thing. There may be several layers of hidden neurons one after another.
  • Eventually the output layer neurons get their inputs and each one uses those inputs to find their results.
  • Example: In CAPTCHA have one output neuron for each possible number of letters and each output neuron's result is like a level of certainty that the input image is that number or letter
  • In the end, the models predicted output is whoever output neuron has the highest result What are two situations when nonparametric are useful? - * we don't know much about the form of the underlying distribution that data comes from, or it doesn't fit a nice distribution
  • it's important to have information about the median
  • Many nonparametric tests focus on the median, and they can be used even when we do not know the form of the underlying distribution.
  • we don't have much data How can Bayesian models incorporate expert opinion when there's not as much data to analyze as we'd like to have? - Expert opinion can be used to define the initial distribution of P(A) , and observed data about can be used with Bayes' theorem to obtain a revised opinion P(A|B). The initial distribution assumed for P(A) is called the 'prior distribution' and the revised distribution P(A|B) is called the 'posterior distribution' when is it useful to empirical bayesian modeling - In the absence of lots of data What is the gist of the Bayesian approach - with a single observation combined with a broader set of observations we can make a deduction or prediction what is a clique - a set of nodes that all have edges between each other what is a comunity - a set of circles that's highly connected within itself what does the Louvian algorithm do - decompose a graph into communities How does the Louvian algorithm work - Step 0: starts with each node being its own community. Step1: Repeat ... make biggest modularity increase by moving a node from its community to an adacent node's community until no move increases modularity Step2: Each community is a super-node. Repeat step 1 using super nodes Suppose we have a graph where all edge weights are equal to 1. In the video, we saw how to split a graph up into highly-interconnected