Loading your deck…
A new card after five completed ratings.
Open a card to practice. EDIT changes its content. DELETE removes it from study.
What is a population in a statistical study?
The complete group about which you want to draw conclusions. It is not necessarily the group actually observed.
What is a sample?
The subset of the population that is observed. How it is selected affects which population conclusions are justified.
What is an observational unit?
The entity being measured, such as one person. Multiple rows from the same person are not automatically independent observations.
Explain the difference between a parameter, a statistic, and an estimate.
A parameter describes a population or model, such as \(\mu\).
A statistic is a function of the sample that does not use unknown parameters, such as \(\bar X\).
An estimate is its observed numerical value, such as \(\bar x=12\).
What is a feature in a predictive model?
An input used to predict an outcome, such as age or flight distance.
What is the target in supervised learning?
The outcome the model is trained to predict. It is supplied as a label during training.
What is supervised learning?
Learning a prediction rule from examples containing both inputs and known targets.
What is unsupervised learning?
Learning structure from inputs without supplied prediction targets, such as finding clusters.
Explain the difference between nominal, ordinal, discrete numerical, and continuous variables.
Nominal: unordered categories, such as device type.
Ordinal: ordered categories, such as low, medium, high.
Discrete numerical: countable values, such as purchase counts.
Continuous: values modeled on a continuum, such as elapsed time.
Explain the difference between the mean and median as summaries of a distribution.
The mean is \(\bar x=n^{-1}\sum_i x_i\). A median places at least half the observations on each side.
The mean minimizes squared deviations. A median minimizes absolute deviations.
The median is more resistant to extreme values. Neither summary alone describes an entire distribution.
What is the sample variance?
For \(n>1\), \[s^2=\frac{1}{n-1}\sum_{i=1}^n(x_i-\bar x)^2.\] It measures squared spread around the sample mean.
What is standard deviation?
The square root of variance. It measures spread in the same units as the observations.
What is the IQR (interquartile range)?
The IQR (interquartile range) is \(Q_{0.75}-Q_{0.25}\), the width of the middle half of the distribution. It is less sensitive to extreme values than standard deviation.
What does a population quantile represent?
The \(u\)-quantile is the smallest threshold at which the cumulative probability reaches \(u\). A median is a \(0.5\)-quantile. This definition also applies to distributions with jumps.
What does the \(1.5\) IQR (interquartile range) rule flag?
Values below \(Q_{0.25}-1.5(Q_{0.75}-Q_{0.25})\) or above \(Q_{0.75}+1.5(Q_{0.75}-Q_{0.25})\). A flagged value needs investigation, not automatic deletion.
How do you standardize a feature without leaking validation information?
Compute the training mean \(\bar x_{\rm tr}\) and standard deviation \(s_{\rm tr}>0\), then transform every split using \[ z=\frac{x-\bar x_{\rm tr}}{s_{\rm tr}}. \] Standardization changes units. It does not make a distribution normal. Handle constant features separately.
Which supports population generalization, and which supports causal comparison?
Random sampling selects units from a population and supports population inference with the sampling design accounted for.
Random assignment allocates treatment and supports causal comparison.
A randomized experiment on a convenience sample need not generalize to the whole population.
What is selection bias?
Distortion caused by who enters the analyzed sample. Collecting more data through the same biased selection process need not remove it.
What is survivorship bias?
Selection bias from examining only cases that remain observable. Studying only firms still operating omits failures and can overstate typical performance.
Why can an association between treatment and outcome fail to be causal?
A third variable may influence both treatment assignment and the outcome.
For example, prior skill can affect both training enrollment and later performance.
Randomization breaks this link in expectation. Observational adjustment requires appropriate measured covariates and additional causal assumptions.
Can a pooled association have the opposite direction from every subgroup association?
Yes. Different subgroup weights can reverse the pooled association.
For example, a treatment may be used more often among high-risk patients and appear worse overall despite better outcomes within risk groups.
The correct comparison depends on the question and causal structure, not a blanket rule to always pool or stratify.
Compare MCAR (missing completely at random), MAR (missing at random), and MNAR (missing not at random).
MCAR (missing completely at random): missingness is independent of data values.
MAR (missing at random): given observed values, missingness is independent of missing values.
MNAR (missing not at random): dependence on missing values remains after conditioning on observed values.
What is the main risk of dropping every row with missing data?
It can change which population the sample represents, as well as reduce sample size. Whether it introduces bias depends on the missingness process and the analysis.
Why can replacing missing values with the mean be misleading?
It can distort variation and relationships between variables. A single filled-in value also hides uncertainty about the missing observation.
What is data leakage?
Using information that would not be available at prediction time or is outside the permitted training data. It makes evaluation look better than the actual deployment task.
Why is selecting features using all labels before cross-validation leakage?
The held-out labels have already influenced feature selection. Feature selection must be fitted inside each training fold.
Why can a table join silently invalidate a data analysis?
A many-to-many join can multiply rows, changing weights and apparent sample size.
Check key uniqueness, expected join cardinality, row counts, and unmatched keys.
Aggregate at the intended unit of analysis. Do not treat duplicated or repeated measurements as new independent people.
How do you calculate the probability that an event does not happen?
\[P(A^c)=1-P(A).\] The event and its complement partition all possible outcomes.
What is the probability of the entire sample space?
\(P(\Omega)=1\). Probabilities are nonnegative, so an event probability lies between zero and one.
How do probabilities combine for disjoint events?
Add them. For pairwise disjoint events \(A_1,A_2,\ldots\), \[P\!\left(\bigcup_i A_i\right)=\sum_iP(A_i).\] Disjoint means the events cannot occur together.
How do you compute \(P(A\cup B)\)?
\[P(A\cup B)=P(A)+P(B)-P(A\cap B).\] Subtract the intersection because it was counted twice.
Explain the difference between disjoint and independent events.
Disjoint events cannot happen together. Independent events do not change one another’s probabilities. Two disjoint events of positive probability are not independent.
What is \(P(A\mid B)\)?
For \(P(B)>0\), \[P(A\mid B)=\frac{P(A\cap B)}{P(B)}.\] The reference population is restricted to outcomes in \(B\).
How can conditional probability be used to calculate \(P(A\cap B)\)?
For \(P(B)>0\), \[P(A\cap B)=P(A\mid B)P(B).\] Independence is not required.
How do you combine conditional probabilities across a partition?
If \(B_1,\ldots,B_k\) partition the sample space and each has positive probability, then \[ P(A)=\sum_{j=1}^kP(A\mid B_j)P(B_j). \] This is a weighted average. The weights are subgroup probabilities, not necessarily equal.
How do you reverse a conditional probability?
For \(P(A),P(B)>0\), \[ P(B\mid A)=\frac{P(A\mid B)P(B)}{P(A)}. \] With hypotheses \(B_j\), obtain \(P(A)\) from total probability.
Posterior odds equal prior odds times the likelihood ratio.
An event has prevalence \(1\%\). A flag has \(90\%\) sensitivity and \(95\%\) specificity. What is \(P(\text{event}\mid\text{flag})\)?
By Bayes' theorem, \[ \frac{0.90(0.01)}{0.90(0.01)+0.05(0.99)} =\frac{2}{13}\approx15.4\%. \] The low prevalence makes false flags common relative to true flags, despite the apparently strong sensitivity and specificity.
When are two events independent?
When \(P(A\cap B)=P(A)P(B)\). If \(P(B)>0\), this is equivalent to \(P(A\mid B)=P(A)\).
What does independence of random variables mean?
Their joint probabilities factor for all measurable sets. For example, \(P(X\in A,Y\in B)=P(X\in A)P(Y\in B)\) for every such \(A,B\).
Does pairwise independence imply mutual independence?
No. Independence of every pair does not guarantee independence of the full collection.
What is a PMF (probability mass function)?
A PMF (probability mass function) gives \(p(x)=P(X=x)\) for a discrete random variable. Its values are nonnegative and sum to one.
What is a PDF (probability density function)?
A PDF (probability density function) gives probabilities by integration: \[P(a<X\le b)=\int_a^b f(x)\,dx.\] Density values may exceed one. They are not point probabilities.
What is a CDF (cumulative distribution function)?
The CDF (cumulative distribution function) is \(F(x)=P(X\le x)\). It exists for every real-valued random variable, discrete or continuous.
How do you calculate the expectation of a discrete random variable?
\[\mathbb E[X]=\sum_x xP(X=x)\] when the expectation exists. It is a probability-weighted average.
How do you calculate the expectation using a density?
\[\mathbb E[X]=\int x f(x)\,dx\] when the expectation exists. The density supplies the weights.
What is the expectation of an event indicator?
\[\mathbb E[\mathbf1_A]=P(A).\] It equals one on \(A\) and zero otherwise.
Does linearity of expectation require independence?
No. For integrable random variables, \[\mathbb E\!\left[\sum_iX_i\right]=\sum_i\mathbb E[X_i].\] This holds for a finite sum even when the variables depend on one another.
What is the variance of a random variable?
For a finite second moment, \[\operatorname{Var}(X)=\mathbb E[(X-\mathbb E X)^2].\] Equivalently, it is \(\mathbb E[X^2]-(\mathbb E X)^2\).
What is \(\operatorname{Var}(aX+b)\)?
\[\operatorname{Var}(aX+b)=a^2\operatorname{Var}(X).\] Adding a constant does not change spread. Multiplying by \(a\) multiplies standard deviation by \(|a|\).
What does covariance measure?
Linear co-movement around two means. For finite second moments, \[\operatorname{Cov}(X,Y)=\mathbb E[XY]-\mathbb E[X]\mathbb E[Y].\] A positive value means the centered variables tend to move together.
What is Pearson correlation?
Covariance standardized by both standard deviations: \[\rho=\frac{\operatorname{Cov}(X,Y)}{\sigma_X\sigma_Y}.\] Both variances must be finite and positive.
Does zero correlation imply independence?
Not generally. Independence implies zero covariance when second moments exist, but the converse fails. It does hold for jointly Gaussian variables.
What is the variance of \(aX+bY\) when the second moments are finite?
\[ \begin{aligned} \Var(aX+bY)={}&a^2\Var(X)+b^2\Var(Y)\\ &+2ab\Cov(X,Y). \end{aligned} \] The last term vanishes for uncorrelated variables, in particular for independent variables with finite variances.
What is the law of total expectation?
For integrable \(Y\), \[\mathbb E[Y]=\mathbb E[\mathbb E(Y\mid X)].\] Average the subgroup means with their probabilities to recover the overall mean.
What is the law of total variance?
For square-integrable \(Y\), \[\operatorname{Var}(Y)=\mathbb E[\operatorname{Var}(Y\mid X)]+\operatorname{Var}(\mathbb E[Y\mid X]).\] Overall variation is average within-group variation plus variation between group means.
What does the law of large numbers say about the sample mean?
If \(X_i\) are i.i.d. (independent and identically distributed) and \(\E|X_1|<\infty\), then \[ \bar X_n\xrightarrow{P}\mu=\E X_1. \] The average stabilizes. The individual observations retain their original distribution.
Convergence does not guarantee a particular finite-sample accuracy.
What becomes approximately normal in the classical central limit theorem?
If \(X_i\) are i.i.d. (independent and identically distributed) with mean \(\mu\) and variance \(0<\sigma^2<\infty\), then \[ \frac{\sqrt n(\bar X_n-\mu)}{\sigma} \xrightarrow{d}N(0,1). \] It is the standardized sample mean, not the raw data, that approaches normality. No universal sample size guarantees a good approximation.
What does a Bernoulli random variable model?
A single binary outcome: \(X=1\) with probability \(p\) and \(X=0\) with probability \(1-p\), where \(0\le p\le1\).
What is the mean of a Bernoulli\((p)\) random variable?
\(\mathbb E[X]=p\). The average of repeated binary outcomes estimates the success probability.
What is the variance of a Bernoulli\((p)\) random variable?
\(\operatorname{Var}(X)=p(1-p)\). It is zero at \(p=0\) and \(p=1\).
When does a success count follow a binomial distribution?
When it counts successes in a fixed number \(n\) of independent binary trials with the same success probability \(p\). The possible counts are \(0,1,\ldots,n\).
For \(X\sim\operatorname{Binomial}(n,p)\), what is \(P(X=k)\)?
\[P(X=k)=\binom nk p^k(1-p)^{n-k},\quad k=0,\ldots,n.\] The binomial coefficient counts arrangements of the \(k\) successes.
What is the mean of \(X\sim\operatorname{Binomial}(n,p)\)?
\(\mathbb E[X]=np\). It is the sum of the means of \(n\) Bernoulli trials.
What is the variance of \(X\sim\operatorname{Binomial}(n,p)\)?
\(\operatorname{Var}(X)=np(1-p)\). This uses the independence of the trials.
For \(X\sim\operatorname{Binomial}(10,0.2)\), calculate \(P(X=2)\).
\[P(X=2)=\binom{10}{2}(0.2)^2(0.8)^8\approx0.3020.\]
For \(X\sim\operatorname{Binomial}(10,0.2)\), calculate \(P(X\ge1)\).
Use the complement: \[P(X\ge1)=1-P(X=0)=1-0.8^{10}\approx0.8926.\]
What does a geometric trial-count random variable measure?
The number of independent Bernoulli trials through the first success, including the successful trial. The success probability \(p>0\) is the same on every trial. Its values start at one.
If \(T\) counts trials through the first success with probability \(p\), what is \(P(T=k)\)?
\[P(T=k)=(1-p)^{k-1}p,\quad k=1,2,\ldots.\] The first \(k-1\) trials fail and trial \(k\) succeeds.
What is the expected number of independent trials through the first success with probability \(p>0\)?
\[\mathbb E[T]=\frac1p.\] For \(p=0.2\), the expected wait is five trials.
What is the variance of a geometric trial count with success probability \(p>0\)?
\[\operatorname{Var}(T)=\frac{1-p}{p^2}.\] The successful trial is included in \(T\).
Explain the difference between counting geometric trials and geometric failures.
If \(T\) includes the first successful trial, the failure count is \(F=T-1\). Thus \(T\) starts at one and \(F\) starts at zero. Subtracting one changes the mean but not the variance.
What is the probability of exactly \(k\) failures before the first success with probability \(p\)?
\[P(F=k)=(1-p)^k p,\quad k=0,1,\ldots.\] Every failure comes before the first success.
What is the expected number of failures before the first success?
For success probability \(p>0\), \[\mathbb E[F]=\frac{1-p}{p}.\] This is \(1/p-1\), because the successful trial is excluded.
What does the memoryless property mean for a geometric trial count?
For \(0<p<1\) and integers \(m,n\ge0\), \[ P(T>m+n\mid T>m)=(1-p)^n=P(T>n). \] Reason: divide \((1-p)^{m+n}\) by \((1-p)^m\).
After failed trials, the remaining waiting time has the original distribution.
What does the negative binomial distribution model in the failure-count convention?
The number \(F\) of failures before the \(r\)th success in independent trials with a fixed success probability \(p\). The count starts at zero.
What is the mean number of failures before the \(r\)th success?
For success probability \(p>0\), \[\mathbb E[F]=\frac{r(1-p)}p.\] It is the sum of the failure counts before \(r\) successive successes.
What is the variance of the failure count before the \(r\)th success?
For independent trials with success probability \(p>0\), \[\operatorname{Var}(F)=\frac{r(1-p)}{p^2}.\]
What does the parameter of a Poisson count represent?
\(\lambda\) is the expected count over a specified exposure. A rate of \(r\) per unit time over duration \(t\) gives \(\lambda=rt\).
For \(X\sim\operatorname{Poisson}(\lambda)\), what is \(P(X=k)\)?
\[P(X=k)=e^{-\lambda}\frac{\lambda^k}{k!},\quad k=0,1,\ldots.\] Here \(\lambda>0\).
What is the relationship between the mean and variance of a Poisson distribution?
They are equal: \(\mathbb E[X]=\operatorname{Var}(X)=\lambda\). This equality is a defining model restriction, not a fact about all count data.
What is the count distribution over a duration in a homogeneous Poisson process?
At rate \(r>0\), a duration \(t\) has count \(N(t)\sim\operatorname{Poisson}(rt)\). Disjoint time intervals have independent counts.
What is the waiting-time distribution in a homogeneous Poisson process?
Successive waiting times are independent exponential random variables with the process rate \(r\). Their common mean is \(1/r\).
Events follow a Poisson process at \(3\) per hour. Find the probability of no events in half an hour.
The half-hour count has mean \(3(0.5)=1.5\). \[ P(N=0)=e^{-1.5}\approx0.2231. \] Using \(e^{-3}\) would ignore the exposure length. The count parameter and the rate are not interchangeable.
When can a binomial count be approximated by a Poisson count?
For large \(n\), small \(p\), and \(\lambda=np\) of moderate size, \[ \mathrm{Binomial}(n,p)\approx\mathrm{Poisson}(np). \] Formally, if \(p_n\to0\) and \(np_n\to\lambda\), the binomial distribution converges to \(\mathrm{Poisson}(\lambda)\).
Check accuracy in the tail used by the analysis.
How do you approximate \(P(X\le k)\) for a binomial count?
When both \(np\) and \(n(1-p)\) are sufficiently large, \[ X\approx N(np,np(1-p)). \] With continuity correction, \[ P(X\le k)\approx \Phi\!\left(\frac{k+0.5-np}{\sqrt{np(1-p)}}\right). \] Rules such as both counts at least \(10\) are heuristics, not universal error guarantees.
When do you use a hypergeometric count model?
When counting successes in a fixed-size random sample drawn without replacement from a finite population. Unlike independent binomial trials, the draws are dependent.
Among \(N\) items there are \(K\) successes. A random sample of size \(n\) is drawn without replacement. What is the probability of \(k\) successes?
\[P(X=k)=\frac{\binom Kk\binom{N-K}{n-k}}{\binom Nn}.\] Use only feasible values of \(k\). The numerator counts samples with the desired composition.
How does sampling without replacement affect the variance of a success count relative to a binomial count?
With \(p=K/N\) and \(N>1\), it multiplies \(np(1-p)\) by \((N-n)/(N-1)\). The count is less variable because the available population changes after each draw.
What is a categorical distribution?
A distribution for one outcome among \(K\) categories with probabilities \(p_1,\ldots,p_K\), where each is nonnegative and they sum to one. Bernoulli is the two-category case.
What does a multinomial distribution model?
The vector of category counts from \(n\) independent categorical trials with fixed class probabilities. The counts sum to \(n\), so they are not independent of one another.
What is the density of \(X\sim\operatorname{Uniform}(a,b)\), with \(a<b\)?
\(f(x)=1/(b-a)\) on \([a,b]\), and zero elsewhere. Equal-length subintervals of the support have equal probability.
What is the mean of \(\operatorname{Uniform}(a,b)\)?
\(\mathbb E[X]=(a+b)/2\), the midpoint of the interval.
What is the variance of \(\operatorname{Uniform}(a,b)\)?
\(\operatorname{Var}(X)=(b-a)^2/12\). It depends on interval length, not location.
What do \(\mu\) and \(\sigma^2\) represent in \(N(\mu,\sigma^2)\)?
\(\mu\) is the mean and \(\sigma^2\) is the variance. The second parameter is not the standard deviation.
How do you convert \(X\sim N(\mu,\sigma^2)\) to a standard normal variable?
For \(\sigma>0\), \[Z=\frac{X-\mu}{\sigma}\sim N(0,1).\] Subtract the mean and divide by the standard deviation.
What does the 68–95–99.7 rule say?
For a normal distribution, about 68%, 95%, and 99.7% of values lie within one, two, and three standard deviations of the mean, respectively. This is one named rule, not a guarantee for arbitrary distributions.
What standard-normal cutoff gives a central 95% interval?
Approximately \(1.96\), giving the interval from \(-1.96\) to \(1.96\). There is 2.5% in each tail.
Explain why a one-sided 5% normal test has a smaller positive cutoff than a two-sided 5% normal test.
The one-sided test puts the whole 5% in one tail, rather than 2.5% in each. The corresponding upper cutoffs are about \(1.645\) and \(1.96\).
For \(X\sim N(100,15^2)\), calculate \(P(X>130)\). You may use \(P(Z>2)=0.0228\).
Standardize: \((130-100)/15=2\). Therefore \(P(X>130)=0.0228\).
Why use Student’s t distribution when estimating a normal mean with unknown variance?
Replacing the unknown population standard deviation with a sample estimate adds uncertainty. The t distribution has heavier tails to account for it. For an independent normal sample of size n, the usual mean statistic has n minus one degrees of freedom.
How is a chi-square variable constructed?
The sum of squares of \(\nu\) independent standard normal variables has a chi-square distribution with \(\nu\) degrees of freedom.
What is the mean of \(\chi^2_\nu\)?
\(\nu\). Each squared standard normal contributes mean one.
What is the variance of \(\chi^2_\nu\)?
\(2\nu\). The independent squared standard normals contribute variances that add.
What kind of quantity has an F distribution?
A ratio of two independent chi-square quantities, each divided by its own degrees of freedom. This distribution is used to compare variation, for example in classical analysis of variance.
What does an exponential distribution commonly model?
The waiting time to the next event in a homogeneous Poisson process. Waiting times are nonnegative, with a constant hazard rate.
What is the exponential density with rate \(\lambda>0\)?
\(f(x)=\lambda e^{-\lambda x}\) for \(x\ge0\), and zero otherwise. Here \(\lambda\) is a rate, not a scale.
For exponential rate \(\lambda>0\), what is \(P(X>t)\) for \(t\ge0\)?
\[P(X>t)=e^{-\lambda t}.\] This is the probability the wait lasts longer than \(t\).
What is the mean waiting time for exponential rate \(\lambda>0\)?
\(\mathbb E[X]=1/\lambda\). A larger rate means a shorter expected wait.
What is the variance of an exponential waiting time with rate \(\lambda>0\)?
\(\operatorname{Var}(X)=1/\lambda^2\). Its standard deviation equals its mean.
What does memorylessness mean for an exponential waiting time?
For \(X\sim\mathrm{Exp}(\lambda)\), \(\lambda>0\), and \(s,t\ge0\), \[ P(X>s+t\mid X>s) =\frac{e^{-\lambda(s+t)}}{e^{-\lambda s}} =e^{-\lambda t}. \] This equals \(P(X>t)\). The future waiting time does not depend on how long one has already waited.
It is a modeling assumption, not a property of all waiting times.
A waiting time is exponential with mean five minutes. Calculate \(P(X>10)\).
The rate is \(1/5\) per minute, so \[P(X>10)=e^{-10/5}=e^{-2}\approx0.1353.\]
A waiting time is exponential with mean five minutes. Find its median.
Solve \(P(X>m)=e^{-m/5}=1/2\). This gives \(m=5\log2\approx3.466\) minutes.
How is a gamma distribution related to exponential waiting times?
With a positive integer shape \(a\), a gamma variable is the sum of \(a\) independent exponential waits with the same rate \(\lambda\). Shape one gives an exponential distribution.
What is the mean of a gamma distribution with shape a and rate λ?
\(\mathbb E[X]=a/\lambda\). Both shape and rate are positive. A scale parameter would instead be \(1/\lambda\).
What is the variance of a gamma distribution with shape a and rate λ?
\(\operatorname{Var}(X)=a/\lambda^2\). This formula uses the rate convention.
Why is the beta distribution useful for modeling an unknown probability?
It is supported between zero and one and has flexible shapes controlled by two positive parameters. It is often used as a prior for a Bernoulli success probability.
What is the mean of \(\operatorname{Beta}(a,b)\), with \(a,b>0\)?
\(\mathbb E[X]=a/(a+b)\). With \(a=b=1\), it is the uniform distribution on \((0,1)\).
Given prior \(p\sim\operatorname{Beta}(a,b)\) and \(s\) successes in \(n\) conditionally independent binary trials, what is the posterior distribution?
\[p\mid\text{data}\sim\operatorname{Beta}(a+s,b+n-s).\] Add successes to the first shape and failures to the second.
What does it mean for a positive random variable to be lognormal?
Its logarithm is normal. Equivalently, \(X=e^Z\) for normal \(Z\). It is a model for positive, right-skewed outcomes.
If \(\log X\sim N(\mu,\sigma^2)\), what is \(\mathbb E[X]\)?
\[\mathbb E[X]=e^{\mu+\sigma^2/2}.\] Exponentiating the mean of the logarithm alone does not give the mean of \(X\).
If \(\log X\sim N(\mu,\sigma^2)\), what is the median of \(X\)?
\(e^\mu\). The increasing exponential transformation preserves the median.
What is the sampling distribution of an estimator?
It is the distribution of the estimator across hypothetical repeated samples from the same sampling model.
The observed estimate is one realization. Its uncertainty comes from how that statistic varies across samples, not just from the spread of the raw observations.
What does it mean for an estimator \(\hat\theta\) to be unbiased?
\[ \operatorname{Bias}_{\theta}(\hat\theta) =\E_{\theta}[\hat\theta]-\theta. \] An estimator is unbiased if this equals zero for every allowed \(\theta\).
Unbiasedness concerns the repeated-sampling average. A particular estimate can still be far from the true value.
Assume \(\E X_i=\mu\) for every \(i\). Prove that \(\bar X\) is unbiased for \(\mu\).
Proof. By linearity of expectation, \[ \E[\bar X] =\frac1n\sum_{i=1}^n\E[X_i] =\frac{n\mu}{n}=\mu. \] Independence is unnecessary for unbiasedness. If the observations are independent with common variance \(\sigma^2\), then \(\Var(\bar X)=\sigma^2/n\).
For an i.i.d. (independent and identically distributed) sample of size \(n\ge2\) with finite variance, explain the unbiased sample variance formula.
Proof. The identity \[ \sum_i(X_i-\bar X)^2 =\sum_i(X_i-\mu)^2-n(\bar X-\mu)^2 \] has expectation \(n\sigma^2-\sigma^2=(n-1)\sigma^2\).
Thus \(S^2=\sum_i(X_i-\bar X)^2/(n-1)\) is unbiased for \(\sigma^2\). This does not make \(S\) unbiased for \(\sigma\).
What does it mean for \(\hat\theta_n\) to be consistent for \(\theta\)?
For every \(\varepsilon>0\), \[ P_{\theta}(|\hat\theta_n-\theta|>\varepsilon)\to0. \] Equivalently, \(\hat\theta_n\xrightarrow{P}\theta\).
Consistency is a large-sample property. It does not say that small-sample bias or variance is small.
Give an estimator of a mean that is unbiased for every \(n\) but inconsistent.
Let \(X_i\) be i.i.d. (independent and identically distributed) \(N(\mu,1)\) and set \(\hat\mu_n=X_1\) for every \(n\).
It is unbiased since \(\E X_1=\mu\), but \[ P(|\hat\mu_n-\mu|>\varepsilon) =P(|Z|>\varepsilon)>0 \] is constant in \(n\). The estimator never uses the additional observations.
Give a familiar biased estimator that is nevertheless consistent.
For i.i.d. (independent and identically distributed) data with \(0<\sigma^2<\infty\), \[ V_n=\frac1n\sum_i(X_i-\bar X)^2 \] has expectation \((n-1)\sigma^2/n\), so it is biased downward.
Yet \(V_n\xrightarrow{P}\sigma^2\) by applying the law of large numbers to \(X_i\) and \(X_i^2\).
How does MSE (mean squared error) combine estimator bias and variance?
MSE (mean squared error) is \[\mathbb E[(\hat\theta-\theta)^2]=\operatorname{Var}(\hat\theta)+\operatorname{Bias}(\hat\theta)^2\] when these moments exist. A biased estimator can be preferable if it reduces variance enough.
Explain the difference between standard error and standard deviation.
Standard deviation describes spread in observations. Standard error is the standard deviation of an estimator’s sampling distribution, describing how estimates vary across repeated samples.
What happens to the standard error of an independent sample mean when sample size quadruples?
It halves, because the standard error is proportional to \(1/\sqrt n\), assuming the population variance stays fixed.
What is likelihood?
The probability mass or density of the observed data, viewed as a function of the unknown parameter. The observed data are held fixed.
What is an MLE (maximum likelihood estimator)?
An MLE (maximum likelihood estimator) is a parameter value that maximizes the likelihood of the observed data within the allowed parameter space.
Is likelihood a probability distribution over parameters?
No. It need not integrate or sum to one over parameter values. A posterior distribution requires a prior and normalization.
What does a frequentist \(95\%\) confidence interval mean?
Under the stated model, the interval-producing procedure covers the fixed true parameter in \(95\%\) of repeated samples, exactly or approximately as specified.
After observing the data, it is not a \(95\%\) posterior probability that the fixed parameter lies in that realized interval.
A sample mean is \(10\), its estimated standard error is \(0.4\), and the appropriate 95% critical multiplier is supplied as \(2.064\). Calculate the interval.
Use estimate plus or minus the supplied multiplier times standard error: \[10\pm2.064(0.4)=[9.1744,10.8256].\] The multiplier is provided rather than memorized.
What is the ordinary nonparametric bootstrap?
Repeatedly draw a sample of the same size with replacement from the observed sample and recompute the statistic. The resulting variation estimates sampling uncertainty when the method is valid.
Why is ordinary row-by-row bootstrap resampling inappropriate for dependent observations?
It can destroy the dependence that affects uncertainty. Clustered observations may require cluster resampling, and time series may require block resampling.
Explain the difference between confidence, credible, and prediction intervals.
A confidence interval targets a fixed parameter through repeated-sampling coverage.
A credible interval contains a specified posterior probability for a parameter, conditional on the Bayesian model and data.
A prediction interval targets a future observation and includes observation noise as well as estimation uncertainty.
What is a hypothesis test?
A prespecified decision rule for assessing data against a null statistical claim. It uses a statistic and a rejection rule chosen to control false rejection under the null model.
What is a null hypothesis?
The claim or set of parameter values treated as the reference for a test. For example, \(H_0:p_A=p_B\) says the two conversion probabilities are equal.
What is an alternative hypothesis?
The competing claim the test is designed to detect. For example, \(H_1:p_A\ne p_B\) allows a difference in either direction.
Why should the test be specified before examining treatment results?
Choosing the hypothesis, statistic, or stopping rule after seeing the result can invalidate the intended false-positive control.
What is a rejection region?
The set of test-statistic values for which the rule rejects the null hypothesis. It is chosen using the null distribution and desired significance level.
What is the null distribution of a test statistic?
Its sampling distribution when the null model holds. It supplies critical values or tail probabilities used to decide whether the observed result is unusual.
What is a p-value?
Under the null model, the probability of a statistic at least as incompatible with the null as the observed statistic, according to the chosen test.
Is a p-value the probability that the null hypothesis is true?
No. It is calculated assuming the null model, not a posterior probability of that model given the observations.
What does it mean for a test to have level \(\alpha\)?
For every parameter value allowed by \(H_0\), \[ P_\theta(\text{reject }H_0)\le\alpha. \] The supremum of these probabilities is the test's size.
The advertised control assumes the model, sampling procedure, and analysis plan are valid. Discrete tests can be conservative.
What is a Type I error?
Rejecting a null hypothesis that is true. It is a false positive.
What is a Type II error?
Failing to reject the null when a specified alternative is true. It is a false negative. Its probability depends on the actual effect.
What is statistical power?
The probability of rejecting the null at a specified true alternative. If the Type II error probability there is \(\beta\), power is \(1-\beta\).
What does 80% power mean for a prespecified effect?
If that effect really exists and the planning assumptions hold, the test rejects the null in about 80% of repeated experiments. It is not an 80% probability that the alternative is true.
Why does a larger sample usually increase power in a mean comparison?
It reduces standard error, making a fixed true difference easier to distinguish from sampling variation. Hold the design, variability, and significance level fixed.
How does a larger true effect affect power in a conventional mean test?
A larger difference from the null is easier to detect, so power increases with its magnitude in the direction the test is designed to detect.
How does greater outcome variability affect power when sample size and effect are fixed?
It usually lowers power by increasing the standard error. The signal becomes harder to distinguish from noise.
What is the tradeoff when increasing the significance level?
It can increase power by making rejection easier, but also permits more Type I errors.
When should a one-sided alternative be selected?
Choose it before seeing the result, when only one direction addresses the scientific decision and the opposite direction would not count as a success.
For \(H_1:\theta>\theta_0\), large positive statistics support rejection. A two-sided test allocates error probability to both directions.
Does \(p>0.05\) establish that the null hypothesis is true?
No. It means the chosen test did not supply enough evidence against \(H_0\) at that threshold.
The data may also be compatible with important effects because the interval is wide or power is low.
Report the interval and detectable effect. Do not replace “fail to reject” with “prove no effect.”
Why can a statistically significant effect be practically unimportant?
A very large sample can make the standard error small enough to detect a tiny effect.
Statistical significance concerns evidence against a null. Practical significance concerns effect size, costs, benefits, and decision thresholds.
Report both magnitude and uncertainty. A p-value alone does not quantify usefulness.
When is a two-sided test equivalent to checking whether a confidence interval excludes the null value?
When the \((1-\alpha)\) confidence set is obtained by inverting the same family of level-\(\alpha\) tests.
Then reject \(H_0:\theta=\theta_0\) exactly when \(\theta_0\) is excluded.
Intervals and tests built from different approximations need not agree, especially for discrete or small samples.
What is FWER (family-wise error rate)?
FWER (family-wise error rate) is the probability of making at least one false rejection among a family of hypothesis tests.
Twenty independent tests each have a 5% false-positive probability and all nulls are true. Using \(1-(1-\alpha)^m\), find the chance of at least one false positive.
\[1-0.95^{20}\approx0.642.\] Testing each at 5% does not control the whole family at 5%.
How does the Bonferroni correction control multiple testing?
For \(m\) prespecified tests, compare each p-value with \(\alpha/m\). For valid p-values this controls the chance of any false rejection at most \(\alpha\), even with dependence.
How does the Holm correction work?
Sort the \(m\) p-values from smallest to largest. Compare the \(j\)th with \(\alpha/(m-j+1)\), stopping at the first failure. Reject only those before that failure.
What is FDR (false discovery rate)?
FDR (false discovery rate) is the expected proportion of rejected hypotheses that are false rejections, taking the proportion as zero if there are no rejections.
What does the Benjamini–Hochberg procedure control?
It controls the false discovery rate at a chosen target under independence or suitable positive dependence of valid p-values. It does not generally control the probability of any false rejection.
Explain the difference between false discovery rate and family-wise error rate.
False discovery rate averages the fraction of false rejections among all rejections. Family-wise error rate is the probability of making even one false rejection.
Why is repeatedly stopping a fixed-horizon test when \(p<0.05\) problematic?
The stopping rule changes the experiment and can inflate the probability of a false positive.
Use the prespecified fixed horizon, or a sequential design with valid stopping boundaries, alpha spending, or anytime-valid inference.
Ordinary fixed-sample p-values do not automatically remain valid under optional stopping.
What does an equivalence test try to establish?
That the effect lies inside a prespecified practically negligible interval \((-\delta,\delta)\). Its null includes effects outside that interval. Failing to detect a difference is not enough.
What does a noninferiority test try to establish?
That a new treatment is not worse than a comparator by more than a prespecified meaningful margin. It addresses one direction of unacceptable difference.
What is the difference between a permutation test and an ordinary bootstrap?
A permutation test rearranges labels under null exchangeability, or follows the actual treatment-randomization scheme under a sharp null.
A bootstrap resamples observations to approximate sampling variability.
Equal population means alone do not guarantee that unrestricted label permutations are valid.
When is a one-proportion z-test appropriate?
To compare one population binary-event probability with a specified null value using independent observations and a reasonable normal approximation to the count.
Which proportion belongs in the standard error when testing \(H_0:p=p_0\)?
The null proportion: \[\sqrt{\frac{p_0(1-p_0)}n}.\] This describes sampling variation under the hypothesis being tested.
For a sample proportion \(\hat p\), what is the z-statistic for testing \(p=p_0\)?
\[Z=\frac{\hat p-p_0}{\sqrt{p_0(1-p_0)/n}}.\] Require independent binary observations and sufficiently large null expected success and failure counts.
What can replace the one-proportion normal approximation for small expected counts?
Under \(H_0\), use the exact distribution \[ X\sim\mathrm{Binomial}(n,p_0). \] For \(H_1:p>p_0\), the p-value is \(P_{p_0}(X\ge x_{\rm obs})\).
Two-sided exact p-values have several conventions. State the convention. Exact does not mean free of independence or common-probability assumptions.
What does a two-proportion z-test compare?
Two independent population success probabilities. Under the equality null, the test estimates one shared probability by pooling the success counts.
How is the shared probability estimated under \(H_0:p_A=p_B\)?
\[\hat p=\frac{x_A+x_B}{n_A+n_B}.\] Add successes and divide by the combined number of trials.
Given pooled null proportion \(\hat p\), what is the test statistic for \(p_A=p_B\)?
\[Z=\frac{\hat p_B-\hat p_A}{\sqrt{\hat p(1-\hat p)(1/n_A+1/n_B)}}.\] This is a large-sample test for independent groups, not paired outcomes.
Why does an equality test pool two proportions while a usual difference interval does not?
The equality test estimates uncertainty under the assumption that both probabilities are the same. A usual difference interval estimates uncertainty without imposing equality.
Why can a Wilson interval be preferable to a simple Wald interval for a proportion?
The usual Wald interval can collapse when all observations are successes or failures and can have poor coverage in small samples. A Wilson interval avoids that collapse by inverting a score test.
Explain when to use a z-test rather than a t-test for a population mean.
For independent normal observations with known \(\sigma>0\), \[ Z=\frac{\bar X-\mu_0}{\sigma/\sqrt n}\sim N(0,1). \] When \(\sigma\) is unknown, use \[ T^*=\frac{\bar X-\mu_0}{S/\sqrt n}\sim t_{n-1} \] under \(H_0\), with \(n\ge2\). Large \(n\) alone is not the definition of a \(Z\)-test.
When would you use Welch’s two-sample t-test?
To compare means from independent groups without assuming equal population variances. Its estimated standard error uses each group’s variance separately, with approximate degrees of freedom.
When should you use a paired t-test?
When observations come in matched pairs, such as before and after measurements on the same people. It tests the mean of the within-pair differences, assuming the pairs are independent.
What extra variance assumption is required for the classical pooled two-sample t-test?
The groups share a common population variance. Together with independent normal observations, this justifies the classical pooled t reference distribution.
What does one-way ANOVA (analysis of variance) test?
ANOVA (analysis of variance) tests whether all group means are equal. A rejection says at least one differs, not which group it is.
What error model justifies the exact classical one-way analysis-of-variance test?
Independent, normally distributed errors with a common variance across groups. The normality assumption concerns errors within groups, not the pooled mixture of all outcomes.
What does a chi-square goodness-of-fit test assess?
Whether categorical counts are compatible with specified category probabilities, assuming independent trials and suitable expected counts.
What is Pearson’s chi-square test statistic?
For observed counts \(O_j\) and positive null expected counts \(E_j\), \[X^2=\sum_j\frac{(O_j-E_j)^2}{E_j}.\] Large values indicate greater disagreement with the null.
Which tail is used for a chi-square goodness-of-fit test?
The upper tail. Larger squared discrepancies supply stronger evidence against the specified count model.
What does a chi-square independence test assess?
Whether two categorical variables are associated in a contingency table. Its null is independence. Association does not establish causation.
How do you compute an expected cell count under independence?
\[E_{ij}=\frac{(\text{row total})(\text{column total})}{\text{grand total}}.\] These expected counts go into Pearson’s chi-square statistic.
What are the degrees of freedom for an \(r\times c\) chi-square independence test?
\((r-1)(c-1)\) in the usual contingency-table setting. For a two-by-two table, this is one.
What does Fisher's exact test do for a \(2\times2\) table?
It tests no association by conditioning on the row and column totals. Under the null, one cell count has a hypergeometric distribution.
It avoids a large-expected-count approximation.
It still requires a suitable sampling model with independent units. It does not replace a paired-data test.
How do you compare two binary outcomes measured on the same subjects?
For i.i.d. (independent and identically distributed) pairs, use the discordant-pair counts \(b\) and \(c\).
Under equal marginal success probabilities, conditional on \(b+c\), one discordant direction has distribution \(\mathrm{Binomial}(b+c,1/2)\).
For enough discordant pairs, \((b-c)^2/(b+c)\) is approximately \(\chi^2_1\). Concordant pairs do not determine the test statistic.
Why is the Mann–Whitney rank-sum test not automatically a test of equal medians?
The usual exact null is equality of the two distributions for independent samples. The statistic reflects pairwise ordering, related to \[ P(X<Y)+\tfrac12P(X=Y). \] A pure location-shift interpretation requires additional common-shape assumptions.
It is not generally a test only of equal means or equal medians.
You have measurements \(M_i\) and trusted reference values \(R_i\). How can you test average measurement bias?
Form errors \(D_i=M_i-R_i\) and test \(H_0:\E D_i=0\) using a one-sample procedure on the errors.
A paired t-test is exact for i.i.d. (independent and identically distributed) normal errors. Also inspect errors against reference magnitude for nonconstant bias.
Zero average error does not imply accurate individual measurements or an unbiased estimator for every parameter.
What is an A/B test?
An experiment that randomly assigns eligible units to control A or treatment B and compares a prespecified outcome.
What is an estimand?
The precise population quantity an analysis seeks to estimate, such as the average effect of assigning a new page to eligible users.
Why randomly assign users in an A/B test?
It balances pretreatment factors in expectation, supporting a causal comparison between the assigned variants. It does not guarantee identical groups in a particular sample.
Why define the conversion observation window before an experiment?
Every user needs a comparable opportunity to convert. Different follow-up windows can change the quantity being estimated and distort the comparison.
Why should randomization and analysis account for repeated visits by the same user?
Visits from one user are dependent and may share treatment exposure.
Randomizing by user but analyzing visits as independent can understate uncertainty, especially with positive within-user correlation.
Aggregate to users or use a model, bootstrap, or standard errors that account for clustering at the appropriate unit.
What assumption fails when one user's treatment changes another user's outcome?
No interference fails. The usual potential-outcome setup also assumes well-defined treatment versions.
Social networks and marketplaces can create spillovers between variants.
Cluster randomization or other exposure-aware designs may be needed. Their estimands and uncertainty calculations differ from an independent-user A/B test.
Explain the difference between a primary experiment metric and a guardrail metric.
The primary metric defines the main success criterion. Guardrails monitor unacceptable harm, such as latency or error rates.
Prespecifying metrics limits selective reporting and multiple-testing ambiguity.
A positive primary metric does not override a material guardrail failure. Define the decision rule before results arrive.
Explain the difference between absolute and relative lift using a change from 10% to 11%.
Absolute lift is \[ 0.11-0.10=0.01, \] which is one percentage point.
Relative lift is \((0.11-0.10)/0.10=10\%\).
Always name the scale. A one-percentage-point increase is not the same as a one-percent relative increase.
Why is a 50/50 allocation often used for an A/B test?
For independent groups with equal outcome variances and equal unit costs, it minimizes the variance of the mean difference at a fixed total sample size. Unequal variances or costs can change the best allocation.
What does the minimum detectable effect mean in experiment planning?
The minimum detectable effect is a planning effect size at which the chosen design attains a specified power and significance level.
It is not a hard boundary below which significance is impossible, and it is not the effect eventually observed.
Choose it from a meaningful decision scale before estimating sample size.
Why are 80% power and a 5% significance level insufficient to determine sample size?
You must also specify the effect to detect and outcome variability or a baseline event rate, together with the test and design. Power is defined at a particular alternative.
What happens to the usual required sample size when the target effect is halved?
It grows by roughly a factor of four because the common approximation has \(n\propto1/\delta^2\), with variance and error targets fixed.
What is SRM (sample-ratio mismatch) in an A/B test?
SRM (sample-ratio mismatch) is a statistically implausible departure of observed group counts from the planned allocation.
A goodness-of-fit test can detect it. Investigate assignment, logging, filtering, and joins before trusting effect estimates.
A non-significant SRM check does not prove that every aspect of the experiment is valid.
Why randomly split traffic between two identical variants?
An A/A test checks the assignment and analysis pipeline when no treatment difference is intended.
It can reveal logging asymmetry, bad uncertainty estimates, or analysis bugs.
Some false positives are expected at the chosen level. One non-significant A/A result does not establish permanent validity.
Why is reaching the planned sample size not always enough to end an experiment immediately?
Outcomes may mature slowly, behavior may vary by weekday, and novelty or carryover effects may matter.
Specify a minimum duration and follow-up window as part of the design.
A short experiment can estimate a short-run effect precisely while missing the sustained effect needed for the decision.
What is intention-to-treat analysis?
Compare units according to their randomized assignment, regardless of whether they actually used the treatment. This preserves the randomized comparison of assignment.
Why can restricting an experiment to users who clicked after treatment cause bias?
Clicking can itself be affected by treatment. Conditioning on it can select different kinds of users in each group, breaking the original randomized comparison.
Why can adjusting an A/B metric using a pretreatment measurement improve precision?
A pretreatment variable correlated with the outcome can remove predictable outcome variation. Because it precedes treatment, it need not remove the treatment effect. The adjustment and uncertainty calculation must respect the experiment design.
An effect is significant in one subgroup but not another. Does that prove the effects differ?
No. “Significant” versus “not significant” is not itself a significant difference.
Test the difference of effects, for example with a treatment-by-subgroup interaction.
Prespecify key subgroups, account for multiple comparisons, and examine uncertainty. Small subgroups often produce unstable estimates.
What is the multiple linear regression model?
\[ Y=\beta_0+\sum_{j=1}^p\beta_jX_j+\varepsilon. \] If \(\E[\varepsilon\mid X]=0\), then \[ \E[Y\mid X]=\beta_0+\sum_{j=1}^p\beta_jX_j. \] The error is the deviation from the population conditional mean, not the fitted residual.
Is \(Y=\beta_0+\beta_1X+\beta_2X^2+\varepsilon\) a linear regression model?
Yes. It is linear in the coefficients \(\beta_0,\beta_1,\beta_2\), even though the fitted curve is nonlinear in \(X\).
Fixed transformations, polynomial terms, splines, and interactions can be included as columns of a design matrix.
Their coefficients remain linear-model parameters.
What does OLS (ordinary least squares) minimize?
OLS (ordinary least squares) minimizes the sum of squared residuals: \[\sum_i(y_i-\hat y_i)^2.\] It chooses the best-fitting coefficients within the specified linear model.
Which population prediction minimizes expected squared loss?
The conditional mean \(\mathbb E[Y\mid X]\), assuming finite second moments. A restricted model may only approximate that mean.
What is the matrix formula for ordinary least-squares coefficients when the design has full column rank?
\[\hat\beta=(X^{\mathsf T}X)^{-1}X^{\mathsf T}y.\] The full-rank condition makes the inverse exist.
What goes wrong when a regression design matrix has linearly dependent columns?
The coefficients need not be uniquely determined. The fitted values can still be unique, even though different coefficient vectors produce them.
What geometric relationship holds between fitted values, residuals, and the design matrix?
\(\hat y\) is the orthogonal projection of \(y\) onto the column space of \(X\). \[ e=y-\hat y,\qquad X^{\mathsf T}e=0. \] If the model contains an intercept, then \(\sum_i e_i=0\).
These are algebraic properties of the fit, not evidence that the population errors are exogenous.
How is \(\beta_j\) interpreted in an additive linear conditional-mean model?
It is the change in the conditional mean of \(Y\) for a one-unit increase in \(X_j\), holding the other modeled predictors fixed.
The interpretation changes when interactions or nonlinear terms involve \(X_j\).
Without a causal design and assumptions, this is a conditional association, not a causal effect.
What determines the fitted slope in simple linear regression with an intercept?
It is sample covariance divided by sample variance of the predictor: \[\hat\beta_1=\frac{\sum_i(x_i-\bar x)(y_i-\bar y)}{\sum_i(x_i-\bar x)^2}.\] The predictor must not be constant.
Given the fitted slope, how do you obtain the intercept in simple linear regression?
\[\hat\beta_0=\bar y-\hat\beta_1\bar x.\] The fitted line passes through the point of sample means.
What does \(R^2\) measure?
Relative reduction in squared error compared with predicting the outcome mean: \[R^2=1-\frac{\sum_i(y_i-\hat y_i)^2}{\sum_i(y_i-\bar y)^2}.\] The outcomes must have nonzero total variation.
Can test-set \(R^2\) be negative?
Yes. This means the predictions have more squared error than using the test-set outcome mean as a constant benchmark.
Why use adjusted \(R^2\) rather than ordinary training \(R^2\) when comparing added predictors?
Ordinary training \(R^2\) cannot decrease when predictors are added to an unpenalized least-squares model. Adjusted \(R^2\) penalizes the extra coefficients, so an unhelpful addition can lower it. Validation is still needed.
A full-rank regression has \(n\) observations and \(k\) coefficients. What residual degrees of freedom are used to estimate the common error variance?
Use \(n-k\), counting the intercept among the coefficients. Dividing the residual sum of squares by \(n-k\) is unbiased under zero conditional error mean and covariance \(\sigma^2I\), with \(n>k\).
What does the usual t-statistic for a regression coefficient compare?
SE (standard error).
The estimated coefficient minus its null value, divided by the estimated standard error: \[t=\frac{\hat\beta_j-b_0}{\widehat{\operatorname{SE}}(\hat\beta_j)}.\] SE (standard error) measures coefficient uncertainty. Exact t inference requires the classical Gaussian linear-model assumptions.
When do you use a nested-model F-test in linear regression?
To jointly test whether added coefficients improve a restricted model. The models must be nested and fitted to the same observations. Exact classical inference requires a full-rank Gaussian model with common error variance and independent errors.
Why is a prediction interval for a new observation wider than a confidence interval for its mean response?
The prediction interval includes new observation noise as well as uncertainty in estimating the mean. The mean-response interval only concerns the estimated conditional mean.
For \(\hat y=2+3x_1-x_2\), calculate the prediction at \((x_1,x_2)=(4,5)\).
\(\hat y=2+3(4)-5=9\).
The observed outcome is \(12\) and the prediction is \(9\). What is the residual?
Using observed minus predicted, \(e=12-9=3\). A positive residual means the model underpredicted.
What error model makes OLS (ordinary least squares) coincide with maximum likelihood for the regression coefficients?
Conditionally independent errors with a common normal distribution \(N(0,\sigma^2)\) give \[ \log L(\beta,\sigma^2) =-\frac n2\log(2\pi\sigma^2) -\frac{\|y-X\beta\|^2}{2\sigma^2}. \] For fixed \(\sigma^2>0\), maximizing this over \(\beta\) is equivalent to minimizing RSS (residual sum of squares).
Why can a linear model be unreliable outside the observed predictor range?
The fitted functional form is weakly checked where there are few or no observations.
Even a precise estimated line can be systematically wrong outside the training domain.
Model-based prediction intervals account for specified noise and parameter uncertainty. They need not account for an incorrect extrapolation shape.
What does the zero conditional error mean assumption mean in linear regression?
\(\mathbb E[\varepsilon\mid X]=0\). Once the modeled predictors are given, there is no systematic remaining error in the mean. With a correct linear model and full rank, this gives unbiased coefficients.
What is homoscedasticity in regression?
Constant conditional error variance across observations: \(\operatorname{Var}(\varepsilon_i\mid X)=\sigma^2\). It concerns errors, not the variation of the predictor columns.
Must predictors in linear regression be normally distributed?
No. The normality assumption used for exact classical inference is about errors conditional on the predictors, not about the predictors themselves.
Must different predictor columns in linear regression be independent?
No. Correlated predictors are allowed. Perfect linear dependence prevents unique coefficients, and strong correlation can increase coefficient uncertainty.
For full-rank \(X\) and \(\E[\varepsilon\mid X]=0\), prove conditional unbiasedness of OLS (ordinary least squares).
Proof. Substitute \(y=X\beta+\varepsilon\): \[ \hat\beta=\beta+(X^{\mathsf T}X)^{-1}X^{\mathsf T}\varepsilon. \] Taking conditional expectations gives \[ \E[\hat\beta\mid X]=\beta. \] Homoscedasticity and normality are not needed for this argument.
What does BLUE (best linear unbiased estimator) mean?
BLUE (best linear unbiased estimator) means minimum covariance, in the positive-semidefinite sense, among estimators linear in the observed outcomes and unbiased under the model. It is not a claim about all possible predictors.
Does the Gauss–Markov theorem require normally distributed errors?
No. For a full-rank linear model, it uses zero conditional error mean, constant error variance, and uncorrelated errors. Normality is not needed for the best-linear-unbiased conclusion.
What does a residual-versus-fitted plot help diagnose?
Systematic structure left in the residuals. Curvature suggests a misspecified mean, while a changing spread suggests a possible variance problem.
What does a normal Q–Q (quantile–quantile) residual plot check?
A normal Q–Q (quantile–quantile) plot compares residual quantiles with normal quantiles. Systematic departures from a line suggest departures from a normal-error approximation.
What can a residual-versus-time plot reveal?
Time patterns remaining after fitting, such as trends or cycles that suggest serial dependence. Absence of an obvious pattern does not prove independence.
What is heteroscedasticity?
Nonconstant conditional error variance. For example, residual spread may grow as fitted values increase.
Does heteroscedasticity automatically bias ordinary least-squares coefficients?
No. Coefficients remain unbiased with full rank and a correct linear conditional mean. However, conventional constant-variance standard errors can be wrong and the fit need not be efficient.
What does a heteroscedasticity-robust standard error estimate?
They estimate coefficient uncertainty without imposing constant error variance, under suitable large-sample conditions.
They do not correct omitted-variable bias, a wrong conditional mean, leakage, or dependence automatically.
Use cluster-robust or time-series-appropriate methods when the dependence structure requires them.
What can go wrong when regression errors are serially correlated?
Usual independent-error standard errors may be wrong. OLS (ordinary least squares) can remain unbiased under conditional mean exogeneity, but may lose efficiency.
With lagged outcomes, exogeneity can fail.
Use a justified dependence model, generalized least squares, or valid dependence-robust inference.
What is multicollinearity?
A strong linear relationship among predictor columns. It can make individual coefficient estimates unstable. Perfect linear dependence prevents a unique coefficient vector.
What does VIF (variance inflation factor) quantify?
VIF (variance inflation factor) measures coefficient variance inflation associated with a predictor’s linear relationship to the others. In the usual setting, \(\operatorname{VIF}_j=1/(1-R_j^2)\), where \(R_j^2\) is from regressing predictor \(j\) on the remaining predictors.
Distinguish a response outlier, high leverage, and an influential observation.
A response outlier has an unusual outcome relative to its fitted value.
High leverage means unusual predictor values, measured by the diagonal of the hat matrix.
Influence measures how much the fit changes when an observation changes or is removed. Cook's distance combines residual and leverage information.
Suppose the true mean is \(\beta_0+\beta_1X+\beta_2Z\). What is the population slope if \(Z\) is omitted?
If \(\E[\varepsilon\mid X,Z]=0\) and \(\Var(X)>0\), the population simple-regression slope is \[ b_1=\beta_1+\beta_2\frac{\Cov(X,Z)}{\Var(X)}. \] Bias relative to \(\beta_1\) requires both \(\beta_2\ne0\) and \(\Cov(X,Z)\ne0\).
This is a population projection identity.
How do you interpret a slope in a model for the conditional mean of log outcome?
In \(\mathbb E[\log Y\mid X]=\beta_0+\beta_1X\), a one-unit increase in \(X\) multiplies the fitted geometric-mean scale by \(e^{\beta_1}\). This is not automatically an arithmetic-mean ratio.
Why does exponentiating a predicted log mean not generally give the arithmetic mean?
Expectation does not commute with exponentiation: \(\mathbb E[Y\mid X]\ne e^{\mathbb E[\log Y\mid X]}\) in general. Recovering the arithmetic mean requires accounting for log-scale variability.
In \(\E[Y\mid X,Z]=\beta_0+\beta_1X+\beta_2Z+\beta_3XZ\), what is the effect of \(X\)?
Holding \(Z\) fixed, \[ \frac{\partial\E[Y\mid X,Z]}{\partial X} =\beta_1+\beta_3Z. \] The slope depends on \(Z\). If \(Z\) is binary, \(\beta_3\) is the difference between the two group slopes.
A main-effect coefficient is interpreted at zero for its interacting variable.
What is ridge regression?
Regression with a squared-coefficient penalty added to the fitting loss. It shrinks slopes toward zero and can stabilize a model with correlated predictors. The intercept is usually not penalized.
Explain the main difference between ridge regression and lasso (least absolute shrinkage and selection operator).
Lasso (least absolute shrinkage and selection operator) minimizes a fitting loss plus \[ \lambda\sum_j|\beta_j|. \] The \(\ell_1\) penalty can set coefficients exactly to zero and therefore performs feature selection.
With correlated predictors, selected variables can be unstable. Ordinary post-selection t-tests do not automatically remain valid.
How does elastic net combine ridge and lasso (least absolute shrinkage and selection operator)?
One convention adds the penalty \[ \lambda\left(a\|\beta\|_1+\frac{1-a}{2}\|\beta\|_2^2\right), \quad 0\le a\le1. \] It combines sparsity with \(\ell_2\) stabilization. Tune both strength and mixture using training-only validation.
Software conventions for loss scaling and parameter names can differ.
Why usually standardize predictors before ridge, lasso (least absolute shrinkage and selection operator), or elastic net?
A coefficient's magnitude depends on its feature's measurement units. A common penalty otherwise treats differently scaled features unequally.
Fit scaling on training data only and reuse those values for validation and test data.
Regularization strength is a hyperparameter, not a hypothesis-test significance level.
State the probability model used in binary logistic regression.
For \(Y\in\{0,1\}\), \[ p(x)=P(Y=1\mid x)=\sigma(\beta_0+x^{\mathsf T}\beta), \qquad \sigma(z)=\frac1{1+e^{-z}}. \] The conditional response is Bernoulli. The output is a probability, not a class label until a decision rule is applied.
How do you convert a probability to odds?
For \(0<p<1\), the odds are \(p/(1-p)\). Odds compare the probability of the event with the probability of its complement.
What are log-odds?
The logarithm of the odds: \[\operatorname{logit}(p)=\log\frac p{1-p}.\] Logistic regression models this quantity as a linear function of the predictors.
How do you interpret \(e^{\beta_j}\) in an additive logistic regression model?
A one-unit increase in \(x_j\), holding the other predictors fixed, multiplies the conditional odds by \(e^{\beta_j}\).
For a change of \(c\) units, the multiplier is \(e^{c\beta_j}\).
This is a constant odds ratio, not a constant change or ratio in probabilities. Interactions alter the interpretation.
A baseline success probability is \(0.20\). A covariate change doubles the odds. What is the new probability?
Baseline odds are \(0.20/0.80=0.25\). Doubling gives odds \(0.50\). \[ p_{\rm new}=\frac{0.50}{1+0.50}=\frac13. \] The probability rises to about \(0.333\), not \(0.40\). Odds ratios and risk ratios are different quantities.
What loss corresponds to maximum likelihood for independent binary outcomes?
Binary cross-entropy, or negative Bernoulli log-likelihood: \[ L=-\sum_i\left[y_i\log p_i+(1-y_i)\log(1-p_i)\right]. \] It penalizes confident wrong probabilities strongly.
For linear logits this objective is convex in the coefficients. A finite minimizer need not exist under separation.
For binary cross-entropy summed over observations with logits \(X\beta\), what is the coefficient gradient?
\[\nabla L=X^{\mathsf T}(p-y).\] Here \(p\) contains predicted probabilities and \(y\) the binary labels. If loss is averaged instead, divide by the sample size.
Why is unpenalized logistic loss convex in linear-model coefficients?
Its Hessian is \(X^{\mathsf T}WX\), where \(W\) has diagonal entries \(p_i(1-p_i)\ge0\). This matrix is positive semidefinite. Convexity alone does not guarantee a finite minimizer.
What is linear in ordinary logistic regression?
The conditional log-odds, not the probability itself. With an additive predictor, \(\log[p(x)/(1-p(x))]=\beta_0+x^{\mathsf T}\beta\).
Does logistic regression require normally distributed predictors?
No. The model specifies a Bernoulli response probability conditional on the predictors, not a Gaussian distribution for the predictors.
Does logistic regression assume constant conditional response variance?
No. For a binary response, \(\operatorname{Var}(Y\mid X)=p(X)(1-p(X))\), which changes with the probability.
Why do repeated outcomes from the same user require care in logistic inference?
The conventional independent-observation likelihood and uncertainty calculation may not describe them. The analysis needs to account for the dependence.
What happens to unpenalized logistic estimates under complete separation?
A linear boundary strictly separates all successes from all failures.
Scaling separating coefficients can continually improve the likelihood, so no finite unpenalized maximizer exists.
Penalization, an appropriate Bayesian prior, or bias-reduced likelihood methods can yield finite estimates. Apparent perfect training classification is not evidence of reliable probabilities.
Is \(0.5\) always the correct classification threshold for logistic regression?
No. For calibrated probability \(p\), let \(c_0>0\) be the cost of a false positive and \(c_1>0\) the cost of a false negative. Correct decisions cost zero.
Predict positive when \[ p>\frac{c_0}{c_0+c_1}. \] Choose or tune the policy without using the final test labels.
For a continuous predictor without interactions, how does probability change with \(x_j\)?
\[ \frac{\partial p(x)}{\partial x_j} =\beta_jp(x)(1-p(x)). \] The effect depends on the starting probability, even though the log-odds slope is constant.
For a binary predictor, compare predicted probabilities at its two values instead of using a derivative.
How can a large-sample coefficient interval be turned into an odds-ratio interval?
SE (standard error).
If a valid Wald approximation gives \[ \hat\beta_j\pm z_{1-\alpha/2}\widehat{\mathrm{SE}}(\hat\beta_j), \] exponentiate both endpoints to obtain the odds-ratio interval.
This can perform poorly with sparse data or separation. Profile-likelihood or other suitable methods may be needed.
How can two nested logistic models be compared inferentially?
Let \(\ell_F\) and \(\ell_R\) be maximized full and restricted log-likelihoods. \[ D=2(\ell_F-\ell_R). \] Under the null and regularity conditions, \(D\) is asymptotically \(\chi^2_q\), where \(q\) is the number of added free parameters.
This is not an exact finite-sample F-test.
What can class weighting or outcome-based resampling change besides classification accuracy?
They change the effective training objective or class prevalence and may change probability calibration.
Use evaluation data representative of deployment. Consider prevalence correction or recalibration when justified.
A rare positive class does not by itself invalidate logistic regression, but sparse events can make estimation unstable.
What is a GLM (generalized linear model)?
A GLM (generalized linear model) combines a response distribution from an exponential family with a link that makes the conditional mean a linear predictor. Logistic and Poisson regression are examples.
What is a link function in a generalized linear model?
It connects the conditional mean to the linear predictor: \[g(\mathbb E[Y\mid x])=x^{\mathsf T}\beta.\] For example, Poisson regression often uses the logarithm.
How does a Poisson GLM (generalized linear model) model counts with differing observation exposure?
For exposure \(t_i>0\), use \[ Y_i\mid x_i\sim\mathrm{Poisson}(\mu_i),\qquad \log\mu_i=\log t_i+x_i^{\mathsf T}\beta. \] The coefficient of the offset \(\log t_i\) is fixed at one. The model describes a rate multiplied by exposure.
The Poisson model implies conditional variance equal to conditional mean.
What is overdispersion relative to a Poisson regression model?
The conditional outcome variance exceeds the conditional mean. Raw variance exceeding raw mean does not by itself diagnose this conditional property.
What is one standard model alternative when a Poisson count model is overdispersed?
A negative binomial model can allow variance to exceed the mean. It must still match the conditional mean and dependence structure of the data.
Explain the different roles of training, validation, and test data.
Training: fit parameters and preprocessing.
Validation: choose hyperparameters, features, model family, and decision thresholds.
Test: estimate performance after those choices are fixed.
Repeatedly choosing models based on test performance makes that set part of model selection.
How does k-fold cross-validation estimate predictive performance?
Split the data into k folds, train on all but one, and evaluate on the held-out fold. Repeat so every fold is held out once, then combine the held-out results.
Where should preprocessing be fitted during cross-validation?
Within each training fold. Apply that fitted transformation to the held-out fold. Fitting it on the full dataset can leak validation information.
Why use an inner and outer cross-validation loop?
The inner loop selects hyperparameters using only the outer training data.
The outer loop evaluates the entire selection-and-fitting procedure on unseen data.
Using the same CV (cross-validation) scores both to search many models and to report the winning score can give an optimistic estimate.
How should data be split when deployment will involve entirely new patients?
Keep all observations from a patient in the same split. Otherwise the evaluation may benefit from seeing that patient during training.
Why can a random row split be inappropriate for forecasting?
It can train the model using information from after the prediction time. Use a past-only training window that matches the intended deployment horizon.
What is the squared prediction-error decomposition at a fixed input?
With independent mean-zero test noise and finite variances, expected squared error equals \[\text{bias}^2+\text{model variance}+\text{noise variance}.\] The averaging includes the randomness of the training data.
Explain the difference between underfitting and overfitting.
Underfitting: the model or training procedure cannot capture useful structure, often producing poor training and validation performance.
Overfitting: the fit exploits sample-specific noise, producing a training advantage that does not generalize.
A gap alone is not a complete diagnosis. Check leakage, distribution shift, and learning curves.
What is a true positive in binary classification?
A case that is actually positive and is predicted positive. The positive class must be defined for the task.
What is a false positive in binary classification?
A case that is actually negative but is predicted positive. It is a false alarm.
What is a true negative in binary classification?
A case that is actually negative and is predicted negative.
What is a false negative in binary classification?
A case that is actually positive but is predicted negative. It is a missed positive.
What does sensitivity, also called recall, measure?
The fraction of actual positives detected: \[\frac{\text{true positives}}{\text{true positives}+\text{false negatives}}.\] It requires at least one actual positive.
What does specificity measure?
The fraction of actual negatives correctly rejected: \[\frac{\text{true negatives}}{\text{true negatives}+\text{false positives}}.\] It requires at least one actual negative.
What is the false-positive rate?
The fraction of actual negatives predicted positive. It equals \(1-\text{specificity}\).
What does precision measure?
The fraction of positive predictions that are correct: \[\frac{\text{true positives}}{\text{true positives}+\text{false positives}}.\] Without positive predictions, the denominator is zero.
Explain the difference between precision and recall.
Precision asks how many positive predictions are correct. Recall asks how many actual positives were found. They condition on different groups.
What is classification accuracy?
The fraction of all predictions that are correct: \[\frac{\text{true positives}+\text{true negatives}}{\text{number of cases}}.\] It can hide poor minority-class detection.
What is balanced accuracy for a binary classifier?
The mean of sensitivity and specificity: \[(\text{sensitivity}+\text{specificity})/2.\] Each true class receives equal weight.
What is the \(F_1\) score?
PR (precision–recall).
The harmonic mean of precision \(P\) and recall \(R\): \[F_1=\frac{2PR}{P+R}.\] It ignores true negatives. A convention is needed when the denominator is zero.
What does a ROC (receiver operating characteristic) curve plot?
A ROC (receiver operating characteristic) curve plots sensitivity vertically against false-positive rate horizontally as the score threshold varies.
What does ROC (receiver operating characteristic) AUC (area under the curve) measure?
ROC (receiver operating characteristic) AUC (area under the curve) measures ranking. It is the probability that a random positive case outranks a random negative case, counting a tie as one half. Higher scores must indicate the positive class.
What does a precision–recall curve show?
Precision against recall across classification thresholds. It displays the tradeoff between the purity of positive alerts and the fraction of actual positives detected.
Is average precision always the trapezoidal area under a precision–recall curve?
No. Average precision is a recall-weighted summary of precision. It generally differs from trapezoidal interpolation of the curve.
Why does log loss penalize confident wrong predictions heavily?
\[ -\frac1n\sum_i\bigl[y_i\log p_i+(1-y_i)\log(1-p_i)\bigr]. \] A probability approaching zero for the event that occurs gives arbitrarily large loss.
Expected log loss is minimized by the true conditional probability. It evaluates probabilities rather than just labels.
What does it mean for a binary classifier’s probabilities to be calibrated?
Among cases assigned a probability of 0.7, the event occurs about 70% of the time, and analogously at other predicted probabilities. Calibration is about agreement between predicted probability and event frequency.
What is the binary Brier score?
The mean squared error of predicted probabilities: \[\frac1n\sum_i(p_i-y_i)^2.\] It evaluates overall probabilistic accuracy, not calibration alone.
What is MAE (mean absolute error)?
MAE (mean absolute error) averages absolute prediction errors: \[\frac1n\sum_i|y_i-\hat y_i|.\] It has the same units as the outcome.
What is RMSE (root mean squared error)?
RMSE (root mean squared error) is \[\sqrt{\frac1n\sum_i(y_i-\hat y_i)^2}.\] Squaring gives large errors extra weight. The final square root restores outcome units.
Which population prediction minimizes expected absolute loss?
A conditional median of the outcome. Squared loss instead targets a conditional mean, assuming the needed expectations exist.
What is covariate shift?
The input distribution \(P(X)\) changes while \(P(Y\mid X)\) remains fixed. The model encounters different kinds of inputs.
What is label shift?
The class distribution \(P(Y)\) changes while \(P(X\mid Y)\) remains fixed. For example, event prevalence changes.
What is concept shift?
The relationship \(P(Y\mid X)\) changes. Old input–outcome relationships no longer describe deployment.
How does a decision tree make a prediction?
It sends the input through a sequence of feature-based split rules to a terminal leaf. That leaf supplies a predicted value or class distribution.
How does limiting tree depth affect model flexibility?
It restricts how many sequential splits can define a prediction region. This limits complexity and can reduce overfitting, but excessive restriction can underfit.
What is Gini impurity?
For class proportions \(p_k\), \[G=1-\sum_kp_k^2.\] It is zero when a node contains only one class.
How does a classification tree evaluate an impurity-based split?
Compare the parent impurity with the sample-size-weighted average impurity of the children. A larger reduction is a better training split.
What is bagging, or bootstrap aggregation?
Train separate models on bootstrap resamples and aggregate their predictions. Averaging unstable learners can reduce variance.
What is an OOB (out-of-bag) prediction?
An OOB (out-of-bag) prediction for a training case uses only models whose bootstrap samples omitted that case. This provides an internal evaluation, but does not repair preprocessing leakage.
How does a random forest differ from bagging ordinary decision trees?
It also randomly restricts candidate features at each split. This can make the trees less correlated and improve the variance reduction achieved by averaging.
How does gradient boosting build an ensemble?
It adds learners sequentially: \[ F_m(x)=F_{m-1}(x)+\eta h_m(x). \] The next learner approximates a descent direction for the chosen loss, often its negative gradients at training cases. For squared loss these are residuals.
Learning rate, learner complexity, and number of rounds interact. Select them using validation data.
How does a k-nearest-neighbor classifier predict?
It finds the k closest training cases and aggregates their labels or class probabilities. The chosen distance determines what counts as close.
How does increasing k affect a nearest-neighbor model?
It smooths the prediction over more cases. Small k is flexible but can be noisy, while large k can miss local structure.
Why should feature scaling be considered for nearest-neighbor models?
A feature measured on a large numerical scale can dominate the distance. The distance should reflect meaningful similarity rather than arbitrary units.
What is a soft-margin SVM (support vector machine)?
A soft-margin SVM (support vector machine) seeks a wide separating margin while penalizing margin violations. It trades boundary simplicity against training errors. Its raw score is not a calibrated probability.
What is the kernel trick?
A positive-semidefinite kernel can represent an inner product \[ K(x,z)=\langle\phi(x),\phi(z)\rangle \] in a possibly high-dimensional feature space. Algorithms using only inner products can operate through \(K\) without constructing \(\phi\) explicitly.
A linear separator in that feature space can be nonlinear in the original inputs.
What conditional-independence assumption defines a naive Bayes classifier?
Features are modeled as conditionally independent given the class: \[ P(Y=k\mid x)\propto P(Y=k)\prod_j p(x_j\mid Y=k). \] This is not marginal independence of the features. Gaussian, multinomial, and Bernoulli variants make different likelihood assumptions.
Correlated evidence may be counted repeatedly, so good classification need not imply calibrated probabilities.
What is LDA (linear discriminant analysis)?
LDA (linear discriminant analysis) is a classifier that models features within each class as Gaussian, using a shared covariance matrix. It combines these class models with class priors through Bayes’ rule, producing linear decision boundaries.
What is QDA (quadratic discriminant analysis)?
QDA (quadratic discriminant analysis) is a classifier that models features within each class as Gaussian, allowing a separate covariance matrix for each class. It combines these models with class priors through Bayes’ rule, allowing quadratic decision boundaries.
What is the first principal component of centered data?
For sample covariance matrix \(S\), its direction solves \[ \max_{\|v\|=1}v^TSv. \] It is an eigenvector associated with the largest eigenvalue. Later components maximize remaining variance subject to orthogonality.
PCA (principal component analysis) is unsupervised. High-variance directions need not be useful for predicting the target.
How is the fraction of variance retained by \(r\) principal components computed?
With covariance eigenvalues \(\lambda_1\ge\cdots\ge\lambda_p\ge0\), \[ \text{fraction retained}=\frac{\sum_{j=1}^r\lambda_j} {\sum_{j=1}^p\lambda_j}, \] provided total variance is positive.
Centering and scaling change the analysis. Fit them and PCA (principal component analysis) on training data only. Choose \(r\) using the actual downstream objective, not variance alone.
What objective does k-means clustering minimize?
The within-cluster sum of squared distances to cluster centers: \[\sum_i\|x_i-\mu_{c_i}\|^2.\] The usual iterative algorithm can converge to a local rather than global optimum.
Why does initialization matter in k-means clustering?
Different starting centers can lead to different local optima. A small final objective from one run does not prove that the best partition was found.
What is agglomerative hierarchical clustering?
Start with one cluster per observation, then repeatedly merge the closest clusters using a chosen linkage rule. The sequence forms a hierarchy.
What is a dendrogram?
A tree recording the merge sequence in hierarchical clustering. Cutting it at a chosen height or cluster count produces a partition.
Explain the difference between single, complete, and average linkage.
They compare clusters using the minimum, maximum, and average pairwise distance between their members, respectively. The linkage choice changes which clusters merge.
What is DBSCAN (density-based spatial clustering of applications with noise)?
DBSCAN (density-based spatial clustering of applications with noise) groups dense connected regions and leaves other points as noise. It uses a neighborhood radius and minimum neighborhood count rather than a prespecified number of clusters.
What is a core point in density-based clustering?
A point with enough observations in its specified-radius neighborhood to satisfy the minimum neighborhood count.
What is a border point in density-based clustering?
A non-core point that lies within the neighborhood of a core point and can be assigned to that cluster.
What does noise mean in density-based clustering?
A point not assigned to a cluster under the chosen density criteria. The label depends on the radius, minimum count, and distance scale.
What does permutation feature importance measure?
The deterioration in a fitted model’s chosen score when a feature is shuffled, preferably on held-out data. It measures reliance by that model, not causality.
Why can correlated predictors complicate permutation importance?
Another feature may substitute for the shuffled feature, hiding its contribution. Shuffling can also create unrealistic feature combinations.
Explain the difference between weak and strict stationarity.
Weak stationarity requires finite second moments, a constant mean, and covariance depending only on lag: \[ \E[Y_t]=\mu,\qquad \Cov(Y_t,Y_{t-h})=\gamma(h). \] Strict stationarity requires all finite-dimensional distributions to be invariant under time shifts. With finite second moments, strict implies weak. The converse need not hold.
How do trend, seasonality, and nonseasonal cycles differ?
A trend is a long-run change in level. Seasonality is a pattern tied to a known recurring period, such as day of week.
Cycles fluctuate over durations that need not be fixed or calendar-based.
Deterministic changes in the mean violate weak stationarity. A stationary stochastic series can still display irregular cycles.
Define the ACF (autocorrelation function) of a weakly stationary series with positive variance.
\[ \gamma(h)=\Cov(Y_t,Y_{t-h}),\qquad \rho(h)=\frac{\gamma(h)}{\gamma(0)}. \] The ACF (autocorrelation function) measures linear association between observations separated by \(h\) time steps. It satisfies \(\rho(0)=1\) and \(\rho(-h)=\rho(h)\).
Sample autocorrelation estimates are noisy estimates, not exact population patterns.
What does the PACF (partial autocorrelation function) at lag \(h\) remove from ordinary autocorrelation?
It is the correlation between \(Y_t\) and \(Y_{t-h}\) after linearly projecting each on the intervening lags \(Y_{t-1},\ldots,Y_{t-h+1}\).
Equivalently, under nonsingular lag covariance, it is the final coefficient in the population regression on the first \(h\) lags for a stationary series.
This is linear adjustment, not general conditional independence.
What is weak white noise?
A process with zero mean, a constant finite variance, and zero covariance between distinct times. These moment conditions do not require independence.
Does zero autocorrelation prove a time series is independent?
No. Zero autocorrelation rules out linear dependence at those lags, not every kind of dependence. Jointly Gaussian white noise is a special case in which independence follows.
Why is a random walk with positive innovation variance not weakly stationary?
With a fixed start and independent mean-zero increments of variance \(\sigma^2>0\), its variance grows as \(t\sigma^2\). Weak stationarity requires constant variance.
What happens when \(Y_t=Y_{t-1}+\varepsilon_t\) is differenced?
\(Y_t-Y_{t-1}=\varepsilon_t\). Differencing removes the accumulated level and leaves the innovations.
What is a first-order autoregressive model?
\[Y_t=c+\phi Y_{t-1}+\varepsilon_t.\] The present value depends linearly on the previous value and a new innovation.
What coefficient condition gives a causal stationary solution for \(Y_t=c+\phi Y_{t-1}+\varepsilon_t\) with white-noise innovations?
\(|\phi|<1\). The effect of a past shock then decays geometrically.
For a stationary \(Y_t=c+\phi Y_{t-1}+\varepsilon_t\) with mean-zero innovations, what is the mean?
\(\mu=c/(1-\phi)\). Taking expectations gives \(\mu=c+\phi\mu\).
For a causal stationary first-order autoregression, what is the variance if innovation variance is \(\sigma^2\)?
\[\operatorname{Var}(Y_t)=\frac{\sigma^2}{1-\phi^2}.\] This requires \(|\phi|<1\).
What is the lag-h autocorrelation of a causal stationary first-order autoregression?
\(\rho(h)=\phi^{|h|}\). Negative \(\phi\) gives alternating signs.
What is an AR (autoregressive) model of order p?
An AR (autoregressive) model predicts the current value from the preceding \(p\) values plus a new innovation: \[Y_t=c+\sum_{j=1}^p\phi_jY_{t-j}+\varepsilon_t.\]
What is an MA (moving average) time-series model?
An MA (moving average) model expresses the current value as a fixed mean plus current and finitely many past white-noise innovations. The order is the number of past innovations used.
Explain the difference between a moving-average time-series model and a rolling average.
The model combines unobserved innovations. A rolling average smooths observed data values. The shared name does not make them the same operation.
What is an ARMA (autoregressive moving average) model?
An ARMA (autoregressive moving average) model combines dependence on previous observations with dependence on previous innovations. The orders \(p,q\) specify how many of each appear in the model.
What does a causal autoregressive representation require of its characteristic roots?
The roots of \(1-\sum_{j=1}^p\phi_jz^j\) must lie outside the unit circle in the usual nondegenerate representation. This gives a stable dependence on present and past innovations.
What does invertibility mean for a moving-average representation?
The innovations can be recovered from current and past observations through a convergent linear filter. It makes the innovation representation usable from observed history.
For a stationary AR (autoregressive) model of order \(p\), which population correlation function cuts off after lag \(p\)?
The PACF (partial autocorrelation function). Sample plots are noisy, so a visual cutoff is evidence to investigate, not proof of the order.
For an MA (moving average) model of order \(q\), which population correlation function is zero beyond lag \(q\)?
The ACF (autocorrelation function). Beyond that lag, the finite innovation windows do not overlap.
Explain the difference between ordinary and seasonal differencing.
Ordinary differencing subtracts the previous observation: \(Y_t-Y_{t-1}\). Seasonal differencing subtracts the previous corresponding season: \(Y_t-Y_{t-m}\), with period \(m\).
Why can unnecessary differencing harm a forecasting model?
It can introduce artificial dynamics and increase forecast uncertainty. Differencing is a modeling choice, not a rule to repeat until every visible pattern disappears.
What does ARIMA (autoregressive integrated moving average) add to an autoregressive moving-average model?
ARIMA (autoregressive integrated moving average) adds differencing. After \(d\) ordinary differences, the transformed series is modeled with autoregressive order \(p\) and moving-average order \(q\).
What does SARIMA (seasonal autoregressive integrated moving average) add to a nonseasonal model?
SARIMA (seasonal autoregressive integrated moving average) includes seasonal dependence and potentially seasonal differencing at a known period, such as 12 months for annual seasonality. It is not just a separate mean for every month.
Explain the difference between the null hypotheses of ADF (augmented Dickey–Fuller) and KPSS (Kwiatkowski–Phillips–Schmidt–Shin).
The augmented Dickey–Fuller test has a unit-root null. KPSS (Kwiatkowski–Phillips–Schmidt–Shin) has a level-stationarity or trend-stationarity null, depending on its specification.
Choose the deterministic terms and lag or bandwidth settings appropriately. Failure to reject either null does not establish it.
Structural breaks, limited data, and model misspecification can complicate both tests.
What is a naive forecast?
The forecast repeats the most recent observation: \(\hat y_{t+h\mid t}=y_t\). It is a useful simple benchmark.
What is a seasonal-naive forecast?
It repeats the most recent observation from the corresponding season. For example, forecast a Monday using the latest Monday.
What is a drift forecast?
It extends the average observed change per time step beyond the latest observation. It differs from repeating the latest value unchanged.
What is simple exponential smoothing?
A forecasting method that updates a level by averaging the latest observation with the previous level. Older observations receive geometrically decreasing weights.
What does a larger exponential-smoothing parameter do?
It gives more weight to the newest observation, so the level reacts faster. A smaller parameter smooths more heavily.
What shape do future forecasts from simple exponential smoothing have?
They are flat at the latest estimated level. The method does not model a trend or seasonal pattern.
How does Holt’s method extend simple exponential smoothing?
It adds an estimated trend to the level. Forecasts project that trend forward instead of remaining flat. The basic method has no seasonal component.
What does damping a forecast trend do?
It reduces how much of the current slope persists at longer horizons. Rather than extending a fixed slope indefinitely, the additional trend contribution gradually levels off.
How does Holt–Winters differ from Holt’s trend method?
It adds a seasonal component with a specified period, alongside the estimated level and trend.
Explain the difference between additive and multiplicative seasonality.
Additive seasonal effects have roughly constant amplitude in the outcome’s units. Multiplicative effects scale with the level of the series. The latter is generally used with positive data.
Use forecast = level + horizon × slope + seasonal effect. For level \(100\), slope \(2\), horizon \(3\), and seasonal effect \(-5\), calculate the forecast.
\(100+3(2)-5=101\). The seasonal adjustment is added in the same units as the outcome.
Use forecast = (level + horizon × slope) × seasonal factor. For level \(100\), slope \(2\), horizon \(3\), and factor \(1.2\), calculate the forecast.
\((100+3(2))1.2=127.2\). The seasonal factor rescales the level-plus-trend forecast.
Explain the modeling difference between ETS (error, trend, and seasonal) and ARIMA (autoregressive integrated moving average).
ETS (error, trend, and seasonal) describes evolving error, trend, and seasonal components in a state-space model.
ARIMA (autoregressive integrated moving average) models serial dependence after differencing, using autoregressive and moving-average terms.
Some additive exponential-smoothing models have integrated-model equivalents. The families are not identical. Compare out-of-sample forecasts.
What does an additive time-series decomposition represent?
It writes the series as \(y_t=T_t+S_t+R_t\): a trend-cycle component, seasonal component, and remaining variation. These are components of one representation.
Why must seasonal decomposition be fitted inside each forecasting training window?
A decomposition fitted to the full series can use future information to estimate past components. That gives the evaluation information unavailable at the forecast origin.
What does a Ljung–Box test assess?
Whether autocorrelations through selected lags are jointly zero. Its approximate reference degrees of freedom must account for fitted model parameters. Nonrejection does not prove independence or optimal forecasting.
How should a forecasting method be evaluated without training on the future?
At each historical origin, fit using only information available then and predict the required future horizon. Move the origin forward and repeat.
Use expanding or fixed-width training windows to match deployment. Tune using these past-only splits, then evaluate a final untouched future period.
Random row-wise cross-validation can leak future structure.
What makes a lag or rolling-window feature valid for an \(h\)-step-ahead forecast?
It must be computable from information available at the forecast origin. A trailing average must not include the future target or later observations.
For multiple steps, future lagged outcomes may require recursive predictions or separate horizon-specific models. Calendar features can be known in advance, but future demand and realized weather usually are not.
What is a key problem with MAPE (mean absolute percentage error)?
MAPE (mean absolute percentage error) divides absolute errors by the magnitude of each actual outcome. It is undefined at zero and can be dominated by outcomes near zero.
What does MASE (mean absolute scaled error) compare?
MASE (mean absolute scaled error) divides test mean absolute error by a training-data naive forecast error scale. A value below one means lower test absolute error than that reference scale. The scale must be positive.
What does AIC (Akaike information criterion) balance?
AIC (Akaike information criterion) balances fitted log-likelihood against parameter count: \(-2\ell+2k\). Smaller is preferred among models with comparable likelihoods for the same response data.
What does BIC (Bayesian information criterion) balance?
BIC (Bayesian information criterion) balances fitted log-likelihood against a sample-size-dependent complexity penalty: \(-2\ell+k\log n\). Compare only compatible likelihoods on the same response data.
Explain the difference between a point forecast and a prediction interval.
A point forecast gives one prediction. A prediction interval expresses uncertainty about a future observation. Its coverage depends on the forecasting model and uncertainty assumptions.
Does a model-based prediction interval automatically include every source of forecasting uncertainty?
No. It may omit uncertainty about model selection, estimated parameters, or future predictor values. An interval is conditional on the assumptions used to construct it.
How can external predictors be combined with serially correlated errors?
Write \(y_t=x_t^T\beta+n_t\), where \(n_t\) follows an ARIMA (autoregressive integrated moving average)-type error model. This is regression with time-series errors.
Forecasts require future predictor values, forecasts, or scenarios. Intervals conditional on a supplied scenario do not automatically include uncertainty about that scenario.
Predictor exogeneity and correct temporal availability still matter.
What does a standard artificial neuron compute?
\[a=\sigma(w^{\mathsf T}x+b).\] It applies an activation function to a weighted input sum plus an offset.
What is a neuron’s bias parameter?
A trainable offset added before activation, analogous to an intercept. It is not the same concept as statistical estimator bias.
Why do several affine layers without nonlinear activation collapse to one affine layer?
For two layers, \[ A_2(A_1x+b_1)+b_2 =(A_2A_1)x+(A_2b_1+b_2). \] This is affine. Repeating the calculation proves the claim for any finite stack.
Nonlinear activations prevent this collapse and allow an MLP (multilayer perceptron) to represent nonlinear input-output relationships.
What is ReLU (rectified linear unit)?
ReLU (rectified linear unit) is \(\max(0,z)\). It keeps positive inputs and returns zero for negative inputs.
What is the sigmoid activation?
\(\sigma(z)=1/(1+e^{-z})\). It maps real-valued scores to values strictly between zero and one.
What is the tanh (hyperbolic tangent) activation?
The tanh (hyperbolic tangent) activation maps real-valued inputs to \((-1,1)\) and is zero-centered. It saturates at large positive or negative inputs.
Why can saturating activations slow learning?
Their derivatives become very small in saturated regions. Backpropagated gradients can therefore shrink, making parameter updates small.
What is a common output-layer and loss pairing for real-valued regression?
A linear output with squared-error loss. It permits real-valued predictions and targets the conditional mean when the relevant moments exist.
What is a common output for a binary neural-network classifier?
One real-valued logit, trained with binary cross-entropy. A sigmoid converts the logit into a probability. Use a numerically stable loss that accepts logits.
What is a common output for mutually exclusive multiclass classification?
One logit per class, converted to probabilities by softmax and trained with categorical cross-entropy. The probabilities sum to one.
Explain why multilabel classification usually uses independent binary outputs instead of one softmax.
Several labels may be true at once. A single softmax forces mutually exclusive probabilities summing to one, which does not match that task.
What does a neural network’s forward pass compute?
It carries the inputs through the layers to compute activations and predictions, followed by the loss when training labels are available.
What is backpropagation?
Applying the chain rule backward through the computational graph to calculate parameter gradients. It computes gradients, not the parameter update itself.
Explain the difference between backpropagation and an optimizer.
Backpropagation computes loss gradients. An optimizer uses those gradients to update the parameters.
What is the basic gradient-descent update?
\[\theta\leftarrow\theta-\eta\nabla L(\theta).\] Move opposite the loss gradient, with step size controlled by learning rate \(\eta\).
Where does randomness in minibatch gradient descent come from?
Each update uses a sampled subset of training cases rather than the entire dataset. At fixed parameters, uniform sampling gives an unbiased estimate of the full mean-loss gradient.
Explain the difference between an epoch, batch size, and optimizer step.
An epoch is one pass through the training set. Batch size is the number of cases used for an update. A step is one optimizer update.
With 1,000 training cases and batch size 64, how many optimizer steps are in an epoch if the last partial batch is retained?
\(\lceil1000/64\rceil=16\). The last step uses a smaller batch.
A dense network has 10 inputs, 16 hidden units, and one output. Each layer has biases. How many trainable parameters?
Each layer has input-width times output-width weights plus one bias per output unit: \[(10\cdot16+16)+(16\cdot1+1)=193.\]
Explain the tradeoff between a learning rate that is too large and one that is too small.
Too large can cause unstable updates or divergence. Too small can make learning very slow. The useful range depends on the model and optimizer.
What is Adam (adaptive moment estimation)?
Adam (adaptive moment estimation) is an optimizer that adapts updates using moving estimates of gradient first and second moments, with bias correction. It still needs a suitable base learning rate.
Why should hidden neurons not all be initialized identically?
If their incoming and outgoing roles are identical, they can receive identical updates and remain redundant. Random initialization breaks that symmetry.
What are vanishing gradients?
Gradients become very small as they are propagated through many layers. Early layers then receive weak learning signals. Repeated contraction or activation saturation can cause this.
What are exploding gradients?
Gradients become very large as they are propagated through layers, potentially causing unstable updates. Repeated expansion through the layer derivatives can cause this.
What does gradient clipping address?
It limits excessively large gradient updates. It can help with exploding gradients but does not directly fix vanishing gradients.
What is dropout in a neural network?
A training-time regularization method that randomly masks some activations. It discourages reliance on particular activation combinations.
How does standard dropout behavior differ between training and inference?
Training randomly masks activations. Standard inference disables masking. With inverted dropout, training scales the retained activations so no extra inference scaling is required.
What is batch normalization?
It normalizes selected activations using batch statistics and then applies a learned scale and shift. It is not the same as standardizing input features.
Why do batch-normalization training and evaluation modes differ?
Training typically uses current-batch statistics. Evaluation typically uses accumulated statistics, so a prediction is not normalized using an arbitrary test batch.
Explain the difference between a weight penalty and early stopping.
A weight penalty changes the training objective to discourage large weights. Early stopping chooses a checkpoint based on validation performance. Neither should be chosen using final test labels.
What is the main structural idea in a CNN (convolutional neural network)?
A CNN (convolutional neural network) reuses local filters across positions. This encodes locality and weight sharing, often useful for spatially structured inputs.
What is the main structural idea in an RNN (recurrent neural network)?
An RNN (recurrent neural network) repeatedly applies a shared state-update rule across a sequence. A hidden state carries information from earlier inputs.
Do neural networks remove the need for statistical assumptions or sound evaluation?
No. Training data, labels, loss, architecture, and validation design must match the intended task and deployment population.
An MLP (multilayer perceptron) does not require Gaussian predictors or a linear conditional mean, but flexibility does not prevent leakage, confounding, distribution shift, or overfitting.
Low training loss and many parameters alone do not establish reliable out-of-sample performance.
What is the entropy of a discrete random variable?
Expected surprise: \[H(X)=-\sum_xp_x\log p_x.\] Use \(0\log0=0\). A deterministic outcome has entropy zero.
What determines the units of entropy?
The logarithm base. Base-two logarithms give bits, while natural logarithms give nats.
When is entropy largest for a fixed finite set of possible outcomes?
When the probabilities are uniform. With \(K\) possible outcomes, the maximum is \(\log K\) in the chosen logarithm base.
What is the entropy of a fair coin in bits?
\(-2(1/2)\log_2(1/2)=1\) bit. Both outcomes are equally likely.
Using \(H(p)=-p\log_2p-(1-p)\log_2(1-p)\), find the entropy for head probability \(1/4\). You may use \(\log_2(3/4)\approx-0.4150\).
\(H(1/4)=1/2-(3/4)\log_2(3/4)\approx0.8113\) bits. Bias makes the outcome more predictable than a fair coin.
What is cross-entropy between a true distribution p and a predicted distribution q?
The expected surprise assigned by \(q\) to outcomes generated by \(p\): \[H(p,q)=-\sum_i p_i\log q_i.\] Assigning zero predicted probability to a possible true outcome produces infinite loss.
What does KL (Kullback–Leibler) divergence compare?
KL (Kullback–Leibler) divergence compares two probability distributions: \[D_{\mathrm{KL}}(p\|q)=\sum_i p_i\log\frac{p_i}{q_i}.\] It is nonnegative but not symmetric.
How does cross-entropy relate to entropy and divergence?
KL (Kullback–Leibler).
For finite distributions when the quantities are finite, \[H(p,q)=H(p)+D_{\mathrm{KL}}(p\|q).\] KL (Kullback–Leibler) divergence is therefore the extra expected surprise from using \(q\) instead of \(p\).
What is conditional entropy for finite discrete variables?
The average remaining entropy of one variable after another is observed. Average over the possible observed values, using their probabilities.
What does mutual information measure?
How much observing one variable reduces uncertainty about the other on average. For finite discrete variables, \[I(X,Y)=H(X)-H(X\mid Y).\] It is zero exactly under independence.
What is differential entropy?
For a continuous density \(f\), \[h(X)=-\int f(x)\log f(x)\,dx\] when defined. It is expected negative log density, measured relative to the chosen coordinate scale.
Can differential entropy be negative?
Yes. A density can exceed one. For a uniform variable on \((0,1/2)\), the density is \(2\), so \(h(X)=-\log2<0\).
Does differential entropy change when measurement units change?
Yes. For finite entropy and \(a\ne0\), \[h(aX+b)=h(X)+\log|a|.\] Changing scale changes the value, while translation does not.
A leaf has 75% positive and 25% negative cases. Using Gini impurity = one minus the sum of squared class proportions, calculate its impurity.
\(1-0.75^2-0.25^2=0.375\). A leaf with only one class would have zero impurity.
Compare binary cross-entropy loss for a true positive label predicted with probability \(0.9\) versus \(0.1\). You may use \(-\log0.9\approx0.1054\) and \(-\log0.1\approx2.3026\).
\[ \ell(y,q)=-y\log q-(1-y)\log(1-q). \] For \(y=1\), the losses are \(-\log0.9\approx0.1054\) and \(-\log0.1\approx2.3026\).
A confidently wrong prediction is costly. Averaging this loss over independent observations is equivalent to maximizing their Bernoulli likelihood.
What is Markov’s inequality?
For \(X\ge0\), finite \(\mathbb E[X]\), and \(a>0\), \[P(X\ge a)\le\frac{\mathbb E[X]}a.\] Nonnegativity is essential.
Why does Markov’s inequality hold?
For nonnegative \(X\), \(X\ge a\mathbf1_{\{X\ge a\}}\). Taking expectations gives \(\mathbb E[X]\ge aP(X\ge a)\). Divide by \(a>0\).
A nonnegative waiting time has mean \(4\) minutes. Without assuming a distribution, bound the probability it is at least \(20\) minutes.
\[ P(X\ge20)\le\frac{\E X}{20}=\frac4{20}=0.20. \] The bound is sharp: put probability \(0.20\) at \(20\) and probability \(0.80\) at \(0\).
This is an upper bound, not an estimate of the actual tail probability.
What is Chebyshev’s inequality?
For mean \(\mu\), finite variance \(\sigma^2\), and \(a>0\), \[P(|X-\mu|\ge a)\le\frac{\sigma^2}{a^2}.\] No normality assumption is needed.
How do you derive Chebyshev’s inequality from Markov’s inequality?
Apply Markov’s inequality to nonnegative \((X-\mu)^2\) with threshold \(a^2\). Its expectation is \(\sigma^2\), giving \(P(|X-\mu|\ge a)\le\sigma^2/a^2\).
Independent observations have mean \(\mu\) and variance \(9\). How large must \(n\) be to guarantee \(P(|\bar X-\mu|\ge0.5)\le0.05\)?
Independence gives \(\Var(\bar X)=9/n\). Hence \[ P(|\bar X-\mu|\ge0.5)\le\frac{9/n}{0.5^2} =\frac{36}{n}. \] It suffices that \(n\ge720\).
No normality assumption is needed. This conservative guarantee differs from an approximate normal calculation.
Independent identically distributed observations have mean \(20\) and variance \(16\). For \(n=100\), approximate \(P(\bar X>20.8)\). You may use \(P(Z>2)=0.0228\).
The CLT (central limit theorem) gives the approximation \[ \bar X\approx N(20,16/100). \] The standard error is \(0.4\), so \[ P(\bar X>20.8)\approx P(Z>2)\approx0.0228. \] This is exact for normal data. Otherwise its accuracy depends on the population distribution and sample size.
Why is “\(n\ge30\) makes the data normal” an incorrect use of the central limit theorem?
The classical CLT (central limit theorem) concerns a standardized sum or mean, not the individual observations.
It assumes independent identically distributed observations and a finite, positive variance. There is no universal cutoff at \(30\). Strong skewness, heavy tails, or dependence can make a simple approximation poor or invalidate this version.
Compare the normal 68–95–99.7 rule with distribution-free bounds within two and three standard deviations.
For a normal variable, the probabilities within one, two, and three standard deviations are about \(68.27\%\), \(95.45\%\), and \(99.73\%\).
Chebyshev guarantees at least \(75\%\) within two and at least \(88.89\%\) within three for any finite positive variance.
The normal percentages are not universal.
Independent trials succeed with probability \(0.2\). If \(T\) counts trials through the first success, find \(P(T>5)\).
The first five trials must all fail: \(P(T>5)=0.8^5=0.32768\).
Independent trials succeed with probability \(0.2\). After five failures, what is the expected number of additional trials until success?
Five additional trials. Memorylessness leaves the expected remaining wait at \(1/0.2\), regardless of the failures already observed.
How can first-step reasoning derive the expected number of trials through the first success?
For success probability \(0<p\le1\), the geometric tail has finite mean \(m\). One trial is always used. After a failure the process restarts, so \(m=1+(1-p)m\). Hence \(m=1/p\).
Independent normal observations have known variance \(16\), \(n=64\), and mean \(11.2\). Test \(H_0:\mu=10\) two-sided using the supplied rule: reject when \(|Z|>1.96\).
\[Z=\frac{11.2-10}{\sqrt{16/64}}=2.4.\] Since \(2.4>1.96\), reject the null. This uses known population variance.
An observed normal-reference statistic is \(z=2.4\). Given \(\Phi(2.4)=0.9918\), calculate its two-sided p-value.
\(p=2(1-0.9918)=0.0164\). Both tails are included because the test is two-sided.
A prespecified test yields p = 0.0164. Compare its decisions at significance levels 0.05 and 0.01.
Reject at 0.05 but not at 0.01. The same evidence is assessed against two different decision thresholds.
What is the one-sample Student’s \(T^*\) test statistic?
\[T^*=\frac{\bar X-\mu_0}{S/\sqrt n}.\] It standardizes the mean’s difference from the null using an estimated standard error. Independent normal data give \(n-1\) degrees of freedom under the null.
Explain the difference between an observed test statistic and a critical value.
The statistic is calculated from the data. The critical value is a cutoff from the null reference distribution and chosen significance level. Compare them to apply the rejection rule.
Independent normal data give \(n=16\), \(\bar x=53\), and \(s=4\). Test mean \(50\) two-sided using the supplied critical value \(2.131\).
\[t=\frac{53-50}{4/\sqrt{16}}=3.\] Reject because \(|3|>2.131\). The critical value is provided, not a number to memorize.
Explain the difference between rejection for an upper one-sided t-test and a two-sided t-test.
An upper one-sided test rejects for a sufficiently large positive statistic. A two-sided test rejects for sufficiently large magnitude in either direction. Choose the direction before examining the result.
For independent observations with common known variance \(v\), what is the standard error of their sample mean?
\(\sqrt{v/n}\). Take the square root of the mean’s variance, not the variance itself.
Independent normal data have known variance \(36\), \(n=100\), and mean \(50\). Use estimate ± \(1.96\) × standard error to calculate the 95% interval.
The standard error is \(\sqrt{36/100}=0.6\). Thus \(50\pm1.96(0.6)=[48.824,51.176]\).
What is the basic structure of a symmetric confidence interval?
\(\text{estimate}\pm\text{margin of error}\). The margin is an appropriate critical multiplier times the standard error. The multiplier depends on the method and its assumptions.
Two independent normal groups have means \(20,22\), sample standard deviations \(5,8\), and \(100\) cases each. Using standard error \(\sqrt{25/100+64/100}\) and a supplied two-sided critical value \(1.975\), is the difference significant?
The statistic is \(2/\sqrt{0.89}\approx2.120\). Since \(2.120>1.975\), reject equal means. Separate variance estimates are used rather than a pooled variance.
Twelve independent normal differences have mean \(1.5\) and standard deviation \(2\). Test zero mean difference using a supplied two-sided critical value \(2.201\).
\(t=1.5/(2/\sqrt{12})\approx2.598\). Reject because \(2.598>2.201\). The sample size counts pairs, not individual before-and-after measurements.
A randomized experiment has conversion rates \(0.10\) and \(0.13\). The pooled test standard error is supplied as \(0.014267\). With independent users and a two-sided 5% cutoff of \(1.96\), do you reject equality?
\(Z=(0.13-0.10)/0.014267\approx2.103\). Reject equality because \(2.103>1.96\). This does not establish that the improvement is large enough to justify a decision.
An estimated conversion difference is \(0.03\), with unpooled standard error \(0.014251\). Use estimate ± \(1.96\) × standard error to calculate an approximate 95% interval.
\(0.03\pm1.96(0.014251)\approx[0.00207,0.05793]\). This is about 0.21 to 5.79 percentage points.
Observed rows are \((100,900)\) and \((130,870)\). Expected rows under independence are both \((115,885)\). Calculate \(\sum(O-E)^2/E\).
\[X^2=2\left(\frac{15^2}{115}+\frac{15^2}{885}\right)\approx4.4215.\] The expected counts are provided, so the target is the statistic calculation.
In \(16\) independent flips, \(12\) heads are observed. For \(H_0:p\le1/2\) against \(H_1:p>1/2\), which binomial tail defines the p-value?
Use \(P(X\ge12)\) for \(X\sim\operatorname{Binomial}(16,1/2)\). The observed value is included. It is approximately \(0.03841\).
In an A/B design, explain the difference between 80% power at a chosen effect and a 5% significance level.
The power describes how often the test rejects when the chosen effect is real. The significance level limits false rejections under the null. Sample-size planning must satisfy both at the specified effect and variability.
Compare Bonferroni and Holm decisions at family-wise level 0.05 for three prespecified p-values: 0.01, 0.03, and 0.04.
Bonferroni compares each with \(0.05/3\approx0.0167\). Only the first is rejected.
Holm compares the sorted values with \(0.0167,0.025,0.05\) in sequence. The first passes, the second fails, so stop. Only the first is rejected.
Both control FWER (family-wise error rate) for valid p-values under arbitrary dependence.
An experiment reports lift \(0.5\) percentage points, a \(95\%\) interval \([-0.4,1.4]\), and \(p=0.28\). What can be concluded?
Do not reject zero effect at \(5\%\). This is not evidence that the effect is exactly zero or that the variants are equivalent.
The interval allows both harm and benefit. Compare it with a prespecified practical margin. Review power and data quality. Do not extend the experiment merely until a conventional p-value becomes significant.
Why maximize log likelihood instead of likelihood?
The logarithm is strictly increasing, so it preserves maximizers. It converts products of independent-observation likelihood terms into sums, making differentiation and numerical calculation easier.
Why is setting the derivative of log likelihood to zero not enough?
A stationary point need not be a global maximum. You must also consider curvature, parameter constraints, and boundaries.
Why must parameter-dependent support be checked in maximum likelihood?
Parameter values that cannot produce the observations have zero likelihood. Differentiating a formula while ignoring those support restrictions can give an invalid answer.
What likelihood describes \(h\) heads in \(n\) independent flips with head probability \(\theta\)?
\(L(\theta)\propto\theta^h(1-\theta)^{n-h}\). The binomial coefficient for a count does not depend on \(\theta\), so it does not affect maximization.
What is the MLE (maximum likelihood estimator) of head probability after \(h\) heads in \(n\ge1\) independent flips?
The MLE (maximum likelihood estimator) is \(\hat\theta=h/n\), the observed head proportion. It is zero with no heads and one with all heads.
A coin shows seven heads in ten independent flips. What is the maximum likelihood estimate of head probability?
\(7/10=0.7\). This uses the observed head proportion, without a prior.
For observed data, \(L(0.7)/L(0.5)=2.277\). How do you interpret that ratio?
Those observed data are 2.277 times as likely under the parameter value 0.7 as under 0.5. This is not automatically a ratio of posterior probabilities for the two parameter values.
For independent normal data with common unknown mean and variance, what maximizes the likelihood over the mean?
The sample mean \(\bar x\), whenever the joint finite maximizer exists. For a nonconstant sample, this is the mean component of the normal maximum likelihood estimate.
Explain the difference between the normal maximum likelihood variance estimate and the unbiased sample variance.
For a nonconstant sample with unknown mean, maximum likelihood divides \(\sum_i(x_i-\bar x)^2\) by \(n\). The unbiased sample variance divides by \(n-1\). For \(n\ge2\), the former is biased downward under the normal model.
For independent exponential waiting times, the log likelihood is \(\ell(\lambda)=n\log\lambda-\lambda\sum_i x_i\), with \(\sum_i x_i>0\). What rate maximizes it?
Setting \(n/\lambda-\sum_i x_i=0\) gives \(\hat\lambda=n/\sum_i x_i=1/\bar x\). The second derivative is \(-n/\lambda^2<0\), so this is the unique maximum.
For \(0<x_i<1\) and \(\theta>-1\), the supplied log likelihood is \(\ell(\theta)=n\log(\theta+1)+\theta\sum_i\log x_i\). Find its maximizer.
\(\ell^\prime(\theta)=n/(\theta+1)+\sum_i\log x_i=0\) gives \[\hat\theta=-\frac n{\sum_i\log x_i}-1.\] It lies above \(-1\), and \(\ell^{\prime\prime}(\theta)<0\), so it uniquely maximizes the objective.
For fixed \(x_i\), let \(Y_i=\theta g(x_i)+\varepsilon_i\), with independent \(N(0,\sigma^2)\) errors and known \(\sigma^2>0\). Find the MLE (maximum likelihood estimator) of \(\theta\).
Write \(g_i=g(x_i)\) and assume \(\sum_i g_i^2>0\). MLE (maximum likelihood estimator) minimizes \(\sum_i(y_i-\theta g_i)^2\). \[ \hat\theta=\frac{\sum_i g_i y_i}{\sum_i g_i^2}. \] This follows by setting \(\sum_i g_i(y_i-\theta g_i)=0\). The objective is strictly convex. There is no intercept in this model.
For \(Y_i=\theta x_i^2+\varepsilon_i\) with independent equal-variance Gaussian errors, use \(\hat\theta=\sum g_i y_i/\sum g_i^2\), where \(g_i=x_i^2\). Given \(x=(1,2,3)\), \(y=(2,7,19)\), calculate the estimate.
The curve values are \((1,4,9)\), so \[\hat\theta=\frac{2+28+171}{1+16+81}=\frac{201}{98}\approx2.0510.\]
How does scalar curve-weight estimation extend to \(f_\theta(x)=\sum_{j=1}^d\theta_jg_j(x)\) with independent equal-variance Gaussian errors?
Let \(G_{ij}=g_j(x_i)\). MLE (maximum likelihood estimator) minimizes \[ \|y-G\theta\|^2. \] If \(G\) has full column rank, \[ \hat\theta=(G^{\mathsf T}G)^{-1}G^{\mathsf T}y. \] Polynomial curves use \(g_j(x)=x^{j-1}\). Linearity in coefficients matters, even when the fitted curve is nonlinear in \(x\).
Why is an unobserved mixture weight not generally estimated by a simple observed success proportion?
The component membership of each observation is unknown. Estimate the weight by maximizing the mixture likelihood, generally using a one-dimensional optimization that includes the allowed endpoints.
What is MAP (maximum a posteriori) estimation?
MAP (maximum a posteriori) estimation chooses a posterior mode. It maximizes likelihood times prior density or mass over the allowed parameter values.
Explain the difference between MLE (maximum likelihood estimation) and MAP (maximum a posteriori) estimation.
MLE (maximum likelihood estimation) maximizes the likelihood. MAP (maximum a posteriori) estimation also includes a prior. With a constant prior over the same support, their maximizers coincide.
Given a posterior \(\operatorname{Beta}(A,B)\) with \(A,B>1\), what is the MAP (maximum a posteriori) estimate?
MAP (maximum a posteriori) is the interior mode: \[\frac{A-1}{A+B-2}.\] The shape conditions ensure that the mode is inside the interval.
Seven heads in ten flips with a \(\operatorname{Beta}(3,3)\) prior give posterior \(\operatorname{Beta}(10,6)\). Using mode \((A-1)/(A+B-2)\), calculate MAP (maximum a posteriori).
The MAP (maximum a posteriori) estimate is \(9/14\approx0.6429\). The prior pulls the estimate toward one half.
What is a posterior mean?
The expectation of the unknown parameter under its posterior distribution. It minimizes posterior expected squared error when those expectations exist.
For a posterior \(\operatorname{Beta}(10,6)\), use mean \(A/(A+B)\) to calculate the posterior mean.
\(10/(10+6)=0.625\). This is a mean, not the posterior mode.
Why check shape conditions before using the beta posterior’s interior-mode formula?
The interior formula requires both shapes to exceed one. Otherwise the highest density can be at a boundary, unbounded there, or constant across the interval. The interior formula should not be applied blindly.
How does a normal prior change the estimate of a normal mean compared with maximum likelihood?
Maximum likelihood uses the sample mean. A normal prior produces a precision-weighted average of the sample mean and prior mean. A more concentrated prior exerts a stronger pull toward its mean.
For positive \(\sigma^2,\tau^2\), the negative log posterior is proportional to \(\sum_i(y_i-\theta g_i)^2/(2\sigma^2)+\theta^2/(2\tau^2)\). Which \(\theta\) minimizes it?
Differentiate the supplied quadratic and set its derivative to zero: \[\hat\theta=\frac{\sum_i g_i y_i}{\sum_i g_i^2+\sigma^2/\tau^2}.\] The positive second derivative gives a unique minimum.
What is the Markov property?
Given the current state, the distribution of the next state does not depend on earlier states. The current state contains the history relevant to the next transition.
What does time homogeneity mean for a Markov chain?
The transition probabilities do not change with the step number. The same transition matrix describes each step.
What is a Markov transition matrix?
An entry \(p_{ij}\) is the probability of moving from current state \(i\) to next state \(j\). Entries are nonnegative and each row sums to one.
How do you find the probability of moving from state \(i\) to state \(j\) in \(n\) steps of a time-homogeneous chain?
Use entry \((i,j)\) of \(P^n\), the nth matrix power of the one-step transition matrix.
For row-vector initial distribution \(\mu_0\) and transition matrix \(P\), what is the distribution after \(n\) steps?
\(\mu_n=\mu_0P^n\). Keep the row-vector convention consistent.
When is one Markov state accessible from another?
State \(j\) is accessible from \(i\) when \((P^n)_{ij}>0\) for some \(n\ge0\). There is a positive-probability path.
When do two Markov states communicate?
When each is accessible from the other. Communication partitions the state space into communicating classes.
What is an irreducible Markov chain?
A chain in which all states communicate. Every state can be reached from every other with positive probability in some number of steps.
What is a recurrent Markov state?
Starting at that state, the probability of ever returning at a positive time is one. The initial time does not itself count as a return.
What is a transient Markov state?
Starting at that state, the probability of ever returning at a positive time is less than one. It can still be revisited, but return is not certain.
Explain the difference between positive and null recurrence.
Both guarantee eventual return with probability one. Positive recurrence has finite expected return time, while null recurrence has infinite expected return time.
What does finiteness and irreducibility guarantee about a Markov chain’s recurrence?
Every state is positive recurrent. The chain has a unique stationary probability distribution, though convergence from a fixed start can still require aperiodicity.
What is the period of a Markov state that can return to itself?
The greatest common divisor of all positive times at which return has positive probability. Period one means aperiodic.
What does a positive self-loop imply for a finite irreducible chain?
The chain is aperiodic. That state can return in one step, and communicating states share a period.
What is a stationary probability distribution for a Markov chain?
A row vector \(\pi\) is stationary when \[ \pi P=\pi,\qquad \pi_i\ge0,\qquad\sum_i\pi_i=1. \] Starting with \(X_0\sim\pi\) keeps every \(X_n\) distributed as \(\pi\).
Stationary does not by itself mean that every starting distribution converges to \(\pi\). Reducible chains can have several stationary distributions.
Find the stationary distribution of \(P=\begin{pmatrix}0.8&0.2\\0.3&0.7\end{pmatrix}\).
Solve flow balance \(0.2\pi_1=0.3\pi_2\) together with \(\pi_1+\pi_2=1\). \[ \pi=(0.6,0.4). \] Directly, \((0.6,0.4)P=(0.6,0.4)\).
This chain is finite, irreducible, and aperiodic, so every initial distribution converges to this stationary distribution.
Why does a stationary distribution not guarantee convergence from a fixed start for \(P=\begin{pmatrix}0&1\\1&0\end{pmatrix}\)?
The unique stationary distribution is \((1/2,1/2)\).
Starting from state one, the distribution alternates between \((1,0)\) and \((0,1)\), so it does not converge.
The chain has period two. For finite irreducible chains, aperiodicity ensures convergence of distributions. Long-run occupation averages do not require aperiodicity.
What is detailed balance?
A probability vector \(\pi\) satisfies \(\pi_i p_{ij}=\pi_j p_{ji}\) for every pair of states. Probability flow is balanced in each direction between every pair.
Why does detailed balance imply stationarity?
Summing \(\pi_i p_{ij}=\pi_j p_{ji}\) over \(i\) gives \((\pi P)_j=\pi_j\sum_i p_{ji}=\pi_j\). The last sum is row \(j\) of the transition matrix, so it equals one.
What is an absorbing state?
A state that cannot be left: \(p_{aa}=1\). After entering it, the chain stays there.
What makes a finite chain an absorbing chain?
It has an absorbing state, and every nonabsorbing state can reach an absorbing state. Merely having an absorbing state somewhere is insufficient.
How do you calculate the probability of absorption in a particular absorbing state \(a\)?
Let \(h_i=P_i(\text{eventually absorbed at }a)\). Set \(h_a=1\) and \(h_b=0\) at other absorbing states.
For transient states, condition on the first step: \[ h_i=\sum_jp_{ij}h_j. \] Solve this linear system with the boundary values. These probabilities are not the same as one-step transition probabilities.
How do you set up equations for expected steps until absorption in a finite absorbing chain?
Let \(T\) be the first time in any absorbing state and \(m_i=\E_i[T]\).
Set \(m_a=0\) at absorbing states. For transient states, \[ m_i=1+\sum_jp_{ij}m_j. \] The extra one counts the step just taken. Solve for \(m_i\). Missing this one is a common error.
What do entries of the fundamental matrix in a finite absorbing chain represent?
With transient-to-transient matrix \(Q\), \(N=(I-Q)^{-1}\). Entry \(N_{ij}\) is the expected number of visits to transient state \(j\) starting from transient state \(i\), counting time zero.
In a finite absorbing chain, given fundamental matrix \(N\) and transient-to-absorbing matrix \(R\), how do you obtain absorption probabilities?
\(B=NR\). Entry \(B_{ia}\) is the probability of eventual absorption at absorbing state \(a\), starting from transient state \(i\).
How do fundamental-matrix visit counts give expected time to absorption?
Sum the entries in the starting state’s row. Each transient visit before absorption contributes one elapsed step, so the row sum is the expected number of steps.
From \(s\), move to absorbing \(a\) with probability \(0.2\), absorbing \(b\) with \(0.3\), or remain at \(s\) with \(0.5\). Find the probability of eventual absorption at \(a\).
Let it be \(h\). First-step reasoning gives \(h=0.2+0.5h\), so \(h=0.4\).
From a transient state, each step is absorbing with probability \(0.5\) and otherwise remains in the same state. What is the expected time to absorption?
\(m=1+0.5m\), so \(m=2\) steps. Equivalently, this is a geometric trial count with success probability \(0.5\).
From \(u\), move to \(v\) or absorbing \(a\), each with probability \(1/2\). From \(v\), move to \(u\) or absorbing \(b\), each with probability \(1/2\). Starting at \(u\), find the probability of absorption at \(a\).
\(h_u=\tfrac12h_v+\tfrac12\) and \(h_v=\tfrac12h_u\). Substitution gives \(h_u=\tfrac14h_u+\tfrac12\), so \(h_u=2/3\).
A chain moves from \(u\) to \(v\) or absorbing \(a\) with equal probability, and from \(v\) to \(u\) or absorbing \(b\) with equal probability. What is the expected absorption time from \(u\)?
At either transient state, the next step absorbs with probability \(1/2\). Thus the absorption time is geometric with mean \(2\) steps.
What is a reasonable first baseline for a rare-event binary classifier?
A regularized logistic model, evaluated with a suitable split and minority-class metrics. Compare weighting or resampling rather than assuming a more complicated method is necessary.
What class prevalence should an imbalanced classifier’s final evaluation reflect?
The intended deployment prevalence. Artificially balancing the final test set can distort quantities such as precision.
Which data should be used to select a classifier’s decision threshold?
Validation data, not final test labels. Fix the threshold policy before the final evaluation.
How can a classifier achieve 99% accuracy while detecting none of the positive cases?
If only 1% of cases are positive, predicting negative for every case is 99% accurate. Its recall is zero. Accuracy alone hides the failure.
A confusion matrix has 40 true positives, 10 false positives, 20 false negatives, and 930 true negatives. Calculate precision.
\(40/(40+10)=0.80\). The denominator counts positive predictions.
A confusion matrix has 40 true positives, 10 false positives, 20 false negatives, and 930 true negatives. Calculate recall.
\(40/(40+20)=2/3\approx0.667\). The denominator counts actual positives.
A confusion matrix has 40 true positives, 10 false positives, 20 false negatives, and 930 true negatives. Calculate specificity.
\(930/(930+10)=93/94\approx0.989\). The denominator counts actual negatives.
A confusion matrix has 40 true positives, 10 false positives, 20 false negatives, and 930 true negatives. Calculate accuracy.
\((40+930)/1000=0.97\). The numerator counts all correct predictions.
Precision is \(0.8\) and recall is \(2/3\). Use \(F_1=2P R/(P+R)\), where \(P\) is precision and \(R\) is recall, to calculate \(F_1\).
\[\frac{2(0.8)(2/3)}{0.8+2/3}=\frac8{11}\approx0.727.\]
Positive scores are \(0.9,0.6\) and negative scores are \(0.7,0.2\). Calculate ROC (receiver operating characteristic) AUC (area under the curve) without selecting a classification threshold.
ROC (receiver operating characteristic) compares sensitivity with false-positive rate over thresholds. Its AUC (area under the curve) counts positive–negative ordering.
Of four pairs, \(0.9\) beats both negatives and \(0.6\) beats only \(0.2\). There are no ties, so \[ \operatorname{AUC}=3/4=0.75. \] Strictly increasing score transformations preserve this area, but may change calibration.
Why inspect a precision–recall curve as well as a receiver operating characteristic curve for rare positives?
A ROC (receiver operating characteristic) curve uses rates within each true class. Many negatives can produce many false alerts even at a low false-positive rate.
Precision–recall curves show detection versus alert purity. No-information population precision equals prevalence.
Report the operating point and confusion counts, not just ranking area.
Where should oversampling, undersampling, or SMOTE (synthetic minority over-sampling technique) be placed during cross-validation?
Apply SMOTE (synthetic minority over-sampling technique) or other resampling only inside each training fold, after the split. Leave validation and test observations untouched and representative of deployment.
Resampling the full dataset first can place duplicates or related synthetic points on both sides of a split.
Resampling is optional, not a universal cure. Compare it with class weighting and threshold tuning.
A calibrated positive probability is \(p\). A false negative costs \(20\), a false positive costs \(1\), and correct decisions cost zero. When should you predict positive?
Predicting positive has expected cost \(1-p\). Predicting negative has expected cost \(20p\). \[ 1-p<20p\quad\Longleftrightarrow\quad p>1/21\approx0.0476. \] This is not a universal \(0.5\) rule. It assumes the probability is calibrated for deployment and the stated costs are correct.
Why can class weighting change probability calibration?
It changes the relative cost of fitting positive and negative outcomes. The probabilities that optimize that weighted loss need not match probabilities at the original prevalence. Check calibration on untouched deployment-like data.
What is model prediction bias at a fixed input?
The difference between the average prediction across repeated training datasets and the true conditional mean at that input.
What is model prediction variance at a fixed input?
How much the model’s prediction varies across repeated training datasets. It is not the same as outcome noise.
What is irreducible noise in a squared prediction-error decomposition?
Outcome variation around the true conditional mean that remains even if that mean were known. It is the conditional noise variance.
At a fixed input, model A has bias \(2\) and prediction variance \(1\). Model B has bias \(1\) and variance \(6\). Noise variance is \(3\). Which has smaller expected squared error?
For A, total error is \[ 2^2+1+3=8. \] For B, it is \(1^2+6+3=10\).
Model B has smaller bias but worse total expected error. The calculation assumes a common target, loss, and noise level, and independent training and test observations.
Explain the difference between robust standard errors and weighted least squares.
Robust standard errors change estimated coefficient uncertainty, not the coefficients. Weighted least squares changes the fitted coefficients by weighting residuals, usually using a justified model of unequal error variances.
How do you interpret a coefficient in a log-link Poisson regression without interactions?
A one-unit predictor increase, holding the others fixed, multiplies the conditional mean count by \(e^{\beta_j}\). It is a multiplicative, not additive, mean effect.
For independent counts with \(\mu_i=e^{x_i^{\mathsf T}\beta}\), what terms determine the Poisson log likelihood?
\[\ell(\beta)=\sum_i\bigl[y_i\log\mu_i-\mu_i-\log(y_i!)\bigr].\] The factorial term is constant in \(\beta\).
For log-rate model \(\log(\mu/t)=-1+0.4x\), find the expected count at \(t=10\), \(x=2\). You may use \(e^{-0.2}=0.81873\).
The rate is \(e^{-0.2}\), so the expected count is \(10(0.81873)=8.1873\).
In a Poisson rate model, what happens to expected count when exposure doubles but the rate stays fixed?
The expected count doubles. Exposure changes the amount of opportunity for events, not the rate per unit exposure.
What sort of outcome can gamma regression model?
A strictly positive, skewed outcome, using a suitable link for its conditional mean. The distributional and mean assumptions must fit the task.
What does quantile regression target?
A conditional quantile, such as the median or 90th percentile, rather than necessarily the conditional mean.
What is probit regression?
A binary response model with \(P(Y=1\mid x)=\Phi(x^{\mathsf T}\beta)\), where \(\Phi\) is the standard normal cumulative distribution function.
Explain the difference between logistic and probit probability links.
Logistic uses the sigmoid \(1/(1+e^{-\eta})\). Probit uses the standard normal cumulative distribution function \(\Phi(\eta)\). Both map a real-valued score to a binary-event probability.