Dealing with Logs and Zeros in Regression Models

summary

The gist

The paper introduces Iterated Ordinary Least Squares (iOLS) as a theoretically grounded estimator for the normalized average treatment effect (AT Enorm = E[Y(1) - Y(0)]/E[Y(0)]) in binary treatment…

In short

Iterated Ordinary Least Squares (iOLS) is a theoretically grounded estimator for normalized average treatment effect in binary settings. It replaces ad-hoc log transformations with an iterative OLS process, offering consistency and asymptotic normality for exponential conditional mean models. iOLS handles heteroskedasticity and incidental parameters better than log-linear methods.

Key concepts

Iterated Ordinary Least Squares (iOLS)
A method that uses a sequence of simple linear regressions to estimate structural parameters. It replaces the standard log transformation for treatment effects, providing a consistent and asymptotically normal estimator that avoids common pitfalls like ad-hoc shifts and incidental parameter bias.
Exponential Conditional Mean Model
A statistical model used when the expected outcome depends on a treatment through an exponential function. iOLS is designed to estimate the underlying structural parameters of this specific type of model, which is more flexible than simple linear models.
Iterated Two-Stage Least Squares (i2SLS)
An extension of iOLS that allows the estimation to handle endogenous regressors. By applying linear regression iteratively, i2SLS maintains computational simplicity while ensuring consistent and asymptotically normal estimates even when variables are correlated with the treatment.
Incidental Parameters
These are extra parameters in a model that do not affect the parameter of primary interest but can cause estimation bias. iOLS addresses this by using Frisch-Waugh-Lovell projections at each step, effectively eliminating incidental parameter bias without needing case-specific corrections.

Terminology used across episodes

This episode discusses

The paper

Dealing with Logs and Zeros in Regression Models · Read on arXiv

HEC Montréal · CREST (UMR 9194) · Telecom Paris

Log regression is widely used to express effects in percentage terms, but zeros, heterogeneity and retransformation pose distinct difficulties. We unify the interpretation, identification and estimation of proportional effects through weighted individual percentage changes. We connect these parameters to pseudo-maximum likelihood estimation, characterize finite-sample differences between consistent estimators, and establish relationships with log regression. We develop a new shifted-log transformation that yields unit-invariant proportional-effect parameters and use it to construct iterated ordinary least squares (IOLS), an algorithm that estimates power-mean models through repeated OLS regressions. We prove global numerical convergence, consistency and asymptotic normality, and extend the approach to endogenous regressors. Furthermore, we develop fixed-effect cancellation and bias-correction methods that address incidental-parameter bias in common panel settings under general working variances, extending beyond the familiar Poisson-PML case. Finally, simulations document numerical reliability, competitive speed and finite-sample performance, while an application to economic development illustrates their empirical relevance.

Transcript

Introduction to the show: ident: Economics. The day in economics: theory, mechanisms and the evidence beneath them.

Ruth: We're looking at the paper: "Dealing with Logs and Zeros in Regression Models".

Warren: Here's the gist of the paper. The paper introduces Iterated Ordinary Least Squares (iOLS) as a theoretically grounded estimator for the normalized average treatment effect (AT Enorm = E

Y(1) - Y(0): /E

Y(0): ) in binary treatment settings, positioning it as the theoretically justified analogue to the ad-hoc (1 + Y) transformation.

Ruth: iOLS is presented as a consistent and asymptotically normal estimator for the structural parameters of an exponential conditional mean model.

Warren: The core methodological contribution lies in reinterpreting the exponential regression model as a sequence of simple iterative linear regressions, thereby leveraging the full machinery of linear models, such as partialling out fixed effects and instrumental variables.

Ruth: iOLS is characterized by its computational efficiency, global convergence properties, and freedom from incidental-parameter bias.

Title and authors: Ruth: So we're looking at this paper today called "Dealing with Logs and Zeros in Regression Models," and it tackles a really common problem where folks use log transformations because they make coefficients interpretable as percentage changes.

Warren: That interpretation is nice, but the authors point out that the log transformation has three major problems: first, it's undefined when the outcome is zero, which causes an identification issue Iris that we need to be careful about.

Ruth: And second, if you have varying treatment effects or heteroskedasticity in your data, using the standard OLS estimator on a log-linear model will give you a biased estimate of what we're actually interested in Warren which is the average treatment effect percentage.

Iris: And third, they say that this standard approach doesn't actually estimate the structural parameter of an exponential conditional mean model, which is what most researchers are aiming for when they want to understand those underlying economic processes Ruth so what’s the solution they’re proposing?

Warren: They propose something called Iterated Ordinary Least Squares, or iOLS.

Iris: Basically, iOLS is their version of the log transformation that's theoretically justified, targeting the normalized average treatment effect instead of just a simple log addition Ruth which preserves that percentage-change meaning while fixing those three limitations at once.

Iris: They frame it as the theoretically justified analogue to that ad hoc log one plus Y transformation, but they deliver a consistent and asymptotically normal estimator for the structural parameters of an exponential conditional mean model Warren which is pretty strong because it has established convergence properties.

Ruth: What makes this approach methodologically interesting to me is how they handle the complexity of linear models by reinterpreting the exponential regression as a sequence of simple iterative linear regressions Iris which lets them use all that machinery we have, like partialling out fixed effects and instrumental variables Warren.

Warren: And computationally, they say iOLS is efficient and has global convergence properties, meaning it should actually find the right answer no matter where you start Iris which is a big deal compared to methods like Iteratively Reweighted Least Squares which they say are only locally convergent.

Ruth: The paper also addresses the incidental parameter problem by using Frisch-Waugh-Lovell projections at each iteration, which means they avoid that bias even in multiway environments Warren so it should work well even if you have a lot of fixed effects to worry about.

Title and authors: Iris: But they do have limitations too, and I want to talk about those. One thing is that the consistency and asymptotic normality of iOLS are established under standard assumptions, but there's a specific condition for the linear convergence proof Ruth which involves a quantity called eta C being greater than max over g q tau squared g equals kappa F E.

Warren: That condition essentially requires that in groups with many zeros, the residualized regressors still have enough within-group variation on observations with Y equal to zero, which basically rules out all-zero fixed effect groups Iris so you need to check your data structure for that kind of issue.

Ruth: Empirically speaking, the paper looks at how people actually handle zeros in regression models, and they find that adding a positive constant is the most common empirical solution with about forty-three percent of cases Warren though mixture models are also used quite often with thirty-five percent.

Iris: The paper suggests that while empirical estimates of both average treatment effect percentage and normalized average treatment effect can be fairly similar, the normalized one has the advantage because it's estimable on the full sample Ruth even if there's some sensitivity to outliers.

Warren: And they also mention that using iterated two-stage least squares, or i2SLS, extends this naturally to endogenous regressors without making things too messy computationally compared to other nonlinear methods Iris which is a huge practical benefit for real-world data.

Ruth: So when we wrap up "Dealing with Logs and Zeros in Regression Models," the main implication is that if you're working with exponential conditional mean models, iOLS gives you a robust way to estimate the structural parameters while handling zeros and endogeneity more cleanly Warren which is a significant step forward from relying on simpler log transformations.

Iris: I think what’s important here for me is seeing how they extend the validity of GPML to arbitrary fixed effects and endogenous regressors through i2SLS, because that broadens how we can apply these kinds of estimation techniques Ruth even when our data structure isn't perfectly clean.

Warren: For the quantitative side, it’s about computational efficiency too; they say it’s implemented using ordinary least squares with just one matrix inversion which makes it fast and directly comparable to standard log-linear regression Iris but we have to remember that this is still an iterative process, not a single direct calculation.

Title and authors: Ruth: So for the listener driving right now, the big picture is that if you're trying to understand how treatment affects things in a way that respects percentage changes but you have zeros or endogeneity, this paper gives you a more theoretically sound tool than just adding an arbitrary constant Warren which is what they call the ad hoc shift.

Iris: I just want to reiterate that while iOLS is powerful, we still need to check those convergence conditions—that eta C greater than max over g q tau squared g equals kappa F E—because if those aren't met, the convergence might not be as fast as expected Ruth and that’s a necessary caveat when you apply any new estimator.

Warren: Exactly. It’s a tool that works well under standard assumptions, but like any quantitative method, it has its boundaries defined by what the underlying data structure allows it to do reliably Iris which is important for anyone trying to set up a model.

Ruth: So we’ve talked about how this paper addresses the three main problems of log transformations—zeros, bias from heterogeneity, and missing structural parameters—and how iOLS tackles them by using an iterative OLS approach Warren which is a big improvement over just picking some constant to add in manually.

Iris: To sum up my view on the method, it’s powerful because it avoids incidental parameter bias through those projections, and that allows us to estimate things in complex settings without needing specific case-by-case corrections Ruth.

Warren: And for a trader or someone running a large simulation, the i2SLS extension is valuable because it lets you tackle endogeneity in this linear framework without jumping into the mess of nonlinear least squares implementations Iris which keeps the computational burden manageable.

Ruth: So we’re going to wrap up this discussion on "Dealing with Logs and Zeros in Regression Models," emphasizing that iOLS gives us a way to estimate normalized average treatment effect while respecting the underlying structure of exponential conditional mean models Warren.

Iris: And I just think the empirical evidence, showing that adding a positive constant is the most frequent fix, grounds this theory in something practical for real-world researchers Ruth even if we have to be mindful of those zero groups.

Title and authors: Warren: That’s right. It’s a consistent and asymptotically normal estimator, achieving that square root of n convergence rate under standard assumptions, which gives us confidence in its performance Iris but always keep an eye on that critical condition for the fixed effect groups.

Ruth: So moving forward, the implication is that when you model these kinds of relationships, instead of just slapping a log onto your outcome variable and hoping for the best, you have a theoretically grounded procedure like iOLS to target what you actually want to measure Warren.

Iris: It gives us a solid framework for choosing our target parameter—the normalized average treatment effect—and reporting it alongside the percentage change one, which should be useful when treatment effects are varying Ruth or when we’re modeling those structural parameters.

Warren: And i2SLS is definitely worth looking into if you have endogenous variables and want to keep the computational structure relatively simple compared to more complex nonlinear estimation tools Iris because it extends the validity of methods like GPML in these ways.

Ruth: That’s our look at "Dealing with Logs and Zeros in Regression Models." it’s a lot of machinery, but the core idea is using iteration to solve a problem that simple log transformations can't handle on their own Warren.

Iris: I just want to say that the paper provides a clear path forward for estimating these parameters consistently without running into those common pitfalls associated with zero values or heteroskedastic errors Ruth as long as you respect the conditions they lay out.

Warren: Agreed. It’s about being systematic about how we approach these estimation problems, rather than just relying on transformations that work most of the time but fail when things get tricky Iris which is a key difference in a quantitative setting.

Ruth: We’ll keep an eye on how this iOLS method is used in future papers to see if it becomes the standard way to handle these specific structural models Warren because I think that’s where the real impact will be felt.

Iris: And we’ll definitely watch for more work extending i2SLS into even more complex settings, like those high-dimensional fixed effects problems they mentioned Ruth which are always a challenge in econometrics.

Warren: It’s good to see this kind of iterative linear approach being applied to these kinds of conditional mean models, because it keeps the estimation framework grounded in the familiar world of linear regression Iris even when we’re dealing with exponential structures.

The paper's summary: Ruth: So, to wrap up on "Dealing with Logs and Zeros in Regression Models," the core idea is that Iterated Ordinary Least Squares, or iOLS, gives us a way to estimate what we actually want—the normalized average treatment effect—while respecting the structure of exponential conditional mean models.

Warren: Right. It’s not just slapping a log onto something; it’s a systematic way to solve the problem of zeros and heteroskedastic errors that standard log-linear methods miss.

Iris: And what I found particularly interesting about the mechanism is how iOLS handles those incidental parameters by using projections at every step, so you don't get that bias you see in some other multiway fixed effects setups.

Ruth: Exactly. So, it’s consistent and asymptotically normal under standard assumptions, meaning if your data looks right, this estimator gives you a reliable number at the square root of n convergence rate.

Warren: And the i2SLS extension is what makes it practical for real-world problems with endogenous variables; it keeps the math linear in each iteration, which is way better than jumping into those messy nonlinear least squares implementations.

Iris: But we can't ignore that condition they lay out for convergence; there’s a specific rule about groups with zeros needing enough variation on observations where the outcome is positive to make sure the process actually converges fast.

Ruth: That’s a necessary caveat, I guess. So, if you aren't careful with your data structure—especially having lots of zero groups—the speed at which iOLS gets its answer might slow down a bit.

Warren: So for someone running these kinds of models, the big picture is that instead of picking an arbitrary constant to add, you have a theoretically grounded procedure like iOLS to target the normalized average treatment effect while keeping the structure intact.

Iris: It gives us a solid framework for choosing our target parameter—the normalized average treatment effect—and reporting it alongside the simple percentage change one when treatment effects are actually different across groups.

Ruth: And that sounds useful when you’re trying to understand those underlying structural parameters of an exponential conditional mean model, which is where most of this work actually goes.

Warren: So, it moves us away from relying on those simpler log transformations that just don't handle the zeros or the varying treatment effects well.

Iris: And we still need to respect those convergence conditions, though; if you ignore them and your data doesn't meet them, the method might not deliver its promised speed.

Ruth: Exactly. So moving forward, instead of just slapping a log onto your outcome variable and hoping for the best, you have a procedure like iOLS to target what you actually want to measure.

Warren: And if you have endogenous variables and want to keep the computational structure relatively simple compared to more complex nonlinear estimation tools, i2SLS is definitely worth looking into.

Iris: I just think the empirical evidence, showing that adding a positive constant is the most frequent fix, grounds this theory in something practical for real-world researchers, even if we have to be mindful of those zero groups.

The paper's improvements: Iris: We've talked about how iOLS tackles the zeros and heterogeneity problem by using an iterative OLS approach, but now we need to look at what improvements the authors suggest for this method itself.

Ruth: So, they are proposing a few things to make this estimator even better than it is right now. They’re saying that instead of just sticking with iOLS, we should consider more specialized algorithms depending on the exact data setup.

Warren: I think they're pointing toward using the iOLS-FD-HDFE algorithm when you’re dealing with those multiway fixed effects scenarios, specifically to handle large and small fixed effect dimensions without that incidental parameter bias kicking in.

Iris: That makes sense because the standard approach can get messy when some of those fixed effect dimensions are short, so this new hybrid approach seems like it keeps the estimation clean across different levels.

Ruth: It’s about being more flexible with the algorithm itself, moving beyond just a single iterative process to something that adapts better to complex environments.

Warren: And they also emphasize how using iterated two-stage least squares, or i2SLS, can be really useful because it extends this linear framework naturally to endogenous regressors without forcing you into those complicated nonlinear least squares implementations.

Iris: That extension is pretty powerful because it means the consistency and asymptotic normality we established for iOLS still hold even when you have those tricky endogenous variables in your dataset.

Ruth: So, if you're running a model with endogeneity, i2SLS seems like a solid path forward because it keeps the computation manageable while still giving you reliable estimates.

Warren: It’s about computational efficiency too; they say the implementation uses ordinary least squares with just one matrix inversion, which keeps it fast and lets you compare it directly to standard log-linear regression, even though it’s still iterative.

Iris: But we can't forget the limitations again; the authors are very clear that this entire framework relies on those specific convergence conditions being met for the process to work reliably.

Ruth: Exactly. So, while iOLS is a powerful tool for estimating normalized average treatment effect under ideal settings, you still need to check if your data structure allows it to converge quickly enough.

Warren: It’s about being systematic about how you approach these estimation problems rather than just relying on transformations that work most of the time but fail when things get tricky.

Conclusion: Ruth: So, to wrap up on "Dealing with Logs and Zeros in Regression Models," we’ve seen how iOLS provides a theoretically grounded way to estimate the normalized average treatment effect while handling zeros and structural parameters in exponential conditional mean models.

Warren: It’s a systematic approach that replaces those arbitrary shifts with an iterative OLS adjustment, which is much more robust than just picking some constant to add manually.

Iris: I think what’s important is how it manages the incidental parameters; by using projections at every step, it avoids that bias even in complex multiway environments.

Ruth: So, the paper shows a way to get a consistent and asymptotically normal estimator for structural parameters without running into the usual pitfalls of zero values or heteroskedastic errors.

Warren: And for anyone dealing with endogenous variables, they’re suggesting i2SLS as a computationally simple way to extend this method, keeping the math linear in each iteration.

Iris: That extension is significant because it maintains the validity of methods like GPML even when you introduce those tricky endogenous regressors into your system.

Ruth: It gives us a solid framework for choosing our target parameter—the normalized average treatment effect—and reporting it alongside the simple percentage change one when treatment effects vary across groups.

Warren: And that’s a practical benefit because it moves us away from relying on those simpler log transformations that just don't handle zeros or varying treatment effects well.

Iris: I just think the convergence conditions they lay out are crucial, though; if you ignore those requirements for groups with zeros, the method might not deliver its promised speed.

Ruth: That’s a necessary caveat, I guess. So moving forward, instead of just slapping a log onto your outcome variable and hoping for the best, you have a procedure like iOLS to target what you actually want to measure.

Warren: And if you have endogenous variables and want to keep the computational structure relatively simple compared to more complex nonlinear estimation tools, i2SLS is definitely worth looking into.

Iris: I just think the empirical evidence showing that adding a positive constant is the most frequent fix grounds this theory in something practical for real-world researchers, even if we have to be mindful of those zero groups.

Ruth: It’s about being systematic about how you approach these estimation problems rather than just relying on transformations that work most of the time but fail when things get tricky.

Warren: So, we’ve seen how this paper addresses the three main problems of log transformations—zeros, bias from heterogeneity, and missing structural parameters—and how iOLS tackles them by using an iterative OLS approach.

Iris: And it gives us a clear path forward for estimating these parameters consistently without running into those common pitfalls associated with zero values or heteroskedastic errors as long as you respect the conditions they lay out.

Ruth: Exactly. So, we’ve talked about how this paper addresses the three main problems of log transformations—zeros, bias from heterogeneity, and missing structural parameters—and how iOLS tackles them by using an iterative OLS approach.

Warren: It’s a consistent and asymptotically normal estimator achieving that square root of n convergence rate under standard assumptions, which gives us confidence in its performance.

Iris: The core idea is using iteration to solve a problem that simple log transformations can't handle on their own, which is pretty neat for modeling those structural parameters.

Ruth: So the implication is that when you model these kinds of relationships, instead of just slapping a log onto your outcome variable and hoping for the best, you have a theoretically grounded procedure like iOLS to target what you actually want to measure.

Warren: And i2SLS is definitely worth looking into if you have endogenous variables and want to keep the computational structure relatively simple compared to more complex nonlinear estimation tools.

Iris: And we’ll definitely watch for more work extending i2SLS into even more complex settings, like those high-dimensional fixed effects problems they mentioned.

Ruth: That’s our look at "Dealing with Logs and Zeros in Regression Models," emphasizing that iOLS gives us a way to estimate normalized average treatment effect while respecting the underlying structure of exponential conditional mean models.

More episodes

← Home