Adapting a Foundation Model for Lunar Surface Height Estimation
summary
The gist
Digital elevation models (DEMs) can provide accurate height information, making it invaluable for analyzing the lunar surface <ref:2609.02448#pg2> As the European Space Agency (ESA) prepares for…
In short
The researchers fine-tuned a foundation model called Depth Anything V2 (DAV2) using lunar surface elevation data to create a more accurate relative height estimator. By injecting new parameters via LoRA and using a hybrid loss function, they significantly improved performance over the original zero-shot model, achieving lower mean MAE and RMSE for lunar terrain analysis.
Key concepts
- Depth Anything V2 (DAV2)
- This is a pre-trained foundation model used for relative depth estimation. The researchers adapted this existing model to handle the specific challenges of estimating heights on the lunar surface, which requires specialized training data.
- LoRA
- Low-Rank Adaptation is a parameter-efficient fine-tuning technique. Instead of retraining the entire large model, LoRA injects small, trainable matrices into specific parts of the Transformer encoder. This allows for significant performance gains with far fewer computational resources and parameters.
- Hybrid Loss Function (Ltotal)
- This combined loss function guides the training process using three distinct goals: LBerhu focuses on depth accuracy, Lgm ensures the residual gradients match across different scales, and Lnorm verifies the accuracy of surface normals. This multi-faceted approach leads to a more robust and accurate final estimation.
Terminology used across episodes
This episode discusses
- Adapting a Foundation Model for Lunar Surface Height Estimation · Paper Radio
- Deep Learning based Systems for Crater Detection: A Review
- Depth Anything V2
- MCTED: A Machine-Learning-Ready Dataset for Digital Elevation Model Generation From Mars Imagery
- DINOv2: Learning Robust Visual Features without Supervision
- LoRA: Low-Rank Adaptation of Large Language Models
The paper
Adapting a Foundation Model for Lunar Surface Height Estimation · Read on arXiv
University of Technology of Troyes · European Space Agency · Hochschule Darmstadt · Technische Hochschule Würzburg-Schweinfurt
DOI: 10.1109/LGRS.2026.3754087
Transcript
Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.
Tom: We're looking at the paper: "Adapting a Foundation Model for Lunar Surface Height Estimation".
Title and authors: Tom: So we're talking about this paper today, "Adapting a Foundation Model for Lunar Surface Height Estimation." It’s about using these huge models, like Depth Anything V2 or DAV2, and making them work better for estimating how high things are on the Moon.
Jane: Right. The main idea here is that digital elevation models, or DEMs—those are maps of height—are super important for understanding the lunar surface because they give you that precise height information <ref:2609.02448#pg1>.
Tom: Exactly. And as the European Space Agency plans future landings, they need a really precise way to estimate heights for tricky terrain where landing approaches could be dangerous <ref:2609.02448#pg1>. So what this paper is doing is trying to get that estimation method much better than just using the base model on its own.
Lu: Basically, they're taking a known deep learning model, DAV2, and fine-tuning it specifically with data from Shape from Shading or SPG—those are traditional methods for getting DEMs from images <ref:2609.02448#pg2>. They’re trying to bridge that gap between the general knowledge the AI has and the specific look of lunar terrain <ref:2609.02448#pg1>.
Meng: Okay, so before we get into how they tuned it, what's the actual problem they are trying to solve? Is it just making a prediction slightly better, or is there a bigger issue with how these models handle this kind of data?
Jane: Well, the paper points out that just applying foundation models directly to remote sensing tasks often doesn't give good results because there’s this big gap between what those models were trained on and what they have to do now <ref:2609.02448#pg2>. They argue that just comparing the zero-shot model against other methods isn't fair because you haven't adapted it to this specific lunar domain yet <ref:2609.02448#pg1>.
Tom: That makes sense. So the core of their work is this fine-tuning strategy they introduce, and they use a technique called LoRA, which stands for low-rank adaptation <ref:2609.02448#pg3>.
Lu: They inject new learnable parameters into the Transformer encoder using LoRA with a rank of eight, so that's the specific setting they used <ref:2609.02448#pg3>. They also selectively unfrozen parts of the decoder network—specifically three out of four RefineNet blocks and the output head—to keep things balanced parameter-wise <ref:2609.02448#pg3>.
Meng: eight million parameters for that adaptation sounds like a reasonable trade-off, but what's guiding this tuning process? What loss functions are they using to make sure it actually gets better?
Jane: They use a hybrid loss function called Ltotal, which is made up of three parts: alpha times the Berhu loss, beta times the gm loss, and gamma times the norm loss <ref:2609.02448#pg2>. These losses are designed to guide the fine-tuning process in different ways.
Title and authors: Tom: Let's break down those losses for our listeners. The first one, LBerhu, is defined as this ratio between a prediction and the ground truth that includes some terms involving tau and r squared <ref:2609.02448#pg2>. It’s focused on getting the basic depth accuracy right across different scales.
Lu: That Berhu loss seems to be about matching the depth predictions against the actual ground truth, incorporating a term that depends on tau and r squared for every patch <ref:2609.02448#pg2>. It’s a way to penalize errors based on that structure.
Meng: Then there's Lgm, which deals with the residual gradient of the prediction relative to the ground truth, and they are incorporating downscale operations at four different scales—K equals four <ref:2609.02448#pg2>. That sounds like they’re trying to enforce consistency across those different resolution levels.
Jane: And finally, Lnorm measures the accuracy of the surface normal by calculating one minus the norm of both the predicted and ground truth surface normals <ref:2609.02448#pg2>. So they're not just looking at height, but also making sure the shape of that lunar surface is correct too.
Tom: So to summarize, they’re using this three-part hybrid loss function to guide the fine-tuning via LoRA on the ViT-L variant of DAV2 <ref:2609.02448#pg3>. The goal is to create a more reliable relative height estimator <ref:2609.02448#pg1>.
Lu: And they claim this approach significantly improves performance compared to the zero-shot model, effectively transforming DAV2 into a reliable relative depth estimator of the lunar surface <ref:2609.02448#pg1>.
Meng: Let's look at the numbers for a minute. What does this mean in practical terms for someone who has to deploy this on a real system? Is it fast enough or stable enough?
Jane: The quantitative results on the test set show that DEM-DAV2 achieves higher metrics compared to DAV2 <ref:2609.02448#pg1>. Specifically, Table one shows that DEM-DAV2 has a mean mean absolute error of four point seven one meters and a root mean square error of five point seven six meters <ref:2609.02448#pg1>.
Tom: That’s a big jump from the zero-shot DAV2, which had a mean absolute error of eleven point six four meters and an root mean square error of fourteen point one three meters <ref:2609.02448#pg1>. That’s a substantial reduction in error, especially for hazardous terrain analysis.
Lu: This improvement confirms what they were seeing qualitatively, showing that fine-tuning foundation models for remote sensing tasks is necessary because of that domain gap <ref:2609.02448#pg1>. It validates the idea of injecting specific domain knowledge into a general model <ref:2609.02448#pg3>.
Meng: So, if I’m running this in production, the implication is that we can trust the relative height estimates more for hazard localization because the error bars are much smaller now <ref:2609.02448#pg1>. But what about deployment challenges? Can this run on a standard CPU without massive computational overhead?
Jane: The authors note some limitations they have to admit, though. One limitation is that the model can't be used for real-time DEM estimation on a CPU <ref:2609.02448#pg1>. Also, they don't have auxiliary global DEMs with absolute height information available for this work <ref:2609.02448#pg1>.
Title and authors: Tom: So the system is great for high-precision relative estimation on a GPU setup, but we can’t rely on it for super low-latency CPU deployment right now <ref:2609.02448#pg1>. And that lack of absolute height data means we still need something else to anchor those heights to the real world <ref:2609.02448#pg1>.
Lu: Looking ahead, they plan to incorporate this DEM-DAV2 as an auxiliary model for crater detection <ref:2609.02448#pg1>. That suggests they see it not just as a height estimator but also a useful tool for other lunar mapping tasks <ref:2609.02448#pg1>.
Meng: That makes sense; using the height data to detect features like craters is a good way to test its utility beyond just measuring elevation <ref:2609.02448#pg1>. It shows the model has generalized well enough for other specific applications.
Jane: So, wrapping up on this paper, they’ve successfully taken DAV2 and fine-tuned it using SPG data and a hybrid loss function to create a reliable relative height estimator <ref:2609.02448#pg1>. This means we have a much more accurate tool for hazard localization on the Moon than we had before <ref:2609.02448#pg1>.
Tom: Exactly. The improvement from an error of eleven point six four meters down to four point seven one meters is pretty substantial when you're talking about precise landing approaches <ref:2609.02448#pg1>. It’s a clear demonstration of how domain adaptation works in practice <ref:2609.02448#pg1>.
Lu: The methodology, using LoRA and those three specific loss functions—Berhu, gm, and norm—is the mechanism that drives this performance gain <ref:2609.02448#pg3>. It’s a concrete way to inject the necessary lunar surface knowledge into the transformer architecture <ref:2609.02448#pg3>.
Meng: For deployment, what does this mean for our pipeline? Does this suggest we should start fine-tuning other foundation models using similar domain-specific loss functions rather than just relying on zero-shot transfers?
Jane: It suggests that for tasks with a huge domain gap, like moving from general vision to lunar topography, you really need that targeted fine-tuning approach <ref:2609.02448#pg1>. It proves that simply feeding the model more data isn't enough; you have to guide it with specific loss functions <ref:2609.02448#pg2>.
Tom: Yeah, it’s about showing that the mechanics of how you train a model matter just as much as the size of the pre-trained weights <ref:2609.02448#pg3>. The concrete numbers on MAE and RMSE are what really sell this result to anyone in engineering <ref:2609.02448#pg1>.
Lu: And for my research, it confirms that the transformer architecture, when modified with LoRA, is very effective at learning these specific surface relationships from limited domain data <ref:2609.02448#pg3>. It shows that small parameter adjustments can yield big results in specialized scenarios <ref:2609.02448#pg1>.
Meng: So, for me, the main thing is that this gives us a baseline for how much performance we can expect when we do this kind of specific fine-tuning on our own models instead of just hoping the general pre-training carries us through <ref:2609.02448#pg1>.
Title and authors: Jane: Ultimately, the paper, "Adapting a Foundation Model for Lunar Surface Height Estimation," shows that with targeted fine-tuning, we can make foundation models capable of providing precise height information for hazardous lunar terrain <ref:2609.02448#pg1>. It’s a practical step toward making autonomous lunar missions safer <ref:2609.02448#pg1>.
Tom: It is a solid result, showing that we can take something like DAV2 and make it perform much better on the Moon than it could out of the box <ref:2609.02448#pg1>. The methodology, especially those three loss functions, gives us a blueprint for how to handle these domain shifts in remote sensing AI <ref:2609.02448#pg3>.
Lu: We need to keep looking at how they plan to use this DEM-DAV2 as an auxiliary model for crater detection because that opens up a whole new area of research <ref:2609.02448#pg1>. It connects height estimation directly to feature extraction for other tasks <ref:2609.02448#pg1>.
Meng: And from a production standpoint, knowing this level of accuracy for relative depth estimation is key because it allows us to build better hazard detection algorithms <ref:2609.02448#pg1>. We need these reliable metrics when we are deploying actual systems <ref:2609.02448#pg1>.
Jane: So, to wrap up on the paper "Adapting a Foundation Model for Lunar Surface Height Estimation," they’ve shown that fine-tuning a model like DAV2 with SPG data and this hybrid loss function yields significantly lower error metrics for lunar surface height estimation <ref:2609.02448#pg1>. This gives us a much more reliable tool for hazard localization on the Moon <ref:2609.02448#pg1>.
Tom: It’s a strong performance boost, and it shows that domain adaptation is a necessary technique when you’re dealing with such distinct remote sensing environments <ref:2609.02448#pg3>. The work provides concrete numbers, like the mean absolute error of four point seven one meters compared to eleven point six four meters <ref:2609.02448#pg1>.
Lu: This research suggests that modifying existing transformer architectures with low-rank adaptation and carefully constructed hybrid loss functions is a viable path for making foundation models useful in specialized areas like lunar surface mapping <ref:2609.02448#pg3>. It’s about showing how to guide the model effectively when the data distribution is very different from its training set <ref:2609.02448#pg1>.
Meng: We need to keep testing if this approach scales well when we move beyond just these specific DEM datasets and try to apply it across other planetary surfaces <ref:2609.02448#pg1>. The lack of absolute height data is the main caveat for real-world deployment right now <ref:2609.02448#pg1>.
Jane: So, that’s what we have on this paper, "Adapting a Foundation Model for Lunar Surface Height Estimation." It gives us a much more reliable relative depth estimator for lunar surfaces through targeted fine-tuning and hybrid loss functions <ref:2609.02448#pg1>. We’ll be looking at how they use it for crater detection next time <ref:2609.02448#pg1>.
The paper's summary: Tom: So, we're looking at this paper "Adapting a Foundation Model for Lunar Surface Height Estimation," and basically, they’ve taken a big model called Depth Anything V2 and fine-tuned it using data from Shape from Shading to make it a much more reliable height estimator for the Moon.
Jane: Right. They’re trying to solve that problem where general AI models don't work well on specific remote sensing tasks because of that big gap between what they were trained on and what they need to do now.
Lu: The core idea is domain adaptation, using SPG derived Digital Elevation Model data to teach the model how lunar terrain looks in terms of height <ref:2609.02448#pg2>.
Meng: And the authors are using a hybrid loss function called Ltotal, which mixes three different types of errors—Berhu loss, gm loss, and norm loss—to guide this training process <ref:2609.02448#pg2>.
Tom: Yeah, they're not just relying on one metric; they’re looking at the height accuracy from the Berhu loss, checking the residual gradients with the gm loss, and even verifying surface normals with the norm loss <ref:2609.02448#pg2>.
Jane: The results are pretty compelling. They found that this DEM-DAV2 model is significantly better than just using the zero-shot DAV2 model on a test set <ref:2609.02448#pg1>.
Lu: Specifically, they report a mean absolute error of four point seven one meters and a root mean square error of five point seven six meters for DEM-DAV2, compared to eleven point six four meters and fourteen point one three meters for the zero-shot DAV2 <ref:2609.02448#pg1>.
Meng: That’s a big reduction in error, especially when you think about landing approaches where precision is everything <ref:2609.02448#pg1>. But there are caveats on the paper, right?
Jane: They do mention two main things they can't do. First, this method isn't ready for real-time DEM estimation on a standard CPU <ref:2609.02448#pg1>. And second, they don't have any global Digital Elevation Models with absolute height information available for this work <ref:2609.02448#pg1>.
Tom: So what that means for us in the field is that we can build a much more reliable relative height estimator on a powerful GPU setup <ref:2609.02448#pg1>. It’s great for hazard localization where you need to know the height difference between two points on the surface <ref:2609.02448#pg1>.
Lu: It proves that modifying existing transformer architectures with low-rank adaptation and carefully constructed hybrid loss functions is a viable way to inject lunar domain knowledge into general foundation models <ref:2609.02448#pg3>.
Meng: I mean, that confirms the idea that targeted fine-tuning, rather than just relying on zero-shot transfers, is necessary when you're dealing with such distinct remote sensing environments <ref:2609.02448#pg1>.
Jane: So, it gives us a blueprint for how to guide models effectively when the data distribution is very different from what they were originally trained on <ref:2609.02448#pg1>.
Tom: Exactly. The concrete numbers on the error reduction show that this isn't just theoretical stuff; it actually improves performance substantially for a real application like lunar surface hazard mapping <ref:2609.02448#pg1>.
Lu: And they’re already looking ahead, planning to use this DEM-DAV2 model as an auxiliary tool for crater detection, which suggests they see it as more than just a height estimator <ref:2609.02448#pg1>.
Meng: Using the height data to detect features like craters is useful because it tests the model’s generalization across different types of surface geometry <ref:2609.02448#pg1>.
Jane: So, this paper sets a strong baseline for how we can make these foundation models useful in specialized areas like lunar surface mapping <ref:2609.02448#pg3>.
Tom: We'll keep an eye on how they use it for crater detection because that could open up a whole new area of research <ref:2609.02448#pg1>.
The paper's improvements: Tom: So, we’re talking about how this paper suggests improving the system they built for lunar height estimation <ref:2609.02448#pg1>. They aren't just stopping at getting a better prediction; they’re proposing specific ways to make the model more robust.
Jane: Right. The improvements focus on how they construct that training signal, using three specific loss functions instead of just one standard error <ref:2609.02448#pg2>.
Lu: They are suggesting this hybrid loss function—the Berhu loss, the gm loss, and the norm loss—is key because it ensures accuracy in depth, consistency across different scales with that four scales K equals four for the gm loss, and correct surface normals with that norm part <ref:2609.02448#pg2>.
Meng: So what does that actually mean for deployment? It means they are building a training signal that checks depth accuracy, residual gradient deviation across those scales, and the actual shape of the surface <ref:2609.02448#pg1>.
Tom: I like that part about checking the residual gradient deviation; it sounds like they’re forcing the model to be consistent not just in one spot, but across different levels of detail <ref:2609.02448#pg3>. That’s a solid mechanic for better generalization.
Jane: And that consistency is important because if the model gets confused at one scale, it can lead to big errors when you try to estimate something bigger or smaller <ref:2609.02448#pg1>. It’s about making sure the prediction makes sense everywhere on the map.
Lu: Exactly. The way they’re defining that loss function is what changes things; it forces the model to learn better representations for those specific geometric relationships on a planetary surface <ref:2609.02448#pg3>.
Meng: From a production standpoint, if we could distill that into a smaller set of constraints, maybe we could speed up inference without losing that accuracy <ref:2609.02448#pg1>. What’s the cost of running all three loss calculations at once?
Jane: Well, they are using LoRA with rank eight to inject the new parameters into the Transformer encoder, which helps keep the parameter count manageable for adaptation <ref:2609.02448#pg3>. That keeps things efficient during training <ref:2609.02448#pg1>.
Tom: So they’re balancing high accuracy with parameter efficiency by only tweaking those specific layers of the model, which is smart engineering <ref:2609.02448#pg3>.
Lu: It shows that you don't always need to retrain the entire giant foundation model; sometimes small, targeted adjustments via techniques like LoRA are enough to unlock better performance for a specific task <ref:2609.02448#pg1>.
Meng: That’s what I want to hear. It means we don't need massive compute resources just to adapt these models for specific planetary conditions; we can focus our training efforts on those crucial loss terms <ref:2609.02448#pg3>.
Jane: It really changes how you think about adapting general AI for specialized science applications, showing that the method matters as much as the size of the model itself <ref:2609.02448#pg1>.
Tom: So, moving forward, we should look at how this hybrid loss function approach works when we try to apply it to other planetary surfaces like Mars or even Earth topography <ref:2609.02448#pg1>.
Conclusion: Tom: So, to wrap up on "Adapting a Foundation Model for Lunar Surface Height Estimation," this paper shows that by fine-tuning models like DAV2 with specific data and three different loss functions, we can get a much more reliable relative height estimator for lunar surfaces <ref:2609.02448#pg1>.
Jane: It’s a solid result, proving that targeted fine-tuning is necessary when you're dealing with such distinct remote sensing environments <ref:2609.02448#pg1>.
Lu: The methodology, especially using LoRA to adjust the encoder and that hybrid loss function for depth, consistency across scales, and surface normals, gives us a blueprint for how to guide models effectively <ref:2609.02448#pg3>.
Meng: I mean, it shows that small parameter adjustments can yield big results in specialized scenarios like this lunar mapping task <ref:2609.02448#pg1>.
Tom: Exactly, and those concrete numbers on error reduction, from eleven point six four meters down to four point seven one meters, really sell the idea that this is a practical improvement for hazard localization <ref:2609.02448#pg1>.
Jane: It’s a practical step toward making autonomous lunar missions safer by giving us a much more reliable tool for relative depth estimation <ref:2609.02448#pg1>.
Lu: We need to keep looking at how they plan to use this DEM-DAV2 model as an auxiliary tool for crater detection because that opens up a whole new area of research <ref:2609.02448#pg1>.
Meng: Using the height data to detect features like craters is useful because it tests the model’s generalization across different types of surface geometry <ref:2609.02448#pg1>.
Tom: So, this paper demonstrates that you can take something like DAV2 and make it perform much better on the Moon than it could out of the box <ref:2609.02448#pg1>.
Jane: It’s a clear demonstration of how domain adaptation works in practice <ref:2609.02448#pg1>.
Lu: The research confirms that modifying existing transformer architectures with low-rank adaptation and carefully constructed hybrid loss functions is a viable path for making foundation models useful in specialized areas like lunar surface mapping <ref:2609.02448#pg3>.
Meng: We’ll need to keep testing if this approach scales well when we move beyond just these specific DEM datasets and try to apply it across other planetary surfaces <ref:2609.02448#pg1>.
Tom: The concrete numbers on MAE and RMSE are what really show the value here for anyone in engineering <ref:2609.02448#pg1>.
Jane: It’s a strong performance boost, and it shows that domain adaptation is a necessary technique when you’re dealing with such distinct remote sensing environments <ref:2609.02448#pg3>.
Lu: We should also keep watching their next steps regarding the auxiliary model for crater detection because that connects height estimation directly to feature extraction for other tasks <ref:2609.02448#pg1>.
Meng: And from a production standpoint, knowing this level of accuracy for relative depth estimation is key because it allows us to build better hazard detection algorithms <ref:2609.02448#pg1>.
Tom: So, the paper "Adapting a Foundation Model for Lunar Surface Height Estimation" provides a blueprint for how to handle these domain shifts in remote sensing AI <ref:2609.02448#pg3>.
More episodes
- 2603.29765-Training-Free Dynamic Upcycling of Expert Language Models
- 2605.00778-Observable Performance Does Not Fully Reflect Adaptive System Organization: A Multi-Level Analysis of Gait Dynamics Under Occlusal Constraint
- 2605.06906-TraXion: Rethinking Pre-training Frameworks for Mobility and Beyond
- 2605.09165-Sparse Layers are Critical to Scaling Looped Language Models
- 2605.09439-Inverse Design for Conditional Distribution Matching
- 2605.10313-Signature Approach for Contextual Bandits with Nonlinear and Path-dependent Rewards: Regret Bound and Statistical Inference
- 2605.10878-Minimum Neural Weight Norm = Kolmogorov Complexity
- 2605.11617-MIST: Reliable Streaming Decision Trees for Online Class-Incremental Learning via McDiarmid Bound
- 2605.15085-From Data to Action: Accelerating Refinery Optimization with AI
- 2605.23500-GRTO: Bootstrapped Group Relative Tool Optimization for Referring Segmentation