OJAL'A: Optimizing J-PAS Astronomy for Large-scale Analysis. A transformer-based model for source classification and physical-parameter inference
summary
The gist
OJALA is a transformer-based autoregressive foundation model designed to simultaneously classify astronomical objects and infer their physical parameters using 54-band photometry from the Javalambre…
In short
OJALA is a transformer model designed to classify astronomical objects and infer their physical properties using 54-band photometry from J-PAS and other surveys. It uses a Transformer architecture trained on synthetic data, employing domain adaptation techniques to bridge the gap between simulated and real observations. The model achieves high accuracy in spectral classification, photometric redshift estimation, and recovering stellar masses.
Key concepts
- Transformer Encoder–Decoder Architecture
- This is the core structure of the AI model. It works like a sophisticated language processor that takes an input (like a set of astronomical data) and predicts missing or related information. The self-attention mechanism allows the model to weigh different parts of the input data simultaneously, enabling it to understand complex relationships between various photometric bands.
- Unsupervised Domain Adaptation (UDA)
- This technique is used to make the model perform well on real astronomical data when it was primarily trained on simulated data. The model learns a representation of the features from synthetic J-spectra and then uses a penalty (Maximum Mean Discrepancy) to align these learned features with the distribution found in real, unlabeled J-PAS observations.
- Multi-task Loss Function
- Instead of training the model on one single goal, OJALA minimizes a combined loss function that includes three separate tasks: spectral classification, photometric redshift estimation, and inferring physical properties like stellar mass. This forces the model to learn a comprehensive understanding of the data simultaneously.
- Weighted F1-score
- This is a metric used to evaluate how well the model classifies objects into categories like stars, galaxies, or QSOs. A weighted F1-score of around 0.9 means that when classifying these objects, the model is highly accurate and reliable across different types of astronomical sources.
Terminology used across episodes
This episode discusses
- OJAL'A: Optimizing J-PAS Astronomy for Large-scale Analysis. A transformer-based model for source classification and physical-parameter inference · Paper Radio
- J-PAS: The Javalambre-Physics of the Accelerated Universe Astrophysical Survey
- Deep Generative Modelling: A Comparative Review of VAEs, GANs, Normalizing Flows, Energy-Based and Autoregressive Models
- FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness
- Data Release 1 of the Dark Energy Spectroscopic Instrument
- Euclid. I. Overview of the Euclid mission
- J-PAS: A value-added catalogue of optical line intensities for nebular emission galaxies (JOLINES)
- Training Compute-Optimal Large Language Models
- The miniJPAS and J-NEP surveys: Machine learning for star-galaxy separation
- DESI Data Release 1: Stellar Catalogue · Paper Radio
- The DESI Experiment, a whitepaper for Snowmass 2013
- J-PAS: Semi-Supervised Sim-to-Obs Transfer for Robust Star--Galaxy--Quasar Classification
- AION-1: Omnimodal Foundation Model for Astronomical Sciences
- The miniJPAS survey: Dissecting galaxy properties across environments with spatially resolved photometry
- AstroPT: Scaling Large Observation Models for Astronomy
- Scaling Laws for Galaxy Images
The paper
OJAL'A: Optimizing J-PAS Astronomy for Large-scale Analysis. A transformer-based model for source classification and physical-parameter inference · Read on arXiv
G. Martínez-Solaeche, R. M. González Delgado, R. García-Benito, A. Hernán-Caballero, I. Pérez-Ráfols, L. A. Díaz-García, L Raul Abramo, J. E Rodríguez-Martín, A. M Conrado, I Breda, H Domínguez SanchḜhe, I Márquez, M Pieri, D López-Cano, V M Placco, L Nakazono, A del Pino, V Marra, J Alcaniz, N Benitez, S Bonoli, S Carneiro, A J Cenarro, D Cristóbal-Hornillos, S Daflon R Dupke A Ederoclite C Hernández-Monteagudo J Liu C López-Sanjuan A Marín-Franch C Mendes de Oliveira M Moles F Roig L Sodré K Taylor J Varela H Vázquez Ramiú J M Víchez, and J Zaragoza-Cardiel
Instituto de Astrofísia de Andalucia · Centro de Estudios de F귬 del Cosmos de Arag᪭a · Departamento de F귬 Fisica, Universitat Politècnica de Catalunya · Departamento de F귬 Fisica Matematica, Instituto de F귬 Física, Universidade do Estado da Bahia · Department of Astrophysics, University of Vienna · Instituto de F귬 Física de Cantabria · Aix Marseille Univ. CNRS. CNES. LAM
Transcript
Introduction to the show: ident: Astrophysics Radio. Generated commentary on the latest astrophysics papers.
Vera: I'm Vera.
Jocelyn: And I'm Jocelyn, here with Subrahmanyan.
Vera: We're looking at the paper: "OJAL'A: Optimizing J-PAS Astronomy for Large-scale Analysis. A transformer-based model for source classification and physical-parameter inference".
Jocelyn: Here's the gist of the paper. OJALA is a transformer-based autoregressive foundation model designed to simultaneously classify astronomical objects and infer their physical parameters using 54-band photometry from the Javalambre Physics of the Accelerating Universe Astrophysical Survey (J-PAS), combined with BB photometry from the DESI Legacy Imaging Surveys and WISE.
Vera: The model achieves a weighted F1-score of ∼ 0.9 for spectral classification (stars, galaxies, and QSOs) at i < 21.
Jocelyn: Model Architecture and Training Strategy The OJALA model builds upon the Transformer-based encoder–decoder architecture introduced by Leung & Bovy (2024).
Vera: It operates analogously to Large Language Models by predicting missing observations or physical properties of an object given the remaining available observations.
Paper summary: Vera: So, we're looking at this paper called "OJAL'A: Optimizing J-PAS Astronomy for Large-scale Analysis. A transformer-based model for source classification and physical-parameter inference." It basically introduces a foundation model, a transformer based one, designed to classify astronomical objects like stars, galaxies, and QSOs while simultaneously figuring out their physical parameters using photometric data.
Jocelyn: That sounds like it's trying to tackle the problem of how we can make sense of all that Javalambre Physics of the Accelerating Universe Astrophysical Survey photometry combined with BB photometry from DESI Legacy Imaging Surveys and WISE. It claims this model can achieve a weighted F1 score of about point nine for spectral classification when looking at sources with an i value less than twenty-one.
Subrahmanyan: Why is that classification accuracy important right now? Because it's the foundation for everything else, isn't it? If you can correctly tell if something is a star or a galaxy, that lets you start inferring its physical properties, which feeds directly into our understanding of cosmic structure formation.
Vera: Exactly. The thesis here is that this model acts like a large language model for astronomy, predicting missing observations or physical properties from the data we already have by looking at the remaining available observations. It minimizes a composite loss function that combines things like regularization, classification loss, and morphological loss.
Jocelyn: And they trained it on about twenty million synthetic spectral energy distributions generated from DESI DR1 spectra. They also used a specific technique called unsupervised domain adaptation with a weighted maximum mean discrepancy penalty to bridge the gap between those synthetic mock data and the actual real J-PAS observations.
Subrahmanyan: Bridging that sim to real gap is crucial because we can't just train these models on perfect simulations; they have to perform well on messy, real observational data from surveys like DESI and WISE. That domain adaptation approach shows a practical way to make these tools useful without needing millions of real labeled objects upfront.
Vera: And the performance metrics are pretty solid across the board for what it's trying to do. For spectral classification, that weighted F1 score of point nine for sources with an i less than twenty-one is a big deal, but they also get good precision in photometric redshift estimation.
Jocelyn: They recover photometric redshifts with a precision of sigma NMAD one point five, reaching about zero point zero zero six at a redshift around three point five. And for galaxies, they can robustly estimate stellar masses and star formation rates with scatter levels of about zero point one one dex and about zero point two two dex respectively.
Subrahmanyan: Stellar masses and star formation rates are key parameters because they tell us how much baryonic matter has turned into stars in a galaxy, which directly impacts the growth of structure in the universe. If we can get those estimates with that kind of scatter, it gives us a good handle on galaxy evolution at those redshifts.
Paper summary: Vera: They also look at diagnostic line predictions, where they reproduce the Star-Forming sequence and identify intense active galactic nuclei in diagnostic diagrams. Specifically, for the BPT space using the Kewley et al. two thousand one line, they get f1 scores of about eighty percent for star-forming galaxies and about sixty eight percent for AGNs.
Jocelyn: Plus, they successfully disentangle H alpha and nitrogen two is despite them being spectrally close together, which is a neat detail given the data complexity of J-PAS. It shows the model isn't just guessing based on general trends.
Subrahmanyan: Being able to separate those line strengths means you're getting cleaner physical constraints on the gas conditions within those galaxies, which is vital for modeling how gas cools and forms stars over cosmic time. I wonder what kind of follow-up observations would be needed to confirm these specific classifications at higher redshifts.
Vera: Well, the modular architecture they built allows them to expand the output vocabulary efficiently by fine-tuning pre-trained embeddings to predict black hole masses. They also successfully recover spectroscopic virial estimates with a precision of about zero point five dex for high quality DESI targets.
Jocelyn: But they did flag some limitations too. The performance degrades when you look at the edges of the parameter space, like for extreme emission line galaxies or populations that aren't well represented in the DESI training set. They caution that performance biases can pop up if you apply it to those specific populations where data is sparse.
Subrahmanyan: That limitation about bias in underrepresented populations is something I think we need to keep watching. If the model works perfectly for bright, typical galaxies but fails on rare, extreme systems, it means we have a blind spot in our current understanding of how those extreme objects behave cosmologically.
Vera: So overall, this paper lays out a comprehensive characterization strategy covering spectral classification and physical properties from J-PAS and other photometric data. The core idea is to use this transformer model as a foundation for large-scale analysis where traditional SED fitting becomes too slow or biased.
Jocelyn: And the authors' conclusion points toward the fact that their predictive accuracy is really tied to how reliable those input catalogs are, meaning the model's success isn't magic; it depends on good inputs. This opens up possibilities for things like analyzing spatially resolved galaxies in the future.
Subrahmanyan: It sounds like a solid step forward in making large-scale surveys much more efficient for us. If this works as advertised across those diverse tasks, it could speed up the discovery pipeline significantly, helping us map out how galaxies evolve across cosmic history.
Conclusion: Vera: So we’re wrapping up this look at OJAL'A, which is about using a transformer model to classify astronomical objects and figure out their physical properties from J-PAS data and other photometric surveys.
Jocelyn: Yeah, it’s a foundation model trying to do two things at once—sorting sources like stars or galaxies and then guessing their mass or star formation rate.
Subrahmanyan: The authors are building this on top of existing transformer architectures, leaning into the idea that these models can predict missing data given what we already have.
Vera: They’ve trained it using synthetic data from DESI spectra, and they used a domain adaptation trick to make sure those simulated results matched real observations from J-PAS.
Jocelyn: It seems like the main goal is making large-scale analysis much faster by letting this model handle the classification and parameter estimation automatically.
Subrahmanyan: The real implication here is how much more we can process of this kind of observational data, which feeds into our models for how galaxies grow over time.
Vera: It’s a big step because it moves us away from slow, manual fitting methods toward something that can handle the whole catalog at once.
Jocelyn: But you have to remember that the performance really depends on the quality of those input catalogs they used for training, which is something they highlight.
Subrahmanyan: That dependency on input quality is key; if the initial data isn't good, the model's output won't be reliable either.
Vera: Right. So it’s a powerful tool for efficiency, but we need to keep looking at those specific limits they mentioned for certain types of galaxies.
Jocelyn: It opens up new ways to study these large surveys, and I think the next thing we can look at is how this model handles spatially resolved galaxies.
More episodes
- 2607.08835-Hybrid Simulations of Proton Acceleration at Oblique High- beta Shocks
- 2606.29552-Improving the Precision of Line-by-Line Radial Velocities: A Data-Driven Iterative Algorithm for Spectral Line Selection
- 2609.05312-Evidence for LP 890-9d via Transit Timing Variations
- 2609.10319-Extending the Little Red Dot population at intermediate redshift with VIPERS
- 2611.14233-Probing for anisotropies at different angular scales in the arrival directions of the cosmic rays detected at the Pierre Auger Observatory
- 2610.21722-Hydrostatic Equilibrium Estimate of the Fuzzy Dark Matter Mass from Dwarf Spheroidals: Method and Systematics
- 2610.40550-An Empirically Calibrated [C II] Luminosity-Halo Mass Relation for Line Intensity Mapping
- 2607.27328-PANTERA. I. Hunting for the Most Metal-Rich Stars in the Solar Neighborhood with High-Resolution Spectroscopy
- 2607.09850-Low-hard to high-soft spectral state transitions in the faintest early-X-ray-detected optical tidal disruption event TDE 2025aarm
- 2611.14028-Constraints on Lorentz Invariance Violation from PeV Gamma Rays of Cygnus X-3 Observed by LHAASO