OJAL'A: Optimizing J-PAS Astronomy for Large-scale Analysis. A transformer-based model for source classification and physical-parameter inference
Listen
Radio episode about this paper
Transcript
Introduction to the show: ident: Astrophysics Radio. Generated commentary on the latest astrophysics papers.
Vera: I'm Vera.
Jocelyn: And I'm Jocelyn, here with Subrahmanyan.
Vera: We're looking at the paper: "OJAL'A: Optimizing J-PAS Astronomy for Large-scale Analysis. A transformer-based model for source classification and physical-parameter inference".
Jocelyn: Here's the gist of the paper. OJALA is a transformer-based autoregressive foundation model designed to simultaneously classify astronomical objects and infer their physical parameters using 54-band photometry from the Javalambre Physics of the Accelerating Universe Astrophysical Survey (J-PAS), combined with BB photometry from the DESI Legacy Imaging Surveys and WISE.
Vera: The model achieves a weighted F1-score of ∼ 0.9 for spectral classification (stars, galaxies, and QSOs) at i < 21.
Jocelyn: Model Architecture and Training Strategy The OJALA model builds upon the Transformer-based encoder–decoder architecture introduced by Leung & Bovy (2024).
Vera: It operates analogously to Large Language Models by predicting missing observations or physical properties of an object given the remaining available observations.
Paper summary: Vera: So, we're looking at this paper called "OJAL'A: Optimizing J-PAS Astronomy for Large-scale Analysis. A transformer-based model for source classification and physical-parameter inference." It basically introduces a foundation model, a transformer based one, designed to classify astronomical objects like stars, galaxies, and QSOs while simultaneously figuring out their physical parameters using photometric data.
Jocelyn: That sounds like it's trying to tackle the problem of how we can make sense of all that Javalambre Physics of the Accelerating Universe Astrophysical Survey photometry combined with BB photometry from DESI Legacy Imaging Surveys and WISE. It claims this model can achieve a weighted F1 score of about point nine for spectral classification when looking at sources with an i value less than twenty-one.
Subrahmanyan: Why is that classification accuracy important right now? Because it's the foundation for everything else, isn't it? If you can correctly tell if something is a star or a galaxy, that lets you start inferring its physical properties, which feeds directly into our understanding of cosmic structure formation.
Vera: Exactly. The thesis here is that this model acts like a large language model for astronomy, predicting missing observations or physical properties from the data we already have by looking at the remaining available observations. It minimizes a composite loss function that combines things like regularization, classification loss, and morphological loss.
Jocelyn: And they trained it on about twenty million synthetic spectral energy distributions generated from DESI DR1 spectra. They also used a specific technique called unsupervised domain adaptation with a weighted maximum mean discrepancy penalty to bridge the gap between those synthetic mock data and the actual real J-PAS observations.
Subrahmanyan: Bridging that sim to real gap is crucial because we can't just train these models on perfect simulations; they have to perform well on messy, real observational data from surveys like DESI and WISE. That domain adaptation approach shows a practical way to make these tools useful without needing millions of real labeled objects upfront.
Vera: And the performance metrics are pretty solid across the board for what it's trying to do. For spectral classification, that weighted F1 score of point nine for sources with an i less than twenty-one is a big deal, but they also get good precision in photometric redshift estimation.
Jocelyn: They recover photometric redshifts with a precision of sigma NMAD one point five, reaching about zero point zero zero six at a redshift around three point five. And for galaxies, they can robustly estimate stellar masses and star formation rates with scatter levels of about zero point one one dex and about zero point two two dex respectively.
Subrahmanyan: Stellar masses and star formation rates are key parameters because they tell us how much baryonic matter has turned into stars in a galaxy, which directly impacts the growth of structure in the universe. If we can get those estimates with that kind of scatter, it gives us a good handle on galaxy evolution at those redshifts.
Paper summary: Vera: They also look at diagnostic line predictions, where they reproduce the Star-Forming sequence and identify intense active galactic nuclei in diagnostic diagrams. Specifically, for the BPT space using the Kewley et al. two thousand one line, they get f1 scores of about eighty percent for star-forming galaxies and about sixty eight percent for AGNs.
Jocelyn: Plus, they successfully disentangle H alpha and nitrogen two is despite them being spectrally close together, which is a neat detail given the data complexity of J-PAS. It shows the model isn't just guessing based on general trends.
Subrahmanyan: Being able to separate those line strengths means you're getting cleaner physical constraints on the gas conditions within those galaxies, which is vital for modeling how gas cools and forms stars over cosmic time. I wonder what kind of follow-up observations would be needed to confirm these specific classifications at higher redshifts.
Vera: Well, the modular architecture they built allows them to expand the output vocabulary efficiently by fine-tuning pre-trained embeddings to predict black hole masses. They also successfully recover spectroscopic virial estimates with a precision of about zero point five dex for high quality DESI targets.
Jocelyn: But they did flag some limitations too. The performance degrades when you look at the edges of the parameter space, like for extreme emission line galaxies or populations that aren't well represented in the DESI training set. They caution that performance biases can pop up if you apply it to those specific populations where data is sparse.
Subrahmanyan: That limitation about bias in underrepresented populations is something I think we need to keep watching. If the model works perfectly for bright, typical galaxies but fails on rare, extreme systems, it means we have a blind spot in our current understanding of how those extreme objects behave cosmologically.
Vera: So overall, this paper lays out a comprehensive characterization strategy covering spectral classification and physical properties from J-PAS and other photometric data. The core idea is to use this transformer model as a foundation for large-scale analysis where traditional SED fitting becomes too slow or biased.
Jocelyn: And the authors' conclusion points toward the fact that their predictive accuracy is really tied to how reliable those input catalogs are, meaning the model's success isn't magic; it depends on good inputs. This opens up possibilities for things like analyzing spatially resolved galaxies in the future.
Subrahmanyan: It sounds like a solid step forward in making large-scale surveys much more efficient for us. If this works as advertised across those diverse tasks, it could speed up the discovery pipeline significantly, helping us map out how galaxies evolve across cosmic history.
Conclusion: Vera: So we’re wrapping up this look at OJAL'A, which is about using a transformer model to classify astronomical objects and figure out their physical properties from J-PAS data and other photometric surveys.
Jocelyn: Yeah, it’s a foundation model trying to do two things at once—sorting sources like stars or galaxies and then guessing their mass or star formation rate.
Subrahmanyan: The authors are building this on top of existing transformer architectures, leaning into the idea that these models can predict missing data given what we already have.
Vera: They’ve trained it using synthetic data from DESI spectra, and they used a domain adaptation trick to make sure those simulated results matched real observations from J-PAS.
Jocelyn: It seems like the main goal is making large-scale analysis much faster by letting this model handle the classification and parameter estimation automatically.
Subrahmanyan: The real implication here is how much more we can process of this kind of observational data, which feeds into our models for how galaxies grow over time.
Vera: It’s a big step because it moves us away from slow, manual fitting methods toward something that can handle the whole catalog at once.
Jocelyn: But you have to remember that the performance really depends on the quality of those input catalogs they used for training, which is something they highlight.
Subrahmanyan: That dependency on input quality is key; if the initial data isn't good, the model's output won't be reliable either.
Vera: Right. So it’s a powerful tool for efficiency, but we need to keep looking at those specific limits they mentioned for certain types of galaxies.
Jocelyn: It opens up new ways to study these large surveys, and I think the next thing we can look at is how this model handles spatially resolved galaxies.
G. Martínez-Solaeche, R. M. González Delgado, R. García-Benito, A. Hernán-Caballero, I. Pérez-Ráfols, L. A. Díaz-García, L Raul Abramo, J. E Rodríguez-Martín, A. M Conrado, I Breda, H Domínguez SanchḜhe, I Márquez, M Pieri, D López-Cano, V M Placco, L Nakazono, A del Pino, V Marra, J Alcaniz, N Benitez, S Bonoli, S Carneiro, A J Cenarro, D Cristóbal-Hornillos, S Daflon R Dupke A Ederoclite C Hernández-Monteagudo J Liu C López-Sanjuan A Marín-Franch C Mendes de Oliveira M Moles F Roig L Sodré K Taylor J Varela H Vázquez Ramiú J M Víchez, and J Zaragoza-Cardiel
Instituto de Astrofísia de Andalucia · Centro de Estudios de F귬 del Cosmos de Arag᪭a · Departamento de F귬 Fisica, Universitat Politècnica de Catalunya · Departamento de F귬 Fisica Matematica, Instituto de F귬 Física, Universidade do Estado da Bahia · Department of Astrophysics, University of Vienna · Instituto de F귬 Física de Cantabria · Aix Marseille Univ. CNRS. CNES. LAM
astro-ph.GA, astro-ph.IM
Submitted: 2026-04-01
Updated: 2026-11-15
Comments: Accepted for publication in Astronomy & Astrophysics
Code: https://github.com/gimarso/OJALA
Project page: https://data.desi.lbl.gov/doc/releases/dr1
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Importance score: 79/100
The gist: OJALA is a transformer-based autoregressive foundation model designed to simultaneously classify astronomical objects and infer their physical parameters using 54-band photometry from the Javalambre…
Key concepts
- Transformer Encoder–Decoder Architecture
- This is the core structure of the AI model. It works like a sophisticated language processor that takes an input (like a set of astronomical data) and predicts missing or related information. The self-attention mechanism allows the model to weigh different parts of the input data simultaneously, enabling it to understand complex relationships between various photometric bands.
- Unsupervised Domain Adaptation (UDA)
- This technique is used to make the model perform well on real astronomical data when it was primarily trained on simulated data. The model learns a representation of the features from synthetic J-spectra and then uses a penalty (Maximum Mean Discrepancy) to align these learned features with the distribution found in real, unlabeled J-PAS observations.
- Multi-task Loss Function
- Instead of training the model on one single goal, OJALA minimizes a combined loss function that includes three separate tasks: spectral classification, photometric redshift estimation, and inferring physical properties like stellar mass. This forces the model to learn a comprehensive understanding of the data simultaneously.
- Weighted F1-score
- This is a metric used to evaluate how well the model classifies objects into categories like stars, galaxies, or QSOs. A weighted F1-score of around 0.9 means that when classifying these objects, the model is highly accurate and reliable across different types of astronomical sources.
Terminology
Summary
OJALA is a transformer-based autoregressive foundation model designed to simultaneously classify astronomical objects and infer their physical parameters using 54-band photometry from the Javalambre Physics of the Accelerating Universe Astrophysical Survey (J-PAS), combined with BB photometry from the DESI Legacy Imaging Surveys and WISE. p. 0 The model achieves a weighted F1-score of ∼ 0. p. 09 for spectral classification (stars, galaxies, and QSOs) at i < 21. p. 0
Model Architecture and Training Strategy
The OJALA model builds upon the Transformer-based encoder–decoder architecture introduced by Leung & Bovy (2024). It operates analogously to Large Language Models by predicting missing observations or physical properties of an object given the remaining available observations. p. 3 The architecture relies on a Transformer Encoder-Decoder structure powered by the self-attention mechanism. p. 3 The model is trained by minimizing a composite multi-task loss function where the supervised component is defined as the sum of three contributions: Lsup = Lreg + Lcls + Lmorph. p. 4
Data Generation and Domain Adaptation
The model is trained on approximately 20 million synthetic SEDs generated from DESI DR1 spectra. To address the sim-to-real domain gap between the synthetic J-spectra (source domain, S) and real observations (target domain, U), the authors implement a form of Unsupervised Domain Adaptation (UDA) using a weighted Maximum Mean Discrepancy (MMD) penalty. p. 4 This approach aligns the latent feature distributions of synthetic mocks with real, unlabeled J-PAS observations. p. 1
Performance Across Astrophysical Tasks
The model demonstrates robust performance across several key astronomical tasks. For spectral classification, the model achieves a global weighted F1-score of ∼ 0. p. 09 for sources with i < 21. p. 1 In photometric redshift estimation, galaxies recover photometric redshifts with a precision of σNMAD 1. p. 05, reaching σNMAD ≈ 0. p. 0006 at z ∼ 3. p. 05. Robust estimation of physical properties for galaxies recovers stellar masses and star formation rates with a scatter of ∼ 0. p. 011 dex and ∼ 0. p. 022 dex, respectively. p. 0
Inference Capabilities and Limitations
The modular architecture allows for the efficient expansion of its output physical vocabulary by fine-tuning pre-trained embeddings to predict black hole masses. The model successfully recovers spectroscopic virial estimates with a precision of ∼ 0. p. 05 dex for high-quality DESI targets. p. 6 However, the performance degrades at the edges of the parameter space (e. p. 5g., extreme emission line galaxies). p. 7 The authors caution that performance biases may arise when applied to populations underrepresented in the DESI training set. p. 3
Diagnostic Line Predictions
The model accurately reproduces the Star-Forming sequence and identifies intense AGN in diagnostic diagrams. In the BPT space, the model achieves F1-scores of ∼ 80% for SF galaxies and ∼ 68% for AGNs using the Kewley et al. p. 8 (2001) line. p. 8 The model also successfully disentangles Hα and [N ii] despite their spectral proximity. p. 7
Summary and Conclusions
The main conclusions of this study are summarized as follows: General Performance, Spectral classification, Photometric redshifts, Stellar Parameters, Emission lines and diagnostics, Stellar Mass and SFR. The model's predictive accuracy is intrinsically bound to the reliability of these input catalogs. The architecture opens avenues for applications beyond single-object analysis, such as the analysis of spatially resolved galaxies. p. 0
References
Baqui, P. O., Marra, V., Casarini, L., et al. 2021, A&A, 645, A87
Benitez, N., Dupke, R., Moles, M., et al. 2014, arXiv e-prints, arXiv:1403. p. 05237
Bertin, E. 2011, in Astronomical Society of the Pacific Conference Series, Vol. p. 1 442, Astronomical Data Analysis Software and Systems XX, ed. p. 0 I. N. Evans, A. Accomazzi, D. J. Mink, & A. H. Rots
Bond-Taylor, S., Leach, A., Long, Y., & Willcocks, C. G., 2021, arXiv e-prints, arXiv:2103. p. 004922
Bonoli, S., Marín-Franch, A. p. 0, Varela, J., et al. 2021, A&A, 653, A31
Breda, I., Amarantidis, S., Vilchez, J. M., et al. 2024, MNRAS, 528, 3340
Bruzual, G. & Charlot, S., 2003, MNRAS, 344, 1000
Bundy, K., Bershady, M. A., Law, D. R., et al. 2015, ApJ, 798, 7
Cappellari, M. & Copin, Y., 2003, MNRAS, 342, 345
Chabrier, G., 2003, PASP, 115, 763
Chaves-Montero, J., Bonoli, S. p. 0 p. 0, Trakhtenbrot, B., et al. 2022, A&A, 660, A95
Cid Fernandes, R., Stasinska, G. p. 8, Mateus, A., & Vale Asari, N., 2011, MNRAS, 413, 1687
Conroy, C., 2013, ARA&A, 51, 393
Dao, T., Fu, D. Y., Ermon, S., Ré, C., & Rudra, A., 2022, arXiv e-prints, arXiv:2205. p. 014135
del Pino, A., López-Sanjuan, C. p. 0 p. 0, Hernán-Caballero, A. p. 6 p. 6, et al. 2024, A&A, 691, A221
Domínguez Sánchez, H., Coelho, P., Bruzual, G., et al. 2024, A&A, 688, A113
Domínguez Sánchez, H., Huertas-Company, M., Bernardi, M., Tuccillo, D., & Fischer, J. L., 2018, MNRAS, 476, 3661
Domínguez Sánchez, H., Margalef, B., Bernardi, M., & Huertas-Company, M., 2022, MNRAS, 509, 4024
Euclid Collaboration, Mellier, Y., Abdurro’uf, et al. 2024, arXiv e-prints, arXiv:2405. p. 013491
Fernández-Ontiveros, J. A., López-Sanjuan, C., Hernán-Caballero, A., et al. 2025, arXiv e-prints, arXiv:2511. p. 0 p. 021822
Galarza, C. A., Daflon, S., Placco, V. M., et al. 2022, A&A, 657, A35
Giménez-Alcázar, A., Amorín, R., Vílchez, J. M., et al. 2026, A&A, 706, A261
González Delgado, R. M., Díaz-García, L. p. 1 p. 1 p. 1 A., de Amorim, A., et al. 2021, A&A, 649, A79
González Delgado, R. M., Rodríguez-Martín, J. p. 0 E., Díaz-García, L. A., et al. 2022, A&A, 666, A84
Gretton, A., Borgwardt, K. M., Rasch, M. J., Sch"olkopf, B., & Smola, A. J., 2007, in Proceedings of the 22nd International Conference on Neural Information Processing Systems (NeurIPS 2007) (Curran Associates Inc. p. 7)
Hernán-Caballero, A. et al. 2021, A&A, 654, A101
Hoffmann, J., Borgeaud, S., Mensch, A., et al. 2022, arXiv e-prints, arXiv:2203. p. 015556
Iglesias-Páramo, J., Arroyo, A., Kehrig, C., et al. 2022, A&A, 665, A95
Ivezic, Ž., Kahn, S. M., Tyson, J. A., et al. 2019, ApJ, 873, 111
Jeakel, A. P., Vieira dos Santos, G., Marra, V., et al. 2025, arXiv e-prints, arXiv:2511.20524
Jin, S., Trager, S. C., Dalton, G. B., et al. 2024, MNRAS, 530, 2688
Kauffmann, G., Heckman, T. M., Tremonti, C., et al. 2003, MNRAS, 346, 1055
Kennicutt, Jr., R. C., 1998, ApJ, 498, 541
Kewley, L. J., Dopita, M. A., Sutherland, R. S., Heisler, C. A., & Trevena, J., 2001, ApJ, 556, 121
Koposov, S. E., Li, T. S., Allende Prieto, C., et al. 2025, arXiv e-prints, arXiv:2505. p. 014787
Leung, H. W. & Bovy, J., 2024, MNRAS, 527, 1494
Levi, M., Bebek, C., Beers, T., et al. 2013, arXiv e-prints, arXiv:1308. p. 00847
López, I. E., Brusa, M., Bonoli, S., et al. 2023, A&A, 672, A137
López-Cano, D., Abramo, L. R., Nakazono, L., et al. 2026, arXiv e-prints, arXiv:2602. p. 013902
Martínez-Solaeche, G., García-Benito, R. p. 0 p. 0 p. 0, González Delgado, R. p. 1 p. 1 p. 1 p. 1 p. 1 p. 1 M., et al. 2024, A&A, 688, A160
Martínez-Solaeche, G., González Delgado, R. M., García-Benito, R., et al. 2021, A&A, 647, A158
Martínez-Solaeche, G., González Delgado, R. M., García-Benito, R., et al. 2022, A&A, 661, A99
Queiroz, C., González Delgado, R. M., Martínez-Solaeche, G. p. 0 p. 0, et al. 2023, A&A, 673, A103
Rodrigues, N. V. N., Raul Abramo, L. p. 0, Queiroz, C., et al. 2023, MNRAS, 520, 3494
Rodríguez-Martín, J. E., Díaz-García, L. A., González Delgado, R. M., et al. 2025, A&A, 704, A52
Rodríguez-Martín, J. E., González Delgado, R. M., Martínez-Solaeche, G., et al. 2026, arXiv e-prints, arXiv:2601. p. 008911
Sánchez, S. F., Kennicutt, R. p. 9 C., Gil de Paz, A., et al. 2012, A&A, 538, A8
Schuldt, S., Suyu, S. H., Cañameras, R., et al. 2021, A&A, 651, A55
Siudek, M., Mezcua, M., Circosta, C., et al. 2025, A&A, 700, A209
Siudek, M., Pucha, R., Mezcua, M., et al. 2024, A&A, 691, A308
Smith, M. J., Roberts, R. J., Angeloudi, E., & Huertas-Company, M., 2024, arXiv e-prints, arXiv:2405. p. 014930
Stasinska, G., Cid Fernandes, R. p. 7, Mateus, A., Sodré, L., & Asari, N. V., 2006, MNRAS, 371, 972
Torralba-Torregrosa, A., Gurung-López, S. p. 0, Arnalte-Mur, P., et al. 2023, A&A, 680, A14
Walmsley, M., Bowles, M., Scaife, A. M. M., et al. 2024, arXiv e-prints, arXiv:2404. p. 002973
Whitten, D. D., Placco, V. M., Beers, T. C., et al. 2019, A&A, 622, A182
Woo, J., Walters, D., Archinuk, F., et al. 2024, MNRAS, 530, 4260
Wright, E. L., Eisenhardt, P. R. M., Mainzer, A. K., et al. 2010, AJ, 140, 1868
York, D. G., Adelman, J., Anderson, John E., J., et al. 2000, AJ, 120, 1579
Yuan, H. B., Yang, L., Cruz, P., et al. 2023, MNRAS, 518, 2018
Zhong, J., Deng, Z., Li, X., et al. 2024, MNRAS, 531, 2011
Appendix A: Architecture and Training Details
OJALA is a Transformer-based model implemented in PyTorch that utilizes Flash Attention (Dao et al.
Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.
Improvements for AI systems
-
Improved model flexibility allows for training on a union of all available observations rather than being restricted to a strict cross-match, enabling robust inference on partial data as stated:
This flexibility provides two important advantages. First, the training set can be constructed from the union of all available observations and catalogs, rather than being restricted to a strict cross-match.
-
Enhanced architecture supports multi-modality by integrating photometry from different surveys alongside physical properties from various catalogs, allowing the model to be applied to other science cases: "The architecture of OJALA opens avenues for applications beyond single-object analysis. A particularly promising direction, which will be the subject of a forthcoming paper, is the analysis of spatially resolved galaxies."
-
Physical parameters can be inferred by fine-tuning pre-trained embeddings, allowing for efficient expansion of the output physical vocabulary beyond the initial training set: "To illustrate this capability, we performed a fine-tuning experiment to predict BH masses for DESI QSO. We utilize the catalog provided by Pan et al. (2025), which provides iron-corrected single-epoch virial mass estimates."
-
The model can perform physicsaware segmentation for spatially resolved galaxies by exploiting embeddings to group pixels based on similarity rather than just spatial proximity: "The similarity search approach used in this work to assign realistic errors to mocks can be adapted to perform physicsaware segmentation, grouping pixels based on the similarity of their embeddings rather than just spatial proximity or luminosity."
-
The system can simultaneously classify objects and infer physical parameters by minimizing a composite multi-task loss function including classification, regression, and morphology terms:
The supervised component, Lsup is defined as the sum of three contributions, Lsup = Lreg + Lcls + Lmorph.
Sources
- J-PAS: The Javalambre-Physics of the Accelerated Universe Astrophysical Survey
- Deep Generative Modelling: A Comparative Review of VAEs, GANs, Normalizing Flows, Energy-Based and Autoregressive Models
- FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness
- Data Release 1 of the Dark Energy Spectroscopic Instrument
- Euclid. I. Overview of the Euclid mission
- J-PAS: A value-added catalogue of optical line intensities for nebular emission galaxies (JOLINES)
- Training Compute-Optimal Large Language Models
- The miniJPAS and J-NEP surveys: Machine learning for star-galaxy separation
- DESI Data Release 1: Stellar Catalogue
- The DESI Experiment, a whitepaper for Snowmass 2013
- J-PAS: Semi-Supervised Sim-to-Obs Transfer for Robust Star--Galaxy--Quasar Classification
- AION-1: Omnimodal Foundation Model for Astronomical Sciences
- The miniJPAS survey: Dissecting galaxy properties across environments with spatially resolved photometry
- AstroPT: Scaling Large Observation Models for Astronomy
- Scaling Laws for Galaxy Images
Related papers
- The Pristine survey XXVIII. Journey to the Galactic outskirts: Mapping the outer halo red giant stars down to the very metal-poor end
- Origins of Extreme Emission-Line Ratios in z > 3 Galaxies: Insights from the Lumen Model
- The Kinematically Hot, Extremely Metal-Poor C-19 Stellar Stream in DESI DR2
- UV slopes of Starforming Galaxies in Strong Lensing fields at the Epoch of Reionization with JWST
- Evolution of Binaries Under Stochastic Perturbations
- The hydrodynamical response of cold circumgalactic clouds to quasar radiation