MolP2SBench: Do LLMs Really Reason from Molecular Properties to Structure?
MolP2SBench is a benchmark designed to test if large language models (LLMs) truly reason from physical molecular properties to predict chemical structures. It masks molecular scaffolds and tests how well models use given properties, even when those properties are changed or shuffled. The findings show that LLM reliance on specific property types varies significantly, suggesting performance depends heavily on the physical data used.