Researchers connect telescope images and spectra to language with AstroBridge

Fourteen researchers authored AstroBridge, a framework that connects astronomical images, spectra, and light curves to a large language model, and Gaurav Sekar submitted version 1 to arXiv on Oct. 26, 2026.

AstroBridge is an astro-ph.IM framework that links observations arriving as images, spectra, and light curves to a large language model through a literature-assisted caption generation pipeline. astro-ph.IM is the arXiv category for Instrumentation and Methods for Astrophysics. A large language model is an AI system trained to understand and generate text. A literature-assisted caption generation pipeline uses published scientific writing to help create descriptions of the observations.

The paper’s abstract contrasts observations, which arrive in these data formats, with literature that describes the same celestial objects in natural language. Observations come as images, spectra, and light curves, while words make up the scientific record. AstroBridge connects the two, using a large language model as the bridge.

The pipeline begins by using catalog metadata to pair each observation with scientific texts describing the matching object. Catalog metadata is data about the observation files themselves. Researchers match observations to scientific texts about the same objects using this catalog metadata. The catalog provides the link between a data file and the published text about it.

This pairing step happens before caption generation.

Once researchers pair an observation with the relevant passages, the matched set moves forward to the captioning stage. A vision-language model writes descriptive captions from observation representations, matched passages, and metadata. A vision-language model combines visual input with text understanding. Observation representations are the mathematical forms of the raw data.

The model reads both the observation and the matched text together.

The process produces observation-caption pairs without manual captioning, meaning no person writes the captions by hand. Manual captioning would require people to write descriptions for every observation by hand. Those paired items feed the model, which generates the text automatically.

Training predicts captions directly from raw observations using modality-specific frozen encoders. Each modality—images, spectra, and light curves—passes through its own frozen encoder. The encoders do not change during training.

A trainable shared fusion network then combines these representations. A fusion network merges information from different sources. It lines the representations up with a large language model’s embedding space. An embedding space is a mathematical representation of meaning used by language models.

The model learns to turn raw inputs into text captions through this alignment. Training connects the observation encoders to the language model’s internal representation of meaning. The alignment lets the language model generate text from the observation data.

The authors call AstroBridge a step toward aligning astronomy’s diverse, exotic data through text as a common language. They frame the work as movement toward expressing astronomy’s varied and exotic data using text as a shared language. Text becomes the medium that lets different data types speak to one another. The goal is to let text carry meaning across observation types.

On spectra, AstroBridge reaches 91.5% accuracy distinguishing quasars from galaxies in a caption-based evaluation. Spectra break light into its component wavelengths. The accuracy figure comes from a caption-based evaluation centered on spectra.

The same evaluation gives the base model 69.0% accuracy before fine-tuning. The pre-fine-tuning base model result provides the comparison point for the 91.5% figure. Fine-tuning adjusts a pretrained model for a specific task. The base model is the starting version before task-specific training.

Reported limitations cover galaxy morphology, classification of minority supernova classes, and the factual quality of generated captions. The authors list these as the remaining weak points alongside the reported accuracy gain. Factual quality refers to whether the captions correctly describe the observations.

The paper The Great Chat in the Sky: Aligning Exotic Astronomical Data to Language with AstroBridge appears under Instrumentation and Methods for Astrophysics. Gaurav Sekar submitted version 1 on Oct. 26, 2026 at 22:44:06 UTC as a 552 KB file. The arXiv listing assigns the paper to the Instrumentation and Methods for Astrophysics subject area.

Instrumentation and Methods for Astrophysics covers the tools and techniques used in astrophysics research. The subject area fits a paper about a framework for handling observational data.

A roster of named researchers follows the abstract. They are Ibrahim Habib, Kevin Li, Trung Nguyen, Gabriel A. Patron, Gaurav Sekar, Nolan Koblischke, Matthieu Le Lain, Shashwat Sourav, Aman Kumar, Kartheik G. Iyer, Joshua S. Speagle, Itai Sfaradi, Michael J. Smith, and Ioana Ciuca.

The Hugging Face release UniverseTBD/astrobridge-model-v6 is an image-plus-spectra astronomy captioner built as a LoRA adapter and fusion stack on frozen Qwen/Qwen3.5-9B. A LoRA adapter is a small trainable module added to a larger model. A fusion stack combines multiple processing stages into one pipeline.

The model card describes the release as an adapter and fusion stack rather than a complete language model. Qwen/Qwen3.5-9B is a large language model.

The release does not include the base Qwen/Qwen3.5-9B model, which users must load separately before applying the adapter. The model card tells users to get the base model themselves first.

Release metadata for UniverseTBD/astrobridge-model-v6 carries config hash bf18d043bdcaa5d5, no quantization, and git sha a5fa267401ffe7e85cb2c0870f8bc3a642acc18e. A config hash is a fingerprint of the settings file. No quantization means the model weights are not compressed to lower precision. A git sha is a unique identifier for a specific commit in a version control system.

The tier histogram for UniverseTBD/astrobridge-model-v6 divides items into single-modality and joint-modality counts. It records 4120 single-modality items and 1318 joint-modality items. Single-modality items use one observation type, while joint-modality items combine types. The counts show the breakdown between observations that use one data type and those that combine types.

Ablation testing checked whether removing or altering each modality changed captions. Ablation testing removes parts of a system to see how each part affects the output. The image ablation test used 122 items, and all changed captions, with a non-null result. A non-null result means the test produced a measurable outcome.

The spectra ablation test used 54 items, and all changed captions, with a non-null result. Spectra break light into its component wavelengths. The result indicates the spectra input affects the generated captions.

Light curve ablation testing used 24 items, all changed captions, with a non-null result. Light curve shuffle testing used 4 items with a mean edit distance of 291.5 and a non-null result. A light curve is a graph of brightness over time. A mean edit distance measures how much the captions changed when shuffling the light curve order.