dmolino/text2ct-weights

Text-to-CT Model Weights

Checkpoints for "From Alignment to Synthesis: Contrastive Volumetric Grounding for Text-to-CT Generation" (Molino et al., BMVC 2026).

Note on versions. The checkpoints currently hosted here correspond to the earlier version of this work (arXiv v2). Weights for the updated model will be added alongside the code release.

---

Model Card

Model Description

• Authors: Daniele Molino, Camillo Maria Caruso, Filippo Ruffini, Valerio Guarrasi, Paolo Soda
• Model type: 3D latent diffusion (RFlow) + 3D VAE + CLIP3D text encoder for CT generation.
• License: Apache 2.0 (same as code release).
• Sources: Code https://github.com/cosbidev/Text2CT | Paper https://arxiv.org/abs/2506.00633
• Demo: Use diff_model_demo.py from the code release for a one-off generation from text.

Intended Use

• Direct use: Research/experimentation on text-conditioned 3D CT synthesis; generating synthetic data for benchmarking or augmentation.
• Downstream use: Fine-tuning or integration into broader research pipelines.
• Out of scope: Clinical decision-making, diagnostic use, or deployment without proper validation and approvals.

Risks & Limitations

• Trained on CT-RATE; may encode dataset biases and is not validated for clinical use.
• Synthetic outputs may contain artifacts; do not use for patient care.

Files included

• autoencoder_epoch273.pt — 3D VAE for latent compression/decoding.
• unet_rflow_200ep.pt — Diffusion UNet trained with rectified flow.
• CLIP3D_Finding_Impression_30ep.pt — CLIP3D weights for encoding reports.

How to Get Started (Python)

from huggingface_hub import snapshot_download

snapshot_download(
    repo_id="dmolino/text2ct-weights",
    repo_type="model",
    local_dir="your_local_path",
)

Use these in the code release configs:

Config field	Points to
trained_autoencoder_path	autoencoder
existing_ckpt_filepath / model_filename	unet
clip_weights	clip

Training Data (for these weights)

• CT-RATE dataset (public on Hugging Face) for CT volumes and reports.

Training Procedure (summary)

• CLIP3D trained for vision-language alignment on CT + reports.
• VAE checkpoint from https://github.com/Project-MONAI/tutorials/tree/main/generation/maisi.
• Diffusion UNet trained with rectified flow (RFlow) in latent space, conditioned on text embeddings.

Evaluation

• See paper for quantitative and qualitative results.

Further Information

• 1,000 generated CT scans are available at https://huggingface.co/datasets/dmolino/CT-RATE_Generated_Scans.

Environmental Impact

• Not reported. Training used a multi-GPU setup.

Citation

If you use these weights or code, please cite:

@misc{molino2026alignment,
      title={From Alignment to Synthesis Contrastive Volumetric Grounding for Text-to-CT Generation}, 
      author={Daniele Molino and Camillo Maria Caruso and Filippo Ruffini and Paolo Soda and Valerio Guarrasi},
      year={2026},
      eprint={2506.00633},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2506.00633}, 
}