CorText
Brain-language fusion enables interactive neural readout
CorText enables query of visually evoked brain activity using natural language. It maps fMRI recording into an LLM’s language embedding space end-to-end, moving neural decoding from static labels toward flexible interaction with brain activity. The resulting interface allows testing how changes in neural data affect semantic output.
What CorText does
- Interactive neural readout: CorText answers open-ended and follow-up questions about the content of visually evoked neural responses.
- Zero-shot generalisation: Neural decoding of concepts beyond those explicitly encountered during training.
- Counterfactual experimentation: In-silico microstimulation enables counterfactual prompts on brain activity, and reveals a consistent and graded mapping between brain-state and language output.
About the project
CorText is the core project of my PhD research, where I have led the central development, implementation and analysis of the model. This project is supervised by Prof. Dr. Tim C Kietzmann at the University of Osnabrück, in collaboration with Daniel Anthes, Adrien Doerig, Sushrut Thorat, and Peter König.
Read the preprint →
Resources and project evolution
- 2026 · Extension of CorText to macaque intracranial recordings (CCN 2026 abstract and poster)
- 2025 · Development of Q&A functionality in CorText, direct decoder-only fusion. Short paper and poster presented at CCN 2025.
- 2024 · Caption decoding with encoder-decoder architecture. The first short paper presented at CCN 2024.
