Encoder-Decoder Transformers: Logical Characterizations and Periodicity
The paper provides a logical framework to formally characterize encoder-decoder transformers by extending propositional logic with counting global and past modalities, termed GPTL−. It shows these models are equivalent to specific modal logics and distributed automata, offering a way to study their expressive power beyond simple text generation metrics.