Language Model Inversion through End-to-End Differentiation

Abstract

Despite emerging research on Language Models (LM), few approaches analyse the invertibility of LMs. That is, given a LM and a desirable target output sequence of tokens, determining what input prompts would yield the target output remains an open problem. We formulate this problem as a classical gradient-based optimisation. First, we propose a simple algorithm to achieve end-to-end differentiability of a given (frozen) LM and then find optimised prompts via gradient descent. Our central insight is to view LMs as functions operating on sequences of distributions over tokens (rather than the traditional view as functions on sequences of tokens). Our experiments and ablations demonstrate that our DLM-powered inversion can reliably and efficiently optimise prompts of lengths 10 and 80 for targets of length 20, for several white-box LMs (out-of-the-box).

Kevin Denamganaï

Independent Researcher

My research investigates the conditions under which AI systems acquire and deploy structured symbolic representations — towards in-context grounding of novel atomic symbols and their systematic recombination into unseen configurations — spanning Compositional Generalisation, Formal Mathematics, Differentiable Language Models, and Physical Simulation. I have also investigated Language Emergence & Grounding (Emergent Communication), Unsupervised Representation Learning, Natural Language Processing, and Multi-Agent Deep Reinforcement Learning.