Generalized Visual Language Models
<p>Processing images to generate text, such as image captioning and visual question-answering, has been studied for years. Traditionally such systems rely on an object detection network as a vision encoder to capture visual features and then produce text via a text decoder. Given a large amount of existing literature, in this post, I would like to only focus on one approach for solving vision language tasks, which is to <em>extend pre-trained <a href="https://lilianweng.github.io/posts/2019-01-31-lm/">generalized language models</a> to be capable of consuming visual signals</em>.</p>