Apple's AIMV2: Multimodal Vision Encoder Pre-training
AI Papers Podcast Daily · AIPPD
Audio is streamed directly from the publisher (media.rss.com) as published in their RSS feed. Play Podcasts does not host this file. Rights-holders can request removal through the copyright & takedown page.
Show Notes
This paper introduces AIMV2, a family of large-scale vision encoders pre-trained using a novel multimodal autoregressive method. Unlike previous methods, AIMV2 simultaneously predicts image patches and text tokens, leading to improved performance across various downstream tasks, including image recognition, object detection, and multimodal understanding. The approach is notably scalable and simpler to implement than comparable models. AIMV2 consistently outperforms state-of-the-art contrastive models on many benchmarks, showcasing its effectiveness as a generalist vision encoder. Extensive experiments demonstrate its strong scaling properties and compatibility with different model architectures and training techniques.