-
Book Overview & Buying
-
Table Of Contents
RAG from First Principles
By :
Anna: Lewis, embedding models for a single modality (such as text or image) are already relatively mature. However, in our actual application scenarios, we often need to process multi-modal data such as text, images, and even audio and video at the same time to achieve richer and more flexible functions.
Lewis: Multimodal embedding models have emerged to address such needs. They can map data from different modalities into the same vector space, thereby supporting cross-modal retrieval, cross-modal generation, and multimodal analysis.
A representative work is the Visualized_BGE model from BAAI, which is a visualized version launched based on the BGE model to meet the needs of multimodal processing. The Visualized_BGE model mainly focuses on the joint processing of image and text modalities. By measuring the similarity between image and text vectors in the same embedding space, the model enables cross-modal retrieval and image-text search...