MC-Lens explores the consciousness mechanism of VLMs
The story so far
Hugging Face introduced the Multimodal Concept Lens (MC-Lens) to explore the 'consciousness' mechanism of Visual Language Models (VLMs). By mapping the intermediate states of visual and language encoders to a shared semantic space, it reveals the accessibility and selective access mechanisms of concepts. Experiments show that the information of accessed concepts participates in downstream computation and reasoning, and it was found that there are differences in the development of cross-modal alignment and selective access.
AI-written from the reports · Updated 2 hours ago
Coverage timeline
Follow the coverage to see every side of the story.
- Hugging Face Daily PapersProbing the "Conscious" Mind: Selective Access and Global Workspace in Vision-Language Models
Multimodal Concept Lens (MC-Lens) reveals the accessibility and selective access mechanisms of VLM concepts by mapping the intermediate states of visual and language encoders to a shared semantic space. Experiments show that the accessed conceptual information participates in downstream computation and reasoning, and it is found that there are differences in the development of cross-modal alignment and selective access.
Heat over time
Not enough data yet to draw a trend.