Falcon audio-visual target speaker extraction model released
The story so far
Falcon has released the audio-visual target speaker extraction model. This model extracts the voice of the target speaker from noisy overlapping speech using the target speaker's mouth video, combined with a single multi-scale time-frequency separator and an encoder spatial speaker contrast loss, reducing interference speaker leakage without increasing inference costs.
AI-written from the reports · Updated 21 days ago
Coverage timeline
Follow the coverage to see every side of the story.
- Hugging Face Daily PapersEfficient Streaming Audio-Visual Target Speaker Extraction for Real-World Acoustic Scenes
Falcon is a causal audio-visual target speaker extraction model that extracts the voice of the target speaker from the mouth video of the target speaker in noisy overlapping speech, combined with a single multi-scale time-frequency separator and an encoder spatial speaker contrast loss, reducing interference speaker leakage without increasing inference cost.