Patch release v5.20.2

Brief overview

Transformers releases patch update v5.20.2, fixing an issue where input types were lost in the generate function, affecting multimodal models; it also addressed several other issues, including input dictionary type conversion, bfloat16 expert parallel gradients being zero, VLM input modal declaration for only reporting text, and host-to-device copy in eager_mask.

Article

Patch release: v5.20.2

This patch notably solves an issue where some inputs could loose their original type inside prepare_inputs_for_generation in generate, leading to problems with multimodal models. It also fixes some other issues listed on the below commits.

It contains the following commits:

Fix inputs dict casting in generate (#49731) by @Cyrilvallez
Fix zero expert gradients in bfloat16 with expert parallelism and ep_size < fsdp_size (#49688) by @qgallouedec
Declare the input modalities of the VLMs that report text only (#49677) by @qgallouedec
Avoid a host-to-device copy in eager_mask (#49684) by @hmellor