Jeff Dean's investment company releases Transformer pre-training results without backpropagation

Transformer models can be pre-trained without backpropagation, Jeff Dean's investment company achieves results. This technology breaks the limitation of traditional neural network training relying on backpropagation and may affect the future architecture and optimizer design of large models.