The development of language models in Artificial Intelligence (AI) has made impressive strides in recent years. Models such as BERT (Bidirectional Encoder Representations from Transformers) and GPT-3 (Generative Pre-trained Transformer 3) have stood out for their ability to process and generate natural language. These models often contain billions of parameters, which increases their performance but also leads to higher computational costs. These costs are particularly challenging for real-time applications of the models.
UltraFastBERT: A Revolution in Language Modeling
In response to the computational cost challenges, researchers from ETH Zurich have developed an innovative solution: UltraFastBERT. This model is based on the BERT architecture but makes significant changes to boost efficiency. The key idea behind UltraFastBERT is the reduction of active neurons in the “feedforward layers” during inference. Instead of using all neurons, UltraFastBERT activates only 0.03% (12 out of 4095 neurons) in each “feedforward layer”.
Optimization of “Feedforward Layers”
Language models like BERT and GPT-3 predominantly use “feedforward layers” for their computations. These layers contain the majority of the model’s parameters. Interestingly, not every neuron in these layers is required for inference, which is the process of processing new inputs and making predictions. This insight allowed the researchers to replace the “feedforward layers” with “fast feedforward networks”, thereby achieving a significant increase in computational efficiency.
The Role of Inference in Language Models
Inference refers to the process where the trained model analyzes new inputs and makes predictions based on them. In language models, this involves interpreting texts and generating responses or summaries. The speed and efficiency of inference are crucial for the practical applicability of language models, especially in real-time applications.
Performance Comparison and Application Potential
UltraFastBERT was tested using the GLUE benchmark for language understanding and achieved up to 96% of the performance of the original BERT model, despite the drastically reduced number of neurons. This performance is particularly significant for large models like GPT-3, where the number of neurons per inference could theoretically be reduced to just 0.03%.
Challenges and Future Prospects
However, implementing UltraFastBERT also poses certain challenges. The technique of Conditional Matrix Multiplication (CMM), which plays a key role in efficiency improvement, is not straightforward to implement as it builds on knowledge that is not freely accessible. Nevertheless, the researchers achieved a 78-fold acceleration over the optimized baseline “feedforward” implementation.
In summary, UltraFastBERT represents a significant advancement in the development of efficient AI language models. It shows the potential to reduce computational costs and increase speed without compromising performance. This opens new possibilities for the application of language models in real-time scenarios and could mark a turning point in how we use AI in language processing.
Additional information:
Git Hub Account UltraFastBERT: https://github.com/pbelcak/UltraFastBERT
Belcak, P., Wattenhofer, R. (2023). Exponentially Faster Language Modelling. https://arxiv.org/abs/2311.10770