A new family of large-scale multimodal models called ERNIE 4.5 has been introduced, featuring 10 distinct variants designed to handle various AI tasks. The announcement marks a significant expansion in the company’s artificial intelligence capabilities.
The ERNIE 4.5 family includes several model architectures, with the most notable being Mixture-of-Experts (MoE) models. These specialized AI systems are designed to process both text and visual information, making them versatile tools for multiple applications.
Technical Specifications
The ERNIE 4.5 lineup features models with varying parameter counts, which determine their computational capacity and potential capabilities. The family includes:
- MoE models with 47 billion active parameters
- MoE models with 3 billion active parameters
- A dense model with 0.3 billion parameters
The largest model in the family contains an impressive 424 billion total parameters, though only a portion of these are activated during any specific task. This architecture allows for greater efficiency while maintaining powerful capabilities.
Mixture-of-Experts Architecture
The Mixture-of-Experts approach used in most ERNIE 4.5 models represents an advanced AI architecture that differs from traditional dense models. In MoE systems, the neural network contains multiple “expert” components that specialize in different tasks or types of data.
When processing information, the system activates only the experts needed for a particular task rather than using the entire network. This selective activation explains the distinction between “active” parameters and “total” parameters in the model specifications.
“The MoE architecture allows us to build much larger models that are still computationally efficient,” a researcher familiar with the technology explained. “By only activating a small portion of the network for each task, we can have models with hundreds of billions of parameters that run with the resource requirements of much smaller systems.”
Multimodal Capabilities
As multimodal models, the ERNIE 4.5 family can process and understand multiple types of information, including text, images, and potentially other data formats. This capability makes them suitable for applications ranging from content creation to complex data analysis.
The smallest model in the family, with 0.3 billion parameters, uses a traditional dense architecture where all parameters are active during processing. While less powerful than its larger siblings, this model may be more suitable for applications with limited computational resources or where lower latency is critical.
The introduction of ERNIE 4.5 follows industry trends toward larger, more capable AI systems that can handle increasingly complex tasks. The model family appears positioned to compete with other major AI systems in the rapidly evolving field of artificial intelligence.
As organizations continue to explore the capabilities of these new models, researchers and developers will likely discover new applications and use cases that take advantage of ERNIE 4.5’s multimodal abilities and varied model sizes.
