Apr 25, 2024
Lately Apple have introduced eight open source language models, the OpenELM models (Open-source Efficient Language Models). What makes them special is that they run directly on the device and not on cloud servers. And in this short guide, we will show you how to run and use them.
Apple have introduced eight open-source language models known as OpenELM (Open-source Efficient Language Models). Unique for their ability to operate directly on devices rather than relying on cloud servers, these models mark a significant advancement in AI technology. This guide will show you how to set up and use these innovative Apple AI models.
Apple's Efficient Language Models
Developers now have access to these large language models, which can be easily downloaded and implemented through the Hugging Face Hub. Notably, four of the OpenELM models were trained with the CoreNet library, a resource also launched by Apple for the training of deep neural networks.
The other four models (Instruct) are designed as instructional tools, capable of interpreting and responding to direct instructions. This full suite of models, along with comprehensive training and evaluation frameworks, is available on public datasets. These include detailed training protocols, various checkpoints, and diverse pre-training configurations.
The OpenELM family includes several models tailored to different needs. Please click to read more details about models.
Running OpenELM via HuggingFace
Install
To help you get started, we've provided a sample function in generate_openelm.py for generating output from OpenELM models via the Hugging Face Hub. To test the model, simply run the following command:
For access to your Hugging Face token, please follow this link.
Additionally, you can customize the generate function with various arguments. For instance, to enhance inference speed, consider using the prompt_lookup_num_tokens argument for lookup token speculative generation.
Or, for an assistive model, use a smaller model via the assistant_model argument as shown below:
Setting up
Make sure to install the necessary dependencies:
Evaluation of OpenELM
Conclusion: Considerations for Using OpenELM Models
The introduction of OpenELM models by Apple marks a significant advancement, offering the research community cutting-edge language models. These tools are trained on publicly available datasets and are provided without guarantees of safety. This may lead to outputs that could be inaccurate, harmful, or biased. Therefore, it is crucial for both users and developers to conduct extensive safety tests and establish robust filtering mechanisms to meet their unique needs and ensure responsible usage.
