Last Monday, the AIFoundry.org team joined the Mozilla.ai Discord Server to present the results of our first open investigation: how to package the OLMo LLM with Llamafile. The Llamafile Project is supported under the Mozilla.ai initiative of the Mozilla Foundation.
Chris Hasiński, a developer with the AIFoundry.org initiative, presented first, starting with an introduction to OLMo. OLMo, created by the Allen Institute for AI, is short for Open Language Model. It's different from other models in that it is entirely open:
All these are liberally licensed under an Apache V2 license.
As compared to Llama2:
Chris then goes into detail about each of the open components of OLMo and where to find them:
“4.5 TB of filtered, de-duplicated data taken from a mix of web content, academic publications, code,
books, and encyclopedic materials.”
- Open tooling: https://github.com/allenai/dolma
All the tooling that was used for creating the data set
- Open weights with checkpoints: https://huggingface.co/allenai/OLMo-7B-hf
- Evaluation Suite including
- Data used for evaluating models: https://huggingface.co/datasets/allenai/paloma
-
- Tooling used for evaluation: https://github.com/allenai/OLMo-Eval
Chris then shifted to our work of packaging OLMo into Llamafile, as a step we took since AIFoundry.org is working on tools that allow comparison of different LLMs. We had decided to standardize on Llamafile since it simplifies packaging of LLMs into a single file.
We shared links of two flavors of our OLMo packaging:
- OLMo in a Llamafile
https://huggingface.co/aifoundry-org/OLMo-1.7-7B-hf.llamafile
- OLMo with Llama.cpp inside a Docker container
https://github.com/aifoundry-org/olmo-llamafile
We then ran a short demo:
Which quickly brings up the UI for Llama.cpp:
Chris then demoed some of the comparison tooling the team is working on.
You can watch the video above or catch a more extended demo at our upcoming virtual AI Hack Lab on Saturday, Jul 13, 2024.
This summarizes the first 15 minutes of the session. The last 15 minutes consist of a very informative Q&A, which I highly recommend you check out.
If you’d like to participate in one of our upcoming Live Podcasts, please subscribe to the AIFoundry.org Calendar on Luma and join the AIFOundry.org Discord server, where we run these as live interactive events.