While visiting Cyprus, our European Engineering team held an in-person AI Hack Lab, gathering ML engineers and AI developers to explore the current state and future of the emerging AI sector. This blog recaps the talks and links to their video replays.

Transforming the AI World (with transformers)

First up, Tanya Dadasheva and Roman Shaposhnik, the founders of AIFoundry.org, captivated the audience with their insightful presentation on the future of AI. They began the session by introducing themselves and interacting with the attendees to gauge their knowledge of machine learning and AI. This interactive approach set a lively tone, encouraging participation and creating a dynamic atmosphere.

The discussion quickly delved into AI's current state, highlighting technologies' transformative impact. The founders reflected on the challenges faced by engineers and investors in keeping pace with the rapidly evolving AI landscape. They underscored the need for adaptive strategies to navigate these advancements effectively.

Providing a historical context, the founders traced the evolution of AI from its roots in statistical methods to the sophisticated neural networks we see today. They acknowledged the staggering complexity of modern AI technologies and the necessity for robust frameworks to manage this complexity. Projects like TVM were highlighted for their efforts to standardize and streamline the industry.

One of the most compelling segments of the talk was the discussion on Transformers, a significant breakthrough in AI architecture. The founders praised the simplicity and versatility of Transformers, showcasing their application in diverse fields such as brain wave analysis and image generation. This led to a broader conversation on innovation in AI hardware and software, including the development of specialized ASICs and the impact of community-driven projects like llama.cpp.

As the session progressed, the importance of open AI models was emphasized. The founders discussed efforts to create and maintain open models, highlighting contributions from the Allen Institute and the BigScience project. They drew parallels between the current state of AI and the early days of Linux, arguing for the establishment of a Linux-like foundation to support AI's growth and standardization.

The event concluded with a call to action, inviting attendees to join the community and participate in further discussions. The founders stressed the importance of collaboration in advancing AI technology and achieving industry-wide standardization. For those who missed the live event, a video recording is available, offering a deeper dive into these transformative insights.

Why Use Rust for Generative AI Projects

Next up, Federico Rampazzo, a consultant at API Plant LTD, discussed the advantages of using Rust for AI development. He emphasized Rust's strong static typing, which reduces bugs, and its ownership system that enhances code safety. Rampazzo also highlighted Rust's straightforward development process, fearless concurrency, and excellent tooling, making it a compelling choice for AI projects. His personal experiences include successfully executing two generative AI projects in Rust, one using Retrieval Augmented Generation (RAG), alongside work in Python and JavaScript/Python for AI, showcasing Rust's versatility and strength in AI development.

Rampazzo also covered the importance of Panda tools in model inference and mentioned Candle and BURN as powerful tools for model training and inference. He provided an overview of the various models available for AI training, giving the audience a clear understanding of the options they can utilize in their own AI development efforts.

Quantizing LLMs to Run on Smaller Systems with Llama.cpp

Yulia Yakovleva, Head of AI Research at Nekko.ai, discussed running large language models (LLMs) on personal machines using llamacpp and llamafile. She explained that before 2023, AI models needed multiple GPUs, making them complex and resource-intensive. However, the open-source release of LLaMA marked a significant shift, allowing for more accessible and efficient AI model operation.

Yulia highlighted Georgi Gerganov's pivotal contributions in 2023, which enabled running LLMs with greater efficiency. She detailed the advancements in memory management that have simplified AI model performance, emphasizing the technical breakthroughs that have transformed the AI landscape.

Meet Llamagator: One Chat Interface for Many LLMs

Chris Hasinski from the AIFoundry hacker team showcased Llamagator, an innovative web application for managing large language models (LLMs). Through a live demo, Hasinski demonstrated how users can interact with Llamagator to manage and test various AI models using different prompts. The demo generated significant interest, prompting many attendees to join AIFoundry's Discord server for further discussions and collaboration.

Llamagator, built on Ruby on Rails 7.1.3 and PostgreSQL 14, provides a seamless platform for creating, managing, and versioning AI models. It offers robust configuration management tools, allowing users to fine-tune models to meet specific performance goals. Additionally, Llamagator facilitates prompt creation and execution, enabling easy testing and deployment of AI solutions. A standout feature is its ability to evaluate and compare prompt execution results, providing valuable insights for model optimization.

BlameAI: Using AI to Ferret Out Problems from Hit History

Sergey Sergyenko, VP of Engineering at Nekko.ai, also gave a taste to the crowd about Git blame, and how a tool can learn from git commands plus to hold discussions and create a new model BlameAI.

Meet the GUT-AI Foundation

Ioannis Kourouklides, co-founder of Gut-ai.org, highlighted the challenges many AI startups face in developing essential AI components. To support these startups, Gut-ai.org was created to provide open-source AI tools and resources.

Kourouklides shared the foundation's ambitious roadmap, which includes securing funding by the end of 2024 to sustain their mission of democratizing AI technology and fostering innovation. Through Gut-ai.org, Kourouklides aims to create an ecosystem where startups can thrive using open-source AI solutions.

The event offered valuable insights into the latest AI developments and fostered community among AI professionals. Key discussions highlighted the importance of open-source initiatives, the potential of Rust, and recent advancements in AI model efficiency. As AI evolves, such events are crucial for bringing experts together to share knowledge and drive innovation.

Join an AIFoundry.org Event

AIFoundry.org runs multiple events per month, both in person and virtually online.  If you would like to join in one of our upcoming events, please subscribe to our calendar.