Armen Atayan, an Armenian programmer based in the diaspora, has unveiled HyGPT – the first high-quality large language model (LLM) specifically designed for the Armenian language. The model is freely accessible online, aiming to bring cutting-edge AI technology to Armenian-speaking communities worldwide.
HyGPT is the first advanced LLM tailored for Eastern Armenian, and its creation marks a significant step toward the digital empowerment of the Armenian language. The project’s primary goal was to develop and train a modern language model that functions natively in Armenian, helping bridge the gap between Armenian speakers and the rapidly evolving field of artificial intelligence.
The project was developed through several key phases. Initially, a suitable base model was selected. This was followed by the careful compilation and curation of a large-scale and representative Armenian-language corpus. The model then underwent a pre-training process to ensure efficient understanding and generation in Eastern Armenian. Instruction-based fine-tuning was also applied, significantly enhancing the model’s conversational capabilities. Several innovative techniques were implemented to ensure the model’s quality and relevance.
Despite recent global progress in AI and language modeling, Armenian has remained largely underserved. This has posed a major barrier for Armenian users wishing to engage with the latest AI tools in their native language.
“The creation of HyGPT is a vital step in preserving the digital identity of our language and culture,” said Armen Atayan. “Armenian deserves state-of-the-art AI tools that can serve education, business, and the creative industries. I’m proud we managed to build the first high-quality large language model specifically adapted to the nuances of our language. This project opens a unique opportunity for Armenian speakers to embrace modern technologies and build systems in Armenian.”
The training dataset for HyGPT included a wide array of Armenian-language content – news articles, curated web content, Wikipedia entries, literary texts, scientific papers, and other publicly available sources. Special credit was given to Artak Hovsepyan, whose collected materials were essential to the completeness of the model.
Armen Atayan spent his childhood in Armenia and currently resides in Kazakhstan, where he founded Gen2B.ai, a company focused on applying AI technologies to business. Under his leadership, Kazakhstan’s first Kazakh-language LLM – Irbis GPT – was also developed, receiving significant attention in local media. Atayan now plans to develop similar models for Luxembourgish and Uzbek in the near future.
He views the creation of HyGPT as a personal contribution to Armenia’s AI future.
“This is my way of giving back to Armenian culture, language, and society,” he said. “They’ve played a crucial role in shaping who I am — both personally and professionally. I hope HyGPT becomes the foundation for many new Armenian digital innovations.”
HyGPT-10b-it, the first released version of the model, was developed over six months, with three of those months devoted entirely to intensive training. According to Atayan, the model has achieved outstanding results, outperforming some larger models in factual accuracy (TruthfulQA) and mathematical reasoning (GSM8K).
To further boost accuracy, the team implemented a key architectural improvement by separating the embedding and output layers—a novel technique that contributed to higher performance. Atayan also credited the project’s success to collaboration with Artak Hovsepyan of NCCAIT in data collection, and support from the Nvidia Inception Program for access to essential computational infrastructure.
“We believe in the power of open-source and have made HyGPT available for everyone,” Atayan wrote in a LinkedIn post. “We are excited to see what the community will build with HyGPT.”
Download the model on Hugging Face: https://huggingface.co/Gen2B/HyGPT-10b-it
