According to aws.amazon.com, Azercell Telecom LLC, Azerbaijan’s leading telecommunications provider, collaborated with the AWS Generative AI Innovation Center to build an Azerbaijani large language model on Amazon SageMaker AI for telecom use cases and a customer-facing chatbot.
The six-week project established a production-ready framework that delivered significant performance improvements. According to the blog post, the implementation achieved 23% higher training throughput and 58% lower peak GPU memory usage through kernel-level optimizations on an ml.p5.48xlarge instance.
The framework addressed challenges specific to Azerbaijani, described as a morphologically rich language with limited training data. According to aws.amazon.com, the team developed a custom monolingual tokenizer that achieved a 2× improvement in tokens per word, “effectively doubling the amount of Azerbaijani text that fits within the model’s context window” compared to baseline English-optimized tokenizers.
The solution implemented three sequential stages: tokenizer development, continued pre-training using Llama 3.2 1B with distributed training and Liger Kernel optimizations, and a final stage (details not fully provided in the excerpt). According to the source, the project built on open source tools including PyTorch, Hugging Face Transformers, and Liger Kernels, with contributions from multiple AWS team members.