Several technical papers published on arXiv in early August 2026 present advances in AI model training and optimization methods.
According to arxiv.org, the DiffusionGemma Team released a technical report on DiffusionGemma on August 5, 2026. The report includes contributions from 20 named researchers including Adrien Ali Taïga, James Assiene, Daniele Calandriello, and others.
In a separate paper also published August 5, 2026, researchers described a practical training recipe for normalized Transformer (nGPT). According to the abstract on arxiv.org, nGPT “realizes hyperspherical representation learning by constraining model parameter vectors and activation vectors to the unit hypersphere.” The training recipe introduces several techniques including Logit Gradient Preconditioning, Logarithmic Learning Rate Decay, and GatedAdamW. The paper reports that a 14B-total-parameter nGPT model “reaches the same validation loss using approximately half as many training tokens” compared to an unnormalized model trained with AdamW.
A third paper examines the Muon optimizer for matrix factorization tasks. According to arxiv.org, the researchers “find that Muon does not consistently outperform AdamW in this setting and that several previously reported advantages are sensitive to hyperparameter choices.”
Finally, researchers introduced ReBRAC-v2, an offline reinforcement learning method. According to arxiv.org, ReBRAC-v2 “averages 74.8 compared to 52.3 for the next-best aggregate result” across ten OGBench categories and “ranks first in eight categories.”