Amazon Web Services has added full fine-tuning (FFT) support to SageMaker AI Serverless Model Customization, the company announced. The service allows users to fine-tune large language models by updating all parameters simultaneously without provisioning GPUs, setting up clusters, or configuring distributed training.
AWS says full fine-tuning addresses limitations of parameter-efficient methods like Low-Rank Adaptation (LoRA), which update only a subset of parameters. According to the company, LoRA’s low-rank structure can limit representation power and compromise performance on complex tasks. Full fine-tuning updates model parameters simultaneously, which AWS says enables models to better absorb complex domain-specific knowledge.
The company positions fine-tuning as delivering better accuracy on domain-specific work while reducing costs, noting that a smaller specialized model can match or exceed a frontier model on specific tasks at lower per-token inference prices.
AWS says its announcement includes practical hyperparameter tuning guidelines backed by controlled experiments and an examination of spectral properties of model weight matrices that the company says explain quality differences between full fine-tuning and LoRA.
The service is now available through Amazon SageMaker AI. Further details are available at https://aws.amazon.com/blogs/machine-learning/full-fine-tuning-fft-on-sagemaker-ai-serverless-model-customization.