LLM2LLM: Boosting LLM with new iterative data augmentation

Content introduction

This paper introduces a breakthrough method for improving the performance of large language models (LLMs) under data scarcity. This technique, called LLM2LLM, utilizes a “teacher” LLM to generate synthetic data based on the errors of a “student” LLM, and then iteratively fine-tune the student model. This approach is of particular interest as it has the potential to significantly enhance the capabilities of LLMs in specialized tasks without the need for extensive and expensive data wrangling. The results presented in the paper show that compared with traditional fine-tuning methods, the performance is improved by up to 52.6% on various data sets. This innovation has the potential to change the way developers and researchers use LLMs in data-constrained environments, making it a fascinating read for those interested in cutting-edge artificial intelligence and machine learning advances.

Automatic summary

– LLM2LLM is an iterative data augmentation strategy for low data situations, using teacher LLM to augment small seed datasets.
– LLM2LLM evaluates and extracts data points where the model is wrong by fine-tuning the baseline student LLM on the initial seed data, and uses the teacher LLM to generate synthetic data based on these wrong data points, which is added back to the training data.
– LLM2LLM significantly improves the performance of LLM in low-data situations, outperforming traditional fine-tuning and other data augmentation methods.
– LLM2LLM reduces reliance on labor-intensive data filtering, paving the way for more scalable and performant LLM solutions.
– Using the LLaMA2-7B student model, in the low data case, LLM2LLM improved by 24.2% on the GSM8K dataset, 32.6% on CaseHOLD, 32.0% on SNIPS, 52.6% on TREC, and It increased by 39.8% on SST-2.

Original link: https://arxiv.org/abs/2403.15042v1