Emergent Complex Reasoning Through Compositional Data Augmentation: How Synthetic Task Diversity Shapes Cognitive Capabilities in Large Language Models

Main Article Content

Fardeen NB
Sameer NB

Abstract

This paper investigates how the composition and distribution of synthetic training data affects emergent reasoning capabilities in large language models (LLMs). We hypothesize that targeted compositional data augmentation strategies lead to significant improvements in complex reasoning tasks, even when model scale is held constant. Through systematic experiments, we demonstrate that diverse synthetic data focused on compositional reasoning skills—compared to uniform or single-task focused augmentation—produces non-linear improvements in multi-hop reasoning, logical consistency, and uncertainty calibration. Our results suggest that synthetic data composition is a key determinant of reasoning emergence, sometimes outweighing the effects of model scale alone. We introduce the Compositional Reasoning Assessment Benchmark (CRAB), a diagnostic suite of tasks requiring different forms of multi-step reasoning. Our findings provide insights into the mechanisms of reasoning in LLMs and suggest practical strategies for enhancing model capabilities without increasing parameter count. The methodologies and synthetic data generation approaches presented here offer a reproducible framework for investigating how data shapes higher-order cognitive abilities in language models.

Article Details

Section
Articles