Handwritten Text Generation with Vision Transformers and Diffusion Models
Main Article Content
Abstract
In Handwritten text synthesis is crucial for augmenting data in handwritten text recognition (HTR) across languages, especially those with complex diacritics and limited samples. We propose WriteViT 2.0, a novel framework that combines Vision Transformers (ViT) with conditional diffusion models to generate high-quality, diverse, and style-faithful handwritten text images. Our multi-scale architecture integrates hierarchical transformer blocks with Conditional Positional Encoding and leverages self-supervised pretrained style embeddings for effective style-content disentanglement. Paired with a ViT-based recognizer and writer identifier, WriteViT 2.0 advances the state of the art in handwriting synthesis. Experiments on IAM, a newly introduced multilingual dataset demonstrate superior performance in visual quality, style consistency, and HTR accuracy compared to GAN and transformer baselines. This work lays a foundation for personalized handwriting synthesis and improved low-resource handwriting recognition.