From BERT To GPT-3: Architectural Innovations and Emergent Capabilities in Large Language Models

Main Article Content

Fardeen NB
Sameer NB

Abstract

Large Language Models (LLMs) have revolutionized natural language processing and artificial intelligence, demonstrating remarkable capabilities across diverse tasks. This comprehensive review examines the evolution of LLMs from their architectural foundations to their current implementations, analyzing scaling laws, training methodologies, and evaluation frameworks. We investigate key aspects including few-shot learning capabilities, cross-lingual transfer, domain adaptation techniques, and parameter-efficient fine-tuning methods. Our analysis encompasses critical concerns surrounding LLMs, including computational requirements, environmental impact, social biases, and ethical considerations. Through synthesizing research from over 50 studies published prior to 2023, we provide a holistic perspective on the current state of LLM research and identify promising directions for future investigation. Our findings reveal that while LLMs have demonstrated unprecedented language understanding and generation capabilities, significant challenges remain in areas of alignment with human values, factual accuracy, computational efficiency, and responsible deployment.

Article Details

Section
Articles