Large language models: A comprehensive survey of architectures, applications and challenges

Abstract

Large Language Models (LLMs) have emerged as a foundational technology in contemporary artificial intelligence, driven largely by the scalability and expressiveness of the Transformer architecture. This survey provides a structured and comprehensive review of recent advances in LLM research across five key dimensions: (i) scaling laws and foundational models, (ii) sparse and efficiency-oriented architectures, (iii) multimodal and tool-augmented models, (iv) privacy, alignment, and safety mechanisms, and (v) domain-specific applications and existing surveys. The reviewed literature indicates that while large-scale model training yields consistent performance gains and emergent capabilities, it also introduces significant challenges related to computational cost, energy consumption, interpretability, and environmental sustainability. Efficiency-driven approaches, including sparse attention mechanisms and Mixture-of-Experts architectures, mitigate some of these constraints but increase system complexity. Multimodal and tool-enhanced LLMs extend reasoning capabilities beyond text; however, they remain susceptible to hallucinations and grounding errors. Furthermore, alignment and privacy-preserving methods are actively evolving, yet the absence of standardized evaluation frameworks and clearly defined utility – safety trade-offs limit their practical deployment. Empirical studies in application domains such as healthcare and education demonstrate substantial promise but emphasize the necessity of trust, explainability, and regulatory compliance. This survey synthesizes current progress, identifies persistent research gaps, and outlines future directions toward the development of efficient, reliable, and responsible large language models.