BTQ-MR: Performance Evaluation of Map Reduce Programme Using Transient Queuing Model with Finite Buffer

Abstract

This paper introduces a transient queueing model with a finite buffer (BTQ-MR) to evaluate the performance of the MapReduce programming model. While earlier research has explored analytical queueing models, many have missed two key aspects: (i) time-dependent workload variations and (ii) the impact of finite buffers for incoming user requests. Buffer size is crucial as it affects scheduling policies, system performance, and resource usage. The BTQ-MR model aims to address these aspects. The study examines the behavior of the BTQ-MR model under different conditions, including job arrival rates, scheduler allocation times, and mapper and reducer completion times. The model includes three service stages which are scheduling, mapper, shuffle, and reducer—with no waiting between them. Using transient differential equations, the study evaluates performance metrics such as average queue length, waiting time, blocking probabilities of mappers, and waiting probabilities in the shuffle phase. MATLAB simulations are used to analyze workload intensity, buffer size, and job completion times of various stages. Increasing buffer size reduces job blocking but may raise resource usage. Changes in arrival rates and completion times highlight the need for adaptive scheduling to ensure stability. The study emphasizes the importance of tuning parameters like the number of mappers and reducers to balance resource use and minimize completion times. The results provide insights into optimizing MapReduce systems to handle large-scale, dynamic workloads effectively. In summary, the BTQ-MR model addresses limitations of earlier approaches by considering finite buffers and time-dependent variations. It offers a practical method to analyze and improve MapReduce performance, laying the groundwork for further research and development.