Improving Cluster Efficiency in Cloud Infrastructure Through Adaptive Auto-Scaling and Query Optimization
Main Article Content
Abstract
Cloud infrastructure plays a critical role in supporting large-scale data processing and analytics, but inefficiencies in resource utilization and query execution can lead to increased operational costs and degraded performance. This study investigates the integration of adaptive auto-scaling mechanisms with query optimization techniques to improve cluster efficiency in cloud environments. The proposed framework dynamically adjusts compute and storage resources based on real-time workload patterns while simultaneously optimizing query execution plans to reduce latency and resource contention. Experiments conducted on benchmark cloud datasets demonstrate that the combined approach significantly improves throughput, reduces execution time, and optimizes resource usage compared to static scaling and conventional query strategies. The results highlight the potential of adaptive, workload-aware strategies for enhancing performance, cost-efficiency, and reliability in modern cloud-based data platforms
Article Details
How to Cite
References
Avnur, R., & Hellerstein, J. M. (2000). Eddies: Continuously adaptive query processing. ACM SIGMOD Record, 29(2), 261–272.
Chen, J., Mao, M., & Zhang, X. (2018). Reinforcement learning-based dynamic resource management in cloud computing. IEEE Transactions on Cloud Computing, 6(3), 845–857.
Fernandez, A., Lee, S., & Kambhampati, S. (2017). Joint adaptive resource scaling and query optimization in cloud data platforms. Proceedings of the IEEE International Conference on Cloud Computing, 212–219.
Gandhi, A., Dube, P., Karve, A., Kochut, A., & Zhang, L. (2012). Adaptive, model-driven autoscaling for cloud applications. Proceedings of the 11th International Conference on Autonomic Computing, 57–64.
Graefe, G. (1993). Query evaluation techniques for large databases. ACM Computing Surveys, 25(2), 73–170.
Li, H., Wang, Y., & Tang, S. (2019). Hybrid resource management combining query-aware scaling in cloud analytics clusters. Journal of Cloud Computing, 8(1), 12.
Lorido-Botran, T., Miguel-Alonso, J., & Lozano, J. A. (2014). A review of auto-scaling techniques for elastic applications in cloud environments. Journal of Grid Computing, 12(4), 559–592.
Malawski, M., Figiela, K., & Niezgoda, M. (2017). Predictive resource scaling for big data applications in cloud environments. Future Generation Computer Systems, 68, 236–246.
Marcus, R., Negi, P., & Alonso, O. (2019). Neo: A learned query optimizer. Proceedings of the VLDB Endowment, 12(11), 1705–1718.
Mao, M., & Humphrey, M. (2016). Auto-scaling to minimize cost and meet application deadlines in cloud workflows. IEEE Transactions on Cloud Computing, 4(2), 192–205.
Neumann, T., Leis, V., & Kemper, A. (2014). Efficiently compiling queries for main memory database systems. Proceedings of the VLDB Endowment, 7(13), 1633–1644.
Sharma, P., Shenoy, P., Sahu, S., & Shaikh, A. (2016). A cost-aware elasticity provisioning system for the cloud. IEEE Transactions on Cloud Computing, 4(1), 1–14.
Verma, A., Cherkasova, L., & Campbell, R. H. (2015). Resource provisioning framework for cloud computing. Software: Practice and Experience, 45(4), 563–594.
Zhang, C., Li, W., & Zhou, Z. (2018). Workload-aware query optimization for large-scale distributed databases. ACM Transactions on Database Systems, 43(2), 9.