Architecting the Future: Scalable Data Infrastructures for Managing and Analyzing IoT Data at Scale

Article Sidebar

pdf
Published: Apr 10, 2024

Main Article Content

Harsh Yadav

Abstract

As the Internet of Things (IoT) continues its exponential growth, the need for robust and scalable data architectures becomes paramount. This research paper explores large-scale data architectures designed to handle the deluge of IoT data effectively. We delve into the intricacies of architecting infrastructures capable of managing, processing, and analyzing vast amounts of sensor-generated data. Theoretical foundations, real-world implementations, and challenges associated with these architectures are scrutinized. Our findings contribute to the evolving discourse on scalable data solutions, offering insights crucial for organizations navigating the complexities of the IoT landscape.

Article Details

How to Cite
Yadav, H. (2024). Architecting the Future: Scalable Data Infrastructures for Managing and Analyzing IoT Data at Scale. International Meridian Journal, 6(6), 1-14. https://meridianjournal.in/index.php/IMJ/article/view/64
Issue
Vol. 6 No. 6 (2024): IMJ
Section
Articles

How to Cite

Yadav, H. (2024). Architecting the Future: Scalable Data Infrastructures for Managing and Analyzing IoT Data at Scale. International Meridian Journal, 6(6), 1-14. https://meridianjournal.in/index.php/IMJ/article/view/64

References

Gray, J., Liu, D. T., Nieto-Santisteban, M., & Szalay, A. (2005). Scientific data management in the coming decade. ACM SIGMOD Record, 34(4), 34-41.

Dean, J., & Ghemawat, S. (2004). MapReduce: Simplified data processing on large clusters. In Proceedings of the 6th conference on Symposium on Opearting Systems Design & Implementation - Volume 6 (pp. 10-10).

Chang, F., Dean, J., Ghemawat, S., Hsieh, W. C., Wallach, D. A., Burrows, M., ... & Gruber, R. E. (2006). Bigtable: A distributed storage system for structured data. ACM Transactions on Computer Systems (TOCS), 26(2), 4.

Ousterhout, J. (2019). A Philosophy of Software Design. CRC Press.

Stonebraker, M., & Çetintemel, U. (2005). "One size fits all": An idea whose time has come and gone. In Data Engineering, 2005. ICDE 2005. Proceedings. 21st International Conference on (pp. 2-11). IEEE.

Abadi, D. J., Madden, S., & Hachem, N. (2008). Column-stores vs. row-stores: How different are they really? In Proceedings of the 2008 ACM SIGMOD international conference on Management of data (pp. 967-980).

Halevy, A., Rajaraman, A., & Ordille, J. (2006). Data integration: The teenage years. In Proceedings of the 32nd international conference on Very large data bases (pp. 9-16).

Stonebraker, M., Brown, P., Zhang, D., & Becla, J. (2011). SciDB: A database management system for applications with complex analytics. Computing in Science & Engineering, 13(4), 22-29.

Ghemawat, S., Gobioff, H., & Leung, S. T. (2003). The Google file system. In ACM SIGOPS operating systems review (Vol. 37, No. 5, pp. 29-43). ACM.

Shvachko, K., Kuang, H., Radia, S., & Chansler, R. (2010). The Hadoop distributed file system. In 2010 IEEE 26th symposium on mass storage systems and technologies (MSST) (pp. 1-10). IEEE.

Abiteboul, S., Buneman, P., & Suciu, D. (1997). Data on the Web: From relations to semistructured data and XML. Morgan Kaufmann.

Bird, S., Klein, E., & Loper, E. (2009). Natural language processing with Python: analyzing text with the natural language toolkit. " O'Reilly Media, Inc.".

Abadi, D. J., Boncz, P. A., & Harizopoulos, S. (2009). Column-oriented database systems. Proceedings of the VLDB Endowment, 2(2), 1664-1665.

Duggan, J., Grannis, S. J., & Perry, T. T. (2017). Privacy, confidentiality, and data access in an era of big data: a report from the NCVHS. Journal of the American Medical Informatics Association, 24(6), 1142-1149.

Cooper, B. F., Silberstein, A., Tam, E., Ramakrishnan, R., & Sears, R. (2010). Benchmarking cloud serving systems with YCSB. In Proceedings of the 1st ACM symposium on Cloud computing (pp. 143-154).

Spector, P. (2008). Data manipulation with R. Springer Science & Business Media.

Abadi, D. J., & Madden, S. R. (2006). Integrating compression and execution in column-oriented database systems. In Proceedings of the 2006 ACM SIGMOD international conference on Management of data (pp. 671-682).

Pinkerton, M., Shneiderman, B., Hertzum, M., & Quigley, J. (2008). With a grain of salt: redesigning salt for the desktop. In Proceedings of Graphics Interface 2008 (pp. 129-136).

He, H., Jagadish, H. V., Ross, K. A., & Zhang, J. (2007). CliqueSquare: Flat plans for mass data warehousing. VLDB Journal, 16(2), 115-139.

Levenshtein, V. I. (1966). Binary codes capable of correcting deletions, insertions, and reversals. Soviet Physics Doklady, 10(8), 707-710.

Whig, P., Yathiraju, N., Modhugu, V. R., & Bhatia, A. B. (2024). 13 Digital Twin for. AI-Driven Digital Twin and Industry 4.0: A Conceptual Framework with Applications, 202.

Whig, P., Battina, D. S., Venkata, S., Bhatia, A. B., & Alkali, Y. J. (2024). Role of Intelligent IoT Applications in Fog Computing. Fog Computing for Intelligent Cloud IoT Systems, 99-118.

Whig, P., Kouser, S., Bhatia, A. B., Purohit, K., & Modhugu, V. R. (2024). 9 Intelligent Control for Energy Management. Microgrid: Design, Optimization, and Applications, 137.

Whig, P., Kouser, I. S., Bhatia, A. B., Nadikattu, I. R. R., & Alkali, Y. J. (2024). 6 IoT Industrial. Wireless Communication Technologies: Roles, Responsibilities, and Impact of IoT, 6G, and Blockchain Practices, 101.

Whig, P., Kasula, B. Y., Bhatia, A. B., Nadikattu, R. R., & Sharma, P. (2024). Digital Twin-Enabled Solution for Smart City Applications. In Transforming Industry using Digital Twin Technology (pp. 259-280). Cham: Springer Nature Switzerland.

Whig, P., Bhatia, A. B., Nadikatu, R. R., Alkali, Y., & Sharma, P. (2024). GIS and Remote Sensing Application for Vegetation Mapping. In Geo-Environmental Hazards using AI-enabled Geospatial Techniques and Earth Observation Systems (pp. 17-39). Cham: Springer Nature Switzerland.

Whig, P., & Kautish, S. (2024). VUCA Leadership Strategies Models for Pre-and Post-pandemic Scenario. In VUCA and Other Analytics in Business Resilience, Part B (pp. 127-152). Emerald Publishing Limited.

Whig, P., Sharma, P., Bhatia, A. B., Nadikattu, R. R., & Bhatia, B. (2024). Machine Learning and its Role in Stock Market Prediction. Deep Learning Tools for Predicting Stock Market Movements, 271-297.

Whig, P., Gera, R., Bhatia, A. B., & Reddy, R. (2024). Convergence of Blockchain and IoT in Healthcare. Convergence of Blockchain and Internet of Things in Healthcare, 277.

Whig, P., Bhatia, A. B., Nadikatu, R. R., Alkali, Y., & Sharma, P. (2024). 3 Security Issues in. Software-Defined Network Frameworks: Security Issues and Use Cases, 34.

Neuromorphic Computing: A Comprehensive Review. Frontiers in Neuroscience, 12, 112-128.

Neha Dhaliwal. (2021). Contributions as a Scrum Master: Facilitating Agile Project Management in Bioinformatics Research with AI. International Journal on Recent and Innovation Trends in Computing and Communication, 9(3), 44–52. Retrieved from https://www.ijritcc.org/index.php/ijritcc/article/view/10671

Priyanka Koushik, S. M. (2024). Elevating Customer Experiences and Maximizing Profits with Predictable Stockout Prevention Modelling. International Journal of Intelligent Systems and Applications in Engineering, 12(21s), 1171–1178. Retrieved from https://ijisae.org/index.php/IJISAE/article/view/5547

Sumit Mittal, "Framework for Optimized Sales and Inventory Control: A Comprehensive Approach for Intelligent Order Management Application," International Journal of Computer Trends and Technology, vol. 72, no. 3, pp. 61-65, 2024. Crossref, https://doi.org/10.14445/22312803/IJCTT-V72I3P109