P-28

A Scalable Architecture for Data Storage and Retrieval in Real-time Data Assimilation in Meteorology

Mayumbo Nyirenda (Graduate School of Information Science and Technology, Hokkaido University), Kimihito Ito (Research Center for Zoonosis Control, Hokkaido University) and Hiroki Arimura (Graduate School of Information Science and Technology, Hokkaido University)

Data assimilation is a statistical method by which actual observations are integrated into computer simulations. So far, data assimilation has been applied in meteorology, achieving significant improvements in the accuracy of predictions. In the meantime, advancements in data assimilation coupled with big data have brought new challenges. Real-time storage and retrieval of massive remote sensing data is a major computational bottleneck in data assimilation systems in meteorology. In order to solve this problem, we propose the use of the distributed quadtree data structure for storing and retrieving real-time observation data in a massively parallel data assimilation in meteorology. We conducted a scale-down parallel data insertion and retrieval experiment using ten single-board computers. The results show that average response time improves as the data is distributed across more servers. In addition the solution scales well. We anticipate that use of a parallelized database with spatial indexing functions can enhance the throughput of data assimilation tasks in meteorology and oceanography