Multi-agent system for multimodal machine learning object detection

Abstract

Multi-agent systems have shown great promise in addressing complex problems that traditional single-agent approaches are not be able to handle. In this article, we propose a multi-agent system for the conception of a multimodal machine learning problem on edge devices. Our architecture leverages docker containers to encapsulate knowledge in the form of models and processes, enabling easy management of the system. Communication between agents is facilitated by Message Queuing Telemetry Transport, a lightweight messaging protocol ideal for Internet of Things and edge computing environments. Additionally, we highlight the significance of object detection in our proposed system, which is a crucial component of many multimodal machine learning tasks, by enabling the identification and localization of objects within diverse data modalities. In this manuscript an overall architecture description is performed, discussing the role of each agent and the communication protocol between them. The proposed system offers a general approach to multimodal machine learning problems on edge devices, demonstrating the advantages of multi-agent systems in handling complex and dynamic environments.