Object detection with artificial vision and neural networks for service robots

Abstract

This dissertation arises from a major project that consists on developing a domestic service robot, named CHARMIE (Collaborative Home Assistant Robot by Minho Industrial Electronics), to cooperate and help on domestic tasks. In general, the project aims to implement artificial intelligence in the whole robot. The main contribution of this dissertation is the development of the vision system, with artificial intelligence, to classify and detect, in real time, the objects represented on the environment that the robot is placed. This dissertation is within two broad areas that revolutionized the robotics industry, namely the artificial vision and artificial intelligence. Knowing that most of the existent information is presented on the vision and with the evolution of robotics, there was a need to introduce the capacity to acquire and process this kind of information. So, the artificial vision algorithms allowed them to acquire information of the environment, namely patterns, objects, formats, through vision sensors (cameras). Although implementing artificial vision can be very complex if it is intended to detect objects, due to image complexity. The introduction of artificial intelligence, more precisely, deep learning, brought the capability of implementing systems that can learn from provided data, without the need of hard coding it, reducing slightly the complexity and the time consumption of implementing complex problems. For artificial vision problems, like this project, there is a deep neural network that is specialized in learning from three dimensional vectors, namely images, named Convolutional Neural Network (CNN). This network uses image data to learn patterns, edges, formats, and many more, that represents a certain object. This type of network is used to classify and detect the objects presented in the image provided by the camera and is implemented with the Tensorflow library. All the image acquisition from the camera is performed by the OpenCv library. At the end of the dissertation, a model that allows real-time detection of objects from camera images is provided.