A web-based voice interaction framework proposal for enhancing information systems user experience

Abstract

Nowadays, numerous organisations of different dimensions and business sectors operate in highly challenging and dynamic environments, wherein the supporting information systems (IS) are becoming increasingly complex. In this context, assistive tools capable of tackling such complexity have the potential to aid users improving their performance and effectiveness, as well as to streamline businesses' processes and promote entrepreneurial-level competitivity. Following this line of research, a web-based speech-to-term recognition approach is presented as a solution to endow IS with advanced capabilities for providing an easier (more natural) and straightforward interaction with baseline functionalities, by combining relevant techniques such as Voice Activity Detection (VAD), Automatic Speech Recognition (ASR), Natural Language Processing (NLP), and an Ontological Database (OD), mapping the IS' functionalities and characteristics, is proposed. The developed interoperable system allows the conversion of speech to text - deriving into IS instructions - that is, in turn, submitted to on an ontological database wherein a term-based query is performed to elicit a set of available commands to be executed in the web context. These commands, fully mapped in the ontological database, are divided into three categories: a) navigation by menus/links, b) buttons interaction (e.g., submit forms) and c) completion of form fields. The proposed framework was experimentally tested in close to real conditions, resorting to an Enterprise Resource Planning (ERP) tool supplied by ERP Company.