Unfolding Conversational Artificial Intelligence: A Systematic Review of Datasets, Techniques and Challenges in Developments
Abstract
Recently, Artificial Intelligence (AI) has seen significant progress, especially in Natural Language Processing (NLP) and Conversational AI, making response generation more efficient. This advancement, combined with increased availability of conversational data, has greatly improved conversational bots, thus enhancing their effectiveness and scope. Despite extensive research which is primarily focused on task-oriented systems, there's a noticeable lack of comprehensive literature reviews that cover conversational bots, datasets, state-of-the-art methodologies, and thorough analytical insights. The paper presents a detailed study exploring these key dimensions of Conversational AI. It analyzes the datasets, methodologies for crafting conversational bots, and performance metrics, while addressing the various challenges inherent in dialogue systems. Additionally, it suggests viable solutions and provides insights into the future trajectory of conversational bots. Conversational bots are broadly categorized into Retrieval-based and Generative systems. The paper also outlines future avenues for exploration, including advancements in data pre-processing, evaluation techniques, and optimization using advanced methods like Generative Adversarial Networks (GANs), transfer learning, self-supervised, or unsupervised learning. These innovative approaches leverage recent developments in conversational AI, laying a strong foundation for future research in this dynamic field.