Improvising diarization for multilingual communication

Abstract

Nowadays, many people use virtual meeting platforms like Google Meet and Microsoft Teams to communicate. In these meetings, it's common for participants to switch between different languages, such as Hindi, English, and Bengali while communicating. However, the current automatic speech recognition (ASR) systems and summarization tools need to work better with these multilingual conversations. They often miss important words or produce inaccurate transcribes because they are not designed to handle multiple languages simultaneously. The main problem is that most ASR systems are built to recognize only one language at a time or struggle when people switch between languages during a conversation. This makes it hard to get accurate transcripts from these meetings, leading to misunderstandings or loss of important information. This thesis focuses on improving automatic speech recognition (ASR) and transcription systems in virtual meetings for Hindi, English, and Bengali languages. The goal is to create a system that can better understand and transcribe conversations where people use multiple languages, making virtual meetings more effective and accessible for everyone. i