Bengali.AI: Democratising AI research in Bangla

Have you ever tried to translate a line or paragraph in Bangla on Google Translate?

You were most likely frustrated as the system cannot translate Bangla well. In fact, some of the translations end up being so hilarious that it can crack you up.

Google's artificial intelligence clearly cannot understand the nuances and quirks of the Bangla language properly. When it comes to Google's voice keyboard, it is a bit of a hit or miss. But when you try it in English, it works just flawlessly.

Bangla is tougher to incorporate into these types of systems partly because it has many dialects which are quite different from standard Bangla. 'Dhakaiya,' 'Chatgaiya,' 'Sylheti,' 'Barishailla'— all these dialects have their very own distinctive tones. Also, Google simply does not have enough voice data to make a perfect voice-to-text system in Bangla. 

Bengali.AI, a non-profit organisation consisting of a large number of researchers and volunteers from different institutions at home and abroad, have stepped in to address this issue.

In December 2017, a group of students who just completed their bachelor's from different universities in Bangladesh started a passion project and named it Bengali.AI. The vision was clear, as the name suggests, to push AI research in the Bangla language. 

Ahmed Imtiaz Humayun, along with some of his peers, started the project with a dream of democratising Bangla AI research.

The founder of Bengali.AI said, "A lot is happening in the AI research arena. But almost all of them are in English. AI research in Bangla was almost nonexistent. My dream was to create a platform for those who dream. For those who will work passionately for AI research in Bangla."

The journey

In early 2018, Bengali.AI launched a project named NumtaDB, an optical character recognition (OCR) model which could detect Bangla numerical characters. For the project, they collected over 85,000 samples of handwritten numbers from over 2,700 people. The same year, they launched an OCR competition on Kaggle in collaboration with Google.

The following year, the organisation collaborated with Google once again and launched another competition on Bangla Graphemes (linguistic segments of word formation). They were able to pull off a total of 7.5 million hours of research work by the data scientists from companies like Google, Nvidia, H2O.AI, etc.

If you are familiar with Murad Takla texts (transliterated Bangla texts), well, in 2020 they worked on a project where the transliterated texts could be transcribed. The same year, they started another project which takes a graphemic approach to Bangla handwritten OCR.

The next year, a team of 40 students from different universities, voluntary researchers from both academia and industry and voluntary annotators launched a GPU-based algorithm which can detect typing mistakes in Bangla texts even to a high degree. This works like Grammarly (an American cloud-based typing assistant that reviews spelling, grammar, punctuation, clarity, etc and mistakes in English language text) but in Bangla. This spell-checker application is on its way to being launched for public use.

Current project: Voice recognition in Bangla

At the beginning of this year, they started an even larger project, "Bangla Speech Recognition." The purpose is to make machines understand the Bangla language. For this, they run a campaign on social media named "Bok Bok Campaign." 

Illustration: TBS

In this campaign, Bangla-speaking people from all over the world donated their voice data to enrich the voice dataset of Bengali.AI. 

Mozilla Commonvoice is the platform where they are hosting the campaign. They have already managed to accumulate a total of around 2,000 hours of voice data from more than 23,000 voluntary voice donors. Their goal is much higher at 10,000 hours. They have partnered with 10 universities in Bangladesh to reach their goal.

Asif Shahriyar Sushmit, one of the masterminds behind this project, said "2023 is the 71st anniversary of 1952's language movement. 52 and 71— two significant numbers in our history. 52 stands for language and 71 stands for freedom. We are aiming for this iconic moment to fulfil our goal— to release 10,000 hours of voice data for our AI researchers."

After the successful completion of the project, a new era will be opened for our researchers and not to mention— the Bangla-speaking community as well. We might get better voice assistants in Bangla, a much smarter voice keyboard which understands different dialects of Bangla. Businesses will be able to increase their productivity with better voice recognition.

Also, people with special needs will be beneficiaries of this as they would be able to operate smart devices in their mother tongue.

All these projects were completely voluntary. Bengali.AI doesn't seek any monetary benefits from their work.

"Many private companies tried to do what we are doing but they failed. People donated their time and data to AI research just because the data will be used for nonprofit purposes. The mission and vision of Bengali.AI remain the same. We will continue working to democratise AI research in Bangla. We are planning to launch another competition very soon to get more data on the Bangla voice recognition project." said the founder.

At this point, more than 6,000 researchers from different parts of the globe are working with Bengali.AI. Bengali.AI aims to continue working on AI research in Bangla to bring the best of artificial intelligence to the masses.

To contribute to the campaign, visit this link: https://commonvoice.mozilla.org/bn/speak