April 12, 2022
3 mins read

NOOR – The world’s largest Arabic NLP model

Technology Innovation Institute (TII) unveils NOOR, the world’s largest Arabic natural language processing (NLP) model which is useful in automated summarization, chatbots, personalised marketing

Technology Innovation Institute (TII), a global research center and applied research pillar of Abu Dhabi’s Advanced Technology Research Council, today announced the launch of NOOR, the world’s largest Arabic natural language processing (NLP) model to date.

TII’s team of advanced researchers and Artificial Intelligence (AI) specialists at its AI Cross-Center Unit, joined forces on this initiative with LightOn, a technology company that unlocks extreme-scale machine intelligence for businesses, to revolutionize Arabic NLP models. The NOOR model carries out varied, cross-domain tasks simply from natural language instructions. To build NOOR, researchers at TII designed an end-to-end pipeline for the collection of high-quality data, including crawling, filtering, and curation at scale. TII’s specialists also built optimized services for extreme-scale distributed training and serving – to deliver applications with efficient inference and model specialization.

Dr. Ray O. Johnson, CEO, TII and ASPIRE, said: “With this development, we are on track to boost our research capabilities and credentials in AI, as well as elevating the status of Abu Dhabi and the UAE as a serious research ecosystem. Our expert teams have demonstrated yet again that this region can achieve breakthrough R&D outcomes that impact the world.”

Dr. Ebtesam Almazrouei, Director, AI Cross-Center Unit, TII, said: “Large language models have taken the world of natural language processing by storm, and we are proud to introduce this cutting-edge model with 10 billion parameters – the world’s largest Arabic NLP model.  The uniquely large Arabic dataset collected to train the model is the result of months of work that included curating, scrapping, and filtering of varied sources. A special thank you to the entire team that worked on this project to make NOOR the go-to exploration model in Arabic for academicians and businesses everywhere.”

Speaking on the upcoming launch, Prof. Mérouane Debbah, Chief Researcher, Digital Science Research Center and AI Cross-Center Unit, TII, said: “With NOOR, TII has expanded the scope of the modern standard Arabic model by leveraging know-how in large language models to build cross-disciplinary, cutting-edge expertise in this new generation of AI research.”

NOOR’s training dataset is the world’s largest high-quality cross-domain Arabic dataset, combining web data with books, poetry, news articles, and technical information to significantly widen the applicability of the model.

Abdulaziz Alshamsi, AI Researcher and PhD student, TII, said: “As an Emirati researcher, I’m proud to be a part of TII. I have enjoyed meeting and working with the passionate researchers and trainers at the AI Cross-Center Unit, who bring profound value to the Arabic language and NLP domain. I learned technical and advanced skills that will pave the way for me as a researcher to discover the world beyond NLP horizons. Training workshops helped upskill and enlighten us about new concepts and equipped us with the right tools to bring the NOOR project to life.”

Dr. Ebtesam Almazrouei said the NOOR model is based on the popular Transformer architecture. As a decoder-only model, similar in structure to GPT-3, it is programmed to tackle generative tasks with architecture upgraded to reflect the latest developments in the world of machine learning, including improvements such as better positional embeddings. To help ensure quality at scale in the NOOR dataset, the TII team designed an automated filtering pipeline based on machine learning techniques. These tools identify text like quality references and safeguard the model from exposure to spam content.

ALSO READ: UAEBBY launches 14th Etisalat Award for Arabic Children’s literature at LBF 2022

Leveraging state-of-the-art 3D parallelism, NOOR was trained on a High-Performance Computing resource with 128 A100 GPUs, allowing for the distribution of computations and ensuring efficient use of the available hardware resources.

The Director of the AI Cross-Center Unit noted that this was only the first step in TII’s efforts to contribute to the wider UAE Strategy for Artificial Intelligence, through supporting AI integration across key sectors of the economy.

Named for the Arabic word “light”, the model has been so called to establish the correlation of the Arabic language model to enlightening the mind. It represents the United Arab Emirates global contribution to advanced technology and artificial intelligence.

Previous Story

IAF may help BSF for Mi17 chopper maintenance

Next Story

Pak PM Sharif offers olive branch to India

Latest from Arab News

Hamas Reiterates Peace Effort After U.S. Jab

Hamas Reaffirms Commitment to Ceasefire Talks After U.S. Criticism…reports Asian Lite News Hamas on Friday reiterated its commitment to the ongoing ceasefire negotiations, responding to remarks by U.S. Special Envoy to the

Israel, US recall teams from Gaza truce talks

Steve Witkoff said mediators had made a great effort but “Hamas does not appear to be coordinated or acting in good faith”…reports Asian Lite News Israel and the United States recalled their

Riyadh welcomes Paris pivot to Palestine

Move stirs global response as Macron breaks G7 ranks; Israel and US condemn recognition bid amid Gaza war…reports Asian Lite News Saudi Arabia has welcomed French President Emmanuel Macron’s pledge to officially

‘UN-OIC unity key to peace push’

Khaled Khiari, particularly highlighted the convergence of UN and OIC efforts in the Middle East, especially in seeking a just, comprehensive resolution to the Palestine question…reports Asian Lite News The United Nations

UAE talks big at global SDG meet

The forum provided a platform for the UAE to highlight its pioneering role in accelerating progress toward the 2030 Sustainable Development Goals..reports Asian Lite News The United Arab Emirates reaffirmed its global
Go toTop

Don't Miss

Rajasthan Pavilion to be inaugurated today at Dubai Expo

The Rajasthan Pavilion will be inaugurated inside the Indian Pavilion

Dubai Ruler approves Dh205bn budget

In its 2023 budget, Dubai continues to focus on social