December 18, 2023
1 min read

Microsoft launches robust AI ‘small language model’ for researchers

Phi-2 is an ideal playground for researchers, including for exploration around mechanistic interpretability, safety improvements, or fine-tuning experimentation on a variety of tasks…reports Asian Lite News

Microsoft has released its newest compact “small language model” titled Phi-2 that continues to perform at par or better than certain larger open-source Llama 2 models with less than 13 billion parameters.

Over the past few months, the Machine Learning Foundations team at Microsoft Research has released a suite of small language models (SLMs) called “Phi” that achieve remarkable performance on a variety of benchmarks.

The first model, the 1.3 billion parameter Phi-1 achieved state-of-the-art performance on Python coding among existing SLMs (specifically on the HumanEval and MBPP benchmarks).

“We are now releasing Phi-2, a 2.7 billion-parameter language model that demonstrates outstanding reasoning and language understanding capabilities, showcasing state-of-the-art performance among base language models with less than 13 billion parameters,” the company said in an update.

Phi-2 is an ideal playground for researchers, including for exploration around mechanistic interpretability, safety improvements, or fine-tuning experimentation on a variety of tasks.

“We have made Phi-2 available in the Azure AI Studio model catalog to foster research and development on language models,” said Microsoft.

The massive increase in the size of language models to hundreds of billions of parameters has unlocked a host of emerging capabilities that have redefined the landscape of natural language processing.

However, a question remains whether such emergent abilities can be achieved at a smaller scale using strategic choices for training, e.g., data selection.

“Our line of work with the Phi models aims to answer this question by training SLMs that achieve performance on par with models of much higher scale (yet still far from the frontier models),” said Microsoft.

The company has also performed extensive testing on commonly used prompts from the research community.

“We observed a behaviour in accordance with the expectation we had given the benchmark results,” said the tech giant.

ALSO READ-Microsoft to invest $3.2 bn in UK to drive future AI growth

Previous Story

BNP cannot win public support through arson, says Hasina

Next Story

Bangladesh War of Liberation is bedrock of our ties, says envoy

Latest from Tech LITE

India’s EV sales need turbo boost

India must accelerate EV adoption by 22% in five years, or risk missing its 2030 green mobility target, warns NITI Aayog….reports Asian Lite News India will need to accelerate electric vehicle (EV)

Uber Targets India Dominance

On the subject of travel, Khosrowshahi observed that booking processes remain outdated and ripe for disruption. “I don’t think that the travel industry has innovated that much Uber CEO Dara Khosrowshahi has

Arab League urges Bigger AI investments

A central message of the Arab AI Forum was the urgent adoption of the league’s recently endorsed ethical AI charter….reports Asian Lite News In a defining moment for the future of artificial

Japan City Limits Smartphones

The proposal comes as new figures from Japan’s Children and Families Agency show that young people in the country spend an average of more than five hours online each weekday A city

India Embraces AI Future

Upskilling is emerging as a critical focus, with 51 per cent of leaders naming it their top priority. Around 63 per cent of managers expect AI training to become a core team
Go toTop

Don't Miss

US Warns On Microsoft Outage, Nadella Assures Fix

CISA warns hackers exploiting Microsoft outage for phishing, as CEO

Demand for AI jobs up 11% in India

The IT sector is leading with 29 per cent, followed