Connect with us

Hi, what are you looking for?

Tech News

Microsoft LASERs away LLM inaccuracies

Microsoft logo
Illustration: The Verge

During the January Microsoft Research Forum, Dipendra Misra, a senior researcher at Microsoft Research Lab NYC and AI Frontiers, explained how Layer-Selective Rank Reduction (or LASER) can make large language models more accurate.

With LASER, researchers can “intervene” and replace one weight matrix with an approximate smaller one. Weights are the contextual connections models make. The heavier the weight, the more the model relies on it. So, does replacing something with more correlations and contexts make the model less accurate? Based on their test results, the answer, surprisingly, is no.

“We are doing intervention using LASER on the LLM, so one would expect that the model loss should go up as we are doing more approximation,…

Continue reading…

You May Also Like

Editor's Pick

One of the key themes, if not THE theme, of 2023 was the dominance of the mega-cap growth trade. The Magnificent 7 stocks handily outperformed...

Editor's Pick

Eric Gomez On January 25, Taiwan’s military inducted its first group of conscripts for a one‐​year service period since Taipei announced the extension of mandatory...

Editor's Pick

The previous week was reduced to just a short 3-day trading week as Monday was declared a special holiday while on Friday, the markets...

Editor's Pick

In this episode of StockCharts TV‘s The MEM Edge, Mary Ellen reviews where the markets stand as we head into earnings from MSFT, GOOGL, AAPL and...