For a while now, companies like OpenAI and Google have been touting advanced "reasoning" capabilities as the next big step in their latest artificial intelligence models. Now, though, a new study from six Apple engineers shows that the mathematical "reasoning" displayed by advanced large language models can be extremely brittle and unreliable in the face of seemingly trivial changes to common benchmark problems. The fragility highlighted in these new results helps support previous research suggesting that LLMs use of probabilistic pattern matching is missing the formal understanding of underlying concepts needed for truly reliable mathematical reasoning capabilities.
Apple Study Exposes Deep Cracks In Llms’ “reasoning” Capabilities
Kyle Orland, Ars Technica
Mon, 10/14/2024 - 2:21pm
Topics: gsm-symbolic: understanding the limitations of mathematical reasoning in large language models openai google current llms mix in gsm-symbolic large language models"—currently sophie bill comments apple gsm8k study reasoning capabilities question problems advanced benchmark blocks building researchers results set
Welcome to Wopular
Wopular is an
online newspaper rack,
giving you a summary view of the top headlines from the top news sites.
Senh Duong (Founder)
Wopular,
MWB,
RottenTomatoes
MoviesWithButter : Our Sister Site
- "Expendables 4" Starts Shooting This Fall, Says Randy Couture at the Arnold Sports Festival more
- “Expendables 4” Gets Funding Interest from “Ip Man 3” Financier more
- "Ip Man 3" Distributor Suspected of Box Office Fraud, Parent Company Loses $200M in Market Cap more
- Yen Targets "Ip Man 4" for 2018 Release; Discusses Film with Director, Producer more
- "Ip Man 3" Shatters Expectations with $72M Opening [China Box Office] more