How knowledge distillation compresses neural networks

If you’ve ever used a neural network to solve a complex problem, you know they can be enormous in size, containing millions of parameters. For instance, the famous BERT model has about ~110 million. To illustrate the point, this is the number of parameters for the most common architectures in (natural language processing) NLP, as summarized in the recent State of AI Report 2020 by Nathan Benaich and Ian Hogarth. You can see this below: In Kaggle competitions, the winner models are often ensembles, composed of several predictors. Although they can beat simple models by a large margin in terms of…

This story continues at The Next Web

from The Next Web https://ift.tt/3miEaGu

Comments

Doomed US lander crushes hopes for first European tech on the Moon

The Peregrine lunar lander is set to crash back to Earth in a sudden end to a mission that was set to put the first piece of European-built technology — and the human remains of a distinguished British sci-fi author — on the Moon. Shortly after launching from Cape Canaveral space station on Monday January 8, the spacecraft suffered a “critical loss of propellant” and was doomed to failure ever since. It is now expected to crash land into Earth on January 18. Astrobotic Technology, the company that built the lander, said it is now working with NASA on a… This story continues at The Next Web from The Next Web https://ift.tt/bHU2dJZ

Macron convinces UK nuclear startup to build its first factory in France

London-based nuclear energy startup Newcleo has scrapped plans to build its first power plant in Britain, opting to invest in France instead. The move follows extensive lobbying by France’s president Emmanuel Macron, who has personally met with Newcleo’s chief executive and founder Stefano Buono several times, to convince him to build the company’s first modular nuclear reactor in France, the Telegraph reports. Founded in 2021, Newcleo develops small nuclear reactors powered by radioactive waste. The company initially sought to tap the UK’s vast stockpile of nuclear waste at the former Sellafield site to power its reactors. But after two years… This story continues at The Next Web from The Next Web https://ift.tt/m9LkMaS

TNW Podcast: Oscar Kneppers on academic startups; ESA wants to detect space-time ripples

Welcome to the new episode of the TNW Podcast — the show where we discuss the latest developments in the European technology ecosystem and feature interviews with some of the most interesting people in the industry. In today’s episode, Andrii and Linnea talk about a new European space mission, AI-generated podcasts, antiquated tech in German trains, and much more. In the interview section, we’re featuring a conversation with Oscar Kneppers, CEO of the Amsterdam Centre for Entrepreneurship, or ACE. He’s also founded several well-known local publications and one of the country’s major startup accelerators, Rockstart. Here are the stories and… This story continues at The Next Web from The Next Web https://ift.tt/a6AlDsr

Technology News Era

Search This Blog