How knowledge distillation compresses neural networks

If you’ve ever used a neural network to solve a complex problem, you know they can be enormous in size, containing millions of parameters. For instance, the famous BERT model has about ~110 million. To illustrate the point, this is the number of parameters for the most common architectures in (natural language processing) NLP, as summarized in the recent State of AI Report 2020 by Nathan Benaich and Ian Hogarth. You can see this below: In Kaggle competitions, the winner models are often ensembles, composed of several predictors. Although they can beat simple models by a large margin in terms of…

This story continues at The Next Web

from The Next Web https://ift.tt/3miEaGu

Comments

Apple’s privacy reputation at risk with new iTunes class-action lawsuit

Apple has marketed itself as a company that puts customers’ privacy first, but a new class-action lawsuit claims to the contrary. Leigh Wheaton, Jill Paul, and Trevor Paul — three iTunes users from Rhode Island and Michigan — filed a federal lawsuit against the Cupertino-based tech giant alleging that the company unlawfully collects and sells their iTunes listening information to third parties without informed consent. The claims, if true, are bound to puncture a hole in Apple’s pro-privacy stance, which it touted earlier this year with the ad slogan: “What happens on your iPhone stays on your iPhone.” Apple never shows… This story continues at The Next Web Or just read more coverage about: iTunes , Apple from The Next Web http://bit.ly/2WuCjFe

Microsoft once owned a chunk of Apple, here’s what it would have been worth today

Back in 1997, Apple wasn’t as shiny as we know it today. It was actually a little bit rotten. It was the year they asked Steve Jobs to return to save the company, as it was struggling to keep up with the competition, Microsoft in particular. In just two years, its sales had fallen from $11 billion to just $7 billion. To survive, the company had to determine what its core assets were, where it was most relevant, and what it should focus on more. And it had to forge new partnerships, even if that meant cutting a deal with the… This story continues at The Next Web Or just read more coverage about: Apple , Microsoft from The Next Web https://ift.tt/32sonws

TNW Podcast: Boris comes over to co-host; Slack’s Cal Henderson talks European tech

Welcome to the new episode of the TNW Podcast — the show where we discuss the latest developments in the European technology ecosystem and feature interviews with some of the most interesting people in the industry. In today’s episode, Andrii is joined by Boris Veldhuijzen van Zanten, co-founder, member of the board, and former CEO of TNW. The topics discussed include the jobs created by Dutch startups, giant state funding for energy projects, translations of the word ‘computer’, and a bunch of other things in between. In the interview section, we’re featuring a conversation with Cal Henderson, co-founder and… This story continues at The Next Web from The Next Web https://ift.tt/jUgcNFD

Technology News Era

Search This Blog