First of all, we would like to thank everyone that participated! We were pleasantly surprised by the quality and depth of the submissions.
After much internal deliberation and thought*, we are pleased to announce the winners of the 1st ever Spectra.pub Machine Learning Review Paper Competition! Without further ado…
1st prize: Towards continual task learning in artificial neural networks
David McCaffary, University of Oxford
This review article covers an important research topic, is well researched, and is well written. It provides an excellent overview of research into continual task learning, transfer learning, and approaches to mitigate the catastrophic forgetting problem that plagues naive approaches to continual task learning. It also draws interesting parallels to neuroscience. Important and interesting stuff!
2nd prize: Co-Tuning: An easy but effective trick to improve transfer learning
Kaichao You, Tsinghua University
This article presents a clear and easy to follow summary of a practical method for transfer learning previously published in a 2020 NeurIPS paper. This method has been shown to perform significantly better than vanilla fine tuning (copy weights, discard fully connected layer). We also love the insight that “transfer learning is the hidden force behind deep learning.” So true!
3rd prize: Adversarial learning on graphs
Jintang Li, Sun Yat-Sen University
A fascinating overview of the emerging field of adversarial learning on graphs, full of links to papers and working code. This line of research is relevant to recommendation systems, where the user-item relationships can be modeled as a graph. By injecting fake users into the system to promote certain items, an attacker can subvert the system. The paper discusses ways to mitigate attacks, metrics to evaluate robustness against attacks, and discusses open research problems in this field.
Winners: Congratulations! Please email nico@mathpix.com to claim your prize and some swag!
Honorable mentions
- Information Theory in Machine Learning - Srishti Saha
- Go-Explore: Reinforcement Learning Algorithms Tackling Hard-Explore Tasks - Guy Leroy
- Causal Machine Learning in Healthcare - Roman Böhringer
Final thoughts
A common theme in the submissions, as evidenced by the winner of the 1st and 2nd prize, has been transfer learning. Clearly, more research into model re-usability and learning in low data regimes is needed to move AI research forward. As AI applications with significant scale (e.g. speech recognition) get swallowed up by corporate research labs with big compute and dataset budgets, better AI will be needed to tackle the “long tail” of AI applications, as well as general AI. Over time, it’s even possible that general AIs will eventually be able to compete with special purpose AIs.
The theme of operational re-usability is seen not just in AI, but in other fields, like space exploration. Building a new rocket every time you want to go to space is not practical. Likewise, training a model for two weeks every time you want to teach an algorithm to perform a new task is also impractical if you want to achieve accessible, affordable AI.
Next Spectra competition!
We will be announcing a new Spectra competition soon! Want to help, with editing, judging, or anything else? Please get in touch with us at support@mathpix.com!
*Votes and article pageviews were also taken into consideration when choosing the winners.