Machine Learning | Artificial Intelligence

959 readers

5 users here now

Welcome to Machine Learning – a versatile digital hub where Artificial Intelligence enthusiasts unite. From news flashes and coding tutorials to ML-themed humor, our community covers the gamut of machine learning topics. Regardless of whether you're an AI expert, a budding programmer, or simply curious about the field, this is your space to share, learn, and connect over all things machine learning. Let's weave algorithms and spark innovation together.

founded 2 years ago

MODERATORS

Hopps

MIT researchers make language models scalable self-learners (news.mit.edu)

submitted 2 years ago* (last edited 2 years ago) by Hopps to c/machinelearning

1 comments fedilink hide all child comments

TLDR Summary:

MIT researchers developed a 350-million-parameter self-training entailment model to enhance smaller language models' capabilities, outperforming larger models with 137 to 175 billion parameters without human-generated labels.
The researchers enhanced the model's performance using 'self-training,' where it learns from its own predictions, reducing human supervision and outperforming models like Google's LaMDA, FLAN, and GPT models.
They developed an algorithm called 'SimPLE' to review and correct noisy or incorrect labels generated during self-training, improving the quality of self-generated labels and model robustness.
This approach addresses inefficiency and privacy issues of larger AI models while retaining high performance. They used 'textual entailment' to train these models, improving their adaptability to different tasks without additional training.
By reformulating natural language understanding tasks like sentiment analysis and news classification as entailment tasks, the model's applications were expanded.
While the model showed limitations in multi-class classification tasks, the research still presents an efficient method for training large language models, potentially reshaping AI and machine learning.

top 1 comments

sorted by: hot top controversial new old

[–] Hopps 1 points 2 years ago

This research by MIT highlights an impressive leap in the field of AI and machine learning, demonstrating that smaller language models can indeed compete with, and even surpass, their larger counterparts in natural language understanding tasks. The introduction of self-training techniques and the innovative use of textual entailment shows a novel approach towards addressing issues like inefficiency and privacy concerns, often associated with larger AI models. This not only makes AI technologies more scalable and cost-effective, but also improves their robustness and adaptability. However, the limitations in multi-class classification tasks indicate there's still room for improvement and exploration. Overall, this study potentially paves the way for a more sustainable and privacy-preserving future in AI technologies, reaffirming the belief that in the world of AI, quality indeed triumphs over sheer size.