ESM-2 outperforms all tested single-sequence protein language models across a range of structure prediction tasks. ESMFold harnesses the ESM-2 language model to generate accurate structure predictions end to end directly from the sequence of a protein.

2023年3月16日 · We trained a family of transformer protein language models, ESM-2, at scales from 8 million parameters up to 15 billion parameters. Relative to our previous generation model ESM-1b, ESM-2 introduces improvements in architecture, training parameters, and increases computational resources and data [supplementary material (SM) sections A.1.1 and A.2].

ESM-2 outperforms all tested single-sequence protein language models across a range of structure prediction tasks, and enables atomic resolution structure prediction.

2023年8月5日 · ESM-1b, ESM-1v, and ESM-2: Three sets of masked language models ranging in size from 30M parameters to 15B parameters. The ESM-1b and ESM-1v models deserve credit for demonstrating the potential of scaled up masked language models for both zero-shot fitness and contact prediction.

ESM-2 is a state-of-the-art protein model trained on a masked language modelling objective. It is suitable for fine-tuning on a wide range of tasks that take protein sequences as input. For detailed information on the model architecture and training data, please refer to the accompanying paper.

2022年11月1日 · The ESM-2 language model is trained to predict amino acids that have been masked out of sequences across evolution. We discovered that, as a result of this training, information about the protein’s structure emerges in the internal states of the model.

ESM-2 language model is trained with ~65 million unique sequences $^\bigstar$. Because of the MLM objective, we ask the model to predict missing pieces (amino acids) of the sequences using the neighbouring amino acid context.

The ESM-2 model is a transformer-based protein language model that has achieved state-of-the-art results in various protein-related tasks. When fine-tuning ESM2, the task-head plays a crucial role.

2023年8月25日 · Specifically, we explore different ESM-2 with the parameter numbers of 8M, 35M, 150M, 650M, and 3B and plot results in Fig. 2. It verifies that scaling the protein language model is...

2023年10月15日 · TLDR: This blog post is about using ESM-2, a protein language model, to score pairs of proteins using masked language modeling loss, in order to predict pairs of proteins that have a high likelihood of binding to one another.