Peptone presents Sentence Transformers research at NeurIPS 2023
Peptone's AI team presented a Sentence Transformers approach for fine-tuning protein language models at the NeurIPS 2023 Biology Track
Protein language models are widely used for in silico protein-engineering tasks and have produced strong results. Their application methods, however, have largely become standardized.
At the NeurIPS 2023 Biology Track, Peptone’s AI team introduced fine-tuning techniques based on Sentence Transformers and combined them with a new data-augmentation procedure. The research showed how this approach can achieve state-of-the-art performance.
Sentence Transformers for protein language models
Sentence Transformers were originally developed for natural-language processing, but their use of sequence pairs and triplets makes them a natural fit for protein-language-model applications.
The team evaluated the approach in two settings that frequently arise in protein research:
- a residue-level prediction task; and
- a sequence-level prediction task.
The work demonstrated how Sentence Transformers can extract more, higher-quality information from protein language models. It also examined the important differences between applying these methods to natural language and applying them to protein sequences.
Download the NeurIPS research paper or learn more about the NeurIPS 2023 Biology Track.