# Peptone presents Sentence Transformers research at NeurIPS 2023

> Peptone's AI team presented a Sentence Transformers approach for fine-tuning protein language models at the NeurIPS 2023 Biology Track.

[View this page on Peptone](https://peptone.io/news/peptone-ai-team-presents-our-sentence-transformers-research-during-neurips-2023-conference/)

Published: 2023-12-12

Protein language models are widely used for in silico protein-engineering tasks and have produced strong results. Their application methods, however, have largely become standardized.

At the NeurIPS 2023 Biology Track, Peptone's AI team introduced fine-tuning techniques based on Sentence Transformers and combined them with a new data-augmentation procedure. The research showed how this approach can achieve state-of-the-art performance.

## Sentence Transformers for protein language models

Sentence Transformers were originally developed for natural-language processing, but their use of sequence pairs and triplets makes them a natural fit for protein-language-model applications.

The team evaluated the approach in two settings that frequently arise in protein research:

- a residue-level prediction task; and
- a sequence-level prediction task.

The work demonstrated how Sentence Transformers can extract more, higher-quality information from protein language models. It also examined the important differences between applying these methods to natural language and applying them to protein sequences.

[Download the NeurIPS research paper](https://www.mlsb.io/papers_2023/Optimizing_protein_language_models_with_Sentence_Transformers.pdf) or [learn more about the NeurIPS 2023 Biology Track](https://www.mlsb.io/).

[Original source](https://www.eu.peptone.io/conference/peptone-ai-team-presents-our-sentence-transformers-research-during-neurips-2023-conference/)