Peptone AI team presents our Sentence Transformers research during NeurIPS 2023 Conference
Protein language models (pLMs) have been widely used in in-silico protein engineering tasks and have shown remarkable results.
However, their application methods are mostly standardized. In this regard, our team introduced a set of fine-tuning techniques based on Sentence Transformers (STs) that were integrated with a novel data augmentation procedure. Our team demonstrated how this can lead to new state-of-the-art performance.
Although STs were initially developed in classic NLP space, they hold natural appeal in pLM-related applications, mainly due to their use of sequence pairs and triplets in the process.
Our team showcased this conceptual approach in two different settings that frequently occur in this domain: a residue and sequence-level prediction task.
Apart from demonstrating how these tools can extract more and higher quality information from pLMs, our team will discuss the main differences between their applications in NLP and protein spaces.
About Peptone
Peptone is a translational biophysics company focused on the discovery of novel therapeutics against diseases driven by intrinsically disordered proteins. IDPs are proteins without a fixed structure that play a significant role in health and disease.