There is an open PhD position in Interpretability for Protein Language Models available at LION Lab, at the Institute for Computer Science at Leipzig University.
The position will be funded by a DFG project within the Priority Programme Artificial Intelligence for Protein Function (AI4PF). The project will be carried out by two PhD students, one working on the computational side, based at LION Lab at the Institute for Computer Science (this position) and one working on the practical side, based at Schoeder Lab at the Institute for Drug Discovery, both at Leipzig University.
The ovearching goal of the project will be to adapt methods from (mechanistic) interpretability developed by the NLP community for LLMs to protein language models (pLMs). pLMs have been trained on billions of protein sequences and acquired complex knowledge of their structure in an unsupervised way, which can be compared to how LLMs have acquired complex linguistic knowledge despite only having been trained on raw text data. pLMs and LLMs are also similar in that they have both achieved far better results at their respective tasks than earlier approaches that explicitly modelled existing insights into the structure of the training data. While in NLP, this has led to high research activity in the field of interpretability, aiming to extract and make explicit the structural knowledge acquired by LLMs, this has so far been underexplored for pLMs.
For the computational side of the project, the first step will be to extend interpretability methods to test pLMs' knowledge of toxicity and immunogenicity, and then use the findings to develop a pLM into a classifier for immunogenicity. This classifier will then be applied for the steering of de novo protein generation with a minimal number of immunogenic peptide sequences. In the meantime, the wetlab PhD student based at Schoeder Lab will establish an immunogenic profile for de novo proteins, and then both PhD students will collaborate on generating minimally immunogenic proteins and evaluating how well this succeeded. Later in the project, we will explore other applications for the transfer of interpretability methods to pLMs.
Beyond the local collaboration with Schoeder Lab, the PhD student will be able to take advantage of a large international network of collaborators, and we will arrange a research stay abroad for several months for each student at some point during their PhD, with a host appropriate for their research topics.
The position is fully funded as TVL-13 100% for three years. Information about the pay scheme is available here. The PhD student will have no formal teaching obligations, but will have the opportunity to teach if they wish, for example giving a tutorial for our NLP and ML undergraduate lectures.
Ideal PhD candidates have a master's degree in computational linguistics, bioinformatics, computer science, data science, or a related discipline, a strong interest in both natural language processing and biology, excellent mathematical and programming skills, and are team players. Knowledge of German is a plus, but not required. However, strong written and spoken English skills are essential.
International applicants can find more information on doing a PhD in Germany here (provided by the Federal Government) and specific to Leipzig here .
Leipzig University is strongly committed to diversity and especially welcomes applications from members of underrepresented groups.
The LION (Linguistically-Oriented Natural Language Processing) Lab is a new research group at Leipzig University, led by Jun.-Prof. Dr. Leonie Weissweiler. For more information, you can check out the lab website and Jun.-Prof. Weissweiler's personal website.
Leipzig University is the second-oldest university in Germany. It looks back on over 600 years of tradition and is a comprehensive university with 14 faculties, 465 professorships, around 31,000 students and a variety of subjects in 158 degree programmes. Leipzig has a well-connected research landscape: the university is part of a local network that includes Max Planck, Helmholtz, Fraunhofer, and Leibniz institutes, providing opportunities for collaboration across different areas of science.
The Institute for Computer Science is located directly in the city centre, in the Paulinum building. Leipzig offers a high quality of life, a vibrant cultural scene, and excellent transport connections — Berlin is only an hour away.
You can find information about pursuing a PhD at Leipzig University, compiled by the PhD student council, here.
To apply, email your CV, transcripts, a copy or draft of your master's thesis and a 1-page research proposal to leonie.weissweiler@uni-leipzig.de under the header [PROTEIN-PHD]. The research proposal should not cover the scope of the entire project, but rather be a description of your own idea within the scope of applying interpretability methods to pLMs, and show that you are able to formulate a research question and make a plan for how to answer it.