Boolean Biotech VHH Design Competition 2025

This is quite a departure for this blog, but I thought it might be fun to follow Adaptyv Bio, Specifica, Ginkgo, et al. and run my own (tiny) protein design competition, the "Boolean Biotech VHH Design Competition 2025"!

Why do this when there are other, larger competitions? The twist is that instead of submitting a design tuned to the target, you submit a script that outputs designs for any target. The goal is to see how good we are at making VHHs with open models, limited compute, and no manual supervision. I am optimistic I'll get at least one submission!

The rules

• For simplicity, entrants should use the standard hNbBCII10 VHH.

>hNbBCII10
QVQLVESGGGLVQPGGSLRLSCAASGGSEYSYSTFSLGWFRQAPGQGLEAVAAIASMGGLTYYADSVKGRFTISRDNSKNTLYLQMNSLRAEDTAVYYCAAVRGYFMRLPSSHNFRYWGQGTLVTVSS
>hNbBCII10_with_CDRs_Xd
QVQLVESGGGLVQPGGSLRLSCAASXXXXXXXXXXXLGWFRQAPGQGLEAVAAXXXXXXXXYYADSVKGRFTISRDNSKNTLYLQMNSLRAEDTAVYYCXXXXXXXXXXXXXXXXXXWGQGTLVTVSS

• The target is Maltose-binding protein from BenchBB (PDB code: 1PEB). BenchBB, an Adaptyv Bio project, has seven targets to choose from. The alternatives are either too large (Cas9), arguably too small (BHRF1, BBF-14) or already well-trodden (EGFR, PD-L1 and IL-7Rα).
• You submit one Python script that I can run using the uvx modal run command below (optionally using --with PyYAML or other libraries.) The script should use ideally use all the chains in the PDB file as the target. For simplicity, if your design tool uses only sequence and not structure, extract the sequence from the PDB file.

uvx modal==1.2.0 run {your_pipeline_name}.py --input-pdb {pdb_name}.pdb

• I will use $50 of compute on any GPU available on modal to produce a binder. The modal script should output a file called {your_pipeline_name}.faa with a maximum of 10 designs that looks like this:

>{optional_info_1}
{binder_seq_1}
>{optional_info_2}
{binder_seq_2}
...

• I can run non-open pipelines (e.g., pipelines that use PyRosetta), but the intention of the competition is to compare open pipelines, e.g., FreeBindCraft over BindCraft.
• To rank designs, I will fold with AF2-Multimer with 3 recycles and MSA, and take the designs with the maximum ipTM. Of course, your script is free to do its own ranking and output a single result.
• I will submit the 10 best submissions to BenchBB, with a max of one per entrant (though if there are fewer than 10 entries, I'll run more than one per entrant.) I'll test more if i can! Ideally I would like to test multiple targets with the same pipeline.
• You have until Friday November 7th to submit. This is not that much time but the hope is that submissions should mostly run existing open pipelines, adapted to run as a single modal script, so there should not be a lot of target-specific tuning going on.
• Since I have no idea if I'll get any submissions within this timeframe, and it's a pretty casual competition, I reserve the right to change the rules above a bit. I think there is a good chance I'll end up just submitting some designs myself, but it's fun to let other people try if they like!

The competition

Obviously, this will be a small competition, so I won't be too strict if there are issues, but I don't want to spend time on environments, jax, cuda, etc. This is a very appealing aspect of forcing the competition to run on uv and modal: one portable script should be able to do whatever you need.

All the code, designs and stats will be made public, and will appear on ProteinBase (Adaptyv Bio's public database), hopefully a few weeks after the competition ends. Adaptyv Bio has its own BenchBB stuff in the works too.

The prize is even better than lucre: it's glory, and maybe a t-shirt? A plausibly easy way to enter would be to use the IgGM modal app from the biomodals repo, which should be almost plug-and-play here.

This competition is difficult, maybe way too difficult! Even the best models today recommend testing 10s of designs for every target. So it might be impossible, but I am struck that one submission from the BindCraft team to the 2024 Adaptyv competition bound, and at 100nM too!

If the expected outcome of everything failing comes to pass, maybe I will try again when the technology has progressed a bit.

(Thanks to Nick Boyd for help with figuring out the rules.)

Comments