Readers of this blog will be well aware that it's an exciting time for AI protein binder design. Since BindCraft was released in 2024, there has been plenty of activity, but things have heated up again over the past couple of months with the release of BindCraft2 from Pacesa Lab, RFD4-Proteina from Baker lab, a report from Anthropic, and several high-profile competitions run by Adaptyv and Anthropic.

There remains a pretty central question of how to rank binders to maximize wet-lab success. Last week I posted a table comparing the main binder ranking scores on twitter. It was a quick and dirty AI-driven experiment, but it generated some interest, so in this article I am going back to do it again a bit more carefully!

How does binder scoring work?

When designing a binder we are attempting to select a sequence that optimizes a certain score (or loss).

Binder design scores are usually a weighted combination of two types of score:

• folding model confidence scores (e.g, pLDDT, ipTM, ipSAE, iPAE, actifpTM);
• heuristics like sequence "naturalness", and physics-based filters like surface complementarity and number of H-bonds.

Only folding model confidence scores correlate strongly with binding, but directly optimizing these scores can easily lead to adversarial sequences. As we know, AIs are great at "hacking the test". It's also worth noting that these scores are not trying to measure binding per se, but more like whether the structures appear together in a PDB file. See the escalante blog for more details on this.

The heuristics and physics-based filters bias the binder sequence and structure to the part of the distribution where real binders live, but this is something of an art for now.

A simple method to help with adversarial sequences is to refold and rank with confidence scores from independent folding algorithms, e.g., design with RFD4-Proteina, and rank with ESMFold2.

Binding score types

The original "binding score", introduced by AlphaFold-Multimer in 2021, is the "interface predicted TM-score" (ipTM). ipTM is calculated directly from the predicted aligned error (PAE) distribution.

PAE evaluates positional error using the local backbone reference frame of each residue, rather than a global alignment. Because each residue defines its own reference frame, PAE is asymmetric. iPAE is just the PAE between chains (e.g., binder and target). Despite the name, ipTM compares two complete chains, not just the interface.

An obvious limitation here is in cases where the model is confident in the interface and not confident about the rest of the protein. In 2025, two scores were published to correct for this problem: ipSAE and actifpTM. This year, BindCraft2 introduced one additional refinement: ipDAE. These three scores are all quite similar, differing mainly in how they select the relevant interface residues.

On twitter, I was fortunate to get some background and feedback from Julia Varga around actifpTM thresholds and Roland Dunbrack on using ipSAE for binders (published in the new ipSAE2 paper).

Julia Varga commenting on actifpTM and suggesting a threshold change. I did not test it here.

Roland Dunbrack recommending a specific ipSAE variant for binders

I was surprised to see that these scores are also in flux: since PAE is asymmetric, you can take the minimum, maximum, mean, or a specific direction (e.g., align on target and use the binder ipSAE). On the BindCraft2 repo, they switched from max to mean ipDAE just a couple of weeks ago!

Scoring

I could only find a few papers on the topic of binder ranking scores:

• Overath et al. 2025, "Predicting Experimental Success in De Novo Binder Design: A Meta-Analysis of 3,766 Experimentally Characterised Binders"
• Cotet et al. 2025, "Crowdsourced Protein Design: Lessons From the Adaptyv EGFR Binder Competition"
• Bozkurt 2026, "Beyond Structure and Affinity: Context-Dependent Signals for de novo Binder Success"
• Liu et al. 2026, "ProtDBench: A Unified Benchmark of Protein Binder Design and Evaluation"

Overath et al. is really the only paper here that tries to compare binder scores across many targets. They took a classic machine learning approach of calculating as many features as they could (over 200) and attempted to predict binding across 3,766 binders and 15 targets. Their conclusion was that AlphaFold 3 ipSAE (minimum) was the single best score, with Rosetta interface energy and shape complementarity also contributing.

ProtDBench is a binder design benchmark. The paper includes some commonly used binder thresholds.

My Proteinbase Benchmark

I used Adaptyv's open Proteinbase database for this experiment. For each of the 24 suitable targets in the database, I took all available binders and an equal number of non-binders. In total, I co-folded 225 binders and 225 non-binders. All are protein binders, of length 10-250 residues, with an average length of 92. There were a negligible number of antibodies or antibody fragments, so these were excluded.

All the data is in this Hugging Face repo, so point your agents here to reanalyze it!

Folding models

Since I work at a company, I can't use AlphaFold 3; instead I use these MIT or Apache-2 licensed tools:

• ESMFold2 is very fast and still reliably high quality;
• OpenDDE is an excellent new model, especially good for antibodies;
• Mol-1 is only a week or two old, but the benchmarks are impressive across the board, including for antibodies. Mol-1 has succeeded Protenix-v2, which confusingly went from open to closed weights, and so is best avoided.

Depending on how much you trust the benchmarks, OpenDDE and Mol-1 can get close to or beat AlphaFold 3, even on antibodies.

Mol-1 showing some impressive performance on FoldBench.

Results

There are many ways to look at the data. I used two main scores: the PPV for the top 10% of scores, and the AUROC. In both cases I take the average value across all targets since some targets have a lot more data than others. Since there are equal numbers of binders and non-binders, random guessing would score 50%; in the real world there will be many more non-binders to filter out. I tested four folding models (ESMFold2, Protenix-v2, OpenDDE, Mol-1), and a consensus "all four models" version where I use the worst percentile rank across the four models.

In all cases I am using the default version of the score. For example, this means I am using max ipSAE, not min ipSAE as recommended in Overath et al. I did some additional analyses where I explored variations of the scores, and found minimal differences.

Top 10% PPV. The best score is highlighted in green; 95% bootstrap CI in small font underneath.

AUROC. The best score is highlighted in green; 95% bootstrap CI in small font underneath.

I was surprised at how well ipTM did. All folding models produce this score by default, so it's easy to get, and it performs well across the board.

ESMFold2 also deserves a shout-out. It's the oldest and fastest method here, but it's still performing very well.

I thought I had a pretty decent amount of data here, but even so, using paired comparisons, I could not detect significant differences between any of the scores. Maybe because they are all derived from the same aligned error data. I don't think this means they all perform the same, but I'd need quite a bit more data to find daylight between them.

Note that many of these Proteinbase submissions were selected for high ipTM, so there is also some selection bias.

Foldism

I have an open-source folding tool called foldism. It runs on modal and includes AlphaFold 2, ESMFold2, OpenDDE, and Mol-1. I removed Boltz-2, Chai-1, and OpenFold to streamline it, but these are easy for an agent to add back in. I updated the code to calculate ipTM, ipSAE, ipDAE, and actifpTM, giving the score and a percentile from the distribution of non-binders in my dataset above.

Insulin's two chains co-folded and scored with foldism. It's not a true binder, but only ipDAE gives a high percentile here (96–100).

One of the minor irritations in my life is that OCR does not consistently work for biological sequences (nucleic acids and proteins).

People think of OCR as a solved problem because AIs are very good at it, but the mistakes AIs make in human language are usually forgivable: maybe an O instead of a 0, or a hyphen instead of an en-dash.

Biological sequences have peculiar challenges: they can be thousands of characters or more; they are often repetitive and high entropy compared to human language; they are fairly often embedded in images, whether that is a scanned PDF, or a rasterized figure in a paper. Most importantly, a single typo can break the whole sequence.

Patents

Patents have some of the most useful biological sequences, and some of the worst OCR problems. Google Patents includes the reference/original PDF of the patent (the USPTO certifies PDFs), and an OCR'd html page, which can have errors. Most of the time, the PDF contains extractable text, and no OCR is necessary, but even then, the sequences often live in images.

I got an AI to examine a sample of 100 patent PDFs, with 25 patents drawn from each of four publication eras (before 2000, 2000–2009, 2010–2019, 2020 onward). 11/100 contained no embedded or extractable text on any page (i.e., all images). Surprisingly, all were published in the last five years (e.g., US11584788B2).

I tried to find some illustrative examples of failed OCR: in WO2017040932A1, the Q in EIVLTQSP is rendered as O for SEQ ID NO: 22; in US11965030B2, Gln is rendered as GIn in two occurrences, SEQ ID NOs: 15 and 19.

Patent WO2017040932A1 PDF (top) vs HTML (bottom). There are multiple OCR errors underlined in pink. This is a grainy image, but readable by a human.

In the process of this search, I was a bit surprised to find that in most cases of impossible DNA or protein sequences, the PDF itself contained the typo.

Patent US8110549 has an impossible codon (underlined in pink).

I tried to do a more comprehensive evaluation of the prevalence of such errors, but it would have taken a ton of manual review. I think it probably makes more sense to wait a year and get a smart AI to do it.

OCR

There are two main kinds of OCR: specialized OCR tools like hoary mainstay tesseract, which works well but is tuned for human language, and AI OCR tools (just asking ChatGPT, Claude, Gemini), which are smarter but have the tendencies of a human reader, and so can get confused or distracted.

Broadly, the most common failure modes I have seen for these are:

• specialized: mistaking Gln for GIn (not enough intelligence to understand that this is a sequence of amino acids)
• AI: miscounting or losing track of the number of repeated characters (homopolymers)

Benchmark

Last year, after failing to extract some not-especially challenging protein sequences from several PDFs—forcing me to spend hours manually typing them in—I got annoyed and decided to make a benchmark.

I chose six sequences. These were an attempt at a representative sample, and were not chosen to be especially challenging. I created a bio_ocr_minibenchmark repo to let me test out methods and see what works best.

The six sequences in the benchmark are not particularly challenging.

These are all monospace fonts, have a constrained alphabet, and clear, uncorrupted images. Yet I have tried every OCR system I could find over the last year and so far nothing has gotten all of them correct. Almost everyone is surprised this is still an issue!

There has been progress in just the past few weeks. Opus 5 and GPT-6 Astra are the best I have tested so far (Fable refuses any biological work, so I can't evaluate it). Specifically, Opus 5 gets 5/6 correct, failing on image 6, which is just some Gs and Cs I typed into Google Docs! It loses count and adds some extra Gs for no reason.

The benchmark includes specialized OCR tools and AI OCR tools. There has been steady improvement over time.

Conclusion

We do not yet have a robust way to extract biological sequences from images. This is surprising, and a pain.

I think we are still maybe a year or more from a fully trustworthy OCR system you don't have to double-check, which is crazy to say given the current capabilities of AI generally.

The nose is the primary entry point for respiratory infections (hence rhinoviruses). This is because most of the air you breathe enters through here. The nose has a host of defenses, including the physical barrier of mucus (which also contains antibodies and peptidases) and the production of nitric oxide, a potent antimicrobial.

The other major entrypoint is the mouth, which has its own defenses including proteases and antibodies in saliva and the fact you can swallow viruses, yet mouth breathing is a significant risk factor for respiratory infections in children.

Respiratory infections are not treated as seriously as they should be. The flu causes tens of thousands of deaths per year in the US and costs tens or hundreds of billions a year; RSV is the leading cause of infant hospitalizations; long COVID affects millions and globally costs an estimated $1T annually; TB remains one of the world's deadliest infectious agents. Even the "common cold", which is actually infection by any of 200 or so viruses, can cause severe complications like pneumonia.

We are increasingly recognizing the accumulated burden of frequent infections, including connections to Alzheimer's and other neurological diseases, and the profound neurological benefits of vaccination, like the huge relative risks in this 2025 paper from Maggi et al.:

Vaccination against herpes zoster was associated with a reduced risk of any dementia (RR 0.76, 95% CI 0.69–0.83) and Alzheimer’s disease (RR 0.53, 95% CI 0.44–0.64). Influenza vaccination was linked to a reduction in dementia risk (RR 0.87, 95% CI 0.77–0.99), as was pneumococcal vaccination (RR 0.64, 95% CI 0.47–0.87) for Alzheimer’s disease. Tetanus, diphtheria, pertussis (Tdap) vaccination was also associated with a significant reduction for any dementia (RR 0.67, 95% CI 0.54–0.83).

We should not be surprised if today's viral load is damaging. The variety and frequency of infections we are now subjected to is an unnatural state for humans; as a species, we are accustomed to living in small tribes with no international travel.

By coincidence, just this week, Stripe launched the Intercept project, "a $500M philanthropic initiative to make respiratory infections, like the common cold and flu, a thing of the past" so at least some people are starting to take the problem seriously.

Nose sprays

This brings me on to one of my favorite subjects: nose sprays! Preventing infectious disease is one of the easiest way to improve long-term health and longevity. Luckily, nose sprays are a pretty simple, inexpensive, and effective intervention. These sprays primarily act outside of your cells, so the risks compared to e.g., antibiotics, are very low.

There are two major types of nose spray: those that act as a physical barrier to prevent entry of viruses, and the more drug-like antimicrobial or antihistamine sprays. The fact that most of the studies below are COVID-specific is just because of the timing of the pandemic and associated funding.

Physical barrier

• Salinex and LUCA V-Defense are carrageenan-based sprays with evidence for COVID infection prevention. $20 per bottle
• Profi is a gellan/pectin-based spray with evidence for influenza infection prevention in mice. $25 per bottle.
• Xlear is a xylitol-based spray with weak evidence as a COVID treatment. $33 per bottle

Antimicrobial / antihistamine

• NOWONDER is a nitric oxide spray with evidence for treating mild COVID. $40 per bottle.
• ViralDine is a povidone iodine spray with evidence for COVID inactivation. $20 per bottle.
• AstePro is an antihistamine spray with evidence for COVID infection prevention. $18 per bottle

Surprisingly, many of the papers above show pretty good evidence. Like masks, the mechanisms of action here are very straightforward.

My personal preference is for the physical barrier sprays. They act as an additional barrier like sunscreen, and appear to be very safe. For example, carrageenan is a GRAS food additive, sometimes used as a vegan alternative to gelatin.

Arguably, the successor to carrageenan sprays is Profi, which essentially builds on the "augmented mucus" concept. Profi has two main advantages over carrageenan: it provides both a physical barrier and pathogen neutralization, and it lasts a claimed eight hours. In 2024, the two professors at Harvard behind Profi published an intriguing study showing complete protection from Influenza A in a mouse model.

Profi acts as a physical barrier and neutralizes pathogens

I currently use Profi a maximum of once per day, but for more protection I would probably recommend Profi in the morning, and maybe NOWONDER nitric oxide spray before bed.

The alternatives

Good evidence and real papers are the exception in the supplement/wellness space. Maybe the nuttiest example is Oscillococcinum, which is somehow both homeopathic and snake oil, yet still gets sold in supermarkets all over the US and Europe.

Despite its insane ingredients, Oscillococcinum had revenue of $15M/yr in the US in 2008

Zicam, a nose spray you can find everywhere for "cold and allergy relief" appears to be exploiting the homeopathy loophole too. The evidence in its favor is weak, and there are hundreds of lawsuits filed against the company, alleging loss of sense of smell.

Despite weak evidence and potential anosmia, Zicam has revenue of approximately $100M/yr

A viral infection case study

The children sometimes get respiratory infections at school. The last time this happened was a couple of months ago, and I decided to sequence some saliva to see what the infectious agent was.

Many of the likely culprits are RNA viruses, so sadly you can't just do DNA sequencing, you need to do metatranscriptome sequencing.

Zymo has a great service where they will do 30 million paired end reads of metatranscriptomic sequencing for $375 from an unprocessed sample (e.g., saliva).

Zymo is very amenable to small projects, and processed my single sample. I did need Zymo DNA/RNA shield ($74) to stabilize the RNA, but I had some from a previous project. The sequencing took around six weeks, and the results look exceptionally clean.

Metatranscriptomic results

The metatranscriptomic analysis found a normal, healthy distribution of bacteria.

Top bacterial species

Species	Abundance		Phylum	Seq identity	Genome coverage
Porphyromonas pasteri	8.5%		Bacteroidota	97.4%	93%
Rothia mucilaginosa	5.8%		Actinobacteriota	98.4%	79%
Rothia sp001808955	2.8%		Actinobacteriota	98.1%	56%
Alloprevotella sp015257125	2.6%		Bacteroidota	97.8%	89%
Prevotella melaninogenica	2.2%		Bacteroidota	98.4%	71%
Actinomyces graevenitzii	2.2%		Actinobacteriota	97.1%	78%
Rothia sp015265375	2.2%		Actinobacteriota	98.2%	47%
Rothia mucilaginosa_B	2.2%		Actinobacteriota	98.2%	47%
Capnocytophaga gingivalis	1.8%		Bacteroidota	97.5%	75%
Neisseria perflava	1.5%		Proteobacteria	98.8%	58%
Streptococcus mitis_BB	1.4%		Firmicutes	99.0%	51%
Alloprevotella sp900095835	1.4%		Bacteroidota	98.3%	79%
Bulleidia sp015256775	1.4%		Firmicutes	98.1%	84%
Rothia aeria	1.3%		Actinobacteriota	98.3%	86%
Gemella sanguinis	1.1%		Firmicutes	97.9%	75%

Top viral species

Virus	Abundance		Note
Tomato brown rugose fruit virus	64%		dietary plant virus (tobamovirus)
uncultured phage	27%		bacteriophage
Human metapneumovirus (HMPV)	9%		real respiratory pathogen → target

The tobamovirus hit is probably from recently eaten food. There is only one human virus in the dataset: human metapneumovirus (HMPV). HMPV is a single-stranded RNA virus with a lipid coat that is one of the most common causes of the common cold. There is no antiviral treatment for HMPV. Like most viruses, the advice is to wait for your immune system to fight it off.

HMPV is usually not serious in older children or adults, but accounts for 5% to 10% of hospitalizations among pediatric patients with acute respiratory tract infections.

Diagram of HMPV from Lianou et al., 2025

Virus sequence

The sequence of the genome is about 3% diverged from the closest reference (FJ168778.1), with 64 missense mutations. It's not that surprising that it all matches a reference sequence so well, but it's still gratifying to see.

The sequence of my HMPV vs a reference sequence

Virus structure

The structure of the HMPV virus is way bigger and more complicated than you would think from the diagram above.

I am used to thinking of viruses as small icosahedra, with a tightly coiled genome inside (see this great article on icosahedral viruses from Asimov Press).

HMPV is pretty different: it's a coiled nucleoprotein, which requires around 1900 "N" proteins to cover the genome, producing a massive structure of hundreds of megadaltons per virion. All of this is squished into a lipid sphere like a ball of yarn.

I used AlphaFold 3 to fold ten nucleoproteins and some RNA from my virus. As you'd expect, given there are good reference structures in PDB, AlphaFold 3 does a fine job folding the nucleoproteins into a coil. In contrast, the RNA has formed a double-stranded hairpin and does not match the crystal structure.

(Left) Ten N proteins from my HMPV in a circular configuration (blue/yellow), with some RNA (orange) wound around. (Right) Eleven N proteins in a spiral configuration (PDB:8PDN)

Viral target

The "F" (Fusion) protein is the obvious target for a therapeutic. It is on the surface of the lipid envelope and mediates cell adhesion and membrane fusion. It has two configurations: pre-fusion and post-fusion. Pre-fusion is the unstable form. When it comes into contact with the host cell, it snaps into the more stable "harpoon" that mediates membrane fusion.

The pre-fusion F protein is compact and the post-fusion F protein is elongated

Luckily, there is a paper (Wen et al., 2012) where the authors created a Fab ("DS7", PDB:4DAG) that binds both the pre-fusion and post-fusion forms. This is the perfect example for us to use as a reference. The sequence of their F protein is 99% similar to ours.

Making a peptide therapy

What if we could design a peptide that binds to the virus and neutralizes it? How hard would that be?

I hear binder design is all the rage these days, so I tried to design a peptide binder. I happened to get some credits for the new BoltzGen API so I decided to try that.

Thanks to the Wen et al. paper, I had a good epitope to go after and a crystal structure of the pre-fusion F protein.

I pointed Claude at the BoltzGen API and asked for a peptide binder of length 20-40, aimed at the DS7 epitope. I spent around $200 on the BoltzGen API, and came up with a length 28 peptide: VKVYDTETPEGYEKWKELARESHGMADV.

Complex	ipTM	ipSAE	iLIS	Notes
Peptide binder + reference pre-fusion F	0.898	0.616	0.563	confident closed-state interface
Peptide binder + sequenced pre-fusion F	0.909	0.635	0.578	confident closed-state interface
Peptide binder + reference post-fusion F	0.159	0.000	0.000	no confident open-state interface

The properties of the binder are pretty good, but not ideal.

The ipTM is high; the ipSAE is relatively high, given the size of the peptide; the iLIS is far into the "confident" range (>0.223), implying a low false positive rate.

One potential limitation is that in theory the pre- and post-fusion forms of the F protein have the same epitope, but when I refold with the post-fusion form it does not appear to bind. In practice, we probably only care about binding the pre-fusion form (before adhesion has occurred).

So I can't say it's definitely a binder, but I think it has a reasonably good shot of binding. Usually, if there is a known binder in PDB, making another binder for the same epitope is not so difficult.

How to make the peptide

There are two main ways to make a peptide: with a ribosome or with chemistry (solid phase synthesis). If you use a ribosome (i.e., translation in a cell or cell-free system), then you need to purify the peptide. For short peptides it's generally easier to synthesize chemically. For example, you can order a peptide from GenScript for around $10-25 per amino acid.

The main advantages of using chemical synthesis are (a) purity: specifically, the lack of endotoxins you get with ribosomal production; (b) the ability to go beyond the simple 20 proteinogenic amino acids.

For this peptide, we may want to add an N-terminal Palmitic acid or a similar fatty acid, which should anchor the peptide in the cell membrane, and prevent it getting flushed as mucus refreshes.

This peptide would cost around $600 and take around 20 business days to arrive

Note, I did not test the binder against the F protein! Maybe I'll do it at Adaptyv at some point just for interest's sake.

Safety

One big open question is whether a peptide like this, sprayed into the nose, would be safe. The main reasons I think it probably would be are that (a) it's extracellular; (b) our noses are exposed to tons of peptides all day (e.g., pollen); (c) if the user experienced irritation, they could stop using it—it doesn't persist.

I did a quick review of the literature, and did not find much on the topic.

Conclusion

It's fun to sequence viruses and design peptide binders, but how would a peptide therapeutic like this actually work in practice?

Detection

First we would need a rapid test that could tell us which virus is present. In theory, sequencing would be best. Oxford Nanopore could do it, but it is still a bit impractical, especially since you'd need RNA, and ideally results within an hour or so.

The most practical thing would probably be an ELISA, similar to the rapid COVID-19 tests. Today you can buy a COVID-19 / Flu A/B / RSV test in the US for around $10. Or, if you go on alibaba, you can buy a 10 in 1 test that includes HMPV for $2.

10 in 1 test kit for "cat, dog, human"(!)

Once you have identified HMPV as the virus, then you would spray the peptide in your nose. Would this actually work post-infection? That is very unclear, though even "protective" sprays like carrageenan do appear to reduce the duration of infection. It is much more likely it could prevent others from getting the virus.

My original idea here was to see if it would make sense to sequence and make a personalized peptide per virus. The answer is probably no, because, as we saw, the viruses are usually not that different, and the steps currently take way too long when a virus can run its course within a week or less.

Instead, we could make a cocktail of peptides to address the top ten common cold viruses. Influenza may evolve too quickly to be included in the panel—it depends on whether we can design a binder to a slowly-evolving part of the virus. Arguably this is all overkill when safe, protective nose sprays exist, but we should do it anyway!