+25 production workflows for heterogeneous compute

Every computational chemistry group we talk to has the same file sitting somewhere in their repo. It is a shell script, four hundred lines long, and it exists to move data between machines. Copy the topology to the GPU box. Run mdrun. Copy the trajectory back. Hope nothing timed out.

The science in that script is maybe thirty lines. The rest is plumbing.

Today we are open-sourcing pantheon, a library of 27 production workflows for horus-runtime. Every one of them is a real multi-stage pipeline, and every stage picks its own compute target. You can browse the whole library here.

The actual problem is that a pipeline is not one workload

Take a virtual screen. It has three stages, and they want completely different hardware.

Preparing the ligands is a CPU job that finishes in seconds. Predicting binding structure and affinity needs a GPU with a lot of memory, and it is the only stage that costs real money. Ranking the results is a CPU job that finishes in seconds again.

Most tools make you pick one machine for all three. So you rent a GPU node and then let it idle through the cheap stages, or you split the pipeline by hand and write the plumbing yourself. Neither is good. The first wastes money and the second wastes your afternoon.

The awkward part is that the right answer changes constantly. It changes when you move from your laptop to the cluster, when a reviewer asks for ten times the sampling, when the GPU queue is full, and when you switch from x86 to Apple Silicon and half your conda packages stop resolving.

What per-stage routing looks like

In Horus, the compute target is a field on the stage, not a property of the workflow. Here is Boltz-2 virtual screening:

prep (local, CPU)          target.fasta + ligands.smi ──► boltz_inputs.tar.gz
   │   LocalToSSH transfer (automatic)
predict (SSH GPU box)      boltz predict (container)   ──► predictions.tar.gz
   │   SSHToLocal transfer (automatic)
rank (local, CPU)          parse affinity + confidence ──► top_hits.csv

The expensive GPU runs exactly one stage. Horus moves the data across each boundary on its own; there is no scp anywhere in the workflow. Moving that GPU stage to a different machine, into a container, or onto an HPC scheduler is a one-field change, and the command definition does not move at all.

That last part matters more than it sounds. It means the workflow you debug on a laptop with a tiny ligand set is byte-for-byte the workflow you run against the full library on a cluster.

The library

All workflows are MIT-licensed and runnable today with uv run horus run workflow.yaml.

Drug discovery

• Boltz-2 Virtual Screening: co-fold a target with each ligand and rank predicted affinity
• AutoDock Vina Docking: prepare, dock, and rank a ligand library

Molecular dynamics, built on the BioExcel building blocks

• GROMACS MD Setup
• GROMACS Protein-Ligand Complex MD Setup
• AMBER MD Setup
• AMBER Protein MD Setup
• AMBER Protein-Ligand Complex MD Setup
• AMBER Constant pH MD Setup
• ABC DNA MD Setup
• Ligand Parameterization
• Mutation Free Energy Calculations
• Protein-Ligand Docking with fpocket
• Protein-Ligand Docking (Cluster90 Binding Site)
• Protein-Ligand Docking (PDBe REST API Binding Site)
• HADDOCK3 Antibody-Antigen Protein-Protein Docking
• DNA Helical Parameters Analysis
• Protein Conformational Ensembles
• Protein Conformational Transitions with GOdMD
• Macromolecular Coarse-Grained Flexibility with FlexServ
• Classical Molecular Interaction Potentials with CMIP
• Molecular Structure Checking and Repair
• AutoEncoders for MD Trajectory Analysis
• Protein-Membrane MD Analysis

Engine capabilities, small workflows that each demonstrate one runtime feature

• Fan-out, Map and Gather
• Programmatic Dynamic DAG
• Bounded Loop with Range Map
• Subworkflow Reuse

A note on ARM64

Several of these were genuinely annoying to get running on Apple Silicon, because biobb_amber, biobb_cmip, biobb_flexserv, and friends have no native arm64 conda build. The fix is the same mechanism as everything else: those stages declare a Docker executor while the neighbouring Python stages stay in a native conda environment, and the workflow does not otherwise change. The per-workflow pages say which stages need this.

Where this is going

pantheon is the open-source half. The other half is Temple Compute OS, which runs these workflows on managed compute so you do not have to own the GPU box or maintain the cluster in the first place. It is in early access and you can ask for beta access.

If you run one of these and it breaks, open an issue. If you have a pipeline that should be in the library, open a PR.