Supported Programs¶

Program Launchers¶

In emwrap, external programs can be defined by specifying “program launchers”. The idea of the launcher is to create a bash script that wraps the program call and sets up the necessary environment. For example, the launcher can load cluster modules, source bash files, or set up environment variables. In that way, the code from emwrap just needs to call the launcher without taking care of local installation details.

Jobs implemented in emwrap itself are launched via emh-tomo --launch MODULE (configured as the default EMWRAP launcher in EMWRAP_CONFIG, typically $ROOT/emh-tomo --launch). Submitting jobs from the Relion GUI uses the relion_dispatcher entry in EMWRAP_CONFIG (typically $ROOT/emh-tomo --submit).

There is a section in the EMWRAP_CONFIG variable related to the launchers:

"programs": {
     "EMWRAP": {"launcher": "$ROOT/emh-tomo --launch"},
     "WARP": {"launcher": "$SCRIPTS/warp_launcher.sh"},
     "PYTOM": {"launcher": "$SCRIPTS/pytom_launcher.sh"},
     "RELION": {"launcher": "$SCRIPTS/relion_launcher.sh"},
     "MOTIONCOR2": {"launcher": "$SCRIPTS/motioncor2.sh"},
     "MOTIONCOR3": {"launcher": "$SCRIPTS/motioncor3.sh"},
     "ARETOMO2": {"launcher": "$SCRIPTS/aretomo2.sh"},
     "ARETOMO3": {"launcher": "$SCRIPTS/aretomo3.sh"},
     "IMOD": {"launcher": "$SCRIPTS/imod_launcher.sh"},
     "CTFFIND": {"launcher": "$SCRIPTS/ctffind5.sh", "version": 5},
     "CRYOCARE": {"launcher": "$SCRIPTS/cryocare_launcher.sh"},
     "DENOISET": {"launcher": "$SCRIPTS/denoiset_launcher.sh"},
     "MISSALIGNMENT": {"launcher": "$SCRIPTS/missalignment.sh"}
 }

After the installation, there is a scripts folder that is created with some of the launcher scripts, but YOU MIGHT NEED TO MODIFY them to work in your environment. In the following sections, there are some examples of launchers.

Warp Launcher¶

In the following example, Warp is loaded from the available modules, together with Aretomo2, version 1.0.0.

#!/bin/bash

PROGRAM=$1
shift

export MODULES="warp/2.0dev33-latest aretomo2/1.0.0"
echo Loading modules $MODULES
module load -s $MODULES

$PROGRAM $@

Or, if we are loading Warp from an SBGrid installation, the launcher could be something like:

#!/bin/bash

PROGRAM=$1
shift

export SBGRID=/programs/sbgrid.shrc
source $SBGRID
echo "Loading Warp from SBGrid file: ${SBGRID}."

$PROGRAM $@

Relion Launcher¶

In the case of the Relion launcher, the first argument is the program name, and the second is the number of MPI processes. The wrapper will take care of adding the _mpi suffix to the program and also the mpirun command. For example:

#!/bin/bash

export SBGRID=/programs/sbgrid.shrc
source $SBGRID
echo "Loading Relion from SBGrid file: ${SBGRID}."

export PROGRAM=$1
shift
export MPI=$1
shift

if [ "$MPI" -eq 1 ]; then
    export CMD="${PROGRAM} $@"
else
    export CMD="mpirun.relion --oversubscribe -np ${MPI} ${PROGRAM}_mpi $@"
fi

echo Running command: ${CMD}
$CMD

Other Launchers¶

emwrap is still under development, and more tools will be integrated in the future. Right now, apart from Warp and Relion, it might be helpful to configure the following launchers:

PyTOM launcher: for 3D template matching
IMOD launcher: for etomo tilt-series alignment

Supported Programs¶

The table below lists the external programs that EMhub-Tomo runs under the hood, the version set in the launcher templates shipped with emwrap (emwrap/config/scripts), and where to find installation instructions. These are the versions used for development and testing. Other versions may work, but they have not been tested. Most programs are also available through SBGrid, which is how several of the example launchers load them.

Program	Config key	Launcher template	Tested version	Installation
Warp / WarpTools / M	`WARP`	`warp_launcher.sh`	2.0.0dev36	Warp installation
RELION	`RELION`	`relion_launcher.sh`	5.1.0-beta (CUDA 13.0)	RELION installation
pytom-match-pick	`PYTOM`	`pytom_launcher.sh`	0.13.2	pytom-match-pick docs
AreTomo2	`ARETOMO2`	`aretomo2.sh`	1.0.0	AreTomo2 on GitHub
AreTomo3	`ARETOMO3`	`aretomo3.sh`	2.2.8	AreTomo3 on GitHub
IMOD	`IMOD`	`imod_launcher.sh`	5.1.9	IMOD download
DenoisET	`DENOISET`	`denoiset.sh`	conda env `denoiset`	DenoisET on GitHub
MissAlignment	`MISSALIGNMENT`	`missalignment.sh`	conda env `miss-alignment`	MissAlignment on GitHub

Note

The Warp tilt-series alignment job (emw-warp-tsalign) also calls AreTomo2/AreTomo3 or IMOD, depending on the alignment method selected, so those launchers must work for the Warp pipeline too.

Supported Jobs¶

The job types currently implemented in emwrap for CryoET processing, grouped by the external package they wrap. This list is generated from ProcessingConfig.get_jobs() and ProcessingConfig.get_packages() (in emwrap/base/config.py); every job listed here ships with a matching form under emwrap/config/forms – see Job Forms for how those forms are structured.

emwrap¶

General-purpose jobs that are not specific to any external package.

Job	Label	Output
`emw-import-ts`	EMwrap Import TS	EMwrap
`emw-subset-ts`	EMwrap Subset TS	EMwrap
`emw-merge-sets`	EMwrap Merge Sets	EMwrap
`emw-missalignment`	MissAlignment	MissAlignment

Warp¶

Jobs wrapping Warp.

Job	Label	Output
`emw-warp-mctf`	Warp Motion and CTF	WarpMctf
`emw-warp-tsalign`	Warp Tilt Series Alignment	WarpTsAlign
`emw-warp-ctfrec`	Warp CTF and Reconstruction	WarpCtfRec
`emw-warp-otf`	Warp OTF Preprocessing	WarpOtf
`emw-warp-export_particles`	Warp Export Particles	WarpExportParticles
`emw-warp-mtools_create`	Warp M Create Population	WarpMCreatePopulation
`emw-warp-mcore`	Warp MCore Refine	WarpMCore
`emw-warp-estimate_weights`	Warp Estimate Weights	WarpEstimateWeights
`emw-warp-mtools_resample`	Warp M Resample	WarpMResample

Relion¶

Jobs wrapping native Relion programs, plus a couple of thin Relion-adjacent utilities.

Job	Label	Output
`relion.reconstructtomograms`	Relion Reconstruct Tomograms	Tomograms
`relion.pseudosubtomo`	Relion Extract Subtomograms	Extract
`relion.reconstructparticletomo`	Relion Tomo Reconstruct Particles	Reconstruct
`relion.initialmodel.tomo`	Relion Tomo Initial Volume	InitialModel
`relion.class3d.tomo`	Relion Tomo 3D Classification	Class3D
`relion.refine3d.tomo`	Relion Tomo Refine	Refine3D
`emw-relion-symmetrize_volume`	(none – falls back to the job id)	–
`emw-relion-mask_create`	(none – falls back to the job id)	–

PyTOM¶

Jobs wrapping PyTOM for 3D template matching.

Job	Label	Output
`emw-pytom-create_template`	PyTom Create Template	PyTom
`emw-pytom`	PyTom Template Matching	PyTom

AreTomo¶

Jobs wrapping AreTomo.

Job	Label	Output
`emw-aretomo3`	Aretomo3	Aretomo3
`emw-denoiset`	DenoisET Pipeline	DenoisET