2D-TPP analysis toolkit (ProSAP + TPP2D)

New here? Start with GETTING-STARTED.md — a step-by-step walkthrough (install ProSAP, set up the R runtime, search in Mascot, build the input file, run the analysis) with screenshots.

Tools, documentation, and an example dataset for running 2D thermal proteome profiling (2D-TPP) analyses with ProSAP and the TPP2D method. There is a script, prosap/Setup-CustomR.cmd` to download and build the correct R runtime and libraries for you.

The converter turns Mascot Daemon quantitation summaries or Mascot Server search results into clean TPP2D long-format files.

From a Mascot search to a result, in three steps

Generate the analysis-ready input straight from the Mascot result files, then point either analysis method at it:

1. Mascot results → per-protein TMT table. scripts/msr_to_quant_summary_pop.py reads the merged .msr results, imports the Percolator FDR from the cache .target.pop, and sums reporter ions per protein — including protein-family members (homologues clustered under a representative, which a plain getHit() loop silently drops). One .txt per sample.
2. Per-protein table → long-format input. scripts/daemon_to_tpp2d_2dplex.py reshapes those tables into the TPP2D long file using a small (sample, channel, concentration, temperature) mapping table.
3. Run either method on the same file. scripts/run_tpp2d_fit.R (TPP2D — F-statistic + bootstrap FDR, headless, no ProSAP GUI) or scripts/run_nparc_fit.R (NPARC). Then use the viz/ scripts (see viz/README.md) to draw the volcano, dose-response and per-protein heatmaps.

The same long-format file feeds both TPP2D and NPARC, so you can run them side by side. See scripts/msr_to_quant_summary_config.json for the layout.

---

What's in here

TPP2D/
├── README.md                     ← you are here (setup + end-to-end workflow)
├── tpp2d_config.json             ← example converter config (copy of scripts/tpp2d_config.json; uses example-data/)
├── scripts/
│   ├── daemon_to_tpp2d.py        ← Daemon .txt  ->  TPP2D long-format CSVs (stdlib only) — layout: one plex = one conc x 10 temps
│   ├── tpp2d_config.json         ← daemon_to_tpp2d.py: settings block + input manifest
│   ├── daemon_to_tpp2d_2dplex.py ← table-driven converter for the Becher layout (one plex = 2 temps x conc series, e.g. PXD038312)
│   ├── daemon_to_tpp2d_2dplex_config.json ← daemon_to_tpp2d_2dplex.py: table path + per-sample input manifest
│   ├── result_to_quant_summary.py ← PATH A: one Mascot result (<resulturl> or path) -> per-protein quant table; standalone or a Daemon "Execute after search" task
│   ├── mascot_to_tpp2d_input.py  ← ONE command: .msr results + sample-map CSV -> the TPP2D input file (chains the two steps below)
│   ├── mascot_to_tpp2d_input_config.json
│   ├── msr_to_quant_summary_pop.py ← per-protein TMT table from Mascot .msr (Percolator .target.pop when reachable, else decoy-FDR fallback; includes protein-family members)
│   ├── msr_to_quant_summary_config.json ← msr_to_quant_summary_pop.py: sample->.msr map + optional FDR/cache overrides
│   ├── run_tpp2d_fit.R              ← STEP 2a: headless TPP2D fit (F-statistic + bootstrap FDR) — no ProSAP GUI needed
│   ├── run_tpp2d_fit_config.R       ← run_tpp2d_fit.R: input/output, bootstrap rounds, FDR, cores
│   ├── run_nparc_fit.R              ← STEP 2b: headless NPARC fit (same input file) — the conservative companion method
│   ├── run_nparc_fit_config.R       ← run_nparc_fit.R: input/output, extra lib path, FDR
│   ├── daemon_to_prosap.py       ← 1D TSA companion (Daemon -> ProSAP wide tables)
│   ├── daemon_to_prosap_config.json ← daemon_to_prosap.py: filter thresholds + channel map
│   ├── verify_no_crash.R         ← QC: confirm a long file won't crash ProSAP's fit
│   └── verify_no_crash_config.R  ← verify_no_crash.R: raw_dir + files + options (sourced; base R, no jsonlite)
├── viz/                          ← figures from the fit result CSVs (run after the analysis)
│   ├── README.md                 ← how to use the figure scripts (start here)
│   ├── tpp2d_viz.py              ← volcano / dose-response / heatmap figures
│   ├── tpp2d_viz_config.json     ← tpp2d_viz.py: inputs, thresholds, colours, style
│   ├── tpp2d_extremes.py         ← stabilizers vs destabilizers heatmap
│   ├── tpp2d_extremes_config.json ← tpp2d_extremes.py: inputs, row count, colour limits, style
│   ├── tpp2d_per_protein_heatmaps.py ← per-protein 2D heatmap grids
│   ├── tpp2d_per_protein_heatmaps_config.json ← tpp2d_per_protein_heatmaps.py: inputs, grid sizes, style
│   ├── nparc_viz.py              ← NPARC volcano + TPP2D-vs-NPARC comparison figures
│   ├── nparc_viz_config.json     ← nparc_viz.py: inputs, gene map, output paths
│   ├── extdatafig3b_replot.py    ← per-protein 2D heatmaps of named proteins, at a fixed publication scale and at data-driven limits
│   ├── stabilized_heatmaps.py    ← same panel style for the top hits in a chosen direction (STABILIZER/DESTABILIZER)
│   ├── tpp2d_common.py           ← shared label/gene-name helper imported by the tpp2d_viz* scripts
│   ├── tpp2d_common.json         ← shared per-experiment labels: ligand/drug, unit, species, controls
│   └── uniprot_to_genename.py    ← UniProt accession → gene-name lookup (builds a gene map for the figures)
├── docs/
│   ├── how-to-run-in-prosap.md   ← step-by-step ProSAP TPP2D run + combined-vs-replicate
│   ├── interpreting-results.md   ← result columns, FDR / F-statistic guidance
│   ├── normalisation-and-noise.md ← why the converters normalise, what each setting does, and the measurements behind the defaults
│   ├── daemon_to_tpp2d-design.md ← converter design + the crash root-cause analysis
│   ├── prosap-bugs.md            ← known ProSAP UI bugs
│   └── prosap-pending-patches.md ← patches to apply when running ProSAP from source
├── prosap/
│   ├── setup-r-packages.R        ← STEP 3 core: installs TPP2D/NPARC + deps (platform-independent; both setups call it)
│   ├── Setup-CustomR.cmd / .ps1  ← STEP 3 (Windows): download R 4.1.0, then run setup-r-packages.R (run once)
│   ├── setup-customr.sh          ← STEP 3 (Linux/macOS): use an R 4.1.x, then run setup-r-packages.R
│   ├── R-4.1.0/                  ← created by Setup-CustomR on Windows; NOT shipped in the repo
│   ├── Launch-ProSAP.cmd         ← starts ProSAP with R_HOME pinned to R-4.1.0 (Windows)
│   ├── Launch-ProSAP-Debug.cmd   ← same, mirroring output to prosap-debug.log (Windows)
│   └── Launch-ProSAP.sh          ← starts ProSAP with R_HOME pinned to your R 4.1.x (Linux/macOS)
├── GETTING-STARTED.md            ← the step-by-step walkthrough (start here)
└── example-data/
    └── PXD038312-2dplex-daemon-quant-summary/  ← worked Vibrio cholerae 2D-TPP example
                                                  (real PXD038312 quant summaries + sample-map CSV)

Each script reads its settings from the config file beside it (the *_config.json, or verify_no_crash_config.R for the R script) — input/output paths, thresholds, and plot style. Pass a different config path as the first argument to override the default.

---

Prerequisites

Need	For	Notes
Python 3.9+	the input-file scripts	Standard library only for the quant-summary converters (daemon_to_tpp2d.py, daemon_to_tpp2d_2dplex.py, daemon_to_prosap.py). Invoke as python, python3, or py -3 (examples below use python).
msparser (Matrix Science SDK)	the .msr route only (mascot_to_tpp2d_input.py → msr_to_quant_summary_pop.py)	Not on PyPI. It ships with Mascot Server, or download from https://www.matrixscience.com/msparser_download.html and add it to PYTHONPATH. It reads the .msr files directly off disk — no Mascot Server URL or login is needed.
numpy, pandas, matplotlib, scipy	the viz/ figure scripts only	Install via pip or conda — see viz/README.md.
ProSAP	running the analysis	Installed separately (Windows app) — see below.
R 4.1.x	ProSAP's TPP2D engine	Bundled here as prosap/R-4.1.0 with all packages. Do not use a newer system R — ProSAP's rpy2 needs 4.1.x.

Platform note: examples use python and forward-slash paths, which work on Windows, macOS, and Linux. On Windows you may need py -3 instead of python, and either slash style works in PowerShell/cmd. ProSAP itself is a Windows application; the converter and viz scripts are cross-platform.

Python must be on your PATH. If python (or py) prints nothing / says "not recognized" / opens the Microsoft Store, the interpreter isn't on your PATH. On Windows, either re-run the python.org installer and tick "Add python.exe to PATH", or add it manually — see Using Python on Windows → PATH and the py launcher. (Tip: py -3 from the bundled launcher works even when bare python is just the do-nothing Microsoft Store alias.)

---

One-time setup

1. Install ProSAP

Download and install ProSAP from https://github.com/hcji/ProSAP (releases). The Windows installer places it at %LOCALAPPDATA%\Programs\ProSAP\ProSAP.exe (per-user). If you install it elsewhere, edit the last line of prosap/Launch-ProSAP.cmd accordingly.

2. Use the bundled R (not the system R)

ProSAP talks to R through rpy2, which needs R 4.1.x plus the TPP2D/TPP/ggplot2/DESeq2 packages. A newer system R (e.g. 4.5) will not work. prosap/R-4.1.0 is a complete, pre-loaded R 4.1.0 — keep it and launch ProSAP through the provided .cmd, which:

• sets R_HOME to prosap/R-4.1.0 (relative to the script),
• puts that R first on PATH,
• pins the library paths and neutralises any personal ~/.Renviron / ~/.Rprofile so they can't shadow the bundled packages.

Building your own R instead of using the bundle: install R 4.1.x and the Bioconductor packages, e.g. in R BiocManager::install("TPP2D"), or via conda conda create -n prosap-r -c conda-forge -c bioconda r-base=4.1 bioconductor-tpp2d. The bundled prosap/R-4.1.0 is the tested path; a hand-built one must still be pinned via R_HOME in the launcher.

3. Launch ProSAP

Run prosap/Launch-ProSAP.cmd (not the Start-menu shortcut — that wouldn't pin the R runtime). Use Launch-ProSAP-Debug.cmd if you need a log of R/rpy2 errors.

4. (Optional) Python environment for the figure scripts

The converter needs only a Python 3.9+ interpreter (standard library). The tpp2d_viz* scripts additionally need three packages — install them however you manage Python:

pip:

python -m pip install numpy pandas matplotlib

conda / mamba:

conda create -n tpp2d python=3.11 numpy pandas matplotlib
conda activate tpp2d

---

Workflow

Mascot Daemon .txt exports
        │   scripts/daemon_to_tpp2d.py   (sanitises + reshapes)
        ▼
TPP2D_long.csv  /  TPP2D_long_perrep.csv
        │   ProSAP → TPP2D module (Data → Confirm)
        ▼
hit table + volcano  →  Save  →  *_fdr.csv / *_hits.csv
        │   viz/*.py   (optional figures — see viz/README.md)
        ▼
publication figures

Step 1 — collect input files

The bundled worked example uses the common two-temperatures-per-plex (Becher) 2D-TPP layout: each TMT10 plex multiplexes 2 temperatures × a concentration series. You provide one Mascot result (or Daemon quant summary) per plex (S-number) plus a small mapping table (experiment, sample, channel, concentration, temperature) that carries all the (temperature, concentration) semantics, and use scripts/daemon_to_tpp2d_2dplex.py (or the one-command scripts/mascot_to_tpp2d_input.py, which reads the .msr files directly). The example lives in example-data/PXD038312-2dplex-daemon-quant-summary/ — real Vibrio cholerae PXD038312 quant summaries plus the sample-map CSV template; see its README.md.

One-temperature-per-plex layout instead? If each plex is a single concentration across 10 temperatures, export one Daemon quantitation summary (.txt) per (replicate, concentration) into one folder and use scripts/daemon_to_tpp2d.py with tpp2d_config.json (point raw_dir at your folder). The output CSV is the same TPP2D long format, so the analysis steps below are identical.

⚠️ Mascot Distiller peak picking — check the reporter-ion region. If you use Mascot Distiller for the peak picking, look at the reporter-ion region of a few MS/MS spectra and adjust the peak-picking options: lower the MS/MS Minimum peak width (Da) parameter to 0.0005 if needed, so it correctly captures very narrow reporter-ion peaks. On high-resolution Orbitrap data the TMT reporters can be ~0.005 Da wide — right at Distiller's default lower limit for a real (non-noise) peak — so at the default setting whole reporter channels are dropped from the peak list and silently read as zero downstream. That corrupts the quantitation without any error. Lowering the minimum peak width recovers all the reporter channels. See docs/normalisation-and-noise.md for the full diagnosis and evidence.

Step 2 — build the TPP2D long files

Copy tpp2d_config.json, edit raw_dir (input folder), out_dir, and the file lists, then run:

python scripts/daemon_to_tpp2d.py your_config.json

To try it on the bundled example (writes to example-data/output/):

python scripts/daemon_to_tpp2d.py tpp2d_config.json

(Windows: use py -3 if python isn't on your PATH.)

This produces two files:

• TPP2D_long_perrep.csv — replicates kept separate (rep1/rep2). Use this for real hit calling (proper, replicate-aware FDR).
• TPP2D_long.csv — replicates merged (exp1). A quick look, or for salvaging a run where one replicate is bad.

Both are normalised and sanitised. The defaults are sensible; the knobs live in the config's settings block and are covered in docs/normalisation-and-noise.md, with the reasoning in docs/daemon_to_tpp2d-design.md.

setting	default	what it does
median_normalize_fc	true	Median-normalises fold changes per (temperature, concentration) cell, correcting per-channel loading. Leave this on — without it, TMT loading differences read as ligand effects.
missing_policy	"floor"	"drop" omits absent measurements instead of writing floor_frac x reference for them. Prefer "drop": a protein that has melted out is genuinely undetectable, and flooring invents a ~1000x depletion exactly where stabilisation is measured.
min_obs	20	Minimum real observations per protein (used when missing_policy is "drop").
min_concs	3	Minimum distinct concentrations per protein. TPP2D's H1 model fits a slope and a pEC50; two dose points cannot determine both, and the underdetermined fit crashes .getFitDf.
normalize_channels	true	(2dplex converter only) Corrects channel loading within each temperature group, against that group's vehicle.
reference_mode	"within_plex"	(concentration-compressed converter only) How rel_value is referenced. See below — the default is the low-noise choice.

A note on reference_mode, because it is the one that changes results most. "within_plex" references each channel to its own plex's ref_temp channel; "vehicle" references it to the vehicle plex at the same temperature (the Becher-style fold change TPP2D assumes). Matching TPP2D's semantics sounds preferable, but in this layout the vehicle lives in a different TMT run, so "vehicle" compares raw abundances across runs and nothing cancels. Measured head-to-head, "within_plex" is about 3x quieter (SD 0.337 vs 0.975 in log2; 1.8% vs 16.7% of proteins beyond 2-fold), because its double ratio cancels per-protein per-run effects. Keep the default unless you have a specific reason.

(Optional QC) confirm a long file won't crash ProSAP, using the bundled R (or any R 4.1.x with TPP2D):

prosap/R-4.1.0/bin/x64/Rscript.exe scripts/verify_no_crash.R

(edit scripts/verify_no_crash_config.R — raw_dir and files — to point at your CSVs, or pass a different config path as the first argument)

Step 3 — run the analysis in ProSAP

Open the TPP2D module → Data → pick a long-format CSV → set Maximum Iter (200 is fine), Bootstrap Round (≥ 20 for a reportable FDR — the default of 2 is too low), Significance Threshold (0.1 = 10 % FDR) → Confirm. Then Save the hit table. Full walkthrough: docs/how-to-run-in-prosap.md.

Step 4 — interpret

What each result column means, what a good-quality measurement looks like, and the recommended FDR / F-statistic guidance: docs/interpreting-results.md. (Short version: call hits at FDR ≤ 0.1; the F-statistic is a within-run ranking, not an absolute cut-off.)

Step 5 — figures (optional)

The figure scripts live in their own folder, viz/, with a dedicated guide: viz/README.md. They read the long CSV plus the FDR/hits tables (TPP2D_fdr.csv, TPP2D_hits.csv, and the NPARC tables) and emit volcano / dose-response / heatmap PNGs. They need numpy, pandas, matplotlib, and scipy. Filenames in each config resolve against the viz/ folder unless absolute, so the simplest path is to copy your result CSVs into viz/.

The highlights (full detail in viz/README.md):

• Experiment labels are not hardcoded. Ligand name, concentration unit, species, and labelled control proteins live in one shared file, viz/tpp2d_common.json — set them once and every figure follows. Controls are keyed by clustername (the gene name). Defaults match the PXD038312 D-Arginine / Vibrio cholerae example.
• Which proteins each figure reports is configurable via a selections block of { source, filter, sort, count } rules over the hits/fdr tables — picked on the fly by criteria, never a hardcoded protein list, so they work on any dataset.
• Gene names label figures out of the box (the clustername is already the GN= gene name). Only if your clusternames are bare accessions do you need an accession→gene map:

python viz/uniprot_to_genename.py --file accessions.txt --output genes.tsv

then set "gene_names": { "enabled": true, "map": "genes.tsv" }. This needs the requests package (python -m pip install requests). Proteins missing from the map fall back to their accession, so nothing is ever unlabelled.

---

Input / output locations at a glance

File kind	Where	Tracked in git?
Quant summaries / .msr inputs	wherever your config points (example: example-data/PXD038312-2dplex-daemon-quant-summary/)	example set: yes
TPP2D_long*.csv (converter output)	out_dir (example: example-data/output/)	no — regenerate with the converter
ProSAP results (*_fdr.csv, *_hits.csv)	wherever you Save them	no
Figures (fig*.png)	beside the viz scripts / your run folder	no

(See .gitignore — generated artefacts are excluded so the repo stays inputs + tools + docs + runtime.)