Reproducing published studies

Each MLPF publication uses a particular detector simulation, target definition, model, and software release. To reproduce a study, start with its archived code and data, then follow the installation and evaluation instructions from that release. The publications page explains the scientific results; this guide connects them to the available artifacts.

Code, dataset, and checkpoint versions are independent. Keep their identifiers together with the model configuration, environment, and evaluation selection in the experiment record. The current dataset catalog and training guide provide the starting point for new studies.

Choosing a release¶

From Delphes to CMS¶

The 2021 proof of concept provides an openly archived simulation dataset and a matching v1.1 code release. The dataset contains 50,000 top-pair and 5,000 QCD events at 14 TeV with average pileup 200, stored as compressed Python pickles. It includes Pythia and Delphes cards and a loading notebook. The paper reports training on 40,000 top-pair events.

Start with README_delphes.md at v1.1 and parameters/delphes-gnn-skipconn.yaml. These instructions cover conversion to TFRecord, training, evaluation, and timing.

The subsequent CMS implementation uses v1.4, explicitly cited in the proceedings. It trains a TensorFlow 2.6 GNN on reference PF candidates and exports the model to ONNX for validation within CMSSW. The paper describes a randomized 80:20 training/test split. The v1.4 CMS instructions obtain training data from a personal EOS directory through lxplus.cern.ch, so this route requires CERN access. They also describe sample generation using local pileup files and a matching CMSSW release.

The 2022 CMS study introduces simulation-derived particle targets. Its accompanying proceedings describe training samples, event-loss scans, and validation, and specify CMSSW_12_3_0_pre6 together with minimum-bias and validation dataset names. A corresponding MLPF code tag and public event archive remain to be identified before assembling a complete reproduction setup.

Full-simulation CLIC reconstruction¶

The 2024 CLIC study provides separate archives for simulation, processed training inputs, and trained models with evaluation results. Choose the artifact according to the stage you want to reproduce.

For simulation and preprocessing, the raw-data record, version 1.1, contains small samples and a dataset_full.txt manifest pointing to bulk data at Jülich. For training, the processed record, version v1.5.0, contains four TFDS archives. The README at v2.3.0 links this processed dataset to the study.

For evaluation, start with the results archive, version v2024.01. Its mlpf-clic-2023-results.tgz contains trained weights, configurations, evaluation outputs, hyperparameter-search summaries, timing, and accelerator-scaling results.

The paper cites code archive 10893930. The repository also links 10928968. Both contain the same v1.6.2 source: all 289 files match after removing the top-level archive directory, and both identify commit cd2ed4100d928193b1fe71910c2819d8b4fa8d5d. Use this version when following the paper; the related GitHub link to v1.6 in record 10893930 is stale.

Fine-tuning from CLIC to CLD¶

The 2025 transfer-learning study pairs code v2.3.0 with CLIC pretraining data and CLD downstream data. The paper describes 24 million CLIC events and 4.2 million CLD events, each divided into training and validation with a 90:10 split.

The cited archives provide subsets of those samples. The CLIC TFDS record contains two of ten subsets per sample. Its dataset label is v2.5.0, and its description explicitly declares compatibility with code v2.3.0. The CLD TFDS record describes approximately 800,000 events. To work from simulation output, the CLD ROOT record contains approximately 50,000 events together with generation and reconstruction settings.

These deposits support working with the study’s data representation. Reproducing the full training-statistics scans also requires establishing access to the remaining samples and their split assignments.

CMS Run-3 reconstruction¶

The published 2026 CMS paper cites v2.5.0. It describes the transformer training environment, including PyTorch 2.7.1 with ROCm 6.2.4, and the conditions used to measure inference performance. Use those settings together with the archived release when reconstructing the software environment.

The journal availability statements refer to CMS data-preservation policy and provide no associated event-data release. The code statement links the public CMSSW repository. Access to the study’s event samples therefore needs to be established separately from the MLPF code checkout.

An earlier v2.4.0 archive describes a preapproval freeze and links training and validation material. It records an earlier stage of the analysis; the published paper’s code reference is v2.5.0.

Verification and version identifiers¶

The links and release mappings were checked on 2026-10-30 against the cited papers, Zenodo metadata and file inventories, and historical repository files. Both CLIC code archives were downloaded, checksum-checked, and compared. Dataset payloads, trained models, external bulk-data hosts, and historical training runs were outside the verification scope.

The repository tags resolved to the following commits: