openff-stats

Stats for OpenFF software packages and publications.

Note: I've only manually checked the Google Scholar numbers for citations. Google Scholar citations require Selenium + Firefox; this is essentially hardcoded to work on my machine (tm), no guarantees it'll work for you. Expect to solve the occasional CAPTCHA by hand in the Firefox window it opens (see the methodology section).

Date last updated: 2026-07-05

Contents

• Contents
• How it works
• Tutorial: add a publication and check its citations
• How to add a new source
  • Examples: querying DOIs
  • Google Scholar lookup by DOI
  • Verifying a DOI or a match by its link
• inputs/ file formats
• Data collection
• Optional bulk discovery
• Methodology and caveats — read before using these numbers
  • Publication citations (data/citations.csv)
  • Zenodo citations (data/zenodo_citations.csv)
  • Conda-forge download counts (data/downloads.csv, data/downloads_yearly.csv)
  • GitHub repo imports and stars (data/github_repo_stars.csv)
• Installation
• CI

How it works

Every source is a manually curated CSV in inputs/. Each source kind is a directory of CSVs where the filename is the group classification — e.g. inputs/publications/force-field.csv holds the force-field papers. Collection commands read every CSV in the directory, tag rows with their group, and print cumulative sums per group. Nothing is scraped into the pipeline automatically — you decide what goes in inputs/.

inputs/publications/<group>.csv   ← papers (DOIs) to track citations for
                                    (force-field.csv, general.csv, …)
inputs/packages/<group>.csv       ← conda-forge packages to track downloads for
                                    (openff.csv, competitor.csv)
inputs/zenodo/<group>.csv         ← Zenodo records to track DataCite citations for
                                    (general.csv, qcsubmit.csv, yammbs.csv, …)
inputs/github_packages.csv        ← curated packages to sweep for GitHub repos
                                    (package, import_name, search_mode)
inputs/github_owner_blacklist.csv ← owners excluded from GitHub adoption counts
                                    (owner, reason — conda-forge, self, …)

data/citations.csv             ← Crossref / OpenAlex / Semantic Scholar /
                                    Google Scholar / ChemRxiv counts, with group
data/downloads.csv             ← total downloads per package, with group
data/downloads_yearly.csv      ← per-package per-year downloads, with group
data/zenodo_citations.csv      ← DataCite citations, long form (one row per
                                    record-year; citation_count = record total)
data/github_repos/<package>.csv ← script-generated repo candidates per package
                                    (status = auto / exclude / manual, evidence in notes)
data/github_repo_stars.csv     ← star counts for the curated repos, with group

Tutorial: add a publication and check its citations

Walkthrough of the full flow for a new paper, from DOI to citation counts.

1. Add the paper. Fetches Crossref metadata (title, authors, year), verifies the DOI resolves, de-dupes against every group file, and appends to inputs/publications/<group>.csv (default group: general; pass --group force-field for a force-field paper):

   openff-stats add-publication-doi "10.1021/acs.jpcb.4c01558" --group force-field

2. Check the row in inputs/publications/<group>.csv. Confirm the metadata looks right. scholar_cluster_id and chemrxiv_id are still blank at this point — that's expected.

3. Get the Google Scholar cluster ID. Either pass --scholar in step 1 to fetch it automatically (best-effort — the paper is still added if Scholar fails to match), or fetch it separately and save it:

   openff-stats scholar-lookup "10.1021/acs.jpcb.4c01558" --save

   This searches Scholar by DOI, falls back to the title, and only saves a confident match. If nothing matches confidently, it prints candidate links instead — see Google Scholar lookup by DOI below for picking one by hand.

4. Run the citation check. Collects Crossref, OpenAlex, Semantic Scholar, Google Scholar (via the cluster ID), and ChemRxiv citation counts for every row of every group file in inputs/publications/, writes data/citations.csv (with a group column), and prints cumulative sums per group and overall:

   openff-stats citations

   (openff-stats run-all runs this plus every other collector in one go.)

How to add a new source

One command per source type. Each fetches metadata, de-duplicates, and appends a row to the curated CSV — so adding a source is a one-liner, and you can always hand-edit the CSV afterwards.

All add-* commands take --group NAME to choose the target file (inputs/<kind>/<NAME>.csv); the duplicate check always spans every group, so a source can only live in one group.

# Any DOI(s) — Zenodo DOIs are auto-detected by their 10.5281/zenodo. prefix
# and routed to inputs/zenodo/<group>.csv; everything else is a publication.
openff-stats add-doi "10.1021/acs.jcim.2c01153" "10.5281/zenodo.5503442"

# A paper (Crossref metadata). Add --scholar to also fill the Scholar cluster ID.
openff-stats add-publication-doi "10.1021/acs.jpcb.4c01558" --group force-field

# A GitHub repo (OWNER/REPO or a github.com URL), added by hand as status=manual;
# --group is the package it imports (default: openff-toolkit). Everything else in
# data/github_repos/ is script-generated by `discover-github-repos` (see below).
openff-stats add-github-repo openforcefield/openff-toolkit

# A Zenodo record (numeric ID, DOI, or record URL).
openff-stats add-zenodo 10.5281/zenodo.18842670 --group qcsubmit

# A conda-forge package: edit inputs/packages/<group>.csv directly (one
# `package` column; the filename is the group, e.g. openff.csv).

Then run openff-stats run-all to refresh the stats.

Examples: querying DOIs

Query a single DOI without adding anything:

# What does this DOI point to? (registered title + publisher, via doi.org)
openff-stats verify-doi "10.1021/acs.jcim.2c01153"

# What are its Google Scholar citations? (prints the match and Cited-by count)
openff-stats scholar-lookup "10.1021/acs.jcim.2c01153"

Add a single DOI — add-doi works for both papers and Zenodo records:

openff-stats add-doi "10.1021/acs.jcim.2c01153"     # → inputs/publications/general.csv
openff-stats add-doi "10.5281/zenodo.5503442"       # → inputs/zenodo/general.csv (Zenodo prefix)

Add a whole list from a text file into its own group (one DOI per line; blank lines and # comments are skipped; mixed papers + Zenodo records is fine — each lands in the right directory under the same group name). For example, DOIs extracted from the qcsubmit and yammbs repos:

openff-stats add-doi --file ~/Downloads/qcsubmit_extracted_dois.txt --group qcsubmit
openff-stats add-doi --file ~/Downloads/yammbs_extracted_dois.txt --group yammbs

Each DOI is reported as it's processed (added / already present / FAILED); failures don't stop the batch, and already-present DOIs are skipped, so re-running the same file is safe.

Then collect citation counts for everything curated — each command prints cumulative sums per group as well as the overall total:

openff-stats citations          # papers   → data/citations.csv
openff-stats zenodo-citations   # Zenodo   → data/zenodo_citations.csv

Google Scholar lookup by DOI

Scholar citation counts need a scholar_cluster_id per paper (Scholar's internal ID for the group of all versions of a paper). Look one up by DOI:

# One DOI — print the matched paper, its cluster ID, and Cited-by count:
openff-stats scholar-lookup "10.1021/acs.jpcb.4c01558"

# ...and write the cluster ID into whichever inputs/publications/*.csv holds the DOI:
openff-stats scholar-lookup "10.1021/acs.jpcb.4c01558" --save

# Every DOI at once — fill scholar_cluster_id for all rows still missing one:
openff-stats scholar-clusters                     # add --overwrite-existing to redo all

Both search Scholar for the DOI, fall back to the paper's title, and validate every candidate against that title — only a confident match is saved, so you won't silently store the wrong cluster. scholar-lookup prints the candidates when nothing matches confidently so you can pick one by hand (paste the ID into the scholar_cluster_id column); scholar-clusters leaves those rows blank and reports the count.

Verifying a DOI or a match by its link

scholar-lookup prints a clickable https://doi.org/… link and a https://scholar.google.com/scholar?cluster=… link for each candidate, so you can open them and eyeball the match. Add --open to launch the DOI page and the best-match Scholar page in your browser automatically.

To just check that a DOI resolves and see what it points to (its registered title and publisher, via doi.org — no scraping):

openff-stats verify-doi "10.1021/acs.jpcb.4c01558"

add-publication-doi runs this resolution check automatically before adding a paper, so a typo'd or dead DOI is rejected rather than stored (skip with --no-verify).

Manual alternative: search the title on https://scholar.google.com, click Cited by on the right result, and copy the number after cluster= (or cites=) in the URL.

inputs/ file formats

Each kind is a directory; the CSV filename is the group classification.

Directory	Columns per CSV
publications/	DOI, title, authors, year, scholar_cluster_id, chemrxiv_id
packages/	package
zenodo/	zenodo_id, doi, title, creators, publication_year, resource_type

scholar_cluster_id / chemrxiv_id may be blank. Group names are load-bearing: e.g. papers in publications/force-field.csv are what the force-field citation subtotal counts.

GitHub repo lists aren't hand-curated the same way — see GitHub repo imports and stars below for inputs/github_packages.csv, inputs/github_owner_blacklist.csv, and the script-generated data/github_repos/<package>.csv files.

Data collection

openff-stats citations          # Crossref + OpenAlex + Semantic Scholar + Google Scholar + ChemRxiv
openff-stats downloads          # conda-forge downloads (Anaconda + condastats), sums per group
openff-stats zenodo-citations   # DataCite citations for Zenodo records, sums per group
openff-stats github-stars       # star counts + repo-import counts per group (needs GITHUB_TOKEN)
openff-stats github-descriptions # descriptions + auto topic category (needs GITHUB_TOKEN)

openff-stats run-all            # everything above except github-descriptions

All commands take --help and accept --input / --output overrides. Conda-forge download sums and GitHub repo-import sums are reported separately, each broken down by group.

Optional bulk discovery

These help you find candidate sources; most write to candidates/ (which is gitignored) and never feed collection directly. Review the output, then add approved rows to inputs/ (or use the add-* commands above).

openff-stats discover-publications --orcid-csv inputs/orcids.csv
openff-stats discover-packages
openff-stats discover-dependents          # conda-forge reverse deps of openff-toolkit
openff-stats discover-zenodo

GitHub repo discovery is different: it verifies every candidate itself and writes straight to data/github_repos/<package>.csv — there's no candidates/ review step for this one.

openff-stats discover-github-repos                   # sweep every package in inputs/github_packages.csv
openff-stats discover-github-repos --package descent # one package (import name + search mode read from the CSV)

See GitHub repo imports and stars below for how candidates are verified and tagged.

Methodology and caveats — read before using these numbers

How each number is measured, and why none of them should be quoted without the caveats. Every count is re-fetched from scratch on each collection run.

Publication citations (data/citations.csv)

One row per curated paper in inputs/publications/, with one column per source. A blank cell means the lookup failed or the paper has no ID for that source — blank is "unknown", never 0.

Column	How it's measured
crossref_citations	The is-referenced-by-count field from the Crossref REST API (api.crossref.org/works/<DOI>): the number of DOI-to-DOI citation links deposited with Crossref. Reliable but a lower bound — citing works whose publishers don't deposit reference lists are missed.
openalex_citations	The cited_by_count field from the OpenAlex API (api.openalex.org/works/<DOI>). OpenAlex aggregates Crossref plus other indexes, so it typically sits between Crossref and Google Scholar. Free, no key, full coverage — the most robust "higher than Crossref" number.
semantic_scholar_citations	The citationCount field from the Semantic Scholar Graph API (api.semanticscholar.org/graph/v1/paper/DOI:<DOI>). Coverage of the chemistry literature is patchy: some of our papers are missing from the corpus entirely (a note is printed) and some counts run well below the other sources — treat as a cross-check, not a headline number.
scholar_citations	The "Cited by N" count scraped from the paper's Google Scholar cluster page (scholar.google.com/scholar?cluster=<scholar_cluster_id>) with Selenium + Firefox. Cluster IDs are matched by DOI/title search and validated against the Crossref title before being stored (see Google Scholar lookup by DOI); collection only ever reads the stored IDs. Scholar counts are the highest of Scholar/Crossref/Scopus (they include preprints, theses, etc.) and fluctuate over time; scraping is fragile, and any failure leaves the cell blank. Requests are paced ~10 s apart, a blank result is retried up to 5 times with a 15 s cooldown before each retry, and a warning is printed for any paper still blank after all retries.
chemrxiv_views, chemrxiv_downloads, chemrxiv_citations	The "Abstract Views", "Content Downloads", and "Citations" metrics from the ChemRxiv public API (/items/<chemrxiv_id>). Preprints only; the citation number is ChemRxiv's own and counts citations of the preprint, not the published paper.

The three broad-coverage sources measure different things. Google Scholar casts the widest net — it counts citations from preprints, theses, books, and other grey literature — so its numbers run the highest, but they come from scraping (fragile, CAPTCHAs, fluctuates over time) and each paper needs a curated cluster ID. OpenAlex aggregates Crossref with other scholarly indexes behind a free, keyless API with full coverage of this corpus; its counts typically land between Crossref and Scholar, making it the best reliability-per-effort number. Semantic Scholar is strongest in computer science and biomedicine; on this chemistry-leaning corpus its coverage is the weakest (several papers missing, some counts far below every other source), so treat it as a cross-check only. Empirically the totals order as Scholar > OpenAlex > Crossref > Semantic Scholar.

Google Scholar is the only manually spot-checked number. The "force-field papers" subtotal is simply the sum over rows whose group file is force-field.csv.

All Scholar traffic (collection and the lookup commands alike) is throttled at the driver level to at most 2 requests per second, on top of the pacing above.

Scraping many papers (or re-running repeatedly) will trip Google's bot detection. Two things can happen: Scholar may serve result pages with the "Cited by" link silently stripped (the count is left blank and a warning is printed), or it may show a CAPTCHA. You may need to fill in CAPTCHAs manually to get a complete run: the Firefox window Selenium drives is visible, so solve one whenever it appears — with the retry cooldowns the run then picks back up on its own. If blanks persist, wait a few hours (or a day) and re-run openff-stats citations; the stored cluster IDs are untouched by collection, so a re-run only re-checks the counts.

Zenodo citations (data/zenodo_citations.csv)

Zenodo mints its DOIs (10.5281/zenodo.*) through DataCite (the DOI registration agency for data/software, Crossref's counterpart), so citation counts come from the DataCite API (api.datacite.org/dois/<doi>). The file is long form: one row per record-year from DataCite's citationsOverTime breakdown, and citation_count is the sum of those per-year counts — DataCite's own citationCount attribute lags its per-year data and is not used. Records with no citations keep a single row with a blank year.

Caveats:

• DataCite only knows about formal DOI-to-DOI citation links, so these counts are conservative — most dataset reuse is never cited this way.
• A record hosted on Zenodo but registered under an external DOI (e.g. conference proceedings with a 10.25080/... DOI) is not in DataCite and would always report 0; track those in inputs/publications/ (Crossref) instead.

Conda-forge download counts (data/downloads.csv, data/downloads_yearly.csv)

Two independent measurements per curated package:

• anaconda_total — the ndownloads field from the Anaconda.org API (api.anaconda.org/package/conda-forge/<pkg>): per-file download counters summed across all currently hosted files. Anaconda publishes no specification of what these counters include.
• condastats_total — the sum of monthly counts from condastats, backed by the public anaconda-package-data dataset, which Anaconda builds by reprocessing raw HTTP request logs. Also the source of the per-year breakdown in data/downloads_yearly.csv. Updated roughly monthly.

These two numbers are not the same quantity and can disagree by ±50% per package — prefer condastats, quoted with a reference date. A per-version spot-check (openff-units) showed the two systems agree within a few percent for old .tar.bz2-era versions, but the API counters undercount every .conda-era version by ~1.5–1.8× — modern .conda artifacts are served via a separate host (conda-static.anaconda.org), whose traffic the log-derived dataset (condastats) counts but the per-file counters (anaconda) largely miss. In the other direction, the API counter (anaconda) reads higher for low-install packages with many files: it counts every HTTP GET at the origin, including mirror syncs and bots, which the log-derived dataset (condastats) does not attribute to installs. Since the dataset is aggregated monthly and Anaconda has revised its history (a CDN double-count in mid-2024 was corrected, and previously missed .conda downloads were backfilled from June 2022), always cite downloads as "N as of YYYY-MM per anaconda-package-data" rather than as a timeless total.

Please take these download counts with a gigantic pile of salt. They are not a reflection of user numbers, and vastly overcount "real" installs:

• Many OpenFF packages depend on each other, so downloading one pulls in others. The "total OpenFF conda-forge downloads" simply sums all downloads across all packages, so it is likely inflated at least 5-fold, probably ~10.
• Many, perhaps most, downloads are just CI and tests.
• Where a package has a metapackage + -base pair (e.g. openff-toolkit and openff-toolkit-base), only the -base variant is in the curated list, so at least those are not double counted.

GitHub repo imports and stars (data/github_repo_stars.csv)

Unlike every other source, this one isn't hand-curated. data/github_repos/<package>.csv (one file per package in inputs/github_packages.csv) is fully script-generated by discover-github-repos, which searches, verifies, and tags every candidate with no human review step.

Discovery. For each package, GitHub code search runs a set of queries — declared dependencies (the conda-forge name in setup.py, setup.cfg, pyproject.toml, environment.yml, pixi.toml) plus runtime imports (import/from <import name> in .py and .ipynb files) — sharded by file size to work around GitHub's 1,000-result-per-query cap. requirements.txt is only searched with --include-requirements: GitHub tokenizes on punctuation, so a short package name matches every hyphenated lookalike (gradient-descent matches a descent query), and the query mostly adds shard-paging time for little precision.

Verification. Every candidate repo is checked against the files that actually matched it, using the search API's text-match fragments: a dependency-declaration regex for manifest files, or a real import X/ from X statement for code. A file is only fetched when the fragment is inconclusive. Notebook (.ipynb) matches inside base64-encoded cell outputs can never satisfy the import check, so those never verify. Repos that verify are tagged status=auto; repos that match the search but fail verification are tagged status=exclude and kept in the CSV for the record, with the reason in notes. Repos added by hand (add-github-repo) are status=manual. Only auto/manual rows are counted — exclude rows are skipped by collection.

Search modes. inputs/github_packages.csv (package, import_name, search_mode) controls what gets searched per package. search_mode=full (the default) runs every manifest and import query above. search_mode=environment-only restricts the search to conda environment files (environment.yml / environment.yaml) only — used for descent, whose generic name drowns every other query in same-token noise (its import query alone reports 800k+ hits). Its counts are therefore a floor: known external users of descent also import openff-toolkit, so they're counted there instead.

stars is stargazers_count from one GitHub Repos API call per repo; repos that have gone private, been deleted, or renamed are recorded with 0 stars.

Caveats: code search only sees public repositories that GitHub has indexed, and only what the search queries match — private/unindexed usage is invisible, so the import counts are a lower bound and only as good as the search + verification.

Two extra columns, external (bool) and external_reason, separate real external adoption from noise:

• self — owner is the OpenFF org or a core maintainer account (openforcefield, lilyminium, jaclark5, mattwthompson, j-wags, Yoshanuikabundi, ntBre — from inputs/github_owner_blacklist.csv, a plain owner,reason CSV): our own repos measure our own activity, not external adoption.
• meta — owner is conda-forge (also from the blacklist): packaging infrastructure, not usage.
• duplicate — the repo is listed under more than one package's group; it counts once, in the first group by inputs/github_packages.csv order, so later rows read as additional adoption rather than double-counting.
• fork — the repo is a fork and another member of the same fork family has more stars; only the highest-starred member of a family counts.

Invalid rows are kept in the CSV for auditing — only the "external" adoption counts exclude them. openff-stats classify-repos re-applies this tagging to an existing data/github_repo_stars.csv offline (e.g. after editing the blacklist), without re-fetching from GitHub; github-stars collection also applies it automatically on every run.

openff-stats discover-github-repos                    # sweep all packages in inputs/github_packages.csv → data/github_repos/*.csv
openff-stats discover-github-repos --package descent  # one package (import name + search mode from the CSV)
openff-stats add-github-repo openforcefield/openff-toolkit --group openff-toolkit  # manual addition (status=manual)
openff-stats github-stars                              # → data/github_repo_stars.csv (stars + external/external_reason + has_python_config)
openff-stats classify-repos                            # re-tag offline after editing the blacklist

Installation

Preferred (pixi):

pixi install
pixi run openff-stats --help

Alternative (conda, also used by CI):

conda env create -f environment.yaml
conda activate openff-stats

All dependencies are declared in pyproject.toml and installed via pip install -e .. Google Scholar scraping additionally requires Firefox and geckodriver.

CI

GitHub Actions runs openff-stats run-all on manual trigger (workflow_dispatch) and commits updated data/ files and the README date stamp back to main. Discovery is not run in CI: discover-github-repos is a manual sweep step, and the other discover-* commands write candidates/ for human review. github-descriptions also isn't run in CI (it needs a hand-reviewed category column).