Skip to content

Genome metadata QC filtering#31

Open
epbrenner wants to merge 5 commits into
mainfrom
qc_filtering
Open

Genome metadata QC filtering#31
epbrenner wants to merge 5 commits into
mainfrom
qc_filtering

Conversation

@epbrenner

Copy link
Copy Markdown
Contributor

Adding new functionality.

  1. .parse_bvbrc_tsv() is a helper to more safely merge TSV data from parallel workers after I noticed a tab shift in data somewhere yielded strain names in the accession ID column and such after a failed merge.
  2. .apply_metadata_qc is the big one, which allows you to screen BV-BRC genome metadata for CheckM completeness and CheckM contamination values. By default, I set these values very loose at 90% complete and 10% contamination being the cutoffs. We can adjust as needed. Additionally, you can enable filters based on CDS counts, genome length, and GC content, which are set based on deviations away from the median value of the distribution of those values for the input taxon. This will not work well for multi-taxon bug sets so this is set to NULL by default!
  3. Updated the retrieveMetadata() and prepareGenomes() functions to accept the apply_metadata_qc steps.
  4. Updated .ftpes_download_one to incorporate the old .is_complete helper inside the loop. This avoids a BiocParallel issue where the workers did not inherit the helpers.
  5. Updated a spot where an old ftp_download_one function was called instead of ftpes_download_one

@amcim Most importantly for you, I know you've been working on the BiocParallel stuff and had since moved on to another implementation (futures?). I just want to confirm that this script is working for me locally after my patches, but this is just based off main. If we're changing implementations, some of my updates aren't necessary.

For testers:
Open GitHub Desktop or your preferred method of fetching a PR, fetch this PR, source data_curation.R into your favorite CLI, and try retrieveMetadata(user_bacs = "Shigella flexneri") for a quick bug, or prepareGenomes(user_bacs = "Shigella flexneri") to also test the genome downloader. I have tried this out locally for Enterobacter spp., K. pnumoniae, and Campylobacter jejuni. Give it a spin!

epbrenner and others added 2 commits July 22, 2026 11:17
Added ability to filter BV-BRC genomes based on genome stat metadata, including CheckM completeness and CheckM contamination as defaults. Also enabled optional checking of genome length, CDS count, and GC content based on deviations from median distribution per bug. Updated genome downloader functions to use ftpes_download_one instead of old ftp_download_one, and moved a dot function helper to inside the ftpes_download_one loop to avoid issues with BiocParallel workers not understanding what that helper is.
epbrenner and others added 3 commits July 22, 2026 13:31
Uncommenting a helper that was still needed for several calls. Fixing stuff I broke myself!
Updating to remove BiocParallel implementation that caused bugs. Patching back to prior Future implementation instead. CLI version of genome downloading appears to have issues that may be separate and requires further troubleshooting.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant