RabbitTClust: enabling fast clustering analysis of millions of bacteria genomes with MinHash sketches

作者：XU Xiao-ming, Zekun Yin, Lifeng Yan, Hao Zhang, Borui Xu, Yanjie Wei, Beifang Niu, Bertil Schmidt, Weiguo Liu · 发表于：Genome biology · 年份：2023 · DOI：10.1186/s13059-023-02961-6 · 被引用次数：17 · 研究领域：Genomics and Phylogenetic Studies、Probiotics and Fermented Foods、Machine Learning in Bioinformatics

We present RabbitTClust, a fast and memory-efficient genome clustering tool based on sketch-based distance estimation. Our approach enables efficient processing of large-scale datasets by combining dimensionality reduction techniques with streaming and parallelization on modern multi-core platforms. 113,674 complete bacterial genome sequences from RefSeq, 455 GB in FASTA format, can be clustered within less than 6 min and 1,009,738 GenBank assembled bacterial genomes, 4.0 TB in FASTA format, within only 34 min on a 128-core workstation. Our results further identify 1269 redundant genomes, with identical nucleotide content, in the RefSeq bacterial genomes database.