遍历.tar.gz目录和连接文件(不解压缩文件夹)

Question

我有一个大约有20000个tar.gz目录的文件夹，每个目录都包含一堆文件。我想进入源文件，遍历tar.gz目录(而不解压缩)，并将文件连接起来，所以在最后我将有三个大文件。

例如，我有一个根文件夹pnoc，它有.tar.gz目录，每个压缩文件夹都有三个文件夹-- Kallisto、RSEM和Hugo。我已经解压缩了一个这样的目录，如下所示：

pnoc/
├── C021_0001_20140916_tumor_RNASeq.tar.gz
├── C021_0002_001113_tumor_RNASeq.tar.gz
├── C021_0003_001409_tumor_RNASeq.tar.gz
├── C021_0004_001418_tumor_RNASeq.tar.gz
├── C021_0005_001661_tumor_RNASeq.tar.gz
├── C021_0007_001669_tumor_RNASeq.tar.gz
├── C021_0008_001699_tumor_RNASeq.tar.gz
├── C021_0009_001766_tumor_RNASeq.tar.gz
├── C021_0010_001774_tumor_RNASeq.tar.gz
├── C021_0011_001786_tumor_RNASeq.tar.gz
├── C021_0012_001825_tumor_RNASeq.tar.gz
├── C021_0013_001872_tumor_RNASeq.tar.gz
├── CPBT_0001_1_tumor_RNASeq.tar.gz
├── CPBT_0003_1_tumor_RNASeq.tar.gz
├── CPBT_0004_1_tumor_RNASeq.tar.gz
├── CPBT_0005_1_tumor_RNASeq.tar.gz
├── CPBT_0006_1_tumor_RNASeq.tar.gz
├── CPBT_0007_1_tumor_RNASeq.tar.gz
├── CPBT_0008_1_tumor_RNASeq.tar.gz
├── CPBT_0009_1_tumor_RNASeq.tar.gz
├── IMPROPERLY_PAIRED.C021_0006_001666_tumor_RNASeq.tar.gz
└── pnoc-manifest


C021_0001_20140916_tumor_RNASeq
├── Kallisto
│   ├── C021_0001_20140916_tumor_RNASeq.abundance.h5
│   ├── C021_0001_20140916_tumor_RNASeq.abundance.tsv
│   └── C021_0001_20140916_tumor_RNASeq.run_info.json
└── RSEM
    ├── C021_0001_20140916_tumor_RNASeq.rsem.genes.norm_counts.tab
    ├── C021_0001_20140916_tumor_RNASeq.rsem.genes.raw_counts.tab
    ├── C021_0001_20140916_tumor_RNASeq.rsem.isoform.norm_counts.tab
    ├── C021_0001_20140916_tumor_RNASeq.rsem.isoform.raw_counts.tab
    ├── C021_0001_20140916_tumor_RNASeq.rsem_genes.results
    ├── C021_0001_20140916_tumor_RNASeq.rsem_isoforms.results
    └── Hugo
        ├── C021_0001_20140916_tumor_RNASeq.rsem.genes.norm_counts.hugo.tab
        ├── C021_0001_20140916_tumor_RNASeq.rsem.genes.raw_counts.hugo.tab
        ├── C021_0001_20140916_tumor_RNASeq.rsem.isoform.norm_counts.hugo.tab
        ├── C021_0001_20140916_tumor_RNASeq.rsem.isoform.raw_counts.hugo.tab
        ├── C021_0001_20140916_tumor_RNASeq.rsem_genes.hugo.results
        └── C021_0001_20140916_tumor_RNASeq.rsem_isoforms.hugo.results

因此，我想将所有*.abundance.tsv连接在一个文件中，*.rsem.genes.norm_counts.tab连接在第二个文件中，*.rsem_genes.hugo.results连接在第三个文件中。做这件事的最好和最有效的方法是什么？我对任何事情都没意见-- R，Python或Bash。

$ find --version
find (GNU findutils) 4.5.11
Copyright (C) 2012 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Written by Eric B. Decker, James Youngman, and Kevin Dalley.
Features enabled: D_TYPE O_NOFOLLOW(enabled) LEAF_OPTIMISATION SELINUX FTS(FTS_CWDFD) CBO(level=2) 

谢谢!

Answer 1

使用bash find命令，如下所示；将exec中的cat命令应用于该命令返回的所有文件。+选项是确保由shell生成的cat实例不超过一个。

这里，{}表示返回的文件，find命令。请参阅更多关于find -exec的信息

find . -type f -name '*.abundance.tsv' -exec cat "{}" + >> ../AbundanceTSV.tsv
find . -type f -name '*.rsem.genes.norm_counts.tab' -exec cat "{}" + >> ../GenesNormCounts.tab
find . -type f -name '*.rsem_genes.hugo.results' -exec cat "{}" + >> ../HugoResults.results