传递awk列说明符的bash变量

Question

将一个shell变量传递给awk有很多线程，我已经很容易理解了，但是我想传递的变量是列说明符变量($1,$2等)。

考虑到shell也使用这些变量作为默认命令行参数变量，这将使人感到困惑。

在这个脚本中，我只是将两个文件排序和连接在一起，但是为了开始稍微概括一下脚本，我希望能够在命令行上指定，awk应该使用的键文件中的字段作为它的排序说明符。

我在这里做错什么了？(我刚开始处理awk，而oneliner是从这里中略作改编的。)

keyfile="$1"
filetosort="$2"
field="$3"

awk -v a="$field"
paste "$keyfile" <(awk 'NR==FNR{o[FNR]=a; next} {t[$1]=$0} END{for(x=1; x<=FNR; x++){y=o[x]; print t[y]}}' $keyfile $filetosort)

编辑在/输出中添加了示例

Keyfile：(从文件中随机抽取10行)

PVClumt18   PAK_2199    PAK_01997
PVClopt2    PAK_2091    PAK_01895
PVCcif7     PAK_1975    PAK_01793
PVClopT12   PAU_02101   PAU_02063
PVCpnf20    PAK_3524    PAK_03184
PVClopt3    PAK_2090    PAK_01894
PVClopT11   PAU_02102   PAU_02064
PVCunit2_11 plu1698     PLT_01726
PVClumT9    afp10       PAU_02198
PVCunit2_17 plu1692     PLT_01720

要排序的文件：

PAU_02064   1pqx    1pqx_A  37.4    13  0.00035 31.4    >1pqx_A Conserved hypothetical protein; ZR18,structure, autostructure,spins,autoassign, northeast structural genomics consortium; NMR {Staphylococcus aureus subsp} SCOP: d.267.1.1 PDB: 2ffm_A 2m6q_A 2m8w_A
PAK_01997   5ftj    5ftj_A  99.9    1.6e-26 4.2e-31 229.2   >5ftj_A Transitional endoplasmic reticulum ATPase; hydrolase, single-particle, AAA ATPase; HET: ADP OJA; 2.30A {Homo sapiens} PDB: 3cf1_A* 3cf3_A* 3cf2_A* 5ftk_A* 5ftl_A* 5ftm_A* 5ftn_A* 1r7r_A* 5c19_A 5c1b_A* 5c18_A* 3cf0_A*
PAK_01894   3j9q    3j9q_A  99.9    1.8e-29 4.6e-34 215.9   >3j9q_A Sheath; pyocin, bacteriocin, sheath, structural protein; 3.50A {Pseudomonas aeruginosa}
PAK_03184   1xju    1xju_A  99.4    4.1e-17 1.1e-21 98.8    >1xju_A Lysozyme; secreted inactive conformation, hydrolase; 1.07A {Enterobacteria phage P1} SCOP: d.2.1.3
PAK_01793   5a3a    5a3a_A  50.8    6   0.00016 31.4    >5a3a_A SIR2 family protein; transferase, P-ribosyltransferase, metalloprotein, NAD-depen lipoylation, regulatory enzyme, rossmann fold; 1.54A {Streptococcus pyogenes} PDB: 5a3b_A* 5a3c_A*
PLT_01720   3ggm    3ggm_A  54.2    4.9 0.00013 26.2    >3ggm_A Uncharacterized protein BT9727_2919; bacillus cereus group., structural genomics, PSI-2, protein structure initiative; 2.00A {Bacillus thuringiensis serovarkonkukian}
PLT_01726   3h2t    3h2t_A  96.8    8e-06   2.1e-10 82.6    >3h2t_A Baseplate structural protein GP6; viral protein, virion; 3.20A {Enterobacteria phage T4} PDB: 3h3w_A 3h3y_A
PAK_01895   3j9q    3j9q_A  100.0   2.5e-35 6.4e-40 248.6   >3j9q_A Sheath; pyocin, bacteriocin, sheath, structural protein; 3.50A {Pseudomonas aeruginosa}
PAU_02198   4jiv    4jiv_D  69.6    1.6 4.2e-05 27.5    >4jiv_D VCA0105, putative uncharacterized protein; PAAR-repeat motif, membrane piercing, type VI secretion SYST vibrio cholerae VGRG2; HET: PLM STE ELA; 1.90A {Vibrio cholerae o1 biovar eltor}
PAU_02063   4yap    4yap_A  31.1    20  0.00052 29.1    >4yap_A Glutathione S-transferase homolog; GSH-lyase GSH-dependent; 1.11A {Sphingobium SP} PDB: 4g10_A 4yav_A*

因此，我需要根据键文件中的第3列和文件中的第1列对行进行排序和匹配。

以及生成的文件：(第3和第4列的复制是我计划之后整理的)

PVClumt18   PAK_2199    PAK_01997   PAK_01997   5ftj    5ftj_A  99.9    1.6e-26 4.2e-31 229.2   >5ftj_A Transitional endoplasmic reticulum ATPase; hydrolase, single-particle, AAA ATPase; HET: ADP OJA; 2.30A {Homo sapiens} PDB: 3cf1_A* 3cf3_A* 3cf2_A* 5ftk_A* 5ftl_A* 5ftm_A* 5ftn_A* 1r7r_A* 5c19_A 5c1b_A* 5c18_A* 3cf0_A*
PVClopt2    PAK_2091    PAK_01895   PAK_01895   3j9q    3j9q_A  100.0   2.5e-35 6.4e-40 248.6   >3j9q_A Sheath; pyocin, bacteriocin, sheath, structural protein; 3.50A {Pseudomonas aeruginosa}
PVCcif7 PAK_1975    PAK_01793   PAK_01793   5a3a    5a3a_A  50.8    6   0.00016 31.4    >5a3a_A SIR2 family protein; transferase, P-ribosyltransferase, metalloprotein, NAD-depen lipoylation, regulatory enzyme, rossmann fold; 1.54A {Streptococcus pyogenes} PDB: 5a3b_A* 5a3c_A*
PVClopT12   PAU_02101   PAU_02063   PAU_02063   4yap    4yap_A  31.1    20  0.00052 29.1    >4yap_A Glutathione S-transferase homolog; GSH-lyase GSH-dependent; 1.11A {Sphingobium SP} PDB: 4g10_A 4yav_A*
PVCpnf20    PAK_3524    PAK_03184   PAK_03184   1xju    1xju_A  99.4    4.1e-17 1.1e-21 98.8    >1xju_A Lysozyme; secreted inactive conformation, hydrolase; 1.07A {Enterobacteria phage P1} SCOP: d.2.1.3
PVClopt3    PAK_2090    PAK_01894   PAK_01894   3j9q    3j9q_A  99.9    1.8e-29 4.6e-34 215.9   >3j9q_A Sheath; pyocin, bacteriocin, sheath, structural protein; 3.50A {Pseudomonas aeruginosa}
PVClopT11   PAU_02102   PAU_02064   PAU_02064   1pqx    1pqx_A  37.4    13  0.00035 31.4    >1pqx_A Conserved hypothetical protein; ZR18,structure, autostructure,spins,autoassign, northeast structural genomics consortium; NMR {Staphylococcus aureus subsp} SCOP: d.267.1.1 PDB: 2ffm_A 2m6q_A 2m8w_A
PVCunit2_11 plu1698 PLT_01726   PLT_01726   3h2t    3h2t_A  96.8    8e-06   2.1e-10 82.6    >3h2t_A Baseplate structural protein GP6; viral protein, virion; 3.20A {Enterobacteria phage T4} PDB: 3h3w_A 3h3y_A
PVClumT9    afp10   PAU_02198   PAU_02198   4jiv    4jiv_D  69.6    1.6 4.2e-05 27.5    >4jiv_D VCA0105, putative uncharacterized protein; PAAR-repeat motif, membrane piercing, type VI secretion SYST vibrio cholerae VGRG2; HET: PLM STE ELA; 1.90A {Vibrio cholerae o1 biovar eltor}
PVCunit2_17 plu1692 PLT_01720   PLT_01720   3ggm    3ggm_A  54.2    4.9 0.00013 26.2    >3ggm_A Uncharacterized protein BT9727_2919; bacillus cereus group., structural genomics, PSI-2, protein structure initiative; 2.00A {Bacillus thuringiensis serovarkonkukian}

Answer 1

当您通过awk -v a="$field"时，awk变量a的规范只适用于单个awk命令。您不能期望a在完全不同的awk调用中可用。

因此，您需要直接将其放在适当的位置。

$ bashvar="2"
$ echo 'foo bar baz' | awk -v awkvar="$bashvar" '{print $awkvar}'
bar

或者在你的情况下：

field=1
awk -v a="$field" '
NR==FNR {
  o[FNR]=$a;
  next;
}

{ t[$1] = $0 }

END {
  for(x=1; x<=FNR; x++) {
    y=o[x]
    printf("%s\t%s\n", y, t[y])
  }
}' "$keyfile" "$filetosort"

注意事项：

• 这里的printf同时发出键和值，因此不需要使用paste将keyfile值放回。
• $a用于将awk变量a (由shell变量field分配)作为变量名称，并执行间接引用--从而查找相关的列号。
• 在展开时，总是引用shell变量。否则，您就无法知道awk的扩展会产生多少参数--它可能是0(如果在$keyfile中找不到字符串中没有字符)；它可能是1，但也可能是一个完全无界的数字(input file.txt将成为两个参数，input和file.txt；* input * .txt将用文件列表替换每个* )。

Answer 2

我要补充这个问题作为一个答案，因为它确实解决了我提出的问题，尽管查尔斯对我出错的(无数)领域提出了很好的建议。

用Charles关于单独awk命令的点来修改上面的代码，我现在可以调用以下内容(抱歉，是的，它仍然在使用paste).

#!/bin/bash

keyfile="$1"
filetosort="$2"
indexfield="$3"

paste "$keyfile" <(awk -v field="$indexfield" 'NR==FNR{o[FNR]=$field; next} {t[$1]=$0} END{for(x=1; x<=FNR; x++){y=o[x]; print t[y]}}' "$keyfile" "$filetosort")

在我在awk命令中调用变量之前，我丢失了$，这是我的原始代码不能工作的(部分)原因之一，并且没有将awk变量声明包含在一个awk调用中。

因此，bash sortandmatch.sh keyfile filetosort 3产生我想要的输出：

PVCunit2_5  plu1704         PLT_01732   PLT_01732   4etv    4etv_A  39.0    12  0.00032 27.6    >4etv_A Ryanodine receptor 2; phosphorylation, cardiac, metal transport; 1.65A {Mus musculus}
PVCunit2_4  plu1705         PLT_01733   PLT_01733   3j9q    3j9q_A  99.9    7.2e-30 1.9e-34 219.0   >3j9q_A Sheath; pyocin, bacteriocin, sheath, structural protein; 3.50A {Pseudomonas aeruginosa}
XVC_pnf15   XBW1_RS06910    XBW1_RS06910    XBW1_RS06910    1fi0    1fi0_A  69.2    1.7 4.4e-05 22.8    >1fi0_A VPR protein, R ORF protein; helix, viral protein; NMR {Synthetic} SCOP: j.11.1.1
PVCcif7     PAU_01999       PAU_01967   PAU_01967   5a3a    5a3a_A  47.5    7.3 0.00019 30.9    >5a3a_A SIR2 family protein; transferase, P-ribosyltransferase, metalloprotein, NAD-depen lipoylation, regulatory enzyme, rossmann fold; 1.54A {Streptococcus pyogenes} PDB: 5a3b_A* 5a3c_A*
PVClumT15   PAU_02233       PAU_02192   PAU_02192   1tdp    1tdp_A  22.1    37.0    0.00096 27.2    >1tdp_A Carnobacteriocin B2 immunity protein; four-helix bundle, antimicrobial protein; NMR {Carnobacterium maltaromaticum} SCOP: a.29.8.1
XVC_pnf3    XBW1_RS06850    XBW1_RS06850    XBW1_RS06850    3eaa    3eaa_A  87.7    0.13    3.4e-06 35.7    >3eaa_A EVPC; T6SS, unknown function; 2.79A {Edwardsiella tarda}
PVCunit1_4  afp4            PAU_02778   PAU_02778   3j9q    3j9q_A  99.9    3.6e-29 9.5e-34 214.6   >3j9q_A Sheath; pyocin, bacteriocin, sheath, structural protein; 3.50A {Pseudomonas aeruginosa}
PVCunit2_3  plu1706         PLT_01734   PLT_01734   3j9q    3j9q_A  100.0   1.6e-34 4.3e-39 253.7   >3j9q_A Sheath; pyocin, bacteriocin, sheath, structural protein; 3.50A {Pseudomonas aeruginosa}
PVClumt17   PAK_2200        PAK_01998   PAK_01998   3k8p    3k8p_C  34.7    16.0    0.00041 34.1    >3k8p_C DSL1, KLLA0C02695P; intracellular trafficking, DSL1 complex, multisubunit tethering complex, snare proteins; 2.60A {Kluyveromyces lactis}
PVClopT12   PAU_02101       PAU_02063   PAU_02063   4yap    4yap_A  31.1    20  0.00052 29.1    >4yap_A Glutathione S-transferase homolog; GSH-lyase GSH-dependent; 1.11A {Sphingobium SP} PDB: 4g10_A 4yav_A*