如何根据另一个文件在bash中解析文件

Question

我真的有危险的问题，我将展示我的尝试制定解决方案的例子。我宁愿用awk来解决这个问题。但任何功能性解决方案都将受到欢迎。

我有模板文件，它告诉我列的顺序(按这个顺序)：

模板：

IMPACT;DISTANCE;STRAND;FLAGS;VARIANT_CLASS;SYMBOL;SYMBOL_SOURCE;HGNC_ID;BIOTYPE;CANONICAL;TSL;APPRIS;CCDS;ENSP;SWISSPROT;TREMBL;UNIPARC;REFSEQ_MATCH;GENE_PHENO;SIFT;PolyPhen;EXON;INTRON;DOMAINS;HGVSc;HGVSp;HGVS_OFFSET;GMAF;AFR_MAF;AMR_MAF;EAS_MAF;EUR_MAF;SAS_MAF;AA_MAF;EA_MAF;ExAC_MAF;ExAC_Adj_MAF;ExAC_AFR_MAF;ExAC_AMR_MAF;ExAC_EAS_MAF;ExAC_FIN_MAF;ExAC_NFE_MAF;ExAC_OTH_MAF;ExAC_SAS_MAF;CLIN_SIG;SOMATIC;PHENO;PUBMED;MOTIF_NAME;MOTIF_POS;HIGH_INF_POS;MOTIF_SCORE_CHANGE

所有的值都是分号分隔的.

这是我的输入文件，我无法解析，因为来自模板的一些值丢失了。

输入：

IMPACT=MODIFIER;STRAND=1;VARIANT_CLASS=deletion;SYMBOL=KIF1B;BIOTYPE=protein_coding;ENSP=NP_055889.2;INTRON=24/46;HGVSc=NM_015074.3:c.2537+467delT;HGVS_OFFSET=9
IMPACT=MODIFIER;STRAND=1;VARIANT_CLASS=deletion;SYMBOL=KIF1B;BIOTYPE=protein_coding;CANONICAL=YES;ENSP=XP_005263490.1;INTRON=26/48;HGVSc=XM_005263433.1:c.2675+467delT;HGVS_OFFSET=9
IMPACT=MODIFIER;DISTANCE=4811;STRAND=-1;VARIANT_CLASS=deletion;SYMBOL=C1orf127;BIOTYPE=protein_coding;CANONICAL=YES;ENSP=NP_001164225.1;GMAF=-:0.1749;AMR_MAF=-:0.3011;EAS_MAF=-:0.1542;EUR_MAF=-:0.0794;SAS_MAF=-:0.2008;AA_MAF=-:0.091
IMPACT=MODIFIER;STRAND=1;VARIANT_CLASS=insertion;BIOTYPE=misc_RNA;CANONICAL=YES;INTRON=1/1;HGVSc=XR_158744.2:n.96+764dupA;HGVS_OFFSET=8;GMAF=A:0.4225;AMR_MAF=A:0.2723;EAS_MAF=A:0.5187;EUR_MAF=A:0.4643;SAS_MAF=A:0.3767;AA_MAF=A:0.5613
IMPACT=MODIFIER;STRAND=1;VARIANT_CLASS=insertion;BIOTYPE=misc_RNA;INTRON=1/1;HGVSc=XR_241119.1:n.41+204dupA;HGVS_OFFSET=8;GMAF=A:0.4225;AMR_MAF=A:0.2723;EAS_MAF=A:0.5187;EUR_MAF=A:0.4643;SAS_MAF=A:0.3767;AA_MAF=A:0.5613
IMPACT=MODIFIER;STRAND=1;VARIANT_CLASS=insertion;SYMBOL=SDHC;BIOTYPE=protein_coding;ENSP=NP_001030588.1;INTRON=2/4;HGVSc=NM_001035511.1:c.77+43dupT;HGVS_OFFSET=11
IMPACT=MODIFIER;STRAND=1;VARIANT_CLASS=insertion;SYMBOL=SDHC;BIOTYPE=protein_coding;ENSP=NP_001030589.1;INTRON=2/4;HGVSc=NM_001035512.1:c.77+43dupT;HGVS_OFFSET=11
IMPACT=MODIFIER;STRAND=1;VARIANT_CLASS=insertion;SYMBOL=SDHC;BIOTYPE=protein_coding;ENSP=NP_001030590.1;INTRON=1/3;HGVSc=NM_001035513.1:c.20+9288dupT;HGVS_OFFSET=11
IMPACT=MODIFIER;STRAND=1;VARIANT_CLASS=insertion;SYMBOL=SDHC;BIOTYPE=protein_coding;ENSP=NP_001265101.1;INTRON=2/3;HGVSc=NM_001278172.1:c.77+43dupT;HGVS_OFFSET=11
IMPACT=MODIFIER;STRAND=1;VARIANT_CLASS=insertion;SYMBOL=SDHC;BIOTYPE=protein_coding;CANONICAL=YES;ENSP=NP_002992.1;INTRON=2/5;HGVSc=NM_003001.3:c.77+43dupT;HGVS_OFFSET=1

我想要选项卡分开的输出。如果模板中缺少任何输入值，则将其放在-标记中。基本上，我需要同一列中相同的值。

输出示例：

IMPACT  DISTANCE    STRAND  FLAGS   VARIANT_CLASS   SYMBOL  SYMBOL_SOURCE   HGNC_ID BIOTYPE
IMPACT=MODIFIER -   STRAND=1    -   VARIANT_CLASS=deletion  SYMBOL=KIF1B    -   -   BIOTYPE=protein_coding
IMPACT=MODIFIER -   STRAND=1    -   VARIANT_CLASS=deletion  SYMBOL=KIF1B    -   -   BIOTYPE=protein_coding
IMPACT=MODIFIER DISTANCE=4811   STRAND=-1   -   VARIANT_CLASS=deletion  -   SYMBOL=C1orf127 -   BIOTYPE=protein_coding
IMPACT=MODIFIER -   STRAND=1    -   VARIANT_CLASS=insertion -   -   -   BIOTYPE=misc_RNA
IMPACT=MODIFIER -   STRAND=1    -   VARIANT_CLASS=insertion     -   -   BIOTYPE=misc_RNA
IMPACT=MODIFIER -   STRAND=1    -   VARIANT_CLASS=insertion SYMBOL=SDHC -   -   BIOTYPE=protein_coding
IMPACT=MODIFIER -   STRAND=1    -   VARIANT_CLASS=insertion SYMBOL=SDHC -   -   BIOTYPE=protein_coding
IMPACT=MODIFIER -   STRAND=1    -   VARIANT_CLASS=insertion SYMBOL=SDHC -   -   BIOTYPE=protein_coding
IMPACT=MODIFIER -   STRAND=1    -   VARIANT_CLASS=insertion SYMBOL=SDHC -   -   BIOTYPE=protein_coding
IMPACT=MODIFIER -   STRAND=1    -   VARIANT_CLASS=insertion SYMBOL=SDHC -   -   BIOTYPE=protein_coding
IMPACT=MODIFIER -   STRAND=1    -   VARIANT_CLASS=insertion SYMBOL=SDHC -   -   BIOTYPE=misc_RNA
IMPACT=MODIFIER -   STRAND=1    -   VARIANT_CLASS=deletion  SYMBOL=SDHC -   -   BIOTYPE=protein_coding
IMPACT=MODIFIER -   STRAND=1    -   VARIANT_CLASS=deletion  SYMBOL=SDHC -   -   BIOTYPE=protein_coding
IMPACT=MODIFIER -   STRAND=1    -   VARIANT_CLASS=deletion  SYMBOL=SDHC -   -   BIOTYPE=protein_coding
IMPACT=MODIFIER -   STRAND=1    -   VARIANT_CLASS=deletion  SYMBOL=SDHC -   -   BIOTYPE=protein_coding
IMPACT=MODIFIER -   STRAND=1    -   VARIANT_CLASS=deletion  SYMBOL=SDHC -   -   BIOTYPE=protein_coding
IMPACT=MODIFIER -   STRAND=1    -   VARIANT_CLASS=deletion  SYMBOL=SDHC -   -   BIOTYPE=misc_RNA
IMPACT=MODIFIER -   STRAND=1    -   VARIANT_CLASS=deletion  SYMBOL=SDHC -   -   BIOTYPE=protein_coding
IMPACT=MODIFIER -   STRAND=1    -   VARIANT_CLASS=deletion  SYMBOL=SDHC -   -   BIOTYPE=protein_coding
IMPACT=MODIFIER -   STRAND=1    -   VARIANT_CLASS=deletion  SYMBOL=SDHC -   -   BIOTYPE=protein_codin

注意:在示例输出中，我没有所有列。

我的尝试只是用awk解析输入文件：

awk -v OFS="\t" '{split($1,arr1,";"); print arr1[1],arr1[2]..}'

这个解析工作很好，但不给我排序，也不处理丢失的值。谢谢你的帮助。

注意：

这是向How to find and print specific character in bash解释的更多问题

Answer 1

很容易在本地bash完成-不需要awk。

#!/usr/bin/env bash
#          ^^^ ^^^^
# uses the bash from your PATH, so on MacOS X, this can use a MacPorts bash 4
# ...vs the 3.x version installed by Apple in /bin.

# read template values into an array
IFS=';' read -r -a template < template

# print header
printf '%s\t' "${template[@]}"; printf '\n'

# declare an associative array (requires bash 4)
declare -A data

# iterate over lines of the input file, reading each into an array
while IFS=';' read -r -a items; do

  # populate the data map with key/value items from this line
  data=( )
  for item in "${items[@]}"; do
    key=${item%%=*}
    value=${item#*=}
    data[$key]=$value
  done

  # iterate over template items, emitting a field for each
  for item in "${template[@]}"; do
    if [[ ${data[$item]} ]]; then
      printf -- '%s=%s\t' "$item" "${data[$item]}"
    else
      printf -- '-\t'
    fi
  done

  # ...and emit a newline after processing each input line
  printf '%s\n'

done <input