比较使用awk和bash的脚本

Question

我尝试使用awk和另一个命令创建比较脚本，并成功地运行。

但我觉得我写的剧本太长了。

下面有人能缩短我的剧本吗？

在计划A shorten the code than之后，计划B是：

 1. I want eliminated a lot temp file (.txt), only need `lengkap.txt`
 2. Put command in variable if can

如果有人能帮我，我很高兴

下面是我创建的基于搜索和尝试的代码。

#!/bin/bash

### Path Folder who will be compare ###
path1=/home/rio/apps1
path2=/home/rio/apps2

### Find all filename and convert to MD5 ###
find $path1 -type f | xargs md5sum > checksums.md5
find $path2 -type f | xargs md5sum > checksums2.md5

### Compare to find different folder ###
awk 'NR==FNR{c[$1]++;next};c[$1] == 0' checksums.md5 checksums2.md5 > hasil1.txt
awk 'NR==FNR{c[$1]++;next};c[$1] == 0' checksums2.md5 checksums.md5 > hasil2.txt

### Merge result of compare ###
awk '{print $0}' hasil1.txt hasil2.txt > perbedaan.txt

### Filter Just Filename Difference ###
cat perbedaan.txt | awk '{print $2}' > hasilperbedaan.txt

### File about result compare (just filename) ###
cekhasil=/home/rio/hasilperbedaan.txt

### Check if File result compare empty or not ###
if [ -s "$cekhasil" ]
then 
   echo " file exists and is not empty "
    ### Find All filename and date, after that put as we want ###
    find $path1 -type f -ls | awk '{print $11" "$8" "$9" "$10 }' > filedate1.txt
    find $path2 -type f -ls | awk '{print $11" "$8" "$9" "$10 }' > filedate2.txt

    ### Compare to get the date of filename ###
    awk 'A[$1]++' hasilperbedaan.txt filedate1.txt > pre_hasil1.txt
    awk 'A[$1]++' hasilperbedaan.txt filedate2.txt > pre_hasil2.txt
    ### Merge result of compare with date ###
    awk '{print $0}' pre_hasil1.txt pre_hasil2.txt > lengkap.txt
else
   echo " file does not exist, or is empty "
fi

Answer 1

嗯，这需要时间来弄清楚你的程序在做什么。所以我把它缩短了两个阶段。

第一阶段:我删除了所有重复的临时文件，并在使用一次临时文件时使用了管道。

STEP1:您只需对$path1和$path2中的所有文件进行校验

### Find all filename and convert to MD5 ###
find $path1 -type f | xargs md5sum > checksums.md5
find $path2 -type f | xargs md5sum > checksums2.md5

您不需要2个临时文件，因为路径包含在文件名中。所以你可以用一个find来代替它。我使用sort可以在之后使用uniq：

find "$path1" "$path2" -type f | xargs md5sum  | sort > cksum.md5

STEP2:您可以在两个文件中找到唯一的校验和，并获得相应的文件名。

### Compare to find different folder ###
awk 'NR==FNR{c[$1]++;next};c[$1] == 0' checksums.md5 checksums2.md5 > hasil1.txt
awk 'NR==FNR{c[$1]++;next};c[$1] == 0' checksums2.md5 checksums.md5 > hasil2.txt

### Merge result of compare ###
awk '{print $0}' hasil1.txt hasil2.txt > perbedaan.txt

### Filter Just Filename Difference ###
cat perbedaan.txt | awk '{print $2}' > hasilperbedaan.txt

由于我们有一个校验和排序文件，我们只需使用uniq进行筛选，并获得文件名。

注意：当我们使用MD5校验和(128位)时，长度为128位/8位* 2hex=32

uniq -u -w32 cksum.md5 | awk '{print $2}' > "$cekhasil"

如果您喜欢awk而不是uniq，只需比较以前的记录$1和当前的。

步骤3:在两个源路径中将已找到的条目与新搜索(find)匹配。

if [ -s "$cekhasil" ]
then
    echo " file exists and is not empty "
    ### Find All filename and date, after that put as we want ###
    find $path1 -type f -ls | awk '{print $11" "$8" "$9" "$10 }' > filedate1.txt
    find $path2 -type f -ls | awk '{print $11" "$8" "$9" "$10 }' > filedate2.txt

    ### Compare to get the date of filename ###
    awk 'A[$1]++' hasilperbedaan.txt filedate1.txt > pre_hasil1.txt
    awk 'A[$1]++' hasilperbedaan.txt filedate2.txt > pre_hasil2.txt
    ### Merge result of compare with date ###
    awk '{print $0}' pre_hasil1.txt pre_hasil2.txt > lengkap.txt
else
    echo " file does not exist, or is empty "
fi

在这里，最好避免新的完整的find，而只是循环文件名。

if [ -s "$cekhasil" ]
then
    echo " file exists and is not empty "
    while read -r fn ; do
        ls -dils "$fn" | awk '{print $11" "$8" "$9" "$10 }'
    done < "$cekhasil" > lengkap2.txt

else
    echo " file does not exist, or is empty "
fi

总之，脚本变成：

#!/bin/bash

path1=/home/br/dev/tools/bash
path2=/home/br/dev/tools/bash2
outfile=hasilperbedaan.txt

find "$path1" "$path2" -type f | xargs md5sum  | sort > cksum.md5

uniq -u -w32 cksum.md5 | awk '{print $2}' > "$outfile"

if [ -s "$outfile" ]; then
    echo " file exists and is not empty "
    while read -r fn ; do
        ls -dils "$fn" | awk '{print $11" "$8" "$9" "$10 }'
    done < "$output" > lengkap2.txt

else
    echo " file does not exist, or is empty "
fi

此时，我们可以注意到剩下的2个临时文件只使用了一次。我们只需去掉它们，只需将所有的pipe放在一起就可以得到输出文件。

    #!/bin/bash
    
    path1=/home/br/dev/tools/bash
    path2=/home/br/dev/tools/bash2
    outputfile="lengkap-v2.txt"
    
    find "$path1" "$path2" -type f | xargs md5sum  | sort |
        uniq -u -w32 | awk '{print $2}' |
        while read -r fn ; do
            ls -dils "$fn" | awk '{print $11" "$8" "$9" "$10 }'
        done > "$outputfile"
    [[ -s "$outputfile" ]] && echo " file exists and is not empty " ||
            echo " file does not exist, or is empty "

说明：

• 我不明白你的第二个问题：“如果可以的话，把命令放在变量中”，所以我跳过了这个问题。
• 您应该<#>绝不使用find的输出(除了-print0选项)或ls作为任何命令的输入。但是这个答案是不可能的。