awk在数据bash上没有得到正确的比较结果

Question

我有一个文件file.txt，

ch.qos.logback:logback-classic 1.2.0
ch.qos.logback:logback-core 1.2.0
com.fasterxml.jackson.core:jackson-databind 2.10.0
com.fasterxml.jackson.core:jackson-databind 2.6.7
com.fasterxml.jackson.core:jackson-databind 2.6.7
com.fasterxml.jackson.core:jackson-databind 2.7.9
com.fasterxml.jackson.core:jackson-databind 2.8.11
com.fasterxml.jackson.core:jackson-databind 2.8.11
com.fasterxml.jackson.core:jackson-databind 2.8.9
com.fasterxml.jackson.core:jackson-databind 2.9.10
com.fasterxml.jackson.core:jackson-databind 2.9.10
com.fasterxml.jackson.core:jackson-databind 2.9.10
com.fasterxml.jackson.core:jackson-databind 2.9.5
com.fasterxml.jackson.core:jackson-databind 2.9.7
com.fasterxml.jackson.core:jackson-databind 2.9.8
com.fasterxml.jackson.core:jackson-databind 2.9.9
com.h2database:h2 2.0.206
com.h2database:h2 2.1.210
com.thoughtworks.xstream:xstream 1.4.11
com.thoughtworks.xstream:xstream 1.4.16
commons-collections:commons-collections 3.2.2
commons-fileupload:commons-fileupload 1.3.3
handlebars 4.3.0
handlebars 4.7.7
io.dropwizard:dropwizard-validation 1.3.21
io.netty:netty-all 4.1.44
io.netty:netty-codec 4.1.66
io.netty:netty-codec-http 4.1.44
io.vertx:vertx-web 3.5.4
net.minidev:json-smart 2.4.1
org.apache.hadoop:hadoop-common 0.23.4
org.apache.logging.log4j:log4j-core 2.3.1
org.apache.shiro:shiro-core 1.7.1
org.apache.shiro:shiro-web 1.5.2
org.apache.shiro:shiro-web 1.5.3
org.apache.shiro:shiro-web 1.7.1
org.apache.tomcat.embed:tomcat-embed-core 7.0.89
org.apache.tomcat.embed:tomcat-embed-core 9.0.31
org.eclipse.jetty:jetty-http 9.2.25
org.jasig.cas.client:cas-client-core 3.3.2
org.springframework.data:spring-data-commons 1.13.11
org.springframework.security.oauth:spring-security-oauth2 2.3.3
org.springframework:spring-web 5.3.0

我想得到文件中提到的最高版本的唯一包名。

我尝试了下面的awk，但结果并不像预期的那样，

cat file.txt | awk '$2 > a[$1]{a[$1] = $2} END{for (i in a) print i, a[i]}'

RESULT:
org.eclipse.jetty:jetty-http 9.2.25
io.netty:netty-codec 4.1.66
com.h2database:h2 2.1.210
org.jasig.cas.client:cas-client-core 3.3.2
org.apache.logging.log4j:log4j-core 2.3.1
io.vertx:vertx-web 3.5.4
handlebars 4.7.7
com.thoughtworks.xstream:xstream 1.4.16
ch.qos.logback:logback-core 1.2.0
net.minidev:json-smart 2.4.1
org.apache.shiro:shiro-core 1.7.1
commons-fileupload:commons-fileupload 1.3.3
org.springframework:spring-web 5.3.0
commons-collections:commons-collections 3.2.2
org.apache.shiro:shiro-web 1.7.1
com.fasterxml.jackson.core:jackson-databind 2.9.9
io.netty:netty-all 4.1.44
org.springframework.security.oauth:spring-security-oauth2 2.3.3
org.apache.hadoop:hadoop-common 0.23.4
io.dropwizard:dropwizard-validation 1.3.21
org.apache.tomcat.embed:tomcat-embed-core 9.0.31
org.springframework.data:spring-data-commons 1.13.11
ch.qos.logback:logback-classic 1.2.0
io.netty:netty-codec-http 4.1.44

但是这个结果是不正确的，就像2.9.0，2.10.0，它需要2.9倍，这是预料不到的。

你能帮帮我吗。

Answer 1

您的脚本的主要问题是$2 > a[$1]正在进行字符串(即字母表，逐字符)比较，而不是版本(即数字，按点分隔的数字)比较，因此10出现在9之前，因为第一个比较字符是1 vs 9和1小于9。

awk没有“版本”的概念，所以您必须在awk中编写一个版本--在awk中进行比较，但是GNU已经内置了--使用GNU进行-V，“版本排序”：

$ sort -k2,2Vr file | awk '!seen[$1]++'
org.eclipse.jetty:jetty-http 9.2.25
org.apache.tomcat.embed:tomcat-embed-core 9.0.31
org.springframework:spring-web 5.3.0
handlebars 4.7.7
io.netty:netty-codec 4.1.66
io.netty:netty-all 4.1.44
io.netty:netty-codec-http 4.1.44
io.vertx:vertx-web 3.5.4
org.jasig.cas.client:cas-client-core 3.3.2
commons-collections:commons-collections 3.2.2
com.fasterxml.jackson.core:jackson-databind 2.10.0
net.minidev:json-smart 2.4.1
org.springframework.security.oauth:spring-security-oauth2 2.3.3
org.apache.logging.log4j:log4j-core 2.3.1
com.h2database:h2 2.1.210
org.springframework.data:spring-data-commons 1.13.11
org.apache.shiro:shiro-core 1.7.1
org.apache.shiro:shiro-web 1.7.1
com.thoughtworks.xstream:xstream 1.4.16
io.dropwizard:dropwizard-validation 1.3.21
commons-fileupload:commons-fileupload 1.3.3
ch.qos.logback:logback-classic 1.2.0
ch.qos.logback:logback-core 1.2.0
org.apache.hadoop:hadoop-common 0.23.4

或者，如果您也关心按字母顺序排序的输出，您也可以这样做(前者在awk脚本中保存内存中的每个唯一$1，而后者在awk脚本中一次只在内存中保存2美元1值)：

sort -k1,1 -k2,2Vr file | awk '!seen[$1]++'
sort -k1,1 -k2,2Vr file | awk '$1!=prev{print; prev=$1}'

例如：

$ sort -k1,1 -k2,2Vr file | awk '!seen[$1]++'
ch.qos.logback:logback-classic 1.2.0
ch.qos.logback:logback-core 1.2.0
com.fasterxml.jackson.core:jackson-databind 2.10.0
com.h2database:h2 2.1.210
com.thoughtworks.xstream:xstream 1.4.16
commons-collections:commons-collections 3.2.2
commons-fileupload:commons-fileupload 1.3.3
handlebars 4.7.7
io.dropwizard:dropwizard-validation 1.3.21
io.netty:netty-all 4.1.44
io.netty:netty-codec 4.1.66
io.netty:netty-codec-http 4.1.44
io.vertx:vertx-web 3.5.4
net.minidev:json-smart 2.4.1
org.apache.hadoop:hadoop-common 0.23.4
org.apache.logging.log4j:log4j-core 2.3.1
org.apache.shiro:shiro-core 1.7.1
org.apache.shiro:shiro-web 1.7.1
org.apache.tomcat.embed:tomcat-embed-core 9.0.31
org.eclipse.jetty:jetty-http 9.2.25
org.jasig.cas.client:cas-client-core 3.3.2
org.springframework.data:spring-data-commons 1.13.11
org.springframework.security.oauth:spring-security-oauth2 2.3.3
org.springframework:spring-web 5.3.0

Answer 2

您还可以使用perl而不是awk来完成此操作，后者具有可以明智地比较的原生版本类型：

$ perl -Mversion -lane '
my $v = version->parse("v$F[1]");
$packages{$F[0]} = $v if $v > $packages{$F[0]};
END {
  foreach my $p (sort keys %packages) {
    print $p, "\t", substr($packages{$p}, 1)
  }
}' input.txt
ch.qos.logback:logback-classic  1.2.0
ch.qos.logback:logback-core 1.2.0
com.fasterxml.jackson.core:jackson-databind 2.10.0
com.h2database:h2   2.1.210
com.thoughtworks.xstream:xstream    1.4.16
commons-collections:commons-collections 3.2.2
commons-fileupload:commons-fileupload   1.3.3
handlebars  4.7.7
io.dropwizard:dropwizard-validation 1.3.21
io.netty:netty-all  4.1.44
io.netty:netty-codec    4.1.66
io.netty:netty-codec-http   4.1.44
io.vertx:vertx-web  3.5.4
net.minidev:json-smart  2.4.1
org.apache.hadoop:hadoop-common 0.23.4
org.apache.logging.log4j:log4j-core 2.3.1
org.apache.shiro:shiro-core 1.7.1
org.apache.shiro:shiro-web  1.7.1
org.apache.tomcat.embed:tomcat-embed-core   9.0.31
org.eclipse.jetty:jetty-http    9.2.25
org.jasig.cas.client:cas-client-core    3.3.2
org.springframework.data:spring-data-commons    1.13.11
org.springframework.security.oauth:spring-security-oauth2   2.3.3
org.springframework:spring-web  5.3.0