Perl使用单个文件打开句柄呈现多个csv文件？

Question

我有这段代码，其中我想处理多个csv(目前只有一个文件)文件，并使用perl在将其发送到linux-box之前呈现其格式，并使用ssh连接替换原始文件内容。这是代码

#!/usr/bin/perl -w
use strict;
# this is a csv which will contains IP addresses of one specific category for e.g malware.
my $seculert_qradar_list = "$seculert_dir/seculert.csv";

#ssh connection information 
my $qradar_console = '10.10.1.22';
my $qradar_ssh_key = "$seculert_dir/qr-id_dsa";
my $qradar_ssh_knownhosts = "$seculert_dir/known_hosts";

#################################################################################

# NOTE: this is the "OUT" file.
# 1 - Name
# 2 - Sub-Name
# 3 - IP Address
# 4 - is colour, deprecated
# 5 - database length, deprecated
# 6 - asset weight, deprecated
# 7 - an ID for the 'record' each unique name pair (first 2 columns) gets an ID
#################################################################################
my $source = 'BAD-IP-Addresses-LABEL';
my $type_description = 'honeypots-for-examnple';

# Based upon the format described above I want to render the csv as written in print OUT statement. This format is important, because the endsystem process the file (remotenet.conf) based upon the provided layout.
open(FP, ">>$seculert_qradar_list");
for my $line (<FP>) {
    my ($hostname, $ip, $something1, $something2) = split(/,/, $line);
    print OUT "$source $type_description $ip #FF0000 0 90  29\n";
}
close(FP);

# Here I just want the contents of modified csv to be written over remotenet.conf. This file is then processed through auto-deploy script by the system. The results get populated on front-end webserver.    
print "Sending to QRadar...\n";
# SSH To QRadar's Console and push out file + trigger update
`scp -i $qradar_ssh_key -o UserKnownHostsFile=$qradar_ssh_knownhosts -o StrictHostKeyChecking=no root\@$qradar_console:/store/configservices/staging/globalconfig/remotenet.conf .`;
`sed -i -e '/^SECULERT/d' remotenet.conf`;
`cat $seculert_qradar_list >> remotenet.conf`;
`scp -i $qradar_ssh_key -o UserKnownHostsFile=$qradar_ssh_knownhosts -o StrictHostKeyChecking=no remotenet.conf root\@$qradar_console:/store/configservices/staging/globalconfig/remotenet.conf`;

print "Cleaning up...\n";
# Remove our SECULERT list and the newly pushed out qradar conf
unlink($seculert_qradar_list); unlink ('remotenet.conf');

print "Deploying in QRadar...(takes time to complete)\n";
# QRadar magic
`ssh -i $qradar_ssh_key -o UserKnownHostsFile=$qradar_ssh_knownhosts -o StrictHostKeyChecking=no root\@$qradar_console /opt/qradar/upgrade/util/setup/upgrades/do_deploy.pl`;
print "Complete!\n\n";

我想知道的是--也许还可以从perl程序员那里得到一些帮助--使用一个文件句柄可以打开多个文件，例如，在我的例子中，我有如下所示

1. virus.csv
2. bot.csv
3. malware.csv

是否需要为每个具有不同句柄的csv文件重新复制循环代码？目标文件remotenet.conf保持不变。

在正确呈现了一个csv文件之后，web上的remotenet.conf将类似于

Virus
    10.10.2.1
     .......

Bot
    10.10.4.1
     ......

如果在一次执行中发生多个更改，只需要一个自动部署(请参阅最后的代码)，那就太好了。我希望我能理解这个问题。如果需要更多的澄清，请告诉我。

谢谢

我想要完成的

我想要一个动态代码，其中有多个csv准备好呈现保存在一个文件夹中。该序列描述如下：

1. 从文件夹中读取每个csv文件。
2. 将其转换为可接受的格式。
3. 根据csv文件名更改两个变量值。如：

文件名 malware.csv

我的$source =‘坏IP地址-标签’；我的$type_description =‘蜜罐-为例子’；

For bot.csv

我的$source = 'bot-net'；我的$type_description = 'top-10'；

最后，通过ssh将格式化的文件连同内容复制到remotenet.conf中。

Answer 1

您不能从以附加>>模式打开的文件中读取。您似乎刚刚将OUT文件句柄重命名为FP，但这还不够。

• 原始代码中包含了一个chdir $seculert_dir --这是有意义的，因为现在我们不必总是在文件名前面加上目录(它相当于shell中的cd命令)。
• 最初的代码没有完全使用最佳实践。我将scp和ssh命令包装在执行最小错误检查的版本中。
• 也没有必要为我们可以使用Perl所做的事情调用外部工具。

下面是我更新的代码，它可以从简单的CSV文件中读取数据：

首先，我们开始序言。use warnings比-w交换机更可取。

#!/usr/bin/perl
use strict;
use warnings;

my $seculert_dir = "!FIXME!";
chdir $seculert_dir;  # go into that dir, so that we don't have to use prefixes

my @csv_files = ("seculert.csv"); # this is a csv which will contains IP addresses
                                  # of one specific category for e.g malware.
                                  # We only use the 2nd column

#ssh connection information 
my $qradar_console = '10.10.1.22';
my $qradar_ssh_key = "qr-id_dsa";
my $qradar_ssh_knownhosts = "known_hosts";

那是我们的配置。接下来，我们使用包装的SCP命令从服务器获取conf：

# fetch the remotenet.conf from QRadar
print STDERR "Fetching configuration from QRadar...\n";
scp("root\@$qradar_console:/store/configservices/staging/globalconfig/remotenet.conf" => '.');

现在打开一个文件，在该文件中放置更改的配置。

# write the changed conf here before uploading:
open my $new_conf, ">", "remotenet.conf.changed" or die qq(Can't open "remotenet.conf.changed" for writing: $!);

现在我们打开旧的配置，复制它，但是跳过以SECULERT开头的行。

# copy old conf over, delete lines starting with "SECULERT"
open my $old_conf, "<", "remotenet.conf" or die qq(Can't open "remotenet.conf" for reading: $!);
while (<$old_conf>) {
    print {$new_conf} $_ unless /^SECULERT/;
}
close $old_conf;

注意我是如何使用“词法文件句柄”(open my $fh, ...)的。这避免了一些问题，而且比使用赤裸裸的词更现代。

接下来，我们循环遍历CSV文件。我们打开每一列，然后提取第二列，并将其与其他内容一起打印到更改的配置文件中。

# append the data from the CSVs
for my $csv_file (@csv_files) {
    my $source = 'BAD-IP-Addresses-LABEL';
    my $type_description = 'honeypots-for-examnple';

    open my $csv, "<", $csv_file or die qq(Can't open "$csv_file" for reading: $!);

    while (my $line = <$csv>) {
        my (undef, $ip) = split /,/, $line; # we're only interested in the 2nd column

        # Based upon the format described below I want to render the csv
        # as written in print OUT statement. This format is important, because the
        # endsystem process the file (remotenet.conf) based upon the provided layout.
        #
        # Columns in the output:
        # 1 - Name
        # 2 - Sub-Name
        # 3 - IP Address
        # 4 - is colour, deprecated
        # 5 - database length, deprecated
        # 6 - asset weight, deprecated
        # 7 - an ID for the 'record' each unique name pair (first 2 columns) gets an ID
        print {$new_conf} "$source $type_description $ip #FF0000 0 90  29\n";
    }
}

现在，我们可以在新的配置文件中获得我们想要的所有信息，并可以将其上传到服务器：

close $new_conf;

# copy the changed remotenet.conf back to QRadar
scp('remotenet.conf.changed' => "root\@$qradar_console:/store/configservices/staging/globalconfig/remotenet.conf");

# Remove our SECULERT list and the newly pushed out qradar conf
print STDERR "Cleaning up...\n";
unlink $_ or warn qq(Can't remove "$_": $!) for 'remotenet.conf', 'remotenet.conf.changed';

接下来，我们运行部署脚本：

# QRadar magic -- run deploy script
print STDERR "Deploying in QRadar...(takes time to complete)\n";
ssh("root\@$qradar_console", '/opt/qradar/upgrade/util/setup/upgrades/do_deploy.pl');

print STDERR "Complete!\n\n";

下面是scp和ssh的包装器。它们是子程序(函数、过程或其他语言中的方法)。参数在@_数组中，从中我们将它们解压到具有更好名称的变量中。system命令接受命令的名称和参数列表。因为这绕过了外壳，所以我们不必考虑外壳外逃。system在sucess上返回零，所以我们使用它来检查错误。

# Wrappers for SSH and SCP commands
sub scp {
    my ($from, $to) = @_;
    my $success = 0 == system 'scp',
        '-i' => $qradar_ssh_key,
        '-o' => "UserKnownHostsFile=$qradar_ssh_knownhosts",
        '-o' => "StrictHostKeyChecking=no",
        $from => $to;
    return $success if defined wantarray;                # return failure when somebody checks for it
    die qq(Can't scp "$from" to "$to") if not $success;  # die when failure, and nobody checks.
}

sub ssh {
    my ($host, $command) = @_;
    my $success = 0 == system 'ssh',
        '-i' => $qradar_ssh_key,
        '-o' => "UserKnownHostsFile=$qradar_ssh_knownhosts",
        '-o' => "StrictHostKeyChecking=no",
        $host, $command;
    return $success if defined wantarray;                           # return failure when somebody checks for it
    die qq(Can't ssh into "$host" for '$command') if not $success;  # die when failure, and nobody checks.
}

这段代码仍然可以改进，例如使用适当的CSV解析器(如Text::CSV )，使用更好的SSH绑定，而不是简单地包装命令行程序，使用autodie自动错误检查，以及更好地处理诱惑文件。

对于不同的文件，您似乎需要不同的$source值。为此，我们应该使用比@csv_files更复杂的数据结构--我会使用数组的散列。

my %csv_files = (
  'malware.csv' => ['BAD-IP-Addresses-LABEL', 'honeypots-for-examnple'],
  'bot.csv'     => ['bot-net', 'top-10'],
);

这是一个字典，它将键(这里是文件名)映射到值(这里是两列的内容)。现在，我们不再循环数组中的条目，而是循环这个散列中的键：

for my $csv_file (keys %csv_files) {
  my ($source, $type_description) = @{ $csv_files{$csv_file} };

  ...
}

表达式$csv_files{$csv_file}访问哈希$csv_files中名为$csv_file的条目。此条目包含作为值的数组引用。将数组引用转换为数组的@{…}，我们可以用列表赋值my ($foo, $bar) = @array解压数组。