crawler4j的实现

Question

我正在尝试让crawler4j的基本形式像here一样运行。我修改了前几行，定义了rootFolder和numberOfCrawlers，如下所示：

public class BasicCrawlController {

    public static void main(String[] args) throws Exception {
            if (args.length != 2) {
                    System.out.println("Needed parameters: ");
                    System.out.println("\t rootFolder (it will contain intermediate crawl data)");
                    System.out.println("\t numberOfCralwers (number of concurrent threads)");
                    return;
            }

            /*
             * crawlStorageFolder is a folder where intermediate crawl data is
             * stored.
             */
             String crawlStorageFolder = args[0];

              args[0] = "/data/crawl/root";

            /*
             * numberOfCrawlers shows the number of concurrent threads that should
             * be initiated for crawling.
             */
            int numberOfCrawlers = Integer.parseInt(args[1]);

            args[1] = "7";


            CrawlConfig config = new CrawlConfig();

            config.setCrawlStorageFolder(crawlStorageFolder);

不管我怎么定义它，我仍然会收到这个错误

Needed parameters: 
 rootFolder (it will contain intermediate crawl data)
 numberOfCralwers (number of concurrent threads)

我认为我需要“在运行配置中设置参数”窗口，但我不知道这是什么意思。如何正确配置此基本crawler以使其启动并运行？

Answer 1

在使用javac关键字编译程序之后，您需要通过键入以下命令来运行它：

java BasicCrawler控制器"arg1“"arg2”

该错误告诉您在运行程序时没有指定arg或arg1。另外，在您已经收到number of crawlers参数之后，这个“args1 = "7";”是怎么回事？

对于您正在尝试做的事情，删除前5行，因为您无论如何都尝试使用硬编码值。然后将crawlForStorage字符串设置为目录路径，将numberOfCrawlers设置为7，这样就不必指定命令行参数了。如果你想使用命令行参数，去掉上面的硬编码值，并在CL中指定它们