Selenium无头测试在EC2实例上运行非常慢

Question

感谢您阅读我的帖子。我有一个问题想请教你一下。我有许多用Java编写的自动化Selenium测试。他们利用JUnit进行测试。我的任务是在亚马逊EC2 CentOS服务器上优化这些测试的运行时间。测试从Github存储库中提取，并存储在EC2实例上的Docker Container中。类似地，这些测试也使用Cucumber从客户机的笔记本电脑通过Eclipse运行。在笔记本电脑上，测试所需的时间是在EC2实例上所用时间的一小部分。EC2实例和笔记本电脑具有相同的内存量和足够的CPU处理能力来运行测试，因此我认为EC2实例上的测试不会花费那么长的时间。EC2实例和笔记本电脑都必须通过代理才能访问互联网。我已确保此部分设置正确，因为我必须将HTTP_PROXY、HTTPS_PROXY和no_proxy设置为特定的IP地址。

在本节中，我将提供在调用chromedriver时实例化chromedriver的代码。我还将提供我用来运行测试的确切命令。如下所示：

下面是实例化时调用的方法：

public static ChromeOptions setupChromeOptions() {
    ChromeOptions options = new ChromeOptions();
    Map<String, Object> p refs = new HashMap<String, Object>();
    prefs.put("profile.default_content_setting_values.notifications", 2);
    options.setHeadless(true);
    options.setProxy(null);
    options.setExperimentalOption("prefs", prefs);
    options.addArguments("--disable-dev-shm-usage");
    options.addArguments("--disable-extensions");
    options.addArguments("--disable-gpu");
    options.addArguments("--headless");
    options.addArguments("--no-proxy-server");
    options.addArguments("--no-sandbox");   
    options.addArguments("--proxy-bypass-list=*");
    options.addArguments("--proxy-server=");
    options.addArguments("--proxy-server='direct://'");
    options.addArguments("--start-maximized");  
    options.addArguments("--window-size=1280,720");
    return options;
}

下面是在EC2实例上通过Jenkins运行的命令：

#!/bin/bash
cd /home/ec2-user/Git/REPOS_NAME                                                            # Navigate to where the repos are at on the EC2.
git fetch --all
git reset --hard origin/master
git pull                                                                                    # Pull down the latest.
sudo docker container ls                                                                    # Display the running containers.
sudo docker exec 0967d39b1967 bash -c 'cd /Selenium/ ; rm -rf REPO_ROOT_FOLDER/ ; exit'     # Delete the automated test folder so that it gets copied over fresh.
sudo docker cp /home/ec2-user/Git/REPO_NAME/REPO_ROOT_FOLDER 0967d39b1967:/Selenium/        # Copy the latest from the Github Repository to the Docker container.
sudo docker exec 0967d39b1967 bash -c 'cd /Selenium/REPO_ROOT_FOLDER/ANOTHER_FOLDER/driver ; chmod 0777 chromedriver ; cd .. ; mvn clean test'  # Activate the Testrunner that is enabled in pom.xml. This kicks off the automated tests.

最后，下面是我正在运行的版本号和操作系统详细信息：

Amazon EC2: CentOS 6.10
Docker Container OS: Ubuntu 16.04
Selenium: selenium-server-standalone-3.141.59.jar
Chromedriver: 84.0.4147.30
Google Chrome: 84.0.4147.105
Java: OpenJDK 1.8.0_252
Cucumber: 2.0.0
I cannot provide links to an HTML Page that I'm scraping because it's all inside of a Salesforce environment.

EC2 Specs:

AWS T2.XLarge
4 vCPUs
64-Bit
16 GB RAM
54 CPU Credits per hour

Laptop Specs:

Intel Core i7-8665U @ 1.90GHz, 2112 MHz, 4 cores, 8 Logocal Processors
16 GB RAM
Windows 10

Group of Tests 1:
EC2: 50 minutes
Laptop through Eclipse: 20 minutes

Group of Tests 2:
EC2: 2 hours 43 minutes
Laptop through Eclipse: 1 hour 15 minutes

对于两组测试，有两个时序分解。正如您所看到的，EC2的关闭时间比笔记本电脑长得多。

最后，这里是运行时间为50分钟的一组测试的日志。我不得不更改一些目录和路径名来保护客户端。

Started by user Scott-IM
Running as SYSTEM
[EnvInject] - Loading node environment variables.
Building remotely on EC2_INSTANCE in workspace /var/lib/jenkins/workspace/Run mvn test
[Run mvn test] $ /bin/bash /tmp/jenkins166122467778007381.sh
Fetching origin
FIPS mode initialized
HEAD is now at 506f4ef Setting proxy to null for Chromeoptions
FIPS mode initialized
Already up to date.
CONTAINER ID        IMAGE               COMMAND                CREATED             STATUS              PORTS                    NAMES
0967d39b1967        IMAGE_NAME          "/Selenium/setup.sh"   4 hours ago         Up 4 hours          0.0.0.0:8080->8080/tcp   CONTAINER_NAME
Picked up JAVA_TOOL_OPTIONS: -Dfile.encoding=UTF8
[INFO] Scanning for projects...
[INFO] 
[INFO] --------------< test:automation >--------------
[INFO] Building test-automation 1.0-SNAPSHOT
[INFO] --------------------------------[ jar ]---------------------------------
[INFO] 
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ automation ---
[INFO] Deleting /Selenium/automation/automation/target
[INFO] 
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ automation ---
[WARNING] Using platform encoding (UTF8 actually) to copy filtered resources, i.e. build is platform dependent!
[INFO] skip non existing resourceDirectory /Selenium/automation/automation/src/main/resources
[INFO] 
[INFO] --- maven-compiler-plugin:3.1:compile (default-compile) @ automation ---
[INFO] No sources to compile
[INFO] 
[INFO] --- maven-resources-plugin:2.6:testResources (default-testResources) @ automation ---
[WARNING] Using platform encoding (UTF8 actually) to copy filtered resources, i.e. build is platform dependent!
[INFO] skip non existing resourceDirectory /Selenium/automation/automation/src/test/resources
[INFO] 
[INFO] --- maven-compiler-plugin:3.1:testCompile (default-testCompile) @ automation ---
[INFO] Changes detected - recompiling the module!
[WARNING] File encoding has not been set, using platform encoding UTF8, i.e. build is platform dependent!
[INFO] Compiling 111 source files to /Selenium/salesforce-automation/salesforce-automation/target/test-classes
[WARNING] /Selenium/automation/automation/src/test/java/com/test/mapper/OIDP_Flow/OIDP_Email_Auto_Approval_Flow_2.java:[797,128] unmappable character for encoding UTF8
[WARNING] /Selenium/automation/automation/src/test/java/com/test/mapper/OIDP_Flow/OIDP_Email_Auto_Approval_Flow_2.java:[1842,130] unmappable character for encoding UTF8
[WARNING] /Selenium/automation/automation/src/test/java/com/test/shared/ExtentReporter.java:[126,33] unmappable character for encoding UTF8
[WARNING] /Selenium/automation/automation/src/test/java/com/test/shared/ExtentReporter.java:[127,32] unmappable character for encoding UTF8
[WARNING] /Selenium/automation/automation/src/test/java/com/test/mapper/rsiaa/HD_ISO_VSC_Auto_Approval_of_Resubmitted_Si.java:[4977,95] unmappable character for encoding UTF8
[WARNING] /Selenium/automation/automation/src/test/java/com/test/pages/aao/TrackingRecord.java:[27,39] unmappable character for encoding UTF8
[WARNING] /Selenium/automation/automation/src/test/java/com/test/pages/aao/TrackingRecord.java:[117,84] unmappable character for encoding UTF8
[WARNING] /Selenium/automation/automation/src/test/java/com/test/pages/aao/TrackingRecord.java:[156,92] unmappable character for encoding UTF8
[WARNING] /Selenium/automation/automation/src/test/java/com/test/pages/aao/TrackingRecord.java:[195,84] unmappable character for encoding UTF8
[WARNING] /Selenium/automation/automation/src/test/java/restApiPOC/PrivacyRestApiConnection.java: Some input files use unchecked or unsafe operations.
[WARNING] /Selenium/automation/automation/src/test/java/restApiPOC/PrivacyRestApiConnection.java: Recompile with -Xlint:unchecked for details.
[INFO] 
[INFO] --- maven-surefire-plugin:2.19:test (default-test) @ automation ---
[WARNING] The parameter forkMode is deprecated since version 2.14. Use forkCount and reuseForks instead.
 
-------------------------------------------------------
 T E S T S
-------------------------------------------------------
Picked up JAVA_TOOL_OPTIONS: -Dfile.encoding=UTF8
Running com.test.runner.TestRunner2
Starting ChromeDriver 84.0.4147.30 (48b3e868b4cc0aa7e8149519690b6f6949e110a8-refs/branch-heads/4147@{#310}) on port 23815
Only local connections are allowed.
Please see https://chromedriver.chromium.org/security-considerations for suggestions on keeping ChromeDriver safe.
ChromeDriver was started successfully.
[1604696323.888][SEVERE]: bind() failed: Cannot assign requested address (99)
Nov 06, 2020 8:58:44 PM org.openqa.selenium.remote.ProtocolHandshake createSession
INFO: Detected dialect: W3C
Frame count :3
Frame count :4
Frame count :6
Newly Generated Service Item Number is :15355359
Frame count :3
Frame count :3
Clicked on new service item :15355359
Frame count :2
Frame count :2
Frame count :4
Frame count :3
Frame count :3
 
1 Scenarios (1 passed)
33 Steps (33 passed)
44m57.274s
 
Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2,698.236 sec - in com.salesforcetest.runner.TestRunner2
 
Results :
 
Tests run: 0, Failures: 0, Errors: 0, Skipped: 0
 
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  45:05 min
[INFO] Finished at: 2020-11-06T21:43:40Z
[INFO] ------------------------------------------------------------------------
Finished: SUCCESS

如果有什么不清楚或不合理，请让我知道，我会解决这个问题。我已经在互联网上到处寻找解决方案，但还没有找到任何东西。我几乎不知所措，因为我不完全确定缓慢的确切原因是什么。如果有人可以提供一些线索或东西来尝试，我非常愿意尝试一下。

非常感谢你的帮助。

Answer 1

本质上不是一个解决方案，但提出了一些值得尝试的东西。

既然你在比较Chrome的运行时间，我建议你尽量保持版本同步。这在windows上是很困难的，因为当Chrome更新时，你通常无法控制。这并不是说它将解决您的问题，但它将有助于消除一个变量。

Chrome (就这一点而言，还有chromedriver )经历了大量的变化，而不是在同一个版本上，这些变化可以而且已经对我产生了影响。

以下是我在过去几年中为了让事情顺利运行而不得不使用的一些chrome选项。我注意到有3个你没有。

--disable-extensions --你永远不会知道他们的本地版本的chrome上可能有什么。

--dns-prefetch-disable --在旧版本的chromedriver中，如果没有这个，脚本就会挂起。

https://bugs.chromium.org/p/chromedriver/issues/detail?id=402#c128

--enable-features=NetworkService,NetworkServiceInProcess --脚本冻结

https://groups.google.com/forum/m/#!topic/chromedriver-users/ktp-s_0M5NM[21-40]