如何在Jenkins上自动重新运行失败的TestNG测试，并在重试通过时更新最终结果以通过？

Question

如何在Jenkins上自动重新运行失败的TestNG测试，并在重试通过时更新最终结果以通过？

描述:我有两个测试运行。Test1通过了，但是Test2在重试时失败了。

实际结果:测试运行: 1，失败: 1，错误: 0，跳过: 0。

预期结果；测试运行: 2，失败: 0，错误: 0，跳过:0

或

测试运行: 2，失败: 0，错误: 0，跳过:1(跳过用于重试测试)

    public class TestRetry implements IRetryAnalyzer {
    private static int MAX_RETRY_COUNT = 1; 
    //Only works for value of 1; Anything more than that causes 
    infinite retries for the same test even when it passes.

    AtomicInteger count = new AtomicInteger(MAX_RETRY_COUNT);

    public boolean isRetryAvailable() {
        return (count.intValue() > 0);
    }

    public boolean retry(ITestResult result) {
        boolean retry = false;
        if (isRetryAvailable()) {
            Reporter.log("Going to retry test case: " + result.getMethod() + ", " + (MAX_RETRY_COUNT - count.intValue() + 1) + " out of " + MAX_RETRY_COUNT, true);
            retry = true;
            count.decrementAndGet();
        }
        return retry;
    }
}

  public class TestListenerSupport extends TestListenerAdapter {
  TestUtilities testUtil = new TestUtilities();

@Override
public void onTestStart(ITestResult result){
    //Handle events before test start
    //Obtain specific annotation info from test methods
    Stable stable = 
  result.getMethod().getConstructorOrMethod().getMethod()
  .getAnnotation(Stable.class);
   Maturing maturing = 
  result.getMethod().getConstructorOrMethod().getMethod()
  .getAnnotation(Maturing.class);
   Preflight preFlight = 
   result.getMethod().getConstructorOrMethod().getMethod()
   .getAnnotation(Preflight.class);
      if (stable != null){
           Reporter.log("Annotation is set to STABLE! for '" + 
    result.getMethod().getMethodName() + "'.", true);
      }
      if (maturing != null){
           Reporter.log("Annotation is set to MATURING! for '" + 
    result.getMethod().getMethodName() + "'.", true);
     }
      if (preFlight != null){
           Reporter.log("Annotation is set to PREFLIGHT! for '" + 
    result.getMethod().getMethodName() + "'.", true);
    }
}


  @Override
  public void onTestFailure(ITestResult result){

      if (result.getMethod().getRetryAnalyzer() != null) {
            TestRetry retryAnalyzer = 
     (TestRetry)result.getMethod().getRetryAnalyzer();

            if(retryAnalyzer.isRetryAvailable()) {
                result.setStatus(ITestResult.SKIP);
            } else {
                result.setStatus(ITestResult.FAILURE);
            }

            Reporter.setCurrentTestResult(result);
        }
}

@Override
public void onTestSkipped(ITestResult result){

    Reporter.log("onTestSkipped invoked! " + result.getThrowable().toString(),true);
    Reporter.log("onTestSkipped Exception Message: " + result.getThrowable().getMessage(),true);
    Reporter.log("onTestSkipped Exception Cause: " + result.getThrowable().getCause(),true);
    Reporter.log("onTestSkipped Exception StackTrace: " + result.getThrowable().getStackTrace(), true);
    AppiumDriver driver = getDriverInstance(result);
    if(driver!=null){
        TestUtilities testUtil = new TestUtilities();
        try {
            testUtil.getScreenShot(driver, result.getMethod().getMethodName() + "_Skipped");
        } catch (Exception e) {
            // TODO Auto-generated catch block
            e.printStackTrace();
        }
        Reporter.log("Killing appium driver instance",true);
        driver.quit();
    }
    else
        Reporter.log("onTestSkipped - null driver detected", true);
}

@Override
public void onTestSuccess(ITestResult result){
    Map map = new HashMap();
    //get the testcase IDs from input xml file used for creating a test run in TestRail
    Map<String,String> test_case_id = result.getMethod().getXmlTest().getAllParameters();
    String test_name = result.getMethod().getMethodName();
    testUtil.writeResult("1", test_case_id.get(test_name));

}

Answer 1

我用尼克在这里提到的https://stackoverflow.com/questions/11622365/testng-how-to-only-output-one-pass-fail-result-when-test-is-rerun来调整我的代码

而且起作用了。

谢谢。

public class RetryAnalyzer implements IRetryAnalyzer {
private int count = 0;
private int maxCount = 2;

@Override
public boolean retry(ITestResult result) {
if (!result.isSuccess()) {
  if (count < maxCount) {
    count++;
    result.setStatus(ITestResult.SUCCESS);
    String message = Thread.currentThread().getName() +
    ": Error in " + result.getName() + " Retrying "
        + (maxCount + 1 - count) + " more times";
    System.out.println(message);
    Reporter.log(message);
    return true;
  } else {
    result.setStatus(ITestResult.FAILURE);
  }
}
return false;
}
}

Answer 2

TestNG提供了IRetryAnalyzer接口，您可以这样实现该接口：

public class MyRetryAnalyzer implements IRetryAnalyzer {

    private AtomicInteger retries = new AtomicInteger(3);

    @Override
    public boolean retry(ITestResult result) {
        return retries.getAndDecrement() > 0;
    }

}

之后，您可以使用它如下：

@Test(retryAnalyzer = MyRetryAnalyzer.class)
public void myTest(){
    // ...
}

然而，薄薄的测试是邪恶的，过度使用重试规则也是如此。看看这个相关的问题：

如何处理有间歇性故障的片状测试？