pytest:如何仅为一个测试用例抑制跟踪

Question

我有以下测试用例：

    @pytest.mark.parametrize("init_resp", [True, False])
    @pytest.mark.parametrize("mechanism", ["login", "plain"])
    def test_byclient(self, auth_peeker_controller, client, mechanism, init_resp):
        self._ehlo(client)
        client.user = "goodlogin"
        client.password = "goodpasswd"
        auth_meth = getattr(client, "auth_" + mechanism)
        try:
            client.auth(mechanism, auth_meth, initial_response_ok=init_resp)
        except SMTPAuthenticationError:
            if (mechanism, init_resp) == ("login", False):
                client.docmd("*")
                pytest.xfail(reason="smtplib.SMTP.auth_login is buggy (bpo-27820)")
            else:
                raise
        peeker = auth_peeker_controller.handler
        assert isinstance(peeker, PeekerHandler)
        assert peeker.login == b"goodlogin"
        assert peeker.password == b"goodpasswd"

如您所见，对于一个参数组合(即("login", False))，测试预计会失败。

问题是，我仍然有两个异常干扰pytest的输出：

aiosmtpd\tests\test_smtp.py:1054: in test_byclient
    client.auth(mechanism, auth_meth, initial_response_ok=init_resp)
C:\Python\36\lib\smtplib.py:642: in auth
    raise SMTPAuthenticationError(code, resp)
E   smtplib.SMTPAuthenticationError: (334, b'UGFzc3dvcmQA')

During handling of the above exception, another exception occurred:
aiosmtpd\tests\test_smtp.py:1058: in test_byclient
    pytest.xfail(reason="smtplib.SMTP.auth_login is buggy (bpo-27820)")
E   _pytest.outcomes.XFailed: smtplib.SMTP.auth_login is buggy (bpo-27820)

(使用--tb=short运行pytest)

是否可以仅针对这个测试用例来抑制这些异常？

编辑1

实际上，在发布了这个问题之后，我将测试用例更改为：

    @pytest.mark.parametrize("init_resp", [True, False])
    @pytest.mark.parametrize("mechanism", ["login", "plain"])
    def test_byclient(self, auth_peeker_controller, client, mechanism, init_resp):
        self._ehlo(client)
        client.user = "goodlogin"
        client.password = "goodpasswd"
        auth_meth = getattr(client, "auth_" + mechanism)
        if (mechanism, init_resp) == ("login", False):
            with pytest.raises(SMTPAuthenticationError):
                client.auth(mechanism, auth_meth, initial_response_ok=init_resp)
            client.docmd("*")
            pytest.xfail(reason="smtplib.SMTP.auth_login is buggy (bpo-27820)")
        client.auth(mechanism, auth_meth, initial_response_ok=init_resp)
        peeker = auth_peeker_controller.handler
        assert isinstance(peeker, PeekerHandler)
        assert peeker.login == b"goodlogin"
        assert peeker.password == b"goodpasswd"

结果会更好一些：

aiosmtpd\tests\test_smtp.py:1035: in test_byclient
    pytest.xfail(reason="smtplib.SMTP.auth_login is buggy (bpo-27820)")
E   _pytest.outcomes.XFailed: smtplib.SMTP.auth_login is buggy (bpo-27820)

不过，如果可能的话，我仍然希望这个“例外”能够被抑制。

上面的代码片段来自GitHub上的这些行

Answer 1

有几种方法可以将xfail标记附加到测试函数之外。例如，在测试集合完成时：

import pytest


def pytest_collection_modifyitems(items):
    for item in items:
        if item.name == "test_byclient[login-False]":
            item.add_marker(pytest.mark.xfail(reason="..."))

(将代码放入测试的conftest.py中，rootdir)。

或通过autouse夹具：

@pytest.fixture(autouse=True)
def _(request):
    if request.node.name == "test_byclient[login-False]":
        request.node.add_marker(pytest.mark.xfail(reason="..."))

这样，login-False-specific代码就可以一起从测试函数中删除：

@pytest.mark.parametrize("init_resp", [True, False])
@pytest.mark.parametrize("mechanism", ["login", "plain"])
def test_byclient(self, auth_peeker_controller, client, mechanism, init_resp):
    self._ehlo(client)
    client.user = "goodlogin"
    client.password = "goodpasswd"
    auth_meth = getattr(client, "auth_" + mechanism)
    client.auth(mechanism, auth_meth, initial_response_ok=init_resp)
    peeker = auth_peeker_controller.handler
    assert isinstance(peeker, PeekerHandler)
    assert peeker.login == b"goodlogin"
    assert peeker.password == b"goodpasswd"

如果client.docmd("*")是login-False-test的一个组成部分，那么我同意@jonrsharpe:您有一个包含两个不同测试用例的测试函数，您应该将它分成两个单独的测试用例。

Answer 2

您的测试用例不应该像try except异常的PROD代码。pytest提供了一种机制来期待异常，然后处理它。下面是代码。

with pytest.raises(SMTPAuthenticationError) as exception:
        client.auth(mechanism, auth_meth, initial_response_ok=init_resp)
        .... do something.