未能通过运行runner脚本在Clickhouse中进行集成测试

Question

我试图通过这个README.md通过docker运行集成测试，但是遇到了一些问题。

以下是我的开发环境：

[boblee@Wsl[19:50:36]server]$ sudo docker version
Client:
 Version:      17.06.1-ce-rc1
 EulerVersion: 17.06.0.10
 API version:  1.30
 Go version:   go1.8.3
 Git commit:   7a547fb
 Built:        Fri Mar 23 05:39:01 2018
 OS/Arch:      linux/amd64

Server:
 Version:      17.06.1-ce-rc1
 EulerVersion: 17.06.0.10
 API version:  1.30 (minimum version 1.12)
 Go version:   go1.8.3
 Git commit:   7a547fb
 Built:        Fri Mar 23 05:38:32 2018
 OS/Arch:      linux/amd64
 Experimental: false

我使用docker pull yandex/clickhouse-integration-tests-runner并从Yandex获取推荐的图像。

[boblee@Wsl[19:52:10]server]$ sudo docker images
REPOSITORY                                   TAG                 IMAGE ID            CREATED             SIZE
yandex/clickhouse-integration-tests-runner   latest              811ba8d0c7f3        5 days ago          1.49GB

唯一的要求是和码头拉yandex/clickhouse-整合-测试-转轮。

所以，我的问题是：fresh configured docker是什么意思？

我是否可以假设拉出的坞图已经有了用于集成测试的所有第三方依赖项？

然后：

我转到$CLICKHOUSE_HOME/tests/integration并运行以下命令，也是从README.md运行

sudo ./runner --binary /home/boblee/ClickHouse/build_20.13.1.1_debug/programs/clickhouse  --bridge-binary /home/boblee/ClickHouse/build_20.13.1.1_debug/programs/clickhouse-odbc-bridge --base-configs-dir /home/boblee/ClickHouse/build_20.13.1.1_debug/programs/server/ 'test_odbc_interaction -ss'

部分错误：

Start tests
======================================================================================================== test session starts =========================================================================================================platform linux -- Python 3.6.9, pytest-6.2.2, py-1.10.0, pluggy-0.13.1
rootdir: /ClickHouse/tests/integration, configfile: pytest.ini
plugins: timeout-1.4.2
timeout: 300.0s
timeout method: signal
timeout func_only: False
collecting ... ENV                  DOCKER_KERBEROS_KDC_TAG latest
ENV         CLICKHOUSE_TESTS_SERVER_BIN_PATH /clickhouse
ENV                                 HOSTNAME DESKTOP-MB3AOEA
ENV                                    SHLVL 0
ENV                              PYTEST_OPTS test_odbc_interaction -ss
ENV                                   OLDPWD /
ENV                                     HOME /root
ENV              DOCKER_MYSQL_PHP_CLIENT_TAG latest
ENV         CLICKHOUSE_TESTS_CLIENT_BIN_PATH /clickhouse
ENV               DOCKER_MYSQL_JS_CLIENT_TAG latest
ENV                                     TERM xterm
ENV                                     PATH /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
ENV                           DOCKER_CHANNEL stable
ENV        DOCKER_POSTGRESQL_JAVA_CLIENT_TAG latest
ENV           DOCKER_MYSQL_GOLANG_CLIENT_TAG latest
ENV                           DOCKER_VERSION 5:19.03.13~3-0~ubuntu-bionic
ENV                                      PWD /ClickHouse/tests/integration
ENV             DOCKER_MYSQL_JAVA_CLIENT_TAG latest
ENV       CLICKHOUSE_ODBC_BRIDGE_BINARY_PATH /clickhouse-odbc-bridge
ENV         CLICKHOUSE_TESTS_BASE_CONFIG_DIR /clickhouse-config
ENV                                       TZ Europe/Moscow
CLUSTER INIT base_config_dir:/clickhouse-config
collected 10 items

test_odbc_interaction/test.py Cluster start called. is_up=False, destroy_dirs=True
Trying to kill unstopped containers...
ERROR:
        Can't find a suitable configuration file in this directory or any
        parent. Are you in the right directory?

        Supported filenames: docker-compose.yml, docker-compose.yaml

ERROR:
        Can't find a suitable configuration file in this directory or any
        parent. Are you in the right directory?

        Supported filenames: docker-compose.yml, docker-compose.yaml

Unstopped containers killed
('Removing instances dir %s', '/ClickHouse/tests/integration/test_odbc_interaction/_instances')
Setup directory for instance: node1 destroy_dirs: True
Copy common default production configuration from /clickhouse-config
Create directory for configuration generated in this helper
Create directory for common tests configuration
Copy common configuration from helpers
Generate and write macros file
Copy custom test config files ['/ClickHouse/tests/integration/test_odbc_interaction/configs/openssl.xml', '/ClickHouse/tests/integration/test_odbc_interaction/configs/odbc_logging.xml', '/ClickHouse/tests/integration/test_odbc_interaction/configs/enable_dictionaries.xml', '/ClickHouse/tests/integration/test_odbc_interaction/configs/dictionaries/sqlite3_odbc_hashed_dictionary.xml', '/ClickHouse/tests/integration/test_odbc_interaction/configs/dictionaries/sqlite3_odbc_cached_dictionary.xml', '/ClickHouse/tests/integration/test_odbc_interaction/configs/dictionaries/postgres_odbc_hashed_dictionary.xml'] to /ClickHouse/tests/integration/test_odbc_interaction/_instances/node1/configs/config.d
Setup database dir /ClickHouse/tests/integration/test_odbc_interaction/_instances/node1/database
Setup logs dir /ClickHouse/tests/integration/test_odbc_interaction/_instances/node1/logs
Env {'LLVM_PROFILE_FILE': '/var/lib/clickhouse/server_%h_%p_%m.profraw'} stored in /ClickHouse/tests/integration/test_odbc_interaction/_instances/node1/env_file
Entrypoint cmd: bash -c "clickhouse server --config-file=/etc/clickhouse-server/config.xml --log-file=/var/log/clickhouse-server/clickhouse-server.log --errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log --daemon; tail -f /dev/null"
Setup MySQL
Stderr:
b'Creating network "roottestodbcinteraction_default" with the default driver\nPulling mysql1 (mysql:5.7)...\nGet https://registry-1.docker.io/v2/: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)\n'

Stdout:
b''

Failed to start cluster:
Command ['docker-compose', '--project-directory', '/ClickHouse/tests/integration/test_odbc_interaction', '--project-name', 'roottestodbcinteraction', '--file', '/compose/docker_compose_mysql.yml', 'up', '-d', '--force-recreate'] return non-zero code 1: b'Creating network "roottestodbcinteraction_default" with the default driver\nPulling mysql1 (mysql:5.7)...\nGet https://registry-1.docker.io/v2/: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)\n'
Traceback (most recent call last):
  File "/ClickHouse/tests/integration/helpers/cluster.py", line 655, in start
    subprocess_check_call(self.base_mysql_cmd + common_opts)
  File "/ClickHouse/tests/integration/helpers/cluster.py", line 60, in subprocess_check_call
    run_and_check(args)
  File "/ClickHouse/tests/integration/helpers/cluster.py", line 54, in run_and_check
    raise Exception('Command {} return non-zero code {}: {}'.format(args, res.returncode, res.stderr))
Exception: Command ['docker-compose', '--project-directory', '/ClickHouse/tests/integration/test_odbc_interaction', '--project-name', 'roottestodbcinteraction', '--file', '/compose/docker_compose_mysql.yml', 'up', '-d', '--force-recreate'] return non-zero code 1: b'Creating network "roottestodbcinteraction_default" with the default driver\nPulling mysql1 (mysql:5.7)...\nGet https://registry-1.docker.io/v2/: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)\n'
None
Command ['docker-compose', '--project-directory', '/ClickHouse/tests/integration/test_odbc_interaction', '--project-name', 'roottestodbcinteraction', '--file', '/compose/docker_compose_mysql.yml', 'up', '-d', '--force-recreate'] return non-zero code 1: b'Creating network "roottestodbcinteraction_default" with the default driver\nPulling mysql1 (mysql:5.7)...\nGet https://registry-1.docker.io/v2/: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)\n'
EEEEEEEEEE

=============================================================================================================== ERRORS ===============================================================================================================__________________________________________________________________________________________ ERROR at setup of test_mysql_simple_select_works __________________________________________________________________________________________

    @pytest.fixture(scope="module")
    def started_cluster():
        try:
            cluster.start()
            sqlite_db = node1.odbc_drivers["SQLite3"]["Database"]

            print("sqlite data received")
            node1.exec_in_container(
                ["bash", "-c", "echo 'CREATE TABLE t1(x INTEGER PRIMARY KEY ASC, y, z);' | sqlite3 {}".format(sqlite_db)],
                privileged=True, user='root')
            node1.exec_in_container(
                ["bash", "-c", "echo 'CREATE TABLE t2(X INTEGER PRIMARY KEY ASC, Y, Z);' | sqlite3 {}".format(sqlite_db)],
                privileged=True, user='root')
            node1.exec_in_container(
                ["bash", "-c", "echo 'CREATE TABLE t3(X INTEGER PRIMARY KEY ASC, Y, Z);' | sqlite3 {}".format(sqlite_db)],
                privileged=True, user='root')
            node1.exec_in_container(
                ["bash", "-c", "echo 'CREATE TABLE t4(X INTEGER PRIMARY KEY ASC, Y, Z);' | sqlite3 {}".format(sqlite_db)],
                privileged=True, user='root')
            print("sqlite tables created")
            mysql_conn = get_mysql_conn()
            print("mysql connection received")
            ## create mysql db and table
            create_mysql_db(mysql_conn, 'clickhouse')
            print("mysql database created")

            postgres_conn = get_postgres_conn()
            print("postgres connection received")

            create_postgres_db(postgres_conn, 'clickhouse')
            print("postgres db created")

            cursor = postgres_conn.cursor()
            cursor.execute(
                "create table if not exists clickhouse.test_table (column1 int primary key, column2 varchar(40) not null)")

            yield cluster

        except Exception as ex:
            print(ex)
>           raise ex

test_odbc_interaction/test.py:98:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

目前，由于找不到docker-compose.yaml，码头似乎无法正确运行。

错误:无法在此目录或任何父目录中找到合适的配置文件。你在正确的目录里吗？支持的文件名: docker-compose.yml，docker-come.yaml

有什么想法吗？

Answer 1

我解决了问题，回到这里来分享我的解决方案。

这个问题可能是由网络环境造成的。如果您为Docker配置了一个代理，它可能有助于更改代理，或者只需手动提取映像(您可以从$ClickHouse/docker/test/integration/runner/compose/xxx.yml)中找到所需的映像。