配置Minio服务器以与Testcontainers一起使用

Question

我的应用程序使用Minio进行与S3兼容的对象存储，并且我想通过Testcontainers在集成测试中使用Minio docker镜像。

对于一些非常基本的测试，我使用minio/minio docker镜像运行一个GenericContainer，除了MINIO_ACCESS_KEY和MINIO_SECRET_KEY之外没有任何配置。然后，我的测试使用的是Minio的Java Client SDK。这些都工作得很好，表现与预期的一样。

但是对于其他集成测试，我需要在Mino中设置单独的用户。据我所知，用户只能使用Admin API添加到Minio，其中没有Java客户端，只有minio/mc docker镜像( mc CLI在用于服务器的minio/minio docker镜像中不可用)。

在命令行上，我可以像这样使用Admin API：

$ docker run --interactive --tty --detach --entrypoint=/bin/sh --name minio_admin minio/mc

为了保持容器的运行，--interactive --tty是一个小技巧，这样我以后就可以运行像这样的命令：

$ docker exec --interactive --tty minio_admin mc admin user add ...

使用Testcontainers，我尝试这样做：

public void testAdminApi() throws Exception {
    GenericContainer mc = new GenericContainer("minio/mc")
            .withCommand("/bin/sh")
            .withCreateContainerCmdModifier(new Consumer<CreateContainerCmd>() {
                @Override
                public void accept(CreateContainerCmd cmd) {
                    cmd
                            .withAttachStdin(true)
                            .withStdinOpen(true)
                            .withTty(true);
                }
            });

    mc.start();
    log.info("mc is running: {}", mc.isRunning());

    String command = "mc";
    Container.ExecResult result = mc.execInContainer(command);
    log.info("Executing command '{}' returned exit code '{}' and stdout '{}'", command, result.getExitCode(), result.getStdout());

    assertEquals(0, result.getExitCode());
}

日志显示容器正在启动，但对它执行命令会返回退出代码126，并声明它处于停止状态：

[minio/mc:latest] - Starting container with ID: 4f96fc7583fe62290925472c4c6b329fbeb7a55b38a3c0ad41ee797db1431841
[minio/mc:latest] - Container minio/mc:latest is starting: 4f96fc7583fe62290925472c4c6b329fbeb7a55b38a3c0ad41ee797db1431841
[minio/mc:latest] - Container minio/mc:latest started
minio.MinioAdminTests - mc is running: true
org.testcontainers.containers.ExecInContainerPattern - /kind_volhard: Running "exec" command: mc
minio.MinioAdminTests - Executing command 'mc' returned exit code '126'
  and stdout 'cannot exec in a stopped state: unknown'

java.lang.AssertionError: Expected: 0, Actual: 126

摆弄了几个小时之后，我就想不出点子了。有人能帮上忙吗？

Answer 1

多亏了@glebsts和@bsideup，我才能让我的集成测试正常工作。下面是一个如何添加用户的最小示例：

public class MinioIntegrationTest {

    private static final String ADMIN_ACCESS_KEY = "admin";
    private static final String ADMIN_SECRET_KEY = "12345678";
    private static final String USER_ACCESS_KEY = "bob";
    private static final String USER_SECRET_KEY = "87654321";

    private static GenericContainer minioServer;
    private static String minioServerUrl;

    @BeforeAll
    static void setUp() throws Exception {
        int port = 9000;
        minioServer = new GenericContainer("minio/minio")
                .withEnv("MINIO_ACCESS_KEY", ADMIN_ACCESS_KEY)
                .withEnv("MINIO_SECRET_KEY", ADMIN_SECRET_KEY)
                .withCommand("server /data")
                .withExposedPorts(port)
                .waitingFor(new HttpWaitStrategy()
                        .forPath("/minio/health/ready")
                        .forPort(port)
                        .withStartupTimeout(Duration.ofSeconds(10)));
        minioServer.start();

        Integer mappedPort = minioServer.getFirstMappedPort();
        Testcontainers.exposeHostPorts(mappedPort);
        minioServerUrl = String.format("http://%s:%s", minioServer.getContainerIpAddress(), mappedPort);

        // Minio Java SDK uses s3v4 protocol by default, need to specify explicitly for mc
        String cmdTpl = "mc config host add myminio http://host.testcontainers.internal:%s %s %s --api s3v4 && "
                + "mc admin user add myminio %s %s readwrite";
        String cmd = String.format(cmdTpl, mappedPort, ADMIN_ACCESS_KEY, ADMIN_SECRET_KEY, USER_ACCESS_KEY, USER_SECRET_KEY);

        GenericContainer mcContainer = new GenericContainer<>("minio/mc")
                .withStartupCheckStrategy(new OneShotStartupCheckStrategy())
                .withCreateContainerCmdModifier(containerCommand -> containerCommand
                        .withTty(true)
                        .withEntrypoint("/bin/sh", "-c", cmd));
        mcContainer.start();
    }

    @Test
    public void canCreateBucketWithAdminUser() throws Exception {
        MinioClient client = new MinioClient(minioServerUrl, ADMIN_ACCESS_KEY, ADMIN_SECRET_KEY);
        client.ignoreCertCheck();

        String bucketName = "foo";
        client.makeBucket(bucketName);
        assertTrue(client.bucketExists(bucketName));
    }

    @Test
    public void canCreateBucketWithNonAdminUser() throws Exception {
        MinioClient client = new MinioClient(minioServerUrl, USER_ACCESS_KEY, USER_SECRET_KEY);
        client.ignoreCertCheck();

        String bucketName = "bar";
        client.makeBucket(bucketName);
        assertTrue(client.bucketExists(bucketName));
    }

    @AfterAll
    static void shutDown() {
        if (minioServer.isRunning()) {
            minioServer.stop();
        }
    }
}

Answer 2

正如@bsideup建议的那样，你可以使用one-shot策略，例如在here中。UPD:新增工作测试。这里很重要的一点是要知道

容器启动时，执行entrypoint +命令(一般为Docker，与Testcontainers无关)。Source from TC github

public class TempTest {
    @Rule
    public Network network = Network.newNetwork();

    private String runMcCommand(String cmd) throws TimeoutException {
        GenericContainer container = new GenericContainer<>("minio/mc")
                .withCommand(cmd)
                .withNetwork(network)
                .withStartupCheckStrategy(new OneShotStartupCheckStrategy())
                .withCreateContainerCmdModifier(command -> command.withTty(true));
        container.start();
        WaitingConsumer waitingConsumer = new WaitingConsumer();
        ToStringConsumer toStringConsumer = new ToStringConsumer();
        Consumer<OutputFrame> composedConsumer = toStringConsumer.andThen(waitingConsumer);
        container.followOutput(composedConsumer);
        waitingConsumer.waitUntilEnd(4, TimeUnit.SECONDS);
        return toStringConsumer.toUtf8String();
    }

    private void showCommandOutput(String cmd) throws TimeoutException {
        String res = runMcCommand(cmd);
        System.out.printf("Cmd '%s' result:\n----\n%s\n----%n", cmd, res);
    }

    @Test
    public void testAdminApi() throws Exception {
        showCommandOutput("ls");
        showCommandOutput("version");
    }
}

另一种选择是使用minio/mc的dockerfile的内容，它很小，修改执行的命令(默认是一次性的mc)，每次测试运行自己的容器一次，如果需要执行多个命令，这将比一次性容器节省一些时间：

@Rule
public Network network = Network.newNetwork();

@Rule
public GenericContainer mc = new GenericContainer(new ImageFromDockerfile()
  .withDockerfileFromBuilder(builder ->
    builder
      .from("alpine:3.7")
      .run("apk add --no-cache ca-certificates && apk add --no-cache --virtual .build-deps curl && curl https://dl.minio.io/client/mc/release/linux-amd64/mc > /usr/bin/mc && chmod +x /usr/bin/mc && apk del .build-deps")
      .cmd("/bin/sh", "-c", "while sleep 3600; do :; done")
      .build())
    )
  .withNetwork(network);

public void myTest() {
  mc.execInContainer("mc blah");
  mc.execInContainer("mc foo");
}

基本上，它在安装了mc的情况下运行image，并休眠1小时，这足以满足您的测试。当它运行时，你可以执行命令等等。在你完成后，它就会被杀死。您的微型容器可以在同一网络中。

Answer 3

您可以使用mc和withCommand("your command")运行一次性容器(使用OneShotStartupCheckStrategy)，连接到与您正在运行的微型服务器相同的网络(请参阅Networking)。