如何让azcopy在docker中工作

Question

我的dockerfile

FROM microsoft/dotnet:2.0.4-runtime

RUN apt-get update \
    && apt-get install -y --no-install-recommends \
         rsync \
    && rm -rf /var/lib/apt/lists/*

RUN set -ex \
    && curl -L -o azcopy.tar.gz https://aka.ms/downloadazcopyprlinux \
    && tar -xf azcopy.tar.gz && rm -f azcopy.tar.gz \
    && ./install.sh && rm -f install.sh \
    && rm -rf azcopy

我正在使用dotnet客户端启动容器

    var container = await client.Containers.CreateContainerAsync(new CreateContainerParameters
    {
        Hostname = "",
        Domainname = "",
        User = "",
        AttachStdin = false,
        AttachStderr = true,
        AttachStdout = true,
        Tty = true,
        Volumes = ...
        Image = ...
        Cmd = arguments,            
        HostConfig = new HostConfig
        {
            LogConfig = new LogConfig { Type = "json-file" },
            AutoRemove = false,
            Binds = binds
        }

    });

等同于

docker run -v dataout:/data1/S2B_MSIL2A_20180107T102359_N0206_R065_T33UUB_20180107T121759.SAFE earthml/azcopy azcopy --source /data1/S2B_MSIL2A_20180107T102359_N0206_R065_T33UUB_20180107T121759.SAFE --destination https://eodata.blob.core.windows.net/test --dest-key  --recursive

但是得到了以下异常：

Unhandled Exception: System.ArgumentOutOfRangeException: Length cannot be less than zero.
Parameter name: length
   at System.String.Substring(Int32 startIndex, Int32 length)
   at Microsoft.WindowsAzure.Storage.AzCopy.Interaction.WriteConsoleAtCursor(String message, Boolean finished)
   at Microsoft.WindowsAzure.Storage.AzCopy.AzCopy.OutputNonVerboseTransferStatus(Nullable`1 transferredBytes, Boolean finished)
   at Microsoft.WindowsAzure.Storage.AzCopy.AzCopy.ProgressUpdate(TransferStatus status)
   at Microsoft.WindowsAzure.Storage.DataMovement.TransferProgressTracker.InvokeProgressHandler()
   at Microsoft.WindowsAzure.Storage.DataMovement.TransferProgressTracker.AddProgress(TransferProgressTracker progressTracker)
   at Microsoft.WindowsAzure.Storage.DataMovement.TransferCollection`1.AddTransfer(Transfer transfer, Boolean updateProgress)
   at Microsoft.WindowsAzure.Storage.DataMovement.TransferManager.GetOrCreateDirectoryTransfer(TransferLocation sourceLocation, TransferLocation destLocation, TransferMethod transferMethod, TransferContext transferContext)
   at Microsoft.WindowsAzure.Storage.DataMovement.TransferManager.<UploadDirectoryInternalAsync>d__69.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.WindowsAzure.Storage.AzCopy.DirectoryTransferLauncher.<UploadDirectory>d__3.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.WindowsAzure.Storage.AzCopy.DirectoryTransferLauncher.<ExecuteImplAsync>d__1.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.WindowsAzure.Storage.AzCopy.TransferLauncherBase.<ExecuteAsync>d__14.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.WindowsAzure.Storage.AzCopy.BlobFileTransfer.DoTransfer()
   at Microsoft.WindowsAzure.Storage.AzCopy.AzCopy.ExecuteTransfer()
   at Microsoft.WindowsAzure.Storage.AzCopy.AzCopy.Execute(String[] args)
   at Microsoft.WindowsAzure.Storage.AzCopy.AzCopy.Main(String[] args)
/usr/bin/../lib/azcopy/azcopy: line 10:    15 Aborted                 dotnet $DIR/bin/azcopy.dll "$@"

更新

我想说清楚一点，我使用代码来运行容器，在azcopy工具中，异常在容器内部抛出。

从callstack中，我们了解到azcopy正在使用控制台进行进度报告。也许这在码头容器中是不允许的。有没有办法给create/start容器调用一些参数来允许它所做的事情。

更新2

我使用CLI运行该命令，该命令正常工作。我用-it和不用都试过了。这两种方法都有效。所以我回到了dotnet docker客户端，什么是正确的参数来模拟CLI正在做的事情。

PS C:\Users\pks> docker run -v E8A9547E29F57C223876982D46564150:/data1/S2B_MSIL2A_20180107T102359_N0206_R065_T33UUB_20180107T121759.SAFE earthml/azcopy azcopy --source /data1/S2B_MSIL2A_20180107T102359_N0206_R065_T33UUB_20180107T121759.SAFE --destination ht
tps://eodata.blob.core.windows.net/test --dest-key "" --recursive
[2018/01/31 21:53:59] Transfer summary:
-----------------
Total files transferred: 113
Transfer successfully:   113
Transfer skipped:        0
Transfer failed:         0
Elapsed time:            00.00:00:07
PS C:\Users\pks> docker run -it -v E8A9547E29F57C223876982D46564150:/data1/S2B_MSIL2A_20180107T102359_N0206_R065_T33UUB_20180107T121759.SAFE earthml/azcopy azcopy --source /data1/S2B_MSIL2A_20180107T102359_N0206_R065_T33UUB_20180107T121759.SAFE --destinatio
n https://eodata.blob.core.windows.net/test1 --dest-key "" --recursive
Finished 113 of total 113 file(s).
[2018/01/31 21:54:47] Transfer summary:
-----------------
Total files transferred: 113
Transfer successfully:   113
Transfer skipped:        0
Transfer failed:         0
Elapsed time:            00.00:00:06
PS C:\Users\pks>

Answer 1

用下面的代码解决了这个问题

var container = await client.Containers.CreateContainerAsync(new CreateContainerParameters
{
    Hostname = "",
    Domainname = "",
    User = "",
    AttachStdin = false,
    AttachStderr = f,alse
    AttachStdout = false,
    Tty = false,
    Volumes = ...
    Image = ...
    Cmd = arguments,            
    HostConfig = new HostConfig
    {
        LogConfig = new LogConfig { Type = "json-file" },
        AutoRemove = false,
        Binds = binds
    }

});