Snakemake不能很好地处理wget

Question

我正在尝试使用snakemake从网站下载文件，然后将它们放在一起。

但是，我总是得到错误：Waiting at most 5 seconds for missing files. MissingOutputException in line 24 of /path/to/Snakefile:

为什么snakemake不等待文件下载完成后再继续？将所有读操作放在不同的目录中对我来说很不方便，而且我不想麻烦地创建一个配置文件，因为这是一个一次性的Snakefile。

谢谢!

下面是我的脚本：

import os
rule all:
    input:
        "ONT/yeastONT_combined.fastq.gz",
        "trimmed/ERR1938684_1.trim.final.fastq.gz",
        "trimmed/ERR1938684_2.trim.final.fastq.gz",
        "trimmed/ERR1938684_1.trim.unpaired.fastq.gz",
        "trimmed/ERR1938684_2.trim.unpaired.fastq.gz"

rule getONTfwd:
    input:

    output:
        "ONT/ERR1883385_1.fastq.gz",
        "ONT/ERR1883386_1.fastq.gz",
        "ONT/ERR1883387_1.fastq.gz",
        "ONT/ERR1883393_1.fastq.gz",
        "ONT/ERR1883395_1.fastq.gz",
        "ONT/ERR1883396_1.fastq.gz"

    shell:
        """cd ONT \
        wget 'ftp://ftp.sra.ebi.ac.uk/vol1/fastq/ERR188/005/ERR1883385/ERR1883385_1.fastq.gz' \
        wget 'ftp://ftp.sra.ebi.ac.uk/vol1/fastq/ERR188/006/ERR1883386/ERR1883386_1.fastq.gz' \
        wget 'ftp://ftp.sra.ebi.ac.uk/vol1/fastq/ERR188/007/ERR1883387/ERR1883387_1.fastq.gz' \
        wget 'ftp://ftp.sra.ebi.ac.uk/vol1/fastq/ERR188/003/ERR1883393/ERR1883393_1.fastq.gz' \
        wget 'ftp://ftp.sra.ebi.ac.uk/vol1/fastq/ERR188/005/ERR1883395/ERR1883395_1.fastq.gz' \
        wget 'ftp://ftp.sra.ebi.ac.uk/vol1/fastq/ERR188/006/ERR1883396/ERR1883396_1.fastq.gz' \
        sleep 300 \
        cd .."""

rule combine_ONT:
    input:
        f1 = "ONT/ERR1883385_1.fastq.gz",
        f2 = "ONT/ERR1883386_1.fastq.gz",
        f3 = "ONT/ERR1883387_1.fastq.gz",
        f4 = "ONT/ERR1883393_1.fastq.gz",
        f5 = "ONT/ERR1883395_1.fastq.gz",
        f6 = "ONT/ERR1883396_1.fastq.gz"
    output:
        "ONT/yeastONT_combined.fastq.gz"
    shell:
        """cat {input.f1} {input.f2} {input.f3} {input.f4} {input.f5} {input.f6} > {output}"""

Answer 1

规则getONTfwd的外壳命令中存在语法错误，您可以使用\对每个换行符进行转义；这会导致将整个外壳命令视为单个命令。删除转义字符\或在转义符号前添加分号以分隔命令(如; \)

此外，如果您只是为了提供下载所有文件的缓冲时间而使用sleep 300，则不需要使用它们。正如Johannes在评论中提到的，wget退出时只下载url中的文件。而且，您的示例脚本中缺少文件trimmed/*.fastq.gz的规则。

下面是您的示例的编辑版本，它应该可以按预期工作：

import os
rule all:
    input:
        "ONT/yeastONT_combined.fastq.gz",
        # "trimmed/ERR1938684_1.trim.final.fastq.gz",
        # "trimmed/ERR1938684_2.trim.final.fastq.gz",
        # "trimmed/ERR1938684_1.trim.unpaired.fastq.gz",
        # "trimmed/ERR1938684_2.trim.unpaired.fastq.gz"

rule getONTfwd:
    output:
        "ONT/ERR1883385_1.fastq.gz",
        "ONT/ERR1883386_1.fastq.gz",
        "ONT/ERR1883387_1.fastq.gz",
        "ONT/ERR1883393_1.fastq.gz",
        "ONT/ERR1883395_1.fastq.gz",
        "ONT/ERR1883396_1.fastq.gz"

    shell:
        """cd ONT
        wget 'ftp://ftp.sra.ebi.ac.uk/vol1/fastq/ERR188/005/ERR1883385/ERR1883385_1.fastq.gz'
        wget 'ftp://ftp.sra.ebi.ac.uk/vol1/fastq/ERR188/006/ERR1883386/ERR1883386_1.fastq.gz'
        wget 'ftp://ftp.sra.ebi.ac.uk/vol1/fastq/ERR188/007/ERR1883387/ERR1883387_1.fastq.gz'
        wget 'ftp://ftp.sra.ebi.ac.uk/vol1/fastq/ERR188/003/ERR1883393/ERR1883393_1.fastq.gz'
        wget 'ftp://ftp.sra.ebi.ac.uk/vol1/fastq/ERR188/005/ERR1883395/ERR1883395_1.fastq.gz'
        wget 'ftp://ftp.sra.ebi.ac.uk/vol1/fastq/ERR188/006/ERR1883396/ERR1883396_1.fastq.gz'
        cd .."""

rule combine_ONT:
    input:
        f1 = "ONT/ERR1883385_1.fastq.gz",
        f2 = "ONT/ERR1883386_1.fastq.gz",
        f3 = "ONT/ERR1883387_1.fastq.gz",
        f4 = "ONT/ERR1883393_1.fastq.gz",
        f5 = "ONT/ERR1883395_1.fastq.gz",
        f6 = "ONT/ERR1883396_1.fastq.gz"
    output:
        "ONT/yeastONT_combined.fastq.gz"
    shell:
        """cat {input.f1} {input.f2} {input.f3} {input.f4} {input.f5} {input.f6} > {output}"""