Snakemake在使用时给出InputFunctionException --profile slurm

Question

我正在创建一个管道，使用snakemake在纳米孔测序数据中调用甲基化。我已经使用--dryrun选项运行了snakenake，并且成功地构建了dag。但是当我添加选项--profile slurm时，我得到以下错误：

(nanopolish) [danielle.perley@talonhead2 nanopolish-CpG-calling]$ snakemake -np --use-conda --profile slurm test_data/20-001-002/20-001-002_fastq_pass.gz

Building DAG of jobs...
Job counts:
    count   jobs
    1   combine_tech_reps
    1
InputFunctionException in line 32 of /home/danielle.perley/nanopolish-CpG-calling/Snakefile:
Error:
  SyntaxError: invalid syntax (<string>, line 1)
Wildcards:
  sample=20-001-002
Traceback:

  File "/home/danielle.perley/miniconda3/envs/nanopolish/lib/python3.7/site-packages/snakemake/executors/__init__.py", line 115, in run_jobs
  File "/home/danielle.perley/miniconda3/envs/nanopolish/lib/python3.7/site-packages/snakemake/executors/__init__.py", line 120, in run
  File "/home/danielle.perley/miniconda3/envs/nanopolish/lib/python3.7/site-packages/snakemake/executors/__init__.py", line 131, in _run
  File "/home/danielle.perley/miniconda3/envs/nanopolish/lib/python3.7/site-packages/snakemake/executors/__init__.py", line 151, in printjob
  File "/home/danielle.perley/miniconda3/envs/nanopolish/lib/python3.7/site-packages/snakemake/executors/__init__.py", line 137, in printjob

第33行是我的snakefile中的规则combine_tech_reps。(我在这里只展示了我的snakefile的第一部分)

from snakemake.utils import validate
import pandas as pd
import os.path
import glob

configfile: "config.yaml"

samples_df = pd.read_table(config["samples"],sep = '\t')
samples_df = samples_df.set_index("Sample")
samples = list(samples_df.index.unique())


wildcard_constraints:
    sample = "|".join(samples)
         
def get_fast5(wildcards):
      
    f5 = glob.glob(os.path.join(config["raw_data"],wildcards.sample,"2*","fast5_pass"))
    return(f5)

localrules: all,build_index

rule all:
    input: 
        expand("results/Methylation/{sample}_frequency.tsv",sample=samples),
        expand("results/alignments/{sample}_flagstat.txt",sample=samples),
        expand("resources/QC/{sample}_pycoQC.json",sample=samples),
        expand("results/QC/{sample}_pycoQC.html",sample=samples),
        "report/multiQC.html"


rule combine_tech_reps:
    input:
        fqs = lambda wildcards: glob.glob(os.path.join(config["raw_data"],"{sample}","2*","{sample}_fastq_pass.gz").format(sample=wildcards.sample))

    output:
        fq = os.path.join(config["raw_data"],"{sample}","{sample}_fastq_pass.gz")

    shell: """
        zcat {input} > {output}
    """

我在~/.config/snakemake/ slurm /config.yaml目录中有一个slurm配置文件

jobs: 10
cluster: "sbatch -p talon -t {resources.time} --mem={resources.mem} -c {resources.cpus} -o logs_slurm/{rule}_{wildcards} -e logs_slurm/{rule}_{wildcards}"
default-resources: [cpus=1, mem=2000, time=10:00]
use-conda: true

我真的很想在我们的HPC上使用这个管道，但我不确定是什么导致了这个错误。

Answer 1

在这篇文章的帮助下，我解决了我的问题：

InputFunctionException: unexpected EOF while parsing

通过添加详细标志：

snakemake -np --verbose --use-conda --profile slurm test_data/20-001-002/20-001-002_fastq_pass.gz

我可以看到snakemake在默认资源上有问题：

10:00
   ^

更改我的config.yaml文件的默认资源行：

default-resources: [cpus=1, mem=2000, time=600]

已删除错误。

Answer 2

我不确定default-resources是否是配置中的有效键。

如果您以config.yaml用户身份尝试，会发生什么情况：

jobs: 10
cluster: "sbatch -p talon -t {resources.time} --mem={resources.mem} -c {resources.cpus} -o logs_slurm/{rule}_{wildcards} -e logs_slurm/{rule}_{wildcards}"
use-conda: true

__default__:
    time: 10
    cpus: 1
    mem: 2GB