并行处理的ForkJoinPool

Question

我试图运行一些代码100万次。我最初是用线程编写的，但这似乎很笨重。我开始做更多的阅读，我偶然发现了ForkJoin。这似乎正是我所需要的，但我不知道如何将下面的内容转换为“scala样式”。有人能解释一下在我的代码中使用ForkJoin的最佳方法吗？

val l = (1 to 1000000) map {_.toLong}
println("running......be patient")
l.foreach{ x =>
    if(x % 10000 == 0) println("got to: "+x)
    val thread = new Thread {
        override def run { 
         //my code (API calls) here. writes to file if call success
        }
    }
}

Answer 1

最简单的方法是使用par (它将自动使用ForkJoinPool )：

 val l = (1 to 1000000) map {_.toLong} toList

 l.par.foreach { x =>
    if(x % 10000 == 0) println("got to: " + x) //will be executed in parallel way
    //your code (API calls) here. will also be executed in parallel way (but in same thread with `println("got to: " + x)`)
 }

另一种方法是使用Future

import scala.concurrent._
import ExecutionContext.Implicits.global //import ForkJoinPool

val l = (1 to 1000000) map {_.toLong}

println("running......be patient")

l.foreach { x =>
    if(x % 10000 == 0) println("got to: "+x)
    Future {
       //your code (API calls) here. writes to file if call success
    }
}

如果你需要偷工作，你应该用scala.concurrent.blocking标记阻塞代码。

Future {
   scala.concurrent.blocking {
      //blocking API call here
   }
}

它将告诉ForkJoinPool用新线程来补偿阻塞的线程--这样您就可以避免线程饥饿(但是有一些缺点)。

Answer 2

在Scala中，您可以使用未来与承诺

val l = (1 to 1000000) map {
  _.toLong
}
println("running......be patient")
l.foreach { x =>
  if (x % 10000 == 0) println("got to: " + x)
  Future{
    println(x)
  }
}