对大型迭代中的循环进行划分

Question

我的问题的目标是通过在一个大的数组列表上拆分循环迭代的范围来增强我的算法的性能。

例如:我有一个数组列表，大约有100亿个长值条目，我试图实现的目标是从0到1亿个条目开始循环，输出循环中所有计算的1亿个条目的结果；然后开始，1亿到2亿执行前面的计算并输出结果，然后300-4亿，400-5亿等等。在我得到所有1000亿/1亿个结果之后，我可以在循环外对它们进行求和，从循环输出中并行收集结果。

我曾尝试使用一个范围，通过尝试使用动态范围移位方法，它可能能够实现类似的结果，但我似乎无法像我希望的那样完全实现逻辑。

public static void tt4() {
    long essir2 = 0;
    long essir3 = 0;

    List cc = new ArrayList<>();  
    List<Long> range = new ArrayList<>();  

    // break point is a method that returns list values, it was converted to 
    // string because of some concatenations and would be converted back to long here
    for (String ari1 : Breakpoint()) {
        cc.add(Long.valueOf(ari1));
    }  

    // the size of the List is huge about 1 trillion entries at the minimum
    long hy = cc.size() - 1;

    for (long k = 0; k < hy; k++) { 
        long t1 = (long) cc.get((int) k);
        long t2 = (long) cc.get((int) (k + 1)); 

        // My main question: I am trying to iterate the entire list in a dynamic way 
        // which would exclude repeated endpoints on each iteration. 

        range = LongStream.rangeClosed(t1 + 1, t2)
                          .boxed()
                          .collect(Collectors.toList());  

        for (long i : range) {
            // Hard is another method call on the iteration
            // complexcalc is a method as well

            essir2 = complexcalc((int) i, (int) Hard(i)); 
            essir3 += essir2;
        } 
    }

    System.out.println("\n" + essir3);  
}

我没有任何错误，我只是在寻找一种方法来提高性能和时间。我可以在不到一秒的时间内直接完成一百万个条目，但当我放置所需的大小时，它将永远运行。我给出的大小是用来说明大小的抽象，我不想要像1000亿这样的观点，如果我能在一秒内做一百万，我说的是巨大的数字，我需要在做复杂任务和调用时迭代，我只是需要帮助，如果我可以的话，我正在尝试实现的逻辑。

Answer 1

我建议马上做的一件事就是将Breakpoint返回值存储在一个简单的数组中，而不是使用List。这将显著改善您的执行时间：

    List<Long> cc = new ArrayList<>();
    for (String ari1 : Breakpoint()) {
        cc.add(Long.valueOf(ari1));
    }
    Long[] ccArray = cc.toArray(new Long[0]);

我相信你正在寻找的是将你的任务分散到多个线程中。你可以用ExecutorService来做到这一点，“它简化了异步模式下的任务执行”。

请注意，我对整个概念并不是非常熟悉，但最近我对它进行了一些实验，并给出了一个如何实现它的快速草稿。

我欢迎那些在多线程方面更有经验的人更正这篇文章，或者在评论中提供更多信息来帮助改进这个答案。

可运行任务类

public class CompartmentalizationTask implements Runnable {

    private final ArrayList<Long> cc;
    private final long index;

    public CompartmentalizationTask(ArrayList<Long> list, long index) {

        this.cc = list;
        this.index = index;
    }

    @Override
    public void run() {
        Main.compartmentalize(cc, index);
    }
}

主类

private static ExecutorService exeService = Executors.newCachedThreadPool();
private static List<Future> futureTasks = new ArrayList<>();

public static void tt4() throws ExecutionException, InterruptedException 
{
    long essir2 = 0;
    long essir3 = 0;

    ArrayList<Long> cc = new ArrayList<>();
    List<Long> range = new ArrayList<>();

    // break point is a method that returns list values, it was converted to
    // string because of some concatenations and would be converted back to long here
    for (String ari1 : Breakpoint()) {
        cc.add(Long.valueOf(ari1));
    }

    // the size of the List is huge about 1 trillion entries at the minimum
    long hy = cc.size() - 1;

    for (long k = 0; k < hy; k++) {
        futureTasks.add(Main.exeService.submit(new CompartmentalizationTask(cc, k)));
    }
    for (int i = 0; i < futureTasks.size(); i++) {
        futureTasks.get(i).get();
    }
    exeService.shutdown();
}

public static void compartmentalize(ArrayList<Long> cc, long index)
{
    long t1 = (long) cc.get((int) index);
    long t2 = (long) cc.get((int) (index + 1));

    // My main question: I am trying to iterate the entire list in a dynamic way
    // which would exclude repeated endpoints on each iteration.

    range = LongStream.rangeClosed(t1 + 1, t2)
            .boxed()
            .collect(Collectors.toList());

    for (long i : range) {
        // Hard is another method call on the iteration
        // complexcalc is a method as well

        essir2 = complexcalc((int) i, (int) Hard(i));
        essir3 += essir2;
    }
}