从大目录和子目录中搜索文件

Question

基本上，我有一个大目录，有许多子文件夹和不同格式的文件。它通过XML、JAVA、.txt等进行读取，以搜索用户输入的选定字符串。结果以片段的形式返回，读取之前和所选字符串之后的一定数量的字符。

它确实按计划运行.我通过Spring和AJAX通过REST调用将结果作为片段和链接发送到主页面。在本地主机服务器上，返回一个目录( 19,405个文件，9,610个文件夹)的结果确实需要5-7秒。

private ArrayList<String> pathNames = new ArrayList();
private HashMap<String, String> snippets = new HashMap<String, String>();

public HashMap readFiles(String search, String dirPath) {

    snippets.clear();
    InputStream stream = null;

    //Method to get all the files in main and sub folders
    listDirectory(dirPath);

    for (int i = 0; i < pathNames.size(); i++) {

        try {
            stream = new FileInputStream(pathNames.get(i));
            BufferedReader br = new BufferedReader(new    InputStreamReader(stream));

            Field chars = br.getClass().getDeclaredField("cb");
            chars.setAccessible(true);

            Field f = br.getClass().getDeclaredField("nChars"); 
            f.setAccessible(true);

            Field ff = br.getClass().getDeclaredField("nextChar");
            ff.setAccessible(true);

            String line;
            //not used
            int lineNum = 0;
            String path = pathNames.get(i);
            while ((line = br.readLine()) != null) {

                //Getting through to Java Source Code - uh oh

                // int charCount = (int) f.get(br);
                char[] cb = (char[]) chars.get(br);
                int nextChar = (Integer) ff.get(br);

                //not used
                lineNum++;

                if (line.contains(search)) {
                    String lines = "";
                    for (int t = nextChar - 200; t < nextChar + 100; t++) {
                        lines += cb[t];
                    }

                    snippets.put(pathNames.get(i), lines);

                }
            }

        } catch (FileNotFoundException ex) {
            Logger.getLogger(FileRead.class.getName()).log(Level.SEVERE, null, ex);
        } catch (IOException ex) {
            Logger.getLogger(FileRead.class.getName()).log(Level.SEVERE, null, ex);
        } catch (IllegalArgumentException ex) {
            Logger.getLogger(FileRead.class.getName()).log(Level.SEVERE, null, ex);
        } catch (IllegalAccessException ex) {
            Logger.getLogger(FileRead.class.getName()).log(Level.SEVERE, null, ex);
        } catch (NoSuchFieldException ex) {
            Logger.getLogger(FileRead.class.getName()).log(Level.SEVERE, null, ex);
        } catch (SecurityException ex) {
            Logger.getLogger(FileRead.class.getName()).log(Level.SEVERE, null, ex);
        }
    }
    return snippets;
}

Answer 1

对于任何感兴趣的人，下面是NetBeans对优化代码(上面讨论的)进行格式化的方式。它确实迭代了大约20,000个文件和10,000个文件夹。这可能不是迭代这么多文件和文件夹的最佳方法，但它确实会以某种及时的方式产生结果。以上两种代码在速度上似乎有很小的差异，但我的测试方法并不理想，而且在大多数情况下似乎是相似的。

private ArrayList<String> pathNames = new ArrayList();
private HashMap<String, String> snippets = new HashMap<String, String>();

public HashMap readFiles(String search, String dirPath) {

    snippets.clear();

    //Method to get all the files in main and sub folders
    listDirectory(dirPath);

    for (int i = 0; i < pathNames.size(); i++) {
        String path = pathNames.get(i);

        try (InputStream stream = new FileInputStream(path); BufferedReader br = new BufferedReader(new InputStreamReader(stream))) {

            Field chars = br.getClass().getDeclaredField("cb");
            chars.setAccessible(true);

            Field ff = br.getClass().getDeclaredField("nextChar");
            ff.setAccessible(true);

            String line;
            while ((line = br.readLine()) != null) {
                if (line.contains(search)) {
                    //Getting through to Java Source Code - uh oh
                    char[] cb = (char[]) chars.get(br);
                    int nextChar = (Integer) ff.get(br);
                    char[] charBuffer = new char[300];
                    System.arraycopy(cb, nextChar - 200, charBuffer, 0, 300);
                    snippets.put(path, new String(charBuffer));

                }
            }

        } catch (IOException | NoSuchFieldException | SecurityException | IllegalAccessException ex) {
            Logger.getLogger(FileRead.class.getName()).log(Level.SEVERE, null, ex);
        }

    }
    return snippets;
}

Answer 2

        //not used
        int lineNum = 0;

            //not used
            lineNum++;

所以它在你的密码里是因为.？

stream = new FileInputStream(pathNames.get(i));

然后以后..。

String path = pathNames.get(i);

您可以先为string变量赋值，然后从string变量生成输入流。

    } catch (FileNotFoundException ex) {
        Logger.getLogger(FileRead.class.getName()).log(Level.SEVERE, null, ex);
    } catch (IOException ex) {
        Logger.getLogger(FileRead.class.getName()).log(Level.SEVERE, null, ex);
    } catch (IllegalArgumentException ex) {
        Logger.getLogger(FileRead.class.getName()).log(Level.SEVERE, null, ex);
    } catch (IllegalAccessException ex) {
        Logger.getLogger(FileRead.class.getName()).log(Level.SEVERE, null, ex);
    } catch (NoSuchFieldException ex) {
        Logger.getLogger(FileRead.class.getName()).log(Level.SEVERE, null, ex);
    } catch (SecurityException ex) {
        Logger.getLogger(FileRead.class.getName()).log(Level.SEVERE, null, ex);
    }

使用多渔获物：

    } catch (FileNotFoundException | IOException | IllegalArgumentException | NoSuchFieldException | SecurityException ex) {
        Logger.getLogger(FileRead.class.getName()).log(Level.SEVERE, null, ex);
    }

其中一些可能是子类，因此需要删除。但是没有必要有6次相同的代码行。

        Field f = br.getClass().getDeclaredField("nChars"); 
        f.setAccessible(true);

此字段也未使用。

                String lines = "";
                for (int t = nextChar - 200; t < nextChar + 100; t++) {
                    lines += cb[t];
                }

考虑声明一个char[]变量并使用System.arraycopy。字符串是不可变的，因此，无论打开BufferedReader对性能有什么好处，您都完全浪费了它，然后在这里浪费了一些。然后，您可以将char[]放入new String中。

snippets.put(pathNames.get(i), lines);

如果您不打算使用path，为什么要声明它？

另外，为什么不在这里使用continue;？HashMap.put将覆盖任何以前的值，所以您只会看到文件的最后一个片段。

        stream = new FileInputStream(pathNames.get(i));
        BufferedReader br = new BufferedReader(new    InputStreamReader(stream));

所有这些都不会被关闭。

既然你有一个try，为什么不使用try-with-resources呢？

String path = pathNames.get(i);
BufferedReader br;
try(stream = new FileInputStream(path);
        br = new BufferedReader(new InputStreamReader(stream))){
    ...
}

差不多是这样的。

private ArrayList<String> pathNames = new ArrayList();
private HashMap<String, String> snippets = new HashMap<String, String>();

变量定义不对接口进行编程，因此任何设置者都必须只接受ArrayList，这使得您的代码很难用于其他任何人。此外，使用菱形类型推断-让编译器从类型定义中获得泛型。

                char[] cb = (char[]) chars.get(br);
                int nextChar = (Integer) ff.get(br);

这些反射调用；每次迭代都会调用它们，但只有在找到搜索字符串时才需要它们。把他们移下来。

最后，您应该考虑找到一种消除反射的方法。这会使代码变得缓慢、丑陋和危险。

“已清除”版本(可能不会编译；请检查“使用资源尝试”和“多捕获”的语法)：

private List<String> pathNames = new ArrayList<>();
private Map<String, String> snippets = new HashMap<>();

public HashMap readFiles(String search, String dirPath) {

    snippets.clear();
    InputStream stream = null;
    BufferedReader br = null;

    //Method to get all the files in main and sub folders
    listDirectory(dirPath);

    for (int i = 0; i < pathNames.size(); i++) {
        String path = pathNames.get(i);
        try (stream = new FileInputStream(path);
            br = new BufferedReader(new InputStreamReader(stream))){

            Field chars = br.getClass().getDeclaredField("cb");
            chars.setAccessible(true);

            Field ff = br.getClass().getDeclaredField("nextChar");
            ff.setAccessible(true);

            String line;
            while ((line = br.readLine()) != null) {
                if (line.contains(search)) {
                    //Getting through to Java Source Code - uh oh
                    char[] cb = (char[]) chars.get(br);
                    int nextChar = (Integer) ff.get(br);
                    char[] charBuffer = new char[300];
                    System.arraycopy(cb, nextChar - 200, charBuffer, 0, 300);
                    snippets.put(path, new String(charBuffer));
                    continue;

                }
            }

        } catch (FileNotFoundException | IOException | IllegalArgumentException | NoSuchFieldException | SecurityException ex) {
            Logger.getLogger(FileRead.class.getName()).log(Level.SEVERE, null, ex);
        }
    }
    return snippets;
}