为什么是HDFS FileNotFoundException:文件不存在？

Question

我从K8S pod访问HDFS，但它找不到HDFS文件。

Exception in thread "main" java.io.FileNotFoundException: File does not exist: hdfs://192.168.65.2:8020/user/flink/[/user/flink/.flink/job-1234/1624538371951/systemShipFiles/examples-wc.jar]
    at org.apache.hadoop.hdfs.DistributedFileSystem$22.doCall(DistributedFileSystem.java:1309)
    at org.apache.hadoop.hdfs.DistributedFileSystem$22.doCall(DistributedFileSystem.java:1301)
    at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
    at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1317)
    at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:337)
    at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:289)
    at org.apache.hadoop.fs.FileSystem.copyToLocalFile(FileSystem.java:2030)
    at org.apache.hadoop.fs.FileSystem.copyToLocalFile(FileSystem.java:1999)
    at org.apache.hadoop.fs.FileSystem.copyToLocalFile(FileSystem.java:1975)
    at org.apache.flink.kubernetes.entrypoint.KubernetesInitContainerEntrypoint.fetchRemoteDependencies(KubernetesInitContainerEntrypoint.java:121)
    at org.apache.flink.kubernetes.entrypoint.KubernetesInitContainerEntrypoint.main(KubernetesInitContainerEntrypoint.java:72)
Error from server (BadRequest): container "flink-job-manager" in pod "job-1234-b8d68f956-c4g58" is waiting to start: PodInitializing

但是我可以使用hadoop命令获取文件作为流：

hadoop fs -ls /user/flink/.flink/job-1234/1624538371951/systemShipFiles/oceanus-examples-wc.jar
21/06/24 20:43:45 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
-rw-r--r--   1 flink supergroup      13981 2021-06-24 20:39 /user/flink/.flink/job-1234/1624538371951/systemShipFiles/examples-wc.jar

HADOOP处于本地模式。

谢谢你的帮助。

Answer 1

您是否可以尝试以hdfs://192.168.65.2:8020/user/.为前缀的完整路径的ls就像它在堆栈跟踪中显示的那样。

检查fs.defaultFs -site.xml，指定的值是正确的。