我正在使用蛇咬伤客户端
https://github.com/spotify/snakebite
当我试图在hdfs中创建目录或移动文件时,我注意到了一种奇怪的行为。这是我的密码。它所做的就是将源目录的内容移动到目标目录。最后,显示目标目录的内容。
def purge_pending(self,source_dir,dest_dir):
if(self.hdfs_serpent.test(path=self.root_dir+"/"+source_dir, exists=True, directory=True)):
print "Source exists ",self.root_dir+source_dir
for x in self.hdfs_serpent.ls([self.root_dir+source_dir]):
print x['path']
else:
print "Source does not exist ",self.root_dir+"/"+source_dir
return
if(self.hdfs_serpent.test(path=self.root_dir+"/"+dest_dir, exists=True, directory=True)):
print "Destination exists ",self.root_dir+dest_dir
else:
print "Destination does not exist ",self.root_dir+dest_dir
print "Will be created"
for y in self.hdfs_serpent.mkdir([self.root_dir+dest_dir],create_parent=True):
print y
for src in self.hdfs_serpent.ls([self.root_dir+source_dir]):
print src['path'].split("/")[-1]
for y in self.hdfs_serpent.rename([src['path']],self.root_dir+dest_dir+"/"+src['path'].split("/")[-1]):
print y
for x in self.hdfs_serpent.ls([self.root_dir+dest_dir]):
print x['path']下面是目标不存在时的示例输出
Source exists /root/source
/root/source/208560.json
/root/source/208571.json
/root/source/208574.json
/root/source/208581.json
/root/source/208707.json
Destination does not exist /root/dest
Will be created
{'path':'/research/dest/'}
208560.json
{'path':'/research/dest/208560.json'}
208571.json
{'path':'/research/dest/208571.json'}
208574.json
{'path':'/research/dest/208574.json'}
208581.json
{'path':'/research/dest/208581.json'}
208707.json
{'path':'/research/dest/208707.json'}奇怪的是,我必须把那些打印的声明放进去,否则什么都不能用。所以
self.hdfs_serpent.mkdir([self.root_dir+dest_dir],create_parent=True)不管用,但是
for y in self.hdfs_serpent.mkdir([self.root_dir+dest_dir],create_parent=True):
print y真的!同为
self.hdfs_serpent.rename([src['path']],self.root_dir+dest_dir+"/"+src['path'].split("/")[-1])由于上面的内容不起作用,但下面的内容确实起作用
for y in self.hdfs_serpent.rename([src['path']],self.root_dir+dest_dir+"/"+src['path'].split("/")[-1]):
print y这是个虫子吗?我做错什么了吗?
发布于 2016-05-12 16:29:04
这看起来是通过设计完成的,因为文档声明方法返回的大多数对象都是生成器。因此,函数通常不会执行任何操作,直到使用了下一次() ( for隐式地使用了这些值)为止。
https://stackoverflow.com/questions/37192027
复制相似问题