文件加载错误: 1.7TB可用存储空间不足

Question

我使用下面的代码以NiFTI格式加载我的文件。

import nibabel as nib 

img_arr = []
for i in range(len(datadir)):
    img = nib.load(datadir[i])
    img_data = img.get_fdata()
    img_arr.append(img_data)
    img.uncache()

少量的图像可以正常工作，但是如果我想加载更多的图像，我会得到以下错误：

OSError                                   Traceback (most recent call last)
<ipython-input-55-f982811019c9> in <module>()
     10     #img = nilearn.image.smooth_img(datadir[i],fwhm = 3) #Smoothing filter for preprocessing (necessary?)
     11     img = nib.load(datadir[i])
---> 12     img_data = img.get_fdata()
     13     img_arr.append(img_data)
     14     img.uncache()

~\AppData\Roaming\Python\Python36\site-packages\nibabel\dataobj_images.py in get_fdata(self, caching, dtype)
    346             if self._fdata_cache.dtype.type == dtype.type:
    347                 return self._fdata_cache
--> 348         data = np.asanyarray(self._dataobj).astype(dtype, copy=False)
    349         if caching == 'fill':
    350             self._fdata_cache = data

~\AppData\Roaming\Python\Python36\site-packages\numpy\core\_asarray.py in asanyarray(a, dtype, order)
    136 
    137     """
--> 138     return array(a, dtype, copy=False, order=order, subok=True)
    139 
    140 

~\AppData\Roaming\Python\Python36\site-packages\nibabel\arrayproxy.py in __array__(self)
    353     def __array__(self):
    354         # Read array and scale
--> 355         raw_data = self.get_unscaled()
    356         return apply_read_scaling(raw_data, self._slope, self._inter)
    357 

~\AppData\Roaming\Python\Python36\site-packages\nibabel\arrayproxy.py in get_unscaled(self)
    348                                        offset=self._offset,
    349                                        order=self.order,
--> 350                                        mmap=self._mmap)
    351         return raw_data
    352 

~\AppData\Roaming\Python\Python36\site-packages\nibabel\volumeutils.py in array_from_file(shape, in_dtype, infile, offset, order, mmap)
    507                              shape=shape,
    508                              order=order,
--> 509                              offset=offset)
    510             # The error raised by memmap, for different file types, has
    511             # changed in different incarnations of the numpy routine

~\AppData\Roaming\Python\Python36\site-packages\numpy\core\memmap.py in __new__(subtype, filename, dtype, mode, offset, shape, order)
    262             bytes -= start
    263             array_offset = offset - start
--> 264             mm = mmap.mmap(fid.fileno(), bytes, access=acc, offset=start)
    265 
    266             self = ndarray.__new__(subtype, shape, dtype=descr, buffer=mm,

OSError: [WinError 8] Not enough storage is available to process this command

我认为img.uncache()会从内存中删除图像，这样它就不会占用太多的存储空间，但仍然能够处理图像数组。但是，将这一点添加到代码中并没有改变什么。

有人知道我能帮上什么忙吗？我正在工作的计算机有24核2,6 GHz CPU，超过52 GB内存，工作目录有超过1.7TB的空闲存储器。我试图从ADNI数据库中加载大约1500张MRI图像。

任何建议都是非常感谢的。

Answer 1

这个错误并不是因为1.7TB硬盘被填满了，而是因为内存耗尽了，也就是内存。理解这两件事的区别是很重要的。

uncache()并不像文档中的这里那样从内存中完全删除项，但是该链接还包含更多的内存保存提示。

如果要从内存中完全删除对象，可以使用垃圾收集器接口，如下所示：

import nibabel as nib 
import gc

img_arr = []
for i in range(len(datadir)):
    img = nib.load(datadir[i])
    img_data = img.get_fdata()
    img_arr.append(img_data)
    img.uncache()
    # Delete the img object and free the memory
    del img
    gc.collect()

这将有助于减少您正在使用的内存量。

Answer 2

如何修复“可用内存不足.”？

• 尝试执行以下步骤：

1. Press the Windows + R key at the same time on your keyboard, then type Regedit.exe in the Run window and click on OK.
2. Then Unfold HKEY\_LOCAL\_MACHINE, then SYSTEM, then CurrentControlSet, then services, then LanmanServer, then Parameters.
3. Locate IRPStackSize (If found skip to step 5), If it does not exist then right-click the right Window and choose New > Dword Value (32)
4. Now type IRPStackSize under the name, then hit enter.
5. Right-click IRPStackSize and click on Modify, then set any value higher then 15 but lower than 50 and click OK
6. Restart your system and try to repeat the same action as you did when the error occurred.

​

• 或：

1. Set the following registry key  HKLM\SYSTEM\CurrentControlSet\Control\Session Manager\Memory Management\LargeSystemCache to value "1"
2. Set the following registry  HKLM\SYSTEM\CurrentControlSet\Services\LanmanServer\Parameters\Size to value "3"

​

另一种在"nibabel“中节省内存的方法:

除了uncache()方法之外，还有其他节省内存的方法，您可以使用：

• 数组代理而不是fdata()
• fdata()的缓存关键字