Solaris ZFS卷:工作负载未命中L2ARC

Question

我已经设置了一个Solaris Express 11机器，在RAID控制器后面安装了一些相当快的HDD，将设备设置为启用压缩的zpool，并向其添加了一个镜像日志和2个缓存设备。这些数据集被公开为用于ESX的FC目标，我已经在其中填充了一些可供使用的数据。L2ARC部分填充(由于某些原因不再填充)，但我几乎看不到它的任何用途。zpool iostat -v显示，过去从缓存中读取的内容不多：

tank           222G  1.96T    189     84   994K  1.95M
  c7t0d0s0     222G  1.96T    189     82   994K  1.91M
  mirror      49.5M  5.51G      0      2      0  33.2K
    c8t2d0p1      -      -      0      2      0  33.3K
    c8t3d0p1      -      -      0      2      0  33.3K
cache             -      -      -      -      -      -
  c11d0p2     23.5G  60.4G      2      1  33.7K   113K
  c10d0p2     23.4G  60.4G      2      1  34.2K   113K

支持L2ARC的arcstat.pl脚本显示当前工作负载的L2ARC 100%未命中：

./arcstat.pl -f read,hits,miss,hit%,l2read,l2hits,l2miss,l2hit%,arcsz,l2size 5
read  hits  miss  hit%  l2read  l2hits  l2miss  l2hit%  arcsz  l2size
[...]
 243   107   136    44     136       0     136       0   886M     39G
 282   144   137    51     137       0     137       0   886M     39G
 454   239   214    52     214       0     214       0   889M     39G
[...]

我最初怀疑这可能是记录大小太大的影响，这样L2ARC就可以将所有内容都识别为流加载，但是zpool只包含zfs卷(我使用zfs create -V 500G -s <datasetname>将它们创建为“稀疏”)，这些卷甚至没有要更改的记录集参数。

我还发现了许多关于L2ARC需要每条记录需要200字节内存的元数据的想法，但是到目前为止，我还无法找到L2ARC会用一个卷数据集来考虑什么“记录”--一个只有512字节的扇区？它会不会因为内存不足而陷入元数据短缺，而到目前为止却被再也没有被阅读过的垃圾填满了？

编辑:在已安装的2GB内存的基础上添加8GB内存效果很好--即使在32位的安装中，附加的RAM也能很好地使用，而且L2ARC现在已经增长，并且正在受到攻击：

    time  read  hit%  l2hit%  arcsz  l2size
21:43:38   340    97      13   6.4G     95G
21:43:48   185    97      18   6.4G     95G
21:43:58   655    91       2   6.4G     95G
21:44:08   432    98      16   6.4G     95G
21:44:18   778    92       9   6.4G     95G
21:44:28   910    99      19   6.4G     95G
21:44:38  4.6K    99      18   6.4G     95G

多亏了白白。

Answer 1

你应该在系统中有更多的内存。指向L2ARC的指针需要保存在RAM (ARC)中，所以我认为您需要大约4GB或6GB的内存来更好地利用您可用的~60 4GB的L2ARC。

这来自ZFS列表中最近的一个线程：

http://opensolaris.org/jive/thread.jspa?threadID=131296

L2ARC is "secondary" ARC. ZFS attempts to cache all reads in the ARC 
(Adaptive Read Cache) - should it find that it doesn't have enough space 
in the ARC (which is RAM-resident), it will evict some data over to the 
L2ARC (which in turn will simply dump the least-recently-used data when 
it runs out of space). Remember, however, every time something gets 
written to the L2ARC, a little bit of space is taken up in the ARC 
itself (a pointer to the L2ARC entry needs to be kept in ARC). So, it's 
not possible to have a giant L2ARC and tiny ARC. As a rule of thumb, I 
try not to have my L2ARC exceed my main RAM by more than 10-15x (with 
really bigMem machines, I'm a bit looser and allow 20-25x or so, but 
still...). So, if you are thinking of getting a 160GB SSD, it would be 
wise to go for at minimum 8GB of RAM. Once again, the amount of ARC 
space reserved for a L2ARC entry is fixed, and independent of the actual 
block size stored in L2ARC. The jist of this is that tiny files eat up 
a disproportionate amount of systems resources for their size (smaller 
size = larger % overhead vis-a-vis large files).