Ubuntu10.04&具有FC多路径的IBM DS3524，非活动路径是[故障][故障]而不是[活动][鬼]

Question

好的，这是我的设置：

FC开关IBM/Brocade，Switch1和Switch2，独立面料。

服务器IBM x3650 M2，2x QLogic QLE2460，1连接到每个FC交换机。

存储IBM，2x控制器，每个端口有4xFC端口，但每个端口上只连接2x。

+-----------------------------------------------------------------------+
|               HBA1             Server              HBA2               |
+-----------------------------------------------------------------------+
                 |                                     |
                 |                                     |
                 |                                     |
+-----------------------------+          +------------------------------+
|            Switch1          |          |            Switch2           |
+-----------------------------+          +------------------------------+
         |                |                  |                 |
         |                |                  |                 |
         |                |                  |                 |
         |                |                  |                 |
         |                |                  |                 |
+-----------------------------------+-----------------------------------+        
| Contr A, port 3 | Contr A, port 4 | Contr B, port 3 | Contr B, port 4 |
+-----------------------------------+-----------------------------------+
|                                 Storage                               |
+-----------------------------------------------------------------------+

我的/etc/mpath. this来自IBM中的DS3500，除了对prio_callout使用不同的设置外，IBM使用/sbin/mpath_prio_tpc，但根据http://changelogs.ubuntu.com/changelogs/pool/main/m/multipath-tools/multipath-tools_0.4.8-7 ubuntu2/changelog，它被重命名为/sbin/mpath_prio_rdac，我正在使用它。

devices {
  device {
    #ds3500
    vendor "IBM"
    product "1746   FAStT"
    hardware_handler "1 rdac"
    path_checker rdac
    failback 0
    path_grouping_policy multibus
    prio_callout "/sbin/mpath_prio_rdac /dev/%n"
  }
}

multipaths {
  multipath {
    wwid xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
    alias array07
    path_grouping_policy multibus
    path_checker readsector0
    path_selector "round-robin 0"
    failback "5"
    rr_weight priorities
    no_path_retry "5"
  }
}

以控制器A为首选路径的multipath -ll的输出：

root@db06:~# multipath -ll
sdg: checker msg is "directio checker reports path is down"
sdh: checker msg is "directio checker reports path is down"
array07 (xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx) dm-2 IBM     ,1746      FASt
[size=4.9T][features=1 queue_if_no_path][hwhandler=0]
\_ round-robin 0 [prio=2][active]
 \_ 5:0:1:0 sdd 8:48  [active][ready]
 \_ 5:0:2:0 sde 8:64  [active][ready]
 \_ 6:0:1:0 sdg 8:96  [failed][faulty]
 \_ 6:0:2:0 sdh 8:112 [failed][faulty]

如果我使用IBM DS存储管理器将首选路径更改为Controller B，则输出将相应地交换：

root@db06:~# multipath -ll
sdd: checker msg is "directio checker reports path is down"
sde: checker msg is "directio checker reports path is down"
array07 (xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx) dm-2 IBM     ,1746      FASt
[size=4.9T][features=1 queue_if_no_path][hwhandler=0]
\_ round-robin 0 [prio=2][active]
 \_ 5:0:1:0 sdd 8:48  [failed][faulty]
 \_ 5:0:2:0 sde 8:64  [failed][faulty]
 \_ 6:0:1:0 sdg 8:96  [active][ready]
 \_ 6:0:2:0 sdh 8:112 [active][ready]

根据IBM的说法，非活动路径应该是"活动的"，而不是"失败“。

尽管如此，我似乎没有任何I/O问题，但是我的syslog每隔5秒就会收到一次垃圾邮件：

Jun  1 15:30:09 db06 multipathd: sdg: directio checker reports path is down
Jun  1 15:30:09 db06 kernel: [ 2350.282065] sd 6:0:2:0: [sdh] Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
Jun  1 15:30:09 db06 kernel: [ 2350.282071] sd 6:0:2:0: [sdh] Sense Key : Illegal Request [current] 
Jun  1 15:30:09 db06 kernel: [ 2350.282076] sd 6:0:2:0: [sdh] <<vendor>> ASC=0x94 ASCQ=0x1ASC=0x94 ASCQ=0x1
Jun  1 15:30:09 db06 kernel: [ 2350.282083] sd 6:0:2:0: [sdh] CDB: Read(10): 28 00 00 00 00 00 00 00 08 00
Jun  1 15:30:09 db06 kernel: [ 2350.282092] end_request: I/O error, dev sdh, sector 0
Jun  1 15:30:10 db06 multipathd: sdh: directio checker reports path is down
Jun  1 15:30:14 db06 kernel: [ 2355.312270] sd 6:0:1:0: [sdg] Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
Jun  1 15:30:14 db06 kernel: [ 2355.312277] sd 6:0:1:0: [sdg] Sense Key : Illegal Request [current] 
Jun  1 15:30:14 db06 kernel: [ 2355.312282] sd 6:0:1:0: [sdg] <<vendor>> ASC=0x94 ASCQ=0x1ASC=0x94 ASCQ=0x1
Jun  1 15:30:14 db06 kernel: [ 2355.312290] sd 6:0:1:0: [sdg] CDB: Read(10): 28 00 00 00 00 00 00 00 08 00
Jun  1 15:30:14 db06 kernel: [ 2355.312299] end_request: I/O error, dev sdg, sector 0

有谁知道我如何使非活动路径显示"活动的“而不是"失败”呢？我假设，一旦我能够正确的，那么垃圾邮件在我的syslog也将结束。

最后值得一提的是，IBM红皮书文档针对的是SLES 11，所以我假设Ubuntu下有一些不一样的地方，我只是还没有弄清楚。

更新:作为米契，我尝试删除/etc/multipath.conf .this，现在multipath -ll的输出如下所示：

root@db06:~# multipath -ll
sdg: checker msg is "directio checker reports path is down"
sdh: checker msg is "directio checker reports path is down"
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxdm-1 IBM     ,1746      FASt
[size=4.9T][features=0][hwhandler=0]
\_ round-robin 0 [prio=1][active]
 \_ 5:0:2:0 sde 8:64  [active][ready]
\_ round-robin 0 [prio=1][enabled]
 \_ 5:0:1:0 sdd 8:48  [active][ready]
\_ round-robin 0 [prio=0][enabled]
 \_ 6:0:1:0 sdg 8:96  [failed][faulty]
\_ round-robin 0 [prio=0][enabled]
 \_ 6:0:2:0 sdh 8:112 [failed][faulty]

因此，它或多或少是一样的，每5分钟在syslog中显示相同的消息，但是分组已经改变了。

Answer 1

我终于找到了一个在这里工作的配置：http://pig.made-it.com/multipath.html。在这里总结一下，以防其他人将来也会有这种情况。

device {
    vendor "IBM"
    product "1745|1746"
    path_grouping_policy group_by_prio
#   getuid_callout "/sbin/scsi_id -g -u -s /block/%n"
    path_selector "round-robin 0"
    path_checker rdac
    features "2 pg_init_retries 50"
    hardware_handler "1 rdac"
    prio_callout "/sbin/mpath_prio_rdac /dev/%n"
    failback immediate
    rr_weight uniform
    no_path_retry 15
    rr_min_io 1000
}

我评论getuid_callout是因为Ubuntu将scsi_id程序放在/lib/udev中，而不是/sbin中，而getuid_callout的默认值是/lib/udev/scsi_id -g -u -s，所以我没有显式地将它放在配置中。

在multipath { ... }块中不需要特定的设置，但是如果需要，可以使用以下方法创建友好的别名：

multipaths {
  multipath {
    wwid xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
    alias array07
  }
}

multipath -ll输出现在如下所示：

root@db06:~# multipath -ll                                                                                                                                                                                                                                             
array07 (xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx) dm-1 IBM     ,1746      FASt
[size=4.9T][features=3 queue_if_no_path pg_init_retries 50][hwhandler=1 rdac]
\_ round-robin 0 [prio=0][enabled]
 \_ 5:0:1:0 sdd 8:48  [active][ghost]
 \_ 5:0:2:0 sde 8:64  [active][ghost]
\_ round-robin 0 [prio=6][active]
 \_ 6:0:1:0 sdg 8:96  [active][ready]
 \_ 6:0:2:0 sdh 8:112 [active][ready]

我不再在/var/log/syslog中接收垃圾邮件了，故障转移/故障回退工作得很好。

Answer 2

我真的不太了解IBM的DS3524。但如果它与EMC Clariion CX-4类似，那么：

尝试将multipath.conf从etc文件夹移开并重新启动multipath-tools。