LVS集群帮助

Question

我正试图为一个项目建立一个LVS集群，但在这些层次的网络中我没有太多的知识。我遇到了一个问题，即真正的服务器拒绝来自主管的任何数据包，但是将允许数据包从我的客户端进入同一个端口。我在2.6.16.60-0.21默认内核上使用suse。

导演剧本：

#!/bin/bash
#----------mini-rc.lvs_dr-director------------------------
#set ip_forward OFF for lvs-dr director (1 on, 0 off)
#(there is no forwarding in the conventional sense for LVS-DR)
cat       /proc/sys/net/ipv4/ip_forward
echo "0" >/proc/sys/net/ipv4/ip_forward

#director is not gw for realservers: leave icmp redirects on
echo 'setting icmp redirects (1 on, 0 off) '
echo "1" >/proc/sys/net/ipv4/conf/all/send_redirects
cat       /proc/sys/net/ipv4/conf/all/send_redirects
echo "1" >/proc/sys/net/ipv4/conf/default/send_redirects
cat       /proc/sys/net/ipv4/conf/default/send_redirects
echo "1" >/proc/sys/net/ipv4/conf/eth1/send_redirects
cat       /proc/sys/net/ipv4/conf/eth1/send_redirects

#add ethernet device and routing for VIP 192.168.1.110
/sbin/ifconfig eth1:110 172.18.9.192 broadcast 172.18.9.192 netmask 255.255.255.255
/sbin/route add -host 172.18.9.192 dev eth1:110
#listing ifconfig info for VIP 192.168.1.110
/sbin/ifconfig eth1:110

#check VIP 192.168.1.110 is reachable from self (director)
/bin/ping -c 1 172.18.9.192
#listing routing info for VIP 192.168.1.110
/bin/netstat -rn

#setup_ipvsadm_table
#clear ipvsadm table
/sbin/ipvsadm -C
#installing LVS services with ipvsadm
#add telnet to VIP with round robin scheduling
/sbin/ipvsadm -A -t 172.18.9.192:8080 -s rr

#forward telnet to realserver using direct routing with weight 1
#/sbin/ipvsadm -a -t 172.18.9.192:telnet -r 172.18.9.191 -g -w 1
/sbin/ipvsadm -a -t 172.18.9.192:8080 -r 172.18.9.191:8080 -g -w 1
#check realserver reachable from director
ping -c 1 172.18.9.191

#forward telnet to realserver using direct routing with weight 1
/sbin/ipvsadm -a -t 172.18.9.192:8080 -r 172.18.1.136:8080 -g -w 1
#check realserver reachable from director
ping -c 1 172.18.1.136

#displaying ipvsadm settings
/sbin/ipvsadm

#not installing a default gw for LVS_TYPE vs-dr
#---------------mini-rc.lvs_dr-director------------------------

真实服务器脚本

#!/bin/bash
#----------mini-rc.lvs_dr-realserver------------------
#installing default gw 192.168.1.254 for vs-dr
#/sbin/route add default gw 192.168.1.254
/sbin/route add default gw 172.18.9.254
#showing routing table
/bin/netstat -rn
#checking if DEFAULT_GW 192.168.1.254 is reachable
ping -c 1 172.18.9.254

#set_realserver_ip_forwarding to OFF (1 on, 0 off).
echo "0" >/proc/sys/net/ipv4/ip_forward
cat       /proc/sys/net/ipv4/ip_forward

#looking for DIP 192.168.1.9
ping -c 1 172.18.9.110

#looking for VIP (will be on director)
ping -c 1 172.18.9.192

#install_realserver_vip

/sbin/ifconfig lo 172.18.9.192 broadcast 172.18.9.192 netmask 0xffffffff up
#ifconfig output
/sbin/ifconfig lo:110
#installing route for VIP 192.168.1.110 on device lo:110
/sbin/route add -host 172.18.9.192 dev lo:110
#listing routing info for VIP 192.168.1.110
/bin/netstat -rn

#hiding interface lo:0, will not arp
echo "1" >/proc/sys/net/ipv4/conf/all/hidden
cat       /proc/sys/net/ipv4/conf/all/hidden
echo "1" >/proc/sys/net/ipv4/conf/lo/hidden
cat       /proc/sys/net/ipv4/conf/lo/hidden

#----------mini-rc.lvs_dr-realserver------------------

由于不存在/hidden，上述arp修复程序无法运行。我修改了/etc/sysctl1.conf以添加：

net.ipv4.conf.lo.arp_ignore = 1
net.ipv4.conf.lo.arp_announce = 2
net.ipv4.conf.all.arp_ignore = 1
net.ipv4.conf.all.arp_announce = 2

并运行sysctl -p

当我在VIP上运行时，我在真正的服务器上使用tcpdump的数据包如下所示：

sudo /usr/sbin/tcpdump -n -i any port 8080
tcpdump: WARNING: Promiscuous mode not supported on the "any" device
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on any, link-type LINUX_SLL (Linux cooked), capture size 96 bytes
14:37:53.617753 IP 172.16.10.74.4592 > 172.18.9.192.8080: S 3169956976:316995697                                                                              6(0) win 65535 
14:37:53.617851 IP 172.18.9.192.8080 > 172.16.10.74.4592: R 0:0(0) ack 316995697                                                                              7 win 0

当我直接运行到真正的服务器时，数据包会被推送：

14:58:23.595729 IP 172.16.10.74.4887 > 172.18.9.191.8080: S 2691152412:2691152412(0) win 65535 
14:58:23.595794 IP 172.18.9.191.8080 > 172.16.10.74.4887: S 3988256836:3988256836(0) ack 2691152413 win 5840 
14:58:23.602995 IP 172.16.10.74.4887 > 172.18.9.191.8080: . ack 1 win 65535
14:58:23.600264 IP 172.16.10.74.4887 > 172.18.9.191.8080: P 1:483(482) ack 1 win 65535
14:58:23.600275 IP 172.18.9.191.8080 > 172.16.10.74.4887: . ack 483 win 6432
14:58:23.605755 IP 172.18.9.191.8080 > 172.16.10.74.4887: P 1:175(174) ack 483 win 6432
14:58:23.622787 IP 172.16.10.74.4887 > 172.18.9.191.8080: P 483:969(486) ack 175 win 65361
14:58:23.619738 IP 172.18.9.191.8080 > 172.16.10.74.4887: P 175:349(174) ack 969 win 7504
14:58:23.674299 IP 172.16.10.74.4887 > 172.18.9.191.8080: P 969:1473(504) ack 349 win 65187
14:58:23.671410 IP 172.18.9.191.8080 > 172.16.10.74.4887: P 349:524(175) ack 1473 win 8576
14:58:23.852775 IP 172.16.10.74.4887 > 172.18.9.191.8080: . ack 524 win 65012

如果需要更多的信息来帮助调试这个问题，请告诉我。

Answer 1

将我的问题张贴在LVS的邮件列表上，第一个答复问道：

在realserver上，httpd是否监听172.18.9.192:8080？

原来我已经正确地设置了LVS集群，但是JBoss并不是用来处理VIP的。