利用haproxy+keepalived来实现基于http 七层负载均衡功能
本文共 19212 字，大约阅读时间需要 64 分钟。
haproxy+keepalived软负载架构
软件负载均衡一般通过两种方式来实现：基于操作系统的软负载实现和基于第三方应用的软负载实现。LVS就是基于Linux操作系统实现的一种软负载，HAProxy就是开源的并且基于第三应用实现的软负载。本案例两台机器作为软负载机器，两台机器上面分布部署了haproxy+keepalived。其中一台为master,另外一台为slave.
HAProxy支持两种主要的代理模式:"tcp"也即4层（大多用于邮件服务器、内部协议通信服务器等），和7层（HTTP）。在4层模式 下，HAproxy仅在客户端和服务器之间转发双向流量。7层模式下，HAProxy会分析协议，并且能通过允许、拒绝、交换、增加、修改或者删除请求 (request)或者回应(response)里指定内容来控制协议，这种操作要基于特定规则。特点：
haproxy支持四七层,而LVS是四层应用，nginx是七层应用支持双机热备，高可用，负载均衡、虚拟主机，应用代理、基于TCP和HTTP的 图形界面查看信息.服务器节点健康检查功能.一般是lvs在前面做4层，haproxy在后面做七层。
静态服务器nginx的安装
yum -y install nginxnginx主配置文件：[root@WHOPRDLBS00AP0001 ~]# cat /etc/nginx/nginx.conf# For more information on configuration, see:# * Official English Documentation: http://nginx.org/en/docs/# * Official Russian Documentation: http://nginx.org/ru/docs/user nginx;worker_processes auto;error_log /var/log/nginx/error.log;pid /var/run/nginx.pid;# Load dynamic modules. See /usr/share/nginx/README.dynamic.include /usr/share/nginx/modules/*.conf;events { worker_connections 1024;}http { log_format main '$remote_addr - $remote_user [$time_local] "$request" ' '$status $body_bytes_sent "$http_referer" ' '"$http_user_agent" "$http_x_forwarded_for"'; access_log /var/log/nginx/access.log main; sendfile on; tcp_nopush on; tcp_nodelay on; keepalive_timeout 65; types_hash_max_size 2048; include /etc/nginx/mime.types; default_type application/octet-stream; # Load modular configuration files from the /etc/nginx/conf.d directory. # See http://nginx.org/en/docs/ngx_core_module.html#include # for more information. include /etc/nginx/conf.d/*.conf;}[root@WHOPRDLBS00AP0001 ~]# # nginx的子配置文件[root@WHOPRDLBS00AP0001 ~]# cat /etc/nginx/conf.d/default.conf ## The default server#server { listen 80 default_server; server_name _; root /usr/share/nginx/html; #指定html主目录 # Load configuration files for the default server block. include /etc/nginx/default.d/*.conf; location / { } error_page 404 /404.html; #指定404 错误页面文件 location = /40x.html { } error_page 500 502 503 504 /50x.html; #指定50X 错误配置文件 location = /50x.html { }}[root@WHOPRDLBS00AP0001 ~]# ll /usr/share/nginx/html/total 28-rw-r--r-- 1 root root 3652 Oct 31 2016 404.html-rw-r--r-- 1 root root 3695 Oct 31 2016 50x.html-rw-r--r-- 1 root root 3698 Oct 31 2016 index.html-rw-r--r-- 1 root root 368 Oct 31 2016 nginx-logo.png-rw-r--r-- 1 root root 2811 Oct 31 2016 poweredby.pngdrwxr-xr-x 2 root root 4096 Mar 13 2017 test-rw-r--r-- 1 root root 24 Mar 13 2017 test.html[root@WHOPRDLBS00AP0001 ~]# cat /usr/share/nginx/html/test/test.html The web's ip:10.36.1.57[root@WHOPRDLBS00AP0001 ~]# 
haproxy的安装
wget http://www.haproxy.org/download/1.7/src/haproxy-1.7.2.tar.gztar xf haproxy-1.7.2.tar.gzcd haproxy-1.7.2make TARGET=linux26 ARCH=x86_64 PREFIX=/usr/local/haproxy-1.7.2make install PREFIX=/usr/local/haproxy-1.7.2groupadd -r haproxyuseradd -g haproxy -M -s /sbin/nologin haproxychown -R haproxy.haproxy haproxy-1.7.2mkdir -p /logs/haproxy-1.7.2/chhown -R haproxy.haproxy /logs/haproxy-1.7.2/mkdir -p /confs/haproxy/ && chhown -R haproxy.haproxy /confs/haproxy//usr/local/haproxy-1.7.2/sbin/haproxy -f /confs/haproxy/haproxy.cfg
haproxy启动配置文件
[root@WHOPRDLBS00AP0001 ~]# cat /etc/init.d/haproxy#!/bin/sh## haproxy## chkconfig: - 85 15# description: HAProxy is a free, very fast and reliable solution \# offering high availability, load balancing, and \# proxying for TCP and HTTP-based applications# processname: haproxy# config: /etc/haproxy/haproxy.cfg# pidfile: /var/run/haproxy.pid# Source function library.. /etc/rc.d/init.d/functions# Source networking configuration.. /etc/sysconfig/network# Check that networking is up.[ "$NETWORKING" = "no" ] && exit 0exec="/usr/local/haproxy-1.7.2/sbin/haproxy"prog=$(basename $exec)[ -e /etc/sysconfig/$prog ] && . /etc/sysconfig/$progcfgfile=/confs/haproxy-1.7.2/haproxy.cfgpidfile=/var/run/haproxy.pidlockfile=/var/lock/subsys/haproxycheck() { $exec -c -V -f $cfgfile $OPTIONS}start() { $exec -c -q -f $cfgfile $OPTIONS if [ $? -ne 0 ]; then echo "Errors in configuration file, check with $prog check." return 1 fi echo -n $"Starting $prog: " # start it up here, usually something like "daemon $exec" daemon $exec -D -f $cfgfile -p $pidfile $OPTIONS retval=$? echo [ $retval -eq 0 ] && touch $lockfile return $retval}stop() { echo -n $"Stopping $prog: " # stop it here, often "killproc $prog" killproc $prog retval=$? echo [ $retval -eq 0 ] && rm -f $lockfile return $retval}restart() { $exec -c -q -f $cfgfile $OPTIONS if [ $? -ne 0 ]; then echo "Errors in configuration file, check with $prog check." return 1 fi stop start}reload() { $exec -c -q -f $cfgfile $OPTIONS if [ $? -ne 0 ]; then echo "Errors in configuration file, check with $prog check." return 1 fi echo -n $"Reloading $prog: " $exec -D -f $cfgfile -p $pidfile $OPTIONS -sf $(cat $pidfile) retval=$? echo return $retval}force_reload() { restart}fdr_status() { status $prog}case "$1" in start|stop|restart|reload) $1 ;; force-reload) force_reload ;; check) check ;; status) fdr_status ;; condrestart|try-restart) [ ! -f $lockfile ] || restart ;; *) echo $"Usage: $0 {start|stop|status|restart|try-restart|reload|force-reload}" exit 2esac[root@WHOPRDLBS00AP0001 ~]# keepalived和haproxy配置
# master节点上面的keepalived配置[root@WHOPRDLBS00AP0001 ~]# cat /etc/keepalived/keepalived.conf! Configuration File for keepalivedglobal_defs { router_id keepalived_haproxy_prd_01_01}vrrp_sync_group VGM_haproxy_prd_01 { group { haproxy_bl_prd_240 }}vrrp_script chk_10.36.1.57_haproxy { #检查脚本，如果haproxy出现故障 script "killall -0 haproxy" interval 2}vrrp_instance haproxy_bl_prd_240 { state MASTER interface bond0.50 virtual_router_id 240 priority 100 advert_int 1 authentication { auth_type PASS auth_pass !QAZxsw2 } virtual_ipaddress { 10.36.1.240 }track_script { chk_10.36.1.57_haproxy}track_interface { bond0.50}notify_master "/.scripts/haproxy/haproxy_master.sh"notify_backup "/.scripts/haproxy/haproxy_backup.sh"notify_fault "/.scripts/haproxy/haproxy_fault.sh"notify_stop "/.scripts/haproxy/haproxy_stop.sh"}[root@WHOPRDLBS00AP0001 ~]# slave节点的keepalived配置：
[root@WH0PRDLBS00APP003 ~]# cat /etc/keepalived/keepalived.conf! Configuration File for keepalivedglobal_defs { router_id keepalived_haproxy_prd_01_02}vrrp_sync_group VGM_haproxy_prd_01 { group { haproxy_bl_prd_240 }}vrrp_script chk_10.36.1.63_haproxy { script "killall -0 haproxy" interval 2}vrrp_instance haproxy_bl_prd_240 { state BACKUP interface eth0 virtual_router_id 240 priority 90 advert_int 1 authentication { auth_type PASS auth_pass !QAZxsw2 } virtual_ipaddress { 10.36.1.240 }track_script { chk_10.36.1.63_haproxy}track_interface { eth0}notify_master "/.scripts/haproxy/haproxy_master.sh"notify_backup "/.scripts/haproxy/haproxy_backup.sh"notify_fault "/.scripts/haproxy/haproxy_fault.sh"notify_stop "/.scripts/haproxy/haproxy_stop.sh"}# 以下是通知脚本[root@WHOPRDLBS00AP0001 ~]# cat /.scripts/haproxy/haproxy_master.sh#!/bin/bashSTARTHAPROXY="service haproxy start"STOPHAPROXY="service haproxy stop"LOGFILE="/logs/keepalived/keepalived-haproxy-state.log"echo "[master]" >> $LOGFILEdate >> $LOGFILEecho "Being master...." >> $LOGFILE 2>&1echo "stop haproxy...." >> $LOGFILE 2>&1$STOPHAPROXY >> $LOGFILE 2>&1echo "start haproxy...." >> $LOGFILE 2>&1$STARTHAPROXY >> $LOGFILE 2>&1echo "haproxy stared ..." >> $LOGFILE[root@WHOPRDLBS00AP0001 ~]# cat /.scripts/haproxy/haproxy_backup.sh#!/bin/bashSTARTHAPROXY="service haproxy start"STOPHAPROXY="service haproxy stop"LOGFILE="/logs/keepalived/keepalived-haproxy-state.log"echo "[backup]" >> $LOGFILEdate >> $LOGFILEecho "Being backup...." >> $LOGFILE 2>&1echo "stop haproxy...." >> $LOGFILE 2>&1$STOPHAPROXY >> $LOGFILE 2>&1echo "start haproxy...." >> $LOGFILE 2>&1$STARTHAPROXY >> $LOGFILE 2>&1echo "haproxy stared ..." >> $LOGFILE[root@WHOPRDLBS00AP0001 ~]# cat /.scripts/haproxy/haproxy_fault.sh#!/bin/bashLOGFILE="/logs/keepalived/keepalived-haproxy-state.log"echo "[fault]" >> $LOGFILEdate >> $LOGFILE[root@WHOPRDLBS00AP0001 ~]# cat /.scripts/haproxy/haproxy_stop.sh#!/bin/bashLOGFILE="/logs/keepalived/keepalived-haproxy-state.log"echo "[stop]" >> $LOGFILEdate >> $LOGFILE[root@WHOPRDLBS00AP0001 ~]# 以下是master节点上面的haproxy服务的配置
[root@WHOPRDLBS00AP0001 ~]# cat /confs/haproxy-1.7.2/haproxy.cfg#---------------------------------------------------------------------# Example configuration for a possible web application. See the# full configuration options online.## http://haproxy.1wt.eu/download/1.4/doc/configuration.txt##---------------------------------------------------------------------#---------------------------------------------------------------------# Global settings#---------------------------------------------------------------------global # to have these messages end up in /var/log/haproxy.log you will # need to: # # 1) configure syslog to accept network log events. This is done # by adding the '-r' option to the SYSLOGD_OPTIONS in # /etc/sysconfig/syslog # # 2) configure local2 events to go to the /var/log/haproxy.log # file. A line like the following can be added to # /etc/sysconfig/syslog # # local2.* /var/log/haproxy.log # log 127.0.0.1 local3 chroot /usr/local/haproxy-1.7.2 pidfile /var/run/haproxy.pid maxconn 65535 user haproxy group haproxy daemon # turn on stats unix socket stats socket /usr/local/haproxy-1.7.2/stats#---------------------------------------------------------------------# common defaults that all the 'listen' and 'backend' sections will# use if not designated in their block#---------------------------------------------------------------------defaults mode http log global option httplog option dontlognull option http-server-close option forwardfor except 127.0.0.0/8 option redispatch retries 3 timeout http-request 10s timeout queue 1m timeout connect 10s timeout client 1m timeout server 1m timeout http-keep-alive 10s timeout check 10s maxconn 10240#---------------------------------------------------------------------# main listen with haproxy#---------------------------------------------------------------------listen authpage bind *:9999 stats enable stats hide-version stats auth haproxy:!QAZxsw2 stats realm haproxy\ staus\ login stats admin if TRUE stats uri /admin?stats#---------------------------------------------------------------------# main frontend which proxys to the backends#---------------------------------------------------------------------frontend staticservers bind *:8080# acl url_static path_beg -i /static /images /javascript /stylesheets# acl url_static path_end -i .jpg .gif .png .css .js# acl url_static path_end -i .html# acl denyfile path /10.html# acl badguy src 192.168.1.100# acl dstipaddr hdr(Host) 192.168.1.110# use_backend static if url_static# use_backend staticservers if url_static# http-request deny if badguy denyfile## default_backend dynamicservers## default_backend dynamicservers reqrep ^GET\ /index.html\ HTTP/1.1 GET\ /test/test.html\ HTTP/1.1 reqrep ^GET\ /\ HTTP/1.1 GET\ /test/test.html\ HTTP/1.1# use_backend staticservers if testurl default_backend staticserversfrontend testbqjrs bind *:9090 reqrep ^GET\ /index.html\ HTTP/1.1 GET\ /bqjr\ HTTP/1.1 reqrep ^GET\ /\ HTTP/1.1 GET\ /bqjr\ HTTP/1.1 default_backend testbqjrs#---------------------------------------------------------------------# static backend for serving up images, stylesheets and such#---------------------------------------------------------------------#backend static# balance roundrobin# server static 127.0.0.1:4331 check#---------------------------------------------------------------------# round robin balancing between the various backends#---------------------------------------------------------------------#backend dynamicservers# balance roundrobin# hash-type consistent# server app1 10.36.1.57:80 check weight 2# server app2 10.36.1.63:80 check weight 1# server backup1 127.0.0.1:8008 check backup# server app2 127.0.0.1:5002 check# server app3 127.0.0.1:5003 check# server app4 127.0.0.1:5004 checkbackend staticservers balance roundrobin# hash-type consistent# server static1 192.168.10.113:80 check weight 1 maxconn 3000 server app1 10.36.1.57:80 check weight 1 rise 3 fall 5 maxconn 5120 server app2 10.36.1.63:80 check weight 1 rise 3 fall 5 maxconn 5120# server backup1 127.0.0.1:8008 check backup# server app2 127.0.0.1:5002 check# server app3 127.0.0.1:5003 check# server app4 127.0.0.1:5004 checkbackend testbqjrs balance roundrobin server app1 10.36.1.14:7001 check weight 1 rise 3 fall 5 maxconn 5120 server app2 10.36.1.15:7001 check weight 1 rise 3 fall 5 maxconn 5120 server app3 10.36.1.16:7001 check weight 1 rise 3 fall 5 maxconn 5120
以下是slave节点的haproxy的配置
[root@WH0PRDLBS00APP003 ~]# cat /confs/haproxy-1.7.2/haproxy.cfg#---------------------------------------------------------------------# Example configuration for a possible web application. See the# full configuration options online.## http://haproxy.1wt.eu/download/1.4/doc/configuration.txt##---------------------------------------------------------------------#---------------------------------------------------------------------# Global settings#---------------------------------------------------------------------global # to have these messages end up in /var/log/haproxy.log you will # need to: # # 1) configure syslog to accept network log events. This is done # by adding the '-r' option to the SYSLOGD_OPTIONS in # /etc/sysconfig/syslog # # 2) configure local2 events to go to the /var/log/haproxy.log # file. A line like the following can be added to # /etc/sysconfig/syslog # # local2.* /var/log/haproxy.log # log 127.0.0.1 local3 chroot /usr/local/haproxy-1.7.2 pidfile /var/run/haproxy.pid maxconn 65535 user haproxy group haproxy daemon # turn on stats unix socket stats socket /usr/local/haproxy-1.7.2/stats#---------------------------------------------------------------------# common defaults that all the 'listen' and 'backend' sections will# use if not designated in their block#---------------------------------------------------------------------defaults mode http log global option httplog option dontlognull option http-server-close option forwardfor except 127.0.0.0/8 option redispatch retries 3 timeout http-request 10s timeout queue 1m timeout connect 10s timeout client 1m timeout server 1m timeout http-keep-alive 10s timeout check 10s maxconn 10240#---------------------------------------------------------------------# main listen with haproxy#---------------------------------------------------------------------listen authpage bind *:9999 stats enable stats hide-version stats auth haproxy:!QAZxsw2 stats realm haproxy\ staus\ login stats admin if TRUE stats uri /admin?stats#---------------------------------------------------------------------#---------------------------------------------------------------------frontend staticservers bind *:8080# acl url_static path_beg -i /static /images /javascript /stylesheets# acl url_static path_end -i .jpg .gif .png .css .js# acl url_static path_end -i .html# acl denyfile path /10.html# acl badguy src 192.168.1.100# acl dstipaddr hdr(Host) 192.168.1.110# use_backend static if url_static# use_backend staticservers if url_static reqrep ^GET\ /index.html\ HTTP/1.1 GET\ /test/test.html\ HTTP/1.1 # 这里是访问index.html会跳转到test/test.html页面； reqrep ^GET\ /\ HTTP/1.1 GET\ /test/test.html\ HTTP/1.1# http-request deny if badguy denyfile default_backend staticserversfrontend testbqjrs bind *:9090 reqrep ^GET\ /index.html\ HTTP/1.1 GET\ /bqjr\ HTTP/1.1 reqrep ^GET\ /\ HTTP/1.1 GET\ /bqjr\ HTTP/1.1# reqrep ^GET\ /\ HTTP/1.1 GET\ /test/test.html\ HTTP/1.1 default_backend testbqjrs#---------------------------------------------------------------------# static backend for serving up images, stylesheets and such#---------------------------------------------------------------------#backend static# balance roundrobin# server static 127.0.0.1:4331 check#---------------------------------------------------------------------# round robin balancing between the various backends#---------------------------------------------------------------------#backend dynamicservers# balance roundrobin# hash-type consistent# server app1 10.36.1.57:80 check weight 2# server app2 10.36.1.63:80 check weight 1# server backup1 127.0.0.1:8008 check backup# server app2 127.0.0.1:5002 check# server app3 127.0.0.1:5003 check# server app4 127.0.0.1:5004 checkbackend staticservers balance roundrobin# hash-type consistent# server static1 192.168.10.113:80 check weight 1 maxconn 3000 server app1 10.36.1.57:80 check weight 1 rise 3 fall 5 maxconn 5120 server app2 10.36.1.63:80 check weight 1 rise 3 fall 5 maxconn 5120# server backup1 127.0.0.1:8008 check backup# server app2 127.0.0.1:5002 check# server app3 127.0.0.1:5003 check# server app4 127.0.0.1:5004 checkbackend testbqjrs balance roundrobin server app1 10.36.1.14:7001 check weight 1 rise 3 fall 5 maxconn 5120 server app2 10.36.1.15:7001 check weight 1 rise 3 fall 5 maxconn 5120 server app3 10.36.1.16:7001 check weight 1 rise 3 fall 5 maxconn 5120[root@WH0PRDLBS00APP003 ~]#
haproxy日志配置
[root@WHOPRDLBS00AP0001 ~]# cat /etc/rsyslog.d/haproxy.conf local3.* /logs/haproxy-1.7.2/haproxy.log&~
静态网页的负载均衡页面
动态网页的负载均衡界面 推荐关注我的个人微信公众号 “云时代IT运维”,周期性更新最新的应用运维类技术文档。关注虚拟化和容器技术、CI/CD、自动化运维等最新前沿运维技术和趋势；
转载于:https://blog.51cto.com/zgui2000/2363151
你可能感兴趣的文章
放开那个程序员
查看>>
构建高性能数据库缓存之Redis（一）
查看>>
测试驱动开发
查看>>
解决MySQL不允许从远程访问
查看>>
puppet介绍及基于httpd实例部署
查看>>
UML常用工具之三--RSA
查看>>
iis7 appcmd的基础命令及简单用法
查看>>
用脚本实现移动某目录下文件名符合指定规则的文件到另一个目录的功能
查看>>
关于SQL镜像配置报错
查看>>
终于找到解决方案了,Qt的Model/View Framework解析
查看>>
线程信息的获取和设置
查看>>
Databricks Scala 编程风格指南
查看>>
Tkinter，label内容随多选框变化
查看>>
PHP开发中的数据类型 ( 第3篇 ) :Heaps
查看>>
网络七层协议
查看>>
4种删除Word空白页的小技巧，都是你需要用到的！
查看>>
单服务器MySQL主从复制实践
查看>>
CentOS 7 root口令恢复
查看>>
| 刘知远：让计算机听懂人话
查看>>
苹果收购初创公司Tueo Health，哮喘监测或将应用到Apple Watch
查看>>