OpenResty
需求：
OpenResty运行在Docker中，后端代理的server存在IP不固定的情况，需要OpenResty动态更新后端地址
实现思路：
- 在OpenResty中定义共享Dict，将后端IP保存在其中
- 设置/add和/delete的location，通过Web调用增删后端IP
- 设置proxy_pass代理，代理的地址从共享Dict中获取实现请求
过程验证(这里所有服务在一个Centos 7 Docker中启动)：
- 启动两个后端服务
- 启动三个终端，在第一个终端中启动web服务test1
[root@f2d9f997edac /]# mkdir -p test1 && cd test1 && echo test1 > test
[root@f2d9f997edac test1]# python -m SimpleHTTPServer 8081
Serving HTTP on 0.0.0.0 port 8081 ...
- 在第二个终端中启动web服务test2
[root@f2d9f997edac /]# mkdir -p test2 && cd test2 && echo test2 > test
[root@f2d9f997edac test2]# python -m SimpleHTTPServer 8082
Serving HTTP on 0.0.0.0 port 8082 ...
- 在第三个终端中验证test1和test2
[root@f2d9f997edac /]# curl 127.0.0.1:8081/test
test1
[root@f2d9f997edac /]# curl 127.0.0.1:8082/test
test2
- 安装OpenResty并配置
- 在第三个终端中，执行下面命令安装Openresty（此处为CentOS系统，其余系统的安装命令请自行搜索）
[root@f2d9f997edac /]# yum install -y openresty
- 编辑配置文件，验证配置文件并启动Openrestry
[root@f2d9f997edac /]# cat /usr/local/openresty/nginx/conf/nginx.conf
worker_processes 4;
events {
worker_connections 1024;
}
http {
include mime.types;
default_type application/octet-stream;
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
#access_log logs/access.log main;
sendfile on;
#tcp_nopush on;
keepalive_timeout 65;
#gzip on;
lua_shared_dict ups 10m;
server {
listen 80;
location /add {
content_by_lua_block {
local ups = ngx.shared.ups
local ip = ngx.req.get_uri_args()["ip"]
local port = ngx.req.get_uri_args()["port"]
local no = ngx.req.get_uri_args()["no"]
if ip == nil or port == nil or no == nil then
ngx.say("usage: /add?ip=x.x.x.x&port=xx&no=xx")
return
end
ups:set("host-"..no, ip)
ups:set("port-"..no, port)
-- ups["host-"..no] = ip
-- ups["port-"..no] = port
local count = 0
for k in pairs(ups:get_keys(0)) do count = count + 1 end
ups:set("servers", count)
ngx.say("OK")
}
}
location /get {
content_by_lua_block {
local ups = ngx.shared.ups
local no = ngx.req.get_uri_args()["no"]
if no == nil then
ngx.say("usage: /get?no=xx")
return
end
if ups:get("host-" .. no) == nil then
ngx.say("the no server is vaild: " .. no)
return
end
local count = 0
for k in pairs(ups:get_keys(0)) do count = count + 1 end
ups:set("servers", count)
ngx.say("ip: " .. ups:get("host-" .. no))
ngx.say("port: " .. ups:get("port-" .. no))
ngx.say("servers: " .. math.floor(ups:get("servers")/2))
}
}
location /delete {
content_by_lua_block {
local ups = ngx.shared.ups
local no = ngx.req.get_uri_args()["no"]
if no == nil then
ngx.say("usage: /get?no=xx")
return
end
if ups:get("host-" .. no) == nil then
ngx.say("the no server is vaild: " .. no)
return
end
ups:delete("host-" .. no)
ups:delete("port-" .. no)
ngx.say("OK")
}
}
location / {
set_by_lua_block $cur_ups {
local ups = ngx.shared.ups
local items = ups:get("servers")
local n = math.random(math.floor(items/2))
local ss = string.gsub(ups:get_keys(0)[n], "%a+-", "")
s = ups:get("host-"..ss) .. ":" .. ups:get("port-"..ss)
return s
}
proxy_next_upstream off;
proxy_set_header Host $host;
proxy_http_version 1.1;
proxy_set_header Connection "";
proxy_pass $scheme://$cur_ups;
}
}
}
[root@f2d9f997edac /]# openresty -t
nginx: the configuration file /usr/local/openresty/nginx/conf/nginx.conf syntax is ok
nginx: configuration file /usr/local/openresty/nginx/conf/nginx.conf test is successful
[root@f2d9f997edac /]# openresty
- 增加后端服务并验证（/get通过no查询后端服务的IP和端口）
- 在add的时候指定后端服务的no(序列号)、IP和端口信息
- no用来区分不同后端服务，执行相同no的提交会覆盖原来的no信息
- proxy_pass拿到的后端服务随机产生(2中的math.random(math.floor(items/2))处)
[root@f2d9f997edac /]# curl '127.0.0.1:80/add?no=1&ip=127.0.0.1&port=8081'
OK
[root@f2d9f997edac /]# curl '127.0.0.1:80/get?no=1'
ip: 127.0.0.1
port: 8081
servers: 1
[root@f2d9f997edac /]# curl '127.0.0.1:80/add?no=2&ip=127.0.0.1&port=8082'
OK
[root@f2d9f997edac /]# curl '127.0.0.1:80/get?no=2'
ip: 127.0.0.1
port: 8082
servers: 2
[root@f2d9f997edac /]# curl '127.0.0.1:80/test'
test1
[root@f2d9f997edac /]# curl '127.0.0.1:80/test'
test2
[root@f2d9f997edac /]#
- 删除后端服务并验证
/delete通过no删除后端服务的IP和端口信息
[root@f2d9f997edac /]# curl '127.0.0.1:80/delete?no=1'
OK
[root@f2d9f997edac /]# curl '127.0.0.1:80/get?no=1'
the no server is vaild: 1
[root@f2d9f997edac /]# curl '127.0.0.1:80/get?no=2'
ip: 127.0.0.1
port: 8082
servers: 1
[root@f2d9f997edac /]# curl '127.0.0.1:80/test'
test2
[root@f2d9f997edac /]# curl '127.0.0.1:80/test'
test2
[root@f2d9f997edac /]#
存在不足：
- 如果由程序控制OpenResty的/add和/delete来维护后端服务信息，需要保证与序号no的对应
- proxy_pass中的后端服务地址和端口对通过随机拿到，存在负载不均衡的情况，可以根据自己的实际需求修改，比如修改为ip_hash，但需要考虑保存在共享内存Dict中后端服务的数量
