来源
同事
问题场景
- 同步任务开始的时候设置一个标记，放到redis，结束的时候删除；进程被kiil -9之后，redis缓存没有删除；重启应用之后，一直认为在同步中
思考
- redis缓存设置时间
- 如何优雅的停机
研究优雅停机时的一点思考
项目实践
停止脚本 使用kill -15；而不是kill -9
- kill -15 献给应用一段时间善后，这时候Spring容器会正常关闭，执行 @PreDestroy 等类似方法
原先
stop_application() {
echo "stop jar"
if [ -f "$PID_FILE" ]; then
kill -9 `cat $PID_FILE`
rm $PID_FILE
else
echo "pid file $PID_FILE does not exist, do noting"
fi
}
修改后
stop_application() {
echo "stop jar"
if [ -f "$PID_FILE" ]; then
kill -15 `cat $PID_FILE`
else
echo "pid file $PID_FILE does not exist, do noting"
fi
SLEEP=5
FORCE=1
if [ ! -z "$PID_FILE" ]; then
if [ -f "$PID_FILE" ]; then
while [ $SLEEP -ge 0 ]; do
kill -0 `cat "$PID_FILE"` >/dev/null 2>&1
if [ $? -gt 0 ]; then
rm -f "$PID_FILE" >/dev/null 2>&1
if [ $? != 0 ]; then
if [ -w "$PID_FILE" ]; then
cat /dev/null > "$PID_FILE"
# If application has stopped don't try and force a stop with an empty PID file
FORCE=0
else
echo "The PID file could not be removed or cleared."
fi
fi
echo "application stopped."
break
fi
if [ $SLEEP -gt 0 ]; then
sleep 1
fi
if [ $SLEEP -eq 0 ]; then
echo "application did not stop in time."
if [ $FORCE -eq 0 ]; then
echo "PID file was not removed."
fi
echo "To aid diagnostics a thread dump has been written to standard out."
kill -3 `cat "$PID_FILE"`
fi
SLEEP=`expr $SLEEP - 1 `
done
fi
else
echo "application normal stopped."
rm -f "$PID_FILE" >/dev/null 2>&1
fi
##强制杀死进程
KILL_SLEEP_INTERVAL=5
if [ $FORCE -eq 1 ]; then
if [ -z "$PID_FILE" ]; then
echo "Kill failed: \$PID_FILE not set"
else
if [ -f "$PID_FILE" ]; then
PID=`cat "$PID_FILE"`
echo "killing application with the PID: $PID force"
kill -9 $PID
while [ $KILL_SLEEP_INTERVAL -ge 0 ]; do
kill -0 `cat "$PID_FILE"` >/dev/null 2>&1
if [ $? -gt 0 ]; then
rm -f "$PID_FILE" >/dev/null 2>&1
if [ $? != 0 ]; then
if [ -w "$PID_FILE" ]; then
cat /dev/null > "$PID_FILE"
else
echo "The PID file could not be removed."
fi
fi
echo "The application process has been killed."
break
fi
if [ $KILL_SLEEP_INTERVAL -gt 0 ]; then
sleep 1
fi
KILL_SLEEP_INTERVAL=`expr $KILL_SLEEP_INTERVAL - 1 `
done
if [ $KILL_SLEEP_INTERVAL -lt 0 ]; then
echo "application has not been killed completely yet. The process might be waiting on some system call or might be UNINTERRUPTIBLE."
fi
fi
fi
fi
}
JAVA代码
Spring容器关闭的钩子
- Spring已经自动帮我们加了钩子
org.springframework.context.support.AbstractApplicationContext#registerShutdownHook
@Override
public void registerShutdownHook() {
if (this.shutdownHook == null) {
// No shutdown hook registered yet.
this.shutdownHook = new Thread() {
@Override
public void run() {
synchronized (startupShutdownMonitor) {
doClose();
}
}
};
Runtime.getRuntime().addShutdownHook(this.shutdownHook);
}
}
- 实现下面这些，Sping容器关闭的时候会自动处理
- @PreDestroy注解
- destory-method方法
- DisposableBean接口
也可以自己加其他的钩子
Runtime.getRuntime().addShutdownHook(this.shutdownHook);
触发了 linux OOM KILLER，还会不会执行钩子？
应该不会，属于强杀
- 这是linux oom kill 的代码
强行翻译：
为了防止OOM受害者耗尽 在它控制下的 从用户空间那保留的 内存，我们应该 在授权它访问保留的内存 之前发送SIGKILL
/*
* We should send SIGKILL before granting access to memory reserves
* in order to prevent the OOM victim from depleting the memory
* reserves from the user space under its control.
*/
do_send_sig_info(SIGKILL, SEND_SIG_PRIV, victim, PIDTYPE_TGID);
- 代码中的SIGKILL
https://linux.cn/article-8771-1.html
image.png
