经验首页 前端设计 程序设计 Java相关 移动开发 数据库/运维 软件/图像 大数据/云计算 其他经验
当前位置:技术经验 » 数据库/运维 » Kubernetes » 查看文章
Kubernetes中Nginx配置热加载的全过程
来源:jb51  时间:2022/1/19 13:44:46  对本文有异议

前言

Nginx本身是支持热更新的,通过nginx -s reload指令,实际通过向进程发送HUB信号实现不停服重新加载配置,然而在Docker或者Kubernetes中,每次都需要进容器执行nginx -s reload指令,单docker容器还好说,可以在外面通过exec指定容器执行该指令进行热加载,Kubernetes的话,就比较难受了

今天介绍一下Kubernetes中Nginx热加载配置的处理方法——reloader

reloader地址:https://github.com/stakater/Reloader

reloader主要就是用来监测ConfigMap或Secret的变化,然后对相关DeploymentConfig的Deployment、DaemonSet执行滚动升级

reloader需要kubernetes1.9以上的版本才支持

使用方法

首先是安装部署reloader

  1. # 直接通过官方yaml文件部署
  2. kubectl apply -f https://raw.githubusercontent.com/stakater/Reloader/master/deployments/kubernetes/reloader.yaml

默认情况下reloader是部署在default命名空间,但是它是监控所有命名空间的configmaps和secrets

当然,如果不想监控某个configmap或secret,可以通过--resources-to-ignore=configMaps/secrets来忽略某个资源

部署成功后,就可以直接使用了,我提前部署了nginx和configmap

这是目前的配置,看一下Nginx目前的配置

接着,我修改Nginx的Deployment,添加reloader,监听nginx-config这个ConfigMap,执行reload

  1. {
  2. "kind": "Deployment",
  3. "apiVersion": "extensions/v1beta1",
  4. "metadata": {
  5. "name": "nginx",
  6. "namespace": "default",
  7. "selfLink": "/apis/extensions/v1beta1/namespaces/default/deployments/nginx",
  8. "uid": "7eee5fa8-7514-11ec-a916-0210d5e9ca3b",
  9. "resourceVersion": "286141",
  10. "generation": 10,
  11. "creationTimestamp": "2022-01-14T08:32:23Z",
  12. "labels": {
  13. "k8s-app": "nginx"
  14. },
  15. "annotations": {
  16. "deployment.kubernetes.io/revision": "9",
  17. "description": "nginx应用"
  18. # 主要是这行
  19. "reloader.stakater.com/reload": "nginx-config"
  20. }
  21. },
  22. "spec": {
  23. "replicas": 1,
  24. "selector": {
  25. "matchLabels": {
  26. "k8s-app": "nginx"
  27. }
  28. }
  29. ……

然后apply该Deployment,之后我们去更新ConfigMap,更新nginx配置文件

更新完成,去掉proxy_redirect,然后去看nginx容器是否执行滚动更新

可以看到,nginx执行了滚动更新,接着看下nginx配置文件是否更新

这样很简单的通过reloader就可以实现Nginx的配置热加载

除了这种方法,常见的方法还有使用sidecar,通过sidecar去做的话,需要自己写监听脚本,比较麻烦,但是有时候也相对灵活,这里也附一个sidecar的python脚本

  1. #!/usr/bin/env python
  2. # -*- encoding: utf8 -*-
  3. """
  4. 需求:nginx配置文件变化,自动更新配置文件,类似nginx -s reload
  5. 实现:
  6. 1、用pyinotify实时监控nginx配置文件变化
  7. 2、如果配置文件变化,给系统发送HUP来reload nginx
  8. """
  9. import os
  10. import re
  11. import pyinotify
  12. import logging
  13. from threading import Timer
  14.  
  15. # Param
  16. LOG_PATH = "/root/python/log"
  17. CONF_PATHS = [
  18. "/etc/nginx",
  19. ]
  20. DELAY = 5
  21. SUDO = False
  22. RELOAD_COMMAND = "nginx -s reload"
  23. if SUDO:
  24. RELOAD_COMMAND = "sudo " + RELOAD_COMMAND
  25.  
  26. # Log
  27. logger = logging.getLogger(__name__)
  28. logger.setLevel(level = logging.INFO)
  29. log_handler = logging.FileHandler(LOG_PATH)
  30. log_handler.setLevel(logging.INFO)
  31. log_formatter = logging.Formatter('%(asctime)s - %(levelname)s - %(message)s')
  32. log_handler.setFormatter(log_formatter)
  33. logger.addHandler(log_handler)
  34.  
  35. # Reloader
  36. def reload_nginx():
  37. os.system(RELOAD_COMMAND)
  38. logger.info("nginx is reloaded")
  39.  
  40. t = Timer(DELAY, reload_nginx)
  41.  
  42. def trigger_reload_nginx(pathname, action):
  43. logger.info("nginx monitor is triggered because %s is %s" % (pathname, action))
  44. global t
  45. if t.is_alive():
  46. t.cancel()
  47. t = Timer(DELAY, reload_nginx)
  48. t.start()
  49. else:
  50. t = Timer(DELAY, reload_nginx)
  51. t.start()
  52.  
  53. events = pyinotify.IN_MODIFY | pyinotify.IN_CREATE | pyinotify.IN_DELETE
  54.  
  55. watcher = pyinotify.WatchManager()
  56. watcher.add_watch(CONF_PATHS, events, rec=True, auto_add=True)
  57.  
  58. class EventHandler(pyinotify.ProcessEvent):
  59. def process_default(self, event):
  60. if event.name.endswith(".conf"):
  61. if event.mask == pyinotify.IN_CREATE:
  62. action = "created"
  63. if event.mask == pyinotify.IN_MODIFY:
  64. action = "modified"
  65. if event.mask == pyinotify.IN_DELETE:
  66. action = "deleted"
  67. trigger_reload_nginx(event.pathname, action)
  68.  
  69. handler = EventHandler()
  70. notifier = pyinotify.Notifier(watcher, handler)
  71.  
  72. # Start
  73. logger.info("Start Monitoring")
  74. notifier.loop()

如果喜欢用go的,这里也提供go脚本

  1. package main
  2.  
  3. import (
  4. "log"
  5. "os"
  6. "path/filepath"
  7. "syscall"
  8.  
  9. "github.com/fsnotify/fsnotify"
  10. proc "github.com/shirou/gopsutil/process"
  11. )
  12.  
  13. const (
  14. nginxProcessName = "nginx"
  15. defaultNginxConfPath = "/etc/nginx"
  16. watchPathEnvVarName = "WATCH_NGINX_CONF_PATH"
  17. )
  18.  
  19. var stderrLogger = log.New(os.Stderr, "error: ", log.Lshortfile)
  20. var stdoutLogger = log.New(os.Stdout, "", log.Lshortfile)
  21.  
  22. func getMasterNginxPid() (int, error) {
  23. processes, processesErr := proc.Processes()
  24. if processesErr != nil {
  25. return 0, processesErr
  26. }
  27.  
  28. nginxProcesses := map[int32]int32{}
  29.  
  30. for _, process := range processes {
  31. processName, processNameErr := process.Name()
  32. if processNameErr != nil {
  33. return 0, processNameErr
  34. }
  35.  
  36. if processName == nginxProcessName {
  37. ppid, ppidErr := process.Ppid()
  38.  
  39. if ppidErr != nil {
  40. return 0, ppidErr
  41. }
  42.  
  43. nginxProcesses[process.Pid] = ppid
  44. }
  45. }
  46.  
  47. var masterNginxPid int32
  48.  
  49. for pid, ppid := range nginxProcesses {
  50. if ppid == 0 {
  51. masterNginxPid = pid
  52.  
  53. break
  54. }
  55. }
  56.  
  57. stdoutLogger.Println("found master nginx pid:", masterNginxPid)
  58.  
  59. return int(masterNginxPid), nil
  60. }
  61.  
  62. func signalNginxReload(pid int) error {
  63. stdoutLogger.Printf("signaling master nginx process (pid: %d) -> SIGHUP\n", pid)
  64. nginxProcess, nginxProcessErr := os.FindProcess(pid)
  65.  
  66. if nginxProcessErr != nil {
  67. return nginxProcessErr
  68. }
  69.  
  70. return nginxProcess.Signal(syscall.SIGHUP)
  71. }
  72.  
  73. func main() {
  74. watcher, watcherErr := fsnotify.NewWatcher()
  75. if watcherErr != nil {
  76. stderrLogger.Fatal(watcherErr)
  77. }
  78. defer watcher.Close()
  79.  
  80. done := make(chan bool)
  81. go func() {
  82. for {
  83. select {
  84. case event, ok := <-watcher.Events:
  85. if !ok {
  86. return
  87. }
  88.  
  89. if event.Op&fsnotify.Create == fsnotify.Create {
  90. if filepath.Base(event.Name) == "..data" {
  91. stdoutLogger.Println("config map updated")
  92.  
  93. nginxPid, nginxPidErr := getMasterNginxPid()
  94. if nginxPidErr != nil {
  95. stderrLogger.Printf("getting master nginx pid failed: %s", nginxPidErr.Error())
  96.  
  97. continue
  98. }
  99.  
  100. if err := signalNginxReload(nginxPid); err != nil {
  101. stderrLogger.Printf("signaling master nginx process failed: %s", err)
  102. }
  103. }
  104. }
  105. case err, ok := <-watcher.Errors:
  106. if !ok {
  107. return
  108. }
  109. stderrLogger.Printf("received watcher.Error: %s", err)
  110. }
  111. }
  112. }()
  113.  
  114. pathToWatch, ok := os.LookupEnv(watchPathEnvVarName)
  115. if !ok {
  116. pathToWatch = defaultNginxConfPath
  117. }
  118.  
  119. stdoutLogger.Printf("adding path: `%s` to watch\n", pathToWatch)
  120.  
  121. if err := watcher.Add(pathToWatch); err != nil {
  122. stderrLogger.Fatal(err)
  123. }
  124. <-done
  125. }

ok,今天的内容就到这里

总结

到此这篇关于Kubernetes中Nginx配置热加载的文章就介绍到这了,更多相关Kubernetes中Nginx配置热加载内容请搜索w3xue以前的文章或继续浏览下面的相关文章希望大家以后多多支持w3xue!

 友情链接:直通硅谷  点职佳  北美留学生论坛

本站QQ群:前端 618073944 | Java 606181507 | Python 626812652 | C/C++ 612253063 | 微信 634508462 | 苹果 692586424 | C#/.net 182808419 | PHP 305140648 | 运维 608723728

W3xue 的所有内容仅供测试,对任何法律问题及风险不承担任何责任。通过使用本站内容随之而来的风险与本站无关。
关于我们  |  意见建议  |  捐助我们  |  报错有奖  |  广告合作、友情链接(目前9元/月)请联系QQ:27243702 沸活量
皖ICP备17017327号-2 皖公网安备34020702000426号