经验首页 前端设计 程序设计 Java相关 移动开发 数据库/运维 软件/图像 大数据/云计算 其他经验
当前位置:技术经验 » Java相关 » Java » 查看文章
log4j2异步日志解读(一)AsyncAppender
来源:cnblogs  作者:lewis09  时间:2018/11/23 10:20:32  对本文有异议

 

log4j、logback、log4j2 历史和关系,我们就在这里不展开讲了。直接上干货,log4j2突出于其他日志的优势,异步日志实现。

看一个东西,首先看官网文档,https://logging.apache.org/log4j/2.x/ ,因为前面文章已经讲解了disruptor源码【https://www.cnblogs.com/lewis09/p/9974617.html】,本文主要展开说说异步日志AsyncAppender和AsyncLogger(基于disruptor实现)。

AsyncLogger笔者下文展开讲。

 

一、AsyncAppender

 

我们先来看看AsyncApperder核心,就是logger将数据通过append方法放入到阻塞队列中,随后后台线程从队列中取出数据然后进行后续的操作。

那这样看来,就很简单了,一个append()方法,一个后台线程执行就是我们要看的核心代码了。围绕我们要看的类AsyncAppender,来看看类关系图。

一、放入队列

主要实现就是logger将数据通过append方法放入到阻塞队列中。

  1. //AsyncAppender.java
  2. /**
  3. * Actual writing occurs here.
  4. *
  5. * @param logEvent The LogEvent.
  6. */
  7. @Override
  8. public void append(final LogEvent logEvent) {
  9. if (!isStarted()) {
  10. throw new IllegalStateException("AsyncAppender " + getName() + " is not active");
  11. }
  12. //创建Log4jLogEvent的对象memento
  13. final Log4jLogEvent memento = Log4jLogEvent.createMemento(logEvent, includeLocation);
  14. InternalAsyncUtil.makeMessageImmutable(logEvent.getMessage());
  15. //transfer(memento)将event放入队列
  16. //默认ArrayBlockingQueueFactory 大小1024
  17. if (!transfer(memento)) {
  18. if (blocking) {
  19. if (AbstractLogger.getRecursionDepth() > 1) { // LOG4J2-1518, LOG4J2-2031
  20. // If queue is full AND we are in a recursive call, call appender directly to prevent deadlock
  21. AsyncQueueFullMessageUtil.logWarningToStatusLogger();
  22. logMessageInCurrentThread(logEvent);
  23. } else {
  24. // delegate to the event router (which may discard, enqueue and block, or log in current thread)
  25. final EventRoute route = asyncQueueFullPolicy.getRoute(thread.getId(), memento.getLevel());
  26. route.logMessage(this, memento);
  27. }
  28. } else {
  29. error("Appender " + getName() + " is unable to write primary appenders. queue is full");
  30. logToErrorAppenderIfNecessary(false, memento);
  31. }
  32. }
  33. }
  34. private boolean transfer(final LogEvent memento) {
  35. return queue instanceof TransferQueue
  36. ? ((TransferQueue<LogEvent>) queue).tryTransfer(memento)
  37. : queue.offer(memento);
  38. }

如流程图所示,首先会判断用户是否设置了blocking选项,默认是true,如果设置为false,则Appender直接会ToErrorAppender,如果用户没有配置或者配置为true,则会按照一定的策略来处理这些消息。策略可以分为2种,他们分别为:

1、DefaultAsyncQueueFullPolicy---等待队列,转为同步操作策略

  1. public class DefaultAsyncQueueFullPolicy implements AsyncQueueFullPolicy {
  2. @Override
  3. public EventRoute getRoute(final long backgroundThreadId, final Level level) {
  4. // LOG4J2-471: prevent deadlock when RingBuffer is full and object
  5. // being logged calls Logger.log() from its toString() method
  6. if (Thread.currentThread().getId() == backgroundThreadId) {
  7. return EventRoute.SYNCHRONOUS;
  8. }
  9. return EventRoute.ENQUEUE;
  10. }


2、DiscardingAsyncQueueFullPolicy---按照日志等级抛弃日志策略

  1. //DiscardingAsyncQueueFullPolicy.java
  2. @Override
  3. public EventRoute getRoute(final long backgroundThreadId, final Level level) {
  4. if (level.isLessSpecificThan(thresholdLevel)) {
  5. if (discardCount.getAndIncrement() == 0) {
  6. LOGGER.warn("Async queue is full, discarding event with level {}. " +
  7. "This message will only appear once; future events from {} " +
  8. "are silently discarded until queue capacity becomes available.",
  9. level, thresholdLevel);
  10. }
  11. return EventRoute.DISCARD;
  12. }
  13. return super.getRoute(backgroundThreadId, level);
  14. }

二、后台线程执行后续操作。

主要就是后台线程从队列中取出数据然后进行后续的操作。

  1. //AsyncAppender.java
  2. private class AsyncThread extends Log4jThread {
  3. private volatile boolean shutdown = false;
  4. private final List<AppenderControl> appenders;
  5. private final BlockingQueue<LogEvent> queue;
  6. public AsyncThread(final List<AppenderControl> appenders, final BlockingQueue<LogEvent> queue) {
  7. super("AsyncAppender-" + THREAD_SEQUENCE.getAndIncrement());
  8. this.appenders = appenders;
  9. this.queue = queue;
  10. setDaemon(true);
  11. }
  12. @Override
  13. public void run() {
  14. while (!shutdown) {
  15. LogEvent event;
  16. try {
  17. event = queue.take();
  18. if (event == SHUTDOWN_LOG_EVENT) {
  19. shutdown = true;
  20. continue;
  21. }
  22. } catch (final InterruptedException ex) {
  23. break; // LOG4J2-830
  24. }
  25. event.setEndOfBatch(queue.isEmpty());
  26. final boolean success = callAppenders(event);
  27. if (!success && errorAppender != null) {
  28. try {
  29. errorAppender.callAppender(event);
  30. } catch (final Exception ex) {
  31. // Silently accept the error.
  32. }
  33. }
  34. }
  35. // Process any remaining items in the queue.
  36. LOGGER.trace("AsyncAppender.AsyncThread shutting down. Processing remaining {} queue events.",
  37. queue.size());
  38. int count = 0;
  39. int ignored = 0;
  40. while (!queue.isEmpty()) {
  41. try {
  42. final LogEvent event = queue.take();
  43. if (event instanceof Log4jLogEvent) {
  44. final Log4jLogEvent logEvent = (Log4jLogEvent) event;
  45. logEvent.setEndOfBatch(queue.isEmpty());
  46. callAppenders(logEvent);
  47. count++;
  48. } else {
  49. ignored++;
  50. LOGGER.trace("Ignoring event of class {}", event.getClass().getName());
  51. }
  52. } catch (final InterruptedException ex) {
  53. // May have been interrupted to shut down.
  54. // Here we ignore interrupts and try to process all remaining events.
  55. }
  56. }
  57. LOGGER.trace("AsyncAppender.AsyncThread stopped. Queue has {} events remaining. "
  58. + "Processed {} and ignored {} events since shutdown started.", queue.size(), count, ignored);
  59. }
  60. ...
  61. }

该线程会一直尝试从阻塞队列中获取LogEvent,如果获取成功,调用AppenderRef所引用Appender的append方法。我们也可以看到,AsyncAppender实际上主要是类似于中转,日志异步化,当消息放入阻塞队列,返回成功,这样能够大幅提高日志记录的吞吐。用户可以在权衡性能与日志收集质量上进行权衡配置策略(设置blocking选项),当然也可以设置不同类型的阻塞队列已到达更好的日志记录吞吐。

AsyncAppender配置参数  

https://logging.apache.org/log4j/2.x/manual/appenders.html#AsyncAppender

 

 友情链接:直通硅谷  点职佳  北美留学生论坛

本站QQ群:前端 618073944 | Java 606181507 | Python 626812652 | C/C++ 612253063 | 微信 634508462 | 苹果 692586424 | C#/.net 182808419 | PHP 305140648 | 运维 608723728

W3xue 的所有内容仅供测试,对任何法律问题及风险不承担任何责任。通过使用本站内容随之而来的风险与本站无关。
关于我们  |  意见建议  |  捐助我们  |  报错有奖  |  广告合作、友情链接(目前9元/月)请联系QQ:27243702 沸活量
皖ICP备17017327号-2 皖公网安备34020702000426号