CLOVER🍀

That was when it all began.

Fluent BitのParser Filter Pluginで、異なるフォーマットのログに複数のパーサーを適用する

これは、なにをしたくて書いたもの?

Fluent BitのParser Filter Pluginでは、複数のパーサーを設定できるようなので、その挙動を確認してみようかなと。

Parser - Fluent Bit: Official Manual

Parser Filter Plugin?

まず最初に、Parser Filter Pluginとはなにか?を見てみます。

The Parser Filter plugin allows to parse field in event records.

Parsers - Fluent Bit: Official Manual

レコード内のフィールドをパースできるプラグイン、ですと。

パーサーの定義は、Fluent Bitの設定ファイルの「parsers_file」で読み込みます。

[SERVICE]
    ...

    parsers_file parsers.conf

「parsers_file」で指定したファイルには、デフォルトでいくつかのログフォーマットに応じたパーサーが定義してあります。

[PARSER]
    Name   apache
    Format regex
    Regex  ^(?<host>[^ ]*) [^ ]* (?<user>[^ ]*) \[(?<time>[^\]]*)\] "(?<method>\S+)(?: +(?<path>[^\"]*?)(?: +\S*)?)?" (?<code>[^ ]*) (?<size>[^ ]*)(?: "(?<referer>[^\"]*)" "(?<agent>[^\"]*)")?$
    Time_Key time
    Time_Format %d/%b/%Y:%H:%M:%S %z

[PARSER]
    Name   apache2
    Format regex
    Regex  ^(?<host>[^ ]*) [^ ]* (?<user>[^ ]*) \[(?<time>[^\]]*)\] "(?<method>\S+)(?: +(?<path>[^ ]*) +\S*)?" (?<code>[^ ]*) (?<size>[^ ]*)(?: "(?<referer>[^\"]*)" "(?<agent>.*)")?$
    Time_Key time
    Time_Format %d/%b/%Y:%H:%M:%S %z

[PARSER]
    Name   apache_error
    Format regex
    Regex  ^\[[^ ]* (?<time>[^\]]*)\] \[(?<level>[^\]]*)\](?: \[pid (?<pid>[^\]]*)\])?( \[client (?<client>[^\]]*)\])? (?<message>.*)$

[PARSER]
    Name   nginx
    Format regex
    Regex ^(?<remote>[^ ]*) (?<host>[^ ]*) (?<user>[^ ]*) \[(?<time>[^\]]*)\] "(?<method>\S+)(?: +(?<path>[^\"]*?)(?: +\S*)?)?" (?<code>[^ ]*) (?<size>[^ ]*)(?: "(?<referer>[^\"]*)" "(?<agent>[^\"]*)")
    Time_Key time
    Time_Format %d/%b/%Y:%H:%M:%S %z

〜省略〜

nginx用のパーサーを見てみましょう。

[PARSER]
    Name   nginx
    Format regex
    Regex ^(?<remote>[^ ]*) (?<host>[^ ]*) (?<user>[^ ]*) \[(?<time>[^\]]*)\] "(?<method>\S+)(?: +(?<path>[^\"]*?)(?: +\S*)?)?" (?<code>[^ ]*) (?<size>[^ ]*)(?: "(?<referer>[^\"]*)" "(?<agent>[^\"]*)")
    Time_Key time
    Time_Format %d/%b/%Y:%H:%M:%S %z

Nameでパーサー名、Formatでパースに使うフォーマットを指定します。

フォーマットには以下の4つがあり、

さらにJSONがネストしているフィールドであればデコードすることもできます。

Decoders - Fluent Bit: Official Manual

こんなやつですね。

{"log":"{\"status\": \"up and running\"}\r\n","stream":"stdout","time":"2018-03-09T01:01:44.851160855Z"}

今回のnginxの例では、正規表現でのパーサーを使用しています。

Regular Expression - Fluent Bit: Official Manual

こんな感じで、正規表現でグループ化してキャプチャした時の名前を、レコード内のフィールドとしてパースすることができます。

    Regex ^(?<remote>[^ ]*) (?<host>[^ ]*) (?<user>[^ ]*) \[(?<time>[^\]]*)\] "(?<method>\S+)(?: +(?<path>[^\"]*?)(?: +\S*)?)?" (?<code>[^ ]*) (?<size>[^ ]*)(?: "(?<referer>[^\"]*)" "(?<agent>[^\"]*)")

Time_KeyとTime_Formatは、パースした結果のフィールドのうち、Timeフィールドとして扱うものがあればそれを指定します。
Time_Formatは、文字通り時間としてパースするためのフォーマットを指定します。

    Time_Key time
    Time_Format %d/%b/%Y:%H:%M:%S %z

パーサーを使う時は、Filter Pluginとして「parser」を指定し、Parserで使うパーサーのNameを、Key_Nameでパースする
フィールドを指定します。

[FILTER]
    Name parser
    Match *
    Key_Name log
    Parser nginx

このParserを複数指定することができるようなので、その動きを確認してみようというのが今回の話です。

Parser … Specify the parser name to interpret the field. Multiple Parser entries are allowed (one per line).

環境

今回の環境は、こちらです。

$ /opt/td-agent-bit/bin/td-agent-bit --version
Fluent Bit v1.5.4

また、取り込むログのお題としては、Apacheを使うことにしましょう。

$ apache2 -v
Server version: Apache/2.4.41 (Ubuntu)
Server built:   2020-08-12T19:46:17

今回は、Apacheのアクセスログとエラーログが混ざったInputを受け取り、パースする設定を書いてみましょう。

デフォルトの設定

まずは、Fluent Bitのデフォルトの設定を記載しておきます。

$ grep -v '^ *#' /etc/td-agent-bit/td-agent-bit.conf
[SERVICE]
    flush        5

    daemon       Off

    log_level    info

    parsers_file parsers.conf

    plugins_file plugins.conf

    http_server  Off
    http_listen  0.0.0.0
    http_port    2020

    storage.metrics on





[INPUT]
    name cpu
    tag  cpu.local

    interval_sec 1

[OUTPUT]
    name  stdout
    match *

パーサーの部分は、使用する最低限の部分だけ載せておきましょう。
/etc/td-agent-bit/parsers.conf

[PARSER]
    Name   apache
    Format regex
    Regex  ^(?<host>[^ ]*) [^ ]* (?<user>[^ ]*) \[(?<time>[^\]]*)\] "(?<method>\S+)(?: +(?<path>[^\"]*?)(?: +\S*)?)?" (?<code>[^ ]*) (?<size>[^ ]*)(?: "(?<referer>[^\"]*)" "(?<agent>[^\"]*)")?$
    Time_Key time
    Time_Format %d/%b/%Y:%H:%M:%S %z

[PARSER]
    Name   apache2
    Format regex
    Regex  ^(?<host>[^ ]*) [^ ]* (?<user>[^ ]*) \[(?<time>[^\]]*)\] "(?<method>\S+)(?: +(?<path>[^ ]*) +\S*)?" (?<code>[^ ]*) (?<size>[^ ]*)(?: "(?<referer>[^\"]*)" "(?<agent>.*)")?$
    Time_Key time
    Time_Format %d/%b/%Y:%H:%M:%S %z

[PARSER]
    Name   apache_error
    Format regex
    Regex  ^\[[^ ]* (?<time>[^\]]*)\] \[(?<level>[^\]]*)\](?: \[pid (?<pid>[^\]]*)\])?( \[client (?<client>[^\]]*)\])? (?<message>.*)$

[PARSER]
    Name   nginx
    Format regex
    Regex ^(?<remote>[^ ]*) (?<host>[^ ]*) (?<user>[^ ]*) \[(?<time>[^\]]*)\] "(?<method>\S+)(?: +(?<path>[^\"]*?)(?: +\S*)?)?" (?<code>[^ ]*) (?<size>[^ ]*)(?: "(?<referer>[^\"]*)" "(?<agent>[^\"]*)")
    Time_Key time
    Time_Format %d/%b/%Y:%H:%M:%S %z

〜省略〜

今回は、apache2とapache_errorを使うことにします。

こんな感じで、Apacheのアクセスログとエラーログを用意。

$ cat /var/log/apache2/access.log /var/log/apache2/error.log 
127.0.0.1 - - [27/Aug/2020:13:45:16 +0000] "GET / HTTP/1.1" 200 11173 "-" "curl/7.68.0"
127.0.0.1 - - [27/Aug/2020:13:45:18 +0000] "GET /index.html HTTP/1.1" 200 11173 "-" "curl/7.68.0"
127.0.0.1 - - [27/Aug/2020:13:45:30 +0000] "GET /index.html HTTP/1.1" 200 11173 "-" "curl/7.68.0"
127.0.0.1 - - [27/Aug/2020:13:45:32 +0000] "GET / HTTP/1.1" 200 11173 "-" "curl/7.68.0"
[Thu Aug 27 13:45:14.957795 2020] [mpm_event:notice] [pid 1863:tid 139821141662784] AH00489: Apache/2.4.41 (Ubuntu) configured -- resuming normal operations
[Thu Aug 27 13:45:14.957893 2020] [core:notice] [pid 1863:tid 139821141662784] AH00094: Command line: '/usr/sbin/apache2'
[Thu Aug 27 13:45:34.983343 2020] [mpm_event:notice] [pid 1863:tid 139821141662784] AH00491: caught SIGTERM, shutting down
[Thu Aug 27 13:45:35.032522 2020] [mpm_event:notice] [pid 1955:tid 139947430849600] AH00489: Apache/2.4.41 (Ubuntu) configured -- resuming normal operations
[Thu Aug 27 13:45:35.032606 2020] [core:notice] [pid 1955:tid 139947430849600] AH00094: Command line: '/usr/sbin/apache2'

こちらをTCP Input Pluginで受け取って

[INPUT]
    Name tcp
    Listen 0.0.0.0
    Port 5170
    Format none
    Tag apache.mixed.log


〜省略〜


[OUTPUT]
    Name  stdout
    Match *
    Format json_lines

標準出力で確認する構成としましょう。

$ sudo journalctl -u td-agent-bit -f

Multiple Parser

では、複数のパーサーを使ってみます。

使い方は至って簡単で、Parser自体の定義をこんな感じで複数並べます。

[FILTER]
    Name     parser
    Match    apache.*
    Key_Name log
    Parser   apache2
    Parser   apache_error

確認。

$ cat /var/log/apache2/access.log /var/log/apache2/error.log | nc localhost 5170

結果。

Aug 27 13:55:34 myhost td-agent-bit[2114]: {"date":1598535916,"host":"127.0.0.1","user":"-","method":"GET","path":"/","code":"200","size":"11173","referer":"-","agent":"curl/7.68.0"}
Aug 27 13:55:34 myhost td-agent-bit[2114]: {"date":1598535918,"host":"127.0.0.1","user":"-","method":"GET","path":"/index.html","code":"200","size":"11173","referer":"-","agent":"curl/7.68.0"}
Aug 27 13:55:34 myhost td-agent-bit[2114]: {"date":1598535930,"host":"127.0.0.1","user":"-","method":"GET","path":"/index.html","code":"200","size":"11173","referer":"-","agent":"curl/7.68.0"}
Aug 27 13:55:34 myhost td-agent-bit[2114]: {"date":1598535932,"host":"127.0.0.1","user":"-","method":"GET","path":"/","code":"200","size":"11173","referer":"-","agent":"curl/7.68.0"}
Aug 27 13:55:34 myhost td-agent-bit[2114]: {"date":1598536534.745031,"time":"Aug 27 13:45:14.957795 2020","level":"mpm_event:notice","pid":"1863:tid 139821141662784","message":"AH00489: Apache/2.4.41 (Ubuntu) configured -- resuming normal operations"}
Aug 27 13:55:34 myhost td-agent-bit[2114]: {"date":1598536534.745031,"time":"Aug 27 13:45:14.957893 2020","level":"core:notice","pid":"1863:tid 139821141662784","message":"AH00094: Command line: '/usr/sbin/apache2'"}
Aug 27 13:55:34 myhost td-agent-bit[2114]: {"date":1598536534.745032,"time":"Aug 27 13:45:34.983343 2020","level":"mpm_event:notice","pid":"1863:tid 139821141662784","message":"AH00491: caught SIGTERM, shutting down"}
Aug 27 13:55:34 myhost td-agent-bit[2114]: {"date":1598536534.745032,"time":"Aug 27 13:45:35.032522 2020","level":"mpm_event:notice","pid":"1955:tid 139947430849600","message":"AH00489: Apache/2.4.41 (Ubuntu) configured -- resuming normal operations"}
Aug 27 13:55:34 myhost td-agent-bit[2114]: {"date":1598536534.745032,"time":"Aug 27 13:45:35.032606 2020","level":"core:notice","pid":"1955:tid 139947430849600","message":"AH00094: Command line: '/usr/sbin/apache2'"}
Aug 27 13:55:34 myhost td-agent-bit[2114]: {"date":1598536534.745033,"time":"Aug 27 13:54:54.859070 2020","level":"mpm_event:notice","pid":"1955:tid 139947430849600","message":"AH00491: caught SIGTERM, shutting down"}
Aug 27 13:55:34 myhost td-agent-bit[2114]: {"date":1598536534.745033,"time":"Aug 27 13:54:54.918733 2020","level":"mpm_event:notice","pid":"2048:tid 140052200950848","message":"AH00489: Apache/2.4.41 (Ubuntu) configured -- resuming normal operations"}
Aug 27 13:55:34 myhost td-agent-bit[2114]: {"date":1598536534.745034,"time":"Aug 27 13:54:54.918866 2020","level":"core:notice","pid":"2048:tid 140052200950848","message":"AH00094: Command line: '/usr/sbin/apache2'"}

アクセスログと

Aug 27 13:55:34 myhost td-agent-bit[2114]: {"date":1598535916,"host":"127.0.0.1","user":"-","method":"GET","path":"/","code":"200","size":"11173","referer":"-","agent":"curl/7.68.0"}
Aug 27 13:55:34 myhost td-agent-bit[2114]: {"date":1598535918,"host":"127.0.0.1","user":"-","method":"GET","path":"/index.html","code":"200","size":"11173","referer":"-","agent":"curl/7.68.0"}
Aug 27 13:55:34 myhost td-agent-bit[2114]: {"date":1598535930,"host":"127.0.0.1","user":"-","method":"GET","path":"/index.html","code":"200","size":"11173","referer":"-","agent":"curl/7.68.0"}
Aug 27 13:55:34 myhost td-agent-bit[2114]: {"date":1598535932,"host":"127.0.0.1","user":"-","method":"GET","path":"/","code":"200","size":"11173","referer":"-","agent":"curl/7.68.0"}

エラーログがそれぞれパースできていますね。

Aug 27 13:55:34 myhost td-agent-bit[2114]: {"date":1598536534.745031,"time":"Aug 27 13:45:14.957795 2020","level":"mpm_event:notice","pid":"1863:tid 139821141662784","message":"AH00489: Apache/2.4.41 (Ubuntu) configured -- resuming normal operations"}
Aug 27 13:55:34 myhost td-agent-bit[2114]: {"date":1598536534.745031,"time":"Aug 27 13:45:14.957893 2020","level":"core:notice","pid":"1863:tid 139821141662784","message":"AH00094: Command line: '/usr/sbin/apache2'"}
Aug 27 13:55:34 myhost td-agent-bit[2114]: {"date":1598536534.745032,"time":"Aug 27 13:45:34.983343 2020","level":"mpm_event:notice","pid":"1863:tid 139821141662784","message":"AH00491: caught SIGTERM, shutting down"}
Aug 27 13:55:34 myhost td-agent-bit[2114]: {"date":1598536534.745032,"time":"Aug 27 13:45:35.032522 2020","level":"mpm_event:notice","pid":"1955:tid 139947430849600","message":"AH00489: Apache/2.4.41 (Ubuntu) configured -- resuming normal operations"}
Aug 27 13:55:34 myhost td-agent-bit[2114]: {"date":1598536534.745032,"time":"Aug 27 13:45:35.032606 2020","level":"core:notice","pid":"1955:tid 139947430849600","message":"AH00094: Command line: '/usr/sbin/apache2'"}
Aug 27 13:55:34 myhost td-agent-bit[2114]: {"date":1598536534.745033,"time":"Aug 27 13:54:54.859070 2020","level":"mpm_event:notice","pid":"1955:tid 139947430849600","message":"AH00491: caught SIGTERM, shutting down"}
Aug 27 13:55:34 myhost td-agent-bit[2114]: {"date":1598536534.745033,"time":"Aug 27 13:54:54.918733 2020","level":"mpm_event:notice","pid":"2048:tid 140052200950848","message":"AH00489: Apache/2.4.41 (Ubuntu) configured -- resuming normal operations"}
Aug 27 13:55:34 myhost td-agent-bit[2114]: {"date":1598536534.745034,"time":"Aug 27 13:54:54.918866 2020","level":"core:notice","pid":"2048:tid 140052200950848","message":"AH00094: Command line: '/usr/sbin/apache2'"}

パーサーはどうやら前の方から順に適用してみて、パースできた時点で止まるようにソースコード上は見えます。

https://github.com/fluent/fluent-bit/blob/v1.5.4/plugins/filter_parser/filter_parser.c#L266-L301

確認してみましょう。

こんな感じで、なんにでもマッチするパーサーを「/etc/td-agent-bit/parsers.conf」に追加してみます。

[PARSER]
    Name    blackhole
    Format  regex
    Regex   (?<message>.*)

「blackhole」と名付けましょう。

Filterの定義にも反映。

[FILTER]
    Name     parser
    Match    apache.*
    Key_Name log
    Parser   blackhole
    Parser   apache2
    Parser   apache_error

再起動して

$ sudo systemctl restart td-agent-bit

確認。

$ cat /var/log/apache2/access.log /var/log/apache2/error.log | nc localhost 5170

すると、こんな感じでアクセスログもエラーログも、「blackhole」で定義したパーサーに取られてしまいます。

Aug 27 13:59:33 myhost td-agent-bit[2140]: {"date":1598536770.593083,"message":"127.0.0.1 - - [27/Aug/2020:13:45:16 +0000] \"GET / HTTP/1.1\" 200 11173 \"-\" \"curl/7.68.0\""}
Aug 27 13:59:33 myhost td-agent-bit[2140]: {"date":1598536770.593086,"message":"127.0.0.1 - - [27/Aug/2020:13:45:18 +0000] \"GET /index.html HTTP/1.1\" 200 11173 \"-\" \"curl/7.68.0\""}
Aug 27 13:59:33 myhost td-agent-bit[2140]: {"date":1598536770.593086,"message":"127.0.0.1 - - [27/Aug/2020:13:45:30 +0000] \"GET /index.html HTTP/1.1\" 200 11173 \"-\" \"curl/7.68.0\""}
Aug 27 13:59:33 myhost td-agent-bit[2140]: {"date":1598536770.593086,"message":"127.0.0.1 - - [27/Aug/2020:13:45:32 +0000] \"GET / HTTP/1.1\" 200 11173 \"-\" \"curl/7.68.0\""}
Aug 27 13:59:33 myhost td-agent-bit[2140]: {"date":1598536770.593087,"message":"[Thu Aug 27 13:45:14.957795 2020] [mpm_event:notice] [pid 1863:tid 139821141662784] AH00489: Apache/2.4.41 (Ubuntu) configured -- resuming normal operations"}
Aug 27 13:59:33 myhost td-agent-bit[2140]: {"date":1598536770.593087,"message":"[Thu Aug 27 13:45:14.957893 2020] [core:notice] [pid 1863:tid 139821141662784] AH00094: Command line: '/usr/sbin/apache2'"}
Aug 27 13:59:33 myhost td-agent-bit[2140]: {"date":1598536770.593088,"message":"[Thu Aug 27 13:45:34.983343 2020] [mpm_event:notice] [pid 1863:tid 139821141662784] AH00491: caught SIGTERM, shutting down"}
Aug 27 13:59:33 myhost td-agent-bit[2140]: {"date":1598536770.593088,"message":"[Thu Aug 27 13:45:35.032522 2020] [mpm_event:notice] [pid 1955:tid 139947430849600] AH00489: Apache/2.4.41 (Ubuntu) configured -- resuming normal operations"}
Aug 27 13:59:33 myhost td-agent-bit[2140]: {"date":1598536770.593088,"message":"[Thu Aug 27 13:45:35.032606 2020] [core:notice] [pid 1955:tid 139947430849600] AH00094: Command line: '/usr/sbin/apache2'"}
Aug 27 13:59:33 myhost td-agent-bit[2140]: {"date":1598536770.593088,"message":"[Thu Aug 27 13:54:54.859070 2020] [mpm_event:notice] [pid 1955:tid 139947430849600] AH00491: caught SIGTERM, shutting down"}
Aug 27 13:59:33 myhost td-agent-bit[2140]: {"date":1598536770.593089,"message":"[Thu Aug 27 13:54:54.918733 2020] [mpm_event:notice] [pid 2048:tid 140052200950848] AH00489: Apache/2.4.41 (Ubuntu) configured -- resuming normal operations"}
Aug 27 13:59:33 myhost td-agent-bit[2140]: {"date":1598536770.593089,"message":"[Thu Aug 27 13:54:54.918866 2020] [core:notice] [pid 2048:tid 140052200950848] AH00094: Command line: '/usr/sbin/apache2'"}

というわけで、複数のパーサーを使う時は、その適用順に気をつけましょうね、と。

どのパーサーが適用された?

複数のパーサーを使用した場合、どのパーサーを使ってパースされたのか(=どのログフォーマットだったのか)知りたくなる
気がします。

パーサーの定義で、マッチしたらタグを追加…みたいなことはできなさそうです。

https://github.com/fluent/fluent-bit/blob/v1.5.4/src/flb_parser.c#L431-L440

フィールドの存在有無で、キーを追加する、とかしたらいいんでしょうか?今回はModify Filterを使い、「pid」フィールドが
あれば「log_type / error」を、「path」フィールドがあれば「log_type / access」を追加してみることにしました。

Modify - Fluent Bit: Official Manual

こんな感じで。

[FILTER]
    Name     parser
    Match    apache.*
    Key_Name log
    Parser   apache2
    Parser   apache_error


[FILTER]
    Name      modify
    Match     apache.mixed.log
    Condition Key_exists pid
    Add       log_type error


[FILTER]
    Name      modify
    Match     apache.mixed.log
    Condition Key_exists path
    Add       log_type access

確認。

$ cat /var/log/apache2/access.log /var/log/apache2/error.log | nc localhost 5170

OKですね。

Aug 27 14:25:46 myhost td-agent-bit[2162]: {"date":1598535916,"host":"127.0.0.1","user":"-","method":"GET","path":"/","code":"200","size":"11173","referer":"-","agent":"curl/7.68.0","log_type":"access"}
Aug 27 14:25:46 myhost td-agent-bit[2162]: {"date":1598535918,"host":"127.0.0.1","user":"-","method":"GET","path":"/index.html","code":"200","size":"11173","referer":"-","agent":"curl/7.68.0","log_type":"access"}
Aug 27 14:25:46 myhost td-agent-bit[2162]: {"date":1598535930,"host":"127.0.0.1","user":"-","method":"GET","path":"/index.html","code":"200","size":"11173","referer":"-","agent":"curl/7.68.0","log_type":"access"}
Aug 27 14:25:46 myhost td-agent-bit[2162]: {"date":1598535932,"host":"127.0.0.1","user":"-","method":"GET","path":"/","code":"200","size":"11173","referer":"-","agent":"curl/7.68.0","log_type":"access"}
Aug 27 14:25:46 myhost td-agent-bit[2162]: {"date":1598538343.940817,"time":"Aug 27 13:45:14.957795 2020","level":"mpm_event:notice","pid":"1863:tid 139821141662784","message":"AH00489: Apache/2.4.41 (Ubuntu) configured -- resuming normal operations","log_type":"error"}
Aug 27 14:25:46 myhost td-agent-bit[2162]: {"date":1598538343.940817,"time":"Aug 27 13:45:14.957893 2020","level":"core:notice","pid":"1863:tid 139821141662784","message":"AH00094: Command line: '/usr/sbin/apache2'","log_type":"error"}
Aug 27 14:25:46 myhost td-agent-bit[2162]: {"date":1598538343.940817,"time":"Aug 27 13:45:34.983343 2020","level":"mpm_event:notice","pid":"1863:tid 139821141662784","message":"AH00491: caught SIGTERM, shutting down","log_type":"error"}
Aug 27 14:25:46 myhost td-agent-bit[2162]: {"date":1598538343.940818,"time":"Aug 27 13:45:35.032522 2020","level":"mpm_event:notice","pid":"1955:tid 139947430849600","message":"AH00489: Apache/2.4.41 (Ubuntu) configured -- resuming normal operations","log_type":"error"}
Aug 27 14:25:46 myhost td-agent-bit[2162]: {"date":1598538343.940818,"time":"Aug 27 13:45:35.032606 2020","level":"core:notice","pid":"1955:tid 139947430849600","message":"AH00094: Command line: '/usr/sbin/apache2'","log_type":"error"}
Aug 27 14:25:46 myhost td-agent-bit[2162]: {"date":1598538343.940818,"time":"Aug 27 13:54:54.859070 2020","level":"mpm_event:notice","pid":"1955:tid 139947430849600","message":"AH00491: caught SIGTERM, shutting down","log_type":"error"}
Aug 27 14:25:46 myhost td-agent-bit[2162]: {"date":1598538343.940819,"time":"Aug 27 13:54:54.918733 2020","level":"mpm_event:notice","pid":"2048:tid 140052200950848","message":"AH00489: Apache/2.4.41 (Ubuntu) configured -- resuming normal operations","log_type":"error"}
Aug 27 14:25:46 myhost td-agent-bit[2162]: {"date":1598538343.940819,"time":"Aug 27 13:54:54.918866 2020","level":"core:notice","pid":"2048:tid 140052200950848","message":"AH00094: Command line: '/usr/sbin/apache2'","log_type":"error"}

Rewrite Tag Filterを試してみたくもなったのですが、タグに関してはもう少しちゃんと理解した方がよさそうなので、また今度。

Rewrite Tag - Fluent Bit: Official Manual

Router - Fluent Bit: Official Manual