CLOVER🍀

That was when it all began.

Quarkus × Micrometerでメトリクスを取得する

Quarkus Prometheus

これは、なにをしたくて書いたもの？

Quarkusでメトリクスを取得する方法は？と思ってドキュメントを見てみたのですが、Micrometerを使うようです。

ちょっと試してみます。

Quarkus Micrometer Extension

Quarkusのメトリクスに関するドキュメントを見ると、以下の2つのExtensionが見つかります。

SmallRye Metricsの方は、非推奨のようです。

Micrometer is the recommended approach to metrics for Quarkus. Use the SmallRye Metrics extension when it is required to retain MicroProfile specification compatibility.

When Quarkus will upgrade to Eclipse MicroProfile 6, the SmallRye Metrics support will be discontinued.

This technology is considered deprecated.

Being deprecated means that this extension is likely to be replaced or removed in a future version of Quarkus.

Smallrye Metrics - Quarkus

チュートリアルの方も、Micrometerを使って書かれています。

Collect metrics using Micrometer - Quarkus

SmallRye Metricsが非推奨になっているのは、こちらに書かれていることが理由な気がしますね。

On the Metrics front, it became evident that while MicroProfile Metrics and SmallRye Metrics were acceptable for simple use cases, they had some issues with more advanced ones. It also didn’t help that users from different technologies had to learn new APIs and constructs to extract metrics information from their applications.

Quarkus Observability Roadmap 2023 / Metrics and Micrometer

Quarkusとしては、今後新しいEclipse MicroProfile Metric仕様をサポートする予定はないみたいです。

というわけで、今回はQuarkusのMicrometer Extensionを見ていきたいと思います。

Micrometer Extensionは、MicrometerをQuarkus内で使えるようにしたものみたいですね。

Micrometer Application Observability

このExtensionを追加すると、Micrometerが追加されQuarkusのメトリクスが取得できるようになったり、自分でメトリクスを定義できたり
します。

またMicrometerにはRegistryの実装が必要ですが、Quarkusが提供しているのはPrometheus向けのものですね。

https://github.com/quarkusio/quarkus/tree/2.16.6.Final/extensions/micrometer-registry-prometheus

それ以外のRegistryは、Quarkiverseで探すことになります。現在のQuarkusのバージョンとはちょっと差がありそうですが。

GitHub - quarkiverse/quarkus-micrometer-registry: Quarkus extensions that pull together required/related dependencies for optional micrometer registries.

とりあえず、今回はドキュメント通りPrometheusのRegistryを使ってみましょう。

環境

今回の環境は、こちら。

$ java --version
openjdk 17.0.6 2023-01-17
OpenJDK Runtime Environment (build 17.0.6+10-Ubuntu-0ubuntu122.04)
OpenJDK 64-Bit Server VM (build 17.0.6+10-Ubuntu-0ubuntu122.04, mixed mode, sharing)


$ mvn --version
Apache Maven 3.8.8 (4c87b05d9aedce574290d1acc98575ed5eb6cd39)
Maven home: $HOME/.sdkman/candidates/maven/current
Java version: 17.0.6, vendor: Private Build, runtime: /usr/lib/jvm/java-17-openjdk-amd64
Default locale: ja_JP, platform encoding: UTF-8
OS name: "linux", version: "5.15.0-69-generic", arch: "amd64", family: "unix"

Prometheusは、172.17.0.2で動作しているものとします。

$ ./prometheus --version
prometheus, version 2.43.0 (branch: HEAD, revision: edfc3bcd025dd6fe296c167a14a216cab1e552ee)
  build user:       root@8a0ee342e522
  build date:       20230321-12:56:07
  go version:       go1.19.7
  platform:         linux/amd64
  tags:             netgo,builtinassets

Quarkusアプリケーションを作成する

まずはQuarkusアプリケーションを作成します。QuarkusのMicrometer Registry Prometheus Extensionを追加して、あとは
RESTEasy Reactive Extensionを加えてプロジェクトを作成。

$ mvn io.quarkus.platform:quarkus-maven-plugin:2.16.6.Final:create \
    -DprojectGroupId=org.littlewings \
    -DprojectArtifactId=micrometer-metrics \
    -DprojectVersion=0.0.1-SNAPSHOT \
    -Dextensions='resteasy-reactive,resteasy-reactive-jackson,micrometer-registry-prometheus' \
    -DnoCode

選択されたExtensionなど。

[INFO] -----------
[INFO] selected extensions:
- io.quarkus:quarkus-resteasy-reactive
- io.quarkus:quarkus-resteasy-reactive-jackson
- io.quarkus:quarkus-micrometer-registry-prometheus

[INFO]
applying codestarts...
[INFO] 📚  java
🔨  maven
📦  quarkus
📝  config-properties
🔧  dockerfiles
🔧  maven-wrapper

ディレクトリ内に移動。

$ cd micrometer-metrics

Maven依存関係など。

  <properties>
    <compiler-plugin.version>3.10.1</compiler-plugin.version>
    <maven.compiler.release>17</maven.compiler.release>
    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    <project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
    <quarkus.platform.artifact-id>quarkus-bom</quarkus.platform.artifact-id>
    <quarkus.platform.group-id>io.quarkus.platform</quarkus.platform.group-id>
    <quarkus.platform.version>2.16.6.Final</quarkus.platform.version>
    <skipITs>true</skipITs>
    <surefire-plugin.version>3.0.0-M7</surefire-plugin.version>
  </properties>
  <dependencyManagement>
    <dependencies>
      <dependency>
        <groupId>${quarkus.platform.group-id}</groupId>
        <artifactId>${quarkus.platform.artifact-id}</artifactId>
        <version>${quarkus.platform.version}</version>
        <type>pom</type>
        <scope>import</scope>
      </dependency>
    </dependencies>
  </dependencyManagement>
  <dependencies>
    <dependency>
      <groupId>io.quarkus</groupId>
      <artifactId>quarkus-resteasy-reactive</artifactId>
    </dependency>
    <dependency>
      <groupId>io.quarkus</groupId>
      <artifactId>quarkus-resteasy-reactive-jackson</artifactId>
    </dependency>
    <dependency>
      <groupId>io.quarkus</groupId>
      <artifactId>quarkus-micrometer-registry-prometheus</artifactId>
    </dependency>
    <dependency>
      <groupId>io.quarkus</groupId>
      <artifactId>quarkus-arc</artifactId>
    </dependency>
    <dependency>
      <groupId>io.quarkus</groupId>
      <artifactId>quarkus-junit5</artifactId>
      <scope>test</scope>
    </dependency>
  </dependencies>

ちなみに、この状態でアプリケーションを起動すると

$ mvn compile quarkus:dev

すでにメトリクスが取得できるようになっています。デフォルトのパスは、/q/metricsですね。

$ curl -s localhost:8080/q/metrics

起動した段階では、88個のメトリクスが取得できました。

$ curl -s localhost:8080/q/metrics | grep -v '#' | wc -l
88

ラベルを切り取って見てみると、こういう種類のメトリクスがあるようです。また、アプリケーションにアクセスすると取得できる
メトリクスは増えていきます。

$ curl -s localhost:8080/q/metrics | grep -v '#' | perl -wp -e 's!(.+?)([{ ].+)!$1!' | sort -u
http_server_active_requests
http_server_bytes_read_count
http_server_bytes_read_max
http_server_bytes_read_sum
http_server_bytes_written_count
http_server_bytes_written_max
http_server_bytes_written_sum
http_server_connections_seconds_active_count
http_server_connections_seconds_duration_sum
http_server_connections_seconds_max
jvm_buffer_count_buffers
jvm_buffer_memory_used_bytes
jvm_buffer_total_capacity_bytes
jvm_classes_loaded_classes
jvm_classes_unloaded_classes_total
jvm_gc_live_data_size_bytes
jvm_gc_max_data_size_bytes
jvm_gc_memory_allocated_bytes_total
jvm_gc_memory_promoted_bytes_total
jvm_gc_overhead_percent
jvm_gc_pause_seconds_count
jvm_gc_pause_seconds_max
jvm_gc_pause_seconds_sum
jvm_info_total
jvm_memory_committed_bytes
jvm_memory_max_bytes
jvm_memory_usage_after_gc_percent
jvm_memory_used_bytes
jvm_threads_daemon_threads
jvm_threads_live_threads
jvm_threads_peak_threads
jvm_threads_states_threads
process_cpu_usage
process_files_max_files
process_files_open_files
process_start_time_seconds
process_uptime_seconds
system_cpu_count
system_cpu_usage
system_load_average_1m
worker_pool_active
worker_pool_completed_total
worker_pool_idle
worker_pool_queue_delay_seconds_count
worker_pool_queue_delay_seconds_max
worker_pool_queue_delay_seconds_sum
worker_pool_queue_size
worker_pool_ratio
worker_pool_usage_seconds_count
worker_pool_usage_seconds_max
worker_pool_usage_seconds_sum

では、ソースコードを追加していきましょう。書籍をお題にすることにします。

エンティティ相当のクラス。

src/main/java/org/littlewings/quarkus/micrometer/Book.java

package org.littlewings.quarkus.micrometer;

public class Book {
    String isbn;
    String title;
    int price;

    // getter／setterは省略
}

RESTEasy Reactiveを使ったJAX-RSリソースクラスを作成。

src/main/java/org/littlewings/quarkus/micrometer/BooksResource.java

package org.littlewings.quarkus.micrometer;

import io.smallrye.mutiny.Multi;
import io.smallrye.mutiny.Uni;
import org.jboss.resteasy.reactive.RestPath;
import org.jboss.resteasy.reactive.RestResponse;

import javax.ws.rs.*;
import javax.ws.rs.core.Context;
import javax.ws.rs.core.MediaType;
import javax.ws.rs.core.UriBuilder;
import javax.ws.rs.core.UriInfo;
import java.util.Comparator;
import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.ConcurrentMap;

@Path("books")
public class BooksResource {
    ConcurrentMap<String, Book> books = new ConcurrentHashMap<>();

    @GET
    @Path("{isbn}")
    @Produces(MediaType.APPLICATION_JSON)
    public Uni<Book> findByIsbn(@RestPath String isbn) {
        return Uni.createFrom().item(books.get(isbn));
    }

    @GET
    @Produces(MediaType.APPLICATION_JSON)
    public Multi<Book> findAll() {
        return Multi
                .createFrom()
                .items(books.values().stream().sorted(Comparator.comparing(Book::getPrice).reversed()));
    }

    @POST
    @Consumes(MediaType.APPLICATION_JSON)
    public Uni<RestResponse<Void>> create(@Context UriInfo uriInfo, Book book) {
        books.put(book.getIsbn(), book);

        return Uni
                .createFrom()
                .item(RestResponse.created(UriBuilder.fromUri(uriInfo.getPath()).path(book.getIsbn()).build()));
    }

    @DELETE
    @Path("{isbn}")
    public Uni<RestResponse<Void>> delete(@RestPath String isbn) {
        books.remove(isbn);

        return Uni
                .createFrom()
                .item(RestResponse.noContent());
    }
}

作成したREST APIにアクセスしてみます。

データの登録。

$ curl -i -XPOST -H 'Content-Type: application/json' localhost:8080/books -d '{"isbn": "978-4621303252", "title": "Effective Java 第3版", "price": 4400}'
HTTP/1.1 201 Created
Location: http://localhost:8080/books/978-4621303252
content-length: 0


$ curl -i -XPOST -H 'Content-Type: application/json' localhost:8080/books -d '{"isbn": "978-1782169970", "title": "Infinispan Data Grid Platform Definitive Guide", "price": 5242}'
HTTP/1.1 201 Created
Location: http://localhost:8080/books/978-1782169970
content-length: 0


$ curl -i -XPOST -H 'Content-Type: application/json' localhost:8080/books -d '{"isbn": "978-4798161488", "title": "MySQL徹底入門 第4版 MySQL 8.0対応", "price": 4180}'
HTTP/1.1 201 Created
Location: http://localhost:8080/books/978-4798161488
content-length: 0


$ curl -i -XPOST -H 'Content-Type: application/json' localhost:8080/books -d '{"isbn": "978-1098116743", "title": "Terraform: Up & Running; Writing Infrastructure As Code", "price": 7404}'
HTTP/1.1 201 Created
Location: http://localhost:8080/books/978-1098116743
content-length: 0

データの全件取得。

$ curl -i localhost:8080/books
HTTP/1.1 200 OK
Content-Type: application/json;charset=UTF-8
transfer-encoding: chunked

[{"isbn":"978-1098116743","title":"Terraform: Up & Running; Writing Infrastructure As Code","price":7404},{"isbn":"978-1782169970","title":"Infinispan Data Grid Platform Definitive Guide","price":5242},{"isbn":"978-4621303252","title":"Effective Java 第3版","price":4400},{"isbn":"978-4798161488","title":"MySQL徹底入門 第4版 MySQL 8.0対応","price":4180}]


$ curl -s localhost:8080/books | jq
[
  {
    "isbn": "978-1098116743",
    "title": "Terraform: Up & Running; Writing Infrastructure As Code",
    "price": 7404
  },
  {
    "isbn": "978-1782169970",
    "title": "Infinispan Data Grid Platform Definitive Guide",
    "price": 5242
  },
  {
    "isbn": "978-4621303252",
    "title": "Effective Java 第3版",
    "price": 4400
  },
  {
    "isbn": "978-4798161488",
    "title": "MySQL徹底入門 第4版 MySQL 8.0対応",
    "price": 4180
  }
]

データの1件取得

$ curl -i localhost:8080/books/978-4621303252
HTTP/1.1 200 OK
content-length: 71
Content-Type: application/json;charset=UTF-8

{"isbn":"978-4621303252","title":"Effective Java 第3版","price":4400}


$ curl -i localhost:8080/books/978-4798161488
HTTP/1.1 200 OK
content-length: 90
Content-Type: application/json;charset=UTF-8

{"isbn":"978-4798161488","title":"MySQL徹底入門 第4版 MySQL 8.0対応","price":4180}

データの削除。

$ curl -i -XDELETE localhost:8080/books/978-4798161488
HTTP/1.1 204 No Content

削除確認。

$ curl -i localhost:8080/books/978-4798161488
Content-Type: application/json;charset=UTF-8

この時点でのメトリクスを見てみます。HTTPサーバーに関するメトリクスは、http_serverで始まるものを見れば良さそうです。

# TYPE http_server_requests_seconds summary
# HELP http_server_requests_seconds
http_server_requests_seconds_count{method="DELETE",outcome="SUCCESS",status="204",uri="/books/{isbn}"} 1.0
http_server_requests_seconds_sum{method="DELETE",outcome="SUCCESS",status="204",uri="/books/{isbn}"} 9.49418E-4
http_server_requests_seconds_count{method="POST",outcome="SUCCESS",status="201",uri="/books"} 4.0
http_server_requests_seconds_sum{method="POST",outcome="SUCCESS",status="201",uri="/books"} 0.087783214
http_server_requests_seconds_count{method="GET",outcome="SUCCESS",status="200",uri="/books/{isbn}"} 2.0
http_server_requests_seconds_sum{method="GET",outcome="SUCCESS",status="200",uri="/books/{isbn}"} 0.005722725
http_server_requests_seconds_count{method="GET",outcome="SUCCESS",status="200",uri="/books"} 2.0
http_server_requests_seconds_sum{method="GET",outcome="SUCCESS",status="200",uri="/books"} 0.025433533
http_server_requests_seconds_count{method="GET",outcome="SUCCESS",status="204",uri="/books/{isbn}"} 1.0
http_server_requests_seconds_sum{method="GET",outcome="SUCCESS",status="204",uri="/books/{isbn}"} 7.89022E-4
# TYPE http_server_requests_seconds_max gauge
# HELP http_server_requests_seconds_max
http_server_requests_seconds_max{method="DELETE",outcome="SUCCESS",status="204",uri="/books/{isbn}"} 9.49418E-4
http_server_requests_seconds_max{method="POST",outcome="SUCCESS",status="201",uri="/books"} 0.083832168
http_server_requests_seconds_max{method="GET",outcome="SUCCESS",status="200",uri="/books/{isbn}"} 0.004878914
http_server_requests_seconds_max{method="GET",outcome="SUCCESS",status="200",uri="/books"} 0.021108196
http_server_requests_seconds_max{method="GET",outcome="SUCCESS",status="204",uri="/books/{isbn}"} 7.89022E-4
# TYPE http_server_bytes_read summary
# HELP http_server_bytes_read Number of bytes received by the server
http_server_bytes_read_count 4.0
http_server_bytes_read_sum 380.0
# TYPE http_server_bytes_read_max gauge
# HELP http_server_bytes_read_max Number of bytes received by the server
http_server_bytes_read_max 109.0
# TYPE http_server_active_requests gauge
# HELP http_server_active_requests
http_server_active_requests 1.0
# TYPE http_server_bytes_written_max gauge
# HELP http_server_bytes_written_max Number of bytes sent by the server
http_server_bytes_written_max 8192.0
# TYPE http_server_bytes_written summary
# HELP http_server_bytes_written Number of bytes sent by the server
http_server_bytes_written_count 46.0
http_server_bytes_written_sum 228701.0
# TYPE http_server_connections_seconds_max gauge
# HELP http_server_connections_seconds_max The duration of the connections
http_server_connections_seconds_max 104.773689224
# TYPE http_server_connections_seconds summary
# HELP http_server_connections_seconds The duration of the connections
http_server_connections_seconds_active_count 2.0
http_server_connections_seconds_duration_sum 104.779142373

パスをパラメーター化した場合は、そちらも反映されるんですね。

http_server_requests_seconds_count{method="GET",outcome="SUCCESS",status="200",uri="/books/{isbn}"} 2.0

独自のメトリクスを定義する

次は、自分でもメトリクスを定義してみましょう。

これは、Quarkusのドキュメントにも一部は書かれていますが

詳細な使い方は、基本的にはMicrometerのドキュメントを見ることになります。

Concepts

今回は、先ほどのJAX-RSリソースクラスをベースにして、こんなクラスを作成しました。

src/main/java/org/littlewings/quarkus/micrometer/CustomMetricsBooksResource.java

package org.littlewings.quarkus.micrometer;

import io.micrometer.core.annotation.Timed;
import io.micrometer.core.instrument.MeterRegistry;
import io.micrometer.core.instrument.Tags;
import io.micrometer.core.instrument.Timer;
import io.smallrye.mutiny.Multi;
import io.smallrye.mutiny.Uni;
import org.jboss.resteasy.reactive.RestPath;
import org.jboss.resteasy.reactive.RestResponse;

import javax.annotation.PostConstruct;
import javax.inject.Inject;
import javax.ws.rs.*;
import javax.ws.rs.core.Context;
import javax.ws.rs.core.MediaType;
import javax.ws.rs.core.UriBuilder;
import javax.ws.rs.core.UriInfo;
import java.time.Duration;
import java.util.Comparator;
import java.util.Random;
import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.ConcurrentMap;

@Path("custom-metrics/books")
@Timed
public class CustomMetricsBooksResource {
    @Inject
    MeterRegistry meterRegistry;
    Timer bookRegisterTimer;

    ConcurrentMap<String, Book> books;

    Random random;

    @PostConstruct
    void init() {
        books = meterRegistry.gaugeMapSize("custom.store.books.size", Tags.empty(), new ConcurrentHashMap<>());
        bookRegisterTimer =
                Timer
                        .builder("custom.store.books.register.seconds")
                        .register(meterRegistry);

        random = new Random();
        random.nextInt();
    }

    @GET
    @Path("{isbn}")
    @Produces(MediaType.APPLICATION_JSON)
    @Timed
    public Uni<Book> findByIsbn(@RestPath String isbn) {
        return Uni.createFrom().item(books.get(isbn));
    }

    @GET
    @Produces(MediaType.APPLICATION_JSON)
    @Timed
    public Multi<Book> findAll() {
        return Multi
                .createFrom()
                .items(books.values().stream().sorted(Comparator.comparing(Book::getPrice).reversed()));
    }

    @POST
    @Consumes(MediaType.APPLICATION_JSON)
    public Uni<RestResponse<Void>> create(@Context UriInfo uriInfo, Book book) {
        return bookRegisterTimer.record(() -> {
            books.put(book.getIsbn(), book);

            return Uni
                            .createFrom()
                            .<RestResponse<Void>>item(RestResponse.created(UriBuilder.fromUri(uriInfo.getPath()).path(book.getIsbn()).build()))
                            .onItem()
                            .delayIt()
                            .by(Duration.ofSeconds(random.nextLong(4) + 1));
        });
    }

    @DELETE
    @Path("{isbn}")
    public Uni<RestResponse<Void>> delete(@RestPath String isbn) {
        books.remove(isbn);

        return Uni
                .createFrom()
                .item(RestResponse.noContent());
    }
}

Gauseとして、Mapのサイズをメトリクスにしてみました。

        books = meterRegistry.gaugeMapSize("custom.store.books.size", Tags.empty(), new ConcurrentHashMap<>());

Micrometer / Concepts / Gauges

また、Timerを定義して

        bookRegisterTimer =
                Timer
                        .builder("custom.store.books.register.seconds")
                        .register(meterRegistry);

データの登録時にランダムにスリープするようにしてレスポンス時間を記録。
※Quarkusに記録させることもできますが、あまり気にせず

    @POST
    @Consumes(MediaType.APPLICATION_JSON)
    public Uni<RestResponse<Void>> create(@Context UriInfo uriInfo, Book book) {
        return bookRegisterTimer.record(() -> {
            books.put(book.getIsbn(), book);

            return Uni
                            .createFrom()
                            .<RestResponse<Void>>item(RestResponse.created(UriBuilder.fromUri(uriInfo.getPath()).path(book.getIsbn()).build()))
                            .onItem()
                            .delayIt()
                            .by(Duration.ofSeconds(random.nextLong(4) + 1));
        });
    }

Micrometer / Concepts / Timers

+1しているのは、0になるのを回避するためです。

また、@Timerアノテーションをクラスとメソッドに付与しています。

@Path("custom-metrics/books")
@Timed
public class CustomMetricsBooksResource {


    @GET
    @Path("{isbn}")
    @Produces(MediaType.APPLICATION_JSON)
    @Timed
    public Uni<Book> findByIsbn(@RestPath String isbn) {
        return Uni.createFrom().item(books.get(isbn));
    }

    @GET
    @Produces(MediaType.APPLICATION_JSON)
    @Timed
    public Multi<Book> findAll() {
        return Multi
                .createFrom()
                .items(books.values().stream().sorted(Comparator.comparing(Book::getPrice).reversed()));
    }

@Timerアノテーションを使うと、クラスに付与した場合はクラスのメソッドに対して、メソッドに付与した場合はそのメソッドの
処理時間を記録できます。

なお、Quarkusでのアノテーションの利用には、注意があります。

Micrometer Metrics / Does Micrometer support annotations?

では、先ほどと同じリクエストを、新しいJAX-RSリソースクラスに向けて実行。

$ curl -i -XPOST -H 'Content-Type: application/json' localhost:8080/custom-metrics/books -d '{"isbn": "978-4621303252", "title": "Effective Java 第3版", "price": 4400}'
$ curl -i -XPOST -H 'Content-Type: application/json' localhost:8080/custom-metrics/books -d '{"isbn": "978-1782169970", "title": "Infinispan Data Grid Platform Definitive Guide", "price": 5242}'
$ curl -i -XPOST -H 'Content-Type: application/json' localhost:8080/custom-metrics/books -d '{"isbn": "978-4798161488", "title": "MySQL徹底入門 第4版 MySQL 8.0対応", "price": 4180}'
$ curl -i -XPOST -H 'Content-Type: application/json' localhost:8080/custom-metrics/books -d '{"isbn": "978-1098116743", "title": "Terraform: Up & Running; Writing Infrastructure As Code", "price": 7404}'
$ curl -i localhost:8080/custom-metrics/books
$ curl -s localhost:8080/custom-metrics/books | jq
$ curl -i localhost:8080/custom-metrics/books/978-4621303252
$ curl -i localhost:8080/custom-metrics/books/978-4798161488
$ curl -i -XDELETE localhost:8080/custom-metrics/books/978-4798161488
$ curl -i localhost:8080/custom-metrics/books/978-4798161488

確認。

$ curl -s localhost:8080/q/metrics | grep -E 'http_server|custom|method'

こんな感じになりました。

# TYPE http_server_connections_seconds_max gauge
# HELP http_server_connections_seconds_max The duration of the connections
http_server_connections_seconds_max 76.551193111
# TYPE http_server_connections_seconds summary
# HELP http_server_connections_seconds The duration of the connections
http_server_connections_seconds_active_count 2.0
http_server_connections_seconds_duration_sum 76.553448684
# TYPE http_server_bytes_written_max gauge
# HELP http_server_bytes_written_max Number of bytes sent by the server
http_server_bytes_written_max 16384.0
# TYPE http_server_bytes_written summary
# HELP http_server_bytes_written Number of bytes sent by the server
http_server_bytes_written_count 42.0
http_server_bytes_written_sum 270754.0
# TYPE http_server_requests_seconds summary
# HELP http_server_requests_seconds
http_server_requests_seconds_count{method="POST",outcome="SUCCESS",status="201",uri="/custom-metrics/books"} 4.0
http_server_requests_seconds_sum{method="POST",outcome="SUCCESS",status="201",uri="/custom-metrics/books"} 10.025620068
http_server_requests_seconds_count{method="DELETE",outcome="SUCCESS",status="204",uri="/books/{isbn}"} 1.0
http_server_requests_seconds_sum{method="DELETE",outcome="SUCCESS",status="204",uri="/books/{isbn}"} 9.59999E-4
http_server_requests_seconds_count{method="POST",outcome="SUCCESS",status="201",uri="/books"} 4.0
http_server_requests_seconds_sum{method="POST",outcome="SUCCESS",status="201",uri="/books"} 0.090636946
http_server_requests_seconds_count{method="DELETE",outcome="SUCCESS",status="204",uri="/custom-metrics/books/{isbn}"} 1.0
http_server_requests_seconds_sum{method="DELETE",outcome="SUCCESS",status="204",uri="/custom-metrics/books/{isbn}"} 0.001569781
http_server_requests_seconds_count{method="GET",outcome="SUCCESS",status="200",uri="/custom-metrics/books"} 2.0
http_server_requests_seconds_sum{method="GET",outcome="SUCCESS",status="200",uri="/custom-metrics/books"} 0.011007882
http_server_requests_seconds_count{method="GET",outcome="SUCCESS",status="200",uri="/custom-metrics/books/{isbn}"} 2.0
http_server_requests_seconds_sum{method="GET",outcome="SUCCESS",status="200",uri="/custom-metrics/books/{isbn}"} 0.005169301
http_server_requests_seconds_count{method="GET",outcome="SUCCESS",status="200",uri="/books/{isbn}"} 2.0
http_server_requests_seconds_sum{method="GET",outcome="SUCCESS",status="200",uri="/books/{isbn}"} 0.00598501
http_server_requests_seconds_count{method="GET",outcome="SUCCESS",status="200",uri="/books"} 2.0
http_server_requests_seconds_sum{method="GET",outcome="SUCCESS",status="200",uri="/books"} 0.024324265
http_server_requests_seconds_count{method="GET",outcome="SUCCESS",status="204",uri="/custom-metrics/books/{isbn}"} 1.0
http_server_requests_seconds_sum{method="GET",outcome="SUCCESS",status="204",uri="/custom-metrics/books/{isbn}"} 0.001087167
http_server_requests_seconds_count{method="GET",outcome="SUCCESS",status="204",uri="/books/{isbn}"} 1.0
http_server_requests_seconds_sum{method="GET",outcome="SUCCESS",status="204",uri="/books/{isbn}"} 9.33564E-4
# TYPE http_server_requests_seconds_max gauge
# HELP http_server_requests_seconds_max
http_server_requests_seconds_max{method="POST",outcome="SUCCESS",status="201",uri="/custom-metrics/books"} 4.004975785
http_server_requests_seconds_max{method="DELETE",outcome="SUCCESS",status="204",uri="/books/{isbn}"} 9.59999E-4
http_server_requests_seconds_max{method="POST",outcome="SUCCESS",status="201",uri="/books"} 0.086196978
http_server_requests_seconds_max{method="DELETE",outcome="SUCCESS",status="204",uri="/custom-metrics/books/{isbn}"} 0.001569781
http_server_requests_seconds_max{method="GET",outcome="SUCCESS",status="200",uri="/custom-metrics/books"} 0.006242274
http_server_requests_seconds_max{method="GET",outcome="SUCCESS",status="200",uri="/custom-metrics/books/{isbn}"} 0.004218526
http_server_requests_seconds_max{method="GET",outcome="SUCCESS",status="200",uri="/books/{isbn}"} 0.005034318
http_server_requests_seconds_max{method="GET",outcome="SUCCESS",status="200",uri="/books"} 0.021170021
http_server_requests_seconds_max{method="GET",outcome="SUCCESS",status="204",uri="/custom-metrics/books/{isbn}"} 0.001087167
http_server_requests_seconds_max{method="GET",outcome="SUCCESS",status="204",uri="/books/{isbn}"} 9.33564E-4
# TYPE http_server_active_requests gauge
# HELP http_server_active_requests
http_server_active_requests 1.0
# TYPE custom_store_books_register_seconds summary
# HELP custom_store_books_register_seconds
custom_store_books_register_seconds_count 4.0
custom_store_books_register_seconds_sum 0.001986586
# TYPE custom_store_books_register_seconds_max gauge
# HELP custom_store_books_register_seconds_max
custom_store_books_register_seconds_max 0.00142105
# TYPE custom_store_books_size gauge
# HELP custom_store_books_size
custom_store_books_size 3.0
# TYPE http_server_bytes_read summary
# HELP http_server_bytes_read Number of bytes received by the server
http_server_bytes_read_count 8.0
http_server_bytes_read_sum 760.0
# TYPE http_server_bytes_read_max gauge
# HELP http_server_bytes_read_max Number of bytes received by the server
http_server_bytes_read_max 109.0
# TYPE method_timed_seconds summary
# HELP method_timed_seconds
method_timed_seconds_count{class="org.littlewings.quarkus.micrometer.CustomMetricsBooksResource",exception="none",method="create"} 4.0
method_timed_seconds_sum{class="org.littlewings.quarkus.micrometer.CustomMetricsBooksResource",exception="none",method="create"} 10.008363919
method_timed_seconds_count{class="org.littlewings.quarkus.micrometer.CustomMetricsBooksResource",exception="none",method="delete"} 1.0
method_timed_seconds_sum{class="org.littlewings.quarkus.micrometer.CustomMetricsBooksResource",exception="none",method="delete"} 4.14856E-4
method_timed_seconds_count{class="org.littlewings.quarkus.micrometer.CustomMetricsBooksResource",exception="none",method="findAll"} 2.0
method_timed_seconds_sum{class="org.littlewings.quarkus.micrometer.CustomMetricsBooksResource",exception="none",method="findAll"} 8.46507E-4
method_timed_seconds_count{class="org.littlewings.quarkus.micrometer.CustomMetricsBooksResource",exception="none",method="findByIsbn"} 3.0
method_timed_seconds_sum{class="org.littlewings.quarkus.micrometer.CustomMetricsBooksResource",exception="none",method="findByIsbn"} 7.50491E-4
# TYPE method_timed_seconds_max gauge
# HELP method_timed_seconds_max
method_timed_seconds_max{class="org.littlewings.quarkus.micrometer.CustomMetricsBooksResource",exception="none",method="create"} 4.000898977
method_timed_seconds_max{class="org.littlewings.quarkus.micrometer.CustomMetricsBooksResource",exception="none",method="delete"} 4.14856E-4
method_timed_seconds_max{class="org.littlewings.quarkus.micrometer.CustomMetricsBooksResource",exception="none",method="findAll"} 7.22673E-4
method_timed_seconds_max{class="org.littlewings.quarkus.micrometer.CustomMetricsBooksResource",exception="none",method="findByIsbn"} 4.61411E-4

自分で追加したメトリクスは、こちらですね。

# TYPE custom_store_books_register_seconds summary
# HELP custom_store_books_register_seconds
custom_store_books_register_seconds_count 4.0
custom_store_books_register_seconds_sum 0.001986586
# TYPE custom_store_books_register_seconds_max gauge
# HELP custom_store_books_register_seconds_max
custom_store_books_register_seconds_max 0.00142105
# TYPE custom_store_books_size gauge
# HELP custom_store_books_size
custom_store_books_size 3.0

@Timerアノテーションによるメトリクスは、こちら。

# TYPE method_timed_seconds summary
# HELP method_timed_seconds
method_timed_seconds_count{class="org.littlewings.quarkus.micrometer.CustomMetricsBooksResource",exception="none",method="create"} 4.0
method_timed_seconds_sum{class="org.littlewings.quarkus.micrometer.CustomMetricsBooksResource",exception="none",method="create"} 10.008363919
method_timed_seconds_count{class="org.littlewings.quarkus.micrometer.CustomMetricsBooksResource",exception="none",method="delete"} 1.0
method_timed_seconds_sum{class="org.littlewings.quarkus.micrometer.CustomMetricsBooksResource",exception="none",method="delete"} 4.14856E-4
method_timed_seconds_count{class="org.littlewings.quarkus.micrometer.CustomMetricsBooksResource",exception="none",method="findAll"} 2.0
method_timed_seconds_sum{class="org.littlewings.quarkus.micrometer.CustomMetricsBooksResource",exception="none",method="findAll"} 8.46507E-4
method_timed_seconds_count{class="org.littlewings.quarkus.micrometer.CustomMetricsBooksResource",exception="none",method="findByIsbn"} 3.0
method_timed_seconds_sum{class="org.littlewings.quarkus.micrometer.CustomMetricsBooksResource",exception="none",method="findByIsbn"} 7.50491E-4
# TYPE method_timed_seconds_max gauge
# HELP method_timed_seconds_max
method_timed_seconds_max{class="org.littlewings.quarkus.micrometer.CustomMetricsBooksResource",exception="none",method="create"} 4.000898977
method_timed_seconds_max{class="org.littlewings.quarkus.micrometer.CustomMetricsBooksResource",exception="none",method="delete"} 4.14856E-4
method_timed_seconds_max{class="org.littlewings.quarkus.micrometer.CustomMetricsBooksResource",exception="none",method="findAll"} 7.22673E-4
method_timed_seconds_max{class="org.littlewings.quarkus.micrometer.CustomMetricsBooksResource",exception="none",method="findByIsbn"} 4.61411E-4

ちゃんと追加できていますね。

MeterFilterを使う

MeterFilterを使うことで、MeterRegistryのカスタマイズができます。

今回は、全メトリクスにapplicationというラベルを付与して、http.server.requestsとcustom（自分で追加したメトリクス）で始まる
メトリクスにパーセンタイルを有効にしてみました。

src/main/java/org/littlewings/quarkus/micrometer/MeterFilterConfiguration.java

package org.littlewings.quarkus.micrometer;

import io.micrometer.core.instrument.Meter;
import io.micrometer.core.instrument.Tag;
import io.micrometer.core.instrument.config.MeterFilter;
import io.micrometer.core.instrument.distribution.DistributionStatisticConfig;

import javax.enterprise.inject.Produces;
import javax.inject.Singleton;
import java.util.List;

@Singleton
public class MeterFilterConfiguration {
    @Produces
    @Singleton
    public MeterFilter configureAllRegistriesTag() {
        return MeterFilter.commonTags(List.of(Tag.of("application", "quarkus")));
    }

    @Produces
    @Singleton
    public MeterFilter histogram() {
        return new MeterFilter() {
            @Override
            public DistributionStatisticConfig configure(Meter.Id id, DistributionStatisticConfig config) {
                if (id.getName().startsWith("http.server.requests") || id.getName().startsWith("custom")) {
                    return DistributionStatisticConfig
                            .builder()
                            .percentiles(0.5, 0.9, 0.95)
                            .percentilesHistogram(true)
                            .build()
                            .merge(config);
                }

                return config;
            }
        };
    }
}

MterFilterは、@ProducesアノテーションでCDI管理Beanとして定義します。

もう1度同じリクエストを全部実行した後に、メトリクスを確認。

こんな感じになりました。
※あんまりにも長いところは省略しています

$ curl -s localhost:8080/q/metrics
# TYPE system_cpu_count gauge
# HELP system_cpu_count The number of processors available to the Java virtual machine
system_cpu_count{application="quarkus"} 8.0
# TYPE worker_pool_completed counter
# HELP worker_pool_completed Number of times resources from the pool have been acquired
worker_pool_completed_total{application="quarkus",pool_name="vert.x-internal-blocking",pool_type="worker"} 0.0
worker_pool_completed_total{application="quarkus",pool_name="vert.x-worker-thread",pool_type="worker"} 13.0
# TYPE http_server_requests_seconds histogram
# HELP http_server_requests_seconds
http_server_requests_seconds{application="quarkus",method="DELETE",outcome="SUCCESS",status="204",uri="/custom-metrics/books/{isbn}",quantile="0.5"} 0.001703936
http_server_requests_seconds{application="quarkus",method="DELETE",outcome="SUCCESS",status="204",uri="/custom-metrics/books/{isbn}",quantile="0.9"} 0.001703936
http_server_requests_seconds{application="quarkus",method="DELETE",outcome="SUCCESS",status="204",uri="/custom-metrics/books/{isbn}",quantile="0.95"} 0.001703936
http_server_requests_seconds_bucket{application="quarkus",method="DELETE",outcome="SUCCESS",status="204",uri="/custom-metrics/books/{isbn}",le="0.001"} 0.0
http_server_requests_seconds_bucket{application="quarkus",method="DELETE",outcome="SUCCESS",status="204",uri="/custom-metrics/books/{isbn}",le="0.001048576"} 0.0
http_server_requests_seconds_bucket{application="quarkus",method="DELETE",outcome="SUCCESS",status="204",uri="/custom-metrics/books/{isbn}",le="0.001398101"} 0.0

〜省略〜

http_server_requests_seconds_bucket{application="quarkus",method="DELETE",outcome="SUCCESS",status="204",uri="/custom-metrics/books/{isbn}",le="17.179869184"} 1.0
http_server_requests_seconds_bucket{application="quarkus",method="DELETE",outcome="SUCCESS",status="204",uri="/custom-metrics/books/{isbn}",le="22.906492245"} 1.0
http_server_requests_seconds_bucket{application="quarkus",method="DELETE",outcome="SUCCESS",status="204",uri="/custom-metrics/books/{isbn}",le="28.633115306"} 1.0
http_server_requests_seconds_bucket{application="quarkus",method="DELETE",outcome="SUCCESS",status="204",uri="/custom-metrics/books/{isbn}",le="30.0"} 1.0
http_server_requests_seconds_bucket{application="quarkus",method="DELETE",outcome="SUCCESS",status="204",uri="/custom-metrics/books/{isbn}",le="+Inf"} 1.0
http_server_requests_seconds_count{application="quarkus",method="DELETE",outcome="SUCCESS",status="204",uri="/custom-metrics/books/{isbn}"} 1.0
http_server_requests_seconds_sum{application="quarkus",method="DELETE",outcome="SUCCESS",status="204",uri="/custom-metrics/books/{isbn}"} 0.001755602
http_server_requests_seconds{application="quarkus",method="GET",outcome="SUCCESS",status="204",uri="/custom-metrics/books/{isbn}",quantile="0.5"} 0.001900544
http_server_requests_seconds{application="quarkus",method="GET",outcome="SUCCESS",status="204",uri="/custom-metrics/books/{isbn}",quantile="0.9"} 0.001900544
http_server_requests_seconds{application="quarkus",method="GET",outcome="SUCCESS",status="204",uri="/custom-metrics/books/{isbn}",quantile="0.95"} 0.001900544
http_server_requests_seconds_bucket{application="quarkus",method="GET",outcome="SUCCESS",status="204",uri="/custom-metrics/books/{isbn}",le="0.001"} 0.0
http_server_requests_seconds_bucket{application="quarkus",method="GET",outcome="SUCCESS",status="204",uri="/custom-metrics/books/{isbn}",le="0.001048576"} 0.0
http_server_requests_seconds_bucket{application="quarkus",method="GET",outcome="SUCCESS",status="204",uri="/custom-metrics/books/{isbn}",le="0.001398101"} 0.0

〜省略〜

http_server_requests_seconds_bucket{application="quarkus",method="GET",outcome="SUCCESS",status="204",uri="/custom-metrics/books/{isbn}",le="17.179869184"} 1.0
http_server_requests_seconds_bucket{application="quarkus",method="GET",outcome="SUCCESS",status="204",uri="/custom-metrics/books/{isbn}",le="22.906492245"} 1.0
http_server_requests_seconds_bucket{application="quarkus",method="GET",outcome="SUCCESS",status="204",uri="/custom-metrics/books/{isbn}",le="28.633115306"} 1.0
http_server_requests_seconds_bucket{application="quarkus",method="GET",outcome="SUCCESS",status="204",uri="/custom-metrics/books/{isbn}",le="30.0"} 1.0
http_server_requests_seconds_bucket{application="quarkus",method="GET",outcome="SUCCESS",status="204",uri="/custom-metrics/books/{isbn}",le="+Inf"} 1.0
http_server_requests_seconds_count{application="quarkus",method="GET",outcome="SUCCESS",status="204",uri="/custom-metrics/books/{isbn}"} 1.0
http_server_requests_seconds_sum{application="quarkus",method="GET",outcome="SUCCESS",status="204",uri="/custom-metrics/books/{isbn}"} 0.001904577
http_server_requests_seconds{application="quarkus",method="GET",outcome="SUCCESS",status="204",uri="/books/{isbn}",quantile="0.5"} 0.001572864
http_server_requests_seconds{application="quarkus",method="GET",outcome="SUCCESS",status="204",uri="/books/{isbn}",quantile="0.9"} 0.001572864
http_server_requests_seconds{application="quarkus",method="GET",outcome="SUCCESS",status="204",uri="/books/{isbn}",quantile="0.95"} 0.001572864
http_server_requests_seconds_bucket{application="quarkus",method="GET",outcome="SUCCESS",status="204",uri="/books/{isbn}",le="0.001"} 0.0
http_server_requests_seconds_bucket{application="quarkus",method="GET",outcome="SUCCESS",status="204",uri="/books/{isbn}",le="0.001048576"} 0.0
http_server_requests_seconds_bucket{application="quarkus",method="GET",outcome="SUCCESS",status="204",uri="/books/{isbn}",le="0.001398101"} 0.0

〜省略〜

http_server_requests_seconds_bucket{application="quarkus",method="GET",outcome="SUCCESS",status="204",uri="/books/{isbn}",le="17.179869184"} 1.0
http_server_requests_seconds_bucket{application="quarkus",method="GET",outcome="SUCCESS",status="204",uri="/books/{isbn}",le="22.906492245"} 1.0
http_server_requests_seconds_bucket{application="quarkus",method="GET",outcome="SUCCESS",status="204",uri="/books/{isbn}",le="28.633115306"} 1.0
http_server_requests_seconds_bucket{application="quarkus",method="GET",outcome="SUCCESS",status="204",uri="/books/{isbn}",le="30.0"} 1.0
http_server_requests_seconds_bucket{application="quarkus",method="GET",outcome="SUCCESS",status="204",uri="/books/{isbn}",le="+Inf"} 1.0
http_server_requests_seconds_count{application="quarkus",method="GET",outcome="SUCCESS",status="204",uri="/books/{isbn}"} 1.0
http_server_requests_seconds_sum{application="quarkus",method="GET",outcome="SUCCESS",status="204",uri="/books/{isbn}"} 0.001601883
http_server_requests_seconds{application="quarkus",method="POST",outcome="SUCCESS",status="201",uri="/books",quantile="0.5"} 0.001900544
http_server_requests_seconds{application="quarkus",method="POST",outcome="SUCCESS",status="201",uri="/books",quantile="0.9"} 0.050266112
http_server_requests_seconds{application="quarkus",method="POST",outcome="SUCCESS",status="201",uri="/books",quantile="0.95"} 0.050266112
http_server_requests_seconds_bucket{application="quarkus",method="POST",outcome="SUCCESS",status="201",uri="/books",le="0.001"} 0.0
http_server_requests_seconds_bucket{application="quarkus",method="POST",outcome="SUCCESS",status="201",uri="/books",le="0.001048576"} 0.0
http_server_requests_seconds_bucket{application="quarkus",method="POST",outcome="SUCCESS",status="201",uri="/books",le="0.001398101"} 0.0
http_server_requests_seconds_bucket{application="quarkus",method="POST",outcome="SUCCESS",status="201",uri="/books",le="0.001747626"} 1.0

〜省略〜

http_server_requests_seconds_bucket{application="quarkus",method="POST",outcome="SUCCESS",status="201",uri="/books",le="17.179869184"} 4.0
http_server_requests_seconds_bucket{application="quarkus",method="POST",outcome="SUCCESS",status="201",uri="/books",le="22.906492245"} 4.0
http_server_requests_seconds_bucket{application="quarkus",method="POST",outcome="SUCCESS",status="201",uri="/books",le="28.633115306"} 4.0
http_server_requests_seconds_bucket{application="quarkus",method="POST",outcome="SUCCESS",status="201",uri="/books",le="30.0"} 4.0
http_server_requests_seconds_bucket{application="quarkus",method="POST",outcome="SUCCESS",status="201",uri="/books",le="+Inf"} 4.0
http_server_requests_seconds_count{application="quarkus",method="POST",outcome="SUCCESS",status="201",uri="/books"} 4.0
http_server_requests_seconds_sum{application="quarkus",method="POST",outcome="SUCCESS",status="201",uri="/books"} 0.112147167
http_server_requests_seconds{application="quarkus",method="DELETE",outcome="SUCCESS",status="204",uri="/books/{isbn}",quantile="0.5"} 0.001507328
http_server_requests_seconds{application="quarkus",method="DELETE",outcome="SUCCESS",status="204",uri="/books/{isbn}",quantile="0.9"} 0.001507328
http_server_requests_seconds{application="quarkus",method="DELETE",outcome="SUCCESS",status="204",uri="/books/{isbn}",quantile="0.95"} 0.001507328
http_server_requests_seconds_bucket{application="quarkus",method="DELETE",outcome="SUCCESS",status="204",uri="/books/{isbn}",le="0.001"} 0.0
http_server_requests_seconds_bucket{application="quarkus",method="DELETE",outcome="SUCCESS",status="204",uri="/books/{isbn}",le="0.001048576"} 0.0
http_server_requests_seconds_bucket{application="quarkus",method="DELETE",outcome="SUCCESS",status="204",uri="/books/{isbn}",le="0.001398101"} 0.0

〜省略〜

http_server_requests_seconds_bucket{application="quarkus",method="DELETE",outcome="SUCCESS",status="204",uri="/books/{isbn}",le="17.179869184"} 1.0
http_server_requests_seconds_bucket{application="quarkus",method="DELETE",outcome="SUCCESS",status="204",uri="/books/{isbn}",le="22.906492245"} 1.0
http_server_requests_seconds_bucket{application="quarkus",method="DELETE",outcome="SUCCESS",status="204",uri="/books/{isbn}",le="28.633115306"} 1.0
http_server_requests_seconds_bucket{application="quarkus",method="DELETE",outcome="SUCCESS",status="204",uri="/books/{isbn}",le="30.0"} 1.0
http_server_requests_seconds_bucket{application="quarkus",method="DELETE",outcome="SUCCESS",status="204",uri="/books/{isbn}",le="+Inf"} 1.0
http_server_requests_seconds_count{application="quarkus",method="DELETE",outcome="SUCCESS",status="204",uri="/books/{isbn}"} 1.0
http_server_requests_seconds_sum{application="quarkus",method="DELETE",outcome="SUCCESS",status="204",uri="/books/{isbn}"} 0.001532955
http_server_requests_seconds{application="quarkus",method="GET",outcome="SUCCESS",status="200",uri="/books/{isbn}",quantile="0.5"} 9.17504E-4
http_server_requests_seconds{application="quarkus",method="GET",outcome="SUCCESS",status="200",uri="/books/{isbn}",quantile="0.9"} 0.005996544
http_server_requests_seconds{application="quarkus",method="GET",outcome="SUCCESS",status="200",uri="/books/{isbn}",quantile="0.95"} 0.005996544
http_server_requests_seconds_bucket{application="quarkus",method="GET",outcome="SUCCESS",status="200",uri="/books/{isbn}",le="0.001"} 1.0
http_server_requests_seconds_bucket{application="quarkus",method="GET",outcome="SUCCESS",status="200",uri="/books/{isbn}",le="0.001048576"} 1.0
http_server_requests_seconds_bucket{application="quarkus",method="GET",outcome="SUCCESS",status="200",uri="/books/{isbn}",le="0.001398101"} 1.0

〜省略〜

http_server_requests_seconds_bucket{application="quarkus",method="GET",outcome="SUCCESS",status="200",uri="/books/{isbn}",le="17.179869184"} 2.0
http_server_requests_seconds_bucket{application="quarkus",method="GET",outcome="SUCCESS",status="200",uri="/books/{isbn}",le="22.906492245"} 2.0
http_server_requests_seconds_bucket{application="quarkus",method="GET",outcome="SUCCESS",status="200",uri="/books/{isbn}",le="28.633115306"} 2.0
http_server_requests_seconds_bucket{application="quarkus",method="GET",outcome="SUCCESS",status="200",uri="/books/{isbn}",le="30.0"} 2.0
http_server_requests_seconds_bucket{application="quarkus",method="GET",outcome="SUCCESS",status="200",uri="/books/{isbn}",le="+Inf"} 2.0
http_server_requests_seconds_count{application="quarkus",method="GET",outcome="SUCCESS",status="200",uri="/books/{isbn}"} 2.0
http_server_requests_seconds_sum{application="quarkus",method="GET",outcome="SUCCESS",status="200",uri="/books/{isbn}"} 0.006754188
http_server_requests_seconds{application="quarkus",method="GET",outcome="SUCCESS",status="200",uri="/books",quantile="0.5"} 0.005767168
http_server_requests_seconds{application="quarkus",method="GET",outcome="SUCCESS",status="200",uri="/books",quantile="0.9"} 0.025952256
http_server_requests_seconds{application="quarkus",method="GET",outcome="SUCCESS",status="200",uri="/books",quantile="0.95"} 0.025952256
http_server_requests_seconds_bucket{application="quarkus",method="GET",outcome="SUCCESS",status="200",uri="/books",le="0.001"} 0.0
http_server_requests_seconds_bucket{application="quarkus",method="GET",outcome="SUCCESS",status="200",uri="/books",le="0.001048576"} 0.0
http_server_requests_seconds_bucket{application="quarkus",method="GET",outcome="SUCCESS",status="200",uri="/books",le="0.001398101"} 0.0
http_server_requests_seconds_bucket{application="quarkus",method="GET",outcome="SUCCESS",status="200",uri="/books",le="0.001747626"} 0.0

〜省略〜

http_server_requests_seconds_bucket{application="quarkus",method="GET",outcome="SUCCESS",status="200",uri="/books",le="17.179869184"} 2.0
http_server_requests_seconds_bucket{application="quarkus",method="GET",outcome="SUCCESS",status="200",uri="/books",le="22.906492245"} 2.0
http_server_requests_seconds_bucket{application="quarkus",method="GET",outcome="SUCCESS",status="200",uri="/books",le="28.633115306"} 2.0
http_server_requests_seconds_bucket{application="quarkus",method="GET",outcome="SUCCESS",status="200",uri="/books",le="30.0"} 2.0
http_server_requests_seconds_bucket{application="quarkus",method="GET",outcome="SUCCESS",status="200",uri="/books",le="+Inf"} 2.0
http_server_requests_seconds_count{application="quarkus",method="GET",outcome="SUCCESS",status="200",uri="/books"} 2.0
http_server_requests_seconds_sum{application="quarkus",method="GET",outcome="SUCCESS",status="200",uri="/books"} 0.031629448
http_server_requests_seconds{application="quarkus",method="GET",outcome="SUCCESS",status="200",uri="/custom-metrics/books",quantile="0.5"} 0.00524288
http_server_requests_seconds{application="quarkus",method="GET",outcome="SUCCESS",status="200",uri="/custom-metrics/books",quantile="0.9"} 0.005505024
http_server_requests_seconds{application="quarkus",method="GET",outcome="SUCCESS",status="200",uri="/custom-metrics/books",quantile="0.95"} 0.005505024
http_server_requests_seconds_bucket{application="quarkus",method="GET",outcome="SUCCESS",status="200",uri="/custom-metrics/books",le="0.001"} 0.0
http_server_requests_seconds_bucket{application="quarkus",method="GET",outcome="SUCCESS",status="200",uri="/custom-metrics/books",le="0.001048576"} 0.0
http_server_requests_seconds_bucket{application="quarkus",method="GET",outcome="SUCCESS",status="200",uri="/custom-metrics/books",le="0.001398101"} 0.0

〜省略〜

http_server_requests_seconds_bucket{application="quarkus",method="GET",outcome="SUCCESS",status="200",uri="/custom-metrics/books",le="17.179869184"} 2.0
http_server_requests_seconds_bucket{application="quarkus",method="GET",outcome="SUCCESS",status="200",uri="/custom-metrics/books",le="22.906492245"} 2.0
http_server_requests_seconds_bucket{application="quarkus",method="GET",outcome="SUCCESS",status="200",uri="/custom-metrics/books",le="28.633115306"} 2.0
http_server_requests_seconds_bucket{application="quarkus",method="GET",outcome="SUCCESS",status="200",uri="/custom-metrics/books",le="30.0"} 2.0
http_server_requests_seconds_bucket{application="quarkus",method="GET",outcome="SUCCESS",status="200",uri="/custom-metrics/books",le="+Inf"} 2.0
http_server_requests_seconds_count{application="quarkus",method="GET",outcome="SUCCESS",status="200",uri="/custom-metrics/books"} 2.0
http_server_requests_seconds_sum{application="quarkus",method="GET",outcome="SUCCESS",status="200",uri="/custom-metrics/books"} 0.011012689
http_server_requests_seconds{application="quarkus",method="POST",outcome="SUCCESS",status="201",uri="/custom-metrics/books",quantile="0.5"} 1.979711488
http_server_requests_seconds{application="quarkus",method="POST",outcome="SUCCESS",status="201",uri="/custom-metrics/books",quantile="0.9"} 3.992977408
http_server_requests_seconds{application="quarkus",method="POST",outcome="SUCCESS",status="201",uri="/custom-metrics/books",quantile="0.95"} 3.992977408
http_server_requests_seconds_bucket{application="quarkus",method="POST",outcome="SUCCESS",status="201",uri="/custom-metrics/books",le="0.001"} 0.0
http_server_requests_seconds_bucket{application="quarkus",method="POST",outcome="SUCCESS",status="201",uri="/custom-metrics/books",le="0.001048576"} 0.0
http_server_requests_seconds_bucket{application="quarkus",method="POST",outcome="SUCCESS",status="201",uri="/custom-metrics/books",le="0.001398101"} 0.0

〜省略〜

http_server_requests_seconds_bucket{application="quarkus",method="GET",outcome="SUCCESS",status="200",uri="/custom-metrics/books/{isbn}",le="17.179869184"} 2.0
http_server_requests_seconds_bucket{application="quarkus",method="GET",outcome="SUCCESS",status="200",uri="/custom-metrics/books/{isbn}",le="22.906492245"} 2.0
http_server_requests_seconds_bucket{application="quarkus",method="GET",outcome="SUCCESS",status="200",uri="/custom-metrics/books/{isbn}",le="28.633115306"} 2.0
http_server_requests_seconds_bucket{application="quarkus",method="GET",outcome="SUCCESS",status="200",uri="/custom-metrics/books/{isbn}",le="30.0"} 2.0
http_server_requests_seconds_bucket{application="quarkus",method="GET",outcome="SUCCESS",status="200",uri="/custom-metrics/books/{isbn}",le="+Inf"} 2.0
http_server_requests_seconds_count{application="quarkus",method="GET",outcome="SUCCESS",status="200",uri="/custom-metrics/books/{isbn}"} 2.0
http_server_requests_seconds_sum{application="quarkus",method="GET",outcome="SUCCESS",status="200",uri="/custom-metrics/books/{isbn}"} 0.003156322
# TYPE http_server_requests_seconds_max gauge
# HELP http_server_requests_seconds_max
http_server_requests_seconds_max{application="quarkus",method="DELETE",outcome="SUCCESS",status="204",uri="/custom-metrics/books/{isbn}"} 0.001755602
http_server_requests_seconds_max{application="quarkus",method="GET",outcome="SUCCESS",status="204",uri="/custom-metrics/books/{isbn}"} 0.001904577
http_server_requests_seconds_max{application="quarkus",method="GET",outcome="SUCCESS",status="204",uri="/books/{isbn}"} 0.001601883
http_server_requests_seconds_max{application="quarkus",method="POST",outcome="SUCCESS",status="201",uri="/books"} 0.100584296
http_server_requests_seconds_max{application="quarkus",method="DELETE",outcome="SUCCESS",status="204",uri="/books/{isbn}"} 0.001532955
http_server_requests_seconds_max{application="quarkus",method="GET",outcome="SUCCESS",status="200",uri="/books/{isbn}"} 0.005828978
http_server_requests_seconds_max{application="quarkus",method="GET",outcome="SUCCESS",status="200",uri="/books"} 0.025768711
http_server_requests_seconds_max{application="quarkus",method="GET",outcome="SUCCESS",status="200",uri="/custom-metrics/books"} 0.005624638
http_server_requests_seconds_max{application="quarkus",method="POST",outcome="SUCCESS",status="201",uri="/custom-metrics/books"} 4.005119149
http_server_requests_seconds_max{application="quarkus",method="GET",outcome="SUCCESS",status="200",uri="/custom-metrics/books/{isbn}"} 0.00224027
# TYPE worker_pool_queue_size gauge
# HELP worker_pool_queue_size Number of pending elements in the waiting queue
worker_pool_queue_size{application="quarkus",pool_name="vert.x-internal-blocking",pool_type="worker"} 0.0
worker_pool_queue_size{application="quarkus",pool_name="vert.x-worker-thread",pool_type="worker"} 0.0
# TYPE worker_pool_ratio gauge
# HELP worker_pool_ratio Pool usage ratio
worker_pool_ratio{application="quarkus",pool_name="vert.x-internal-blocking",pool_type="worker"} NaN
worker_pool_ratio{application="quarkus",pool_name="vert.x-worker-thread",pool_type="worker"} 0.05
# TYPE jvm_gc_live_data_size_bytes gauge
# HELP jvm_gc_live_data_size_bytes Size of long-lived heap memory pool after reclamation
jvm_gc_live_data_size_bytes{application="quarkus"} 0.0
# TYPE jvm_gc_memory_allocated_bytes counter
# HELP jvm_gc_memory_allocated_bytes Incremented for an increase in the size of the (young) heap memory pool after one GC to before the next
jvm_gc_memory_allocated_bytes_total{application="quarkus"} 0.0
# TYPE http_server_active_requests gauge
# HELP http_server_active_requests
http_server_active_requests{application="quarkus"} 1.0
# TYPE jvm_info counter
# HELP jvm_info JVM version info
jvm_info_total{application="quarkus",runtime="OpenJDK Runtime Environment",vendor="Private Build",version="17.0.6+10-Ubuntu-0ubuntu122.04"} 1.0
# TYPE jvm_classes_unloaded_classes counter
# HELP jvm_classes_unloaded_classes The total number of classes unloaded since the Java virtual machine has started execution
jvm_classes_unloaded_classes_total{application="quarkus"} 0.0
# TYPE worker_pool_queue_delay_seconds_max gauge
# HELP worker_pool_queue_delay_seconds_max Time spent in the waiting queue before being processed
worker_pool_queue_delay_seconds_max{application="quarkus",pool_name="vert.x-internal-blocking",pool_type="worker"} 0.0
worker_pool_queue_delay_seconds_max{application="quarkus",pool_name="vert.x-worker-thread",pool_type="worker"} 0.001447286
# TYPE worker_pool_queue_delay_seconds summary
# HELP worker_pool_queue_delay_seconds Time spent in the waiting queue before being processed
worker_pool_queue_delay_seconds_count{application="quarkus",pool_name="vert.x-internal-blocking",pool_type="worker"} 0.0
worker_pool_queue_delay_seconds_sum{application="quarkus",pool_name="vert.x-internal-blocking",pool_type="worker"} 0.0
worker_pool_queue_delay_seconds_count{application="quarkus",pool_name="vert.x-worker-thread",pool_type="worker"} 14.0
worker_pool_queue_delay_seconds_sum{application="quarkus",pool_name="vert.x-worker-thread",pool_type="worker"} 0.002581931
# TYPE jvm_buffer_total_capacity_bytes gauge
# HELP jvm_buffer_total_capacity_bytes An estimate of the total capacity of the buffers in this pool
jvm_buffer_total_capacity_bytes{application="quarkus",id="mapped"} 0.0
jvm_buffer_total_capacity_bytes{application="quarkus",id="direct"} 602552.0
jvm_buffer_total_capacity_bytes{application="quarkus",id="mapped - 'non-volatile memory'"} 0.0
# TYPE jvm_threads_peak_threads gauge
# HELP jvm_threads_peak_threads The peak live thread count since the Java virtual machine started or peak was reset
jvm_threads_peak_threads{application="quarkus"} 58.0
# TYPE process_files_open_files gauge
# HELP process_files_open_files The open file descriptor count
process_files_open_files{application="quarkus"} 315.0
# TYPE jvm_gc_max_data_size_bytes gauge
# HELP jvm_gc_max_data_size_bytes Max size of long-lived heap memory pool
jvm_gc_max_data_size_bytes{application="quarkus"} 4.17333248E9
# TYPE system_load_average_1m gauge
# HELP system_load_average_1m The sum of the number of runnable entities queued to available processors and the number of runnable entities running on the available processors averaged over a period of time
system_load_average_1m{application="quarkus"} 3.2841796875
# TYPE jvm_memory_usage_after_gc_percent gauge
# HELP jvm_memory_usage_after_gc_percent The percentage of long-lived heap pool used after the last GC event, in the range [0..1]
jvm_memory_usage_after_gc_percent{application="quarkus",area="heap",pool="long-lived"} 0.0
# TYPE jvm_memory_max_bytes gauge
# HELP jvm_memory_max_bytes The maximum amount of memory in bytes that can be used for memory management
jvm_memory_max_bytes{application="quarkus",area="nonheap",id="Metaspace"} -1.0
jvm_memory_max_bytes{application="quarkus",area="heap",id="G1 Old Gen"} 4.17333248E9
jvm_memory_max_bytes{application="quarkus",area="heap",id="G1 Survivor Space"} -1.0
jvm_memory_max_bytes{application="quarkus",area="heap",id="G1 Eden Space"} -1.0
jvm_memory_max_bytes{application="quarkus",area="nonheap",id="CodeCache"} 5.0331648E7
jvm_memory_max_bytes{application="quarkus",area="nonheap",id="Compressed Class Space"} 1.073741824E9
# TYPE worker_pool_idle gauge
# HELP worker_pool_idle The number of resources from the pool currently used
worker_pool_idle{application="quarkus",pool_name="vert.x-internal-blocking",pool_type="worker"} 20.0
worker_pool_idle{application="quarkus",pool_name="vert.x-worker-thread",pool_type="worker"} 19.0
# TYPE custom_store_books_size gauge
# HELP custom_store_books_size
custom_store_books_size{application="quarkus"} 3.0
# TYPE method_timed_seconds summary
# HELP method_timed_seconds
method_timed_seconds_count{application="quarkus",class="org.littlewings.quarkus.micrometer.CustomMetricsBooksResource",exception="none",method="findAll"} 2.0
method_timed_seconds_sum{application="quarkus",class="org.littlewings.quarkus.micrometer.CustomMetricsBooksResource",exception="none",method="findAll"} 9.71138E-4
method_timed_seconds_count{application="quarkus",class="org.littlewings.quarkus.micrometer.CustomMetricsBooksResource",exception="none",method="delete"} 1.0
method_timed_seconds_sum{application="quarkus",class="org.littlewings.quarkus.micrometer.CustomMetricsBooksResource",exception="none",method="delete"} 4.1948E-4
method_timed_seconds_count{application="quarkus",class="org.littlewings.quarkus.micrometer.CustomMetricsBooksResource",exception="none",method="create"} 4.0
method_timed_seconds_sum{application="quarkus",class="org.littlewings.quarkus.micrometer.CustomMetricsBooksResource",exception="none",method="create"} 9.011837585
method_timed_seconds_count{application="quarkus",class="org.littlewings.quarkus.micrometer.CustomMetricsBooksResource",exception="none",method="findByIsbn"} 3.0
method_timed_seconds_sum{application="quarkus",class="org.littlewings.quarkus.micrometer.CustomMetricsBooksResource",exception="none",method="findByIsbn"} 8.39793E-4
# TYPE method_timed_seconds_max gauge
# HELP method_timed_seconds_max
method_timed_seconds_max{application="quarkus",class="org.littlewings.quarkus.micrometer.CustomMetricsBooksResource",exception="none",method="findAll"} 7.61149E-4
method_timed_seconds_max{application="quarkus",class="org.littlewings.quarkus.micrometer.CustomMetricsBooksResource",exception="none",method="delete"} 4.1948E-4
method_timed_seconds_max{application="quarkus",class="org.littlewings.quarkus.micrometer.CustomMetricsBooksResource",exception="none",method="create"} 4.00099564
method_timed_seconds_max{application="quarkus",class="org.littlewings.quarkus.micrometer.CustomMetricsBooksResource",exception="none",method="findByIsbn"} 4.01224E-4
# TYPE http_server_connections_seconds_max gauge
# HELP http_server_connections_seconds_max The duration of the connections
http_server_connections_seconds_max{application="quarkus"} 24.336646476
# TYPE http_server_connections_seconds summary
# HELP http_server_connections_seconds The duration of the connections
http_server_connections_seconds_active_count{application="quarkus"} 2.0
http_server_connections_seconds_duration_sum{application="quarkus"} 24.345742658
# TYPE jvm_threads_live_threads gauge
# HELP jvm_threads_live_threads The current number of live threads including both daemon and non-daemon threads
jvm_threads_live_threads{application="quarkus"} 27.0
# TYPE system_cpu_usage gauge
# HELP system_cpu_usage The \"recent cpu usage\" of the system the application is running in
system_cpu_usage{application="quarkus"} 0.260844250363901
# TYPE jvm_memory_committed_bytes gauge
# HELP jvm_memory_committed_bytes The amount of memory in bytes that is committed for the Java virtual machine to use
jvm_memory_committed_bytes{application="quarkus",area="nonheap",id="Metaspace"} 4.6989312E7
jvm_memory_committed_bytes{application="quarkus",area="heap",id="G1 Old Gen"} 7.7594624E7
jvm_memory_committed_bytes{application="quarkus",area="heap",id="G1 Survivor Space"} 4194304.0
jvm_memory_committed_bytes{application="quarkus",area="heap",id="G1 Eden Space"} 7.9691776E7
jvm_memory_committed_bytes{application="quarkus",area="nonheap",id="CodeCache"} 1.2386304E7
jvm_memory_committed_bytes{application="quarkus",area="nonheap",id="Compressed Class Space"} 6684672.0
# TYPE jvm_gc_memory_promoted_bytes counter
# HELP jvm_gc_memory_promoted_bytes Count of positive increases in the size of the old generation memory pool before GC to after GC
jvm_gc_memory_promoted_bytes_total{application="quarkus"} 0.0
# TYPE custom_store_books_register_seconds histogram
# HELP custom_store_books_register_seconds
custom_store_books_register_seconds{application="quarkus",quantile="0.5"} 2.048E-4
custom_store_books_register_seconds{application="quarkus",quantile="0.9"} 0.001368064
custom_store_books_register_seconds{application="quarkus",quantile="0.95"} 0.001368064
custom_store_books_register_seconds_bucket{application="quarkus",le="0.001"} 3.0
custom_store_books_register_seconds_bucket{application="quarkus",le="0.001048576"} 3.0
custom_store_books_register_seconds_bucket{application="quarkus",le="0.001398101"} 4.0

〜省略〜

custom_store_books_register_seconds_bucket{application="quarkus",le="17.179869184"} 4.0
custom_store_books_register_seconds_bucket{application="quarkus",le="22.906492245"} 4.0
custom_store_books_register_seconds_bucket{application="quarkus",le="28.633115306"} 4.0
custom_store_books_register_seconds_bucket{application="quarkus",le="30.0"} 4.0
custom_store_books_register_seconds_bucket{application="quarkus",le="+Inf"} 4.0
custom_store_books_register_seconds_count{application="quarkus"} 4.0
custom_store_books_register_seconds_sum{application="quarkus"} 0.002065362
# TYPE custom_store_books_register_seconds_max gauge
# HELP custom_store_books_register_seconds_max
custom_store_books_register_seconds_max{application="quarkus"} 0.001347034
# TYPE jvm_threads_daemon_threads gauge
# HELP jvm_threads_daemon_threads The current number of live daemon threads
jvm_threads_daemon_threads{application="quarkus"} 17.0
# TYPE process_uptime_seconds gauge
# HELP process_uptime_seconds The uptime of the Java virtual machine
process_uptime_seconds{application="quarkus"} 33.421
# TYPE process_cpu_usage gauge
# HELP process_cpu_usage The \"recent cpu usage\" for the Java Virtual Machine process
process_cpu_usage{application="quarkus"} 8.741258741258741E-4
# TYPE jvm_memory_used_bytes gauge
# HELP jvm_memory_used_bytes The amount of used memory
jvm_memory_used_bytes{application="quarkus",area="nonheap",id="Metaspace"} 4.639248E7
jvm_memory_used_bytes{application="quarkus",area="heap",id="G1 Old Gen"} 3.8484992E7
jvm_memory_used_bytes{application="quarkus",area="heap",id="G1 Survivor Space"} 3654848.0
jvm_memory_used_bytes{application="quarkus",area="heap",id="G1 Eden Space"} 4.4040192E7
jvm_memory_used_bytes{application="quarkus",area="nonheap",id="CodeCache"} 1.1197568E7
jvm_memory_used_bytes{application="quarkus",area="nonheap",id="Compressed Class Space"} 6391424.0
# TYPE worker_pool_usage_seconds summary
# HELP worker_pool_usage_seconds Time spent using resources from the pool
worker_pool_usage_seconds_count{application="quarkus",pool_name="vert.x-internal-blocking",pool_type="worker"} 0.0
worker_pool_usage_seconds_sum{application="quarkus",pool_name="vert.x-internal-blocking",pool_type="worker"} 0.0
worker_pool_usage_seconds_count{application="quarkus",pool_name="vert.x-worker-thread",pool_type="worker"} 13.0
worker_pool_usage_seconds_sum{application="quarkus",pool_name="vert.x-worker-thread",pool_type="worker"} 0.12950493
# TYPE worker_pool_usage_seconds_max gauge
# HELP worker_pool_usage_seconds_max Time spent using resources from the pool
worker_pool_usage_seconds_max{application="quarkus",pool_name="vert.x-internal-blocking",pool_type="worker"} 0.0
worker_pool_usage_seconds_max{application="quarkus",pool_name="vert.x-worker-thread",pool_type="worker"} 0.039114791
# TYPE process_start_time_seconds gauge
# HELP process_start_time_seconds Start time of the process since unix epoch.
process_start_time_seconds{application="quarkus"} 1.680959526301E9
# TYPE http_server_bytes_written_max gauge
# HELP http_server_bytes_written_max Number of bytes sent by the server
http_server_bytes_written_max{application="quarkus"} 131072.0
# TYPE http_server_bytes_written summary
# HELP http_server_bytes_written Number of bytes sent by the server
http_server_bytes_written_count{application="quarkus"} 18.0
http_server_bytes_written_sum{application="quarkus"} 319859.0
# TYPE http_server_bytes_read summary
# HELP http_server_bytes_read Number of bytes received by the server
http_server_bytes_read_count{application="quarkus"} 8.0
http_server_bytes_read_sum{application="quarkus"} 760.0
# TYPE http_server_bytes_read_max gauge
# HELP http_server_bytes_read_max Number of bytes received by the server
http_server_bytes_read_max{application="quarkus"} 109.0
# TYPE jvm_gc_overhead_percent gauge
# HELP jvm_gc_overhead_percent An approximation of the percent of CPU time used by GC activities over the last lookback period or since monitoring began, whichever is shorter, in the range [0..1]
jvm_gc_overhead_percent{application="quarkus"} 0.0
# TYPE jvm_classes_loaded_classes gauge
# HELP jvm_classes_loaded_classes The number of classes that are currently loaded in the Java virtual machine
jvm_classes_loaded_classes{application="quarkus"} 10433.0
# TYPE jvm_buffer_count_buffers gauge
# HELP jvm_buffer_count_buffers An estimate of the number of buffers in the pool
jvm_buffer_count_buffers{application="quarkus",id="mapped"} 0.0
jvm_buffer_count_buffers{application="quarkus",id="direct"} 20.0
jvm_buffer_count_buffers{application="quarkus",id="mapped - 'non-volatile memory'"} 0.0
# TYPE jvm_threads_states_threads gauge
# HELP jvm_threads_states_threads The current number of threads
jvm_threads_states_threads{application="quarkus",state="new"} 0.0
jvm_threads_states_threads{application="quarkus",state="runnable"} 11.0
jvm_threads_states_threads{application="quarkus",state="timed-waiting"} 6.0
jvm_threads_states_threads{application="quarkus",state="terminated"} 0.0
jvm_threads_states_threads{application="quarkus",state="blocked"} 0.0
jvm_threads_states_threads{application="quarkus",state="waiting"} 10.0
# TYPE worker_pool_active gauge
# HELP worker_pool_active The number of resources from the pool currently used
worker_pool_active{application="quarkus",pool_name="vert.x-internal-blocking",pool_type="worker"} 0.0
worker_pool_active{application="quarkus",pool_name="vert.x-worker-thread",pool_type="worker"} 1.0
# TYPE jvm_buffer_memory_used_bytes gauge
# HELP jvm_buffer_memory_used_bytes An estimate of the memory that the Java virtual machine is using for this buffer pool
jvm_buffer_memory_used_bytes{application="quarkus",id="mapped"} 0.0
jvm_buffer_memory_used_bytes{application="quarkus",id="direct"} 602554.0
jvm_buffer_memory_used_bytes{application="quarkus",id="mapped - 'non-volatile memory'"} 0.0
# TYPE process_files_max_files gauge
# HELP process_files_max_files The maximum file descriptor count
process_files_max_files{application="quarkus"} 1048576.0
# EOF

全メトリクスに、{application="quarkus"}というラベルが入りました。

また、条件で指定したメトリクスにはパーセンタイルが入るようになっています。

# TYPE http_server_requests_seconds histogram
# HELP http_server_requests_seconds
http_server_requests_seconds{application="quarkus",method="DELETE",outcome="SUCCESS",status="204",uri="/custom-metrics/books/{isbn}",quantile="0.5"} 0.001703936
http_server_requests_seconds{application="quarkus",method="DELETE",outcome="SUCCESS",status="204",uri="/custom-metrics/books/{isbn}",quantile="0.9"} 0.001703936
http_server_requests_seconds{application="quarkus",method="DELETE",outcome="SUCCESS",status="204",uri="/custom-metrics/books/{isbn}",quantile="0.95"} 0.001703936
http_server_requests_seconds_bucket{application="quarkus",method="DELETE",outcome="SUCCESS",status="204",uri="/custom-metrics/books/{isbn}",le="0.001"} 0.0
http_server_requests_seconds_bucket{application="quarkus",method="DELETE",outcome="SUCCESS",status="204",uri="/custom-metrics/books/{isbn}",le="0.001048576"} 0.0
http_server_requests_seconds_bucket{application="quarkus",method="DELETE",outcome="SUCCESS",status="204",uri="/custom-metrics/books/{isbn}",le="0.001398101"} 0.0

〜省略〜

http_server_requests_seconds_bucket{application="quarkus",method="DELETE",outcome="SUCCESS",status="204",uri="/custom-metrics/books/{isbn}",le="17.179869184"} 1.0
http_server_requests_seconds_bucket{application="quarkus",method="DELETE",outcome="SUCCESS",status="204",uri="/custom-metrics/books/{isbn}",le="22.906492245"} 1.0
http_server_requests_seconds_bucket{application="quarkus",method="DELETE",outcome="SUCCESS",status="204",uri="/custom-metrics/books/{isbn}",le="28.633115306"} 1.0
http_server_requests_seconds_bucket{application="quarkus",method="DELETE",outcome="SUCCESS",status="204",uri="/custom-metrics/books/{isbn}",le="30.0"} 1.0
http_server_requests_seconds_bucket{application="quarkus",method="DELETE",outcome="SUCCESS",status="204",uri="/custom-metrics/books/{isbn}",le="+Inf"} 1.0
http_server_requests_seconds_count{application="quarkus",method="DELETE",outcome="SUCCESS",status="204",uri="/custom-metrics/books/{isbn}"} 1.0
http_server_requests_seconds_sum{application="quarkus",method="DELETE",outcome="SUCCESS",status="204",uri="/custom-metrics/books/{isbn}"} 0.001755602


# TYPE custom_store_books_register_seconds histogram
# HELP custom_store_books_register_seconds
custom_store_books_register_seconds{application="quarkus",quantile="0.5"} 2.048E-4
custom_store_books_register_seconds{application="quarkus",quantile="0.9"} 0.001368064
custom_store_books_register_seconds{application="quarkus",quantile="0.95"} 0.001368064
custom_store_books_register_seconds_bucket{application="quarkus",le="0.001"} 3.0
custom_store_books_register_seconds_bucket{application="quarkus",le="0.001048576"} 3.0
custom_store_books_register_seconds_bucket{application="quarkus",le="0.001398101"} 4.0

〜省略〜

custom_store_books_register_seconds_bucket{application="quarkus",le="17.179869184"} 4.0
custom_store_books_register_seconds_bucket{application="quarkus",le="22.906492245"} 4.0
custom_store_books_register_seconds_bucket{application="quarkus",le="28.633115306"} 4.0
custom_store_books_register_seconds_bucket{application="quarkus",le="30.0"} 4.0
custom_store_books_register_seconds_bucket{application="quarkus",le="+Inf"} 4.0
custom_store_books_register_seconds_count{application="quarkus"} 4.0
custom_store_books_register_seconds_sum{application="quarkus"} 0.002065362

Prometheusにメトリクスを取り込む

最後に、Prometheusでメトリクスを取り込んでみましょう。

こんな感じで設定を用意。スクレイピング先は、172.17.0.1とします。

prometheus.yml

global:
  scrape_interval: 5s
  evaluation_interval: 5s

scrape_configs:
  - job_name: "prometheus"
    static_configs:
      - targets: ["localhost:9090"]
  - job_name: "quarkus"
    metrics_path: "/q/metrics"
    static_configs:
      - targets: ["172.17.0.1:8080"]

アプリケーション側は、別ホストからアクセスがあるのでバインドするアドレスを変更します（デフォルトはlocalhostです）。

src/main/resources/application.properties

quarkus.http.host=0.0.0.0

PrometheusのWeb UIで確認。

OKそうですね。

オマケ

今回完全に飛ばしましたが、QuarkusのMicrometer Extensionでは設定がいろいろできます。

Micrometer Metrics / Configuration Reference

Micrometerを有効にするかどうかの設定や、記録するメトリクスをカテゴリー単位で有効、無効にできます。
たとえばquarkus.micrometer.binder.http-server.enabled、quarkus.micrometer.binder.redis.enabledなどですね。

このカテゴリーのような単位をBinderと呼ぶみたいです。

このBinderの実装を見ると、Quakursが自動的に設定するメトリクスに関する情報を見ることができます。

https://github.com/quarkusio/quarkus/tree/2.16.6.Final/extensions/micrometer/runtime/src/main/java/io/quarkus/micrometer/runtime/binder

なので、各Extensionの実装ではなくMicromter Extensionを見ればだいたいメトリクスに関する情報はわかることになります。

たとえば、HTTPリクエスト（サーバー側）に関するメトリクスは、このあたりを見ると良さそうです。

https://github.com/quarkusio/quarkus/blob/2.16.6.Final/extensions/micrometer/runtime/src/main/java/io/quarkus/micrometer/runtime/binder/vertx/HttpRequestMetric.java

https://github.com/quarkusio/quarkus/blob/2.16.6.Final/extensions/micrometer/runtime/src/main/java/io/quarkus/micrometer/runtime/binder/RequestMetricInfo.java

https://github.com/quarkusio/quarkus/blob/2.16.6.Final/extensions/micrometer/runtime/src/main/java/io/quarkus/micrometer/runtime/binder/HttpBinderConfiguration.java

有効にしているのも、MeterFilterですね。

https://github.com/quarkusio/quarkus/blob/2.16.6.Final/extensions/micrometer/runtime/src/main/java/io/quarkus/micrometer/runtime/binder/HttpMeterFilterProvider.java

こんなところでしょうか。

まとめ

QuarkusのMicrometer Extensionでメトリクスの取得や独自のメトリクス定義などを行ってみました。

動かすだけなら割とあっさりできたのですが、そもそもMicromterなどに詳しくならないと使いこなせないですね…。