InfinispanのFunctional Map APIで、関数がどのNodeで実行されるのかを確認する

これは、なにをしたくて書いたもの？

• Stack Overflowで、いわゆるEntry Processorにあたるものは？という質問を見かけたので
• 回答としては、Functional Map APIだということだったので、関数がどのNodeで実行されるのかを改めて確認しようと思って

元ネタの質問
java - Infinispan: how to invoke EntryProcessor asynchronous? - Stack Overflow

こんな感じの動機です。まんまなので、早速確認してみましょう。

Funcational Map API？

Infinispan 8.0で導入された、Mapに対するCompletableFutureを使った非同期操作を提供するAPIです。

用途に応じて、Read-Only、Write-Only、Read-WriteなMapの3種類があります。各Mapのメソッドには、Functionなどを
渡して処理を行うのですが、リモートで実行されることもあるため多くはSerializableであることが求められます。

で、このFunction（関数）が、どのNodeで実行されるのかという話。

ドキュメントを見た感じ、特にデータローカリティについては書かれていない感じに見えますが、Stack Overflowの
回答を見るとそうでもなさそうなので、確認してみましょう、と。

環境

今回の環境は、こちら。

$ java -version
openjdk version "1.8.0_181"
OpenJDK Runtime Environment (build 1.8.0_181-8u181-b13-0ubuntu0.18.04.1-b13)
OpenJDK 64-Bit Server VM (build 25.181-b13, mixed mode)


$ mvn -version
Apache Maven 3.5.4 (1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-18T03:33:14+09:00)
Maven home: $HOME/.sdkman/candidates/maven/current
Java version: 1.8.0_181, vendor: Oracle Corporation, runtime: /usr/lib/jvm/java-8-openjdk-amd64/jre
Default locale: ja_JP, platform encoding: UTF-8
OS name: "linux", version: "4.15.0-38-generic", arch: "amd64", family: "unix"

準備

Maven依存関係は、こちら。

        <dependency>
            <groupId>org.infinispan</groupId>
            <artifactId>infinispan-core</artifactId>
            <version>9.4.0.Final</version>
        </dependency>

Infinispan 9.4.0.Finalを使います。

Cacheは、デフォルト設定のDistributed Cacheを用意。
src/main/resources/infinispan.xml

<?xml version="1.0" encoding="UTF-8"?>
<infinispan
        xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
        xsi:schemaLocation="urn:infinispan:config:9.4 http://www.infinispan.org/schemas/infinispan-config-9.4.xsd"
        xmlns="urn:infinispan:config:9.4">

    <jgroups>
        <stack-file name="udp" path="default-configs/default-jgroups-udp.xml"/>
    </jgroups>

    <cache-container shutdown-hook="REGISTER">
        <distributed-cache name="distributedCache"/>
    </cache-container>
</infinispan>

起動していてもらうだけの、Cache Server

最初に、単純に起動後に浮いていてもらうだけの、Cache Serverを書いてみます。
src/main/java/org/littlewings/infinispan/functionalmap/EmbeddedCacheServer.java

package org.littlewings.infinispan.functionalmap;

import java.io.IOException;

import org.infinispan.Cache;
import org.infinispan.manager.DefaultCacheManager;
import org.infinispan.manager.EmbeddedCacheManager;

public class EmbeddedCacheServer {
    public static void main(String... args) throws IOException {
        EmbeddedCacheManager manager = new DefaultCacheManager("infinispan.xml");

        Cache<String, Integer> cache = manager.getCache("distributedCache");

        System.out.printf("server[%s] started.%n", manager.getAddress());

        System.console().readLine("> Enter stop.");

        cache.stop();
        manager.stop();
    }
}

このサーバーは、データの格納と計算Nodeとして使用します。停止は、Enterで。

Functional Map APIを使用するプログラム

続いて、Functional Map APIを使用するプログラムを書いてみます。

まずは、ざっと雛形的な。
src/main/java/org/littlewings/infinispan/functionalmap/FunctionalMapRunner.java

package org.littlewings.infinispan.functionalmap;

import java.io.IOException;
import java.util.HashSet;
import java.util.Set;
import java.util.stream.IntStream;

import org.infinispan.Cache;
import org.infinispan.distribution.DistributionInfo;
import org.infinispan.distribution.DistributionManager;
import org.infinispan.functional.FunctionalMap;
import org.infinispan.functional.impl.FunctionalMapImpl;
import org.infinispan.functional.impl.ReadOnlyMapImpl;
import org.infinispan.functional.impl.ReadWriteMapImpl;
import org.infinispan.functional.impl.WriteOnlyMapImpl;
import org.infinispan.manager.DefaultCacheManager;
import org.infinispan.manager.EmbeddedCacheManager;
import org.jboss.logging.Logger;

public class FunctionalMapRunner {
    public static void main(String... args) throws IOException {
        EmbeddedCacheManager manager = new DefaultCacheManager("infinispan.xml");

        Cache<String, Integer> cache = manager.getCache("distributedCache");

        System.out.printf("server[%s] started.%n", manager.getAddress());

        IntStream.rangeClosed(1, 100).forEach(i -> cache.put("key" + i, i));

        // Functional Map APIを使う

        cache.stop();
        manager.stop();
    }
}

コメントしているところで、Functional Map APIを使います。データは、100件格納。

Functional Mapの作成。3種類のMap、すべて使うことにします。あと、データの配置状況を確認するためにDistributionManagerも併用。

        FunctionalMapImpl<String, Integer> functionalMap = FunctionalMapImpl.create(cache.getAdvancedCache());
        FunctionalMap.ReadOnlyMap<String, Integer> readOnlyMap = ReadOnlyMapImpl.create(functionalMap);
        FunctionalMap.WriteOnlyMap<String, Integer> writeOnlyMap = WriteOnlyMapImpl.create(functionalMap);
        FunctionalMap.ReadWriteMap<String, Integer> readWriteMap = ReadWriteMapImpl.create(functionalMap);

        DistributionManager dm = cache.getAdvancedCache().getDistributionManager();

まずは、単一キーを使用するAPIを使って、確認。

        System.out.println("============ single key ============");

        cache.keySet().forEach(key -> {
            DistributionInfo di = dm.getCacheTopology().getDistribution(key);
            System.out.printf("pattern-1: location / key[%s]: primary[%s] / backup%s%n", key, di.primary(), di.writeBackups());

            readOnlyMap.eval(key, entry -> {
                Logger logger = Logger.getLogger("lambda-logger");
                logger.infof("pattern-1: read-only / key[%s]%n", key);
                return entry.get();
            }).join();

            writeOnlyMap.eval(key, di.primary().toString(), (param, entry) -> {
                Logger logger = Logger.getLogger("lambda-logger");
                logger.infof("pattern-1: write-only / key[%s]: primary = %s%n", key, param);
            }).join();

            readWriteMap.eval(key, di.primary().toString(), (param, entry) -> {
                Logger logger = Logger.getLogger("lambda-logger");
                logger.infof("pattern-1: read-write / key[%s]: primary = %s%n", key, param);
                return null;
            }).join();
        });

DistributionInfoで、データの配置状況を出力しつつ、各Mapのevalメソッド内で実行時にログ出力。

このログが、どのNodeで出力されるかを確認します。書き込み可能なMapには引数が渡せるので、データのPrimary Ownerの
情報も合わせて出力するようにしています。

あと、evalManyも一応確認。ここでは、キーは4つ単位で確認していくことにします。

        System.out.println("============ many key ============");
        for (int i = 1; i <= 100; i += 4) {
            Set<String> keys = new HashSet<>();
            for (int j = 0; j < 4; j++) {
                if (i + j <= 100) {
                    keys.add("key" + (i + j));
                }
            }

            readOnlyMap.evalMany(keys, entry -> {
                Logger logger = Logger.getLogger("lambda-logger");
                logger.infof("pattern-2: read-only / key[%s]%n", entry.key());
                return entry.get();
            });

            writeOnlyMap.evalMany(keys, entry -> {
                Logger logger = Logger.getLogger("lambda-logger");
                logger.infof("pattern-2: write-only / key[%s]%n", entry.key());
            }).join();

            readWriteMap.evalMany(keys, entry -> {
                Logger logger = Logger.getLogger("lambda-logger");
                logger.infof("pattern-2: read-write / key[%s]%n", entry.key());
                return null;
            });
        }

これで、準備は完了です。

確認

それでは、確認してみましょう。

まずは、Cache Serverを2つ浮かせます。

## ひとつ目
$ mvn compile exec:java -Dexec.mainClass=org.littlewings.infinispan.functionalmap.EmbeddedCacheServer

## ふたつ目
$ mvn compile exec:java -Dexec.mainClass=org.littlewings.infinispan.functionalmap.EmbeddedCacheServer

そして、Functional Map APIを使う起点となるNodeを起動。

$ mvn compile exec:java -Dexec.mainClass=org.littlewings.infinispan.functionalmap.FunctionalMapRunner

ここで、各Nodeの名前は以下のようになっています。

• Cache Server × 2 … xxxxx-43836、xxxxx-21429
• Functional Map APIを起動するNode … xxxxx-38730

では、実行結果を。
※各パターンの出力結果のログをソートのうえ、20件をピックアップして記載

xxxxx-38730

#### Single Key

## ReadOnlyMap
INFO: pattern-1: read-only / key[key100]
INFO: pattern-1: read-only / key[key12]
INFO: pattern-1: read-only / key[key13]
INFO: pattern-1: read-only / key[key15]
INFO: pattern-1: read-only / key[key17]
INFO: pattern-1: read-only / key[key18]
INFO: pattern-1: read-only / key[key19]
INFO: pattern-1: read-only / key[key1]
INFO: pattern-1: read-only / key[key20]
INFO: pattern-1: read-only / key[key21]
INFO: pattern-1: read-only / key[key23]
INFO: pattern-1: read-only / key[key24]
INFO: pattern-1: read-only / key[key25]
INFO: pattern-1: read-only / key[key26]
INFO: pattern-1: read-only / key[key27]
INFO: pattern-1: read-only / key[key28]
INFO: pattern-1: read-only / key[key2]
INFO: pattern-1: read-only / key[key31]
INFO: pattern-1: read-only / key[key32]
INFO: pattern-1: read-only / key[key34]

...

## WriteOnlyMap
INFO: pattern-1: write-only / key[key18]: primary = xxxxx-38730
INFO: pattern-1: write-only / key[key19]: primary = xxxxx-38730
INFO: pattern-1: write-only / key[key20]: primary = xxxxx-38730
INFO: pattern-1: write-only / key[key23]: primary = xxxxx-38730
INFO: pattern-1: write-only / key[key24]: primary = xxxxx-38730
INFO: pattern-1: write-only / key[key2]: primary = xxxxx-38730
INFO: pattern-1: write-only / key[key31]: primary = xxxxx-38730
INFO: pattern-1: write-only / key[key35]: primary = xxxxx-38730
INFO: pattern-1: write-only / key[key39]: primary = xxxxx-38730
INFO: pattern-1: write-only / key[key3]: primary = xxxxx-38730
INFO: pattern-1: write-only / key[key44]: primary = xxxxx-38730
INFO: pattern-1: write-only / key[key45]: primary = xxxxx-38730
INFO: pattern-1: write-only / key[key46]: primary = xxxxx-38730
INFO: pattern-1: write-only / key[key53]: primary = xxxxx-38730
INFO: pattern-1: write-only / key[key55]: primary = xxxxx-38730
INFO: pattern-1: write-only / key[key58]: primary = xxxxx-38730
INFO: pattern-1: write-only / key[key64]: primary = xxxxx-38730
INFO: pattern-1: write-only / key[key72]: primary = xxxxx-38730
INFO: pattern-1: write-only / key[key74]: primary = xxxxx-38730

...

## ReadWriteMap
INFO: pattern-1: read-write / key[key18]: primary = xxxxx-38730
INFO: pattern-1: read-write / key[key19]: primary = xxxxx-38730
INFO: pattern-1: read-write / key[key20]: primary = xxxxx-38730
INFO: pattern-1: read-write / key[key23]: primary = xxxxx-38730
INFO: pattern-1: read-write / key[key24]: primary = xxxxx-38730
INFO: pattern-1: read-write / key[key2]: primary = xxxxx-38730
INFO: pattern-1: read-write / key[key31]: primary = xxxxx-38730
INFO: pattern-1: read-write / key[key35]: primary = xxxxx-38730
INFO: pattern-1: read-write / key[key39]: primary = xxxxx-38730
INFO: pattern-1: read-write / key[key3]: primary = xxxxx-38730
INFO: pattern-1: read-write / key[key44]: primary = xxxxx-38730
INFO: pattern-1: read-write / key[key45]: primary = xxxxx-38730
INFO: pattern-1: read-write / key[key46]: primary = xxxxx-38730
INFO: pattern-1: read-write / key[key53]: primary = xxxxx-38730
INFO: pattern-1: read-write / key[key55]: primary = xxxxx-38730
INFO: pattern-1: read-write / key[key58]: primary = xxxxx-38730
INFO: pattern-1: read-write / key[key64]: primary = xxxxx-38730
INFO: pattern-1: read-write / key[key72]: primary = xxxxx-38730
INFO: pattern-1: read-write / key[key74]: primary = xxxxx-38730

...


#### Many Keys

## ReadOnlyMap
INFO: pattern-2: read-only / key[key100]
INFO: pattern-2: read-only / key[key12]
INFO: pattern-2: read-only / key[key13]
INFO: pattern-2: read-only / key[key15]
INFO: pattern-2: read-only / key[key17]
INFO: pattern-2: read-only / key[key18]
INFO: pattern-2: read-only / key[key19]
INFO: pattern-2: read-only / key[key1]
INFO: pattern-2: read-only / key[key20]
INFO: pattern-2: read-only / key[key21]
INFO: pattern-2: read-only / key[key23]
INFO: pattern-2: read-only / key[key24]
INFO: pattern-2: read-only / key[key25]
INFO: pattern-2: read-only / key[key26]
INFO: pattern-2: read-only / key[key27]
INFO: pattern-2: read-only / key[key28]
INFO: pattern-2: read-only / key[key2]
INFO: pattern-2: read-only / key[key31]

...

## WriteOnlyMap
INFO: pattern-2: write-only / key[key100]
INFO: pattern-2: write-only / key[key12]
INFO: pattern-2: write-only / key[key13]
INFO: pattern-2: write-only / key[key15]
INFO: pattern-2: write-only / key[key17]
INFO: pattern-2: write-only / key[key18]
INFO: pattern-2: write-only / key[key19]
INFO: pattern-2: write-only / key[key1]
INFO: pattern-2: write-only / key[key20]
INFO: pattern-2: write-only / key[key21]
INFO: pattern-2: write-only / key[key23]
INFO: pattern-2: write-only / key[key24]
INFO: pattern-2: write-only / key[key25]
INFO: pattern-2: write-only / key[key26]
INFO: pattern-2: write-only / key[key27]
INFO: pattern-2: write-only / key[key28]
INFO: pattern-2: write-only / key[key2]
INFO: pattern-2: write-only / key[key31]
INFO: pattern-2: write-only / key[key32]
INFO: pattern-2: write-only / key[key34]

...

## ReadWriteMap
INFO: pattern-2: read-write / key[key100]
INFO: pattern-2: read-write / key[key12]
INFO: pattern-2: read-write / key[key13]
INFO: pattern-2: read-write / key[key15]
INFO: pattern-2: read-write / key[key17]
INFO: pattern-2: read-write / key[key18]
INFO: pattern-2: read-write / key[key19]
INFO: pattern-2: read-write / key[key1]
INFO: pattern-2: read-write / key[key20]
INFO: pattern-2: read-write / key[key21]
INFO: pattern-2: read-write / key[key23]
INFO: pattern-2: read-write / key[key24]
INFO: pattern-2: read-write / key[key25]
INFO: pattern-2: read-write / key[key26]
INFO: pattern-2: read-write / key[key27]
INFO: pattern-2: read-write / key[key28]
INFO: pattern-2: read-write / key[key2]
INFO: pattern-2: read-write / key[key31]
INFO: pattern-2: read-write / key[key32]
INFO: pattern-2: read-write / key[key34]

...

xxxxx-43836

#### Single Key

## ReadOnlyMap
INFO: pattern-1: read-only / key[key10]
INFO: pattern-1: read-only / key[key11]
INFO: pattern-1: read-only / key[key14]
INFO: pattern-1: read-only / key[key16]
INFO: pattern-1: read-only / key[key22]
INFO: pattern-1: read-only / key[key29]
INFO: pattern-1: read-only / key[key30]
INFO: pattern-1: read-only / key[key33]
INFO: pattern-1: read-only / key[key38]
INFO: pattern-1: read-only / key[key40]
INFO: pattern-1: read-only / key[key43]
INFO: pattern-1: read-only / key[key48]
INFO: pattern-1: read-only / key[key49]
INFO: pattern-1: read-only / key[key50]
INFO: pattern-1: read-only / key[key52]
INFO: pattern-1: read-only / key[key56]
INFO: pattern-1: read-only / key[key61]
INFO: pattern-1: read-only / key[key63]
INFO: pattern-1: read-only / key[key65]
INFO: pattern-1: read-only / key[key66]

...

## WriteOnlyMap
INFO: pattern-1: write-only / key[key13]: primary = xxxxx-43836
INFO: pattern-1: write-only / key[key16]: primary = xxxxx-43836
INFO: pattern-1: write-only / key[key17]: primary = xxxxx-43836
INFO: pattern-1: write-only / key[key25]: primary = xxxxx-43836
INFO: pattern-1: write-only / key[key26]: primary = xxxxx-43836
INFO: pattern-1: write-only / key[key28]: primary = xxxxx-43836
INFO: pattern-1: write-only / key[key29]: primary = xxxxx-43836
INFO: pattern-1: write-only / key[key32]: primary = xxxxx-43836
INFO: pattern-1: write-only / key[key36]: primary = xxxxx-43836
INFO: pattern-1: write-only / key[key37]: primary = xxxxx-43836
INFO: pattern-1: write-only / key[key40]: primary = xxxxx-43836
INFO: pattern-1: write-only / key[key41]: primary = xxxxx-43836
INFO: pattern-1: write-only / key[key42]: primary = xxxxx-43836
INFO: pattern-1: write-only / key[key43]: primary = xxxxx-43836
INFO: pattern-1: write-only / key[key48]: primary = xxxxx-43836
INFO: pattern-1: write-only / key[key4]: primary = xxxxx-43836
INFO: pattern-1: write-only / key[key50]: primary = xxxxx-43836
INFO: pattern-1: write-only / key[key51]: primary = xxxxx-43836
INFO: pattern-1: write-only / key[key56]: primary = xxxxx-43836
INFO: pattern-1: write-only / key[key57]: primary = xxxxx-43836

...

## ReadWriteMap
INFO: pattern-1: read-write / key[key13]: primary = xxxxx-43836
INFO: pattern-1: read-write / key[key16]: primary = xxxxx-43836
INFO: pattern-1: read-write / key[key17]: primary = xxxxx-43836
INFO: pattern-1: read-write / key[key25]: primary = xxxxx-43836
INFO: pattern-1: read-write / key[key26]: primary = xxxxx-43836
INFO: pattern-1: read-write / key[key28]: primary = xxxxx-43836
INFO: pattern-1: read-write / key[key29]: primary = xxxxx-43836
INFO: pattern-1: read-write / key[key32]: primary = xxxxx-43836
INFO: pattern-1: read-write / key[key36]: primary = xxxxx-43836
INFO: pattern-1: read-write / key[key37]: primary = xxxxx-43836
INFO: pattern-1: read-write / key[key40]: primary = xxxxx-43836
INFO: pattern-1: read-write / key[key41]: primary = xxxxx-43836
INFO: pattern-1: read-write / key[key42]: primary = xxxxx-43836
INFO: pattern-1: read-write / key[key43]: primary = xxxxx-43836
INFO: pattern-1: read-write / key[key48]: primary = xxxxx-43836
INFO: pattern-1: read-write / key[key4]: primary = xxxxx-43836
INFO: pattern-1: read-write / key[key50]: primary = xxxxx-43836
INFO: pattern-1: read-write / key[key51]: primary = xxxxx-43836
INFO: pattern-1: read-write / key[key56]: primary = xxxxx-43836
INFO: pattern-1: read-write / key[key57]: primary = xxxxx-43836

...


#### Many Keys

## ReadOnlyMap
INFO: pattern-2: read-only / key[key29]
INFO: pattern-2: read-only / key[key30]
INFO: pattern-2: read-only / key[key43]
INFO: pattern-2: read-only / key[key48]
INFO: pattern-2: read-only / key[key56]
INFO: pattern-2: read-only / key[key65]
INFO: pattern-2: read-only / key[key66]
INFO: pattern-2: read-only / key[key67]
INFO: pattern-2: read-only / key[key94]
（以上）

...

## WriteOnlyMap
INFO: pattern-2: write-only / key[key10]
INFO: pattern-2: write-only / key[key11]
INFO: pattern-2: write-only / key[key13]
INFO: pattern-2: write-only / key[key14]
INFO: pattern-2: write-only / key[key16]
INFO: pattern-2: write-only / key[key17]
INFO: pattern-2: write-only / key[key18]
INFO: pattern-2: write-only / key[key19]
INFO: pattern-2: write-only / key[key22]
INFO: pattern-2: write-only / key[key23]
INFO: pattern-2: write-only / key[key25]
INFO: pattern-2: write-only / key[key26]
INFO: pattern-2: write-only / key[key28]
INFO: pattern-2: write-only / key[key29]
INFO: pattern-2: write-only / key[key30]
INFO: pattern-2: write-only / key[key32]
INFO: pattern-2: write-only / key[key33]
INFO: pattern-2: write-only / key[key36]
INFO: pattern-2: write-only / key[key37]
INFO: pattern-2: write-only / key[key38]

...

## ReadWriteMap
INFO: pattern-2: read-write / key[key10]
INFO: pattern-2: read-write / key[key11]
INFO: pattern-2: read-write / key[key13]
INFO: pattern-2: read-write / key[key14]
INFO: pattern-2: read-write / key[key16]
INFO: pattern-2: read-write / key[key17]
INFO: pattern-2: read-write / key[key18]
INFO: pattern-2: read-write / key[key19]
INFO: pattern-2: read-write / key[key22]
INFO: pattern-2: read-write / key[key23]
INFO: pattern-2: read-write / key[key25]
INFO: pattern-2: read-write / key[key26]
INFO: pattern-2: read-write / key[key28]
INFO: pattern-2: read-write / key[key29]
INFO: pattern-2: read-write / key[key30]
INFO: pattern-2: read-write / key[key32]
INFO: pattern-2: read-write / key[key33]
INFO: pattern-2: read-write / key[key36]
INFO: pattern-2: read-write / key[key37]
INFO: pattern-2: read-write / key[key38]

...

xxxxx-21429

#### Single Key

## ReadOnlyMap
INFO: pattern-1: read-only / key[key10]
INFO: pattern-1: read-only / key[key11]
INFO: pattern-1: read-only / key[key14]
INFO: pattern-1: read-only / key[key16]
INFO: pattern-1: read-only / key[key22]
INFO: pattern-1: read-only / key[key29]
INFO: pattern-1: read-only / key[key30]
INFO: pattern-1: read-only / key[key33]
INFO: pattern-1: read-only / key[key38]
INFO: pattern-1: read-only / key[key40]
INFO: pattern-1: read-only / key[key43]
INFO: pattern-1: read-only / key[key48]
INFO: pattern-1: read-only / key[key49]
INFO: pattern-1: read-only / key[key50]
INFO: pattern-1: read-only / key[key52]
INFO: pattern-1: read-only / key[key56]
INFO: pattern-1: read-only / key[key61]
INFO: pattern-1: read-only / key[key63]
INFO: pattern-1: read-only / key[key65]
INFO: pattern-1: read-only / key[key66]

...

## WriteOnlyMap
INFO: pattern-1: write-only / key[key100]: primary = xxxxx-21429
INFO: pattern-1: write-only / key[key10]: primary = xxxxx-21429
INFO: pattern-1: write-only / key[key11]: primary = xxxxx-21429
INFO: pattern-1: write-only / key[key12]: primary = xxxxx-21429
INFO: pattern-1: write-only / key[key14]: primary = xxxxx-21429
INFO: pattern-1: write-only / key[key15]: primary = xxxxx-21429
INFO: pattern-1: write-only / key[key1]: primary = xxxxx-21429
INFO: pattern-1: write-only / key[key21]: primary = xxxxx-21429
INFO: pattern-1: write-only / key[key22]: primary = xxxxx-21429
INFO: pattern-1: write-only / key[key27]: primary = xxxxx-21429
INFO: pattern-1: write-only / key[key30]: primary = xxxxx-21429
INFO: pattern-1: write-only / key[key33]: primary = xxxxx-21429
INFO: pattern-1: write-only / key[key34]: primary = xxxxx-21429
INFO: pattern-1: write-only / key[key38]: primary = xxxxx-21429
INFO: pattern-1: write-only / key[key47]: primary = xxxxx-21429
INFO: pattern-1: write-only / key[key49]: primary = xxxxx-21429
INFO: pattern-1: write-only / key[key52]: primary = xxxxx-21429
INFO: pattern-1: write-only / key[key54]: primary = xxxxx-21429
INFO: pattern-1: write-only / key[key61]: primary = xxxxx-21429
INFO: pattern-1: write-only / key[key65]: primary = xxxxx-21429

...

## ReadWriteMap
INFO: pattern-1: read-write / key[key100]: primary = xxxxx-21429
INFO: pattern-1: read-write / key[key10]: primary = xxxxx-21429
INFO: pattern-1: read-write / key[key11]: primary = xxxxx-21429
INFO: pattern-1: read-write / key[key12]: primary = xxxxx-21429
INFO: pattern-1: read-write / key[key14]: primary = xxxxx-21429
INFO: pattern-1: read-write / key[key15]: primary = xxxxx-21429
INFO: pattern-1: read-write / key[key1]: primary = xxxxx-21429
INFO: pattern-1: read-write / key[key21]: primary = xxxxx-21429
INFO: pattern-1: read-write / key[key22]: primary = xxxxx-21429
INFO: pattern-1: read-write / key[key27]: primary = xxxxx-21429
INFO: pattern-1: read-write / key[key30]: primary = xxxxx-21429
INFO: pattern-1: read-write / key[key33]: primary = xxxxx-21429
INFO: pattern-1: read-write / key[key34]: primary = xxxxx-21429
INFO: pattern-1: read-write / key[key38]: primary = xxxxx-21429
INFO: pattern-1: read-write / key[key47]: primary = xxxxx-21429
INFO: pattern-1: read-write / key[key49]: primary = xxxxx-21429
INFO: pattern-1: read-write / key[key52]: primary = xxxxx-21429
INFO: pattern-1: read-write / key[key54]: primary = xxxxx-21429
INFO: pattern-1: read-write / key[key61]: primary = xxxxx-21429
INFO: pattern-1: read-write / key[key65]: primary = xxxxx-21429

...


#### Many Keys

## ReadOnlyMap
INFO: pattern-2: read-only / key[key10]
INFO: pattern-2: read-only / key[key11]
INFO: pattern-2: read-only / key[key14]
INFO: pattern-2: read-only / key[key16]
INFO: pattern-2: read-only / key[key22]
INFO: pattern-2: read-only / key[key33]
INFO: pattern-2: read-only / key[key38]
INFO: pattern-2: read-only / key[key40]
INFO: pattern-2: read-only / key[key49]
INFO: pattern-2: read-only / key[key50]
INFO: pattern-2: read-only / key[key52]
INFO: pattern-2: read-only / key[key61]
INFO: pattern-2: read-only / key[key63]
INFO: pattern-2: read-only / key[key70]
INFO: pattern-2: read-only / key[key71]
INFO: pattern-2: read-only / key[key76]
INFO: pattern-2: read-only / key[key79]
INFO: pattern-2: read-only / key[key88]
（以上）

## WriteOnlyMap
INFO: pattern-2: write-only / key[key100]
INFO: pattern-2: write-only / key[key10]
INFO: pattern-2: write-only / key[key11]
INFO: pattern-2: write-only / key[key12]
INFO: pattern-2: write-only / key[key14]
INFO: pattern-2: write-only / key[key15]
INFO: pattern-2: write-only / key[key16]
INFO: pattern-2: write-only / key[key1]
INFO: pattern-2: write-only / key[key20]
INFO: pattern-2: write-only / key[key21]
INFO: pattern-2: write-only / key[key22]
INFO: pattern-2: write-only / key[key24]
INFO: pattern-2: write-only / key[key27]
INFO: pattern-2: write-only / key[key29]
INFO: pattern-2: write-only / key[key2]
INFO: pattern-2: write-only / key[key30]
INFO: pattern-2: write-only / key[key31]
INFO: pattern-2: write-only / key[key33]
INFO: pattern-2: write-only / key[key34]
INFO: pattern-2: write-only / key[key35]

...

## ReadWriteMap
INFO: pattern-2: read-write / key[key100]
INFO: pattern-2: read-write / key[key10]
INFO: pattern-2: read-write / key[key11]
INFO: pattern-2: read-write / key[key12]
INFO: pattern-2: read-write / key[key14]
INFO: pattern-2: read-write / key[key15]
INFO: pattern-2: read-write / key[key16]
INFO: pattern-2: read-write / key[key1]
INFO: pattern-2: read-write / key[key20]
INFO: pattern-2: read-write / key[key21]
INFO: pattern-2: read-write / key[key22]
INFO: pattern-2: read-write / key[key24]
INFO: pattern-2: read-write / key[key27]
INFO: pattern-2: read-write / key[key29]
INFO: pattern-2: read-write / key[key2]
INFO: pattern-2: read-write / key[key30]
INFO: pattern-2: read-write / key[key31]
INFO: pattern-2: read-write / key[key33]
INFO: pattern-2: read-write / key[key34]
INFO: pattern-2: read-write / key[key35]

...

さて、ここで各データの配置状況（バックアップ含む）を見てみます。

pattern-1: location / key[key100]: primary[xxxxx-21429] / backup[xxxxx-38730]
pattern-1: location / key[key10]: primary[xxxxx-21429] / backup[xxxxx-43836]
pattern-1: location / key[key11]: primary[xxxxx-21429] / backup[xxxxx-43836]
pattern-1: location / key[key12]: primary[xxxxx-21429] / backup[xxxxx-38730]
pattern-1: location / key[key13]: primary[xxxxx-43836] / backup[xxxxx-38730]
pattern-1: location / key[key14]: primary[xxxxx-21429] / backup[xxxxx-43836]
pattern-1: location / key[key15]: primary[xxxxx-21429] / backup[xxxxx-38730]
pattern-1: location / key[key16]: primary[xxxxx-43836] / backup[xxxxx-21429]
pattern-1: location / key[key17]: primary[xxxxx-43836] / backup[xxxxx-38730]
pattern-1: location / key[key18]: primary[xxxxx-38730] / backup[xxxxx-43836]
pattern-1: location / key[key19]: primary[xxxxx-38730] / backup[xxxxx-43836]
pattern-1: location / key[key1]: primary[xxxxx-21429] / backup[xxxxx-38730]
pattern-1: location / key[key20]: primary[xxxxx-38730] / backup[xxxxx-21429]
pattern-1: location / key[key21]: primary[xxxxx-21429] / backup[xxxxx-38730]
pattern-1: location / key[key22]: primary[xxxxx-21429] / backup[xxxxx-43836]
pattern-1: location / key[key23]: primary[xxxxx-38730] / backup[xxxxx-43836]
pattern-1: location / key[key24]: primary[xxxxx-38730] / backup[xxxxx-21429]
pattern-1: location / key[key25]: primary[xxxxx-43836] / backup[xxxxx-38730]
pattern-1: location / key[key26]: primary[xxxxx-43836] / backup[xxxxx-38730]
pattern-1: location / key[key27]: primary[xxxxx-21429] / backup[xxxxx-38730]
pattern-1: location / key[key28]: primary[xxxxx-43836] / backup[xxxxx-38730]
pattern-1: location / key[key29]: primary[xxxxx-43836] / backup[xxxxx-21429]
pattern-1: location / key[key2]: primary[xxxxx-38730] / backup[xxxxx-21429]
pattern-1: location / key[key30]: primary[xxxxx-21429] / backup[xxxxx-43836]
pattern-1: location / key[key31]: primary[xxxxx-38730] / backup[xxxxx-21429]
pattern-1: location / key[key32]: primary[xxxxx-43836] / backup[xxxxx-38730]
pattern-1: location / key[key33]: primary[xxxxx-21429] / backup[xxxxx-43836]
pattern-1: location / key[key34]: primary[xxxxx-21429] / backup[xxxxx-38730]
pattern-1: location / key[key35]: primary[xxxxx-38730] / backup[xxxxx-21429]
pattern-1: location / key[key36]: primary[xxxxx-43836] / backup[xxxxx-38730]
pattern-1: location / key[key37]: primary[xxxxx-43836] / backup[xxxxx-38730]
pattern-1: location / key[key38]: primary[xxxxx-21429] / backup[xxxxx-43836]
pattern-1: location / key[key39]: primary[xxxxx-38730] / backup[xxxxx-43836]
pattern-1: location / key[key3]: primary[xxxxx-38730] / backup[xxxxx-21429]
pattern-1: location / key[key40]: primary[xxxxx-43836] / backup[xxxxx-21429]
pattern-1: location / key[key41]: primary[xxxxx-43836] / backup[xxxxx-38730]
pattern-1: location / key[key42]: primary[xxxxx-43836] / backup[xxxxx-38730]
pattern-1: location / key[key43]: primary[xxxxx-43836] / backup[xxxxx-21429]
pattern-1: location / key[key44]: primary[xxxxx-38730] / backup[xxxxx-21429]
pattern-1: location / key[key45]: primary[xxxxx-38730] / backup[xxxxx-21429]
pattern-1: location / key[key46]: primary[xxxxx-38730] / backup[xxxxx-21429]
pattern-1: location / key[key47]: primary[xxxxx-21429] / backup[xxxxx-38730]
pattern-1: location / key[key48]: primary[xxxxx-43836] / backup[xxxxx-21429]
pattern-1: location / key[key49]: primary[xxxxx-21429] / backup[xxxxx-43836]
pattern-1: location / key[key4]: primary[xxxxx-43836] / backup[xxxxx-38730]
pattern-1: location / key[key50]: primary[xxxxx-43836] / backup[xxxxx-21429]
pattern-1: location / key[key51]: primary[xxxxx-43836] / backup[xxxxx-38730]
pattern-1: location / key[key52]: primary[xxxxx-21429] / backup[xxxxx-43836]
pattern-1: location / key[key53]: primary[xxxxx-38730] / backup[xxxxx-21429]
pattern-1: location / key[key54]: primary[xxxxx-21429] / backup[xxxxx-38730]
pattern-1: location / key[key55]: primary[xxxxx-38730] / backup[xxxxx-21429]
pattern-1: location / key[key56]: primary[xxxxx-43836] / backup[xxxxx-21429]
pattern-1: location / key[key57]: primary[xxxxx-43836] / backup[xxxxx-38730]
pattern-1: location / key[key58]: primary[xxxxx-38730] / backup[xxxxx-43836]
pattern-1: location / key[key59]: primary[xxxxx-43836] / backup[xxxxx-38730]
pattern-1: location / key[key5]: primary[xxxxx-43836] / backup[xxxxx-38730]
pattern-1: location / key[key60]: primary[xxxxx-43836] / backup[xxxxx-38730]
pattern-1: location / key[key61]: primary[xxxxx-21429] / backup[xxxxx-43836]
pattern-1: location / key[key62]: primary[xxxxx-43836] / backup[xxxxx-38730]
pattern-1: location / key[key63]: primary[xxxxx-43836] / backup[xxxxx-21429]
pattern-1: location / key[key64]: primary[xxxxx-38730] / backup[xxxxx-43836]
pattern-1: location / key[key65]: primary[xxxxx-21429] / backup[xxxxx-43836]
pattern-1: location / key[key66]: primary[xxxxx-43836] / backup[xxxxx-21429]
pattern-1: location / key[key67]: primary[xxxxx-43836] / backup[xxxxx-21429]
pattern-1: location / key[key68]: primary[xxxxx-21429] / backup[xxxxx-38730]
pattern-1: location / key[key69]: primary[xxxxx-21429] / backup[xxxxx-38730]
pattern-1: location / key[key6]: primary[xxxxx-43836] / backup[xxxxx-38730]
pattern-1: location / key[key70]: primary[xxxxx-43836] / backup[xxxxx-21429]
pattern-1: location / key[key71]: primary[xxxxx-21429] / backup[xxxxx-43836]
pattern-1: location / key[key72]: primary[xxxxx-38730] / backup[xxxxx-43836]
pattern-1: location / key[key73]: primary[xxxxx-21429] / backup[xxxxx-38730]
pattern-1: location / key[key74]: primary[xxxxx-38730] / backup[xxxxx-43836]
pattern-1: location / key[key75]: primary[xxxxx-38730] / backup[xxxxx-21429]
pattern-1: location / key[key76]: primary[xxxxx-21429] / backup[xxxxx-43836]
pattern-1: location / key[key77]: primary[xxxxx-38730] / backup[xxxxx-21429]
pattern-1: location / key[key78]: primary[xxxxx-38730] / backup[xxxxx-21429]
pattern-1: location / key[key79]: primary[xxxxx-21429] / backup[xxxxx-43836]
pattern-1: location / key[key7]: primary[xxxxx-21429] / backup[xxxxx-38730]
pattern-1: location / key[key80]: primary[xxxxx-38730] / backup[xxxxx-21429]
pattern-1: location / key[key81]: primary[xxxxx-21429] / backup[xxxxx-38730]
pattern-1: location / key[key82]: primary[xxxxx-43836] / backup[xxxxx-38730]
pattern-1: location / key[key83]: primary[xxxxx-38730] / backup[xxxxx-21429]
pattern-1: location / key[key84]: primary[xxxxx-21429] / backup[xxxxx-38730]
pattern-1: location / key[key85]: primary[xxxxx-38730] / backup[xxxxx-43836]
pattern-1: location / key[key86]: primary[xxxxx-38730] / backup[xxxxx-21429]
pattern-1: location / key[key87]: primary[xxxxx-43836] / backup[xxxxx-38730]
pattern-1: location / key[key88]: primary[xxxxx-21429] / backup[xxxxx-43836]
pattern-1: location / key[key89]: primary[xxxxx-38730] / backup[xxxxx-21429]
pattern-1: location / key[key8]: primary[xxxxx-21429] / backup[xxxxx-38730]
pattern-1: location / key[key90]: primary[xxxxx-43836] / backup[xxxxx-38730]
pattern-1: location / key[key91]: primary[xxxxx-38730] / backup[xxxxx-21429]
pattern-1: location / key[key92]: primary[xxxxx-21429] / backup[xxxxx-38730]
pattern-1: location / key[key93]: primary[xxxxx-38730] / backup[xxxxx-43836]
pattern-1: location / key[key94]: primary[xxxxx-43836] / backup[xxxxx-21429]
pattern-1: location / key[key95]: primary[xxxxx-43836] / backup[xxxxx-38730]
pattern-1: location / key[key96]: primary[xxxxx-43836] / backup[xxxxx-38730]
pattern-1: location / key[key97]: primary[xxxxx-43836] / backup[xxxxx-38730]
pattern-1: location / key[key98]: primary[xxxxx-38730] / backup[xxxxx-43836]
pattern-1: location / key[key99]: primary[xxxxx-38730] / backup[xxxxx-21429]
pattern-1: location / key[key9]: primary[xxxxx-21429] / backup[xxxxx-38730]

…さて、実際どうなってるんでしょう。一部のキーを、ピックアップしてフォーカスしてみましょう。

Nodeの構成は、こうでしたね。

• Cache Server × 2 … xxxxx-43836、xxxxx-21429
• Functional Map APIを起動するNode … xxxxx-38730

最初に選ぶキーは、key10。

配置はこう。

pattern-1: location / key[key10]: primary[xxxxx-21429] / backup[xxxxx-43836]

結果は…

パターン	Mapの種類	実行されたNode
単一キー指定	ReadOnlyMap	xxxxx-43836、xxxxx-21429
単一キー指定	WriteOnlyMap	xxxxx-21429
単一キー指定	ReadWriteMap	xxxxx-21429
複数キー指定	ReadOnlyMap	xxxxx-21429
複数キー指定	WriteOnlyMap	xxxxx-43836、xxxxx-21429
複数キー指定	ReadWriteMap	xxxxx-43836、xxxxx-21429

なんか、単一キー指定の時と複数キー指定の時、それからMapの種類で、実行されるNodeの数が裏返っているような…？

ただ、データのあるNodeで実行されることには変わりなさそうですね。

もうひとつ確認。key20。

配置はこう。

pattern-1: location / key[key20]: primary[xxxxx-38730] / backup[xxxxx-21429]

結果は…

パターン	Mapの種類	実行されたNode
単一キー指定	ReadOnlyMap	xxxxx-38730
単一キー指定	WriteOnlyMap	xxxxx-38730
単一キー指定	ReadWriteMap	xxxxx-38730
複数キー指定	ReadOnlyMap	xxxxx-38730
複数キー指定	WriteOnlyMap	xxxxx-38730、xxxxx-21429
複数キー指定	ReadWriteMap	xxxxx-38730、xxxxx-21429

今度は、単一キー指定時のReadOnlyMapの実行が、単一Nodeになりました…。

とはいえ、やっぱり実行されるのはデータを持つNodeです。

というわけで、Functional Map APIを使った操作は、データを持つNodeで実行されることが確認できました、と。

もうちょっとソースコードを見てみる

動作からはデータの配置に合わせて、データを持つNode上で処理が実行されることは確認できましたが、もうちょっと
ソースコードから追ってみましょう。
※いずれも、トランザクションを有効にしていないケースでのトレースをしています

各Mapで、以下のような箇所でローカル実行するか、リモートに投げるかが決まってそうな感じです。

ReadOnlyMapの場合。
https://github.com/infinispan/infinispan/blob/9.4.0.Final/core/src/main/java/org/infinispan/interceptors/distribution/BaseDistributionInterceptor.java#L837-L843 https://github.com/infinispan/infinispan/blob/9.4.0.Final/core/src/main/java/org/infinispan/interceptors/distribution/BaseDistributionInterceptor.java#L858-L860

どうして複数NodeでReadOnlyMapが実行されていたかというと、ここでRPCをデータを持つ全Nodeに対して
実行しているからですね。
https://github.com/infinispan/infinispan/blob/9.4.0.Final/core/src/main/java/org/infinispan/interceptors/distribution/BaseDistributionInterceptor.java#L858-L860

WriteOnlyMapの場合。
https://github.com/infinispan/infinispan/blob/9.4.0.Final/core/src/main/java/org/infinispan/interceptors/distribution/TriangleDistributionInterceptor.java#L156-L165 https://github.com/infinispan/infinispan/blob/9.4.0.Final/core/src/main/java/org/infinispan/interceptors/distribution/TriangleDistributionInterceptor.java#L180-L204

単一キーの場合ですが、ローカルで実行するか他のNodeに転送するかは、このあたりで判断しています。
https://github.com/infinispan/infinispan/blob/9.4.0.Final/core/src/main/java/org/infinispan/interceptors/distribution/TriangleDistributionInterceptor.java#L398-L411

必要に応じて、別Nodeに転送。
https://github.com/infinispan/infinispan/blob/9.4.0.Final/core/src/main/java/org/infinispan/interceptors/distribution/TriangleDistributionInterceptor.java#L523-L539

複数キーの場合は、このあたりで。
https://github.com/infinispan/infinispan/blob/9.4.0.Final/core/src/main/java/org/infinispan/interceptors/distribution/TriangleDistributionInterceptor.java#L249-L256

この場合、OwnerとBackupに、一気にコマンドを投げているみたい…それで、複数Nodeで実行されているのかなぁと。

ReadWriteMapの場合。
https://github.com/infinispan/infinispan/blob/9.4.0.Final/core/src/main/java/org/infinispan/interceptors/distribution/TriangleDistributionInterceptor.java#L145-L154

https://github.com/infinispan/infinispan/blob/9.4.0.Final/core/src/main/java/org/infinispan/interceptors/distribution/TriangleDistributionInterceptor.java#L206-L231

実行Node選択のロジックは、WriteOnlyMapと同じです。

WriteOnlyMapおよびReadWriteMapの場合は、ローカルNodeにデータがあることを前提にしています。
特に、単一キー操作の場合は。
https://github.com/infinispan/infinispan/blob/9.4.0.Final/core/src/main/java/org/infinispan/commands/functional/WriteOnlyKeyValueCommand.java#L92-L93 https://github.com/infinispan/infinispan/blob/9.4.0.Final/core/src/main/java/org/infinispan/commands/functional/WriteOnlyKeyValueCommand.java#L92-L93 https://github.com/infinispan/infinispan/blob/9.4.0.Final/core/src/main/java/org/infinispan/commands/functional/ReadWriteKeyCommand.java#L97-L98 https://github.com/infinispan/infinispan/blob/9.4.0.Final/core/src/main/java/org/infinispan/commands/functional/ReadWriteKeyValueCommand.java#L111-L112

ReadOnlyMapの場合は、そうでもないみたいですね。

あと、やっぱりロックはしているようで。
https://github.com/infinispan/infinispan/blob/9.4.0.Final/core/src/main/java/org/infinispan/interceptors/locking/AbstractLockingInterceptor.java#L125-L126 https://github.com/infinispan/infinispan/blob/9.4.0.Final/core/src/main/java/org/infinispan/interceptors/locking/NonTransactionalLockingInterceptor.java#L55-L56

まとめ

Functional Map APIで、処理がどのNodeで実行されるのか、その挙動を確認してみました。

初めて使った時は、なんとなく「データがあるNodeで実行されるのだろう」とは思っていましたが、あんまり深く追うことは
していなかったので、良い勉強になりました。

今回作成したソースコードは、こちらに置いています。

https://github.com/kazuhira-r/infinispan-getting-started/tree/master/embedded-functional-map-locality