CLOVER🍀

That was when it all began.

Selenium RemoteWebDriverで、LogType.PERFORMANCEを取得する

Java Selenium

Seleniumを使っていて、ネットワーク部分のログが取れないのかな？と思いまして。

ChromeやFirefoxでいう、下のキャプチャの赤で囲った部分が欲しい、と。

で、これを見てLogTypeのperformanceを使用すれば取得できるのでは？

Log Types

これを見ると、どうもFirefoxでしかダメっぽい感じが。

Browser Specific Behavior

というわけで、ちょっと試してみます。

あ、前提として、アクセス対象のWebサーバーは起動済みとします。キャプチャしているのは、Ubuntu Linuxのデフォルト状態のApache（SSL有効）です。

Maven依存関係

利用するMaven依存関係は、こちら。

        <dependency>
            <groupId>org.seleniumhq.selenium</groupId>
            <artifactId>selenium-java</artifactId>
            <version>2.48.2</version>
        </dependency>

プログラム

とりあえず、FirefoxDriverを使ったプログラムを書いてみます。
src/main/java/org/littlewings/selenium/App.java

package org.littlewings.selenium;

import java.util.concurrent.TimeUnit;
import java.util.logging.Level;

import org.openqa.selenium.firefox.FirefoxDriver;
import org.openqa.selenium.logging.LogType;
import org.openqa.selenium.logging.LoggingPreferences;
import org.openqa.selenium.logging.Logs;
import org.openqa.selenium.remote.CapabilityType;
import org.openqa.selenium.remote.DesiredCapabilities;

public class App {
    public static void main(String... args) throws Exception {
        LoggingPreferences logs = new LoggingPreferences();
        logs.enable(LogType.PERFORMANCE, Level.ALL);

        DesiredCapabilities capabilities =
                DesiredCapabilities
                        .firefox();
        capabilities.setCapability(CapabilityType.LOGGING_PREFS, logs);

        FirefoxDriver driver = new FirefoxDriver(capabilities);

        try {
            driver.get("https://localhost/");

            TimeUnit.SECONDS.sleep(5L);

            Logs log = driver.manage().logs();
            log.get(LogType.PERFORMANCE).getAll().forEach(System.out::println);
        } finally {
            driver.quit();
        }
    }
}

待機を雑にTimeUnit#sleepを使ってしまいましたが…。

DesiredCapabilitiesに、LoggingPreferencesを設定すればよいみたいです。

        LoggingPreferences logs = new LoggingPreferences();
        logs.enable(LogType.PERFORMANCE, Level.ALL);

        DesiredCapabilities capabilities =
                DesiredCapabilities
                        .firefox();
        capabilities.setCapability(CapabilityType.LOGGING_PREFS, logs);

        FirefoxDriver driver = new FirefoxDriver(capabilities);

参考）
GitHub - SeleniumHQ/selenium-google-code-issue-archive: Archive, please see main selenium repo

これを実行してみると…

何も出なかったりします。
あれ…？
で、いろいろ調べてこういうのにたどり着きまして。
RemoteWebDriver to get LogType=performance produces exception, works fine with ChromeDriver · Issue #8457 · SeleniumHQ/selenium-google-code-issue-archive · GitHub
RemoteWebDriver？むしろChromeの方がいい？
というわけで、ダウンロードページよりSelenium Standalone ServerとGoogle Chrome Driverをダウンロード。
Downloads
これを使って進めてみます。






    
RemoteWebDriver＋Google Chrome Driverで

    
ダウンロードしたGoogle Chrome Driverを展開。


$ unzip chromedriver_linux64.zip

chromedriverというファイルが現れるので、このパスを覚えておきます。
また、Selenium Standalone Serverも起動しておきます。


$ java -jar ~/path/to/selenium-server-standalone-2.48.2.jar

で、先ほどのプログラムをこのように修正。


        System.setProperty("webdriver.chrome.driver", "/path/to/chromedriver");

        DesiredCapabilities capabilities =
                DesiredCapabilities
                        .chrome();
        capabilities.setCapability(CapabilityType.LOGGING_PREFS, logs);

        //RemoteWebDriver driver = new RemoteWebDriver(capabilities);  // デフォルトのアクセス先はこちら。
        RemoteWebDriver driver = new RemoteWebDriver(new URL("http://localhost:4444/wd/hub"), capabilities);

DesiredCapabilitiesをchromeに、FirefoxDriverをRemoteWebDriverに変更しました。
今回、Selenium Standalone Serverのアクセスは明示しましたが、デフォルトのままなら特に書かなくてもOKです。
実行ログ。


[2016-01-01T22:02:22+0900] [INFO] {"message":{"method":"Page.loadEventFired","params":{"timestamp":195096.433474}},"webview":"C91F4E2C-E670-4AEB-8A77-35889CF96BCA"}
[2016-01-01T22:02:22+0900] [INFO] {"message":{"method":"Page.frameStoppedLoading","params":{"frameId":"71261.2"}},"webview":"C91F4E2C-E670-4AEB-8A77-35889CF96BCA"}
[2016-01-01T22:02:22+0900] [INFO] {"message":{"method":"Page.domContentEventFired","params":{"timestamp":195096.433702}},"webview":"C91F4E2C-E670-4AEB-8A77-35889CF96BCA"}
[2016-01-01T22:02:22+0900] [INFO] {"message":{"method":"Page.frameResized","params":{}},"webview":"C91F4E2C-E670-4AEB-8A77-35889CF96BCA"}
[2016-01-01T22:02:22+0900] [INFO] {"message":{"method":"Page.frameStartedLoading","params":{"frameId":"71287.2"}},"webview":"C91F4E2C-E670-4AEB-8A77-35889CF96BCA"}
[2016-01-01T22:02:22+0900] [INFO] {"message":{"method":"Network.requestWillBeSent","params":{"documentURL":"https://localhost/","frameId":"71287.2","initiator":{"type":"other"},"loaderId":"71287.3","request":{"headers":{"Accept":"text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8","Upgrade-Insecure-Requests":"1","User-Agent":"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/47.0.2526.106 Safari/537.36","X-DevTools-Emulate-Network-Conditions-Client-Id":"C91F4E2C-E670-4AEB-8A77-35889CF96BCA"},"initialPriority":"VeryHigh","method":"GET","mixedContentType":"none","url":"https://localhost/"},"requestId":"71287.1","timestamp":195096.72467,"type":"Document","wallTime":1451653342.3939}},"webview":"C91F4E2C-E670-4AEB-8A77-35889CF96BCA"}
[2016-01-01T22:02:22+0900] [INFO] {"message":{"method":"Network.responseReceived","params":{"frameId":"71287.2","loaderId":"71287.3","requestId":"71287.1","response":{"connectionId":19,"connectionReused":false,"encodedDataLength":-1,"fromDiskCache":false,"fromServiceWorker":false,"headers":{"Accept-Ranges":"bytes","Connection":"Keep-Alive","Content-Encoding":"gzip","Content-Length":"3256","Content-Type":"text/html","Date":"Fri, 01 Jan 2016 13:02:22 GMT","ETag":"\"2cf6-52629d6ee2840-gzip\"","Keep-Alive":"timeout=5, max=100","Last-Modified":"Sat, 05 Dec 2015 17:20:25 GMT","Server":"Apache/2.4.7 (Ubuntu)","Vary":"Accept-Encoding"},"headersText":"HTTP/1.1 200 OK\r\nDate: Fri, 01 Jan 2016 13:02:22 GMT\r\nServer: Apache/2.4.7 (Ubuntu)\r\nLast-Modified: Sat, 05 Dec 2015 17:20:25 GMT\r\nETag: \"2cf6-52629d6ee2840-gzip\"\r\nAccept-Ranges: bytes\r\nVary: Accept-Encoding\r\nContent-Encoding: gzip\r\nContent-Length: 3256\r\nKeep-Alive: timeout=5, max=100\r\nConnection: Keep-Alive\r\nContent-Type: text/html\r\n\r\n","mimeType":"text/html","protocol":"http/1.1","remoteIPAddress":"::1","remotePort":443,"requestHeaders":{"Accept":"text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8","Accept-Encoding":"gzip, deflate, sdch","Accept-Language":"ja,en-US;q=0.8,en;q=0.6","Connection":"keep-alive","Host":"localhost","Upgrade-Insecure-Requests":"1","User-Agent":"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/47.0.2526.106 Safari/537.36"},"requestHeadersText":"GET / HTTP/1.1\r\nHost: localhost\r\nConnection: keep-alive\r\nAccept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8\r\nUpgrade-Insecure-Requests: 1\r\nUser-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/47.0.2526.106 Safari/537.36\r\nAccept-Encoding: gzip, deflate, sdch\r\nAccept-Language: ja,en-US;q=0.8,en;q=0.6\r\n\r\n","securityDetails":{"certificateId":1,"cipher":"AES_128_GCM","keyExchange":"ECDHE_RSA","protocol":"TLS 1.2"},"securityState":"insecure","status":200,"statusText":"OK","timing":{"connectEnd":30.2149999770336,"connectStart":26.7519999761134,"dnsEnd":26.7519999761134,"dnsStart":26.7469999962486,"proxyEnd":-1,"proxyStart":-1,"receiveHeadersEnd":31.4489999727812,"requestTime":195096.725219,"sendEnd":31.1719999881461,"sendStart":30.2999999839813,"sslEnd":30.200999986846,"sslStart":26.912999979686,"workerReady":-1,"workerStart":-1},"url":"https://localhost/"},"timestamp":195096.763386,"type":"Document"}},"webview":"C91F4E2C-E670-4AEB-8A77-35889CF96BCA"}
[2016-01-01T22:02:22+0900] [INFO] {"message":{"method":"Network.dataReceived","params":{"dataLength":11510,"encodedDataLength":3594,"requestId":"71287.1","timestamp":195096.763675}},"webview":"C91F4E2C-E670-4AEB-8A77-35889CF96BCA"}
[2016-01-01T22:02:22+0900] [INFO] {"message":{"method":"Page.frameNavigated","params":{"frame":{"id":"71287.2","loaderId":"71287.3","mimeType":"text/html","securityOrigin":"https://localhost","url":"https://localhost/"}}},"webview":"C91F4E2C-E670-4AEB-8A77-35889CF96BCA"}
[2016-01-01T22:02:22+0900] [INFO] {"message":{"method":"Network.loadingFinished","params":{"encodedDataLength":3594,"requestId":"71287.1","timestamp":195096.757206}},"webview":"C91F4E2C-E670-4AEB-8A77-35889CF96BCA"}
[2016-01-01T22:02:22+0900] [INFO] {"message":{"method":"Network.requestWillBeSent","params":{"documentURL":"https://localhost/","frameId":"71287.2","initiator":{"lineNumber":196,"type":"parser","url":"https://localhost/"},"loaderId":"71287.3","request":{"headers":{"Accept":"image/webp,image/*,*/*;q=0.8","Referer":"https://localhost/","User-Agent":"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/47.0.2526.106 Safari/537.36","X-DevTools-Emulate-Network-Conditions-Client-Id":"C91F4E2C-E670-4AEB-8A77-35889CF96BCA"},"initialPriority":"VeryLow","method":"GET","mixedContentType":"none","url":"https://localhost/icons/ubuntu-logo.png"},"requestId":"71287.2","timestamp":195096.906463,"type":"Other","wallTime":1451653342.5757}},"webview":"C91F4E2C-E670-4AEB-8A77-35889CF96BCA"}
[2016-01-01T22:02:22+0900] [INFO] {"message":{"method":"Network.requestWillBeSent","params":{"documentURL":"https://localhost/","frameId":"71287.2","initiator":{"lineNumber":373,"type":"parser","url":"https://localhost/"},"loaderId":"71287.3","request":{"headers":{"Accept":"image/webp,image/*,*/*;q=0.8","User-Agent":"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/47.0.2526.106 Safari/537.36","X-DevTools-Emulate-Network-Conditions-Client-Id":"C91F4E2C-E670-4AEB-8A77-35889CF96BCA"},"initialPriority":"VeryLow","method":"GET","mixedContentType":"optionally-blockable","url":"http://www.w3.org/Icons/valid-xhtml10"},"requestId":"71287.3","timestamp":195096.906692,"type":"Other","wallTime":1451653342.57592}},"webview":"C91F4E2C-E670-4AEB-8A77-35889CF96BCA"}
[2016-01-01T22:02:22+0900] [INFO] {"message":{"method":"Page.domContentEventFired","params":{"timestamp":195096.93919}},"webview":"C91F4E2C-E670-4AEB-8A77-35889CF96BCA"}
[2016-01-01T22:02:22+0900] [INFO] {"message":{"method":"Network.responseReceived","params":{"frameId":"71287.2","loaderId":"71287.3","requestId":"71287.2","response":{"connectionId":19,"connectionReused":true,"encodedDataLength":-1,"fromDiskCache":false,"fromServiceWorker":false,"headers":{"Accept-Ranges":"bytes","Connection":"Keep-Alive","Content-Length":"3404","Content-Type":"image/png","Date":"Fri, 01 Jan 2016 13:02:22 GMT","ETag":"\"d4c-5221145d61640\"","Keep-Alive":"timeout=5, max=99","Last-Modified":"Wed, 14 Oct 2015 14:21:05 GMT","Server":"Apache/2.4.7 (Ubuntu)"},"headersText":"HTTP/1.1 200 OK\r\nDate: Fri, 01 Jan 2016 13:02:22 GMT\r\nServer: Apache/2.4.7 (Ubuntu)\r\nLast-Modified: Wed, 14 Oct 2015 14:21:05 GMT\r\nETag: \"d4c-5221145d61640\"\r\nAccept-Ranges: bytes\r\nContent-Length: 3404\r\nKeep-Alive: timeout=5, max=99\r\nConnection: Keep-Alive\r\nContent-Type: image/png\r\n\r\n","mimeType":"image/png","protocol":"http/1.1","remoteIPAddress":"::1","remotePort":443,"requestHeaders":{"Accept":"image/webp,image/*,*/*;q=0.8","Accept-Encoding":"gzip, deflate, sdch","Accept-Language":"ja,en-US;q=0.8,en;q=0.6","Connection":"keep-alive","Host":"localhost","Referer":"https://localhost/","User-Agent":"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/47.0.2526.106 Safari/537.36"},"requestHeadersText":"GET /icons/ubuntu-logo.png HTTP/1.1\r\nHost: localhost\r\nConnection: keep-alive\r\nAccept: image/webp,image/*,*/*;q=0.8\r\nUser-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/47.0.2526.106 Safari/537.36\r\nReferer: https://localhost/\r\nAccept-Encoding: gzip, deflate, sdch\r\nAccept-Language: ja,en-US;q=0.8,en;q=0.6\r\n\r\n","securityDetails":{"certificateId":1,"cipher":"AES_128_GCM","keyExchange":"ECDHE_RSA","protocol":"TLS 1.2"},"securityState":"insecure","status":200,"statusText":"OK","timing":{"connectEnd":-1,"connectStart":-1,"dnsEnd":-1,"dnsStart":-1,"proxyEnd":-1,"proxyStart":-1,"receiveHeadersEnd":2.73700000252575,"requestTime":195096.906742,"sendEnd":1.06500001857057,"sendStart":0.939999998081476,"sslEnd":-1,"sslStart":-1,"workerReady":-1,"workerStart":-1},"url":"https://localhost/icons/ubuntu-logo.png"},"timestamp":195096.975156,"type":"Image"}},"webview":"C91F4E2C-E670-4AEB-8A77-35889CF96BCA"}
[2016-01-01T22:02:22+0900] [INFO] {"message":{"method":"Network.dataReceived","params":{"dataLength":3404,"encodedDataLength":3688,"requestId":"71287.2","timestamp":195096.975468}},"webview":"C91F4E2C-E670-4AEB-8A77-35889CF96BCA"}
[2016-01-01T22:02:22+0900] [INFO] {"message":{"method":"Network.loadingFinished","params":{"encodedDataLength":3688,"requestId":"71287.2","timestamp":195096.909911}},"webview":"C91F4E2C-E670-4AEB-8A77-35889CF96BCA"}
[2016-01-01T22:02:22+0900] [INFO] {"message":{"method":"Network.responseReceived","params":{"frameId":"71287.2","loaderId":"71287.3","requestId":"71287.3","response":{"connectionId":28,"connectionReused":false,"encodedDataLength":-1,"fromDiskCache":false,"fromServiceWorker":false,"headers":{"Accept-Ranges":"bytes","Cache-Control":"max-age=2592000","Content-Length":"1882","Content-Location":"valid-xhtml10.png","Content-Type":"image/png; qs=0.7","Date":"Fri, 01 Jan 2016 13:02:16 GMT","ETag":"\"75a-41880ced83900;520010edf82df\"","Expires":"Sat, 23 Jan 2016 21:01:40 GMT","Last-Modified":"Fri, 14 Jul 2006 01:43:32 GMT","P3P":"policyref=\"http://www.w3.org/2014/08/p3p.xml\"","Server":"Apache/2","TCN":"choice","Vary":"negotiate,accept"},"headersText":"HTTP/1.1 200 OK\r\nServer: Apache/2\r\nContent-Location: valid-xhtml10.png\r\nVary: negotiate,accept\r\nTCN: choice\r\nLast-Modified: Fri, 14 Jul 2006 01:43:32 GMT\r\nETag: \"75a-41880ced83900;520010edf82df\"\r\nCache-Control: max-age=2592000\r\nExpires: Sat, 23 Jan 2016 21:01:40 GMT\r\nP3P: policyref=\"http://www.w3.org/2014/08/p3p.xml\"\r\nContent-Type: image/png; qs=0.7\r\nContent-Length: 1882\r\nAccept-Ranges: bytes\r\nDate: Fri, 01 Jan 2016 13:02:16 GMT\r\n\r\n","mimeType":"image/png","protocol":"http/1.1","remoteIPAddress":"128.30.52.100","remotePort":80,"requestHeaders":{"Accept":"image/webp,image/*,*/*;q=0.8","Accept-Encoding":"gzip, deflate, sdch","Accept-Language":"ja,en-US;q=0.8,en;q=0.6","Connection":"keep-alive","Host":"www.w3.org","User-Agent":"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/47.0.2526.106 Safari/537.36"},"requestHeadersText":"GET /Icons/valid-xhtml10 HTTP/1.1\r\nHost: www.w3.org\r\nConnection: keep-alive\r\nAccept: image/webp,image/*,*/*;q=0.8\r\nUser-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/47.0.2526.106 Safari/537.36\r\nAccept-Encoding: gzip, deflate, sdch\r\nAccept-Language: ja,en-US;q=0.8,en;q=0.6\r\n\r\n","securityState":"neutral","status":200,"statusText":"OK","timing":{"connectEnd":203.525000018999,"connectStart":11.1829999950714,"dnsEnd":11.1829999950714,"dnsStart":0.533000013092533,"proxyEnd":-1,"proxyStart":-1,"receiveHeadersEnd":398.987000022316,"requestTime":195096.906979,"sendEnd":203.67499999702,"sendStart":203.586000017822,"sslEnd":-1,"sslStart":-1,"workerReady":-1,"workerStart":-1},"url":"http://www.w3.org/Icons/valid-xhtml10"},"timestamp":195097.306625,"type":"Image"}},"webview":"C91F4E2C-E670-4AEB-8A77-35889CF96BCA"}
[2016-01-01T22:02:22+0900] [INFO] {"message":{"method":"Network.dataReceived","params":{"dataLength":1024,"encodedDataLength":1460,"requestId":"71287.3","timestamp":195097.306783}},"webview":"C91F4E2C-E670-4AEB-8A77-35889CF96BCA"}
[2016-01-01T22:02:22+0900] [INFO] {"message":{"method":"Network.dataReceived","params":{"dataLength":858,"encodedDataLength":858,"requestId":"71287.3","timestamp":195097.306852}},"webview":"C91F4E2C-E670-4AEB-8A77-35889CF96BCA"}
[2016-01-01T22:02:22+0900] [INFO] {"message":{"method":"Network.loadingFinished","params":{"encodedDataLength":2318,"requestId":"71287.3","timestamp":195097.306827}},"webview":"C91F4E2C-E670-4AEB-8A77-35889CF96BCA"}
[2016-01-01T22:02:22+0900] [INFO] {"message":{"method":"Page.loadEventFired","params":{"timestamp":195097.307169}},"webview":"C91F4E2C-E670-4AEB-8A77-35889CF96BCA"}
[2016-01-01T22:02:22+0900] [INFO] {"message":{"method":"Page.frameStoppedLoading","params":{"frameId":"71287.2"}},"webview":"C91F4E2C-E670-4AEB-8A77-35889CF96BCA"}
[2016-01-01T22:02:22+0900] [INFO] {"message":{"method":"Network.requestWillBeSent","params":{"documentURL":"https://localhost/","frameId":"71287.2","initiator":{"type":"other"},"loaderId":"71287.3","request":{"headers":{"Referer":"https://localhost/","User-Agent":"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/47.0.2526.106 Safari/537.36","X-DevTools-Emulate-Network-Conditions-Client-Id":"C91F4E2C-E670-4AEB-8A77-35889CF96BCA"},"initialPriority":"Medium","method":"GET","mixedContentType":"none","url":"https://localhost/favicon.ico"},"requestId":"71287.4","timestamp":195097.30913,"type":"Other","wallTime":1451653342.97836}},"webview":"C91F4E2C-E670-4AEB-8A77-35889CF96BCA"}
自分の最初に挙げた目的だと、「"method":"Network.requestWillBeSent"」や「"method":"Network.responseReceived"」などを見れば良さそうですね。

このログの内容は、LogEntryからtimestamp、level、messageとして取れますが、mesageは思いきりStringなので、JSONパースしたりするとよいでしょう。
でも、なんでFirefoxだとダメなんでしょう…。