Seleniumを使っていて、ネットワーク部分のログが取れないのかな?と思いまして。
ChromeやFirefoxでいう、下のキャプチャの赤で囲った部分が欲しい、と。
で、これを見てLogTypeのperformanceを使用すれば取得できるのでは?
これを見ると、どうもFirefoxでしかダメっぽい感じが。
というわけで、ちょっと試してみます。
あ、前提として、アクセス対象のWebサーバーは起動済みとします。キャプチャしているのは、Ubuntu Linuxのデフォルト状態のApache(SSL有効)です。
Maven依存関係
利用するMaven依存関係は、こちら。
<dependency> <groupId>org.seleniumhq.selenium</groupId> <artifactId>selenium-java</artifactId> <version>2.48.2</version> </dependency>
プログラム
とりあえず、FirefoxDriverを使ったプログラムを書いてみます。
src/main/java/org/littlewings/selenium/App.java
package org.littlewings.selenium; import java.util.concurrent.TimeUnit; import java.util.logging.Level; import org.openqa.selenium.firefox.FirefoxDriver; import org.openqa.selenium.logging.LogType; import org.openqa.selenium.logging.LoggingPreferences; import org.openqa.selenium.logging.Logs; import org.openqa.selenium.remote.CapabilityType; import org.openqa.selenium.remote.DesiredCapabilities; public class App { public static void main(String... args) throws Exception { LoggingPreferences logs = new LoggingPreferences(); logs.enable(LogType.PERFORMANCE, Level.ALL); DesiredCapabilities capabilities = DesiredCapabilities .firefox(); capabilities.setCapability(CapabilityType.LOGGING_PREFS, logs); FirefoxDriver driver = new FirefoxDriver(capabilities); try { driver.get("https://localhost/"); TimeUnit.SECONDS.sleep(5L); Logs log = driver.manage().logs(); log.get(LogType.PERFORMANCE).getAll().forEach(System.out::println); } finally { driver.quit(); } } }
待機を雑にTimeUnit#sleepを使ってしまいましたが…。
DesiredCapabilitiesに、LoggingPreferencesを設定すればよいみたいです。
LoggingPreferences logs = new LoggingPreferences(); logs.enable(LogType.PERFORMANCE, Level.ALL); DesiredCapabilities capabilities = DesiredCapabilities .firefox(); capabilities.setCapability(CapabilityType.LOGGING_PREFS, logs); FirefoxDriver driver = new FirefoxDriver(capabilities);
参考)
GitHub - SeleniumHQ/selenium-google-code-issue-archive: Archive, please see main selenium repo
これを実行してみると…
何も出なかったりします。
あれ…?
で、いろいろ調べてこういうのにたどり着きまして。
RemoteWebDriver?むしろChromeの方がいい?
というわけで、ダウンロードページよりSelenium Standalone ServerとGoogle Chrome Driverをダウンロード。
これを使って進めてみます。
RemoteWebDriver+Google Chrome Driverで
ダウンロードしたGoogle Chrome Driverを展開。
$ unzip chromedriver_linux64.zip
chromedriverというファイルが現れるので、このパスを覚えておきます。
また、Selenium Standalone Serverも起動しておきます。
$ java -jar ~/path/to/selenium-server-standalone-2.48.2.jar
で、先ほどのプログラムをこのように修正。
System.setProperty("webdriver.chrome.driver", "/path/to/chromedriver"); DesiredCapabilities capabilities = DesiredCapabilities .chrome(); capabilities.setCapability(CapabilityType.LOGGING_PREFS, logs); //RemoteWebDriver driver = new RemoteWebDriver(capabilities); // デフォルトのアクセス先はこちら。 RemoteWebDriver driver = new RemoteWebDriver(new URL("http://localhost:4444/wd/hub"), capabilities);
DesiredCapabilitiesをchromeに、FirefoxDriverをRemoteWebDriverに変更しました。
今回、Selenium Standalone Serverのアクセスは明示しましたが、デフォルトのままなら特に書かなくてもOKです。
実行ログ。
[2016-01-01T22:02:22+0900] [INFO] {"message":{"method":"Page.loadEventFired","params":{"timestamp":195096.433474}},"webview":"C91F4E2C-E670-4AEB-8A77-35889CF96BCA"} [2016-01-01T22:02:22+0900] [INFO] {"message":{"method":"Page.frameStoppedLoading","params":{"frameId":"71261.2"}},"webview":"C91F4E2C-E670-4AEB-8A77-35889CF96BCA"} [2016-01-01T22:02:22+0900] [INFO] {"message":{"method":"Page.domContentEventFired","params":{"timestamp":195096.433702}},"webview":"C91F4E2C-E670-4AEB-8A77-35889CF96BCA"} [2016-01-01T22:02:22+0900] [INFO] {"message":{"method":"Page.frameResized","params":{}},"webview":"C91F4E2C-E670-4AEB-8A77-35889CF96BCA"} [2016-01-01T22:02:22+0900] [INFO] {"message":{"method":"Page.frameStartedLoading","params":{"frameId":"71287.2"}},"webview":"C91F4E2C-E670-4AEB-8A77-35889CF96BCA"} [2016-01-01T22:02:22+0900] [INFO] {"message":{"method":"Network.requestWillBeSent","params":{"documentURL":"https://localhost/","frameId":"71287.2","initiator":{"type":"other"},"loaderId":"71287.3","request":{"headers":{"Accept":"text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8","Upgrade-Insecure-Requests":"1","User-Agent":"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/47.0.2526.106 Safari/537.36","X-DevTools-Emulate-Network-Conditions-Client-Id":"C91F4E2C-E670-4AEB-8A77-35889CF96BCA"},"initialPriority":"VeryHigh","method":"GET","mixedContentType":"none","url":"https://localhost/"},"requestId":"71287.1","timestamp":195096.72467,"type":"Document","wallTime":1451653342.3939}},"webview":"C91F4E2C-E670-4AEB-8A77-35889CF96BCA"} [2016-01-01T22:02:22+0900] [INFO] {"message":{"method":"Network.responseReceived","params":{"frameId":"71287.2","loaderId":"71287.3","requestId":"71287.1","response":{"connectionId":19,"connectionReused":false,"encodedDataLength":-1,"fromDiskCache":false,"fromServiceWorker":false,"headers":{"Accept-Ranges":"bytes","Connection":"Keep-Alive","Content-Encoding":"gzip","Content-Length":"3256","Content-Type":"text/html","Date":"Fri, 01 Jan 2016 13:02:22 GMT","ETag":"\"2cf6-52629d6ee2840-gzip\"","Keep-Alive":"timeout=5, max=100","Last-Modified":"Sat, 05 Dec 2015 17:20:25 GMT","Server":"Apache/2.4.7 (Ubuntu)","Vary":"Accept-Encoding"},"headersText":"HTTP/1.1 200 OK\r\nDate: Fri, 01 Jan 2016 13:02:22 GMT\r\nServer: Apache/2.4.7 (Ubuntu)\r\nLast-Modified: Sat, 05 Dec 2015 17:20:25 GMT\r\nETag: \"2cf6-52629d6ee2840-gzip\"\r\nAccept-Ranges: bytes\r\nVary: Accept-Encoding\r\nContent-Encoding: gzip\r\nContent-Length: 3256\r\nKeep-Alive: timeout=5, max=100\r\nConnection: Keep-Alive\r\nContent-Type: text/html\r\n\r\n","mimeType":"text/html","protocol":"http/1.1","remoteIPAddress":"::1","remotePort":443,"requestHeaders":{"Accept":"text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8","Accept-Encoding":"gzip, deflate, sdch","Accept-Language":"ja,en-US;q=0.8,en;q=0.6","Connection":"keep-alive","Host":"localhost","Upgrade-Insecure-Requests":"1","User-Agent":"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/47.0.2526.106 Safari/537.36"},"requestHeadersText":"GET / HTTP/1.1\r\nHost: localhost\r\nConnection: keep-alive\r\nAccept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8\r\nUpgrade-Insecure-Requests: 1\r\nUser-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/47.0.2526.106 Safari/537.36\r\nAccept-Encoding: gzip, deflate, sdch\r\nAccept-Language: ja,en-US;q=0.8,en;q=0.6\r\n\r\n","securityDetails":{"certificateId":1,"cipher":"AES_128_GCM","keyExchange":"ECDHE_RSA","protocol":"TLS 1.2"},"securityState":"insecure","status":200,"statusText":"OK","timing":{"connectEnd":30.2149999770336,"connectStart":26.7519999761134,"dnsEnd":26.7519999761134,"dnsStart":26.7469999962486,"proxyEnd":-1,"proxyStart":-1,"receiveHeadersEnd":31.4489999727812,"requestTime":195096.725219,"sendEnd":31.1719999881461,"sendStart":30.2999999839813,"sslEnd":30.200999986846,"sslStart":26.912999979686,"workerReady":-1,"workerStart":-1},"url":"https://localhost/"},"timestamp":195096.763386,"type":"Document"}},"webview":"C91F4E2C-E670-4AEB-8A77-35889CF96BCA"} [2016-01-01T22:02:22+0900] [INFO] {"message":{"method":"Network.dataReceived","params":{"dataLength":11510,"encodedDataLength":3594,"requestId":"71287.1","timestamp":195096.763675}},"webview":"C91F4E2C-E670-4AEB-8A77-35889CF96BCA"} [2016-01-01T22:02:22+0900] [INFO] {"message":{"method":"Page.frameNavigated","params":{"frame":{"id":"71287.2","loaderId":"71287.3","mimeType":"text/html","securityOrigin":"https://localhost","url":"https://localhost/"}}},"webview":"C91F4E2C-E670-4AEB-8A77-35889CF96BCA"} [2016-01-01T22:02:22+0900] [INFO] {"message":{"method":"Network.loadingFinished","params":{"encodedDataLength":3594,"requestId":"71287.1","timestamp":195096.757206}},"webview":"C91F4E2C-E670-4AEB-8A77-35889CF96BCA"} [2016-01-01T22:02:22+0900] [INFO] {"message":{"method":"Network.requestWillBeSent","params":{"documentURL":"https://localhost/","frameId":"71287.2","initiator":{"lineNumber":196,"type":"parser","url":"https://localhost/"},"loaderId":"71287.3","request":{"headers":{"Accept":"image/webp,image/*,*/*;q=0.8","Referer":"https://localhost/","User-Agent":"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/47.0.2526.106 Safari/537.36","X-DevTools-Emulate-Network-Conditions-Client-Id":"C91F4E2C-E670-4AEB-8A77-35889CF96BCA"},"initialPriority":"VeryLow","method":"GET","mixedContentType":"none","url":"https://localhost/icons/ubuntu-logo.png"},"requestId":"71287.2","timestamp":195096.906463,"type":"Other","wallTime":1451653342.5757}},"webview":"C91F4E2C-E670-4AEB-8A77-35889CF96BCA"} [2016-01-01T22:02:22+0900] [INFO] {"message":{"method":"Network.requestWillBeSent","params":{"documentURL":"https://localhost/","frameId":"71287.2","initiator":{"lineNumber":373,"type":"parser","url":"https://localhost/"},"loaderId":"71287.3","request":{"headers":{"Accept":"image/webp,image/*,*/*;q=0.8","User-Agent":"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/47.0.2526.106 Safari/537.36","X-DevTools-Emulate-Network-Conditions-Client-Id":"C91F4E2C-E670-4AEB-8A77-35889CF96BCA"},"initialPriority":"VeryLow","method":"GET","mixedContentType":"optionally-blockable","url":"http://www.w3.org/Icons/valid-xhtml10"},"requestId":"71287.3","timestamp":195096.906692,"type":"Other","wallTime":1451653342.57592}},"webview":"C91F4E2C-E670-4AEB-8A77-35889CF96BCA"} [2016-01-01T22:02:22+0900] [INFO] {"message":{"method":"Page.domContentEventFired","params":{"timestamp":195096.93919}},"webview":"C91F4E2C-E670-4AEB-8A77-35889CF96BCA"} [2016-01-01T22:02:22+0900] [INFO] {"message":{"method":"Network.responseReceived","params":{"frameId":"71287.2","loaderId":"71287.3","requestId":"71287.2","response":{"connectionId":19,"connectionReused":true,"encodedDataLength":-1,"fromDiskCache":false,"fromServiceWorker":false,"headers":{"Accept-Ranges":"bytes","Connection":"Keep-Alive","Content-Length":"3404","Content-Type":"image/png","Date":"Fri, 01 Jan 2016 13:02:22 GMT","ETag":"\"d4c-5221145d61640\"","Keep-Alive":"timeout=5, max=99","Last-Modified":"Wed, 14 Oct 2015 14:21:05 GMT","Server":"Apache/2.4.7 (Ubuntu)"},"headersText":"HTTP/1.1 200 OK\r\nDate: Fri, 01 Jan 2016 13:02:22 GMT\r\nServer: Apache/2.4.7 (Ubuntu)\r\nLast-Modified: Wed, 14 Oct 2015 14:21:05 GMT\r\nETag: \"d4c-5221145d61640\"\r\nAccept-Ranges: bytes\r\nContent-Length: 3404\r\nKeep-Alive: timeout=5, max=99\r\nConnection: Keep-Alive\r\nContent-Type: image/png\r\n\r\n","mimeType":"image/png","protocol":"http/1.1","remoteIPAddress":"::1","remotePort":443,"requestHeaders":{"Accept":"image/webp,image/*,*/*;q=0.8","Accept-Encoding":"gzip, deflate, sdch","Accept-Language":"ja,en-US;q=0.8,en;q=0.6","Connection":"keep-alive","Host":"localhost","Referer":"https://localhost/","User-Agent":"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/47.0.2526.106 Safari/537.36"},"requestHeadersText":"GET /icons/ubuntu-logo.png HTTP/1.1\r\nHost: localhost\r\nConnection: keep-alive\r\nAccept: image/webp,image/*,*/*;q=0.8\r\nUser-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/47.0.2526.106 Safari/537.36\r\nReferer: https://localhost/\r\nAccept-Encoding: gzip, deflate, sdch\r\nAccept-Language: ja,en-US;q=0.8,en;q=0.6\r\n\r\n","securityDetails":{"certificateId":1,"cipher":"AES_128_GCM","keyExchange":"ECDHE_RSA","protocol":"TLS 1.2"},"securityState":"insecure","status":200,"statusText":"OK","timing":{"connectEnd":-1,"connectStart":-1,"dnsEnd":-1,"dnsStart":-1,"proxyEnd":-1,"proxyStart":-1,"receiveHeadersEnd":2.73700000252575,"requestTime":195096.906742,"sendEnd":1.06500001857057,"sendStart":0.939999998081476,"sslEnd":-1,"sslStart":-1,"workerReady":-1,"workerStart":-1},"url":"https://localhost/icons/ubuntu-logo.png"},"timestamp":195096.975156,"type":"Image"}},"webview":"C91F4E2C-E670-4AEB-8A77-35889CF96BCA"} [2016-01-01T22:02:22+0900] [INFO] {"message":{"method":"Network.dataReceived","params":{"dataLength":3404,"encodedDataLength":3688,"requestId":"71287.2","timestamp":195096.975468}},"webview":"C91F4E2C-E670-4AEB-8A77-35889CF96BCA"} [2016-01-01T22:02:22+0900] [INFO] {"message":{"method":"Network.loadingFinished","params":{"encodedDataLength":3688,"requestId":"71287.2","timestamp":195096.909911}},"webview":"C91F4E2C-E670-4AEB-8A77-35889CF96BCA"} [2016-01-01T22:02:22+0900] [INFO] {"message":{"method":"Network.responseReceived","params":{"frameId":"71287.2","loaderId":"71287.3","requestId":"71287.3","response":{"connectionId":28,"connectionReused":false,"encodedDataLength":-1,"fromDiskCache":false,"fromServiceWorker":false,"headers":{"Accept-Ranges":"bytes","Cache-Control":"max-age=2592000","Content-Length":"1882","Content-Location":"valid-xhtml10.png","Content-Type":"image/png; qs=0.7","Date":"Fri, 01 Jan 2016 13:02:16 GMT","ETag":"\"75a-41880ced83900;520010edf82df\"","Expires":"Sat, 23 Jan 2016 21:01:40 GMT","Last-Modified":"Fri, 14 Jul 2006 01:43:32 GMT","P3P":"policyref=\"http://www.w3.org/2014/08/p3p.xml\"","Server":"Apache/2","TCN":"choice","Vary":"negotiate,accept"},"headersText":"HTTP/1.1 200 OK\r\nServer: Apache/2\r\nContent-Location: valid-xhtml10.png\r\nVary: negotiate,accept\r\nTCN: choice\r\nLast-Modified: Fri, 14 Jul 2006 01:43:32 GMT\r\nETag: \"75a-41880ced83900;520010edf82df\"\r\nCache-Control: max-age=2592000\r\nExpires: Sat, 23 Jan 2016 21:01:40 GMT\r\nP3P: policyref=\"http://www.w3.org/2014/08/p3p.xml\"\r\nContent-Type: image/png; qs=0.7\r\nContent-Length: 1882\r\nAccept-Ranges: bytes\r\nDate: Fri, 01 Jan 2016 13:02:16 GMT\r\n\r\n","mimeType":"image/png","protocol":"http/1.1","remoteIPAddress":"128.30.52.100","remotePort":80,"requestHeaders":{"Accept":"image/webp,image/*,*/*;q=0.8","Accept-Encoding":"gzip, deflate, sdch","Accept-Language":"ja,en-US;q=0.8,en;q=0.6","Connection":"keep-alive","Host":"www.w3.org","User-Agent":"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/47.0.2526.106 Safari/537.36"},"requestHeadersText":"GET /Icons/valid-xhtml10 HTTP/1.1\r\nHost: www.w3.org\r\nConnection: keep-alive\r\nAccept: image/webp,image/*,*/*;q=0.8\r\nUser-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/47.0.2526.106 Safari/537.36\r\nAccept-Encoding: gzip, deflate, sdch\r\nAccept-Language: ja,en-US;q=0.8,en;q=0.6\r\n\r\n","securityState":"neutral","status":200,"statusText":"OK","timing":{"connectEnd":203.525000018999,"connectStart":11.1829999950714,"dnsEnd":11.1829999950714,"dnsStart":0.533000013092533,"proxyEnd":-1,"proxyStart":-1,"receiveHeadersEnd":398.987000022316,"requestTime":195096.906979,"sendEnd":203.67499999702,"sendStart":203.586000017822,"sslEnd":-1,"sslStart":-1,"workerReady":-1,"workerStart":-1},"url":"http://www.w3.org/Icons/valid-xhtml10"},"timestamp":195097.306625,"type":"Image"}},"webview":"C91F4E2C-E670-4AEB-8A77-35889CF96BCA"} [2016-01-01T22:02:22+0900] [INFO] {"message":{"method":"Network.dataReceived","params":{"dataLength":1024,"encodedDataLength":1460,"requestId":"71287.3","timestamp":195097.306783}},"webview":"C91F4E2C-E670-4AEB-8A77-35889CF96BCA"} [2016-01-01T22:02:22+0900] [INFO] {"message":{"method":"Network.dataReceived","params":{"dataLength":858,"encodedDataLength":858,"requestId":"71287.3","timestamp":195097.306852}},"webview":"C91F4E2C-E670-4AEB-8A77-35889CF96BCA"} [2016-01-01T22:02:22+0900] [INFO] {"message":{"method":"Network.loadingFinished","params":{"encodedDataLength":2318,"requestId":"71287.3","timestamp":195097.306827}},"webview":"C91F4E2C-E670-4AEB-8A77-35889CF96BCA"} [2016-01-01T22:02:22+0900] [INFO] {"message":{"method":"Page.loadEventFired","params":{"timestamp":195097.307169}},"webview":"C91F4E2C-E670-4AEB-8A77-35889CF96BCA"} [2016-01-01T22:02:22+0900] [INFO] {"message":{"method":"Page.frameStoppedLoading","params":{"frameId":"71287.2"}},"webview":"C91F4E2C-E670-4AEB-8A77-35889CF96BCA"} [2016-01-01T22:02:22+0900] [INFO] {"message":{"method":"Network.requestWillBeSent","params":{"documentURL":"https://localhost/","frameId":"71287.2","initiator":{"type":"other"},"loaderId":"71287.3","request":{"headers":{"Referer":"https://localhost/","User-Agent":"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/47.0.2526.106 Safari/537.36","X-DevTools-Emulate-Network-Conditions-Client-Id":"C91F4E2C-E670-4AEB-8A77-35889CF96BCA"},"initialPriority":"Medium","method":"GET","mixedContentType":"none","url":"https://localhost/favicon.ico"},"requestId":"71287.4","timestamp":195097.30913,"type":"Other","wallTime":1451653342.97836}},"webview":"C91F4E2C-E670-4AEB-8A77-35889CF96BCA"}
自分の最初に挙げた目的だと、「"method":"Network.requestWillBeSent"」や「"method":"Network.responseReceived"」などを見れば良さそうですね。
このログの内容は、LogEntryからtimestamp、level、messageとして取れますが、mesageは思いきりStringなので、JSONパースしたりするとよいでしょう。
でも、なんでFirefoxだとダメなんでしょう…。