executescript - Usando Selenium RemoteWebDriver en C#

selenium-webdriver

Question

Estoy tratando de usar el Selenium RemoteWebDriver en C #, básicamente todo lo que quiero hacer es raspar programáticamente el html de una página web después de que el javascript haya terminado de manipular el DOM (sin que aparezca una ventana del navegador).

Primero, inicié selenium-server.jar de la siguiente manera:

C: / Archivos de programa / selenium-server> java -jar "C: / Archivos de programa / selenium-server / selenium-server.jar"

13:34:46.163 INFO - Java: Sun Microsystems Inc. 19.1-b02
13:34:46.166 INFO - OS: Windows 7 6.1 amd64
13:34:46.174 INFO - v2.0 [a2], with Core v2.0 [a2]
13:34:46.277 INFO - RemoteWebDriver instances should connect to: **http://127.0.0.1:4444/wd/hub**
13:34:46.278 INFO - Version Jetty/5.1.x
13:34:46.279 INFO - Started HttpContext[/selenium-server/driver,/selenium-server
/driver]
13:34:46.280 INFO - Started HttpContext[/selenium-server,/selenium-server]
13:34:46.280 INFO - Started HttpContext[/,/]
13:34:46.311 INFO - Started org.openqa.jetty.jetty.servlet.ServletHandler@6019d0
a1
13:34:46.312 INFO - Started HttpContext[/wd,/wd]
13:34:46.316 INFO - Started SocketListener on 0.0.0.0:4444
13:34:46.316 INFO - Started org.openqa.jetty.jetty.Server@199a0c7c

Luego traté de ejecutar esta línea desde un caso de prueba:

var driver = new RemoteWebDriver(new Uri("http://127.0.0.1:4444/wd/hub"), DesiredCapabilities.Chrome());

Esta linea de errores:

Test ''Housters.Test.ScrapeTest.TestSelenium'' failed: OpenQA.Selenium.WebDriverException : Unexpected error. {"message":"java.lang.NullPointerException","localizedMessage":"java.lang.NullPointerException","cause":{"class":"java.lang.NullPointerException","stackTrace":[{"fileName":"DriverFactory.java","class":"java.lang.StackTraceElement","lineNumber":43,"className":"org.openqa.selenium.remote.server.DriverFactory","methodName":"getBestMatchFor","nativeMethod":false},{"fileName":"DriverFactory.java","class":"java.lang.StackTraceElement","lineNumber":76,"className":"org.openqa.selenium.remote.server.DriverFactory","methodName":"newInstance","nativeMethod":false},{"fileName":"Session.java","class":"java.lang.StackTraceElement","lineNumber":48,"className":"org.openqa.selenium.remote.server.Session$1","methodName":"call","nativeMethod":false},{"fileName":"Session.java","class":"java.lang.StackTraceElement","lineNumber":46,"className":"org.openqa.selenium.remote.server.Session$1","methodName":"call","nativeMethod":false},{"class":"java.lang.StackTraceElement","lineNumber":-1,"className":"java.util.concurrent.FutureTask$Sync","methodName":"innerRun","nativeMethod":false},{"class":"java.lang.StackTraceElement","lineNumber":-1,"className":"java.util.concurrent.FutureTask","methodName":"run","nativeMethod":false},{"class":"java.lang.StackTraceElement","lineNumber":-1,"className":"java.util.concurrent.ThreadPoolExecutor$Worker","methodName":"runTask","nativeMethod":false},{"class":"java.lang.StackTraceElement","lineNumber":-1,"className":"java.util.concurrent.ThreadPoolExecutor$Worker","methodName":"run","nativeMethod":false},{"class":"java.lang.StackTraceElement","lineNumber":-1,"className":"java.lang.Thread","methodName":"run","nativeMethod":false}]},"class":"java.util.concurrent.ExecutionException","stackTrace":[{"class":"java.lang.StackTraceElement","lineNumber":-1,"className":"java.util.concurrent.FutureTask$Sync","methodName":"innerGet","nativeMethod":false},{"class":"java.lang.StackTraceElement","lineNumber":-1,"className":"java.util.concurrent.FutureTask","methodName":"get","nativeMethod":false},{"fileName":"Session.java","class":"java.lang.StackTraceElement","lineNumber":68,"className":"org.openqa.selenium.remote.server.Session","methodName":"execute","nativeMethod":false},{"fileName":"Session.java","class":"java.lang.StackTraceElement","lineNumber":54,"className":"org.openqa.selenium.remote.server.Session","methodName":"<init>","nativeMethod":false},{"fileName":"DriverSessions.java","class":"java.lang.StackTraceElement","lineNumber":76,"className":"org.openqa.selenium.remote.server.DriverSessions","methodName":"newSession","nativeMethod":false},{"fileName":"NewSession.java","class":"java.lang.StackTraceElement","lineNumber":46,"className":"org.openqa.selenium.remote.server.handler.NewSession","methodName":"handle","nativeMethod":false},{"fileName":"ResultConfig.java","class":"java.lang.StackTraceElement","lineNumber":144,"className":"org.openqa.selenium.remote.server.rest.ResultConfig","methodName":"handle","nativeMethod":false},{"fileName":"DriverServlet.java","class":"java.lang.StackTraceElement","lineNumber":271,"className":"org.openqa.selenium.remote.server.DriverServlet","methodName":"handleRequest","nativeMethod":false},{"fileName":"DriverServlet.java","class":"java.lang.StackTraceElement","lineNumber":256,"className":"org.openqa.selenium.remote.server.DriverServlet","methodName":"doPost","nativeMethod":false},{"fileName":"HttpServlet.java","class":"java.lang.StackTraceElement","lineNumber":727,"className":"javax.servlet.http.HttpServlet","methodName":"service","nativeMethod":false},{"fileName":"HttpServlet.java","class":"java.lang.StackTraceElement","lineNumber":820,"className":"javax.servlet.http.HttpServlet","methodName":"service","nativeMethod":false},{"fileName":"ServletHolder.java","class":"java.lang.StackTraceElement","lineNumber":428,"className":"org.openqa.jetty.jetty.servlet.ServletHolder","methodName":"handle","nativeMethod":false},{"fileName":"ServletHandler.java","class":"java.lang.StackTraceElement","lineNumber":677,"className":"org.openqa.jetty.jetty.servlet.ServletHandler","methodName":"dispatch","nativeMethod":false},{"fileName":"ServletHandler.java","class":"java.lang.StackTraceElement","lineNumber":568,"className":"org.openqa.jetty.jetty.servlet.ServletHandler","methodName":"handle","nativeMethod":false},{"fileName":"HttpContext.java","class":"java.lang.StackTraceElement","lineNumber":1530,"className":"org.openqa.jetty.http.HttpContext","methodName":"handle","nativeMethod":false},{"fileName":"HttpContext.java","class":"java.lang.StackTraceElement","lineNumber":1482,"className":"org.openqa.jetty.http.HttpContext","methodName":"handle","nativeMethod":false},{"fileName":"HttpServer.java","class":"java.lang.StackTraceElement","lineNumber":909,"className":"org.openqa.jetty.http.HttpServer","methodName":"service","nativeMethod":false},{"fileName":"HttpConnection.java","class":"java.lang.StackTraceElement","lineNumber":820,"className":"org.openqa.jetty.http.HttpConnection","methodName":"service","nativeMethod":false},{"fileName":"HttpConnection.java","class":"java.lang.StackTraceElement","lineNumber":986,"className":"org.openqa.jetty.http.HttpConnection","methodName":"handleNext","nativeMethod":false},{"fileName":"HttpConnection.java","class":"java.lang.StackTraceElement","lineNumber":837,"className":"org.openqa.jetty.http.HttpConnection","methodName":"handle","nativeMethod":false},{"fileName":"SocketListener.java","class":"java.lang.StackTraceElement","lineNumber":245,"className":"org.openqa.jetty.http.SocketListener","methodName":"handleConnection","nativeMethod":false},{"fileName":"ThreadedServer.java","class":"java.lang.StackTraceElement","lineNumber":357,"className":"org.openqa.jetty.util.ThreadedServer","methodName":"handle","nativeMethod":false},{"fileName":"ThreadPool.java","class":"java.lang.StackTraceElement","lineNumber":534,"className":"org.openqa.jetty.util.ThreadPool$PoolThread","methodName":"run","nativeMethod":false}]}
    at OpenQA.Selenium.Remote.RemoteWebDriver.UnpackAndThrowOnError(Response errorResponse)
    at OpenQA.Selenium.Remote.RemoteWebDriver.Execute(DriverCommand driverCommandToExecute, Dictionary`2 parameters)
    at OpenQA.Selenium.Remote.RemoteWebDriver.StartSession(ICapabilities desiredCapabilities)
    at OpenQA.Selenium.Remote.RemoteWebDriver..ctor(Uri remoteAddress, ICapabilities desiredCapabilities)
    ScrapeTest.cs(36,0): at Housters.Test.ScrapeTest.TestSelenium()

En la ventana de la consola de servicio, muestra este error:

13:44:55.558 INFO - WebDriver remote server: Executing: [new session: null] at URL: /session)
13:44:55.560 INFO - WebDriver remote server: Exception: java.lang.NullPointerException

Estoy tratando de hacer esto desde Windows 7 x64. ¿Qué estoy haciendo mal? Esto parece mucho trabajo para lo que quiero hacer ...

Answer 1

Obtener el HTML de la página que le interesa es fácil, especialmente en .NET (use HttpWebRequest). Analizarlo en una estructura tipo DOM y aplicar las transformaciones indicadas por JavaScript en la página, no tanto. Requiere un navegador, o como mínimo, un analizador HTML con un motor de construcción DOM, así como un motor de scripts JS para manipular el DOM resultante. No es un problema trivial. En este punto, sus opciones son: (1) utilizar HtmlUnit, env.js o uno de los otros proyectos de navegador "sin cabeza", ninguno de los cuales requiere que WebDriver haga lo que desea hacer; o (2) vive con una ventana del navegador apareciendo. Y en el caso de (2), el uso de InternetExplorerDriver o FirefoxDriver sin procesar (o ChromeDriver en el próximo 2.0b4) es una opción más simple que usar el servidor remoto.

Answer 2

Si está intentando ejecutar Selenium2 en localhost, no necesita usar RemoveWebDriver () y Selenium Server. Puede usar lo siguiente:

WebDriver driver = new ChromeDriver();

He encontrado menos problemas que se ejecutan localmente de esta manera que cuando uso RemoveWebDriver, y debería encontrar que obtiene más información fácilmente disponible si hay un problema.

Alternativamente, puede usar HtmlUnit directamente como se describe en http://blog.stevensanderson.com/2010/03/30/using-htmlunit-on-net-for-headless-browser-automation/

Answer 3

Veo que ya tiene una respuesta aceptada, pero como responde una pregunta completamente diferente, agregaré mis 5 centavos.

Por lo tanto, si desea conectarse a un controlador web eliminado, en lugar de la línea:

var driver = new FirefoxDriver();

Tu tienes que hacer:

var driver = new RemoteWebDriver(new Uri("http://localhost:4444/wd/hub"), DesiredCapabilities.Firefox()); // instead of this url you can put the url of your remote hub

No te olvides de incluir el espacio de nombres remoto antes de la línea anterior:

using OpenQA.Selenium.Remote;