data big hadoop oracle11g sqoop apache-sqoop bigdata

hadoop - big - sqoop hive



Importación de Sqoop usando el conector ojdbc6 (1)

Estoy usando sqoop para importar datos de Oracle 11g, ya que no tengo el permiso para poner el jar ojdbc en la biblioteca de sqoop en clúster. Estoy proporcionando explícitamente el jar usando -libjars pero está lanzando excepción. El código que he usado es: sqoop eval -libjars /root/shared_folder/ojdbc6.jar --driver oracle.jdbc.OracleDriver --connect jdbc:oracle:thin:@127.0.0.1:1521:XE --username srivastavaaman --password manager --query ''SELECT * from TestTable1''

El resultado que sigue es: Warning: /usr/lib/sqoop/../accumulo does not exist! Accumulo imports will fail. Please set $ACCUMULO_HOME to the root of your Accumulo installation. 17/04/05 02:48:40 INFO sqoop.Sqoop: Running Sqoop version: 1.4.4.2.1.1.0-385 17/04/05 02:48:41 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead. 17/04/05 02:48:41 WARN sqoop.ConnFactory: Parameter --driver is set to an explicit driver however appropriate connection manager is not being set (via --connection-manager). Sqoop is going to fall back to org.apache.sqoop.manager.GenericJdbcManager. Please specify explicitly which connection manager should be used next time. 17/04/05 02:48:41 INFO manager.SqlManager: Using default fetchSize of 1000 17/04/05 02:48:41 ERROR sqoop.Sqoop: Got exception running Sqoop: java.lang.RuntimeException: Could not load db driver class: oracle.jdbc.OracleDriver java.lang.RuntimeException: Could not load db driver class: oracle.jdbc.OracleDriver at org.apache.sqoop.manager.SqlManager.makeConnection(SqlManager.java:797) at org.apache.sqoop.manager.GenericJdbcManager.getConnection(GenericJdbcManager.java:52) at org.apache.sqoop.tool.EvalSqlTool.run(EvalSqlTool.java:64) at org.apache.sqoop.Sqoop.run(Sqoop.java:147) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183) at org.apache.sqoop.Sqoop.runTool(Sqoop.java:222) at org.apache.sqoop.Sqoop.runTool(Sqoop.java:231) at org.apache.sqoop.Sqoop.main(Sqoop.java:240) Warning: /usr/lib/sqoop/../accumulo does not exist! Accumulo imports will fail. Please set $ACCUMULO_HOME to the root of your Accumulo installation. 17/04/05 02:48:40 INFO sqoop.Sqoop: Running Sqoop version: 1.4.4.2.1.1.0-385 17/04/05 02:48:41 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead. 17/04/05 02:48:41 WARN sqoop.ConnFactory: Parameter --driver is set to an explicit driver however appropriate connection manager is not being set (via --connection-manager). Sqoop is going to fall back to org.apache.sqoop.manager.GenericJdbcManager. Please specify explicitly which connection manager should be used next time. 17/04/05 02:48:41 INFO manager.SqlManager: Using default fetchSize of 1000 17/04/05 02:48:41 ERROR sqoop.Sqoop: Got exception running Sqoop: java.lang.RuntimeException: Could not load db driver class: oracle.jdbc.OracleDriver java.lang.RuntimeException: Could not load db driver class: oracle.jdbc.OracleDriver at org.apache.sqoop.manager.SqlManager.makeConnection(SqlManager.java:797) at org.apache.sqoop.manager.GenericJdbcManager.getConnection(GenericJdbcManager.java:52) at org.apache.sqoop.tool.EvalSqlTool.run(EvalSqlTool.java:64) at org.apache.sqoop.Sqoop.run(Sqoop.java:147) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183) at org.apache.sqoop.Sqoop.runTool(Sqoop.java:222) at org.apache.sqoop.Sqoop.runTool(Sqoop.java:231) at org.apache.sqoop.Sqoop.main(Sqoop.java:240)

Si el mismo ojdbc6.jar se coloca en la lib del sqoop en la máquina local, funciona correctamente.

El argumento -libjars no se usa normalmente con Sqoop, sino que se agrega como parte del sistema interno de análisis argumental de Hadoop.

Agregue la ruta del archivo Jar a la variable $HADOOP_CLASSPATH .

export HADOOP_CLASSPATH=$HADOOP_CLASSPATH:/root/shared_folder/ojdbc6.jar sqoop eval --driver oracle.jdbc.OracleDriver --connect jdbc:oracle:thin:@127.0.0.1:1521:XE --username ****** --password ***** --query ''SELECT * from TestTable1''