hadoop - experience - hue editor

Error de Hive en todas las consultas: no se pudo encontrar el registro asociado con el identificador de operación

Question

Obteniendo un error en todas las consultas de colmenas que se ejecutan en Hadoop.

Simplemente ejecute una instrucción select de una tabla que exista y obtenga los siguientes resultados:

Fetching results ran into the following error(s):

Couldn''t find log associated with operation handle: 
OperationHandle[opType=EXECUTE_STATEMENT,
getHandleIdentifier()=263df9c3-c076-454f-a025-23f5b919e957]

Y los registros que acompañan a la consulta son:

Estado incorrecto para la solicitud TFetchResultsReq (fetchType = 1, operationHandle = TOperationHandle (hasResultSet = False, modifiedRowCount = None, operationType = 0, operationId = THandleIdentifier (secret = ''% / x1d / x0f / x11 / xce / xf4B / xd6 / xb8 / xb5 / x8ddv / xceKV '', guid = "/ xbc5 / xd9 | / xfa / x13J'' / x86 / x8e4 / xb3 / x87 / xa0 / xf4 / x05")), orientation = 4, maxRows = -1): TFetchResultsResp (estado = TStatus (errorCode = 0, errorMessage = "No se pudo encontrar el registro asociado con el identificador de operación: OperationHandle [opType = EXECUTE_STATEMENT, getHandleIdentifier () = bc35d97c-fa13-4a27-868e-34b387a0f405]", sqlState = None, infoMessages = [" * org.apache.hive.service.cli.HiveSQLException: no se pudo encontrar el registro asociado con el identificador de operación: OperationHandle [opType = EXECUTE_STATEMENT, getHandleIdentifier () = bc35d97c-fa13-4a27-868e-34b387a0f405]: 25: 24 ", '' org.apache.hive.service.cli.operation.OperationManager: getOperationLogRowSet: OperationManager.java: 257 '','' org.apache.hive.service.cli.session.HiveSessionImpl: fetchResults: HiveSessionImpl.java: 656 '','' sun. reflejar.GeneratedMe thodAccessor29: invoke :: - 1 '','' sun.reflect.DelegatingMethodAccessorImpl: invoke: DelegatingMethodAccessorImpl.java: 43 '','' java.lang.reflect.Method: invoke: Method.java: 606 '','' org.apache.hive. service.cli.session.HiveSessionProxy: invoke: HiveSessionProxy.java: 79 '','' org.apache.hive.service.cli.session.HiveSessionProxy: acceso $ 000: HiveSessionProxy.java: 37 '','' org.apache.hive.service .cli.session.HiveSessionProxy $ 1: run: HiveSessionProxy.java: 64 '','' java.security.AccessController: doPrivileged: AccessController.java: -2 '','' javax.security.auth.Subject: doAs: Subject.java: 415 '','' org.apache.hadoop.security.UserGroupInformation: doAs: UserGroupInformation.java: 1628 '','' org.apache.hadoop.hive.shims.HadoopShimsSecure: doAs: HadoopShimsSecure.java: 536 '','' org.apache.hive .service.cli.session.HiveSessionProxy: invoke: HiveSessionProxy.java: 60 '','' com.sun.proxy. $ Proxy28: fetchResults :: - 1 '','' org.apache.hive.service.cli.CLIService: fetchResults: CLIService.java:427 '','' org.apache.hive.service.cli.thrift.ThriftCLIService: FetchResults: ThriftCLIService.java:587 '','' org.apache.hive.service.cli.thrift.TCLIService $ Processor $ FetchResults: getResult: TCLIService.java: 1553 '','' org.apache.hive.service.cli.thrift.TCLIService $ Procesador $ FetchResults: getResult: TCLIService.java: 1538 '','' org.apache.thrift.ProcessFunction: process: ProcessFunction.java: 39 '','' org.apache.thrift.TBaseProcessor: process: TBaseProcessor.java: 39 '','' org.apache.hive.service.auth.TSetIpAddressProcessor: process: TSetIpAddressProcessor.java: 56 '','' org.apache.thrift.server.TThreadPoolServer $ WorkerProcess: run: TThreadPoolServer.java: 206 '','' java.util.concurrent. ThreadPoolExecutor: runWorker: ThreadPoolExecutor.java: 1145 '','' java.util.concurrent.ThreadPoolExecutor $ Worker: ejecutar: ThreadPoolExecutor.java: 615 '','' java.lang.Thread: ejecutar: Thread.java: 744 ''], statusCode = 3), results = None, hasMoreRows = None) Estado incorrecto para la solicitud TFetchResultsReq (fetchType = 1, operationHandle = TOperationHandle (hasResultSet = False, modifiedRowCount = None, operationType = 0, operationId = THandleIdentifier (secret = ''c / xdf / xfe / xb7 / x8a / x10D / xea / xbaD / x93} B / xb5 / xc3 / xe7 '', guid ='' & = / xf9 / xc3 / xc0vEO / xa0% # / xf5 / xb9 / x19 / xe9W '')) , orientation = 0, maxRows = -1): TFetchResultsResp (status = TStatus (errorCode = 0, errorMessage = "No se pudo encontrar el registro asociado con el identificador de operación: OperationHandle [opType = EXECUTE_STATEMENT, getHandleIdentifier () = 263df9c3-c076-454f- a025-23f5b919e957] ", sqlState = None, infoMessages = [" * org.apache.hive.service.cli.HiveSQLException: no se pudo encontrar el registro asociado con el identificador de operación: OperationHandle [opType = EXECUTE_STATEMENT, getHandleIdentifier () = 263df9c3-c076 -454f-a025-23f5b919e957]: 25: 24 ", ''org.apache.hive.service.cli.operation.OperationManager: getOperationLogRowSet: OperationManager.java: 257'', ''org.apache.hive.service.cli.session. HiveSessionImpl: fetchResults: HiveSessionImpl.java: 656 '','' sun.reflect.GeneratedMethodAccessor29: invoke :: - 1 '','' sun.reflect.DelegatingMethodAccessorImpl: invoke: DelegatingMethodAccessorImpl.java: 43 '','' java.lang.reflect.Method: invoke: Method.java: 606 '','' org.apache.hive.service.cli .session.HiveSessionProxy: invoke: HiveSessionProxy.java: 79 '','' org.apache.hive.service.cli.session.HiveSessionProxy: access $ 000: HiveSessionProxy.java: 37 '','' org.apache.hive.service.cli. session.HiveSessionProxy $ 1: run: HiveSessionProxy.java: 64 '','' java.security.AccessController: doPrivileged: AccessController.java: -2 '','' javax.security.auth.Subject: doAs: Subject.java: 415 '','' org.apache.hadoop.security.UserGroupInformation: doAs: UserGroupInformation.java: 1628 '','' org.apache.hadoop.hive.shims.HadoopShimsSecure: doAs: HadoopShimsSecure.java: 536 '','' org.apache.hive.service. cli.session.HiveSessionProxy: invoke: HiveSessionProxy.java: 60 '','' com.sun.proxy. $ Proxy28: fetchResults :: - 1 '','' org.apache.hive.service.cli.CLIService: fetchResults: CLIService.java : 427 '','' org.apache.hive.service.cli.thrift.ThriftCLIService: FetchResults: ThriftCLIService.java: 587 '','' org.apache.hive.service.cli.thrift.TCLIService $ Processor $ FetchResults: getResult: TCLIService .java: 1553 '','' org.apache.hive.service.cli.thrift.TCLIService $ Processor $ Fetc hResults: getResult: TCLIService.java: 1538 '','' org.apache.thrift.ProcessFunction: process: ProcessFunction.java: 39 '','' org.apache.thrift.TBaseProcessor: process: TBaseProcessor.java: 39 '','' org. apache.hive.service.auth.TSetIpAddressProcessor: process: TSetIpAddressProcessor.java: 56 '','' org.apache.thrift.server.TThreadPoolServer $ WorkerProcess: run: TThreadPoolServer.java: 206 '','' java.util.concurrent.ThreadPoolExecutor: runWorker: ThreadPoolExecutor.java: 1145 '','' java.util.concurrent.ThreadPoolExecutor $ Worker: ejecutar: ThreadPoolExecutor.java: 615 '','' java.lang.Thread: ejecutar: Thread.java: 744 ''], statusCode = 3) , results = None, hasMoreRows = None) Estado incorrecto para la solicitud TFetchResultsReq (fetchType = 1, operationHandle = TOperationHandle (hasResultSet = False, modifiedRowCount = None, operationType = 0, operationId = THandleIdentifier (secret = ''c / xdf / xfe / xb7 / x8a / x10D / xea / xbaD / x93} B / xb5 / xc3 / xe7 '', guid ='' & = / xf9 / xc3 / xc0vEO / xa0% # / xf5 / xb9 / x19 / xe9W '')), orientation = 0, maxRows = -1): TFetchResultsResp (estado = TStatus (errorCode = 0, errorMessage = "No se pudo buscar el registro asociado con el identificador de operación: OperationHandle [opType = EXECUTE_STATEMENT, getHandleIdentifier () = 263df9c3-c076-454f-a025-23f5b919e957] ", sqlState = None, infoMessages = [" * org.apache.hive.service.cli.HiveSQLException: No se pudo encontrar el registro asociado con el identificador de operación: OperationHandle [opType = EXECUTE_STATEMENT, getHandleIdentifier () = 263df9c3-c076-454f-a025-23f5b919e957]: 25: 24 ", ''org.apache.hive.service.cli.operation.OperationManager : getOperationLogRowSet: OperationManager.java: 257 '','' org.apache.hive.service.cli.session.HiveSessionImpl: fetchResults: HiveSessionImpl.java: 656 '','' sun.reflect.GeneratedMethodAccessor29: invoke :: - 1 '','' sun .reflect.DelegatingMethodAccessorImpl: invoke: DelegatingMethodAccessorImpl.java: 43 '','' java.lang.reflect.Method: invoke: Method.java: 606 '','' org.apache.hive.service.cli.session.HiveSessionProxy: invoke: HiveSessionProxy .java: 79 '','' org.apache.hive.service.cli.session.HiveSessionProxy: acceso $ 000: HiveSessionProxy.java: 37 '','' org.apache.hive.service.cli.session.HiveSe ssionProxy $ 1: ejecutar: HiveSessionProxy.java: 64 '','' java.security.AccessController: doPrivileged: AccessController.java: -2 '','' javax.security.auth.Subject: doAs: Subject.java: 415 '','' org. apache.hadoop.security.UserGroupInformation: doAs: UserGroupInformation.java: 1628 '','' org.apache.hadoop.hive.shims.HadoopShimsSecure: doAs: HadoopShimsSecure.java: 536 '','' org.apache.hive.service.cli. session.HiveSessionProxy: invoke: HiveSessionProxy.java: 60 '','' com.sun.proxy. $ Proxy28: fetchResults :: - 1 '','' org.apache.hive.service.cli.CLIService: fetchResults: CLIService.java: 427 '','' org.apache.hive.service.cli.thrift.ThriftCLIService: FetchResults: ThriftCLIService.java: 587 '','' org.apache.hive.service.cli.thrift.TCLIService $ Processor $ FetchResults: getResult: TCLIService.java : 1553 '','' org.apache.hive.service.cli.thrift.TCLIService $ Processor $ FetchResults: getResult: TCLIService.java: 1538 '','' org.apache.thrift.ProcessFunction: process: ProcessFunction.java: 39 '', ''org.apache.thrift.TBaseProcessor: process: TBaseProcessor.java: 39'', ''org.apache.hive .service.auth.TSetIpAddressProcessor: process: TSetIpAddressProcessor.java: 56 '','' org.apache.thrift.server.TThreadPoolServer $ WorkerProcess: run: TThreadPoolServer.java: 206 '','' java.util.concurrent.ThreadPoolExecutor: runWorker: ThreadPoolExecutor .java: 1145 '','' java.util.concurrent.ThreadPoolExecutor $ Worker: ejecutar: ThreadPoolExecutor.java: 615 '','' java.lang.Thread: ejecutar: Thread.java: 744 ''], statusCode = 3), results = Ninguno, hasMoreRows = Ninguno)

Answer 1

Si está utilizando AWS EMR, agregue lo siguiente al archivo hive-site.xml en /etc/hive/conf.dist/hive-site.xml :

<property>
    <name>hive.server2.logging.operation.enabled</name>
    <value>true</value>
</property>

A continuación, inicie sesión en el nodo maestro y reinicie el servidor de hue y colmena

sudo stop hive-server2
sudo stop hue
sudo start hue
sudo start hive-server2

Esto lo arregló para mí (probado en la etiqueta de lanzamiento emr-4.6.0 ).

Answer 2

Este error puede ocurrir con el ie ("hive.server2.logging.operation.log.location") apunta a una ubicación que no existe para escribir registros.

Mira el código de OperationManager.java

OperationLog operationLog = getOperation(opHandle).getOperationLog();
if (operationLog == null) {
  throw new HiveSQLException("Couldn''t find log associated with operation handle: " + opHandle);
}

Answer 3

Si está viendo esto en una distribución de Cloudera cuando usa Hue, agregue lo siguiente al Fragmento de configuración avanzada de Hive Service (Valve de seguridad) para hive-site.xml en Cloudera Manager (o directamente a hive-site.xml para otro distros):

<property>
    <name>hive.server2.logging.operation.enabled</name>
    <value>true</value>
</property>

Answer 4

En Windows 8.1, también me enfrenté a la misma parte del problema dentro de la parte de consulta de la hive .

org.apache.hive.service.cli.HiveSQLException: Couldn''t find log associated with operation handle: OperationHandle [opType=EXECUTE_STATEMENT, getHandleIdentifier()=230485c4-a1be-4a17-94c8-a1ebce426319]

por lo tanto, fui al %HIVE_HOME%/conf/hive-site.xml y cambié el nombre de la propiedad hive.server2.logging.operation.log.location valor anterior ${system:user.name}/operation_logs para que me gustara a continuación dijo -

  <property>
    <name>hive.server2.logging.operation.log.location</name>
    <value>/tmp/${user.name}/operation_logs</value>
    <description>Top level directory where operation logs are stored if logging functionality is enabled</description>
  </property>

y también reemplazó todos los valores que contienen el ${system:user.name} a /tmp/${user.name} y esto me lleva al problema del registro de trabajos en la colmena, gracias.