java - para - directorio web-inf

No se puede cerrar el servidor Neo4j Jetty dentro de la aplicación web Tomcat

Question

Actualmente estoy trabajando en una aplicación web que utiliza Neo4j. Nuestra aplicación debe implementarse en un entorno Tomcat (requisito del cliente). Decidimos incrustar Neo4j, porque de esa manera podemos utilizar la API Java proporcionada por Neo4j, es más fácil de implementar y obtenemos un mejor rendimiento. Sin embargo, también necesitamos acceso a la API REST, porque tenemos una aplicación web de una sola página escrita en Angular que está haciendo uso actualmente de esta interfaz. Sin embargo, la base de datos Neo4j incorporada no expone la API REST. El artefacto del servidor Neo4j contiene un código que puede iniciar un servidor embarcadero con una base de datos grafica incrustada. Entonces, nuestra aplicación web Tomcat está iniciando un servidor Jetty. Podemos acceder al graphDb a través de la aplicación web desplegada en Tomcat en un puerto (8080) y la interfaz REST de Neo4j y el navegador Neo4j en otro puerto (7474). Aunque esto es un poco extraño, funciona bien, excepto cuando tratamos de detener nuestra aplicación web (por ejemplo, para la redistribución). Al cerrar nuestra aplicación web recibimos estos errores de tomcat:

SEVERE: The web application [/chainmonitor] appears to have started a thread named [GC-Monitor] but has failed to stop it. This is very likely to create a memory leak.
jun 10, 2014 6:43:17 PM org.apache.catalina.loader.WebappClassLoader clearReferencesThreads
SEVERE: The web application [/chainmonitor] appears to have started a thread named [RRD4J Sync Pool [Thread-1]] but has failed to stop it. This is very likely to create a memory leak.
jun 10, 2014 6:43:17 PM org.apache.catalina.loader.WebappClassLoader clearReferencesThreads
SEVERE: The web application [/chainmonitor] appears to have started a thread named [Statistics Gatherer[primitives]] but has failed to stop it. This is very likely to create a memory leak.
jun 10, 2014 6:43:17 PM org.apache.catalina.loader.WebappClassLoader clearReferencesThreads
SEVERE: The web application [/chainmonitor] appears to have started a thread named [pool-1-thread-1] but has failed to stop it. This is very likely to create a memory leak.
jun 10, 2014 6:43:17 PM org.apache.catalina.loader.WebappClassLoader clearReferencesThreads
SEVERE: The web application [/chainmonitor] appears to have started a thread named [pool-1-thread-2] but has failed to stop it. This is very likely to create a memory leak.
jun 10, 2014 6:43:17 PM org.apache.catalina.loader.WebappClassLoader clearReferencesThreads
SEVERE: The web application [/chainmonitor] appears to have started a thread named [DateCache] but has failed to stop it. This is very likely to create a memory leak.
jun 10, 2014 6:43:17 PM org.apache.catalina.loader.WebappClassLoader clearReferencesThreads
SEVERE: The web application [/chainmonitor] appears to have started a thread named [qtp835579386-31-selector-3] but has failed to stop it. This is very likely to create a memory leak.
jun 10, 2014 6:43:17 PM org.apache.catalina.loader.WebappClassLoader clearReferencesThreads
SEVERE: The web application [/chainmonitor] appears to have started a thread named [qtp835579386-32-selector-0] but has failed to stop it. This is very likely to create a memory leak.
jun 10, 2014 6:43:17 PM org.apache.catalina.loader.WebappClassLoader clearReferencesThreads
SEVERE: The web application [/chainmonitor] appears to have started a thread named [qtp835579386-33-selector-1] but has failed to stop it. This is very likely to create a memory leak.
jun 10, 2014 6:43:17 PM org.apache.catalina.loader.WebappClassLoader clearReferencesThreads
SEVERE: The web application [/chainmonitor] appears to have started a thread named [qtp835579386-34-selector-2] but has failed to stop it. This is very likely to create a memory leak.
jun 10, 2014 6:43:17 PM org.apache.catalina.loader.WebappClassLoader clearReferencesThreads
SEVERE: The web application [/chainmonitor] appears to have started a thread named [qtp835579386-35-selector-4] but has failed to stop it. This is very likely to create a memory leak.
jun 10, 2014 6:43:17 PM org.apache.catalina.loader.WebappClassLoader clearReferencesThreads
SEVERE: The web application [/chainmonitor] appears to have started a thread named [qtp835579386-36-selector-5] but has failed to stop it. This is very likely to create a memory leak.
jun 10, 2014 6:43:17 PM org.apache.catalina.loader.WebappClassLoader clearReferencesThreads
SEVERE: The web application [/chainmonitor] appears to have started a thread named [qtp835579386-37-acceptor-0-ServerConnector@1425c689{HTTP/1.1}{localhost:7474}] but has failed to stop it. This is very likely to create a memory leak.
jun 10, 2014 6:43:17 PM org.apache.catalina.loader.WebappClassLoader clearReferencesThreads
SEVERE: The web application [/chainmonitor] appears to have started a thread named [qtp835579386-38-acceptor-1-ServerConnector@1425c689{HTTP/1.1}{localhost:7474}] but has failed to stop it. This is very likely to create a memory leak.
jun 10, 2014 6:43:17 PM org.apache.catalina.loader.WebappClassLoader clearReferencesThreads
SEVERE: The web application [/chainmonitor] appears to have started a thread named [HashSessionScavenger-0] but has failed to stop it. This is very likely to create a memory leak.
jun 10, 2014 6:43:17 PM org.apache.catalina.loader.WebappClassLoader clearReferencesThreads
SEVERE: The web application [/chainmonitor] appears to have started a thread named [qtp835579386-54] but has failed to stop it. This is very likely to create a memory leak.
jun 10, 2014 6:43:17 PM org.apache.catalina.loader.WebappClassLoader checkThreadLocalMapForLeaks
SEVERE: The web application [/chainmonitor] created a ThreadLocal with key of type [org.eclipse.jetty.http.HttpFields$1] (value [org.eclipse.jetty.http.HttpFields$1@4a3d3a5e]) and a value of type [org.eclipse.jetty.http.HttpFields.DateGenerator] (value [org.eclipse.jetty.http.HttpFields$DateGenerator@24f0137b]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.
jun 10, 2014 6:43:17 PM org.apache.catalina.loader.WebappClassLoader checkThreadLocalMapForLeaks
SEVERE: The web application [/chainmonitor] created a ThreadLocal with key of type [scala.util.DynamicVariable$$anon$1] (value [scala.util.DynamicVariable$$anon$1@281a099d]) and a value of type [java.lang.Integer] (value [0]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.
jun 10, 2014 6:43:17 PM org.apache.catalina.loader.WebappClassLoader checkThreadLocalMapForLeaks
SEVERE: The web application [/chainmonitor] created a ThreadLocal with key of type [scala.util.DynamicVariable$$anon$1] (value [scala.util.DynamicVariable$$anon$1@281a099d]) and a value of type [java.lang.Integer] (value [0]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.
jun 10, 2014 6:43:17 PM org.apache.catalina.loader.WebappClassLoader checkThreadLocalMapForLeaks
SEVERE: The web application [/chainmonitor] created a ThreadLocal with key of type [scala.util.DynamicVariable$$anon$1] (value [scala.util.DynamicVariable$$anon$1@281a099d]) and a value of type [java.lang.Integer] (value [0]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.
jun 10, 2014 6:43:17 PM org.apache.catalina.loader.WebappClassLoader checkThreadLocalMapForLeaks
SEVERE: The web application [/chainmonitor] created a ThreadLocal with key of type [scala.util.DynamicVariable$$anon$1] (value [scala.util.DynamicVariable$$anon$1@281a099d]) and a value of type [java.lang.Integer] (value [0]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.
jun 10, 2014 6:43:17 PM org.apache.catalina.loader.WebappClassLoader checkThreadLocalMapForLeaks
SEVERE: The web application [/chainmonitor] created a ThreadLocal with key of type [scala.util.DynamicVariable$$anon$1] (value [scala.util.DynamicVariable$$anon$1@281a099d]) and a value of type [java.lang.Integer] (value [0]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.
jun 10, 2014 6:43:17 PM org.apache.catalina.loader.WebappClassLoader checkThreadLocalMapForLeaks
SEVERE: The web application [/chainmonitor] created a ThreadLocal with key of type [scala.util.DynamicVariable$$anon$1] (value [scala.util.DynamicVariable$$anon$1@281a099d]) and a value of type [java.lang.Integer] (value [0]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.
jun 10, 2014 6:43:17 PM org.apache.catalina.loader.WebappClassLoader checkThreadLocalMapForLeaks
SEVERE: The web application [/chainmonitor] created a ThreadLocal with key of type [scala.util.DynamicVariable$$anon$1] (value [scala.util.DynamicVariable$$anon$1@281a099d]) and a value of type [java.lang.Integer] (value [0]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.
jun 10, 2014 6:43:17 PM org.apache.catalina.loader.WebappClassLoader checkThreadLocalMapForLeaks
SEVERE: The web application [/chainmonitor] created a ThreadLocal with key of type [scala.util.DynamicVariable$$anon$1] (value [scala.util.DynamicVariable$$anon$1@281a099d]) and a value of type [java.lang.Integer] (value [0]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.
jun 10, 2014 6:43:17 PM org.apache.catalina.loader.WebappClassLoader checkThreadLocalMapForLeaks
SEVERE: The web application [/chainmonitor] created a ThreadLocal with key of type [scala.util.DynamicVariable$$anon$1] (value [scala.util.DynamicVariable$$anon$1@281a099d]) and a value of type [java.lang.Integer] (value [0]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.
jun 10, 2014 6:43:17 PM org.apache.catalina.loader.WebappClassLoader checkThreadLocalMapForLeaks
SEVERE: The web application [/chainmonitor] created a ThreadLocal with key of type [scala.util.DynamicVariable$$anon$1] (value [scala.util.DynamicVariable$$anon$1@281a099d]) and a value of type [java.lang.Integer] (value [0]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.
jun 10, 2014 6:43:17 PM org.apache.catalina.loader.WebappClassLoader checkThreadLocalMapForLeaks
SEVERE: The web application [/chainmonitor] created a ThreadLocal with key of type [scala.util.DynamicVariable$$anon$1] (value [scala.util.DynamicVariable$$anon$1@281a099d]) and a value of type [java.lang.Integer] (value [0]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.
jun 10, 2014 6:43:17 PM org.apache.catalina.loader.WebappClassLoader checkThreadLocalMapForLeaks
SEVERE: The web application [/chainmonitor] created a ThreadLocal with key of type [scala.util.DynamicVariable$$anon$1] (value [scala.util.DynamicVariable$$anon$1@281a099d]) and a value of type [java.lang.Integer] (value [0]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.
jun 10, 2014 6:43:17 PM org.apache.catalina.loader.WebappClassLoader checkThreadLocalMapForLeaks
SEVERE: The web application [/chainmonitor] created a ThreadLocal with key of type [scala.util.DynamicVariable$$anon$1] (value [scala.util.DynamicVariable$$anon$1@281a099d]) and a value of type [java.lang.Integer] (value [0]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.
jun 10, 2014 6:43:17 PM org.apache.catalina.loader.WebappClassLoader checkThreadLocalMapForLeaks
SEVERE: The web application [/chainmonitor] created a ThreadLocal with key of type [scala.util.DynamicVariable$$anon$1] (value [scala.util.DynamicVariable$$anon$1@281a099d]) and a value of type [java.lang.Integer] (value [0]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.
jun 10, 2014 6:43:17 PM org.apache.catalina.loader.WebappClassLoader checkThreadLocalMapForLeaks
SEVERE: The web application [/chainmonitor] created a ThreadLocal with key of type [scala.util.DynamicVariable$$anon$1] (value [scala.util.DynamicVariable$$anon$1@281a099d]) and a value of type [java.lang.Integer] (value [0]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.
jun 10, 2014 6:43:17 PM org.apache.catalina.loader.WebappClassLoader checkThreadLocalMapForLeaks
SEVERE: The web application [/chainmonitor] created a ThreadLocal with key of type [scala.util.DynamicVariable$$anon$1] (value [scala.util.DynamicVariable$$anon$1@281a099d]) and a value of type [java.lang.Integer] (value [0]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.
jun 10, 2014 6:43:17 PM org.apache.catalina.loader.WebappClassLoader checkThreadLocalMapForLeaks
SEVERE: The web application [/chainmonitor] created a ThreadLocal with key of type [scala.util.DynamicVariable$$anon$1] (value [scala.util.DynamicVariable$$anon$1@281a099d]) and a value of type [java.lang.Integer] (value [0]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.
jun 10, 2014 6:43:27 PM org.apache.catalina.startup.HostConfig undeploy
INFO: Undeploying context [/chainmonitor]
jun 10, 2014 6:43:27 PM org.apache.catalina.startup.ExpandWar deleteDir
SEVERE: [C:/dev/apache-tomcat-7.0.54/webapps/chainmonitor/WEB-INF/lib] could not be completely deleted. The presence of the remaining files may cause problems
jun 10, 2014 6:43:27 PM org.apache.catalina.startup.ExpandWar deleteDir
SEVERE: [C:/dev/apache-tomcat-7.0.54/webapps/chainmonitor/WEB-INF] could not be completely deleted. The presence of the remaining files may cause problems
jun 10, 2014 6:43:27 PM org.apache.catalina.startup.ExpandWar deleteDir
SEVERE: [C:/dev/apache-tomcat-7.0.54/webapps/chainmonitor] could not be completely deleted. The presence of the remaining files may cause problems
jun 10, 2014 6:43:27 PM org.apache.catalina.startup.ExpandWar delete
SEVERE: [C:/dev/apache-tomcat-7.0.54/webapps/chainmonitor] could not be completely deleted. The presence of the remaining files may cause problems

Nuestro pom.xml se ve así:

    <dependency>
        <groupId>org.neo4j</groupId>
        <artifactId>neo4j</artifactId>
        <version>2.0.3</version>
    </dependency>
    <dependency>
        <groupId>org.neo4j.app</groupId>
        <artifactId>neo4j-server</artifactId>
        <version>2.0.3</version>
    </dependency>

Así es como iniciamos la base de datos integrada y comenzamos el servidor de embarcadero:

graphDb = (EmbeddedGraphDatabase) new GraphDatabaseFactory().
      newEmbeddedDatabaseBuilder(dataDir).loadPropertiesFromURL(Neo4jPropertiesUrl).newGraphDatabase();
bootstrapper = new WrappingNeoServerBootstrapper(graphDb);

Actualmente neo4j-server.properties está vacío. No he probado muchas configuraciones diferentes, pero hacerlo no parece ser un problema.

Así es como dejamos la base de datos integrada y el servidor de embarcadero:

    @Override
    public void contextDestroyed(ServletContextEvent event) {
​            graphDb.shutdown();
            bootstrapper.stop();
    }

Esto aparentemente se debe a la protección de fuga de memoria de Tomcat ( http://wiki.apache.org/tomcat/MemoryLeakProtection ).

El resultado es que nuestra aplicación web solo puede redistribuirse matando por completo el proceso Tomcat (kill -9), lo cual es muy indeseable.

Hemos intentado muchas cosas:

• no cerrar el grapbDb incrustado. Sin efecto.
• no cerrando explícitamente el bootstrapper. Sin efecto. (el programa de arranque tiene su propio gancho de apagado)
• cerrando primero el bootstrapper y luego la base de datos integrada.
• inmolador de hilos: https://github.com/Neo4j/Neo4j/issues/1070 . El código se ejecuta e inmola 59 valores locales de subprocesos, pero sin efecto aparente. Curiosamente, cuando ejecuto esto en un bucle durante un minuto durante el apagado, sigue diciendo que inmolaron 6 valores. Para mí, esto es una indicación de que este proceso no funciona en absoluto, porque Neo4j tarda un tiempo mientras se apaga.
• poner en un retraso después del cierre. La razón por la que probamos esto es porque después de que tomcat haya recibido una señal de apagado, tenemos que esperar unos segundos antes de matar el proceso, de lo contrario, obtenemos un error en nuestro siguiente inicio que nuestros datos de gráfico se han corrompido (con un mensaje que no tiene se ha cerrado correctamente). Retrasar por más de 5 segundos parece tener un efecto también, porque luego solo recibimos 6 errores graves en lugar de 16. Esta es también la razón por la que creo que el proceso de inmolación no funciona en absoluto.
• Intentamos como alternativa agregar nuestro Neo4j incorporado a un clúster, pudiendo usar tanto la APLICACIÓN REST como la Neo4j incorporada. Sin embargo, después de aproximadamente cuatro horas tratando de hacer que esto funcione, nos dimos por vencidos. Durante el inicio de la base de datos Neo4j, nuestra aplicación weba se congeló. Después de algunas depuraciones, parecía un punto muerto en el código de Neo4j. Intentamos establecer todas las configuraciones de tiempo de espera de HA en algunos valores más pequeños, pero sin ningún efecto.

El problema es causado por el servidor de embarcadero Neo4j. Sin él, este problema no surge. Sin embargo, necesitamos la API REST. ¿Sabes cómo podemos solucionar esto? ¿O tiene una forma alternativa de usar la base de datos incrustada y la API REST dentro de un entorno Tomcat?

Estamos usando:

• Java: 1.7.0_60
• Tomcat 7.0.54
• Neo4j: 2.0.3 (hemos intentado 2.1.1 pero obtuvimos exactamente el mismo resultado)
• El mismo comportamiento se ha observado en Windows 7 y Linux Redhat

Answer 1

Es posible que tenga transacciones con fugas que aún no están cerradas, entonces Neo4j espera hasta 20 segundos al apagar para cerrar esas transacciones.

Si esperas más de 5 segundos, ¿cuáles de esos problemas informados se mantienen?

Answer 2

Tenemos una configuración y una necesidad muy similares. Aquí está la solución que tenemos para este problema. El JVM Runtime shutdownHook no funciona en Tomcat, por lo que mantenemos una referencia a un objeto envoltorio que contiene nuestro GraphDatabaseService y colocamos ese contenedor en un objeto del administrador del repositorio. Utilice la anotación @PreDestroy en un método en el objeto del administrador de repositorio que apagará los repositorios antes de que se destruya la clase de gestor de repositorio.

import javax.annotation.PreDestroy;

...

@PreDestroy
public void destroy() {
    log.info("Destroying Neo4jRepositoryManager...");
    shutdown(Neo4jPropertyKeys.SANDBOX_DATABASE.getUrlKey());
    shutdown(Neo4jPropertyKeys.PRODUCTION_DATABASE.getUrlKey());
    shutdown(Neo4jPropertyKeys.TEST_DATABASE.getUrlKey());
}

private void shutdown(String urlKey) {
    if(repositories.containsKey(urlKey)) {
        LabeledNodeGraphRepository graphdb = repositories.get(urlKey);
        graphdb.getGraphDatabaseService().shutdown();
        log.info(urlKey + " shutdown...");
    }
}

<bean id="repositoryManager" class="mypath.neo4j.repository.Neo4jRepositoryManager"/>

Nuestro ambiente es:

• Java 1.7.0_72
• Tomcat 7.0.56
• Neo4j 2.1.6
• Primavera 4.0.5
• Jersey 2.1.4