我做的應用是以Spring為系統的基礎框架,mysql為后臺數據庫.在tomcat上發布后,總是不能進行熱部署(reload),多次reload后,就會出OutOfMemory PermGen,
為此煩惱了很久,總于下定決心找找根源.
經過3天的不懈努力,小有成果,記錄下來
實際上下面的分析都已經沒什么用了,如果你使用tomcat6.0.26及以后的版本,我所說的這些情況都已經被處理了,并且比我處理的還要多很多.可以下載tomcat6.0.26的源代碼
看看WebappClassLoader類的處理就成了.
通過分析工具的分析(用了YourKit,以及JDK1.6/bin下的jps/jmap/jhat),發現有下面幾個方面會造成memory leak.
1.SystemClassLoader與WebappClassLoader加載的類相互引用,tomcat reload只是卸載WebappClassloader中的class,SystemClassLoader是不會卸載的(否則其他應用也停止了).但是WebappClassloader加載的類被SystemClassLoader引用的化,WebappClassloader中的相關類就不會被JVM進行垃圾收集
目前發現2種容易產生這種leak的現象.
a.在使用java.lang.ThreadLocal的時候很容易產生這種情況
b.使用jdbc驅動,而且不是在tomcat中配置的公共連接池.則java.sql.DriverManager一定會產生這種現象
ThreadLocal.set(Object),如果這個Object是WebappsClassLoader加載的,使用之后沒有做ThreadLocal.set(null)或者ThreadLocal.remove(),就會產生memory leak.
由于ThreadLocal實際上操作的是java.lang.Thread類中的ThreadLocalMap,Thread類是由SystemClassLoder加載的.而這個線程實例(main thread)在tomcat reload的時候不會銷毀重建,必然就產生了SystemClassLoder中的類引用WebappsClassLoader的類.
DriverManager也是由SystemClassLoder載入的,當初始化某個JDBC驅動的時候,會向DriverManager中注冊該驅動,通常是***.driver,例如com.mysql.jdbc.Driver
這個Driver是通過class.forName()加載的,通常也是加載到WebappClassLoader.這就出現了兩個classLoader中的類的交叉引用.導致memory leak.
解決辦法:
寫一個ServletContextListener,在contextDestroyed方法中統一刪除當前Thread的ThreadLocalMap中的內容.
public class ApplicationCleanListener implements ServletContextListener {
public void contextInitialized(ServletContextEvent event) {
}
public void contextDestroyed(ServletContextEvent event) {
//處理ThreadLocal
ThreadLocalCleanUtil.clearThreadLocals();
/*
* 如果數據故驅動是通過應用服務器(tomcat etc...)中配置的<公用>連接池,這里不需要 否則必須卸載Driver
*
* 原因: DriverManager是System classloader加載的, Driver是webappclassloader加載的,
* driver保存在DriverManager中,在reload過程中,由于system
* classloader不會銷毀,driverManager就一直保持著對driver的引用,
* driver無法卸載,與driver關聯的其他類
* ,例如DataSource,jdbcTemplate,dao,service....都無法卸載
*/
try {
System.out.println("clean jdbc Driver......");
for (Enumeration e = DriverManager.getDrivers(); e
.hasMoreElements();) {
Driver driver = (Driver) e.nextElement();
if (driver.getClass().getClassLoader() == getClass()
.getClassLoader()) {
DriverManager.deregisterDriver(driver);
}
}
} catch (Exception e) {
System.out
.println("Exception cleaning up java.sql.DriverManager's driver: "
+ e.getMessage());
}
}
}
/**
* 這個類根據
*/
public class ThreadLocalCleanUtil {
/**
* 得到當前線程組中的所有線程 description:
*
* @return
*/
private static Thread[] getThreads() {
ThreadGroup tg = Thread.currentThread().getThreadGroup();
while (tg.getParent() != null) {
tg = tg.getParent();
}
int threadCountGuess = tg.activeCount() + 50;
Thread[] threads = new Thread[threadCountGuess];
int threadCountActual = tg.enumerate(threads);
while (threadCountActual == threadCountGuess) {
threadCountGuess *= 2;
threads = new Thread[threadCountGuess];
threadCountActual = tg.enumerate(threads);
}
return threads;
}
public static void clearThreadLocals() {
ClassLoader classloader = Thread
.currentThread()
.getContextClassLoader();
Thread[] threads = getThreads();
try {
Field threadLocalsField = Thread.class
.getDeclaredField("threadLocals");
threadLocalsField.setAccessible(true);
Field inheritableThreadLocalsField = Thread.class
.getDeclaredField("inheritableThreadLocals");
inheritableThreadLocalsField.setAccessible(true);
Class tlmClass = Class
.forName("java.lang.ThreadLocal$ThreadLocalMap");
Field tableField = tlmClass.getDeclaredField("table");
tableField.setAccessible(true);
for (int i = 0; i < threads.length; ++i) {
if (threads[i] == null)
continue;
Object threadLocalMap = threadLocalsField.get(threads[i]);
clearThreadLocalMap(threadLocalMap, tableField, classloader);
threadLocalMap = inheritableThreadLocalsField.get(threads[i]);
clearThreadLocalMap(threadLocalMap, tableField, classloader);
}
} catch (Exception e) {
e.printStackTrace();
}
}
private static void clearThreadLocalMap(Object map,
Field internalTableField, ClassLoader classloader)
throws NoSuchMethodException, IllegalAccessException,
NoSuchFieldException, InvocationTargetException {
if (map != null) {
Method mapRemove = map.getClass().getDeclaredMethod("remove",
new Class[] { ThreadLocal.class });
mapRemove.setAccessible(true);
Object[] table = (Object[]) internalTableField.get(map);
int staleEntriesCount = 0;
if (table != null) {
for (int j = 0; j < table.length; ++j) {
if (table[j] != null) {
boolean remove = false;
Object key = ((Reference) table[j]).get();
if ((key != null)
&& (key.getClass().getClassLoader() == classloader)) {
remove = true;
System.out.println("clean threadLocal key,class="
+ key.getClass().getCanonicalName()
+ ",value=" + key.toString());
}
Field valueField = table[j]
.getClass()
.getDeclaredField("value");
valueField.setAccessible(true);
Object value = valueField.get(table[j]);
if ((value != null)
&& (value.getClass().getClassLoader() == classloader)) {
remove = true;
System.out.println("clean threadLocal value,class="
+ value.getClass().getCanonicalName()
+ ",value=" + value.toString());
}
if (remove) {
if (key == null)
++staleEntriesCount;
else {
mapRemove.invoke(map, new Object[] { key });
}
}
}
}
}
if (staleEntriesCount > 0) {
Method mapRemoveStale = map
.getClass()
.getDeclaredMethod("expungeStaleEntries", new Class[0]);
mapRemoveStale.setAccessible(true);
mapRemoveStale.invoke(map, new Object[0]);
}
}
}
}
2.對于使用mysql JDBC驅動的:mysql JDBC驅動會啟動一個Timer Thread,這個線程在reload的時候也是無法自動銷毀.
因此,需要強制結束這個timer
可以在 上面的ApplicationCleanListener中加入如下代碼:
try {
Class ConnectionImplClass = Thread
.currentThread()
.getContextClassLoader()
.loadClass("com.mysql.jdbc.ConnectionImpl");
if (ConnectionImplClass != null
&& ConnectionImplClass.getClassLoader() == getClass()
.getClassLoader()) {
System.out.println("clean mysql timer......");
Field f = ConnectionImplClass.getDeclaredField("cancelTimer");
f.setAccessible(true);
Timer timer = (Timer) f.get(null);
timer.cancel();
}
} catch (java.lang.ClassNotFoundException e1) {
// do nothing
} catch (Exception e) {
System.out
.println("Exception cleaning up MySQL cancellation timer: "
+ e.getMessage());
}
3.common-logging+log4j似乎也會導致leak,看網上有人說在ApplicationCleanListene6中加入這行代碼就可以:
LogFactory.release(Thread.currentThread().getContextClassLoader());
我沒試成功,懶得再找原因,直接換成了slf4j+logback,沒有問題.據說slf4j+logback的性能還要更好.
后記:
tomcat-6.0.26之前的版本(我用的是tomcat-6.0.18),加入上述ApplicationCleanListener后,多次reload,不會出現outOfMemory.
但要注意,第一次啟動后,reload一次,內存會增加,也就是看著還是由memory Leak,但是重復reload,內存始終保持在第一次reload的大小.似乎tomcat始終保留了雙WebappClassLoader.因此,配置內存要小心些,至少要保證能夠load兩倍的你的所有jar包的大小(當然,是指Perm的內存大小).
測試過程中最好加上 JVM參數 -verbosegc,這樣,在做GC的時候可以直觀的看到class被卸載.