客戶(hù)提了這么個(gè)問(wèn)題:admin server因?yàn)閛ut of memory導(dǎo)致性能很差,此時(shí)managed server連接admin server的話(huà),可能因?yàn)榫W(wǎng)絡(luò)原因或者admin server因?yàn)镺OM而導(dǎo)致的低性能,最終導(dǎo)致managed server很多線(xiàn)程掛死在socket read上。如果沒(méi)有超時(shí)機(jī)制,這樣的線(xiàn)程會(huì)一直掛著,如果這樣的線(xiàn)程很多的話(huà),系統(tǒng)線(xiàn)程資源逐漸會(huì)被消耗完,從而引起managed server無(wú)法提供響應(yīng)。下面是這個(gè)客戶(hù)的線(xiàn)程堆棧,
Thread-90 "[STUCK] ExecuteThread: '6' for queue: 'weblogic.kernel.Default (self-tuning)'" <alive, in native, suspended, priority=1, DAEMON> {
jrockit.net.SocketNativeIO.readBytesPinned(SocketNativeIO.java:???)
jrockit.net.SocketNativeIO.socketRead(SocketNativeIO.java:25)
java.net.SocketInputStream.socketRead0(SocketInputStream.java:???)
java.net.SocketInputStream.read(SocketInputStream.java:107)
java.io.BufferedInputStream.fill(BufferedInputStream.java:189)
java.io.BufferedInputStream.read(BufferedInputStream.java:234)
^-- Holding lock: java.io.BufferedInputStream@5cc3faf[thin lock]
weblogic.net.http.MessageHeader.isHTTP(MessageHeader.java:214)
weblogic.net.http.MessageHeader.parseHeader(MessageHeader.java:141)
weblogic.net.http.HttpClient.parseHTTP(HttpClient.java:477)
^-- Holding lock: java.io.BufferedInputStream@5cc3faf[thin lock]
weblogic.net.http.HttpURLConnection.getInputStream(HttpURLConnection.java:329)
^-- Holding lock: java.io.BufferedInputStream@5cc3faf[thin lock]
weblogic.rjvm.http.HTTPClientJVMConnection.connect(HTTPClientJVMConnection.java:161)
^-- Holding lock: java.io.BufferedInputStream@5cc3faf[thin lock]
weblogic.rjvm.http.HTTPClientJVMConnection.createConnection(HTTPClientJVMConnection.java:86)
weblogic.rjvm.ConnectionManager.createConnection(ConnectionManager.java:1723)
weblogic.rjvm.ConnectionManager.findOrCreateConnection(ConnectionManager.java:1330)
^-- Holding lock: java.io.BufferedInputStream@5cc3faf[thin lock]
^-- Holding lock: weblogic.net.http.HttpClient@5cc3ea0[thin lock]
weblogic.rjvm.ConnectionManager.bootstrap(ConnectionManager.java:440)
weblogic.rjvm.ConnectionManager.bootstrap(ConnectionManager.java:315)
weblogic.rjvm.RJVMManager.findOrCreateRemoteInternal(RJVMManager.java:220)
^-- Holding lock: java.io.BufferedInputStream@5cc3faf[thin lock]
weblogic.rjvm.RJVMManager.findOrCreate(RJVMManager.java:206)
weblogic.rjvm.RJVMFinder.findOrCreateRemoteServer(RJVMFinder.java:226)
weblogic.rjvm.RJVMFinder.findOrCreate(RJVMFinder.java:170)
^-- Holding lock: java.io.BufferedInputStream@5cc3faf[thin lock]
weblogic.rjvm.ServerURL.findOrCreateRJVM(ServerURL.java:154)
weblogic.jndi.WLInitialContextFactoryDelegate.getInitialReference(WLInitialContextFactoryDelegate.java:384)
weblogic.jndi.Environment.getInitialReference(Environment.java:223)
weblogic.server.channels.RemoteChannelServiceImpl.registerInternal(RemoteChannelServiceImpl.java:153)
^-- Holding lock: java.io.BufferedInputStream@5cc3faf[thin lock]
weblogic.server.channels.RemoteChannelServiceImpl.access$300(RemoteChannelServiceImpl.java:46)
weblogic.server.channels.RemoteChannelServiceImpl$TimerListenerImpl.timerExpired(RemoteChannelServiceImpl.java:101)
weblogic.timers.internal.TimerImpl.run(TimerImpl.java:253)
weblogic.work.SelfTuningWorkManagerImpl$WorkAdapterImpl.run(SelfTuningWorkManagerImpl.java:464)
weblogic.work.ExecuteThread.execute(ExecuteThread.java:197)
weblogic.work.ExecuteThread.run(ExecuteThread.java:164)
現(xiàn)在的問(wèn)題是,對(duì)于這種問(wèn)題可不可以通過(guò)超時(shí)設(shè)定來(lái)釋放線(xiàn)程。weblogic中, RJVM(即server之間,可能是admin-to-managed,也可能是managed-to-managed)之間,連接的協(xié)議有兩類(lèi)五種,兩類(lèi)是http、t3,五種是http、https、t3、t3s、local。超時(shí)設(shè)定時(shí)協(xié)議層面的東西,所以我們這里只討論http和t3。
對(duì)于t3,從上面的trace可以看到,連接的創(chuàng)建從ServerURL.findOrCreateRJVM()開(kāi)始,這個(gè)方法有多種實(shí)現(xiàn),不同的實(shí)現(xiàn)使用不同的timeout,客戶(hù)端程序可以通過(guò)向environment中set一個(gè)叫做weblogic.jndi.requestTimeout的變量,如果不做設(shè)定,則使用系統(tǒng)默認(rèn)的DEFAULT_CONNECTION_TIMEOUT,這是個(gè)靜態(tài)值(0)。而在上面的stack trace中,我們可以看到,environment是在RemoteChannelServiceImpl中定義的,這個(gè)environment對(duì)于客戶(hù)而言是不可配置的,而weblogic自己的這個(gè)env中是不設(shè)定request timeotu的,也就是,無(wú)論哪種方式,connectionTimeout/readTimeout對(duì)于t3,都是不可配置的,而且默認(rèn)是沒(méi)有超時(shí)的。
而對(duì)于http,HTTPClientJVMConnection在創(chuàng)建HttpURLConnection的時(shí)候,會(huì)讀取系統(tǒng)環(huán)境變量中的如下兩個(gè)變量,
weblogic.http.client.defaultReadTimeout
weblogic.http.client.defaultConnectTimeout
如果沒(méi)有設(shè)定,這兩個(gè)變量的默認(rèn)值均為-1,即不做timeout。如果我們作了設(shè)定,這兩個(gè)值即讀超時(shí)、連接超時(shí)都會(huì)生效。這兩個(gè)值可以用于解決上述的問(wèn)題。