When to generate ? How to generate ? how to debug ?
If you get unexplained server hangs under WebSphere, you can obtain, from the WebSphere server, a thread dump to help diagnose the problem.
In the case of a server hang, you can force an application to create a thread dump.
On unix/Linux machines find the process id (PID) of the hung JVM and issue kill -3 PID. Look for an output file in the installation root directory with a name like javacore.date.time.id.txt.
Using wasadmin prompt,
get the handle of the server
wsadmin>set jvm [$AdminControl completeObjectName type=JVM,process=server1,*]
wsadmin>$AdminControl invoke $jvm dumpThreadsRead More »
IBM Says, they have a very low overhead monitoring tool solution to do the task. It is Java Health Center or known as HC.
“Health Center is a very low overhead monitoring tool. It runs alongside an IBM Java application with a very small impact on the application’s performance. Health Center monitors several application areas, using the information to provide recommendations and analysis that help you improve the performance and efficiency of your application. Health Center can save the data obtained from monitoring an application and load it again for analysis at a later date.”
Health Center provides visibility, monitoring and profiling in the following application areas:
Java method profiling: The Health Center uses a sampling method profiler to diagnose applications showing high CPU usage. It’s low overhead which means there is no need to specify in advance which parts of the application to monitor, the Health Center simply monitors everything. It works without recompilation or byte code instrumentation and shows where the application is spending its time, by giving full call stack information for all sampled methods.
Lock analysis: Synchonization can be a big performance bottleneck on multi-CPU systems. It is often difficult to identify a hot lock or assess the impact locking is having on your application. Health Center records all locking activity and identifies the objects with most contention. Health Center analyses this information, and uses it to provide guidance about whether synchronization is impacting performance
Garbage collection: The performance of Garbage Collection (GC) affects the entire application. Tuning GC correctly can potentially deliver significant performance gains. Health Center identifies where garbage collection is causing performance problems and suggests more appropriate command line options.Read More »
A tool that detects delays and hangs in WebSphere trace and HTTP plug-in trace
WebSphere Application Server trace is used to obtain detailed information about running components, including application servers, clients, and other processes in the environment, such as the time and sequence of methods called by base classes.
IBM Trace and Request Analyzer for WebSphere Application Server allows you to find delays and possible hangs from WebSphere trace files and HTTP plug-in traces by parsing call trees of methods and traces and calculating delays in each method and trace.
The package includes a Java graphical user interface (GUI) application that displays delays in lists and tables.
With IBM Trace and Request Analyzer for WebSphere Application Server you get a useful tool for analyzing and identifying problems in trace files.
Here is how to use it:Read More »
When you enable the security on WebSphere Application Server [WAS], it will prompt you for authentication when you access admin console, stop server and wsadmin prompt. As we discussed earlier in other blog post about WebSphere Security, all the security related settings are stored in config file under Profile_root/config/cells/cell_name. File name is security.xml. The workaround when the administrator forgot the password is to change the security settings by manually modifying the security.xml file
- Locate the security.xml file and take a backup of it
- open security.xml file for editing and search for enabled=”true”
- modify it to enabled=”false” [you need to do this only for the very first occurrence of enabled=”true” ]
- Restart the servers
- Note: since you do not have the password you cannot stop the servers, so use KILL command
- Log into admin console
- Enable Security again
- Restart the servers.
[9/28/09 22:23:29:538 EST] 00000040 SRTServletReq E SRVE0133E: An error occurred while parsing parameters. java.net.SocketTimeoutException: Async operation timed out at com.ibm.ws.tcp.channel.impl.AioTCPReadRequestContextImpl.processSyncReadRequest(AioTCPReadRequestContextImpl.java:157) at com.ibm.ws.tcp.channel.impl.TCPReadRequestContextImpl.read(TCPReadRequestContextImpl.java:109) at com.ibm.ws.http.channel.impl.HttpServiceContextImpl.fillABuffer(HttpServiceContextImpl.java:4127) at com.ibm.ws.http.channel.impl.HttpServiceContextImpl.readSingleBlock(HttpServiceContextImpl.java:3371) at com.ibm.ws.http.channel.impl.HttpServiceContextImpl.readBodyBuffer(HttpServiceContextImpl.java:3476) at com.ibm.ws.http.channel.inbound.impl.HttpInboundServiceContextImpl.getRequestBodyBuffer(HttpInboundServiceContextImpl.java:1604) at com.ibm.ws.webcontainer.channel.WCCByteBufferInputStream.bufferIsGood(WCCByteBufferInputStream.java:133) at com.ibm.ws.webcontainer.channel.WCCByteBufferInputStream.read(WCCByteBufferInputStream.java:95) at com.ibm.ws.webcontainer.srt.http.HttpInputStream.read(HttpInputStream.java:296) at com.ibm.ws.webcontainer.servlet.RequestUtils.parsePostData(RequestUtils.java:297) at com.ibm.ws.webcontainer.srt.SRTServletRequest.parseParameters(SRTServletRequest.java:1722
As you can see this error will originate from Web Container’s http transport channel. And the reason for this error to come is Network delay. By default, in WebSphere application server, ConnectionTimeout will be set to 5 seconds for http transport channels. So if your network is slow, you may see this error in the logs
If you would like to change this timeout settings, follow these steps:
- Application Servers -> serverA -> Web Container -> HTTP Transport -> Application Name
- go to custom properties
- Add these two parameters (where xx is the time in seconds)
- ConnectionIOTimeOut=xx and ConnectionKeepAliveTimeout=xx
- Repeat this for all servers, if your application is mapped to multiple servers.
- Restart the server (s)
When you have a “WebGroup / virtual host not defined” error what to do?
PLGN0021E: Servlet Request Processor Exception: Virtual Host/WebGroup Not Found : The host xxxxxxx has not been defined.
Access application from app server:
By pass the web server and access the application directly from the application server using app_server_name:port/context_root
Check any recent application install/update:
If you have recently updated or installed the application, ensure that the Web server plug-in was regenerated and propagated to the Web server. Also, ensure
that the Web server is using the new plug-in configuration file.
Verify Plug-in is working:
Attempt to access other applications running on the same application server through the Web server. If you can access another application, this will verify that the plug-in and Web server are working with the application server, but not necessarily that the plug-in configuration is correct.
Verify Virtual Host/ Host Alias etc..
Verify that the virtual host associated with the Web module has a host alias defined containing the host name and port matching the host name and port in the URL causing the failure.
Find the URL pattern for the servlet.
Find the context root of the Web module containing the servlet.
Find the virtual host where the Web module is installed.
Find the aliases by which the virtual host is known.