WebSphere Application Server v8 Primer, Part-6 : Application deployment by adding properties file to Monitored directory

The properties file based configuration feature can be used copy configuration properties from one environment to another. And also use the properties file to troubleshoot configuration issues and apply one set of configuration properties across multiple profiles, nodes, cells, servers or applications. The Monitored Directory deployment has extended to use the properties file to install, uninstall or update an application in version8.


WebSphere Application Server v8 Primer, Part-5 : Monitored Deployment

Monitored directory/deployment is a new, simple and fast way for administrators and developers to install, update, and uninstall applications by moving archive files in or out of a monitored directory.  A user who has prepackaged an application file with all required bindings specified can quickly deploy that application without having to resort to any tools other than a running application server, or, in a network deployment environment, a deployment manager.

WebSphere Application Server Troubleshooting tool : IBM Support Assistant v5

IBM Support Assistant (ISA) 5.0 Tech Preview introduces the next generation of IBM’s popular problem determination and troubleshooting platform. This new, complimentary release provides the richest tools and functions you need to perform root-cause analysis and enhance your productivity.

New features:

  • Case Management – simplify the organization of your problem determination investigation
  • File Management – enhanced capabilities to help you search through files quickly
  • Tools – automated analysis tools to generate reports and assist you in problem discovery
  • Intuitive web UI – with an emphasis on simplicity, you can easily open a browser to access the ISA application
  • Server-based application – install ISA once and collaborate with multiple team members. Analysis processing can now be off-loaded from your desktop!
  • Single-user desktop – run the ISA application on your desktop if you prefer a single-user mode

The Enigma of WebSphere Application Server Plug-in, Part-1: What, why and How

This is a 4 part series which covers WebSphere Application Server Plugin related topics like: what is plug-in, why should you use plug-in, how does plug-in works, Different components invloved in plugin operation, Plgin configuration file, Configurable parameters, Troubleshooting and merging multiple plug-in files. This article series is written assuming that IBM http server is used as web server in front of the WebSphere application server. 

The first part of this article, discuss the basics of the websphere application server plug-in. In this part, you’ll understand what is plugin, advantages of plugin and how a plugin routes requests to the backend web containers.

What is WebSphere Application Server Plugin?
The IBM WebSphere Application Server Web server plug-in is the “glue” between a Web server and WebSphere Application Server. The plug-in was developed using native programming language “C”.

What does plugin do?
Plug-in forwards HTTP requests from the Web server to the WebSphere Application Server.

How do i get the Plugin?
It comes along with the WebSphere Application Server installation CD or part of your downloaded image.

Before we start ……
take a look at the below image.

In general, there will a web server sitting infront of the websphere application server. The request from browser/internet will first comes to web server and then it gets redirected/forwarded to application server. This forwarding/redirection is handled by the plugin. This redirection happens based on a set of rules defined in the plugin configuration file. The Web server gives the WebSphere plug-in an opportunity to handle every request first, and only when WebSphere is not configured to service that URL does the request go to the Web server for servicing.

What are advantages of using webserver and plugin?

  1. The plug-in provides work load management and failover capabilities by distributing requests evenly to multiple application servers and routing requests away from a failed application server.
  2. Static content can be served by the Web server without doing a full round trip to the application server.
  3. The Web server provides an additional hardware layer between the Web browser and the application server, thus strengthening the application server’s security.

components involved in plugin operation

  • Web server : which first receives the request
  • httpd.conf : This contains the reference to the plugin so/dll file and reference to the plugin configuration file.
  • Plugin : It is loaded into memory by the Web server during startup and runs inside the same process as the Web server.
  • plugin-cfg.xml : configuration file of the plugin, which contains iinformation about which URLs will be served by backend application servers
  • http transport : which accepts the forwarded request from plugin, on the application server side
  • web container: which actually handles the request

How does Plug-in work ?

A)Plug-in initialization

  • When web server is starting, httpd.conf will be read
  • httpd.conf will have: 1)LoadModule reference to the path of the plugin so/dll file. 2)WebSpherePluginConfig reference will have the path to plugin-cfg.xml
  • While web server is starting , it loads websphere plugin using LoadModule reference
  • Plugin will load configuration file using WebSpherePluginConfig reference, into the plugin process.
  • Then configuration file will be parsed and validated

Whenever the Web server receives a new HTTP request, it first passes that request to the plug-in. The plug-in determines whether this request should be handled by WebSphere by matching the URL requested to all the URLs defined in plugin-cfg.xml.

If plugin-cfg.xml is updated, it can automatically be reloaded by the plug-in without requiring a restart of the Web server. By default, the plug-in attempts to reload the plug-in configuration file every 60 seconds

<?xml version=”1.0″ encoding=”ISO-8859-1″?>
<Config ASDisableNagle=”false” AcceptAllContent=”false” AppServerPortPreference=”HostHeader” ChunkedResponse=”false” FIPSEnable=”false
” HTTPMaxHeaders=”300″ IISDisableNagle=”false” IISPluginPriority=”High” IgnoreDNSFailures=”false” RefreshInterval=”60″ ResponseChunkSi
ze=”64″ SSLConsolidate=”false” TrustedProxyEnable=”false” VHostMatchingCompat=”false”>
<Log LogLevel=”Error” Name=”/www/logs/ibmhttp/http_plugin.log”/>
<Property Name=”ESIEnable” Value=”true”/>
<Property Name=”ESIMaxCacheSize” Value=”1024″/>
<Property Name=”ESIInvalidationMonitor” Value=”false”/>
<Property Name=”ESIEnableToPassCookies” Value=”false”/>

The plug-in does this by checking the timestamp of plugin-cfg.xml, and if the timestamp has changed since the previous check, it reloads it.

B) How does plug-in determines where to send the request?

The plug-in compares the request to each route.

<Route ServerCluster=”app1_cluster” UriGroup=”joseph-app1_app1l_cluster_URIs” VirtualHostGroup=”joseph-app1″/>

<UriGroup Name=”joseph-app1_app1l_cluster_URIs”>
<Uri AffinityCookie=”JSESSIONID” AffinityURLIdentifier=”jsessionid” Name=”/app1_context/*”/>

<VirtualHostGroup Name=”joseph-app1″>
<VirtualHost Name=”testapp1:80″/>
<VirtualHost Name=”testapp1.joseph.com:80″/>
<VirtualHost Name=”app1-dev.joseph.com:9081″/>
<VirtualHost Name=”app1-dev.joseph.com:9444″/>
<VirtualHost Name=”app2-dev.joseph.com:9081″/>
<VirtualHost Name=”app2-dev.joseph.com:9444″/>

If a match for URIGroup and VirtualHostGroup is found, the plug-in sends the request to the associated Server.

If there are multiple servers in the servercluster, the request is sent to one of them depending upon various factors such as WLM policy, session information, etc.

<ServerCluster CloneSeparatorChange=”false” GetDWLMTable=”false” IgnoreAffinityRequests=”true” LoadBalance=”Round Robin” Name=”appl
_cluster” PostBufferSize=”64″ PostSizeLimit=”-1″ RemoveSpecialHeaders=”true” RetryInterval=”60″>

<Server CloneID=”abcdxyz” ConnectTimeout=”0″ ExtendedHandshake=”false” LoadBalanceWeight=”2″ MaxConnections=”-1″ Name=”appnode1_server1_app1″ ServerIOTimeout=”0″ WaitForContinue=”false”>
<Transport Hostname=”app1-dev” Port=”9081″ Protocol=”http”/>
<Transport Hostname=”app1-dev” Port=”9444″ Protocol=”https”>
<Property Name=”keyring” Value=”/www/ibmhttp/Plugins/config/dev/plugin-key.kdb”/>
<Property Name=”stashfile” Value=”/www/ibmhttp/Plugins/config/dev/plugin-key.sth”/>

<Server CloneID=”xyzabcd” ConnectTimeout=”0″ ExtendedHandshake=”false” LoadBalanceWeight=”2″ MaxConnections=”-1″ Name=”appnode2_server1_app2″ ServerIOTimeout=”0″ WaitForContinue=”false”>
<Transport Hostname=”app2-dev” Port=”9081″ Protocol=”http”/>
<Transport Hostname=”app2-dev” Port=”9444″ Protocol=”https”>
<Property Name=”keyring” Value=”/www/ibmhttp/Plugins/config/dev/plugin-key.kdb”/>
<Property Name=”stashfile” Value=”/www/ibmhttp/Plugins/config/dev/plugin-key.sth”/>

<Server Name=”appnode1_server1_app1″/>
<Server Name=”appnode2_server1_app2″/>


C) Sending requests to application server?

This is handled in two steps:

  1. Once the server/clone determination has been made [step-B], the plug-in breaks down the HTTP request into its various components such as HTTP header, GET/POST data, etc. A new data packet is created by populating an empty HTTP client object with data from the user request. Once this new HTTP client object has been populated, it is ready to be sent. If the plug-in is configured to communicate with the HttpTransport over SSL, it is required to perform security functions as well.
  2. The request will be sent by flushing out the client object stream to the network using TCP/IP.

D) Receiving response from application Server

After sending a request to the application server, plug-in will wait for the response.

  • If a response from the application server has been received by the plug-in, the response is passed on to the Web server and then to the Web browser
  • If there is no response from the selected server, then that application server is marked as unreachable and is then taken out of the routing algorithm for a period specified by the RetryInterval. In a ServerCluster, request will be passed on to other clones. If communication with every clone in the cluster fails due timeouts, then you receive an HTTP 500 error.

<ServerCluster CloneSeparatorChange=”false” GetDWLMTable=”false” IgnoreAffinityRequests=”true” LoadBalance=”Round Robin” Name=”appl
_cluster” PostBufferSize=”64″ PostSizeLimit=”-1″ RemoveSpecialHeaders=”true” RetryInterval=”60″>


In the next part, we will learn about different parameters, how plugin handles servers and configuration options.

Hung thread detection policy in WebSphere Application Server

The hung thread detection policy in websphere application server lets you detect the hung threads by sending notifications in the log files. You can configure a hang detection policy to, accommodate your applications and environment so that potential hangs can be reported, providing earlier detection of failing servers. Using a hang thread detection policy you can specify a time that is too long for a unit of work to complete. A thread monitor will check all the threads in the systems.

Hang thread detection option is enabled by default. You can adjust the values of the detection policy or disable it by using the below procedure.

  • Navigate to Servers –> Applicaiton Servers –> server_name –> administration –> custom properties
  • Add the following 4 custom properties
# Property Description Default
A com.ibm.websphere.threadmonitor.interval  How frequently thread monitor should check all the managed threads for hung threads 180
B com.ibm.websphere.threadmonitor.threshold  After how many seconds a thread can be considered as hung 600
C com.ibm.websphere.threadmonitor.false.alarm.threshold The number of times that false alarms can occur before automatically increasing the threshold (T) 100
D com.ibm.websphere.threadmonitor.dump.java  when set to ‘true’, creates a java core when a hung thread is detected false
  • Click OK and save the changes.
  • Sync the changes and restart the servers for the changes to take effect.

False Alarms:

What happens if a thread eventually finishes its work after it been reported as hung? This situation is called a ‘false alarm’. A large number of these events indicate that the threshold value is too small. The hang detection facility can automatically respond to this situation. For every (T) false alarm, the threshold is increased by a factor of 1.5

Notifications for J2EE applications:

The notifications from the hand thread detector will appear in the log file.

Hung thread report  WSVR0605W Thread ‘thread_name’ has been active for ‘xxx sec’ and may be hung.  There are ‘N’ threads in total in the server that may be hung.
False Alarm  WSVR0606W Thread ‘thread_name’ was previously reported to be hung but has completed. It was active for approximately ‘yyy sec’. There are ‘N’ threads in total in the server that still may be hung.
Auto adjustment  WSVR0607W Too many thread hangs have been falsely reported.  The hang threshold is now being set to ‘zzz sec’.