Troubleshooting WebSphere Security-related Issues

IBM WebSphere Application Server v7.0 Security

Secure your IBM WebSphere applications with Java EE and JAAS security standards using this book and eBook

Troubleshooting general security configuration exceptions

The selected cases in this subsection concerns the situations when various aspects of configuring security are carried out and, as a result, error conditions occur.

Identifying problems with the Deployment Manager—node agent communication blues

Several of the problems that may take place due to either wrong or incomplete security configuration are found in the communication of the administrative layers of the WebSphere environment, i.e., between the deployment manager and the node agent(s).

A couple of the most common situations are shown below, along with recommendations as to how to correct the condition.

Receiving the message HMGR0149E: node agent rejected

The message HMGR0149E is the result of the Deployment Manager rejecting a request to connect from the node agent. This type of error and the display of this message normally takes place when security changes in the Deployment Manager were not synchronized with the node in question. An example of log file clip where this message is found can be seen in the following screenshot:

One way to fix this problem is by using the syncNode.sh command. The syntax for this command is:

syncNode.sh dmgr_host [dmgr_port] [-conntype <type>]
 [-stopservers] [-restart] [-quiet] [-nowait]
 [-logfile <filename>] [-replacelog] [-trace]
 [-username <username>] [-password <password>]
 [-localusername <localusername>]
 [-localpassword <localpassword>]
 [-profileName <profile>]
 syncNode.sh [-help]

Furthermore, a very simple procedure to correct this problem is given next:

1. Stop the affected node agent(s).
2. Execute, on the node agent OS host, the syncNode.sh command.
3. Monitor the SystemOut.log file for both dmgr and nodeagent processes.
4. Start the node agent.

For additional information on messages from the high availability manager, refer to the WAS ND7 Information Center link:
http://publib.boulder.ibm.com/infocenter/wasinfo/ v7r0/topic/com.ibm.websphere.messages.doc/com.ibm. ws.hamanager.nls.HAManagerMessages.html

Receiving the message ADMS0005E: node agent unable to synchronize

This message, ADMS0005E, is the result of the node agent attempting to synchronize configuration with the Deployment Manager. It is likely caused when changes in security-related configuration occurred and the node agent were not available. The following screenshot shows an example of this type of error.

One way to solve the issue is to shut down the node agent, and then, manually execute the command syncNode.sh from the node OS host using a user ID and password that has administrative privileges on the Deployment Manager. For syntax or usage information about this command, kindly refer to the previous example.

In case this action does not solve the problem, follow the next procedure:

1. Stop the node agent(s)
2. Using the ISC, disable global security
3. Restart the Deployment Manager
4. Start the node agent(s)
5. Perform a full synchronization using the ISC
6. Using the ISC, enable global security
7. Synchronize changes with all nodes
8. Stop the node agent(s)
9. Restart the Deployment Manager to activate global security
Unlock access to the largest independent learning library in Tech for FREE!
Get unlimited access to 7500+ expert-authored eBooks and video courses covering every tech area you can think of.
Renews at $19.99/month. Cancel anytime
START FREE TRIAL
10. Start the node agent(s)

For additional information on messages about the administrative synchronization, refer to the WAS ND7 Information Center link:
http://publib.boulder.ibm.com/infocenter/wasinfo/ v7r0/topic/com.ibm.websphere.messages.doc/com.ibm. ws.management.resources.sync.html

Troubleshooting runtime security exceptions

To close the section on troubleshooting, this subsection presents several cases of error or exception conditions that occur due to security configuration of various WAS ND7 environment components. Such components can be all within WAS or some components could be external, for example, the IHS/WebSphere Plug-in.

Troubleshooting HTTPS communication between WebSphere Plug-in and Application Server

When setting up the HTTPS communication between the WebSphere Plug-in and the WebSphere Application Server there may be instances in which exceptions and errors may occur during the configuration phase. Some of the most common are listed next.

Receiving the message SSL0227E: SSL handshake fails

The message SSL0227E is a common one when the main IHS process is attempting to retrieve the SSL certificate indicated by the property SSLServerCert located in the httpd.conf file. What this message is stating is that the intended SSL certificate cannot be found by its label from the key ring indicated by the directive KeyFile in the same configuration file.

An example of this type of message is shown in the following screenshot. In order to correct this error, there are two possibilities that can be explored.

On the one hand, one needs to insure that the directive KeyFile is pointing to the correct key ring file. That is, that the key ring file actually stores the target SSL certificate to be used with this IHS server.

On the other hand, there may be a typographic error in the value of the property SSLServerCert. In other words, the label that is mapped to the target SSL certificate was misspelled in the httpd.conf file.

In both cases, the command gsk7capicmd can be used to list the content of the key ring file. The syntax for listing the contents of a key ring file is:

<IHS_ROOT_Directory>/bin/gsk7capicmd -cert -list all -db <Path_To_
 kdb_File> -pw <kdb_File_Password>

For additional information on messages about handshaking issues, refer to the IHS v7 Information Center link:
http://publib.boulder.ibm.com/infocenter/wasinfo/v7r0/ topic/com.ibm.websphere.ihs.doc/info/ihs/ihs/rihs_ troubhandmsg.html

Receiving ws_config_parser errors while loading the plugin configuration file

If the configParserParse message of the ws_config_parser component is observed in the errors log file of the IBM HTTP Server; the following screenshot is an example of a possible output that may be found in the error logs. There may be a couple of reasons for this type of message to appear in the logs.

One reason for this type of message is that it occurs at the time in which the IHS process is being brought down. The WebSphere Plug-in module is in its cycle to reparse the plugin-cfg.xml file while the IHS process is shutting down, therefore the ws_config_parser component does not have enough resources to perform the parsing of the configuration file and throws this message, possibly multiple times in a row. In order to ensure that this is the correct interpretation of the message, it is necessary to find an indicator, such as a 'shutting down' type of message like the one shown in the next screenshot:

The other reason why this message may appear in the logs is very likely that the process owner of the IHS process does not have the correct privileges to read the plugin-cfg.xml file. In this case, ensure that the definition for the property User in the httpd.conf file has enough privileges to read the plug-in configuration file defined for the property WebSpherePluginConfig of the httpd.conf file.

For additional information on messages about WebSphere Plug-in issues, refer to the article Error message definitions for WebSphere Application Server's webserver plugin component.

Receiving the message GSK_ERROR_BAD_CERT: No suitable certificate found

The message GSK_ERROR_BAD_CERT appears in log files when the WebSphere Plug-in is attempting to establish an SSL connection with the back-end WebSphere Application Server and it does not have a way to validate the SSL certificate sent by the WebSphere Application Server. An example of this type of message is shown in the next screenshot:

One way to solve this problem is by adding to the IHS key ring file the signer certificate from the WebSphere Application Server. When doing this, care must be taken to correctly select the WebSphere trust store. In other words, the correct scope for your target Application Server needs to be identified so that the appropriate trust store can be accessed.

For instance, if it was desired to obtain the root certificate (aka, signer certificate) used by the Chap7AppServer Application Server, one needs to identify the scope for that application server. Therefore, one should start with the following breadcrumb in the ISC (Deployment Manager console): Security | SSL certificate and key management | Manage endpoint security configurations. The following screenshot illustrates a portion of the resulting page:

Once the appropriate scope is identified, continue by completing the breadcrumb: Security | SSL certificate and key management | Manage endpoint security configurations | Chap7AppServer | Key stores and certificates | NodeDefaultTrustStore | Signer certificates. The following screenshot shows a portion of a resulting page.

You are now in position to extract the Application Server signer SSL certificate. Once this certificate is extracted, it needs to be imported into the IHS key ring file as a root certificate.