Showing posts with label JVMs. Show all posts
Showing posts with label JVMs. Show all posts

Thursday, 28 January 2016

java.lang.OutOfMemoryError: GC overhead limit exceeded

Hello Everyone,

Last time we discussed the issue of java.lang.OutOfMemoryError: Java heap space for Apache Tomcat server. Similarly today we are having another JVM related issue for WebSphere application server. Here my users are complaining about the some kind of slowness while application access and when I check for the error I am getting below error string in logs.

Error:


java.lang.OutOfMemoryError: GC overhead limit exceeded

Cause:


Before solution we need to know what is GC and why we are getting this error called  GC overhead limit exceeded.
Java runtime environment contains a built-in Garbage Collection (GC)  daemon process. Whenever a particular space in memory is no longer used, a process called Garbage Collection clears that memory space and GC do this work on regular basis. There are different algorithms and parameter of GC which defines how GC will work. And if in case GC is not performing well the it will write the error code similar to what we get.

The java.lang.OutOfMemoryError: GC overhead limit exceeded error is displayed when your application has exhausted pretty much all the available memory and GC has repeatedly failed to clean it. The java.lang.OutOfMemoryError: GC overhead limit exceeded error is the JVM’s way of signaling that your application spends too much time doing garbage collection with too little result. By default the JVM is configured to throw this error if it spends more than 98% of the total time doing GC and when after the GC only less than 2% of the heap is recovered. This means that the small amount of heap the GC is able to clean will likely be quickly filled again, forcing the GC to restart the cleaning process again. This forms a vicious cycle where the CPU is 100% busy with GC and no actual work can be done and end users of the application face extreme slowdowns

Solution: 


You can resolve this issue by using below options.

Step 1
Disable the error check using -XX:-UseGCOverheadLimit. You can disable the error check by adding the option  -XX:-UseGCOverheadLimit to the Generic JVM arguments. This will help you to stopping these error alerts. Please keep in mind that this error is very likely to be a symptom of a JVM Heap / tuning problem so my recommendation to you is always to focus on the root cause as opposed to the symptom instated disabling the error alert .

Step 2
You may also try to change Garbage Collector Policy. You can add the below policy of Garbage collector option to the Generic JVM arguments.
-XX:+UseConcMarkSweepGC 
-XX:+UseConcMarkSweepGC this can help you in the space issue as it compact all fragment free space after GC run and make you available more contiguous space.
-XX:+UseParallelGC
This -XX:+UseParallelGC parameter help you to run the GC continuously instated of reaching to the memory space threshold.

Step 3
If the option two not works you can  Increase heap size using -Xmx switch in Generic JVM arguments

Step 4
If option three not works add -XX:+HeapDumpOnOutOfMemoryError to Generic JVM arguments which will automatically generate heap dump in case of out of memory issue. Then analyze the dump with IBM Memory Analyzer looking for memory leaks. On the basis of analysis you can work with the development team, optimize your code to use less memory and/or reuse objects instead of creating new ones thus reducing the number of times the garbage collector runs.

Sometime dealing with the OutOfMemoryError is a log process, you need to analyze lot of things for perfect tuning and sometime it just resolved by JVM parameter setting.

Hope this will help you. Kindly comment for your suggestion and quires.  

"Effort only fully releases its reward after a person refuses to quit.”

 Regards,
 Akhilesh B. Humbe

Friday, 22 January 2016

java.lang.OutOfMemoryError: Java heap space

Hell Everyone,

After a very long time I am working on Apache tomcat. While accessing application login page user facing some issue and getting some errors on the web browser. When I check the tomcat output logs I seen the below error.

Log Location:

/usr/local/apache-tomcat-5.5.23/logs/catalina.out

Error:

Jan 14, 2016 10:40:36 AM org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor processChildren
SEVERE: Exception invoking periodic operation:
java.lang.OutOfMemoryError: Java heap space
        at java.util.HashMap.<init>(HashMap.java:203)
        at java.util.LinkedHashMap.<init>(LinkedHashMap.java:178)
        at org.apache.catalina.startup.HostConfig$DeployedApplication.<init>(HostConfig.java:1314)
        at org.apache.catalina.startup.HostConfig.deployDirectory(HostConfig.java:898)
        at org.apache.catalina.startup.HostConfig.deployDirectories(HostConfig.java:883)
        at org.apache.catalina.startup.HostConfig.deployApps(HostConfig.java:492)
        at org.apache.catalina.startup.HostConfig.check(HostConfig.java:1206)
        at org.apache.catalina.startup.HostConfig.lifecycleEvent(HostConfig.java:293)
        at org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(LifecycleSupport.java:120)
        at org.apache.catalina.core.ContainerBase.backgroundProcess(ContainerBase.java:1306)

Solution:

As you can see in error it's a java.lang.OutOfMemoryError: Java heap space means we don’t have enough heap memory available  to write new objects of an application. There is not a single specific way to resolve this issue. There may be a different reasons of OutOfMemoryError. You can find out that by generating and analyzing heap dump.

In my case the allocated heap memory for running the application was too less as compared to my requirement. So I increase the Java Heap memory  and it resolve my issue.

How to increase the java heap space in Apache tomcat ?


When you initiate the command to start the Apache tomcat instance it take the configuration from the file catalina.sh. If you want to increase the java heap space you need to manually edit this file. For increasing the java heap space increase the value of -Xmx parameter for the CATALINA_OPTS
E.g. 
cd /usr/local/apache-tomcat-5.5.23/bin/
vi catalina.sh

JAVA_HOME=/usr/java
CATALINA_OPTS="-Xmx512m " [In my case previously it was -Xmx126m]

Save the Configuration changes and start restart the Apache tomcat service.

/usr/local/apache-tomcat-5.5.23/bin/shutdown.sh
/usr/local/apache-tomcat-5.5.23/bin/startup.sh

Note: Do not Increase the heap size blindly, it will not help you every time in case of the Out Of Memory Error. Make a habit to analyze the heap dump and try to find out the best suitable JVM  arguments for your environment.

Important:

How to take a Heap Dump.
Best Guide for Garbage Collection
Best Guide for JVM tuning.

Hope this will help you. Kindly comment for your suggestion and quires.  

"Effort only fully releases its reward after a person refuses to quit.”

 Regards,
 Akhilesh B. Humbe

Sunday, 10 January 2016

CRIMC1066E ERROR: Unable to acquire lock on file .imlock''.

Hello Everyone,

Happy New Year 2016 to all readers. It's 1st week of the new year and I am getting new error while initiating IBM Installation Manager console.

CRIMC1066E ERROR:   Unable to acquire lock on file  /export/localhome/was/var/ibm/InstallationManager/.imlock''.

Problem:

While executing command ./imcl -c to launch the command line console of IBM installation manager it displays the following error message.
CRIMC1066E ERROR:   Unable to acquire lock on file  /export/localhome/was/var/ibm/InstallationManager/.imlock''.
The installation manager could not be started because another installation manager instance is already running. Only a single installation manager may be running at any time.

Cause:

We are getting this error if the installation manager is already running by some other user. When we start Installation manager the user create lock file .imlock which not allow other user to run installation manager to other location. This file get deleted when you close the installation manager. In some case if installation manager process got killed accidentally the lock file remains as it is and not allow you to start the installation manager next time and showing the error code. 

Resolution:

1. Ensure that there is no running Installation Manager instance.
2. If there is an Installation manager process running, delete that process
Unix: ps -ef | grep imcl or  ps -ef | grep java (Run these commands to find the process)
Windows: Go to the Task Manager and terminate all the java.exe, javaw.exe and IBMIM.exe processes.
3. Re-launch Installation Manager.

If the above steps do not resolve the issue, follow these steps:

1. Go to location list in the error code and manually delete .imlock file.
Unix: ls -a (use this command to list down the hidden files)
2. Hope this will not require, but if necessary, restart the system.

Hope this will help you. Kindly comment for your suggestion and quires.   

"Effort only fully releases its reward after a person refuses to quit.”

 Regards,

 Akhilesh B. Humbe

Sunday, 6 December 2015

Uninstall WebSphere Application Server from Command line

Hello All,

It is a very common request to uninstall the WebSphere products from machine if you are working as an administrator. If it's a windows environment and you have installation manager console available then it's a very easy task to do. But if it's a UNIX OS and you don't even have a Xmanager to open a graphical user interface the it's critical task to uninstall WebSphere Application server.

In my case I have a Solaris as an OS and didn't have Xmanager available for graphical user interface. Then I use the command line to uninstall WebSphere application Server .  You can do this task in two ways which are described as below.

Procedure:


Uninstall WebSphere Environment Using IBM Installation Manager Command line console.


1. Go to the below location
cd /opt/InstallationManager/eclipse/tools

2. Execute the below command and you will get the command line console
./imcl -c

=====> IBM Installation Manager

Select:
      1. Install - Install software packages
      2. Update - Find and install updates and fixes to installed software packages
      3. Modify - Change installed software packages
      4. Roll Back - Revert to an earlier version of installed software packages
      5. Uninstall - Remove installed software packages

Other Options:
      L. View Logs
      S. View Installation History
      V. View Installed Packages
         ------------------------
      P. Preferences
         ------------------------
      E. Export Data for Problem Analysis
      A. About IBM Installation Manager
         ------------------------
      X. Exit Installation Manager

3. Type 5 to uninstall the installed software packages and press Enter.

4. Similarly by following the instructions given on the command line console you can uninstall all installed packages and Installation Manager also.

Uninstall WebSphere Environment Using IBM Installation Manager's ./imcl uninstall utility.


1. Go to the below location
cd /opt/InstallationManager/eclipse/tools/

2. Execute the below command to list down all installed packages.

./imcl listInstalledPackages

com.ibm.cic.agent_1.5.2000.20120223_0907
com.ibm.websphere.IBMJAVA.v70_7.0.1000.20120424_1539
com.ibm.websphere.ND.v85_8.5.2.20130327_1831
com.ibm.websphere.PLG.v85_8.5.1.20121017_1724

3. Execute the below commands to uninstall all installed packages including Installation manager if you want
[Note: Follow the below sequence for error free uninstall ]

bash-3.2$ ./imcl uninstall com.ibm.websphere.IBMJAVA.v70_7.0.1000.20120424_1539
Uninstalled com.ibm.websphere.IBMJAVA.v70_7.0.1000.20120424_1539 from the /opt/WebSphere85 directory.

bash-3.2$ ./imcl uninstall com.ibm.websphere.PLG.v85_8.5.1.20121017_1724
Uninstalled com.ibm.websphere.PLG.v85_8.5.1.20121017_1724 from the /opt/wasplugins directory.

bash-3.2$ ./imcl uninstall com.ibm.websphere.ND.v85_8.5.2.20130327_1831
Uninstalled com.ibm.websphere.PLG.v85_8.5.1.20121017_1724 from the /opt/WebSphere85 directory.

bash-3.2$ ./imcl uninstall com.ibm.cic.agent_1.5.2000.20120223_0907
Uninstalled com.ibm.websphere.IBMJAVA.v70_7.0.1000.20120424_1539 from the
/opt/InstallationManager directory.


Hope this will help you. Kindly comment for your suggestion and quires.  

"Effort only fully releases its reward after a person refuses to quit.”

 Regards,
 Akhilesh B. Humbe

Saturday, 31 October 2015

Caused by: com.ibm.wsspi.management.bla.op.OpExecutionException: CWWMH0121E: Could not read configuration data for composition unit

Hello Everyone

We always think that installation/UnInstalltion and deletion of Websphere Application server  is easy task. It's just a thing like click on next button for some time and you are done with the work. But it is always not easy as we think. Today I was trying to delete one JVM from my Websphere Application server v8.5.0.0 ND environment and I was not able to delete it from the console. Then I tried the same thing using wsadmin tool but still getting some errors. When I was checking the logs I found bello errors in SystemOut.log. While resolving issue I came to know very interesting facts which I have noted down below.

Error :

Caused by: com.ibm.wsspi.management.bla.op.OpExecutionException: CWWMH0121E: Could not read configuration data for composition unit "WebSphere:cuname=<Application Name>,cuedition=BASE". Error data: com.ibm.wsspi.management.bla.op.OpExecutionException: Enterprise application data for composition unit "WebSphere:cuname=<Application_Name>,cuedition=BASE" could not be read. at com.ibm.ws.management.bla.model.CompositionUnitConfig.read(CompositionUnitConfig.java:293)

Cause: 

In my case I have deleted all the applications installed on the JVM which I want to delete and then try to delete that JVM. But as you can seen in above error it was saying that some application are still pointed to the JVMs which are not uninstall. But I never install these applications on my JVM may be did installed on some other JVMs in same domain but before some time, but currently these applications are not installed on any of the JVM in my domain. And when I search for the reason I came to know that when I install an application there are some folders in configuration which keep a track of these folders. These folders are as below.

WAS_INSTALL/profiles/ProfileName/installedApp/Cell_Name/
WAS_INSTALL/profiles/ProfileName/config/cells/Cell_Name/applications
WAS_INSTALL/profiles/ProfileName/config/cells/Cell_Name/applications/blas 
WAS_INSTALL/profiles/ProfileName/config/cells/Cell_Name/applications/cus 

When you delete the application the entries of application should be remove from the all these folders otherwise it will cause the error which we are facing. Most of the times the application entries are there in /blas and /cus folders.

We can solve this issue using below solution.

Resolution 


Perform the following procedure
  1. Login to the WebSphere console and list down all the applications with exact version information which are currently install .
  2. Stop the DeploymentManager/Adminagent
  3. Go to the below listed folders location and delete all the listed application directories other than you noted down from admin console. WAS_INSTALL/profiles/ProfileName/installedApp/Cell_Name/ WAS_INSTALL/profiles/ProfileName/config/cells/Cell_Name/applications WAS_INSTALL/profiles/ProfileName/config/cells/Cell_Name/applications/blas  WAS_INSTALL/profiles/ProfileName/config/cells/Cell_Name/applications/cus 
  4. Stop the DeploymentManager/Adminagent
  5. Log back into the Admin Console
  6. Perform a 'Full Resynchronize' operation on all nodes using the Admin Console
  7. After re-sync fully completes for all nodes, attempt to delete the JVM/Cluster again using the Administrative Console, or the wsadmin utility
Hope this will help you. Kindly comment for your suggestion and quires.  

"Effort only fully releases its reward after a person refuses to quit.”

 Regards,
 Akhilesh B. Humbe

Thursday, 14 May 2015

Step by Step Installation Of WebSphere Application Server

Today here we are going to discuss about the installation of WebSphere Application Server. Which is the very basic thing we need to know before learning to administrate any kind of technology. Here I have the trail version of WebSphere Application Server v8.5. You can also download this version from the Websphere Site

This one is just the trial version you can install any of  the WebSphere Application Server (Base/ND) product using these steps. For Installation in WebSphere Application Server v8.0 and onward we require a tool called Installation Manager. So here we divide the installation process in four sub processes and we will learn about the each process very briefly.

Process for Installing WebSphere Application Server.

For Installing WebSphere Application Server v8.0 and onwards we need to follow the below steps.

Step 1 : Installation of IMB Installation Manager .
Step 2 : Installation of WebSphere Application Server.
Step 3 : Creating Profile Definition. 
Step 4 : First Step towards WebSphere Application Server. 

Here we have windows environment. But the procedure is similar for all operating systems.

"Effort only fully releases its reward after a person refuses to quit.”

Regards,
Akhilesh B. Humbe

Installation of WebSphere Application Server.

 Step 1 : Installation of IMB Installation Manager .

Step 2 : Installation of WebSphere Application Server.


Installation of WebSphere Application Server using IBM Installation Manager is very user friendly and easy task. Here we are going to install the trail base version 8.5 using IBM Installation Manager. Below are the steps to install WebSphere Application Server trail base version.

Download the dump for WebSphere Application Server and  extract the .zip files. There are three .zip files for this installation.


Start IBM Installation Manager from start menu in windows. Or else you can start it using IBMIM command from location C:\Program Files\IBM\Installation Manager\eclipse . Here it's already started from Step 1


Before installation you need to register the product. For that click on the file menu in left corner of screen and click on preferences.


Click on Add Repository and  the  brows the location of repository.xml from disc1.


Click on apply and ok.


Then click on install in main screen and you will get the following window. Which will list you the products which you have registered. You can select one of then or all of then to install. Here we are selecting WebSphere Application Server trail v8.5.5.0 to install.


Accept the terms in license agreement and click on Next.


Select the installation Directory, you can keep it as default or can change according to you.



Select the language and click on next.


Select the products which you want to install. You can install the sample application for testing purpose.


Brows the Disc2 and then Disc3 location as asked and click on ok.


Review the summary information and click on install and I will start the installation.



After installation you can see the below screen  click on finish to start the profile management tool and start the profile creation.

Step 3 : Creating Profile Definition. 

Creating Profile Definition

Step 1 : Installation of IMB Installation Manager.
Step 2 : Installation of WebSphere Application Server.

Step 3 : Creating Profile Definition . 


After Installation profile creation is the step where we defile the definition for application environment. There are different profiles with there different significance.  Here we are creation stand alone application server profile as we have a base version installed. For managed profiles you can install the ND setup.
To create the profile definition you can follow the below steps.

Start the profile management tool from start menu in windows. Or else you can start it using command "C:\Program Files (x86)\IBM\WebSphere\AppServer\bin\ProfileManagement\pmt.bat".  Here it's already started from Step 2


Click on create for new profile creation.


Select the type of profile you want to create. Here we are creating Application Server Profile. Click on next.


In this window you can select any of the type, but when you use Advanced Profile Creation it allow you to change the default things. Here we are selecting Advanced Profile Creation.


Click on next in Optional Application Deployment window.


Here you can set the profile name and the profile directory path. Here we are using default one. Click on Next.


Now you can set Node name, Host name and server/jvm name. Here we are using default one. Click on Next.


In Administrative Security page you can enable or disable the security. And specify the user name and password for secure access. Click on Next.


Here you can configure SSL certificate with your profile. You can create one or import one. Here we are creating default one. Click on Next.


In this window you can see the default certificate details. Click on Next.


Here in this window you can check and change the port assign for profile.


This window will ask you that if you want to run the JVM as windows service.


If you have webserver already installed, here you can configure the webserver to application server. We don’t have webserver installed so we ignore this click on Next.


Check the profile creation summary and clock on Finish.


It will start the profile creation process.


Once you done with the profile creation process click on finish the first step console.


Step 4 : First Step towards WebSphere Application Server 

First Step towards WebSphere Application Server

Step 1 : Installation of IMB Installation Manager.
Step 2 : Installation of WebSphere Application Server.
Step 3 : Creating Profile Definition.

Step 4 : First Step towards WebSphere Application Server. 


First Step Console allow you to perform the very basic admin activities on newly created profile.

You can open the first step console from start menu in windows. Or else you can start it using command "C:\Program Files (x86)\IBM\WebSphere\AppServer\profiles\AppSrv01\firststeps\firststeps.bat". Here it's already started from Step 3


After profile creation and installation click on installation verification. It will verify your installation and start the default JVM.




From first step console you can stop the jvm and redirect to admin console.


It will brows you the admin console link and using username and password you can login to the Administration console.


Here we done with the installation of WebSphere Application Server

Hope this will help you. Kindly comment for your suggestion and quires.  

"Effort only fully releases its reward after a person refuses to quit.”

 Regards,
 Akhilesh B. Humbe

Installation of IMB Installation Manager

Step 1 : Installation of IMB Installation Manager


IBM installation Manager is the product/utility used for installation of WebSphere products. We are using this utility from WebSphere Application Server v8.0 and onwards. Installation Manager allow you to install/uninstall/update the products. Below are the steps to install IBM Installation Manager.

Download the IBM Installation Manager suitable for your operating system and extract the .zip file.


Run the file Install and it's good if you run it as an administrator.


You can see, it will start installing IBM Installation Manager.


You can see the version of Installation Manager in install package page. Click on Next to proceed.


Accept the terms in license agreement and click on Next.


Select the installation Directory, you can keep it as default or can change according to you.


Verify the summary and click on Install.


It will Initiate the installation process.


Once you get finish you will get the following page window.


Click on Restart Installation Manager to start the installation manager tool.

Step 2 : Installation of WebSphere Application Server 

Tuesday, 21 April 2015

SSLHandshakeE E SSLC0008E: Unable to initialize SSL connection

Hello Everyone,

Hope you all aware of SSL (Secure Socket Listener) . We normally face lot of issues during SSL configuration and after configuration while handshake. We have discussed many of them before also. Here today I was getting another one while handshake between client and server.

Error:


[4/10/15 10:44:20:024 EDT] 00000067 SSLHandshakeE E   SSLC0008E: Unable to initialize SSL connection.  Unauthorized access was denied or security settings have expired.  Exception is javax.net.ssl.SSLHandshakeException: Client requested protocol TLSv1 not enabled or not supported
        at com.ibm.jsse2.K.B(K.java:141)
        at com.ibm.jsse2.SSLEngineImpl.b(SSLEngineImpl.java:106)
        at com.ibm.jsse2.SSLEngineImpl.c(SSLEngineImpl.java:184)
        at com.ibm.jsse2.SSLEngineImpl.wrap(SSLEngineImpl.java:582)
        at javax.net.ssl.SSLEngine.wrap(SSLEngine.java:16)
        at com.ibm.ws.ssl.channel.impl.SSLUtils.handleHandshake(SSLUtils.java:746)
        at com.ibm.ws.ssl.channel.impl.SSLConnectionLink.readyInbound(SSLConnectionLink.java:566)
        at com.ibm.ws.ssl.channel.impl.SSLConnectionLink.ready(SSLConnectionLink.java:295)
        at com.ibm.ws.tcp.channel.impl.NewConnectionInitialReadCallback.sendToDiscriminators(NewConnectionInitialReadCallback.java:214)
        at com.ibm.ws.tcp.channel.impl.NewConnectionInitialReadCallback.complete(NewConnectionInitialReadCallback.java:113)
        at com.ibm.ws.tcp.channel.impl.WorkQueueManager.requestComplete(WorkQueueManager.java:558)
        at com.ibm.ws.tcp.channel.impl.WorkQueueManager.attemptIO(WorkQueueManager.java:608)
        at com.ibm.ws.tcp.channel.impl.WorkQueueManager.workerRun(WorkQueueManager.java:985)
        at com.ibm.ws.tcp.channel.impl.WorkQueueManager$Worker.run(WorkQueueManager.java:1074)
        at com.ibm.ws.util.ThreadPool$Worker.run(ThreadPool.java:1881)
Caused by: javax.net.ssl.SSLHandshakeException: Client requested protocol TLSv1 not enabled or not supported

Cause :


While Searching for this error I came across very important and helpful information. Here we consider that you all aware of the  SSL Handshake term or we will discuss it in brief later. In SSL hand shake the two communicators i.e. client and server initiate connection with each other to perform the transaction. For handshake we are having a different protocols based upon client and server. There is condition that for handshake need the communication protocols on both side should be compatible. Here in case WebSphere application server is our server and the requesting client may be different. It may be Webserver or your java application or you web browser or any internal service.

Here consider in my case the client which is asking foe handshake is a java application. Almost all HTTPS requesting traffic from application is Java generated. So the protocol requested by my java application is not enabled on server side so it's not supporting the handshake.

Let's check the below table for more clarification.



Here in my case the application is running using the java 6 means it will support the minimum SSLv3** and Maximum TSLv1 Version. Here in case if I disable the SSLv3** support from my server side the my application will use the higher version of SSL to handshake and will establish the connection for transaction. But I disable the higher version i.e. TSLv1 then it is not possible to establish the hand shake. So it will show the error Client requested protocol TLSv1 not enabled or not supported. Same thin will happen vise versa .

Solution:


In such case you can go and check on Websphere application server console which version of SSL handshake protocol your server is supporting. For that go to the
SSL certificate and key management > SSL configurations > NodeDefaultSSLSettings (your Custom setting ) > Quality of protection (QoP) settings and check. Here in my case it was showing TLSv1.2 which is not supported by a client.



So here I go and set the protocol to TLSv1, which one client asking for. And now it's working fine.



For this click on the drop box, select the suitable protocol, Apply the change and don't forget to save it.


Hope this will help you. Kindly comment for your suggestion and quires.  

"Effort only fully releases its reward after a person refuses to quit.”

 Regards,
 Akhilesh B. Humbe

Wednesday, 15 April 2015

referenced in VMStructs::localHotSpotVMStructs in the remote VM, was not present in the remote VMStructs

Hello Everyone,

Hope you all are doing well. Today here we are discussing about taking the heapdump using a jmap command. Here I have one command which successfully generate the heapdump for your running java process. While executing this command I had faced one error, which is very command and I think need to discuss with you people.

How to generate a heap dump using java process id (pid) :


The jmap is a very good utility to generate a heapdump. Using below command you can generate the heapdump for running java process

Command : jmap -dump:format=b,file=TestFile.hprof PID

Where TestFile.hprof is the dump file name you can change it accordingly.
And PID is process id of running java process for which you want to generate the heapdump

But sometime while executing this command we get the below error:

Error :

Attaching to process ID XXXX, please wait...
Exception in thread "main" java.lang.RuntimeException: Type "nmethodBucket*", referenced in VMStructs::localHotSpotVMStructs in the remote VM, was not present in the remote VMStructs::localHotSpotVMTypes table (should have been caught in the debug build of that VM). Can not continue.
        at sun.jvm.hotspot.HotSpotTypeDataBase.lookupOrFail(HotSpotTypeDataBase.java:345)
        at sun.jvm.hotspot.HotSpotTypeDataBase.readVMStructs(HotSpotTypeDataBase.java:236)
        at sun.jvm.hotspot.HotSpotTypeDataBase.<init>(HotSpotTypeDataBase.java:70)
        at sun.jvm.hotspot.bugspot.BugSpotAgent.setupVM(BugSpotAgent.java:544)
        at sun.jvm.hotspot.bugspot.BugSpotAgent.go(BugSpotAgent.java:476)
        at sun.jvm.hotspot.bugspot.BugSpotAgent.attach(BugSpotAgent.java:314)
        at sun.jvm.hotspot.tools.Tool.start(Tool.java:146)
        at sun.jvm.hotspot.tools.JMap.main(JMap.java:126)

Cause:

I was getting this error because my path to jmap wasn't the same as the path to the java process i.e. targeting two different versions. In such a case you need to set a proper path before executing the jmap command or execute command with full path of the java you are currently using.

i.e.  /opt/WebSphere85/java_1.7_64/bin/jmap -dump:format=b,file=TestFile.hprof 27693.

where /opt/WebSphere85/java_1.7_64 is my current version of java for webSphere server.

This command will generate you heapdump in your current working directory.

Hope this will help you. Kindly comment for your suggestion and quires.  

"Effort only fully releases its reward after a person refuses to quit.”

 Regards,
 Akhilesh B. Humbe

Wednesday, 8 April 2015

java.lang.OutOfMemoryError: PermGen space

Hello Everyone,

Today one of WAS 8.0.0.5 Server get crashed and was showing the below error in SystemOut.log

Error:


[04/03/15 10:27:47:004 GMT] 00013e2b servlet       E com.ibm.ws.webcontainer.servlet.ServletWrapper service SRVE0014E: Uncaught service() exception root caus
e spring-ws: org.springframework.web.util.NestedServletException: Request processing failed; nested exception is java.lang.OutOfMemoryError: PermGen space

Cause:


As it's a java application running of WebSphere Application server it allocate some memory to run. The size of memory is split into different areas including PermGen Space. The size of all these areas set during the JVM startup. If you do not set the sizes yourself, platform specific defaults will be used. So  the “java.lang.OutOfMemoryError: PermGen space” message indicates that the Permanent Size area in memory is exhausted. And there are lot of reasons for this.

All Java classes are loaded and stored in the Java PermGen.All of these are allocated to PermGen and stay in PermGen.As you can see, the PermGen size requirements depend both upon the number of classes loaded as well as the size of such class declarations. So it is easy to see the main cause for the “java.lang.OutOfMemoryError: PermGen space“: either too many classes or too big classes are being loaded to the permanent generation.

Solution:


The first solution to the OutOfMemoryError due to PermGen should be obvious. If we have exhausted the PermGen area in the memory we need to increase its size. This solution is indeed helpful if you just have not given your JVM enough below room. So alter your application launch configuration and add (or increase if present) the following:
-XX:MaxPermSize=512m (Suitable Size)
This configuration parameter is indicating to the JVM that PermGen is allowed to grow up to 512 MB before complaining in the form of OutOfMemoryError.

Second Solution  is you can take the heap dump at the time of issue and analyze it to find which objects causes the OutOfMemoryError in PermGen space

Third possibility is to allow GC to unload classes from PermGen. The standard JVM is rather conservative in this regard  Classes are born to live forever. So once loaded, classes stay in memory even if no one is really using them anymore. This can become a problem when the application is creating lots of classes dynamically and the generated classes are not needed for longer periods. In such a case, allowing JVM to unload class definitions can be helpful. This is achieved by adding again just one configuration parameter to your startup scripts:
-XX:+CMSClassUnloadingEnabled
By default this is set to false and so to enable this you need to explicitly set the following option in Java options. If you enable CMSClassUnloadingEnabled, GC will sweep PermGen too and remove classes which are no longer used. Keep in mind that this option will work only when UseConcMarkSweepGC is also enabled using the below option. So when running parallel or, God forbid, serial GCs, make sure you have set your GC to CMS by specifying:
-XX:+UseConcMarkSweepGC
But before calling it a night, be warned  more often than not usage of the recommended “quick fixes” means you are just masking the symptoms by hiding “java.lang.OutOfMemoryError: Permgen space” and are not tackling the underlying problem.

I personally recommend you to use the 1st and 2nd solution. And hope it will help you to resolve the issue.

"Effort only fully releases its reward after a person refuses to quit.”

 Regards,
 Akhilesh B. Humbe

Popular Posts