Thursday 28 January 2016

java.lang.OutOfMemoryError: GC overhead limit exceeded

Hello Everyone,

Last time we discussed the issue of java.lang.OutOfMemoryError: Java heap space for Apache Tomcat server. Similarly today we are having another JVM related issue for WebSphere application server. Here my users are complaining about the some kind of slowness while application access and when I check for the error I am getting below error string in logs.

Error:


java.lang.OutOfMemoryError: GC overhead limit exceeded

Cause:


Before solution we need to know what is GC and why we are getting this error called  GC overhead limit exceeded.
Java runtime environment contains a built-in Garbage Collection (GC)  daemon process. Whenever a particular space in memory is no longer used, a process called Garbage Collection clears that memory space and GC do this work on regular basis. There are different algorithms and parameter of GC which defines how GC will work. And if in case GC is not performing well the it will write the error code similar to what we get.

The java.lang.OutOfMemoryError: GC overhead limit exceeded error is displayed when your application has exhausted pretty much all the available memory and GC has repeatedly failed to clean it. The java.lang.OutOfMemoryError: GC overhead limit exceeded error is the JVM’s way of signaling that your application spends too much time doing garbage collection with too little result. By default the JVM is configured to throw this error if it spends more than 98% of the total time doing GC and when after the GC only less than 2% of the heap is recovered. This means that the small amount of heap the GC is able to clean will likely be quickly filled again, forcing the GC to restart the cleaning process again. This forms a vicious cycle where the CPU is 100% busy with GC and no actual work can be done and end users of the application face extreme slowdowns

Solution: 


You can resolve this issue by using below options.

Step 1
Disable the error check using -XX:-UseGCOverheadLimit. You can disable the error check by adding the option  -XX:-UseGCOverheadLimit to the Generic JVM arguments. This will help you to stopping these error alerts. Please keep in mind that this error is very likely to be a symptom of a JVM Heap / tuning problem so my recommendation to you is always to focus on the root cause as opposed to the symptom instated disabling the error alert .

Step 2
You may also try to change Garbage Collector Policy. You can add the below policy of Garbage collector option to the Generic JVM arguments.
-XX:+UseConcMarkSweepGC 
-XX:+UseConcMarkSweepGC this can help you in the space issue as it compact all fragment free space after GC run and make you available more contiguous space.
-XX:+UseParallelGC
This -XX:+UseParallelGC parameter help you to run the GC continuously instated of reaching to the memory space threshold.

Step 3
If the option two not works you can  Increase heap size using -Xmx switch in Generic JVM arguments

Step 4
If option three not works add -XX:+HeapDumpOnOutOfMemoryError to Generic JVM arguments which will automatically generate heap dump in case of out of memory issue. Then analyze the dump with IBM Memory Analyzer looking for memory leaks. On the basis of analysis you can work with the development team, optimize your code to use less memory and/or reuse objects instead of creating new ones thus reducing the number of times the garbage collector runs.

Sometime dealing with the OutOfMemoryError is a log process, you need to analyze lot of things for perfect tuning and sometime it just resolved by JVM parameter setting.

Hope this will help you. Kindly comment for your suggestion and quires.  

"Effort only fully releases its reward after a person refuses to quit.”

 Regards,
 Akhilesh B. Humbe

Friday 22 January 2016

java.lang.OutOfMemoryError: Java heap space

Hell Everyone,

After a very long time I am working on Apache tomcat. While accessing application login page user facing some issue and getting some errors on the web browser. When I check the tomcat output logs I seen the below error.

Log Location:

/usr/local/apache-tomcat-5.5.23/logs/catalina.out

Error:

Jan 14, 2016 10:40:36 AM org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor processChildren
SEVERE: Exception invoking periodic operation:
java.lang.OutOfMemoryError: Java heap space
        at java.util.HashMap.<init>(HashMap.java:203)
        at java.util.LinkedHashMap.<init>(LinkedHashMap.java:178)
        at org.apache.catalina.startup.HostConfig$DeployedApplication.<init>(HostConfig.java:1314)
        at org.apache.catalina.startup.HostConfig.deployDirectory(HostConfig.java:898)
        at org.apache.catalina.startup.HostConfig.deployDirectories(HostConfig.java:883)
        at org.apache.catalina.startup.HostConfig.deployApps(HostConfig.java:492)
        at org.apache.catalina.startup.HostConfig.check(HostConfig.java:1206)
        at org.apache.catalina.startup.HostConfig.lifecycleEvent(HostConfig.java:293)
        at org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(LifecycleSupport.java:120)
        at org.apache.catalina.core.ContainerBase.backgroundProcess(ContainerBase.java:1306)

Solution:

As you can see in error it's a java.lang.OutOfMemoryError: Java heap space means we don’t have enough heap memory available  to write new objects of an application. There is not a single specific way to resolve this issue. There may be a different reasons of OutOfMemoryError. You can find out that by generating and analyzing heap dump.

In my case the allocated heap memory for running the application was too less as compared to my requirement. So I increase the Java Heap memory  and it resolve my issue.

How to increase the java heap space in Apache tomcat ?


When you initiate the command to start the Apache tomcat instance it take the configuration from the file catalina.sh. If you want to increase the java heap space you need to manually edit this file. For increasing the java heap space increase the value of -Xmx parameter for the CATALINA_OPTS
E.g. 
cd /usr/local/apache-tomcat-5.5.23/bin/
vi catalina.sh

JAVA_HOME=/usr/java
CATALINA_OPTS="-Xmx512m " [In my case previously it was -Xmx126m]

Save the Configuration changes and start restart the Apache tomcat service.

/usr/local/apache-tomcat-5.5.23/bin/shutdown.sh
/usr/local/apache-tomcat-5.5.23/bin/startup.sh

Note: Do not Increase the heap size blindly, it will not help you every time in case of the Out Of Memory Error. Make a habit to analyze the heap dump and try to find out the best suitable JVM  arguments for your environment.

Important:

How to take a Heap Dump.
Best Guide for Garbage Collection
Best Guide for JVM tuning.

Hope this will help you. Kindly comment for your suggestion and quires.  

"Effort only fully releases its reward after a person refuses to quit.”

 Regards,
 Akhilesh B. Humbe

Sunday 10 January 2016

CRIMC1066E ERROR: Unable to acquire lock on file .imlock''.

Hello Everyone,

Happy New Year 2016 to all readers. It's 1st week of the new year and I am getting new error while initiating IBM Installation Manager console.

CRIMC1066E ERROR:   Unable to acquire lock on file  /export/localhome/was/var/ibm/InstallationManager/.imlock''.

Problem:

While executing command ./imcl -c to launch the command line console of IBM installation manager it displays the following error message.
CRIMC1066E ERROR:   Unable to acquire lock on file  /export/localhome/was/var/ibm/InstallationManager/.imlock''.
The installation manager could not be started because another installation manager instance is already running. Only a single installation manager may be running at any time.

Cause:

We are getting this error if the installation manager is already running by some other user. When we start Installation manager the user create lock file .imlock which not allow other user to run installation manager to other location. This file get deleted when you close the installation manager. In some case if installation manager process got killed accidentally the lock file remains as it is and not allow you to start the installation manager next time and showing the error code. 

Resolution:

1. Ensure that there is no running Installation Manager instance.
2. If there is an Installation manager process running, delete that process
Unix: ps -ef | grep imcl or  ps -ef | grep java (Run these commands to find the process)
Windows: Go to the Task Manager and terminate all the java.exe, javaw.exe and IBMIM.exe processes.
3. Re-launch Installation Manager.

If the above steps do not resolve the issue, follow these steps:

1. Go to location list in the error code and manually delete .imlock file.
Unix: ls -a (use this command to list down the hidden files)
2. Hope this will not require, but if necessary, restart the system.

Hope this will help you. Kindly comment for your suggestion and quires.   

"Effort only fully releases its reward after a person refuses to quit.”

 Regards,

 Akhilesh B. Humbe

Popular Posts