JAVA Memory Allocation Failures Due to Insufficient Swap Space
If you see JAVA Runtime insufficient memory errors like these – then swap space is the issue.
- for jira failures –
- for jenkins failures –
There is insufficient memory
The memory allocation error log files often begin with this text.
# # There is insufficient memory for the Java Runtime Environment to continue. # Native memory allocation (mmap) failed to map 65536 bytes for committing reserved memory. # Possible reasons: # The system is out of physical RAM or swap space # In 32 bit mode, the process size limit was hit # Possible solutions: # Reduce memory load on the system # Increase physical memory or swap space # Check if swap backing store is full # Use 64 bit Java on a 64 bit OS # Decrease Java heap size (-Xmx/-Xms) # Decrease number of Java threads # Decrease Java thread stack sizes (-Xss) # Set larger code cache with -XX:ReservedCodeCacheSize= # This output file may be truncated or incomplete. # # Out of Memory Error (os_linux.cpp:2627), pid=9477, tid=139985815168768 # # JRE version: Java(TM) SE Runtime Environment (8.0_74-b02) (build 1.8.0_74-b02) # Java VM: Java HotSpot(TM) 64-Bit Server VM (25.74-b02 mixed mode linux-amd64 compressed oops) # Failed to write core dump. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again #
Amazon AWS EC2 and Oracle VirtualBox Virtual Machines Swap Space
free -m to confirm that the problem is swap space. Two common strands of the same problem emerge.
An Oracle VirtualBox machine running Ubuntu 16.04 or Ubuntu 15.04 may run out of swap space like below.
Swap Space Used Up on Oracle VirtualBox Virtual Machine
It says that 454MB of the 509MB configured swap space is utilised. The little left causes VM (virtual machines) issues like this.
bbw@sandbox:~$ free -m total used free shared buffers cached Mem: 489 482 6 0 53 41 -/+ buffers/cache: 387 101 Swap: 509 454 55
No Swap Space on Amazon AWS Ubuntu EC2 Server
This is an Amazon AWS EC2 machine that is saying that it has no swap space.
ubuntu@ip-172-31-43-133:~$ free -m total used free shared buffers cached Mem: 992 475 517 26 10 93 -/+ buffers/cache: 370 621 Swap: 0 0 0
Configure Swap Space on Ubuntu 14.04, 15.04 and 16.04
This is the swap space configuration trace on an Ubuntu virtual machine that has too little swap. The commands are exactly the same for a cloud server like the Amazon AWS EC2 machines.
Too Little Swap Space
total used free shared buffers cached Mem: 489 482 6 0 53 41 -/+ buffers/cache: 387 101 Swap: 509 454 55
Enough Hard Disk Space to Borrow From
Filesystem Size Used Avail Use% Mounted on /dev/sda1 16G 12G 2.5G 84% / none 4.0K 0 4.0K 0% /sys/fs/cgroup udev 235M 4.0K 235M 1% /dev tmpfs 49M 392K 49M 1% /run none 5.0M 0 5.0M 0% /run/lock none 245M 0 245M 0% /run/shm none 100M 0 100M 0% /run/user
The Commands to Allocate New (Or More) Swap Space
sudo fallocate -l 1G /swapfile
ls -lh /swapfile
-rw-r--r-- 1 root root 1.0G May 19 13:14 /swapfile
Other Machine Users Should Not Be Allowed to Read From or Write To the Swap File
sudo chmod 600 /swapfile
ls -lh /swapfile
-rw------- 1 root root 1.0G May 19 13:14 /swapfile
Tell Ubuntu it is a Swap File
sudo mkswap /swapfile
The response from the system should be similar to this.
Setting up swapspace version 1, size = 1048572 KiB no label, UUID=79e4554e-931b-4e0b-81d7-1baaf8f0b1e0
sudo swapon /swapfile
Ask Ubuntu – How Much Swap Space Do You Have?
Now the tables have turned.
sudo swapon -s
The report on the Oracle VirtualBox Ubuntu Sandbox has an extra line of swap space available.
Filename Type Size Used Priority /dev/sda5 partition 522236 457760 -1 /swapfile file 1048572 0 -2
The Amazon Web Services (AWS) EC2 instance which had no swap space to speak of now reports that it has 1Gig of swappable space.
Filename Type Size Used Priority /swapfile file 1048572 0 -1
Query Ubuntu – For a Free Memory Report
total used free shared buffers cached Mem: 489 481 7 0 51 36 -/+ buffers/cache: 393 95 Swap: 1533 447 1086
The above free command now tells us that there is
1G (not the 55K) of swap available.
Server Restart – the Swap Space must Survive a Restart
# /etc/fstab: static file system information. # # Use 'blkid' to print the universally unique identifier for a # device; this may be used with UUID= as a more robust way to name devices # that works even if disks are added and removed. See fstab(5). # #
# / was on /dev/sda1 during installation UUID=c72ba3d9-16b6-474e-903c-5a26db4cf431 / ext4 errors=remount-ro 0 1 # swap was on /dev/sda5 during installation UUID=dc7f44e0-c436-4ecc-bdbf-6265a5938101 none swap sw 0 0 /swapfile none swap sw 0 0
Make sure you tell Ubuntu at the end of the file – about our new swap file.