Compcache on Ubuntu on Amazon EC2

The following fully-automatic Bash script downloads, compiles, and initializes compcache version 0.6.2 on Ubuntu Karmic Koala (9.10). This script creates two swaps with a maximum of 4GB uncompressed size each. Two swaps are used to take advantage of 2 CPUs (or CPU cores in a multicore CPU).

Compcache is a fascinating memory compression system. The old alternative to compcache is to use a swap file backed by a hard disk (without using compression), but this kind of swapping is extremely slowed to physical memory or memory compressed by compcache. Compcache creates a swap device backed by compressed memory or by another swap file. Basically, you get more RAM—almost for free—though compcache adds a small overhead for compression. Compression should actually increase performance when backed by a hard drive because it reduces expensive disk I/O. When memory pages are empty (which happens more often than you’d think), compcache stores nothing. I run data analysis using party in R, and it needs huge amounts of memory (well over 10GB). Amazon EC2 gives me a large starting memory capacity, and compcache extends the physical RAM.

The script doesn’t install or make permanent changes, so if you don’t like it, reboot to start over. It was tested on Amazon EC2 with Canonical 64-bit server (ami-55739e3c), and it should work fine with other most Ubuntu versions such as the new Lucid Lynx LTS (10.04) [update: see fix for asm/asm-offsets.h error on Lucid Lynx]. For other ways to use Compcache, see compcache: Compiling And Using New .

DIR=/tmp/compcache-0.6.2
CONTROL=$DIR/sub-projects/rzscontrol/rzscontrol

# install kernel headers
sudo apt-get install linux-headers-$(uname -r) gawk

# show memory before
free
cat /proc/swaps

# compile
cd /tmp
wget http://compcache.googlecode.com/files/compcache-0.6.2.tar.gz
rm -rf $DIR
tar xvzf compcache-0.6.2.tar.gz
cd $DIR
make

# load dependency modules
sudo modprobe lzo_compress
sudo modprobe lzo_decompress

# load ramzswap module
sudo insmod ramzswap.ko num_devices=2
sleep 1s

# Initialize devices with 4GB each upper limit of uncompressed memory.  This device does not have a backing swap.
sudo $CONTROL /dev/ramzswap0 --disksize_kb=4456448 --init
sudo $CONTROL /dev/ramzswap1 --disksize_kb=4456448 --init

# activate
sudo swapon -p 5 /dev/ramzswap0
sudo swapon -p 5 /dev/ramzswap1

# show stats
free
cat /proc/swaps
sudo $CONTROL /dev/ramzswap0 --stats
  • About these ads
  • One thought on “Compcache on Ubuntu on Amazon EC2

    1. Pingback: error: asm/asm-offsets.h: No such file or directory « Heuristic Andrew

    Leave a Reply

    Fill in your details below or click an icon to log in:

    WordPress.com Logo

    You are commenting using your WordPress.com account. Log Out / Change )

    Twitter picture

    You are commenting using your Twitter account. Log Out / Change )

    Facebook photo

    You are commenting using your Facebook account. Log Out / Change )

    Google+ photo

    You are commenting using your Google+ account. Log Out / Change )

    Connecting to %s