ActiveMQ purge – file system is full

ACTIVEMQ Purge – how to fix a full disk

/var full files in /var/lib/activemq/data/kahadb directory

This is by no means an activeMQ tutorial, it’s just a brief document aimed at helping you solve an issue I had and wasn’t able to find any answers that helped me.

The Issue

You might find your disk is filling up with log files in/var/lib/activemq/data. You need to do an activemq purge. You’ll probably want to speak to someone in your organisation that understand what ActiveMQ is and how it’s used. You’ll definitely need to speak to someone, if you don’t know yourself if you’re dealing with a production server.

When I encountered this issue,  /var was full and I’d already moved some log files out to another disk. As the file system was full, I couldn’t copy the log files back to where they came from. This mean I had to extend the LV and FS (detailed below). This was all done without stopping activeMQ or un mounting the file system and took all of 15 minutes.

I’ve carried out this procedure to great effect. This is a bit of a brain dump, but here for reference. I couldn’t find any information as specific as this.

The Solution

Run activemq dstat  this gives you queue stats, there will be a large number under the QueueSize column.

activemq purge {your_queue_name}

Enter the queue that had the large queue size in place of {your_queue_size}

This should move the number in the Queue Size column into the Dequeue column and the files in /var/lib/activemq/data/kahadb should have decreased accordingly. The log files in /var/lib/activemq/data/kahadb aren’t normal text log files, they are similar to transaction logs that need to be processed. If you read more about activemq you can find out the meaning of these files, I will try to add more info on this when I can.

You don’t need to stop the process to run activemq purge.

I have also had to move older log files out of the way to make space available. Make sure you move the old files back into you kahadb directory before running the purge, it will error otherwise.

Now your disk space will be recovered and your Queue Size should be empty.

If you need to know how to extend the logical volume and file system I have created a couple of pages to give you examples. It’s not a full blown LVM tutorial, but if everything is as it should be it will certainly tell you what you need to do, without necessarily being a RedHat Certified SA. So, have a look at http://devopslinux.net/logical-volume-manager/ and http://devopslinux.net/increase-filesystem-online-linux/

As always, feedback welcome and I can expand on anything you think needs expanded on.

Graham Bell

Owner, author, grafter, tea boy - you guessed it, one man band. Now a "hired hand" Linux and DevOps expert. Spent over 14 years in the Finance industry but has also worked in Pharmaceuticals, Telcos, Public Sector and a few Blue Chip global organisations.

You may also like...

Leave a Reply

Your email address will not be published. Required fields are marked *