Friday, August 26, 2011

Asterisk disk full

Just after my last class today I though I'd copy the asterisk box from the presenter machine to my backup drive. I was shocked to see a 44GB file! Obviously there was some logging problem bloating the hard disk, but it was not asterisk's log files under var/asterisk but instead core files found in tmp.
These core files are effectively memory dump files created by CentOS when Asterisk mibehaves and crashes. I had 43 GB worth of dump files already. The problem is, that simple errors in your sip.conf can cause Asterisk to crash and restart and subsequently crash, going on until your HDD is full of dump files.
I found a easy post on how to control the core dumps here:
http://aplawrence.com/Linux/limit_core_files.html

But that only combats half the problem. The main part was why was asterisk is crashing so often. the answer lied in my case in the qualify=yes entries in my sip.config. If the Lync Mediation server does not expect to respond to the gateway (topology not published yet, Mediation Service not restarted) or is not online at all then the qualify will fail. Worse even, it will crash my Asterisk 1.6 box causing those dumps to quickly fill up the HDD (50MB per dump). So setting qualify=no has helped me keep my asterisk box more stable between my demonstrations.