Whenever we want to carry out lengthy tasks or process large batches of data, we should consider executing that process in the background. The most typical way of doing this is through “cron jobs” – tasks which are scheduled to be executed at regular intervals; every 2 minutes, or at 6am every Sunday. Alternatively, you can have the web browser instigate the task by using something like shell_exec(). This guide presents a third alternative, system level daemons.

Introduction

Put simply, a daemon is a very long running process. Even when it has completed all its tasks, it stays resident, waiting for some work to do. They may spend a lot of their time sleeping, but if designed correctly, they spring to life when required. Of course, they may be constantly listening out for events (by reading a named pipe for example), rather than periodically checking to see if work is available.

Why is a daemon better than a cron job

For the same reason that web pages are not served by a cronjob running every minute. Daemons are designed to be much more responsive. Cron jobs are very reliable, but sometimes they are less “elegant”.

Why is a daemon better than shell_exec()

Irrespective of whether your process is triggered by an HTTP request or some other event, it might not be appropriate to create one process per request. If your process requires considerable resources, then you may end up swamping the server.

Broadly speaking, daemons can come in two flavours:

• Single background process
• Multiple background processes managed by a parent

Single background process

The parent creates a copy of itself (a process known as “forking”) and then dies. This leaves the single process running in the background. This type of daemon is very easy to manage and develop.

Multiple background processes managed by a parent

The parent creates a child, and then dies. That new child then creates multiple children. Each of these children is a “worker thread” and responsible for the heavy lifting. The parent of those children is responsible for distributing their workload amongst them. Apache HTTPd is a prime example of this – in the configuration you can choose how many children it can create, the minimum number of children to have available, etc.

Daemons provide a nice balance between the managed rigidity of a cron job while still being responsive.

Sample Application

I have recently found a handy PEAR library for creating PHP daemons – System_Daemon. It greatly simplifies the task of creating a daemon, although the library only supports “Single background process” type daemons.

1
2
3
4
5
6
7
8
9
10

require_once "System/Daemon.php";
System_Daemon::setOption("appName", "myDaemon");
System_Daemon::start();
while(true) {
   sleep(1); // prevent CPU hogging
   if (mt_rand(0,10) == 1) {
      // simulate a random event which requires handling
      System_Daemon::log(System_Daemon::LOG_INFO, "Doing something interesting...");
   }
}

This daemon does pretty much nothing, other than stay in the background, slowly adding to a log file. However, it demonstrates that you can have a very “lightweight” process listening for system events in only a few lines of code. It’s also important to have some element of a “forever” loop in there to keep the process going.

Signal Handlers

Signals are a very basic form of Inter-process communication. In its most common usage, you would send a SIGTERM signal to a process by using the kill command. We can trap SIGTERM signals explicitly and shutdown “nicely”. This means any work being carried out can be finished off cleanly, and then the process will terminate. We can trap many other signals, all of which can be sent via “kill” on the command line, or more interestingly, the PHP function posix_signal().

Important tip for signals

I discovered this recently : PHP’s sleep() method is interrupted by signal interrupts. This means you can have a fairly lazy daemon (e.g. sleeps for 5 seconds every interation) and when you want to wake it up, you send it a SIGTERM signal. When interrupted this way, sleep() returns how many seconds were left at the point of interruption.

System_Daemon wraps up pcntl_signal() for us into

1

System_Daemon::setSigHandler($signalConstant, $handlerName);

Init.d scripts

The System_Daemon library can generate init.d scripts for us, which means (for me at least) I can and stop/start a daemon with:

/sbin/service myDaemon restart

Because we studiously assigned a SIGTERM handler, when we ask the daemon to stop using our init.d script, it will shutdown cleanly.

Real Life Examples

Reporting System

Reports can take anywhere between 10 seconds and 3 hours to generate. Requests for report generation were stored in a database and the Reporting Daemon picked them up and processed them. The report was written to disk and made available in a reports area. The system remained responsive irrespective of how many reports were requested.

SMS Processing

A large, multinational mobile promotions company received hundreds of SMSes a second. In order to be an effective product, the SMS request must have a response as quickly as possible. During promotional periods SMS traffic would sky rocket. Each inbound SMS was written to a database and then awaited processing by the daemon. A similar system was written for outbound SMS messages. The entire SMS platform used about 5 PHP Daemons and was the bread-and-butter of a £40m+ company.

Co-ordinating remote service requests

On a much smaller scale than the previous two examples, I wrote a simple caching daemon. The daemon would listen out for SIGINT signals (sent by a web page using posix_signal()) During quiet periods, the daemon would forward requests to the remote service as they came in. During busy periods, the daemon would batch them together. Because the daemon acted as a centralised point for communications, it was the best place to judge work load. For those interested, the web page communicated with the daemon using System V queues, with more information here.

Things to watch out for

• The PCNTL library (required for this excercise) is only available when running PHP on the command line (cli mode)
• You will soon notice that calls to echo or print are quite distracting when the daemon is running.
• Going from personal experience, fatal errors can be hard to track down. Sometimes the process will die with nothing having been written to /var/log/php_error.
• Any resources opened before forking, may not be available after the fork (e.g file pointers, db connections). Because pcntl_fork() creates a copy of the current process, when the parent is killed off, all resources are closed. The child, now holding a reference to a closed resource, will trigger an error when that resource is accessed.
• Certain resources close after a period of inactivity (I’m looking at you, MySQL) – check that your resources are available before using them.
• Because the script never shuts down (or shuts down rarely) memory leaks can be problematic.