How to post an asynchronous HTTP request in PHP

Photo by Zombizi

The way users make things happen with web applications is by fetching from a URL, passing in some parameters, eg http://somewhere.com/somescript.php?action=dosomething. For an architecture like Facebook, this is also how API calls are made. I was pretty surprised to find that PHP has a hard time doing something this simple asynchronously.

The key problem is that most HTTP libraries are designed around two-way communication. You send off a request and then wait for a response. In this case I don't want a reply, I just want the request to trigger some action on the other end, which might eventually involve that server calling me back with a similar fetch with some result, or it might just update an internal database. I want my PHP script to fire off that request and then continue executing, but the cURL functionality that's built in always waits for the response before carrying on.

At first, I just needed to make an occasional call like this, and I found a hacky solution that set the timeout on the cURL fetch to 1 second. This meant the request was fired off, but then almost immediately timed-out. The problem is that almost immediately wasn't fast enough once you start calling this frequently, that 1 second every time builds up, and you can't set the timeout to 0. Here's that code:

function curl_post_async($url, $params)
{
    foreach ($params as $key => &$val) {
      if (is_array($val)) $val = implode(',', $val);
        $post_params[] = $key.'='.urlencode($val);
    }
    $post_string = implode('&', $post_params);

    $ch = curl_init();
    curl_setopt($ch, CURLOPT_URL, $url);
    curl_setopt($ch, CURLOPT_POSTFIELDS, $post_string);
    curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
    curl_setopt($ch, CURLOPT_USERAGENT, 'curl');
    curl_setopt($ch, CURLOPT_TIMEOUT, 1);
    $result = curl_exec($ch);
    curl_close($ch);
}

I needed something that didn't have that one-second penalty every time. PHP doesn't support threads out of the box, so I looked at using its flavor of fork, pcnt_fork(). It was looking promising until I realized that it's disabled by default in Apache, with some reason since there's a lot of process baggage to copy when it's running in that environment. I then toyed with the idea of using exec to spawn a cURL command-line instance to carry out the command, but that just seemed ugly, fragile and too heavy-weight. I looked at PHP's HTTP streams too, but they are also synchronous.

I was getting frustrated because HTTP is a simple protocol at heart, and it shouldn't be this hard to do what I need.

At last, White Shadow came to the rescue. His post talks about a few different ways of doing what I need, but importantly he has one based on raw sockets, and closing the connection immediately after writing the post data. This is exactly what I needed, it fires off the request and then returns almost immediately. I was able to get a lot better performance using this technique.

function curl_post_async($url, $params)
{
    foreach ($params as $key => &$val) {
      if (is_array($val)) $val = implode(',', $val);
        $post_params[] = $key.'='.urlencode($val);
    }
    $post_string = implode('&', $post_params);

    $parts=parse_url($url);

    $fp = fsockopen($parts['host'],
        isset($parts['port'])?$parts['port']:80,
        $errno, $errstr, 30);

    pete_assert(($fp!=0), "Couldn't open a socket to ".$url." (".$errstr.")");

    $out = "POST ".$parts['path']." HTTP/1.1\r\n";
    $out.= "Host: ".$parts['host']."\r\n";
    $out.= "Content-Type: application/x-www-form-urlencoded\r\n";
    $out.= "Content-Length: ".strlen($post_string)."\r\n";
    $out.= "Connection: Close\r\n\r\n";
    if (isset($post_string)) $out.= $post_string;

    fwrite($fp, $out);
    fclose($fp);
}

Comments

[Garry]

thanks man! just what I was looking for