That is most certainly strange. Personally I would suspect the cURL module first since PHP and Apache are specifically designed to run in parallel. Perhaps it's implemented in such a way that PHP speaks to only one cURL process, that's possible, although I'd call that a catastrophic design flaw if that were the case.

PHP's cURL library can however do asynchronous requests, even from a single PHP execution. While it's arguably not a proper solution to the weirdness of your problem, you may have more luck with it.

Here's one tutorial that I found and looks promising:

http://www.phpied.com/simultaneuos-http-requests-in-php-with-curl/

Hope it helps! Let us know. :)

Kind regards, Helgi Hrafn Gunnarsson helgi@binary.is

On Tue, Mar 15, 2011 at 4:02 PM, John Lange john@johnlange.ca wrote:

I have some code (written in PHP) that checks on the status of a single "server" but I need this code to scale up to hundreds of parallel checks.

For example, you call the script like this:

http://mytest.com/checkserver.php?ip=10.11.12.13

where the ip=X.X.X.X is different every time depending on which server you are checking.

And the PHP looks something like this:

<?php $ch = curl_init(); curl_setopt($ch, CURLOPT_URL, "http://".$ip."/status.html"); curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1); $output = curl_exec($ch); curl_close($ch); ?>

To my way of thinking, this should scale easily. Lets say 200 people hit the page at the same time all passing different "ip=" values, this should check all 200 different servers in parallel and return results.

It doesn't. I wrote this little bash script to test it (yes I'm using command line curl to test php-curl, please don't be confused by that):

---

#!/bin/bash

IPS="10.18.136.20 10.18.136.21 10.18.136.22 ... ( a hundred more IPs)"

for IP in $IPS ; do curl -s http://mytest.com/checkserver.php?ip=$IP & done

---

All the curl commands launch into the background as you expect and I see all the apache child threads startup, but the results return one by one.

I can't for the life of me figure out how this can be possible. Each apache thread should run it's PHP in a separate thread and return in parallel. The only thing I can think of is that php is tracking all the requests as being part of the same session and imposing some limit on the outbound curl requests... Or maybe it's apache blocking it?

Just thought someone might have had to do something similar in the past and run into this.

By the way, if I go direct like this:

for IP in $IPS ; do curl -s http://%22.$IP%22/status.html & done

It works just exactly as you'd expect, all results return in parallel so it's got to be a problem with either PHP or apache.

-- John Lange www.johnlange.ca _______________________________________________ Roundtable mailing list Roundtable@muug.mb.ca http://www.muug.mb.ca/mailman/listinfo/roundtable

On 03/15/2011 04:02 PM, John Lange wrote:

I have some code (written in PHP) that checks on the status of a single "server" but I need this code to scale up to hundreds of parallel checks.

I can't for the life of me figure out how this can be possible. Each apache thread should run it's PHP in a separate thread and return in parallel. The only thing I can think of is that php is tracking all the requests as being part of the same session and imposing some limit on the outbound curl requests... Or maybe it's apache blocking it?

If your apache is really threaded, try switching to the non-threaded prefork httpd.

Peter