An Architect's View

ColdFusion MX 7 makes it so easy to run code in parallel that you're probably going to go mad with it once you start. If you're using my Concurrency library, it can be very tempting to run many of your methods concurrently. You'll do it the name of performance, to make a request run faster. When I started this series of blog entries, I mentioned that you don't want too many simultaneous requests to be processed by ColdFusion because it can cause two much contention for CPU resources and you can end up thrashing your server as it constantly switches from request to request trying to give each some CPU time. That's why the "3-5 requests per CPU" rule of thumb is useful. The same consideration is true of your event gateways. If you try to run everything in parallel, you may well end up thrashing your server as it spends more time switching between threads than actually getting any work done. The default setting for the number of event gateway threads is 10. You might think that setting this higher will mean better performance but you probably don't want it set higher than 5 times the number of CPUs you have if your event gateway tasks are CPU intensive. If your event gateway tasks are I/O-bound, or are mostly waiting for external systems (e.g., running a slow web service call in the background), then you can afford to have more of them executing at once and you can set the number of event gateway threads higher. We had a 4 CPU server and someone had pushed the number of threads up to 40. The result was that under load, all the threads were active and the server started thrashing. We cut the number of threads back to 20 and the performance improved dramatically. We are actually still able to thrash the server under load so we are going to throttle the parallelism a little more at some point. Oh, the load test scenario we'd forgotten? If you've read all three of these Asynchronous Development entries, you've probably guessed by now: we did not test the scenario where we have a lot of data queued up, ready to be consumed, and you start all of the event gateways at the same time. We had too much parallelism and too many automatically started gateways - when we brought up ColdFusion, it immediately started consuming data and overwhelmed the server. We now know how much of a backlog of data we can consume without causing a problem and we have also tested a process for consuming larger backlogs: starting and stopping the event gateway to let it process part of the backlog each time. One very useful thing to note about the event gateways: if you ask them to stop, they do so gracefully, accepting no more entries on the queue but waiting for active threads to complete before actually stopping.