Suspend and Resume thread (Windows, C) -

i'm developing heavily multi-threaded application, dealing lots of small data batch process.

the problem many threads being spawns, slows down system considerably. in order avoid that, i've got table of handles limits number of concurrent threads. "waitformultipleobjects", , when 1 slot being freed, create new thread, own data batch handle.

now, i've got many threads want (typically, 1 per core). then, load incurred multi-threading extremely sensible. reason this: data batch small, i'm creating new threads.

the first idea i'm implementing regroup jobs longer serial lists. therefore, when i'm creating new thread, have 128 or 512 data batch handle before being terminated. works well, destroys granularity.

i asked scenario: if problem comes "creating" threads often, "pausing" them, loading data batch , "resuming" thread?

unfortunately, i'm not successful. problem is: when thread in "suspend" mode, "waitformultipleobjects" not detect available. in fact, can't efficiently distinguish between active , suspended thread.

so i've got 2 questions:

1. how detect "suspended thread", can load new data , resume it?

2. is idea? after all, "createthread" ressource hog?

edit

after testings, here findings concerning thread pooling , io completion port, both advised in post.

thread pooling tested using older version "queueuserworkitem". io completion port requires using createiocompletionport, getqueuedcompletionstatus , postqueuedcompletionstatus;

1) first on performance : creating many threads costly, , both thread pooling , io completion ports doing great job avoid cost. down 8-jobs per batch, earlier 512-jobs per batch, no slowdown. considerable. when going 1-job per batch, performance impact less 5%. remarkable.

from performance standpoint, queueuserworkitem wins, albeit such small margin (about 1% better) negligible.

2) on usage simplicity : regarding starting threads : no question, queueuserworkitem far easiest setup. io completion port heavyweight in comparison. regarding ending threads : win io completion port. unknown reason, ms provides no function in c know when jobs completed queueuserworkitem. requires nasty tricks implement basic critical function. there no excuse such lack of feature.

3) on resource control : big win io completion port, allows finely tune number of concurrent threads, while there no such control queueuserworkitem, happily spend cpu cycles available cores. that, in itself, deal breaker queueuserworkitem. note newer version of completion port seems allow control, available on windows vista , later.

4) on compatibility : small win io completion port, available since windows nt4. queueuserworkitem exists since windows 2000. enough. newer version of completion port no-go windows xp.

as can guessed, i'm pretty tied between 2 solutions. both answer correctly needs. general situation, suggest i/o completion port, resource control. on other hand, queueuserworkitem easier setup. quite pity loses of simplicity on requiring programmer deal alone end-of-jobs detection.

if want support windows xp, cannot use createthreadpool -- otherwise, if vista , newer sufficient, windows thread pools easiest way.

if windows xp support needed, spawn number of threads , assign them io completion port, have each thread block on getqueuedcompletionstatus(). completion ports let post events port wake 1 thread per event, , efficient. use lifo strategy on waking threads keep caches warm, too.

in case, never want suspend thread. never ever. block, wait, don't suspend.

the reason suspend problem describe, plus create deadlocks, e.g. if thread within critical section or mutex. aside debugger, nobody should ever need suspend thread.