c# - Odd behavior with yield and Parallel.ForEach -

at work 1 of our processes uses sql database table queue. i've been designing queue reader check table queued work, update row status when work starts, , delete row when work finished. i'm using parallel.foreach give each process own thread , setting maxdegreeofparallelism 4.

when queue reader starts checks unfinished work , loads work list, concat list , method returns ienumerable runs in infinite loop checking new work do. idea unfinished work should processed first , new work can worked threads available. i'm seeing fetchqueuedwork change dozens of rows in queue table 'processing' work on few items @ time.

what expected happen fetchqueuedwork new work , update table when slot opened in parallel.foreach. what's odd me behaves expect when run code in local developer environment, in production above problem.

i'm using .net 4. here code:

public void go() {     list<workdata> unfinishedwork = workdata.loadunfinishedwork();     ienumerable<workdata> work = unfinishedwork.concat(fetchqueuedwork());          parallel.foreach(work, new paralleloptions { maxdegreeofparallelism = 4 }, dowork); }  private ienumerable<workdata> fetchqueuedwork() {     while (true)     {         var workunit = workdata.getqueuedworkandsetstatustoprocessing();         yield return workunit;     } }  private void dowork(workdata workunit) {     if (!workunit.loaded)     {         system.threading.thread.sleep(5000);         return;     }     work(); } 

i suspect default (release mode?) behaviour buffer input. might need create own partitioner , pass nobuffering option:

list<workdata> unfinishedwork = workdata.loadunfinishedwork(); ienumerable<workdata> work = unfinishedwork.concat(fetchqueuedwork());      var options = new paralleloptions { maxdegreeofparallelism = 4 }; var partitioner = partitioner.create(work, enumerablepartitioneroptions.nobuffering); parallel.foreach(partioner, options, dowork);