threading using for..next

Hi,
I have some intensive number crunching code that I want to break up
into threads so PCs with several processors can do the job quicker.

I have a loop that will go from say 1 to 10, and for each value,
different numbers will be crunched depending on what the value is.

As it is an intensive process the user may cancel the loop. I need to
know at what value of the loop the process was cancelled, so it can be
restarted at the next value next time.

My initial thought was to allocate 1-5 to thread 1 and 6-10 to thread
2 (for a 2 processor machine) - which shouldn't be too difficult. But
the issue with this if that if the user cancels then its not so neat
to resume as 1 and 6 might be the only ones completed.

Ideally I would like to do the following but am not quite sure how.
Any pointers or code snippets would be appreciated.

for 1 = 1 to 10 step numProcessors (say 2)



next i

baldrick
Sat May 10 16:11:45 CDT 2008

On 11 May, 07:04, baldrick <philbrier...@hotmail.com> wrote:
> Hi,
> I have some intensive number crunching code that I want to break up
> into threads so PCs with several processors can do the job quicker.
>
> I have a loop that will go from say 1 to 10, and for each value,
> different numbers will be crunched depending on what the value is.
>
> As it is an intensive process the user may cancel the loop. I need to
> know at what value of the loop the process was cancelled, so it can be
> restarted at the next value next time.
>
> My initial thought was to allocate 1-5 to thread 1 and 6-10 to thread
> 2 (for a 2 processor machine) - which shouldn't be too difficult. But
> the issue with this if that if the user cancels then its not so neat
> to resume as 1 and 6 might be the only ones completed.
>
> Ideally I would like to do the following but am not quite sure how.
> Any pointers or code snippets would be appreciated.
>


for i = 1 to 10 step numThreads (say 2)

allocate i to the next free thread

next i

This way as I loop through 1 to 10, and the user cancels at x, then I
know I have completed 1 through x and need to resume at x+1.

How do I set up the loop to wait for one of the 2 threads to finish
and then allocate the current unprocessed 'i' to that thread?

Hope this makes sense?

Pb

David
Sat May 10 17:11:13 CDT 2008

that doesn't require threading. you can use a delegate to stop a loop, and
start it again from sT (startThread)

for i = sT to eT

--
David Glienna
MVP - Visual Developer (Visual Basic)
2006 thru 2008
"baldrick" <philbrierley@hotmail.com> wrote in message
news:5ef87706-1981-40c6-9be0-ff4a6936606a@f36g2000hsa.googlegroups.com...
> On 11 May, 07:04, baldrick <philbrier...@hotmail.com> wrote:
>> Hi,
>> I have some intensive number crunching code that I want to break up
>> into threads so PCs with several processors can do the job quicker.
>>
>> I have a loop that will go from say 1 to 10, and for each value,
>> different numbers will be crunched depending on what the value is.
>>
>> As it is an intensive process the user may cancel the loop. I need to
>> know at what value of the loop the process was cancelled, so it can be
>> restarted at the next value next time.
>>
>> My initial thought was to allocate 1-5 to thread 1 and 6-10 to thread
>> 2 (for a 2 processor machine) - which shouldn't be too difficult. But
>> the issue with this if that if the user cancels then its not so neat
>> to resume as 1 and 6 might be the only ones completed.
>>
>> Ideally I would like to do the following but am not quite sure how.
>> Any pointers or code snippets would be appreciated.
>>
>
>
> for i = 1 to 10 step numThreads (say 2)
>
> allocate i to the next free thread
>
> next i
>
> This way as I loop through 1 to 10, and the user cancels at x, then I
> know I have completed 1 through x and need to resume at x+1.
>
> How do I set up the loop to wait for one of the 2 threads to finish
> and then allocate the current unprocessed 'i' to that thread?
>
> Hope this makes sense?
>
> Pb
>

baldrick
Sun May 11 01:16:49 CDT 2008

On 11 May, 08:11, "David Glienna" <dglie...@hotmail.com> wrote:
> that doesn't require threading. =A0you can use a delegate to stop a loop, =
and
> start it again from sT (startThread)
>
> for i =3D sT to eT
>
> --
> David Glienna
> MVP - Visual Developer (Visual Basic)
> 2006 thru 2008"baldrick" <philbrier...@hotmail.com> wrote in message
>
> news:5ef87706-1981-40c6-9be0-ff4a6936606a@f36g2000hsa.googlegroups.com...
>
>
>
> > On 11 May, 07:04, baldrick <philbrier...@hotmail.com> wrote:
> >> Hi,
> >> I have some intensive number crunching code that I want to break up
> >> into threads so PCs with several processors can do the job quicker.
>
> >> I have a loop that will go from say 1 to 10, and for each value,
> >> different numbers will be crunched depending on what the value is.
>
> >> As it is an intensive process the user may cancel the loop. I need to
> >> know at what value of the loop the process was cancelled, so it can be
> >> restarted at the next value next time.
>
> >> My initial thought was to allocate 1-5 to thread 1 and 6-10 to thread
> >> 2 (for a 2 processor machine) - which shouldn't be too difficult. But
> >> the issue with this if that if the user cancels then its not so neat
> >> to resume as 1 and 6 might be the only ones completed.
>
> >> Ideally I would like to do the following but am not quite sure how.
> >> Any pointers or code snippets would be appreciated.
>
> > for i =3D 1 to 10 step numThreads (say 2)
>
> > =A0 allocate i to the next free thread
>
> > next i
>
> > This way as I loop through 1 to 10, and the user cancels at x, then I
> > know I have completed 1 through x and need to resume at x+1.
>
> > How do I set up the loop to wait for one of the 2 threads to finish
> > and then allocate the current unprocessed 'i' to that thread?
>
> > Hope this makes sense?
>
> > Pb- Hide quoted text -
>
> - Show quoted text -

David,
I must have not made myself very clear. How to stop the loop was not
the question. I can do that quite easily.

I want to loop through 1 to 10, allocate 1 to the first thread, 2 to
the second, then 3 to either thread 1 or 2, depending on which one has
finished, or wait if neither have finished, same for 4 etc...

I could easily allocate 1-5 to thread 1 and 6-10 to thread 2 but I
want to do it a bit more in order.

The reason for threading is that each i of the loop will take about 10
minutes so I want to do 2 at once so it will run on 2 processors an be
done in half the time.

Would appreciate any pointers or code snippets.

Phil

Cor
Sun May 11 02:14:52 CDT 2008


"baldrick"

> I have some intensive number crunching code that I want to break up
> into threads so PCs with several processors can do the job quicker.
>
It is exactly as you write here, it "can" do the job quicker with threading,
but in the same case it can do the job "slower".

Threading uses managment, so be aware that you are not only because of
marketing information are doing things that has a reverse result than you
think. Threading can be greath as you are doing several jobs which have wait
times like retrieving data from internet.

Have a look simple at the TaskManager and see how many threads are already
running and how they perform on your 2 processors. As it is a function for
your own organisation, and you are allowed to set threadpriority for all
threads to the highest, then maybe you can get some quicker througput time.
Your total processor time will however always be more.

As you persist on using threads in this case, have then a look at the que
class as with that and locking the enque en deque operations you can in a
simple way create a smooth operation.

Cor

Armin
Sun May 11 04:48:49 CDT 2008

"baldrick" <philbrierley@hotmail.com> schrieb

David,
I must have not made myself very clear. How to stop the loop was not
the question. I can do that quite easily.

I want to loop through 1 to 10, allocate 1 to the first thread, 2 to
the second, then 3 to either thread 1 or 2, depending on which one has
finished, or wait if neither have finished, same for 4 etc...

I could easily allocate 1-5 to thread 1 and 6-10 to thread 2 but I
want to do it a bit more in order.

The reason for threading is that each i of the loop will take about 10
minutes so I want to do 2 at once so it will run on 2 processors an be
done in half the time.

Would appreciate any pointers or code snippets.

=======

Maybe I'm wrong (early in the morning), though..:
1. Enqueue _all_ work items into a Queue object. (symbol search "queue")
2. Start two (or more) threads (one per processor (core))
3. Each thread has access to the queue object. Each thread runs a loop
that pulls one work item from the queue (use Synclock on the queue
object while doing this).
4. That's all
5. Really

However, I don't know if additional work items will have to be processed
later, that means after they some been enqueued in step 1.

I forgot: In order to cancel a thread, just set a boolean flag that is
accessible to the threads and signals them to return.


AZ

Armin
Sun May 11 04:53:40 CDT 2008

"Armin Zingler" <az.nospam@freenet.de> schrieb
> However, I don't know if additional work items will have to be
> processed later, that means after they some been enqueued in step 1.

correction:
...after they have been enqueued...
...after some have been...

Choose one.


AZ

Jay
Sun May 11 17:11:56 CDT 2008

baldrick,
In addition to the other comments, you may want to review PLINQ.

http://blogs.msdn.com/pfxteam/archive/tags/PLINQ/default.aspx

PLINQ will take care of spreading your computations across multiple threads,
more importantly across multiple cores for you.

--
Hope this helps
Jay B. Harlow [MVP - Outlook]
.NET Application Architect, Enthusiast, & Evangelist
T.S. Bradley - http://www.tsbradley.net


"baldrick" <philbrierley@hotmail.com> wrote in message
news:e5778b71-5470-464a-80e9-ce08973d51c8@s50g2000hsb.googlegroups.com...
> Hi,
> I have some intensive number crunching code that I want to break up
> into threads so PCs with several processors can do the job quicker.
>
> I have a loop that will go from say 1 to 10, and for each value,
> different numbers will be crunched depending on what the value is.
>
> As it is an intensive process the user may cancel the loop. I need to
> know at what value of the loop the process was cancelled, so it can be
> restarted at the next value next time.
>
> My initial thought was to allocate 1-5 to thread 1 and 6-10 to thread
> 2 (for a 2 processor machine) - which shouldn't be too difficult. But
> the issue with this if that if the user cancels then its not so neat
> to resume as 1 and 6 might be the only ones completed.
>
> Ideally I would like to do the following but am not quite sure how.
> Any pointers or code snippets would be appreciated.
>
> for 1 = 1 to 10 step numProcessors (say 2)
>
>
>
> next i

Cor
Sun May 11 22:36:41 CDT 2008

I forgot to give you this link.

http://msdn.microsoft.com/en-us/library/system.threading.waitcallback(VS.85).aspx

Cor

"Cor Ligthert[MVP]" <notmyfirstname@planet.nl> schreef in bericht
news:D6737E0E-A120-41DC-945E-B78C2F2175CC@microsoft.com...
>
> "baldrick"
>
>> I have some intensive number crunching code that I want to break up
>> into threads so PCs with several processors can do the job quicker.
>>
> It is exactly as you write here, it "can" do the job quicker with
> threading, but in the same case it can do the job "slower".
>
> Threading uses managment, so be aware that you are not only because of
> marketing information are doing things that has a reverse result than you
> think. Threading can be greath as you are doing several jobs which have
> wait times like retrieving data from internet.
>
> Have a look simple at the TaskManager and see how many threads are already
> running and how they perform on your 2 processors. As it is a function for
> your own organisation, and you are allowed to set threadpriority for all
> threads to the highest, then maybe you can get some quicker througput
> time. Your total processor time will however always be more.
>
> As you persist on using threads in this case, have then a look at the que
> class as with that and locking the enque en deque operations you can in a
> simple way create a smooth operation.
>
> Cor