Async Sockets and IOCPS

Hang my head in shame; I only found out yesterday that .NET sockets are
already doing overlapped I/O with IOCPs when you make async calls. The
only problem I see (experimentally via debugging, so I don't know if
it's true), is that each socket gets it's own IOCP. Is there any way
to attach a socket to a specific IOCP, the way you do in Win32?

Thanks,

-- kov

Chris
Sat Aug 26 12:48:45 CDT 2006

<ken.overton@gmail.com> wrote
> Is there any way
> to attach a socket to a specific IOCP, the way you do in Win32?

I've never seen a way to do this without writing a large amount of your own
code, and I spend quite a bit of time using Async Sockets. There are a
number of .Net IOCP Wrapper classes sitting around out there - the most
complete being found in Jeff Richter's Power Threading library that you can
pull down off the Wintellect site.

The .Net sockets, as they sit now, scale to incredible numbers on even low
end hardware. On high end 64 bit hardware the scalability is absolutly
outrageous. What are you trying to accomplish by writing a custom IOCP
Sockets integration?

I wrote a detailed blog entry on socket sever scalability and architectural
options with .Net recently:
http://www.coversant.net/dotnetnuke/Coversant/Blogs/tabid/88/EntryID/10/Default.aspx

This blog entry mostly examines the different architectures available to
Sockets applications, and the pros and cons of each that we've run into.

--
Chris Mullins, MCSD.Net, MCPD:Enterprise
http://www.coversant.net/blogs/cmullins

ken
Sat Aug 26 15:04:54 CDT 2006

Chris Mullins wrote:
> The .Net sockets, as they sit now, scale to incredible numbers on even low
> end hardware. On high end 64 bit hardware the scalability is absolutly
> outrageous. What are you trying to accomplish by writing a custom IOCP
> Sockets integration?

Well, of course I'm trying to *avoid* writing custom IOCP sockets
integration :-)

The clients I typically have to write connect to lots and lots of
different sockets. Some of those are very active, some are not so
active. It just seems ludicrous to dedicate an entire IOCP thread for
each of these. It seems pretty logical to find a way to group a bunch
of less-active sockets together on one IOCP, a high-volume with it's
own, etc etc.

I'm investigating how easy it will be to do that.

Thanks for the input,

-- kov

Chris
Sat Aug 26 15:26:39 CDT 2006

<ken.overton@gmail.com> wrote
> Chris Mullins wrote:
> What are you trying to accomplish by writing a custom
> IOCP Sockets integration?
>
> The clients I typically have to write connect to lots and lots of
> different sockets. Some of those are very active, some are not so
> active. It just seems ludicrous to dedicate an entire IOCP thread for
> each of these. It seems pretty logical to find a way to group a bunch
> of less-active sockets together on one IOCP, a high-volume with it's
> own, etc etc.

I would just use the out-of-the-box technology and not worry about it.

We've had to build some crazy client-side sockets applications in order to
drive load against the SoapBox Server. We actually built a full XMPP based
botnet (bots, controllers, etc) that can run on N number of machines,
connect to a control channel server, listen for work items and then execute
those work items. Driving load on a server than can handle 500k simultanious
users has been..... challenging.

We can (and have) had tens of thousands of these little bots running on lots
and lots of machines.

32-bit Windows Sockets scales, client side, to about 30k sockets before
things really start to taper off. This assumes you have enough memory, and
have set the magic registry switches.

--
Chris Mullins, MCSD.NET, MCPD:Enterprise
http://www.coversant.net/blogs/cmullins

ken
Sat Aug 26 17:34:11 CDT 2006

Chris Mullins wrote:
>
> 32-bit Windows Sockets scales, client side, to about 30k sockets before
> things really start to taper off. This assumes you have enough memory, and
> have set the magic registry switches.

Wow. And that's all using Async sockets? I.e., at least a thread per
socket? The reason I'm looking into this is precisely because the last
job I worked on the clients had performance problems due to excessive
context-switches. This was a large institution so their engineering
department had built a library that allowed multiple sockets' I/O to be
scheduled on one thread, and using this technology the thread count
dropped from 40 down to about 8 and improved performance in a huge way.
But now I'm in a different place without those kind of engineering
resources so I'm debating whether I'll have to build my own version of
it.

I suppose part of it that needs to be said is that the destination
boxes are not allocated just for my process(es), they're already doing
many other things.

--kov

Chris
Sat Aug 26 17:47:42 CDT 2006

<ken.overton@gmail.com> Wrote:

> And that's all using Async sockets? I.e., at least a thread per
> socket?

That blog entry I linked earlier goes into quite a bit of detail on this
topic. It's probably worth reading, if you're into building scalable socket
applciations.

Async sockets (and by extension, IOCP) is in no way the same as 1 thread per
socket. Quite the opposite actually. IOCP is very smart about the number of
threads that are awake, the processor they're scheduled on, and what they're
doing.

I would recommend a 1 thread per socket approach to a novice dealing with a
few dozen sockets. There's no real scalability there at all, although for
some people it's an easier programming model than the async stuff.

> But now I'm in a different place without those kind of engineering
> resources so I'm debating whether I'll have to build my own version of
> it.

If you try to build this yourself, you'll be wasing alot of time. The stuff
out of the box is... excellent.


> I suppose part of it that needs to be said is that the destination
> boxes are not allocated just for my process(es), they're already doing
> many other things.

That's just fine. In fact this pushes even more towards using out-of-the-box
async sockets. This stuff works, and it works really well.

--
Chris Mullins

ken
Sun Aug 27 06:22:14 CDT 2006

Chris Mullins wrote:
> <ken.overton@gmail.com> Wrote:
>
> > And that's all using Async sockets? I.e., at least a thread per
> > socket?
>
> That blog entry I linked earlier goes into quite a bit of detail on this
> topic. It's probably worth reading, if you're into building scalable socket
> applciations.

Thanks, I just read it again now that I know IOCPs are in .NET -- the
first time I thought everytime you said "IOCP" it was some custom thing
you were adding -- and I get it now.

> > But now I'm in a different place without those kind of engineering
> > resources so I'm debating whether I'll have to build my own version of
> > it.
>
> If you try to build this yourself, you'll be wasing alot of time. The stuff
> out of the box is... excellent.

Thanks for the advice, I think I will save my time for other problems.

Cheers,

-- kov