Miniport's IOCTL interface performance issue.

Hi all,
I have written a NDIS 5.0 virtual miniport driver that send and receive
packets from a usermode application via Direct IOCTL interface. All
seems to be working well, except that the miniport drops packets if it
receives more that about 3000/second. I have implemented a receive
buffer in the miniport and it is clear that the miniport receive all
the packets from the above protocols and queue them accordingly but the
IOCTL interface seems to be to slow to handle all of the packets, so
the queue overflows. Is there any way to improve the IOCTL
interface's performance?

Thanks,
Amos

Stephan
Wed Oct 12 06:31:53 CDT 2005

In general, people tend to more and more request performance
improvements from "the system" rather than revising the design of their
software.

I guess the answer is no, you cannot expect faster IOCTLs.

A similar problem arose years ago when the number of packets per second
on the wire grew by a factor of ten (Gigabit Ethernet vs. Fast
Ethernet). Since we just cannot burden the system with one interrupt
per packet, what we did was introduce a mechanism known as "interrupt
moderation", which simply bundles several packets to be handled in just
one interrupt.

So you could pass more than one packet in a single IOCTL, both send and
receive.

If a packet is received, do not immediately complete an outstanding IRP
for it. Instead, start a timer. If either more than a certain number of
packets has been received *or* the timer expires, whichever comes
first, complete the IRP.

On the send side (application), similarly gather several send packets
before passing them down to the driver in a single IRP.

HTH, Stephan
---
Amos wrote:
> Hi all,
> I have written a NDIS 5.0 virtual miniport driver that send and receive
> packets from a usermode application via Direct IOCTL interface. All
> seems to be working well, except that the miniport drops packets if it
> receives more that about 3000/second. I have implemented a receive
> buffer in the miniport and it is clear that the miniport receive all
> the packets from the above protocols and queue them accordingly but the
> IOCTL interface seems to be to slow to handle all of the packets, so
> the queue overflows. Is there any way to improve the IOCTL
> interface's performance?
>
> Thanks,
> Amos

Calvin
Wed Oct 12 13:44:14 CDT 2005

Sounds like "software interrupt coalescing" to me.

Yep, batching TX/Rx can help throughput. I see in NDIStest environment, if
upper layer was sending a lot of net buffers per net buffer list, my driver
can saturate the Gbe link with less than 10% CPU.

OP, note that aggressive interrupt coalescing can hurt latency. I guess this
is the #1 trick in net driver design and tuning. It is the area where
innovation goes.

good luck,
Calvin
--
Calvin Guan (Windows DDK MVP)
NetXtreme Longhorn Miniport Prime
Broadcom Corp. www.broadcom.com

"Stephan Wolf [MVP]" <stewo68@hotmail.com> wrote in message
news:1129116713.752256.76060@g43g2000cwa.googlegroups.com...
> In general, people tend to more and more request performance
> improvements from "the system" rather than revising the design of their
> software.
>
> I guess the answer is no, you cannot expect faster IOCTLs.
>
> A similar problem arose years ago when the number of packets per second
> on the wire grew by a factor of ten (Gigabit Ethernet vs. Fast
> Ethernet). Since we just cannot burden the system with one interrupt
> per packet, what we did was introduce a mechanism known as "interrupt
> moderation", which simply bundles several packets to be handled in just
> one interrupt.
>
> So you could pass more than one packet in a single IOCTL, both send and
> receive.
>
> If a packet is received, do not immediately complete an outstanding IRP
> for it. Instead, start a timer. If either more than a certain number of
> packets has been received *or* the timer expires, whichever comes
> first, complete the IRP.
>
> On the send side (application), similarly gather several send packets
> before passing them down to the driver in a single IRP.
>
> HTH, Stephan
> ---
> Amos wrote:
>> Hi all,
>> I have written a NDIS 5.0 virtual miniport driver that send and receive
>> packets from a usermode application via Direct IOCTL interface. All
>> seems to be working well, except that the miniport drops packets if it
>> receives more that about 3000/second. I have implemented a receive
>> buffer in the miniport and it is clear that the miniport receive all
>> the packets from the above protocols and queue them accordingly but the
>> IOCTL interface seems to be to slow to handle all of the packets, so
>> the queue overflows. Is there any way to improve the IOCTL
>> interface's performance?
>>
>> Thanks,
>> Amos
>