ndisprot, ndisuio, packet - Help plz?

Hi,

For the last 5-6 months I have tried the "ndisprot like" examples from the 3
versions of the DDKs (packet, ndisuio, ndisprot) and practically I have
learned a few things through trial and error.
I have been working on an application that needs to move from layer 3 of the
Network stack to as close as possible to the layer 2. I thought to take
(initially) ndisuio example and use it.
The problem is that I am actually trying to create a network bridge between
two NIC interfaces but to do so I need a good throughput which basically
comes down to asynchronous operation.
I am familiar with the implications of that (dublicate frames, forwarded
frame unless broadcast needs to have the dest MAC of the neighbour device
etc.). I have solutions for these. The problem is the bridging! I am really
desperate for a solution...

I 've figured out that I can't open 2 handles per interface (or am i wrong
here?) so that leaves me with overlapped operation.

Now i tried that but it all seems to fail.

Does any of this DDK examples really supports overlapped operation? Either
on opening a device or on the createfile? And when using overlapped operation
is it better (faster) to use createfile with readfile/writefile and
getoverlappedresult or a callback function?
And okay, we are not supposed to use the same overlapped structure but
assuming that we have 100 frames per second am i supposed to initialise 100
frame structs containing 100 different overlapped structs? Isn't there
another way??

Is there anyone out there who has achieved proper overlapped operation using
any of these three DDK examples?
Can that someone kindly provide me with some advice? Please?

ps: At some point I have tried 4 different versions of the same driver (2
per interface, one for reading one for writing) in an attempt tto overcome
the "only one handle per interface" rule, but the throughput of the bridge
has been pathetic (8-10Kb/s on a FTP transfer on a 10/100Mbps bridging). Any
idea why that can be? Are multiple ndisuio drivers not supposed to be loaded
together?

Kind regards,
dimitris
de4rag@hotmail.com

Stephan
Fri Sep 09 09:21:06 CDT 2005

First of all, *why* are you trying to implement a network bridge?
Windows already implements this. See e.g.

"How the WindowsXP Network Bridge Works"
http://www.microsoft.com/technet/columns/cableguy/cg0102.asp

"Windows XP Bridging and Media Support for Home Networking"

http://www.eu.microsoft.com/technet/prodtechnol/winxppro/Plan/XPBrdgNt.asp

Second, why are you trying to implement the bridge in *user* mode? That
does not sound like a high performance solution.

Stephan
---
De4rag wrote:
> Hi,
>
> For the last 5-6 months I have tried the "ndisprot like" examples from the 3
> versions of the DDKs (packet, ndisuio, ndisprot) and practically I have
> learned a few things through trial and error.
> I have been working on an application that needs to move from layer 3 of the
> Network stack to as close as possible to the layer 2. I thought to take
> (initially) ndisuio example and use it.
> The problem is that I am actually trying to create a network bridge between
> two NIC interfaces but to do so I need a good throughput which basically
> comes down to asynchronous operation.
> I am familiar with the implications of that (dublicate frames, forwarded
> frame unless broadcast needs to have the dest MAC of the neighbour device
> etc.). I have solutions for these. The problem is the bridging! I am really
> desperate for a solution...
>
> I 've figured out that I can't open 2 handles per interface (or am i wrong
> here?) so that leaves me with overlapped operation.
>
> Now i tried that but it all seems to fail.
>
> Does any of this DDK examples really supports overlapped operation? Either
> on opening a device or on the createfile? And when using overlapped operation
> is it better (faster) to use createfile with readfile/writefile and
> getoverlappedresult or a callback function?
> And okay, we are not supposed to use the same overlapped structure but
> assuming that we have 100 frames per second am i supposed to initialise 100
> frame structs containing 100 different overlapped structs? Isn't there
> another way??
>
> Is there anyone out there who has achieved proper overlapped operation using
> any of these three DDK examples?
> Can that someone kindly provide me with some advice? Please?
>
> ps: At some point I have tried 4 different versions of the same driver (2
> per interface, one for reading one for writing) in an attempt tto overcome
> the "only one handle per interface" rule, but the throughput of the bridge
> has been pathetic (8-10Kb/s on a FTP transfer on a 10/100Mbps bridging). Any
> idea why that can be? Are multiple ndisuio drivers not supposed to be loaded
> together?
>
> Kind regards,
> dimitris
> de4rag@hotmail.com

Thomas
Fri Sep 09 10:23:40 CDT 2005

The main point is that NDISUIO and NDISPROT are just sample drivers. They
illustrate very basic NDIS techniques and NOT high-performance packet
collection techniques. The design goal of NDISUIO was probably to provide a
verys simple interface to NDIS protocol drivers to support managing 802.11
adapters and very simple packet send/receive needed to support some types of
authentication.

These samples do have restrictions how many handles can be opened, etc.

Basically as-is they simply are not suitable for the use that you are trying
to put them to. They are not flawed with respect to performance, they simply
were not designed to offer the performance you are looking for.

You CAN'T use either of these sample drivers for your intended use. You must
develop your own driver that fits your specific needs. To do this you will
need to study non-NDIS driver samples that illustrate high-performance
asynchronous I/O . You have to stop a moment and decide what the
architecture should be to fit your needs - then develop a driver of your own
that fits your needs.

You can, of course, study things like WinPCap. If you continue along this
route you will be attempting to develop features like those of Rawether.NET,
so you might as well study that product's capabilities as well.

You need to understand that implementing a bridge in user-mode will be
slower that doing it in the kernel no matter how well you refine your
asynchronous I/O.

You should also take a deep breath, sit back and analyze if you have the
correct approach at all.

As Stephan said, Windows XP already has some bridging capability. How does
what you are trying to do differ from what comes in the box?

Are other alternatives better than what you are attempting? I, for one,
would consider implementing a bridge in an NDIS Intermediate (IM) driver
instead of a NDIS protocol driver if such a beast was necessary.

Good luck,

Thomas F. Divine


"De4rag" <De4rag@discussions.microsoft.com> wrote in message
news:A2B83F79-44C0-4158-A379-B8D9E94A88A1@microsoft.com...
> Hi,
>
> For the last 5-6 months I have tried the "ndisprot like" examples from the
> 3
> versions of the DDKs (packet, ndisuio, ndisprot) and practically I have
> learned a few things through trial and error.
> I have been working on an application that needs to move from layer 3 of
> the
> Network stack to as close as possible to the layer 2. I thought to take
> (initially) ndisuio example and use it.
> The problem is that I am actually trying to create a network bridge
> between
> two NIC interfaces but to do so I need a good throughput which basically
> comes down to asynchronous operation.
> I am familiar with the implications of that (dublicate frames, forwarded
> frame unless broadcast needs to have the dest MAC of the neighbour device
> etc.). I have solutions for these. The problem is the bridging! I am
> really
> desperate for a solution...
>
> I 've figured out that I can't open 2 handles per interface (or am i wrong
> here?) so that leaves me with overlapped operation.
>
> Now i tried that but it all seems to fail.
>
> Does any of this DDK examples really supports overlapped operation? Either
> on opening a device or on the createfile? And when using overlapped
> operation
> is it better (faster) to use createfile with readfile/writefile and
> getoverlappedresult or a callback function?
> And okay, we are not supposed to use the same overlapped structure but
> assuming that we have 100 frames per second am i supposed to initialise
> 100
> frame structs containing 100 different overlapped structs? Isn't there
> another way??
>
> Is there anyone out there who has achieved proper overlapped operation
> using
> any of these three DDK examples?
> Can that someone kindly provide me with some advice? Please?
>
> ps: At some point I have tried 4 different versions of the same driver (2
> per interface, one for reading one for writing) in an attempt tto overcome
> the "only one handle per interface" rule, but the throughput of the bridge
> has been pathetic (8-10Kb/s on a FTP transfer on a 10/100Mbps bridging).
> Any
> idea why that can be? Are multiple ndisuio drivers not supposed to be
> loaded
> together?
>
> Kind regards,
> dimitris
> de4rag@hotmail.com
>
>

De4rag
Fri Sep 09 16:32:02 CDT 2005


I appreciate both of your replies. For someone like me who works alone on an
area which I have to admit is slightly beyond my league your comments are
more than appreciated/welcome.

Your questions are justified. To make things clearer: I do have a pretty
good idea on Networking (been teaching Networking courses at the University
as part of my PhD obligations), I am not however a Windows driver developer.
I might be good with C/C++ but the first time I looked at the code of the DDK
samples I canâ??t say that I could figure out how it really worked.

With regards to why I am attempting this, at the University of Greenwich
(UK) where I am doing my PhD we are (actually its oneâ??s man army â?? just me)
working for the last two and a half years now, on Quality of Service for
TCP/IP networks. Having read about implementations on Linux systems, but none
for Windows OS (and as I am more into Windows than Linux), I have decided to
go ahead and implement something for Windows that could be used for a
test-bed that its primary target would be to evaluate the performance of
individual DiffServ components and/or component sets. I know that Windows
have support for QoS but I wanted to create something slightly different
(more flexible, more portable).

So I have developed that test-bed using Winsock. Apart from not be able to
select how the switching worked (traffic arrives at one interface but how to
control where is to be sent from Winsock?) there was the hope of increased
performance.

I donâ??t deny it that I hoped (especially now that fast machines are cheaper)
that I could reach a point where I could take an ordinary PC with 2 NICs,
install Windows, run my program and have a basic DiffServ router. Moving
closer to layer 2 of the network stack by eliminating Winsock layer
processing sounded as a good idea.

I wanted to be something Windows based but also something lightweight, not
too complicated as I am not into driver development, and something that I
could modify (if needed). So the DDK examples sounded like the obvious answer.

You might call it the wrong choice and maybe it is but I thought: I will
take ndisuio, add multi-thread support (2 threads per NIC) and then use my
program on top of it.
Thatâ??s it basically. Although my aspiration has been to be able to reach at
a point the performance of a Cisco router, this is a test-bed after all. Yes
the performance is important but if I could get it close to 50-60% of a 100
Mbps Kernel based bridging that would still be adequate.

I have read most of the ndis related driver articles although not many when
googled/yahooed (including your Pass Through modification Thomas â?? very nice
work there) but as I am working on this alone its difficult to re-write a
driver from scratch.

So, any suggestions? Any books that you can recommend? Any ideas before i go
blind spending all these hours looking at driver examples that probably are
not good for what I am looking for?

Kind regards
Dimitris
fd26@gre.ac.uk

ps: With regards to the DDK UK Microsoft support I can say that it could get
better. When I have asked them about any DDK based newsgroups they didn't
have a clue.
Nevermind.


"Thomas F. Divine [DDK MVP]" wrote:

> The main point is that NDISUIO and NDISPROT are just sample drivers. They
> illustrate very basic NDIS techniques and NOT high-performance packet
> collection techniques. The design goal of NDISUIO was probably to provide a
> verys simple interface to NDIS protocol drivers to support managing 802.11
> adapters and very simple packet send/receive needed to support some types of
> authentication.
>
> These samples do have restrictions how many handles can be opened, etc.
>
> Basically as-is they simply are not suitable for the use that you are trying
> to put them to. They are not flawed with respect to performance, they simply
> were not designed to offer the performance you are looking for.
>
> You CAN'T use either of these sample drivers for your intended use. You must
> develop your own driver that fits your specific needs. To do this you will
> need to study non-NDIS driver samples that illustrate high-performance
> asynchronous I/O . You have to stop a moment and decide what the
> architecture should be to fit your needs - then develop a driver of your own
> that fits your needs.
>
> You can, of course, study things like WinPCap. If you continue along this
> route you will be attempting to develop features like those of Rawether.NET,
> so you might as well study that product's capabilities as well.
>
> You need to understand that implementing a bridge in user-mode will be
> slower that doing it in the kernel no matter how well you refine your
> asynchronous I/O.
>
> You should also take a deep breath, sit back and analyze if you have the
> correct approach at all.
>
> As Stephan said, Windows XP already has some bridging capability. How does
> what you are trying to do differ from what comes in the box?
>
> Are other alternatives better than what you are attempting? I, for one,
> would consider implementing a bridge in an NDIS Intermediate (IM) driver
> instead of a NDIS protocol driver if such a beast was necessary.
>
> Good luck,
>
> Thomas F. Divine
>
>
> "De4rag" <De4rag@discussions.microsoft.com> wrote in message
> news:A2B83F79-44C0-4158-A379-B8D9E94A88A1@microsoft.com...
> > Hi,
> >
> > For the last 5-6 months I have tried the "ndisprot like" examples from the
> > 3
> > versions of the DDKs (packet, ndisuio, ndisprot) and practically I have
> > learned a few things through trial and error.
> > I have been working on an application that needs to move from layer 3 of
> > the
> > Network stack to as close as possible to the layer 2. I thought to take
> > (initially) ndisuio example and use it.
> > The problem is that I am actually trying to create a network bridge
> > between
> > two NIC interfaces but to do so I need a good throughput which basically
> > comes down to asynchronous operation.
> > I am familiar with the implications of that (dublicate frames, forwarded
> > frame unless broadcast needs to have the dest MAC of the neighbour device
> > etc.). I have solutions for these. The problem is the bridging! I am
> > really
> > desperate for a solution...
> >
> > I 've figured out that I can't open 2 handles per interface (or am i wrong
> > here?) so that leaves me with overlapped operation.
> >
> > Now i tried that but it all seems to fail.
> >
> > Does any of this DDK examples really supports overlapped operation? Either
> > on opening a device or on the createfile? And when using overlapped
> > operation
> > is it better (faster) to use createfile with readfile/writefile and
> > getoverlappedresult or a callback function?
> > And okay, we are not supposed to use the same overlapped structure but
> > assuming that we have 100 frames per second am i supposed to initialise
> > 100
> > frame structs containing 100 different overlapped structs? Isn't there
> > another way??
> >
> > Is there anyone out there who has achieved proper overlapped operation
> > using
> > any of these three DDK examples?
> > Can that someone kindly provide me with some advice? Please?
> >
> > ps: At some point I have tried 4 different versions of the same driver (2
> > per interface, one for reading one for writing) in an attempt tto overcome
> > the "only one handle per interface" rule, but the throughput of the bridge
> > has been pathetic (8-10Kb/s on a FTP transfer on a 10/100Mbps bridging).
> > Any
> > idea why that can be? Are multiple ndisuio drivers not supposed to be
> > loaded
> > together?
> >
> > Kind regards,
> > dimitris
> > de4rag@hotmail.com
> >
> >
>
>

Stephan
Sat Sep 10 11:55:32 CDT 2005

Just a real quick and short (and incomplete) answer...

De4rag wrote:
[..]
> So, any suggestions? Any books that you can recommend? Any ideas before i go
> blind spending all these hours looking at driver examples that probably are
> not good for what I am looking for?

No books on NDIS whatsoever, sorry. All you can get is the DDK
documentation, this newsgroup, some info on WHDC
(http://www.microsoft.com/whdc/device/network/ndis/) and Thomas' web
site (http://www.ndis.com).

Also do a search in this forum for "bridge" and "ndis"
(http://groups.google.com/group/microsoft.public.development.device.drivers/search?q=bridge+ndis).

Stephan