How to shared virtual space between driver and application?

Many posts have talked about this topic, but I still have some questions.

A normal solution to share buffer between driver and application is,

1. application allocates a buffer,
2. pass the buffer pointer to driver by an IOCTL.
3. driver calls ProbeForWrite to probe the buffer, creates an MDL for that
buffer and calls MmProbeAndLockPages to lock pages, then calls
MmGetSystemAddressForMdlSafe to get a system-space virtual address.

After calling MmProbeAndLockPages() and MmGetSystemAddressForMdlSafe(),
driver gets a nonpaged buffer pointer and buffer occupies physical memory.
That is not what I wanted. In my case, a driver thread runs always at
PASSIVE_LEVEL and it needs to manipulate many massive buffers allocated by
an application. If they are all commited to nonpaged buffer, memory resource
may runs out. Can I get a page-able system-space virtual adress which memory
is shared with application?

Maxim
Tue Nov 18 21:20:48 CST 2003

> That is not what I wanted. In my case, a driver thread runs always at
> PASSIVE_LEVEL and it needs to manipulate many massive buffers allocated by
> an application.

Why not send them down to the driver as IRPs?

--
Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
maxim@storagecraft.com
http://www.storagecraft.com

Valentin
Wed Nov 19 03:29:25 CST 2003

I think because buffers are large. He asks how to handle them so that driver
does not make a copy. There are two ways driver can deal with buffer, one is
for large buffers. Described in DDK, does not know more.

Kirk
Wed Nov 19 03:45:02 CST 2003

"Brilliance Qian" <brilliance.qian%AT%nojunkmail.dactron.spx.com> wrote in
message news:uQbUTUjrDHA.2304@TK2MSFTNGP12.phx.gbl...
> A normal solution to share buffer between driver and application is,
>
> 1. application allocates a buffer,
> 2. pass the buffer pointer to driver by an IOCTL.
> 3. driver calls ProbeForWrite to probe the buffer, creates an MDL for that
> buffer and calls MmProbeAndLockPages to lock pages, then calls
> MmGetSystemAddressForMdlSafe to get a system-space virtual address.

Normal solution would be to send a buffered IO IOCTL every time you need to
talk to the device. Your approach is tough to get right and has certain
security ramifications. What requirements do you have that force you to do
that?

ProbeForWrite is a waste (it just probes the first byte of the buffer) you
can use ProbeForRead instead. In addition to the above you also have to have
_try/_except block and ensure that whole thing always gets executed at
PASSIVE_LEVEL. If one day you get filter driver on top that happens to use
StartIo your code will bugcheck.

Also note that you are essentially creating a double mapping. Every time you
remove this mapping you will flush the TLB.

-Kirk

Kirk
Wed Nov 19 03:53:33 CST 2003

"Valentin Tihomirov" <valentin@abelectron.com> wrote in message
news:bpfd60$1o71uc$1@ID-212430.news.uni-berlin.de...
> I think because buffers are large. He asks how to handle them so that
driver
> does not make a copy. There are two ways driver can deal with buffer, one
is
> for large buffers. Described in DDK, does not know more.

The alternative to a copy (buffered IO) is to create a double mapping
(direct IO). But when you remove the mapping you will flush the TLB. This is
not something that you want to do very often.

-Kirk

Maxim
Wed Nov 19 11:37:49 CST 2003

> I think because buffers are large. He asks how to handle them so that driver
> does not make a copy.

Sending them to the DO_DIRECT_IO driver by WriteFile or by METHOD_OUT_DIRECT
IOCTL is fine, it will not do the extra copy.

The multimedia stack does this by IOCTL_KS_WRITE_STREAM.

So, the way is good, and by far better then shared memory due to security and
synchronization issues.

As about map/unmap - the system PTE code in the OS is rather smart, it collects
a large list of "dirty" PTEs (freed but not invalidated and thus unsuitable for
immediate reuse) and then flushes the whole TLB when this list becomes too
huge.

Also - if you need the shared space for DMA, then no need in map/unmap.

--
Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
maxim@storagecraft.com
http://www.storagecraft.com

Brilliance
Wed Nov 19 13:21:17 CST 2003

Let me explain more on my case.

In my driver, a thread is created when a device is added and runs at
PASSIVE_LEVLE definitely. After the device is activated to run by
application, the thread sucks data from a USB bulk pipe then dispatchs data
to massive buffer array by an ID in received data. The massive buffer array
is allocated by application and the pointers are passed to driver by an
IOCTL.

1. Both buffer size and buffer number could be large depending on user
requests. So I want to avoid allocating double buffer. It is better to share
buffers between application and driver, and use page-able memory.

2. If my understanding is correct, it can't use buffered IO IOCTL because
buffer pointer is freed after IOCTL is completed. But in the case, buffer
pointers are not used in the IOCTL lifetime, they are used in another thread
in the driver.

3. If sending buffer pointers to the DO_DIRECT_IO driver by WriteFile or by
METHOD_OUT_DIRECT
IOCTL, are the buffers still page-able memory? Can the thread use the
pointers directly, or it just gets some pointers of MDL and it has to lock
MDL pages then call MmGetSystemAddressForMdlSafe () to get buffer pointer
thus buffer memory is committed to physical memory.


"Brilliance Qian" <brilliance.qian%AT%nojunkmail.dactron.spx.com> wrote in
message news:uQbUTUjrDHA.2304@TK2MSFTNGP12.phx.gbl...
> Many posts have talked about this topic, but I still have some questions.
>
> A normal solution to share buffer between driver and application is,
>
> 1. application allocates a buffer,
> 2. pass the buffer pointer to driver by an IOCTL.
> 3. driver calls ProbeForWrite to probe the buffer, creates an MDL for that
> buffer and calls MmProbeAndLockPages to lock pages, then calls
> MmGetSystemAddressForMdlSafe to get a system-space virtual address.
>
> After calling MmProbeAndLockPages() and MmGetSystemAddressForMdlSafe(),
> driver gets a nonpaged buffer pointer and buffer occupies physical memory.
> That is not what I wanted. In my case, a driver thread runs always at
> PASSIVE_LEVEL and it needs to manipulate many massive buffers allocated by
> an application. If they are all commited to nonpaged buffer, memory
resource
> may runs out. Can I get a page-able system-space virtual adress which
memory
> is shared with application?
>
>