mapping device memory to userspace through MDL direct io interface

I am trying to map the device memory(128MB) to userspace through direct IO
interface using MDL's.
I have choosen below approach for mapping the memory to user space:
IoAllocateMdl for allocating buffer for MDL(with virtual address of the
device memory, from MmMapIoSpace)
MmBuildMdlForNonPagedPool
MmMapLockedPagesSpecifyCache

By doing in above method, i have observed that, i could map the memory to
less limit onlyi.e <0x20000, but if i tried to use the full size(128MB),
system
is giving Blue screen with error as KERNEL_IN_PAGE_ERROR.

I am building the driver using WDF tool kit & trying in Visat-32 bit OS.
Kindly suggest me, is this is a right approach to map the memory to user
space or any other method can be used to map the memory to user space.

Attached the BUGCHECK windbg output:
KERNEL_DATA_INPAGE_ERROR (7a)
The requested page of kernel data could not be read in. Typically caused by
a bad block in the paging file or disk controller error. Also see
KERNEL_STACK_INPAGE_ERROR.
If the error status is 0xC000000E, 0xC000009C, 0xC000009D or 0xC0000185,
it means the disk subsystem has experienced a failure.
If the error status is 0xC000009A, then it means the request failed because
a filesystem failed to make forward progress.
Arguments:
Arg1: 00000001, lock type that was held (value 1,2,3, or PTE address)
Arg2: d0000006, error status (normally i/o status code)
Arg3: 83b674e8, current process (virtual address for lock type 3, or PTE)
Arg4: c000e000, virtual address that could not be in-paged (or PTE contents
if arg1 is a PTE address)

Debugging Details:
------------------


ERROR_CODE: (NTSTATUS) 0xd0000006 - The instruction at "0x%08lx" referenced
memory at "0x%08lx". The required data was not placed into memory because of
an I/O error status of "0x%08lx".

BUGCHECK_STR: 0x7a_d0000006

DEFAULT_BUCKET_ID: VISTA_RC

PROCESS_NAME: rfm2g_util.exe

CURRENT_IRQL: 2

LAST_CONTROL_TRANSFER: from 818d873f to 81881760

STACK_TEXT:
a95df4d4 818d873f 00000003 a95d4e64 00000000
nt!RtlpBreakWithStatusInstruction
a95df524 818d91ac 00000003 c0603000 83d57580 nt!KiBugCheckDebugBreak+0x1c
a95df8d0 818263d2 0000007a 00000001 d0000006 nt!KeBugCheck2+0x5f4
a95df914 818bb1da c000e000 83b674e8 00000000
nt!MiMakeSystemAddressValid+0x19e
a95df938 81815137 83b674e8 00000000 83d57580
nt!MiMakePdeExistAndMakeValid+0x68
a95df980 819d3606 01c00000 83a6a09c 00000000
nt!MiMapLockedPagesInUserSpaceHelper+0xb8
a95df9c4 818ba69b 83a6a000 00000001 01be0000
nt!MiMapLockedPagesInUserSpace+0x38c
a95dfa24 8ad5e1a4 83a6a000 00000001 00000001
nt!MmMapLockedPagesSpecifyCache+0x29
a95dfa74 8ad5d35a 839cfa70 7c4589d0 839cfc58 pmc55xx!CreateMDL+0x104
[d:\test\rfm_xp32fromxp64\legacy\pmc5565_wdf_driver\plx9x5x\sys\init.c @
1131]
a95dfb70 804e2ba4 7c2d31e8 7c400270 0000000c
pmc55xx!PLxDrvEvtIoDeviceControl+0x1aa
[d:\test\rfm_xp32fromxp64\legacy\pmc5565_wdf_driver\plx9x5x\sys\pci9656.c @
920]
a95dfb94 804e3f7c 7c2d31e8 7c400270 0000000c
Wdf01000!FxIoQueueIoDeviceControl::Invoke+0x30
a95dfbc4 804e6598 7c400270 83bffd88 83d2ce10
Wdf01000!FxIoQueue::DispatchRequestToDriver+0x445
a95dfbe0 804e7d2c 83d2ce00 8050d188 83d2ce10
Wdf01000!FxIoQueue::DispatchEvents+0x485
a95dfbfc 804e8e67 00000000 83bcfe88 83d85020
Wdf01000!FxIoQueue::QueueRequest+0x237
a95dfc20 804d7d9a 8377e560 a95dfc44 81827ecf Wdf01000!FxPkgIo::Dispatch+0x377
a95dfc2c 81827ecf 83d85020 8377e560 8377e560 Wdf01000!FxDevice::Dispatch+0x7f
a95dfc44 81988f65 83bcfe88 8377e560 8377e618 nt!IofCallDriver+0x63
a95dfc64 81989f25 83d85020 83bcfe88 019c1200
nt!IopSynchronousServiceTail+0x1e0
a95dfd00 8198ee8d 83d85020 8377e560 00000000 nt!IopXxxControlFile+0x6b7
a95dfd34 8188c96a 00000098 00000000 00000000 nt!NtDeviceIoControlFile+0x2a
a95dfd34 77680f34 00000098 00000000 00000000 nt!KiFastCallEntry+0x12a
0012fd2c 7767f850 75f27c92 00000098 00000000 ntdll!KiFastSystemCallRet
0012fd30 75f27c92 00000098 00000000 00000000 ntdll!ZwDeviceIoControlFile+0xc
0012fd90 10001b15 00000098 00122000 00000000 kernel32!DeviceIoControl+0x14a
WARNING: Stack unwind information not available. Following frames may be
wrong.
0012fdf8 75f28574 00412558 00000000 00425900 rfm2gdll_stdc+0x1b15
0012feb0 00409241 0012ff00 01bd0144 01030f78 kernel32!ReadFile+0x294
00000000 00000000 00000000 00000000 00000000 rfm2g_util+0x9241


STACK_COMMAND: kb

FOLLOWUP_IP:
pmc55xx!CreateMDL+104
[d:\test\rfm_xp32fromxp64\legacy\pmc5565_wdf_driver\plx9x5x\sys\init.c @
1131]
8ad5e1a4 8b55e4 mov edx,dword ptr [ebp-1Ch]

FAULTING_SOURCE_CODE:
1127: UserMode,
1128: MmCached,
1129: NULL,
1130: FALSE,
> 1131: NormalPagePriority)) == NULL)
1132: {
1133: /* Uh-oh */
1134:
1135: Status = STATUS_INSUFFICIENT_RESOURCES;
1136: break;

Gary
Wed Mar 26 11:16:54 CDT 2008

What is it you are trying to do? Typically, mapping memory between user and
kernel is a poor design decision, with better ways available; filter driver,
inverted callbacks, etc.

--
The personal opinion of
Gary G. Little

"kota" <kota@discussions.microsoft.com> wrote in message
news:364BE160-E919-43D6-8868-A733EDD29077@microsoft.com...
>I am trying to map the device memory(128MB) to userspace through direct IO
> interface using MDL's.
> I have choosen below approach for mapping the memory to user space:
> IoAllocateMdl for allocating buffer for MDL(with virtual address of the
> device memory, from MmMapIoSpace)
> MmBuildMdlForNonPagedPool
> MmMapLockedPagesSpecifyCache
>
> By doing in above method, i have observed that, i could map the memory to
> less limit onlyi.e <0x20000, but if i tried to use the full size(128MB),
> system
> is giving Blue screen with error as KERNEL_IN_PAGE_ERROR.
>
> I am building the driver using WDF tool kit & trying in Visat-32 bit OS.
> Kindly suggest me, is this is a right approach to map the memory to user
> space or any other method can be used to map the memory to user space.
>
> Attached the BUGCHECK windbg output:
> KERNEL_DATA_INPAGE_ERROR (7a)
> The requested page of kernel data could not be read in. Typically caused
> by
> a bad block in the paging file or disk controller error. Also see
> KERNEL_STACK_INPAGE_ERROR.
> If the error status is 0xC000000E, 0xC000009C, 0xC000009D or 0xC0000185,
> it means the disk subsystem has experienced a failure.
> If the error status is 0xC000009A, then it means the request failed
> because
> a filesystem failed to make forward progress.
> Arguments:
> Arg1: 00000001, lock type that was held (value 1,2,3, or PTE address)
> Arg2: d0000006, error status (normally i/o status code)
> Arg3: 83b674e8, current process (virtual address for lock type 3, or PTE)
> Arg4: c000e000, virtual address that could not be in-paged (or PTE
> contents
> if arg1 is a PTE address)
>
> Debugging Details:
> ------------------
>
>
> ERROR_CODE: (NTSTATUS) 0xd0000006 - The instruction at "0x%08lx"
> referenced
> memory at "0x%08lx". The required data was not placed into memory because
> of
> an I/O error status of "0x%08lx".
>
> BUGCHECK_STR: 0x7a_d0000006
>
> DEFAULT_BUCKET_ID: VISTA_RC
>
> PROCESS_NAME: rfm2g_util.exe
>
> CURRENT_IRQL: 2
>
> LAST_CONTROL_TRANSFER: from 818d873f to 81881760
>
> STACK_TEXT:
> a95df4d4 818d873f 00000003 a95d4e64 00000000
> nt!RtlpBreakWithStatusInstruction
> a95df524 818d91ac 00000003 c0603000 83d57580 nt!KiBugCheckDebugBreak+0x1c
> a95df8d0 818263d2 0000007a 00000001 d0000006 nt!KeBugCheck2+0x5f4
> a95df914 818bb1da c000e000 83b674e8 00000000
> nt!MiMakeSystemAddressValid+0x19e
> a95df938 81815137 83b674e8 00000000 83d57580
> nt!MiMakePdeExistAndMakeValid+0x68
> a95df980 819d3606 01c00000 83a6a09c 00000000
> nt!MiMapLockedPagesInUserSpaceHelper+0xb8
> a95df9c4 818ba69b 83a6a000 00000001 01be0000
> nt!MiMapLockedPagesInUserSpace+0x38c
> a95dfa24 8ad5e1a4 83a6a000 00000001 00000001
> nt!MmMapLockedPagesSpecifyCache+0x29
> a95dfa74 8ad5d35a 839cfa70 7c4589d0 839cfc58 pmc55xx!CreateMDL+0x104
> [d:\test\rfm_xp32fromxp64\legacy\pmc5565_wdf_driver\plx9x5x\sys\init.c @
> 1131]
> a95dfb70 804e2ba4 7c2d31e8 7c400270 0000000c
> pmc55xx!PLxDrvEvtIoDeviceControl+0x1aa
> [d:\test\rfm_xp32fromxp64\legacy\pmc5565_wdf_driver\plx9x5x\sys\pci9656.c
> @
> 920]
> a95dfb94 804e3f7c 7c2d31e8 7c400270 0000000c
> Wdf01000!FxIoQueueIoDeviceControl::Invoke+0x30
> a95dfbc4 804e6598 7c400270 83bffd88 83d2ce10
> Wdf01000!FxIoQueue::DispatchRequestToDriver+0x445
> a95dfbe0 804e7d2c 83d2ce00 8050d188 83d2ce10
> Wdf01000!FxIoQueue::DispatchEvents+0x485
> a95dfbfc 804e8e67 00000000 83bcfe88 83d85020
> Wdf01000!FxIoQueue::QueueRequest+0x237
> a95dfc20 804d7d9a 8377e560 a95dfc44 81827ecf
> Wdf01000!FxPkgIo::Dispatch+0x377
> a95dfc2c 81827ecf 83d85020 8377e560 8377e560
> Wdf01000!FxDevice::Dispatch+0x7f
> a95dfc44 81988f65 83bcfe88 8377e560 8377e618 nt!IofCallDriver+0x63
> a95dfc64 81989f25 83d85020 83bcfe88 019c1200
> nt!IopSynchronousServiceTail+0x1e0
> a95dfd00 8198ee8d 83d85020 8377e560 00000000 nt!IopXxxControlFile+0x6b7
> a95dfd34 8188c96a 00000098 00000000 00000000 nt!NtDeviceIoControlFile+0x2a
> a95dfd34 77680f34 00000098 00000000 00000000 nt!KiFastCallEntry+0x12a
> 0012fd2c 7767f850 75f27c92 00000098 00000000 ntdll!KiFastSystemCallRet
> 0012fd30 75f27c92 00000098 00000000 00000000
> ntdll!ZwDeviceIoControlFile+0xc
> 0012fd90 10001b15 00000098 00122000 00000000
> kernel32!DeviceIoControl+0x14a
> WARNING: Stack unwind information not available. Following frames may be
> wrong.
> 0012fdf8 75f28574 00412558 00000000 00425900 rfm2gdll_stdc+0x1b15
> 0012feb0 00409241 0012ff00 01bd0144 01030f78 kernel32!ReadFile+0x294
> 00000000 00000000 00000000 00000000 00000000 rfm2g_util+0x9241
>
>
> STACK_COMMAND: kb
>
> FOLLOWUP_IP:
> pmc55xx!CreateMDL+104
> [d:\test\rfm_xp32fromxp64\legacy\pmc5565_wdf_driver\plx9x5x\sys\init.c @
> 1131]
> 8ad5e1a4 8b55e4 mov edx,dword ptr [ebp-1Ch]
>
> FAULTING_SOURCE_CODE:
> 1127: UserMode,
> 1128: MmCached,
> 1129: NULL,
> 1130: FALSE,
>> 1131: NormalPagePriority)) == NULL)
> 1132: {
> 1133: /* Uh-oh */
> 1134:
> 1135: Status = STATUS_INSUFFICIENT_RESOURCES;
> 1136: break;
>

kota
Thu Mar 27 09:30:01 CDT 2008

Hi Gary,

Thanks for very quick & short reply, could you be little more brief about
your suggestions (or) plz provide any link about your suggesting for mapping
kernel device memory to user space.

I am currently following the method suggested in below paper:
http://www.microsoft.com/whdc/driver/kernel/mem-mgmt.mspx
But instead of getting the virtual kernel address space, i am taking user
virtual address space from below function:
MmMapLockedPagesSpecifyCache
After looking to the below article, i modified the code:
http://www.osronline.com/article.cfm?id=423
Instead of using MmBuildNonPagedPool, i used MmMapLockedPages, with this
method, i could map the memory till 1MB, where as it was working only 100KB
with earlier method.
if i increase the memory limit more than this, i am getting the bluse screen
with same info.

>

Don
Thu Mar 27 09:33:06 CDT 2008

And as Gary asked why do you think you need this? What problem are you
trying to solve, that can justify the security problems you are creating
with this approach? What is the characteristics of your device that need
the level of performance that can only be justified by this?


--
Don Burn (MVP, Windows DDK)
Windows 2k/XP/2k3 Filesystem and Driver Consulting
Website: http://www.windrvr.com
Blog: http://msmvps.com/blogs/WinDrvr
Remove StopSpam to reply


"kota" <kota@discussions.microsoft.com> wrote in message
news:04A19F1C-ECB3-411C-94BF-33C83B0FE85C@microsoft.com...
> Hi Gary,
>
> Thanks for very quick & short reply, could you be little more brief about
> your suggestions (or) plz provide any link about your suggesting for
> mapping
> kernel device memory to user space.
>
> I am currently following the method suggested in below paper:
> http://www.microsoft.com/whdc/driver/kernel/mem-mgmt.mspx
> But instead of getting the virtual kernel address space, i am taking user
> virtual address space from below function:
> MmMapLockedPagesSpecifyCache
> After looking to the below article, i modified the code:
> http://www.osronline.com/article.cfm?id=423
> Instead of using MmBuildNonPagedPool, i used MmMapLockedPages, with this
> method, i could map the memory till 1MB, where as it was working only
> 100KB
> with earlier method.
> if i increase the memory limit more than this, i am getting the bluse
> screen
> with same info.
>
>>

kota
Fri Mar 28 00:11:00 CDT 2008

Hi Don,

My PCI device has variants of memory size from 64MB to 256MB, this device
will be connected to other devices through Fibre optic cable for data
transfer. Data transfer rate is 2Gb/sec.
Applications uses this user mapped memory to transfer the data to
device, there will be continuos data transfer takes place from different
applications. Earlier driver release(NT model) was also used same methods to
ensure that, data transfer will takes place in short time(in view of
applications) with old NT model functions and the driver is working fine.
Instead of applications sending the data through IOCTL/any other method and
it should wait, till that completes for next transfer.

Hope i have answered your queries regarding my device & applications.
Your suggestons are always welcome to improve much better.

Thanks,
Prafulla

Maxim
Fri Mar 28 08:16:37 CDT 2008

> My PCI device has variants of memory size from 64MB to 256MB, this device
> will be connected to other devices through Fibre optic cable for data
> transfer. Data transfer rate is 2Gb/sec.

Not so good a design for such data rates.

CPU-side accesses to the device memory are usually much slower then DMA (DMA is
a nearly always a burst, CPU-side access is usually not a burst), so, another
design is better, with no large memory on the device and with chain DMA running
over a buffer list in the main RAM.

--
Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
maxim@storagecraft.com
http://www.storagecraft.com

kota
Fri Mar 28 08:49:03 CDT 2008

Shatskih,

I forgot to mention, we are using the DMA for buffer>64bytes.
If the memory is less than 64bytes, we are using the user mapped memory
access directly.

Thanks,
Prafulla

"Maxim S. Shatskih" wrote:

> > My PCI device has variants of memory size from 64MB to 256MB, this device
> > will be connected to other devices through Fibre optic cable for data
> > transfer. Data transfer rate is 2Gb/sec.
>
> Not so good a design for such data rates.
>
> CPU-side accesses to the device memory are usually much slower then DMA (DMA is
> a nearly always a burst, CPU-side access is usually not a burst), so, another
> design is better, with no large memory on the device and with chain DMA running
> over a buffer list in the main RAM.
>
> --
> Maxim Shatskih, Windows DDK MVP
> StorageCraft Corporation
> maxim@storagecraft.com
> http://www.storagecraft.com
>
>

Gary
Fri Mar 28 12:56:45 CDT 2008

And just how do you intend to sync access to that memory between those
applications and most importantly the kernel? Even Remote DMA over fibre
channel is best done without mapping memory between user and kernel. It's a
bad design that begs to be updated. Was the old driver ever given heavy use
or testing on multi-processing?

--
The personal opinion of
Gary G. Little

"kota" <kota@discussions.microsoft.com> wrote in message
news:E50C63BB-039C-4D7C-8B69-DDDDFDA4115A@microsoft.com...
> Hi Don,
>
> My PCI device has variants of memory size from 64MB to 256MB, this device
> will be connected to other devices through Fibre optic cable for data
> transfer. Data transfer rate is 2Gb/sec.
> Applications uses this user mapped memory to transfer the data
> to
> device, there will be continuos data transfer takes place from different
> applications. Earlier driver release(NT model) was also used same methods
> to
> ensure that, data transfer will takes place in short time(in view of
> applications) with old NT model functions and the driver is working fine.
> Instead of applications sending the data through IOCTL/any other method
> and
> it should wait, till that completes for next transfer.
>
> Hope i have answered your queries regarding my device & applications.
> Your suggestons are always welcome to improve much better.
>
> Thanks,
> Prafulla
>
>
>