cancel-safe queues and Cleanup

Assume a driver working with the same logic as the DDK Cancel sample
implements. Suppose that one routine just removed a pending irp from the
queue and then the Cleanup dispatch is called. Because the irp is not in the
queue anymore, Cleanup won't find it and the irp won't be cancelled.

In most cases this won't be a problem, the irp will be completed soon
enough. But in some devices it is possible that the irp can't be completed
for some reason. Then the irp would keep pending, possibly inserted again in
the cancel-safe queue. In extreme cases the irp might be never completed, and
it won't ever be cancelled either.

Am I missing something, or do I need to take special care if irps removed
from the queue might be not possible to complete?

Tim
Sat Mar 10 18:12:00 CST 2007

ijor <ijor@nospam.nospam> wrote:
>
>Assume a driver working with the same logic as the DDK Cancel sample
>implements. Suppose that one routine just removed a pending irp from the
>queue and then the Cleanup dispatch is called. Because the irp is not in the
>queue anymore, Cleanup won't find it and the irp won't be cancelled.
>
>In most cases this won't be a problem, the irp will be completed soon
>enough. But in some devices it is possible that the irp can't be completed
>for some reason. Then the irp would keep pending, possibly inserted again in
>the cancel-safe queue. In extreme cases the irp might be never completed, and
>it won't ever be cancelled either.
>
>Am I missing something, or do I need to take special care if irps removed
>from the queue might be not possible to complete?

Once you remove an IRP from the queue, the responsibility for handling
cancellation falls squarely upon YOUR shoulders. Any time there's a
potential delay in processing, you should check for cancellation yourself.
--
Tim Roberts, timr@probo.com
Providenza & Boekelheide, Inc.

ijor
Sat Mar 10 21:53:29 CST 2007

"Tim Roberts" wrote:

> ijor <ijor@nospam.nospam> wrote:
> >
> >Assume a driver working with the same logic as the DDK Cancel sample
> >implements. Suppose that one routine just removed a pending irp from the
> >queue and then the Cleanup dispatch is called. Because the irp is not in the
> >queue anymore, Cleanup won't find it and the irp won't be cancelled.
> >
> >In most cases this won't be a problem, the irp will be completed soon
> >enough. But in some devices it is possible that the irp can't be completed
> >for some reason. Then the irp would keep pending, possibly inserted again in
> >the cancel-safe queue. In extreme cases the irp might be never completed, and
> >it won't ever be cancelled either.
> >
> >Am I missing something, or do I need to take special care if irps removed
> >from the queue might be not possible to complete?
>
> Once you remove an IRP from the queue, the responsibility for handling
> cancellation falls squarely upon YOUR shoulders. Any time there's a
> potential delay in processing, you should check for cancellation yourself.

I realize that I need to check for cancellation requests, that wouldn't be
too difficult to handle. Delivering the irp back to the cancel-safe queue
would be enough. But the problem here is not about checking for cancel
requests, the problem here is that nobody ever would send a cancel request
for that irp.

Actually, I just realized that there is an additional race condition between
cleanup and queuing. If a new irp comes about the same time than a cleanup
request, and if for some reason cleanup finishes before the new irp is
queued, then again that irp would never be cancelled.

I guess the solution involves marking the file object as "cleaned up"
somehow in the Fscontext fields. Then the routine deferring or requeing irps
could check this field.

Btw, can somebody explain to me why the system requires the driver to cancel
irps during cleanup. Doesn't the system mantain a list of all pending irps
belonging to a file object? Then why the system can't mark all those irps for
cancellation by itself?

Doron
Sun Mar 11 00:02:44 CST 2007

the system does cancel all IRPs pending on the file object during file
handle close. there is still a race though between one thread sending i/o
and another thread closing the handle that you can account for in the
application. you cancel i/o on cleanup so the application can exit and is
not a zombie. the app will stick around until all pended i/o has completed.

d

--
Please do not send e-mail directly to this alias. this alias is for
newsgroup purposes only.
This posting is provided "AS IS" with no warranties, and confers no rights.


"ijor" <ijor@nospam.nospam> wrote in message
news:5EBE0BAE-2C69-4B2F-B1B0-EB74DEBEB25F@microsoft.com...
> "Tim Roberts" wrote:
>
>> ijor <ijor@nospam.nospam> wrote:
>> >
>> >Assume a driver working with the same logic as the DDK Cancel sample
>> >implements. Suppose that one routine just removed a pending irp from the
>> >queue and then the Cleanup dispatch is called. Because the irp is not in
>> >the
>> >queue anymore, Cleanup won't find it and the irp won't be cancelled.
>> >
>> >In most cases this won't be a problem, the irp will be completed soon
>> >enough. But in some devices it is possible that the irp can't be
>> >completed
>> >for some reason. Then the irp would keep pending, possibly inserted
>> >again in
>> >the cancel-safe queue. In extreme cases the irp might be never
>> >completed, and
>> >it won't ever be cancelled either.
>> >
>> >Am I missing something, or do I need to take special care if irps
>> >removed
>> >from the queue might be not possible to complete?
>>
>> Once you remove an IRP from the queue, the responsibility for handling
>> cancellation falls squarely upon YOUR shoulders. Any time there's a
>> potential delay in processing, you should check for cancellation
>> yourself.
>
> I realize that I need to check for cancellation requests, that wouldn't be
> too difficult to handle. Delivering the irp back to the cancel-safe queue
> would be enough. But the problem here is not about checking for cancel
> requests, the problem here is that nobody ever would send a cancel request
> for that irp.
>
> Actually, I just realized that there is an additional race condition
> between
> cleanup and queuing. If a new irp comes about the same time than a cleanup
> request, and if for some reason cleanup finishes before the new irp is
> queued, then again that irp would never be cancelled.
>
> I guess the solution involves marking the file object as "cleaned up"
> somehow in the Fscontext fields. Then the routine deferring or requeing
> irps
> could check this field.
>
> Btw, can somebody explain to me why the system requires the driver to
> cancel
> irps during cleanup. Doesn't the system mantain a list of all pending irps
> belonging to a file object? Then why the system can't mark all those irps
> for
> cancellation by itself?
>

Alexander
Sun Mar 11 20:44:11 CDT 2007

To avoid such a race condition it can be good idea to duplicate a hadnle
before handing it to another thread. The thread then will be responsible for
closing it.

"Doron Holan [MS]" <doronh@nospam.microsoft.com> wrote in message
news:uL7JkL6YHHA.4396@TK2MSFTNGP06.phx.gbl...
> the system does cancel all IRPs pending on the file object during file
> handle close. there is still a race though between one thread sending i/o
> and another thread closing the handle that you can account for in the
> application. you cancel i/o on cleanup so the application can exit and is
> not a zombie. the app will stick around until all pended i/o has
> completed.
>
> d
>
> --
> Please do not send e-mail directly to this alias. this alias is for
> newsgroup purposes only.
> This posting is provided "AS IS" with no warranties, and confers no
> rights.
>

Doron
Mon Mar 12 00:30:54 CDT 2007

you won't get the cleanup until the last duplicated handle is closed, so you
could still have a race if this is not done correctly

d

--
Please do not send e-mail directly to this alias. this alias is for
newsgroup purposes only.
This posting is provided "AS IS" with no warranties, and confers no rights.


"Alexander Grigoriev" <alegr@earthlink.net> wrote in message
news:eh2TxfEZHHA.4552@TK2MSFTNGP05.phx.gbl...
> To avoid such a race condition it can be good idea to duplicate a hadnle
> before handing it to another thread. The thread then will be responsible
> for closing it.
>
> "Doron Holan [MS]" <doronh@nospam.microsoft.com> wrote in message
> news:uL7JkL6YHHA.4396@TK2MSFTNGP06.phx.gbl...
>> the system does cancel all IRPs pending on the file object during file
>> handle close. there is still a race though between one thread sending
>> i/o and another thread closing the handle that you can account for in the
>> application. you cancel i/o on cleanup so the application can exit and
>> is not a zombie. the app will stick around until all pended i/o has
>> completed.
>>
>> d
>>
>> --
>> Please do not send e-mail directly to this alias. this alias is for
>> newsgroup purposes only.
>> This posting is provided "AS IS" with no warranties, and confers no
>> rights.
>>
>
>

ijor
Mon Mar 12 15:26:42 CDT 2007

"Doron Holan [MS]" wrote:

> the system does cancel all IRPs pending on the file object during file
> handle close.

Hmm, this is in contradiction to the DDK/WDK, a couple of KB articles, and
everything I read about that so far. I thought that the system does NOT
cancel pending IRPs on close, and that's why cleanup is supposed to cancel
them.

> there is still a race though between one thread sending i/o
> and another thread closing the handle ...

I'm not sure I understand. Do you mean the whole idea is that *both* the
system and the driver's cleanup should cancel pending IRPs, and this
redundancy is just for the purpose of avoiding potential races?

Alexander
Mon Mar 12 23:11:22 CDT 2007

The point is that each thread then owns its own handle to issue IRP and can
close it without need to synchronize with other threads. Then the last
handle won't get closed (and CLEANUP issued) while another thread is issuing
IRPs, so there's no possibility of IOCTL/READ/WRITE arriving after CLEANUP
because of a race condition.

"Doron Holan [MS]" <doronh@nospam.microsoft.com> wrote in message
news:uxtRceGZHHA.984@TK2MSFTNGP04.phx.gbl...
> you won't get the cleanup until the last duplicated handle is closed, so
> you could still have a race if this is not done correctly
>
> d
>
> --
> Please do not send e-mail directly to this alias. this alias is for
> newsgroup purposes only.
> This posting is provided "AS IS" with no warranties, and confers no
> rights.
>
>
> "Alexander Grigoriev" <alegr@earthlink.net> wrote in message
> news:eh2TxfEZHHA.4552@TK2MSFTNGP05.phx.gbl...
>> To avoid such a race condition it can be good idea to duplicate a hadnle
>> before handing it to another thread. The thread then will be responsible
>> for closing it.
>>
>> "Doron Holan [MS]" <doronh@nospam.microsoft.com> wrote in message
>> news:uL7JkL6YHHA.4396@TK2MSFTNGP06.phx.gbl...
>>> the system does cancel all IRPs pending on the file object during file
>>> handle close. there is still a race though between one thread sending
>>> i/o and another thread closing the handle that you can account for in
>>> the application. you cancel i/o on cleanup so the application can exit
>>> and is not a zombie. the app will stick around until all pended i/o has
>>> completed.
>>>
>>> d
>>>
>>> --
>>> Please do not send e-mail directly to this alias. this alias is for
>>> newsgroup purposes only.
>>> This posting is provided "AS IS" with no warranties, and confers no
>>> rights.
>>>
>>
>>
>
>

Doron
Tue Mar 13 00:02:30 CDT 2007

i understand what you are saying. my point is that you have to still be
careful about how you manage the thread/file handle pair.

d

--
Please do not send e-mail directly to this alias. this alias is for
newsgroup purposes only.
This posting is provided "AS IS" with no warranties, and confers no rights.


"Alexander Grigoriev" <alegr@earthlink.net> wrote in message
news:OMtIsWSZHHA.4772@TK2MSFTNGP05.phx.gbl...
> The point is that each thread then owns its own handle to issue IRP and
> can close it without need to synchronize with other threads. Then the last
> handle won't get closed (and CLEANUP issued) while another thread is
> issuing IRPs, so there's no possibility of IOCTL/READ/WRITE arriving after
> CLEANUP because of a race condition.
>
> "Doron Holan [MS]" <doronh@nospam.microsoft.com> wrote in message
> news:uxtRceGZHHA.984@TK2MSFTNGP04.phx.gbl...
>> you won't get the cleanup until the last duplicated handle is closed, so
>> you could still have a race if this is not done correctly
>>
>> d
>>
>> --
>> Please do not send e-mail directly to this alias. this alias is for
>> newsgroup purposes only.
>> This posting is provided "AS IS" with no warranties, and confers no
>> rights.
>>
>>
>> "Alexander Grigoriev" <alegr@earthlink.net> wrote in message
>> news:eh2TxfEZHHA.4552@TK2MSFTNGP05.phx.gbl...
>>> To avoid such a race condition it can be good idea to duplicate a hadnle
>>> before handing it to another thread. The thread then will be responsible
>>> for closing it.
>>>
>>> "Doron Holan [MS]" <doronh@nospam.microsoft.com> wrote in message
>>> news:uL7JkL6YHHA.4396@TK2MSFTNGP06.phx.gbl...
>>>> the system does cancel all IRPs pending on the file object during file
>>>> handle close. there is still a race though between one thread sending
>>>> i/o and another thread closing the handle that you can account for in
>>>> the application. you cancel i/o on cleanup so the application can exit
>>>> and is not a zombie. the app will stick around until all pended i/o
>>>> has completed.
>>>>
>>>> d
>>>>
>>>> --
>>>> Please do not send e-mail directly to this alias. this alias is for
>>>> newsgroup purposes only.
>>>> This posting is provided "AS IS" with no warranties, and confers no
>>>> rights.
>>>>
>>>
>>>
>>
>>
>
>

Doron
Tue Mar 13 00:02:54 CDT 2007

please point to the KB article which says the OS does not cancel i/o on
close

d

--
Please do not send e-mail directly to this alias. this alias is for
newsgroup purposes only.
This posting is provided "AS IS" with no warranties, and confers no rights.


"ijor" <ijor@nospam.nospam> wrote in message
news:BE94D287-203A-4915-9316-F50A134E1344@microsoft.com...
> "Doron Holan [MS]" wrote:
>
>> the system does cancel all IRPs pending on the file object during file
>> handle close.
>
> Hmm, this is in contradiction to the DDK/WDK, a couple of KB articles, and
> everything I read about that so far. I thought that the system does NOT
> cancel pending IRPs on close, and that's why cleanup is supposed to cancel
> them.
>
>> there is still a race though between one thread sending i/o
>> and another thread closing the handle ...
>
> I'm not sure I understand. Do you mean the whole idea is that *both* the
> system and the driver's cleanup should cancel pending IRPs, and this
> redundancy is just for the purpose of avoiding potential races?
>

ijor
Tue Mar 13 00:39:16 CDT 2007

"Doron Holan [MS]" wrote:

> please point to the KB article which says the OS does not cancel i/o on
> close

http://support.microsoft.com/kb/120170

"Conversely, IRP_MJ_CLEANUP can be issued with no cancel routines being
called. As explained earlier, this happens when the last handle to a file
object is closed. In this case, even if there are pending IRPs associated
with the file object, the system does not try to cancel them. The driver
handles the IRPs appropriately in the IRP_MJ_CLEANUP routine."

Eliyas
Tue Mar 13 01:42:30 CDT 2007

What the article says is correct until Vista. On Vista, if you close a file
handle that's associated with a completion port for asynchronous I/O
operation, I/O manager will cancel IRPs. This change is made as part of new
feature called "thread agnostic I/O" to improve performance of completion
ports.

Prior to Vista OS when an application makes an I/O request (for synchronous,
asynchronous, or asynchronous handle associated with completion ports), the
IRP created by the I/O manager is queued to the thread. On Vista, if the
handle is opened for overlapped (asynchronous) I/O and associated with
completion ports, then the I/O manager queues the IRP to the fileobject.
When such an handle is closed, I/O manager will cancel all the IRPs
associated with the fileobject.

-Eliyas