DPC/ISR synchronization (windows ce)

I haven't had a lot of response in the platform builder newsgroups, so I'll
try here.

In Windows CE 4.1, I had no problem with a NDIS ISR (miniportISR)
interrupting the DPC(MiniportHandleInterrupt). In upgrading to Windows CE
5.0, I'm seeing that the ISR is getting held off by the DPC. I've tried
serialized/deserialized. I've verified that the interrupt is not disabled in
the kernel and can see with independent means that kernel is seeing the
interrupt and setting the interrupt event in time with the hardware, but the
ISR waits until the DPC finishes until it is run.

Does anyone have visibility into what has changed between the CE 4.x and 5.x
NDIS implementation ? The driver registers as an NDIS 5.0 miniport.

GaryGLittle
Wed Jun 29 09:06:04 CDT 2005

Does the DPC raise IRQL, and if so what does it raise it to? The DPC is
nromally at DISPATCH_LEVEL in XP, and given the common use of terms, I would
expect that to be the same. If the DPC raises IRQL to a level higher than the
DIRQL causing the intterupt, the yeah ... the ISR will wait till IRQL drops.

The personal opinion of
Gary G. Little

"anonymous" wrote:

> I haven't had a lot of response in the platform builder newsgroups, so I'll
> try here.
>
> In Windows CE 4.1, I had no problem with a NDIS ISR (miniportISR)
> interrupting the DPC(MiniportHandleInterrupt). In upgrading to Windows CE
> 5.0, I'm seeing that the ISR is getting held off by the DPC. I've tried
> serialized/deserialized. I've verified that the interrupt is not disabled in
> the kernel and can see with independent means that kernel is seeing the
> interrupt and setting the interrupt event in time with the hardware, but the
> ISR waits until the DPC finishes until it is run.
>
> Does anyone have visibility into what has changed between the CE 4.x and 5.x
> NDIS implementation ? The driver registers as an NDIS 5.0 miniport.

anonymous
Wed Jun 29 10:48:02 CDT 2005

Thanks,

That is the issue. Previously, the ISR was two priority levels above the
DPC. In 5.0, they are both at the same level.

So, the next question is why were they changed ?

"Gary G. Little" wrote:

> Does the DPC raise IRQL, and if so what does it raise it to? The DPC is
> nromally at DISPATCH_LEVEL in XP, and given the common use of terms, I would
> expect that to be the same. If the DPC raises IRQL to a level higher than the
> DIRQL causing the intterupt, the yeah ... the ISR will wait till IRQL drops.
>
> The personal opinion of
> Gary G. Little
>
> "anonymous" wrote:
>
> > I haven't had a lot of response in the platform builder newsgroups, so I'll
> > try here.
> >
> > In Windows CE 4.1, I had no problem with a NDIS ISR (miniportISR)
> > interrupting the DPC(MiniportHandleInterrupt). In upgrading to Windows CE
> > 5.0, I'm seeing that the ISR is getting held off by the DPC. I've tried
> > serialized/deserialized. I've verified that the interrupt is not disabled in
> > the kernel and can see with independent means that kernel is seeing the
> > interrupt and setting the interrupt event in time with the hardware, but the
> > ISR waits until the DPC finishes until it is run.
> >
> > Does anyone have visibility into what has changed between the CE 4.x and 5.x
> > NDIS implementation ? The driver registers as an NDIS 5.0 miniport.

anonymous
Wed Jun 29 10:54:07 CDT 2005

Well, it appears they are actually running in the same thread now.

"anonymous" wrote:

> Thanks,
>
> That is the issue. Previously, the ISR was two priority levels above the
> DPC. In 5.0, they are both at the same level.
>
> So, the next question is why were they changed ?
>
> "Gary G. Little" wrote:
>
> > Does the DPC raise IRQL, and if so what does it raise it to? The DPC is
> > nromally at DISPATCH_LEVEL in XP, and given the common use of terms, I would
> > expect that to be the same. If the DPC raises IRQL to a level higher than the
> > DIRQL causing the intterupt, the yeah ... the ISR will wait till IRQL drops.
> >
> > The personal opinion of
> > Gary G. Little
> >
> > "anonymous" wrote:
> >
> > > I haven't had a lot of response in the platform builder newsgroups, so I'll
> > > try here.
> > >
> > > In Windows CE 4.1, I had no problem with a NDIS ISR (miniportISR)
> > > interrupting the DPC(MiniportHandleInterrupt). In upgrading to Windows CE
> > > 5.0, I'm seeing that the ISR is getting held off by the DPC. I've tried
> > > serialized/deserialized. I've verified that the interrupt is not disabled in
> > > the kernel and can see with independent means that kernel is seeing the
> > > interrupt and setting the interrupt event in time with the hardware, but the
> > > ISR waits until the DPC finishes until it is run.
> > >
> > > Does anyone have visibility into what has changed between the CE 4.x and 5.x
> > > NDIS implementation ? The driver registers as an NDIS 5.0 miniport.

pavel_a
Wed Jun 29 16:12:04 CDT 2005

"anonymous" wrote:
> I haven't had a lot of response in the platform builder newsgroups, so I'll
> try here.
>
> In Windows CE 4.1, I had no problem with a NDIS ISR (miniportISR)
> interrupting the DPC(MiniportHandleInterrupt). In upgrading to Windows CE
> 5.0, I'm seeing that the ISR is getting held off by the DPC.

Hi,
First of all, could you please explain why your ISR occurs while the DPC
is scheduled?
When interrupt occurs and the ISR queues a DPC, it should normally prevent
the device from generating more interrupts.
Then the DPC runs and re-enables the interrupts just before returning.
Why this design isn't good for you?

Regards,
--PA

>I've tried
> serialized/deserialized. I've verified that the interrupt is not disabled in
> the kernel and can see with independent means that kernel is seeing the
> interrupt and setting the interrupt event in time with the hardware, but the
> ISR waits until the DPC finishes until it is run.
>
> Does anyone have visibility into what has changed between the CE 4.x and 5.x
> NDIS implementation ? The driver registers as an NDIS 5.0 miniport.

anonymous
Wed Jun 29 18:10:12 CDT 2005

My problem is that my NIC has four buffers for receive and transmit. Assuming
I'm full duplex and one is going out, then I've room for three packets.

Three packets at 100Mbit, is approximately 1024(header)+64byte*8 = 1536 bit
times + interpacket delay of 960ns = 16.32us * 3 = 48.96us

By the time fourth packet posts at 65.28us, I'm dropping packets.

A call into NdisMIndicateReceivePacket is taking much longer than 65us even
with just one packet. If I have to wait around for that indication to return
before I can service the hardware, I'm just dropping too many packets too
frequently.

Thanks!

GaryGLittle
Thu Jun 30 09:31:07 CDT 2005

Not necessarily. I have dealt with drivers that could not diasble interrupts
and then let the DPC re-enable them. Done properly, there is no reason why
the device's interrupt should be disabled in the ISR and enabled in the DPC.
You do have to sync access to resources that are shared between the two.

The personal opinion of
Gary G. Little

"Pavel A." wrote:

> "anonymous" wrote:
> > I haven't had a lot of response in the platform builder newsgroups, so I'll
> > try here.
> >
> > In Windows CE 4.1, I had no problem with a NDIS ISR (miniportISR)
> > interrupting the DPC(MiniportHandleInterrupt). In upgrading to Windows CE
> > 5.0, I'm seeing that the ISR is getting held off by the DPC.
>
> Hi,
> First of all, could you please explain why your ISR occurs while the DPC
> is scheduled?
> When interrupt occurs and the ISR queues a DPC, it should normally prevent
> the device from generating more interrupts.
> Then the DPC runs and re-enables the interrupts just before returning.
> Why this design isn't good for you?
>
> Regards,
> --PA
>
> >I've tried
> > serialized/deserialized. I've verified that the interrupt is not disabled in
> > the kernel and can see with independent means that kernel is seeing the
> > interrupt and setting the interrupt event in time with the hardware, but the
> > ISR waits until the DPC finishes until it is run.
> >
> > Does anyone have visibility into what has changed between the CE 4.x and 5.x
> > NDIS implementation ? The driver registers as an NDIS 5.0 miniport.

pavel_a
Thu Jun 30 11:55:03 CDT 2005

"Gary G. Little" wrote:
> Not necessarily. I have dealt with drivers that could not diasble interrupts
> and then let the DPC re-enable them. Done properly, there is no reason why
> the device's interrupt should be disabled in the ISR and enabled in the DPC.
> You do have to sync access to resources that are shared between the two.

You're correct, but as you see, this design is not quite portable to WinCE.
How exactly hardware interrupts behave on CE depends on the
manufacturer of the CE device.
I'm not sure about CE 5, but in previous CE versions
installable drivers could not receive hardware interrupts at all, and thus
did not have real ISRs. What they get is callbacks in context of a high
priority user mode threads. DPCs are emulated with usermode
threads with lesser priority.
So the different behavior in CE5 could be caused by mistake of
somebody who configured it for the specific platform
(assigned wrong priorities to interrupt and DPC threads, etc).

Regards,
--PA


> The personal opinion of
> Gary G. Little
>
> "Pavel A." wrote:
>
> > "anonymous" wrote:
> > > I haven't had a lot of response in the platform builder newsgroups, so I'll
> > > try here.
> > >
> > > In Windows CE 4.1, I had no problem with a NDIS ISR (miniportISR)
> > > interrupting the DPC(MiniportHandleInterrupt). In upgrading to Windows CE
> > > 5.0, I'm seeing that the ISR is getting held off by the DPC.
> >
> > Hi,
> > First of all, could you please explain why your ISR occurs while the DPC
> > is scheduled?
> > When interrupt occurs and the ISR queues a DPC, it should normally prevent
> > the device from generating more interrupts.
> > Then the DPC runs and re-enables the interrupts just before returning.
> > Why this design isn't good for you?
> >
> > Regards,
> > --PA
> >
> > >I've tried
> > > serialized/deserialized. I've verified that the interrupt is not disabled in
> > > the kernel and can see with independent means that kernel is seeing the
> > > interrupt and setting the interrupt event in time with the hardware, but the
> > > ISR waits until the DPC finishes until it is run.
> > >
> > > Does anyone have visibility into what has changed between the CE 4.x and 5.x
> > > NDIS implementation ? The driver registers as an NDIS 5.0 miniport.

Arkady
Fri Jul 01 09:51:50 CDT 2005

Pavel !
Just a little addition : that's try that CE model is similar to vxWorks
InterruptHandler ( ISR ) just set event for usermode servicethread
( IST ) thus excluding DCP at all , but for NDIS in CE that still emulate(d)
the same DPC behavious as in big windows , so maybe
some general concept changed in the last CE , because OP pointed that they
run on the same thread which wasn't possible in previous versions of little
window for any driver especially for network ( NDIS ).
Arkady

"Pavel A." <pavel_a@NOwritemeNO.com> wrote in message
news:273A0DF7-E7A3-49CD-B791-957F4E87882B@microsoft.com...
> "Gary G. Little" wrote:
>> Not necessarily. I have dealt with drivers that could not diasble
>> interrupts
>> and then let the DPC re-enable them. Done properly, there is no reason
>> why
>> the device's interrupt should be disabled in the ISR and enabled in the
>> DPC.
>> You do have to sync access to resources that are shared between the two.
>
> You're correct, but as you see, this design is not quite portable to
> WinCE.
> How exactly hardware interrupts behave on CE depends on the
> manufacturer of the CE device.
> I'm not sure about CE 5, but in previous CE versions
> installable drivers could not receive hardware interrupts at all, and thus
> did not have real ISRs. What they get is callbacks in context of a high
> priority user mode threads. DPCs are emulated with usermode
> threads with lesser priority.
> So the different behavior in CE5 could be caused by mistake of
> somebody who configured it for the specific platform
> (assigned wrong priorities to interrupt and DPC threads, etc).
>
> Regards,
> --PA
>
>
>> The personal opinion of
>> Gary G. Little
>>
>> "Pavel A." wrote:
>>
>> > "anonymous" wrote:
>> > > I haven't had a lot of response in the platform builder newsgroups,
>> > > so I'll
>> > > try here.
>> > >
>> > > In Windows CE 4.1, I had no problem with a NDIS ISR (miniportISR)
>> > > interrupting the DPC(MiniportHandleInterrupt). In upgrading to
>> > > Windows CE
>> > > 5.0, I'm seeing that the ISR is getting held off by the DPC.
>> >
>> > Hi,
>> > First of all, could you please explain why your ISR occurs while the
>> > DPC
>> > is scheduled?
>> > When interrupt occurs and the ISR queues a DPC, it should normally
>> > prevent
>> > the device from generating more interrupts.
>> > Then the DPC runs and re-enables the interrupts just before returning.
>> > Why this design isn't good for you?
>> >
>> > Regards,
>> > --PA
>> >
>> > >I've tried
>> > > serialized/deserialized. I've verified that the interrupt is not
>> > > disabled in
>> > > the kernel and can see with independent means that kernel is seeing
>> > > the
>> > > interrupt and setting the interrupt event in time with the hardware,
>> > > but the
>> > > ISR waits until the DPC finishes until it is run.
>> > >
>> > > Does anyone have visibility into what has changed between the CE 4.x
>> > > and 5.x
>> > > NDIS implementation ? The driver registers as an NDIS 5.0 miniport.

pavel_a
Fri Jul 01 20:15:02 CDT 2005

"Arkady Frenkel" wrote:
> Pavel !
> Just a little addition : that's try that CE model is similar to vxWorks
> InterruptHandler ( ISR ) just set event for usermode servicethread
> ( IST ) thus excluding DCP at all , but for NDIS in CE that still emulate(d)
> the same DPC behavious as in big windows , so maybe
> some general concept changed in the last CE , because OP pointed that they
> run on the same thread which wasn't possible in previous versions of little
> window for any driver especially for network ( NDIS ).
> Arkady

The number and priorities of ISTs vary by platform, this is why
I suspected that the CE build the OP works with, is misconfigured.
This could be just because of some changes in CE5
config options and PB.

Regards,
--PA

Arkady
Sun Jul 03 00:34:44 CDT 2005


"Pavel A." <pavel_a@NOwritemeNO.com> wrote in message
news:924DCC74-0DCD-4AAE-9E9B-44917C51ECA7@microsoft.com...
> "Arkady Frenkel" wrote:
>> Pavel !
>> Just a little addition : that's try that CE model is similar to vxWorks
>> InterruptHandler ( ISR ) just set event for usermode servicethread
>> ( IST ) thus excluding DCP at all , but for NDIS in CE that still
>> emulate(d)
>> the same DPC behavious as in big windows , so maybe
>> some general concept changed in the last CE , because OP pointed that
>> they
>> run on the same thread which wasn't possible in previous versions of
>> little
>> window for any driver especially for network ( NDIS ).
>> Arkady
>
> The number and priorities of ISTs vary by platform, this is why
> I suspected that the CE build the OP works with, is misconfigured.
> This could be just because of some changes in CE5
> config options and PB.
But windows CE open source is not so open to change NDIS behaviour by OEM.
They can't run on the same thread unless MSFT want it so
Arkady

>
> Regards,
> --PA
>
>

Arkady
Mon Jul 04 03:31:43 CDT 2005

Pavel ! That IMHO ( possible to say I'm sure it in :) ) will be interesting
for you , look at the answer of Soemin Tjong [MSFT] on that question of OP
on microsoft.public.windowsce.platbuilder .
Arkady

"Arkady Frenkel" <arkadyf@hotmailxdotx.com> wrote in message
news:eKe3Gj4fFHA.2152@TK2MSFTNGP14.phx.gbl...
>
> "Pavel A." <pavel_a@NOwritemeNO.com> wrote in message
> news:924DCC74-0DCD-4AAE-9E9B-44917C51ECA7@microsoft.com...
>> "Arkady Frenkel" wrote:
>>> Pavel !
>>> Just a little addition : that's try that CE model is similar to vxWorks
>>> InterruptHandler ( ISR ) just set event for usermode servicethread
>>> ( IST ) thus excluding DCP at all , but for NDIS in CE that still
>>> emulate(d)
>>> the same DPC behavious as in big windows , so maybe
>>> some general concept changed in the last CE , because OP pointed that
>>> they
>>> run on the same thread which wasn't possible in previous versions of
>>> little
>>> window for any driver especially for network ( NDIS ).
>>> Arkady
>>
>> The number and priorities of ISTs vary by platform, this is why
>> I suspected that the CE build the OP works with, is misconfigured.
>> This could be just because of some changes in CE5
>> config options and PB.
> But windows CE open source is not so open to change NDIS behaviour by OEM.
> They can't run on the same thread unless MSFT want it so
> Arkady
>
>>
>> Regards,
>> --PA
>>
>>
>
>

Arkady
Mon Jul 04 04:10:50 CDT 2005

Additionally : I heard that they rewrite TCP stack , but thought that only
in connection with LSP they added into the chain , I see now that really
they re-write much more :)
Arkady

"Arkady Frenkel" <arkadyf@hotmailxdotx.com> wrote in message
news:%2367TlnGgFHA.2472@TK2MSFTNGP15.phx.gbl...
> Pavel ! That IMHO ( possible to say I'm sure it in :) ) will be
> interesting for you , look at the answer of Soemin Tjong [MSFT] on that
> question of OP on microsoft.public.windowsce.platbuilder .
> Arkady
>
> "Arkady Frenkel" <arkadyf@hotmailxdotx.com> wrote in message
> news:eKe3Gj4fFHA.2152@TK2MSFTNGP14.phx.gbl...
>>
>> "Pavel A." <pavel_a@NOwritemeNO.com> wrote in message
>> news:924DCC74-0DCD-4AAE-9E9B-44917C51ECA7@microsoft.com...
>>> "Arkady Frenkel" wrote:
>>>> Pavel !
>>>> Just a little addition : that's try that CE model is similar to vxWorks
>>>> InterruptHandler ( ISR ) just set event for usermode servicethread
>>>> ( IST ) thus excluding DCP at all , but for NDIS in CE that still
>>>> emulate(d)
>>>> the same DPC behavious as in big windows , so maybe
>>>> some general concept changed in the last CE , because OP pointed that
>>>> they
>>>> run on the same thread which wasn't possible in previous versions of
>>>> little
>>>> window for any driver especially for network ( NDIS ).
>>>> Arkady
>>>
>>> The number and priorities of ISTs vary by platform, this is why
>>> I suspected that the CE build the OP works with, is misconfigured.
>>> This could be just because of some changes in CE5
>>> config options and PB.
>> But windows CE open source is not so open to change NDIS behaviour by
>> OEM.
>> They can't run on the same thread unless MSFT want it so
>> Arkady
>>
>>>
>>> Regards,
>>> --PA
>>>
>>>
>>
>>
>
>

Pavel
Mon Jul 04 17:55:53 CDT 2005

Thanks.
--PA

"Arkady Frenkel" <arkadyf@hotmailxdotx.com> wrote in message news:OleMc9GgFHA.1412@TK2MSFTNGP09.phx.gbl...
> Additionally : I heard that they rewrite TCP stack , but thought that only in connection with LSP they added into the chain ,
> I see now that really they re-write much more :)
> Arkady