AVStream: Do we need to KsEdit the AllocatorFraming member?

HI,

I noticed that the AVSHWS sample driver KsEdit the AllocatorFraming
member of KSPIN_DESCRIPTOR_EX during Pin->Create().

Do we need to do this?

Why?


Thanks

Igor
Tue Jun 01 08:27:26 CDT 2004

Since it supports multiple KSDATAFORMATs; If You are
using the only one data format or multiple formats with
the same alloc. requirements You are free not to do this.

KsEdit is ised fro simplicity; this function guarantees that memory
wolud be freed.

"WDD" <anonymous@microsoft.com> wrote in message
news:eRqHQa9REHA.2520@TK2MSFTNGP11.phx.gbl...
> HI,
>
> I noticed that the AVSHWS sample driver KsEdit the AllocatorFraming
> member of KSPIN_DESCRIPTOR_EX during Pin->Create().
>
> Do we need to do this?
>
> Why?
>
>
> Thanks

WDD
Tue Jun 01 08:39:06 CDT 2004

If I follow the samples code and use it in my driver, when I get a call
to process the KsPinGetLeadingEdgeStreamPointer returns NULL.

Igor Slewsarev wrote:
> Since it supports multiple KSDATAFORMATs; If You are
> using the only one data format or multiple formats with
> the same alloc. requirements You are free not to do this.
>
> KsEdit is ised fro simplicity; this function guarantees that memory
> wolud be freed.
>
> "WDD" <anonymous@microsoft.com> wrote in message
> news:eRqHQa9REHA.2520@TK2MSFTNGP11.phx.gbl...
>
>>HI,
>>
>>I noticed that the AVSHWS sample driver KsEdit the AllocatorFraming
>>member of KSPIN_DESCRIPTOR_EX during Pin->Create().
>>
>>Do we need to do this?
>>
>>Why?
>>
>>
>>Thanks
>
>
>

WDD
Tue Jun 01 09:11:55 CDT 2004

Does it matter that I'm not using my own kernel-level allocator? I'm
using the system allocator.

Since this is the case shouldn't the
KSPIN_DESCRIPTOR_EX.AllocatorFraming be set to NULL in this case?





Igor Slewsarev wrote:
> Since it supports multiple KSDATAFORMATs; If You are
> using the only one data format or multiple formats with
> the same alloc. requirements You are free not to do this.
>
> KsEdit is ised fro simplicity; this function guarantees that memory
> wolud be freed.
>
> "WDD" <anonymous@microsoft.com> wrote in message
> news:eRqHQa9REHA.2520@TK2MSFTNGP11.phx.gbl...
>
>>HI,
>>
>>I noticed that the AVSHWS sample driver KsEdit the AllocatorFraming
>>member of KSPIN_DESCRIPTOR_EX during Pin->Create().
>>
>>Do we need to do this?
>>
>>Why?
>>
>>
>>Thanks
>
>
>

Igor
Tue Jun 01 10:36:57 CDT 2004

> Does it matter that I'm not using my own kernel-level allocator? I'm
> using the system allocator.
No. I hope AVStream calls KsCreateAllocator for you;
As far as I understand you have to write your own custom
allocator only if your pin follows source model (or if you
capture board have some special type of memory, e.g.
common buffer maped to user space) .

> Since this is the case shouldn't the
> KSPIN_DESCRIPTOR_EX.AllocatorFraming be set to NULL in this case?
No!

Igor
Tue Jun 01 10:37:55 CDT 2004


"WDD" <anonymous@microsoft.com> wrote in message
news:etZhe29REHA.4020@TK2MSFTNGP11.phx.gbl...
> If I follow the samples code and use it in my driver, when I get a call
> to process the KsPinGetLeadingEdgeStreamPointer returns NULL.
So what?

Max
Tue Jun 01 17:47:55 CDT 2004

A few more comments on this.

When you describe a pin you can choose to provide a description of memory
allocator _properties_. This is not a memory allocator that you provide, but
rather a properties on the buffers that "some" allocator will provide to
your pin.
Why bother? Because you may know in advance that your pin can deal with
paged memory, for example. In that case you can (and should) tell it to KS
so that you can be nice to the rest of the system by not wasting nonpaged
pool.

However the size of a buffer as well as the number of buffers can depend on
the data format negotiated on the pin. And you won't know the format until
the pin gets connected. So what AVStream sample does, it fixes the allocator
properties after the pin is connected and after you know exactly the data
format that has been negotiated.

As Igor said if your pin supports only one format then you may not need all
this because you can specify exact buffer requirements in advance in static
memory.

Also KSPIN_DESCRIPTOR_EX.AllocatorFraming _can_ be set to NULL.
It is a way for pin to say that it does not care about allocator
requirements, i.e. it agrees with everything.
In that case the allocator properties are either dictated by the peer pin or
default ones are selected.

-- Max.


"Igor Slewsarev" <slewsarev_no_spam_@nospam.yahoo.com> wrote in message
news:e4uHG5%23REHA.904@TK2MSFTNGP12.phx.gbl...
> > Does it matter that I'm not using my own kernel-level allocator? I'm
> > using the system allocator.
> No. I hope AVStream calls KsCreateAllocator for you;
> As far as I understand you have to write your own custom
> allocator only if your pin follows source model (or if you
> capture board have some special type of memory, e.g.
> common buffer maped to user space) .
>
> > Since this is the case shouldn't the
> > KSPIN_DESCRIPTOR_EX.AllocatorFraming be set to NULL in this case?
> No!
>
>

WDD
Wed Jun 02 08:55:48 CDT 2004



How do I perform any work(DMA) with NULL pointer?


Igor Slewsarev wrote:
> "WDD" <anonymous@microsoft.com> wrote in message
> news:etZhe29REHA.4020@TK2MSFTNGP11.phx.gbl...
>
>>If I follow the samples code and use it in my driver, when I get a call
>>to process the KsPinGetLeadingEdgeStreamPointer returns NULL.
>
> So what?
>
>

Igor
Wed Jun 02 10:24:44 CDT 2004

Don't start DMA. This happens very offen and
there are a lot of reasons why your queue is empty (it seems
to me typing mistakes in pin descriptions in you case):
e.g. renderer blocks stream while processing media sample,
number of outstanding frames is not enough and etc.
Please, first thoroughly inspect your code. This is useful in
any case.

WDD
Wed Jun 02 11:18:20 CDT 2004

1. MY Pin descriptor is Identical to the Descriptor used by the sample,
the only difference I see in the driver code is that I do NOT set the
flag for ScatterGather = TRUE in the DeviceDescription when I get my DMA
Adapter. I don't think that's needed since I set
KSPIN_FLAG_GENERATE_MAPPINGS in the KSPIN_DESCRIPTOR_EX for the Pin.

2. If i set the following KSPIN_DESCRIPTOR_EX.Flags,
KSPIN_FLAG_GENERATE_MAPPINGS |
KSPIN_FLAG_INITIATE_PROCESSING_ON_EVERY_ARRIVAL.

Which should cause my Process() to be called when data arrives into a
previously empty Queue. Now if my Process() gets invoked, that means
there is Data, right? Then if there's data in the Queue, why would my
attempt to get the Leading edge be NULL?


Igor Slewsarev wrote:
> Don't start DMA. This happens very offen and
> there are a lot of reasons why your queue is empty (it seems
> to me typing mistakes in pin descriptions in you case):
> e.g. renderer blocks stream while processing media sample,
> number of outstanding frames is not enough and etc.
> Please, first thoroughly inspect your code. This is useful in
> any case.
>
>

Max
Wed Jun 02 16:04:07 CDT 2004

If you specify KSPIN_FLAG_INITIATE_PROCESSING_ON_EVERY_ARRIVAL, then your
Process() will be called upon EVERY new frame arrival, not just arrival on
an empty queue.
This does not explain the leading edge SP being NULL.

Process() can be also called as the result of KsPinAttemptProcessing. Check
your code and see if you call it.

Also there can be a following racing condition:
1. KS receives a frame and decides to call your Process().
2. before the call actually ends up in your Process() the thread get
preempted (by your ISR's DPC or something else) and part of your logic
advances the SP passed the end of the queue.
3. your Process() executes and then the leading edge can be NULL.

Conclusions:
1. You got a bug in your driver. Fix it.
2. Be prepared to deal with NULL returned as the leading edge SP no matter
what your belief of the current state of KS queue is. It can get screwed up
in some other strange way. No matter what shape the queue in, it is hardly a
reason for a blue screen.
There are cases when it is better to crash, but this is NOT one of them.

-- Max.




"WDD" <anonymous@microsoft.com> wrote in message
news:O4XbzzLSEHA.2772@TK2MSFTNGP10.phx.gbl...
> 1. MY Pin descriptor is Identical to the Descriptor used by the sample,
> the only difference I see in the driver code is that I do NOT set the
> flag for ScatterGather = TRUE in the DeviceDescription when I get my DMA
> Adapter. I don't think that's needed since I set
> KSPIN_FLAG_GENERATE_MAPPINGS in the KSPIN_DESCRIPTOR_EX for the Pin.
>
> 2. If i set the following KSPIN_DESCRIPTOR_EX.Flags,
> KSPIN_FLAG_GENERATE_MAPPINGS |
> KSPIN_FLAG_INITIATE_PROCESSING_ON_EVERY_ARRIVAL.
>
> Which should cause my Process() to be called when data arrives into a
> previously empty Queue. Now if my Process() gets invoked, that means
> there is Data, right? Then if there's data in the Queue, why would my
> attempt to get the Leading edge be NULL?
>
>
> Igor Slewsarev wrote:
> > Don't start DMA. This happens very offen and
> > there are a lot of reasons why your queue is empty (it seems
> > to me typing mistakes in pin descriptions in you case):
> > e.g. renderer blocks stream while processing media sample,
> > number of outstanding frames is not enough and etc.
> > Please, first thoroughly inspect your code. This is useful in
> > any case.
> >
> >

Igor
Thu Jun 03 04:24:06 CDT 2004

For what a hell You set KSPIN_FLAG_GENERATE_MAPPINGS
if Your device doesn't support scatter gather? If your device
follows driver model with DMA common buffer You shouldn't
use mappings - just copy your common buffer contents at DPC.
If not - set ScatterGather flag.
Check Your code!!!

"WDD" <anonymous@microsoft.com> wrote in message
news:O4XbzzLSEHA.2772@TK2MSFTNGP10.phx.gbl...
> 1. MY Pin descriptor is Identical to the Descriptor used by the sample,
> the only difference I see in the driver code is that I do NOT set the
> flag for ScatterGather = TRUE in the DeviceDescription when I get my DMA
> Adapter. I don't think that's needed since I set
> KSPIN_FLAG_GENERATE_MAPPINGS in the KSPIN_DESCRIPTOR_EX for the Pin.
>
> 2. If i set the following KSPIN_DESCRIPTOR_EX.Flags,
> KSPIN_FLAG_GENERATE_MAPPINGS |
> KSPIN_FLAG_INITIATE_PROCESSING_ON_EVERY_ARRIVAL.
>
> Which should cause my Process() to be called when data arrives into a
> previously empty Queue. Now if my Process() gets invoked, that means
> there is Data, right? Then if there's data in the Queue, why would my
> attempt to get the Leading edge be NULL?
>
>
> Igor Slewsarev wrote:
> > Don't start DMA. This happens very offen and
> > there are a lot of reasons why your queue is empty (it seems
> > to me typing mistakes in pin descriptions in you case):
> > e.g. renderer blocks stream while processing media sample,
> > number of outstanding frames is not enough and etc.
> > Please, first thoroughly inspect your code. This is useful in
> > any case.
> >
> >

WDD
Thu Jun 03 09:06:50 CDT 2004

Well, as per DDK "A KSPIN that is performing DMA to or from hardware
must specify the flag KSPIN_FLAG_GENERATE_MAPPINGS." Doesn't mention
anything about ScatterGather support or not, actually I should be able
to program the DMA op with the PhysicalAddress given in the Mappings
directly. No? Especially if they give me 1 mapping in each stream
pointer since I don't set the scatter gather support ot TRUE.

Anyway,

Initially I wasn't using Common Buffer DMA, but have since changed to
that. In any case what would be no difference in setting the
KSPIN_FLAG_GENERATE_MAPPINGS flag, admittedly I cause KS to do more work
for me, but I still populate the buffer via Pointer->StreamHeader->Data
member(which is identical to the OffsetOut->Data) at DPC time?

BTW, I'm just learning from others here and asking alot of questions, I
do appreciate any comments & suggestions, I don't mean to frustrate but
sometimes it happens. ;)


Igor Slewsarev wrote:
> For what a hell You set KSPIN_FLAG_GENERATE_MAPPINGS
> if Your device doesn't support scatter gather? If your device
> follows driver model with DMA common buffer You shouldn't
> use mappings - just copy your common buffer contents at DPC.
> If not - set ScatterGather flag.
> Check Your code!!!
>
> "WDD" <anonymous@microsoft.com> wrote in message
> news:O4XbzzLSEHA.2772@TK2MSFTNGP10.phx.gbl...
>
>>1. MY Pin descriptor is Identical to the Descriptor used by the sample,
>>the only difference I see in the driver code is that I do NOT set the
>>flag for ScatterGather = TRUE in the DeviceDescription when I get my DMA
>>Adapter. I don't think that's needed since I set
>>KSPIN_FLAG_GENERATE_MAPPINGS in the KSPIN_DESCRIPTOR_EX for the Pin.
>>
>>2. If i set the following KSPIN_DESCRIPTOR_EX.Flags,
>>KSPIN_FLAG_GENERATE_MAPPINGS |
>>KSPIN_FLAG_INITIATE_PROCESSING_ON_EVERY_ARRIVAL.
>>
>>Which should cause my Process() to be called when data arrives into a
>>previously empty Queue. Now if my Process() gets invoked, that means
>>there is Data, right? Then if there's data in the Queue, why would my
>>attempt to get the Leading edge be NULL?
>>
>>
>>Igor Slewsarev wrote:
>>
>>>Don't start DMA. This happens very offen and
>>>there are a lot of reasons why your queue is empty (it seems
>>>to me typing mistakes in pin descriptions in you case):
>>>e.g. renderer blocks stream while processing media sample,
>>>number of outstanding frames is not enough and etc.
>>>Please, first thoroughly inspect your code. This is useful in
>>>any case.
>>>
>>>
>
>
>

Igor
Thu Jun 03 11:54:27 CDT 2004

> Well, as per DDK "A KSPIN that is performing DMA to or from hardware
> must specify the flag KSPIN_FLAG_GENERATE_MAPPINGS." Doesn't mention
> anything about ScatterGather support or not, actually I should be able
> to program the DMA op with the PhysicalAddress given in the Mappings
> directly. No? Especially if they give me 1 mapping in each stream
> pointer since I don't set the scatter gather support ot TRUE.

This is quotation from MSDN:
"Packet-based direct memory access (DMA) occurs when your minidriver
reads data directly from and writes data directly to capture buffers
received from user mode...
To implement a packet-based DMA scheme:
Specify KSPIN_FLAG_GENERATE_MAPPINGS in the Flags member of relevant
KSPIN_DESCRIPTOR_EX structures. Note that this flag should only be used
by a bus master with SCATTER/GATHER support.
Register an interrupt service routine (ISR) as described in Writing
AVStream Minidrivers for Hardware.
Then in the AVStrMiniDeviceStart start dispatch:
Set up a DMA adapter object using IoGetDmaAdapter.
Register the DMA adapter object with AVStream by calling
KsDeviceRegisterAdapterObject."

If You don't specify ScatterGather flag this means that you want media
samples with physically contigious memory buffer. Now think, could
usermode allocators allocate such memory? I also suppose that even
KM allocator provided by ks.sys is unable to allocate such memory.
(and sometimes it's really impossible. I have some sad experience in
allocating 512K contigiuos blocks with
MmAllocateContiguousMemorySpecifyCache)

> Initially I wasn't using Common Buffer DMA, but have since changed to
> that. In any case what would be no difference in setting the
> KSPIN_FLAG_GENERATE_MAPPINGS flag, admittedly I cause KS to do more work
> for me, but I still populate the buffer via Pointer->StreamHeader->Data
> member(which is identical to the OffsetOut->Data) at DPC time?
Another quotation:
"To use common buffer DMA in your AVStream minidriver:
Acquire a DMA adapter by calling IoGetDmaAdapter, as in packet-based
DMA. Do not specify KSPIN_FLAG_GENERATE_MAPPINGS in the Flags member
of the KSPIN_DESCRIPTOR_EX structure, and do not register your DMA
adapter with AVStream. You might need to implement your own private
buffer/copy scheme." and etc.

Pointer->StreamHeader->Data points to the begining of frame buffer;
Pointer->OffsetOut.Data points to the current write position;
Suppose that you receive media sample which cann't be filled in one
DMA xaction (e.g. Your capure board transfers one field per xaction).
in this case You have to advance stream pointer Offset field.
Next time Your DPC is called You retrieve the second field and
write it right after first and finally eject frame;

BTW what's Your name and from where You are?

S.Y.
Igor V. Slewsarev

WDD
Thu Jun 03 12:24:07 CDT 2004

Thanks, I would have loved to read this before, can you direct me to the
MSDN documentation(www link if available).

BTW, Bond, "James" Bond
Great White North.

Igor Slewsarev wrote:
>>Well, as per DDK "A KSPIN that is performing DMA to or from hardware
>>must specify the flag KSPIN_FLAG_GENERATE_MAPPINGS." Doesn't mention
>>anything about ScatterGather support or not, actually I should be able
>>to program the DMA op with the PhysicalAddress given in the Mappings
>>directly. No? Especially if they give me 1 mapping in each stream
>>pointer since I don't set the scatter gather support ot TRUE.
>
>
> This is quotation from MSDN:
> "Packet-based direct memory access (DMA) occurs when your minidriver
> reads data directly from and writes data directly to capture buffers
> received from user mode...
> To implement a packet-based DMA scheme:
> Specify KSPIN_FLAG_GENERATE_MAPPINGS in the Flags member of relevant
> KSPIN_DESCRIPTOR_EX structures. Note that this flag should only be used
> by a bus master with SCATTER/GATHER support.
> Register an interrupt service routine (ISR) as described in Writing
> AVStream Minidrivers for Hardware.
> Then in the AVStrMiniDeviceStart start dispatch:
> Set up a DMA adapter object using IoGetDmaAdapter.
> Register the DMA adapter object with AVStream by calling
> KsDeviceRegisterAdapterObject."
>
> If You don't specify ScatterGather flag this means that you want media
> samples with physically contigious memory buffer. Now think, could
> usermode allocators allocate such memory? I also suppose that even
> KM allocator provided by ks.sys is unable to allocate such memory.
> (and sometimes it's really impossible. I have some sad experience in
> allocating 512K contigiuos blocks with
> MmAllocateContiguousMemorySpecifyCache)
>
>
>>Initially I wasn't using Common Buffer DMA, but have since changed to
>>that. In any case what would be no difference in setting the
>>KSPIN_FLAG_GENERATE_MAPPINGS flag, admittedly I cause KS to do more work
>>for me, but I still populate the buffer via Pointer->StreamHeader->Data
>>member(which is identical to the OffsetOut->Data) at DPC time?
>
> Another quotation:
> "To use common buffer DMA in your AVStream minidriver:
> Acquire a DMA adapter by calling IoGetDmaAdapter, as in packet-based
> DMA. Do not specify KSPIN_FLAG_GENERATE_MAPPINGS in the Flags member
> of the KSPIN_DESCRIPTOR_EX structure, and do not register your DMA
> adapter with AVStream. You might need to implement your own private
> buffer/copy scheme." and etc.
>
> Pointer->StreamHeader->Data points to the begining of frame buffer;
> Pointer->OffsetOut.Data points to the current write position;
> Suppose that you receive media sample which cann't be filled in one
> DMA xaction (e.g. Your capure board transfers one field per xaction).
> in this case You have to advance stream pointer Offset field.
> Next time Your DPC is called You retrieve the second field and
> write it right after first and finally eject frame;
>
> BTW what's Your name and from where You are?
>
> S.Y.
> Igor V. Slewsarev
>
>

WDD
Thu Jun 03 15:52:03 CDT 2004

Found the DOC, this is newer DDK info than the 3790.

http://msdn.microsoft.com/library/default.asp?url=/library/en-us/stream/hh/stream/avsover_2753.asp

Cheers

WDD wrote:
> Thanks, I would have loved to read this before, can you direct me to the
> MSDN documentation(www link if available).
>
> BTW, Bond, "James" Bond
> Great White North.
>
> Igor Slewsarev wrote:
>
>>> Well, as per DDK "A KSPIN that is performing DMA to or from hardware
>>> must specify the flag KSPIN_FLAG_GENERATE_MAPPINGS." Doesn't mention
>>> anything about ScatterGather support or not, actually I should be able
>>> to program the DMA op with the PhysicalAddress given in the Mappings
>>> directly. No? Especially if they give me 1 mapping in each stream
>>> pointer since I don't set the scatter gather support ot TRUE.
>>
>>
>>
>> This is quotation from MSDN:
>> "Packet-based direct memory access (DMA) occurs when your minidriver
>> reads data directly from and writes data directly to capture buffers
>> received from user mode...
>> To implement a packet-based DMA scheme:
>> Specify KSPIN_FLAG_GENERATE_MAPPINGS in the Flags member of relevant
>> KSPIN_DESCRIPTOR_EX structures. Note that this flag should only be
>> used
>> by a bus master with SCATTER/GATHER support.
>> Register an interrupt service routine (ISR) as described in Writing
>> AVStream Minidrivers for Hardware.
>> Then in the AVStrMiniDeviceStart start dispatch:
>> Set up a DMA adapter object using IoGetDmaAdapter.
>> Register the DMA adapter object with AVStream by calling
>> KsDeviceRegisterAdapterObject."
>>
>> If You don't specify ScatterGather flag this means that you want media
>> samples with physically contigious memory buffer. Now think, could
>> usermode allocators allocate such memory? I also suppose that even
>> KM allocator provided by ks.sys is unable to allocate such memory.
>> (and sometimes it's really impossible. I have some sad experience in
>> allocating 512K contigiuos blocks with
>> MmAllocateContiguousMemorySpecifyCache)
>>
>>
>>> Initially I wasn't using Common Buffer DMA, but have since changed to
>>> that. In any case what would be no difference in setting the
>>> KSPIN_FLAG_GENERATE_MAPPINGS flag, admittedly I cause KS to do more work
>>> for me, but I still populate the buffer via Pointer->StreamHeader->Data
>>> member(which is identical to the OffsetOut->Data) at DPC time?
>>
>>
>> Another quotation:
>> "To use common buffer DMA in your AVStream minidriver:
>> Acquire a DMA adapter by calling IoGetDmaAdapter, as in packet-based
>> DMA. Do not specify KSPIN_FLAG_GENERATE_MAPPINGS in the Flags member
>> of the KSPIN_DESCRIPTOR_EX structure, and do not register your DMA
>> adapter with AVStream. You might need to implement your own private
>> buffer/copy scheme." and etc.
>>
>> Pointer->StreamHeader->Data points to the begining of frame buffer;
>> Pointer->OffsetOut.Data points to the current write position;
>> Suppose that you receive media sample which cann't be filled in one
>> DMA xaction (e.g. Your capure board transfers one field per xaction).
>> in this case You have to advance stream pointer Offset field.
>> Next time Your DPC is called You retrieve the second field and
>> write it right after first and finally eject frame;
>>
>> BTW what's Your name and from where You are?
>>
>> S.Y.
>> Igor V. Slewsarev
>>
>>