XP USB driver stack/lost bytes on bulk transfer

Hello

I loose bytes ( some multiple of 64 bytes ( =MaxPacketSize ) ) with
Windows XP SP1 on a Gibabyte PC 566 MHz ( single CPU ) with a VIA
chipset ( USB 1.1 HC ).

The following happens:
My USB device driver sends 'read' urbs down the stack ( reading a
picture ~1.5MB from scanner ). USBD_SHORT_TRANSFER_OK ist set and
MaximumTransferSize ist default ( 4KB ).
If i read the image with
urb->UrbBulkOrInterruptTransfer.TransferBufferLength=64 bytes, some 64
byte-packets get lost in lower drivers.
When TransferBufferLength is set on some higher value ( multiple of 64
) the image is transfered correctly.

Tests:
Tested some ( faster ) ASUS pcs with Intel chipset -> No problems.
Downloaded and installed the latest USB hotfix -> No changes.
Wrote a filter driver 'lower' my driver -> Yes, bytes get lost when
requesting 64 bytes.
Wrote a 'HC debug driver' permanent checking the HC registers -> HC
never reports errors ( did not check transfer descriptors ).
Counted bytes on USB analyzer -> The whole image is transferred.
Activated debug outputs in my driver -> OK, also when
TransferBufferLength=64!
Changed internal flowings in my usb driver to be faster -> Bytes get
lost on TransferBufferLength=64 AND TransferBufferLength=128 ( OK, on
higher values )!
Tested my own NT and OS/2 drivers on that motherboard ( the image is
transferred twice faster than under XP ) -> OK.
Used checked build versions of the XP usb drivers -> did not see any
errors reported in windbg, but i'm not very experienced in that.

I know, that VIA has ( maybe had? ) some special 'features'
implemented in USB ( in contrast to Intel ), but for me it looks like
a bug in the XP USB driver stack; maybe some 'timing issue' ( the
faster the usb device driver, the more bytes it must request each
urb!? ).

Any ideas?

Thank you,
Bernhard Ruhsam

PS: The reason, why i have to request just 64 bytes is, that there's a
bug in the firmware of the scanner, which does not send zero length
packets.

Leo
Thu Jan 13 05:59:15 CST 2005

"bench" <ruh@keba.co.at> wrote in message
news:ad44867b.0501130204.35c548c9@posting.google.com...
> Hello
>
> I loose bytes ( some multiple of 64 bytes ( =MaxPacketSize ) ) with
> Windows XP SP1 on a Gibabyte PC 566 MHz ( single CPU ) with a VIA
> chipset ( USB 1.1 HC ).

AFAIK VIA use Open Host Controller, and the ASUS with Intel chipset uses
Universal Host Controller.
Lots of devices have problems with Open host controllers.
Find some USB PCI cards, and try with a Open host controller in the ASUS PC
and with a Universal host controller in the VIA PC.

Leo Havmøller.

Bench
Thu Jan 13 08:19:06 CST 2005

The VT82C596B from VIA is 'UHCI' and they call it 'Intel Universal HCI v.1.1
compatible' .
So XPs miniport driver usbuhci.sys is loaded.

- Bernhard Ruhsam


"Leo Havmøller" wrote:

> "bench" <ruh@keba.co.at> wrote in message
> news:ad44867b.0501130204.35c548c9@posting.google.com...
> > Hello
> >
> > I loose bytes ( some multiple of 64 bytes ( =MaxPacketSize ) ) with
> > Windows XP SP1 on a Gibabyte PC 566 MHz ( single CPU ) with a VIA
> > chipset ( USB 1.1 HC ).
>
> AFAIK VIA use Open Host Controller, and the ASUS with Intel chipset uses
> Universal Host Controller.
> Lots of devices have problems with Open host controllers.
> Find some USB PCI cards, and try with a Open host controller in the ASUS PC
> and with a Universal host controller in the VIA PC.
>
> Leo Havmøller.
>
>
>

Alexander
Thu Jan 13 10:14:26 CST 2005

I think there is another bug in the scanner's firmware that causes it to
lose data that cannot be transferred fast enough.

64 bytes per transfer will give you 64 kbytes/s, which is too little for
most good scanners.

"bench" <ruh@keba.co.at> wrote in message
news:ad44867b.0501130204.35c548c9@posting.google.com...
> Hello
>
> I loose bytes ( some multiple of 64 bytes ( =MaxPacketSize ) ) with
> Windows XP SP1 on a Gibabyte PC 566 MHz ( single CPU ) with a VIA
> chipset ( USB 1.1 HC ).
>
> The following happens:
> My USB device driver sends 'read' urbs down the stack ( reading a
> picture ~1.5MB from scanner ). USBD_SHORT_TRANSFER_OK ist set and
> MaximumTransferSize ist default ( 4KB ).
> If i read the image with
> urb->UrbBulkOrInterruptTransfer.TransferBufferLength=64 bytes, some 64
> byte-packets get lost in lower drivers.
> When TransferBufferLength is set on some higher value ( multiple of 64
> ) the image is transfered correctly.
>
> Tests:
> Tested some ( faster ) ASUS pcs with Intel chipset -> No problems.
> Downloaded and installed the latest USB hotfix -> No changes.
> Wrote a filter driver 'lower' my driver -> Yes, bytes get lost when
> requesting 64 bytes.
> Wrote a 'HC debug driver' permanent checking the HC registers -> HC
> never reports errors ( did not check transfer descriptors ).
> Counted bytes on USB analyzer -> The whole image is transferred.
> Activated debug outputs in my driver -> OK, also when
> TransferBufferLength=64!
> Changed internal flowings in my usb driver to be faster -> Bytes get
> lost on TransferBufferLength=64 AND TransferBufferLength=128 ( OK, on
> higher values )!
> Tested my own NT and OS/2 drivers on that motherboard ( the image is
> transferred twice faster than under XP ) -> OK.
> Used checked build versions of the XP usb drivers -> did not see any
> errors reported in windbg, but i'm not very experienced in that.
>
> I know, that VIA has ( maybe had? ) some special 'features'
> implemented in USB ( in contrast to Intel ), but for me it looks like
> a bug in the XP USB driver stack; maybe some 'timing issue' ( the
> faster the usb device driver, the more bytes it must request each
> urb!? ).
>
> Any ideas?
>
> Thank you,
> Bernhard Ruhsam
>
> PS: The reason, why i have to request just 64 bytes is, that there's a
> bug in the firmware of the scanner, which does not send zero length
> packets.

Randy
Fri Jan 14 12:10:49 CST 2005

>>>64 bytes per transfer will give you 64 kbytes/s, which is too little
for
>>>most good scanners.

Two points: (1) most *good* scanners will stop when their internal
buffer fills up. (2) there is no generic reason, bandwidth being
available, that you cannot schedule additional bulk transfers during
the same USB frame. Of course, this doesn't answer the original
question, but...
Cheers,
Randy Hyde

Tim
Sat Jan 15 00:17:45 CST 2005

"Randy" <rhyde@troygroup.com> wrote:

>>>>64 bytes per transfer will give you 64 kbytes/s, which is too little
>>>>for most good scanners.
>
>Two points: (1) most *good* scanners will stop when their internal
>buffer fills up. (2) there is no generic reason, bandwidth being
>available, that you cannot schedule additional bulk transfers during
>the same USB frame. Of course, this doesn't answer the original
>question, but...

Remember that the original poster's problem only occurred when he sent down
his IRPs with TransferBufferLength set to 64 bytes. You can't turn around
an IRP/URB fast enough to get two in a frame.
--
- Tim Roberts, timr@probo.com
Providenza & Boekelheide, Inc

Alexander
Sun Jan 16 09:51:31 CST 2005

OP also said that the scanner firmware was so crappy it didn't supply
zero-length packets. No surprize it might lose data on overrun.

"Tim Roberts" <timr@probo.com> wrote in message
news:u7dhu05ausr2dpbp3kidbrfud8uooonpvi@4ax.com...
> "Randy" <rhyde@troygroup.com> wrote:
>
>>>>>64 bytes per transfer will give you 64 kbytes/s, which is too little
>>>>>for most good scanners.
>>
>>Two points: (1) most *good* scanners will stop when their internal
>>buffer fills up. (2) there is no generic reason, bandwidth being
>>available, that you cannot schedule additional bulk transfers during
>>the same USB frame. Of course, this doesn't answer the original
>>question, but...
>
> Remember that the original poster's problem only occurred when he sent
> down
> his IRPs with TransferBufferLength set to 64 bytes. You can't turn around
> an IRP/URB fast enough to get two in a frame.
> --
> - Tim Roberts, timr@probo.com
> Providenza & Boekelheide, Inc

Bench
Mon Jan 17 01:05:01 CST 2005

The image is completly transmitted from scanner to HC - this is checked by an
USB analyzer.

Thank you,
- Bernhard Ruhsam

"Alexander Grigoriev" wrote:

> OP also said that the scanner firmware was so crappy it didn't supply
> zero-length packets. No surprize it might lose data on overrun.
>
> "Tim Roberts" <timr@probo.com> wrote in message
> news:u7dhu05ausr2dpbp3kidbrfud8uooonpvi@4ax.com...
> > "Randy" <rhyde@troygroup.com> wrote:
> >
> >>>>>64 bytes per transfer will give you 64 kbytes/s, which is too little
> >>>>>for most good scanners.
> >>
> >>Two points: (1) most *good* scanners will stop when their internal
> >>buffer fills up. (2) there is no generic reason, bandwidth being
> >>available, that you cannot schedule additional bulk transfers during
> >>the same USB frame. Of course, this doesn't answer the original
> >>question, but...
> >
> > Remember that the original poster's problem only occurred when he sent
> > down
> > his IRPs with TransferBufferLength set to 64 bytes. You can't turn around
> > an IRP/URB fast enough to get two in a frame.
> > --
> > - Tim Roberts, timr@probo.com
> > Providenza & Boekelheide, Inc
>
>
>

Jan
Mon Jan 17 06:58:38 CST 2005

"Alexander Grigoriev" <alegr@earthlink.net> schrieb im Newsbeitrag
news:OkZ50rY%23EHA.1604@TK2MSFTNGP12.phx.gbl...
>I think there is another bug in the scanner's firmware that causes it to
>lose data that cannot be transferred fast enough.
>
> 64 bytes per transfer will give you 64 kbytes/s, which is too little for
> most good scanners.

Hi Alexander!
Could you please point me to where this "one-transfer-per-millisecond"
comes from? I've made the same experiance on Windows (I could not send more
then 1000 URBs per second, almost independent of the size (check 8..1024
bytes)), but have not yet found out, if this is a fundamental (part of the
USB specifications) of a due to Microsoft's implementation.

Cheers Jan

Jan
Mon Jan 17 07:07:40 CST 2005

"Alexander Grigoriev" <alegr@earthlink.net> schrieb im Newsbeitrag
news:74wGd.8851$Ii4.3960@newsread3.news.pas.earthlink.net...
> OP also said that the scanner firmware was so crappy it didn't supply
> zero-length packets. No surprize it might lose data on overrun.
>
Hi all!
The firmware is not supposed to send zero length packages. According to
USB specifications, a Bulk transaction has finished if either all data was
send or if one packet has less then wMaxPacketSize bytes.
Neven the less, I also think, that there is a problem with the firmware:
I could imagine, that the Bulk endpoints are double buffered and that the
firmware is not checking both buffer-full flags. In that case it might
overright just one of the two bufferes if the host controler is not reading
fast enough. (As happend to me while using a PDIUSBD12 ...)

Best Regards
Jan

Alexander
Mon Jan 17 09:43:34 CST 2005

Not sending ZLP is OK if you know that your transfers will always end with
incomplete packets or you know exact maximum transfer length. If the actual
transfer size is unknown and it may be multiple of packet size, the transfer
should end with ZLP, otherwise the host will wait for data.

"Jan Menzel" <jan.menzelSPAM@gmx.de> wrote in message
news:351ri3F4h3rvoU1@uni-berlin.de...
> "Alexander Grigoriev" <alegr@earthlink.net> schrieb im Newsbeitrag
> news:74wGd.8851$Ii4.3960@newsread3.news.pas.earthlink.net...
>> OP also said that the scanner firmware was so crappy it didn't supply
>> zero-length packets. No surprize it might lose data on overrun.
>>
> Hi all!
> The firmware is not supposed to send zero length packages. According to
> USB specifications, a Bulk transaction has finished if either all data was
> send or if one packet has less then wMaxPacketSize bytes.
> Neven the less, I also think, that there is a problem with the
> firmware: I could imagine, that the Bulk endpoints are double buffered and
> that the firmware is not checking both buffer-full flags. In that case it
> might overright just one of the two bufferes if the host controler is not
> reading fast enough. (As happend to me while using a PDIUSBD12 ...)
>
> Best Regards
> Jan
>
>

Alexander
Mon Jan 17 09:44:21 CST 2005

Queue update is usually done on end-of-frame interrupt.

"Jan Menzel" <jan.menzelSPAM@gmx.de> wrote in message
news:351r1dF4gt5m4U1@uni-berlin.de...
> "Alexander Grigoriev" <alegr@earthlink.net> schrieb im Newsbeitrag
> news:OkZ50rY%23EHA.1604@TK2MSFTNGP12.phx.gbl...
>>I think there is another bug in the scanner's firmware that causes it to
>>lose data that cannot be transferred fast enough.
>>
>> 64 bytes per transfer will give you 64 kbytes/s, which is too little for
>> most good scanners.
>
> Hi Alexander!
> Could you please point me to where this "one-transfer-per-millisecond"
> comes from? I've made the same experiance on Windows (I could not send
> more then 1000 URBs per second, almost independent of the size (check
> 8..1024 bytes)), but have not yet found out, if this is a fundamental
> (part of the USB specifications) of a due to Microsoft's implementation.
>
> Cheers Jan
>
>

Bench
Tue Jan 18 02:45:07 CST 2005

64 Byte per URB does not mean 64KB/sec.
There are a lot of things affecting the tranferspeed - TransferBuffer seems
not to be ( directly ) one.
With 64 Bytes per ( asynchronous ) Irp/Urb i reach about 800KB/sec (
measured in usermode application ) - this is still slow ( 1.5MB on OS/2 and
NT ).
How fast are good drivers on XP and how are they designed?

On USB 1.1 the HC just can transfer 64 Byte packages ( bulk trasnfer ). So
somewhere lower the stack someone has to build transfer descriptors with
MaxLen set to 0x3F. The HC sends as many packets as possible within 1ms. I
think it should not matter ( in the view of the HC ) who break the request
into 64 Byte packages. On an ASUS with i845 chipset i measured higher
transferspeed when requesting 64 Bytes instead of 2KB for example.

Well, maybe there is some design failure in my driver, but in my case
TransferBufferLength does not directly affect the transfer speed.

- Bernhard Ruhsam




"Alexander Grigoriev" wrote:

> Queue update is usually done on end-of-frame interrupt.
>
> "Jan Menzel" <jan.menzelSPAM@gmx.de> wrote in message
> news:351r1dF4gt5m4U1@uni-berlin.de...
> > "Alexander Grigoriev" <alegr@earthlink.net> schrieb im Newsbeitrag
> > news:OkZ50rY%23EHA.1604@TK2MSFTNGP12.phx.gbl...
> >>I think there is another bug in the scanner's firmware that causes it to
> >>lose data that cannot be transferred fast enough.
> >>
> >> 64 bytes per transfer will give you 64 kbytes/s, which is too little for
> >> most good scanners.
> >
> > Hi Alexander!
> > Could you please point me to where this "one-transfer-per-millisecond"
> > comes from? I've made the same experiance on Windows (I could not send
> > more then 1000 URBs per second, almost independent of the size (check
> > 8..1024 bytes)), but have not yet found out, if this is a fundamental
> > (part of the USB specifications) of a due to Microsoft's implementation.
> >
> > Cheers Jan
> >
> >
>
>
>