Hi everyone.

I'm trying to develop mirror video driver for VNC-like aplication.
I've made a "stupid" version that just shares framebuffer into user-
space and provide another buffer to store screen updates (just a
bounding rect of a CLIPOBJ). But such a driver "produces" too much
traffic. It is not much less than while using "blit-and-XOR" approach
(BitBlt from screen, XOR with the previous frame and send the result
over a wire) in most cases.

So, I decided to implement a kind of updates z-Buffer, so that you can
discard previous update if it was covered by another one the next
moment. But it is not the case in point.

The second way of bandwidth utilization improvement is taking screen-
to-screen BitBlts into account. They often happen when scrolling
appears. When screen-to-screen BitBlt occurs we need to send just a
notification with the two rects coordinates instead of the whole
bitmap, so this approach can decrease network traffic significally.

But there is one problem. If screen-2-screen BitBlt occurs while user-
mode application reads frame buffer, data sent to the other client
might be inconsistent. And this inconsistency won't be fixed when
"screen-to-screen" command is sent because this command carries no
bitmap data, and data on the other side of the wire is taken from it's
local screen copy which is corrupted.

I see two ways of resolving this problem. First is to refuse the idea
of taking screen-to-screen BitBlt into account and treat them as a
bitmap-to-screen. The second way is to refuse the idea of grabbing the
image directly from the framebuffer and create a separate bitmap for
each BitBlt and share it's bits into userspace. Please tell me what
approach is preferable. I tried to figure it out by disassembling
Microsoft's rdpdd.dll, but it's to complex and has many obscure things
like additional parameters cheks in DrvXxx berfore calling the
corresponding EngXxx and use of private data structures.

Re: Mirror driver, screen-to-screen BitBlt and consistency. by Maxim

Maxim
Thu Aug 30 01:17:39 PDT 2007

I think that VNC sends _drawing commands_ over wire, like X11 and RDP do,
and not _the bitmaps of screen updated regions_, like the old pcAnywhere and
the most newbie products.

Sending commands is by far smarter. For instance, opening the Start menu in
Server 2003 thru Remote Desktop produces only aroud 550 bytes of network
traffic (measured).

--
Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
maxim@storagecraft.com
http://www.storagecraft.com

"mofa" <polukhovich@gmail.com> wrote in message
news:1188458128.197722.63860@22g2000hsm.googlegroups.com...
> Hi everyone.
>
> I'm trying to develop mirror video driver for VNC-like aplication.
> I've made a "stupid" version that just shares framebuffer into user-
> space and provide another buffer to store screen updates (just a
> bounding rect of a CLIPOBJ). But such a driver "produces" too much
> traffic. It is not much less than while using "blit-and-XOR" approach
> (BitBlt from screen, XOR with the previous frame and send the result
> over a wire) in most cases.
>
> So, I decided to implement a kind of updates z-Buffer, so that you can
> discard previous update if it was covered by another one the next
> moment. But it is not the case in point.
>
> The second way of bandwidth utilization improvement is taking screen-
> to-screen BitBlts into account. They often happen when scrolling
> appears. When screen-to-screen BitBlt occurs we need to send just a
> notification with the two rects coordinates instead of the whole
> bitmap, so this approach can decrease network traffic significally.
>
> But there is one problem. If screen-2-screen BitBlt occurs while user-
> mode application reads frame buffer, data sent to the other client
> might be inconsistent. And this inconsistency won't be fixed when
> "screen-to-screen" command is sent because this command carries no
> bitmap data, and data on the other side of the wire is taken from it's
> local screen copy which is corrupted.
>
> I see two ways of resolving this problem. First is to refuse the idea
> of taking screen-to-screen BitBlt into account and treat them as a
> bitmap-to-screen. The second way is to refuse the idea of grabbing the
> image directly from the framebuffer and create a separate bitmap for
> each BitBlt and share it's bits into userspace. Please tell me what
> approach is preferable. I tried to figure it out by disassembling
> Microsoft's rdpdd.dll, but it's to complex and has many obscure things
> like additional parameters cheks in DrvXxx berfore calling the
> corresponding EngXxx and use of private data structures.
>


Re: Mirror driver, screen-to-screen BitBlt and consistency. by mofa

mofa
Thu Aug 30 03:49:33 PDT 2007

I know that. But if user-mode calls GDI32!BitBlt, drawing command will
be "draw bitmap". And BitBlt is called often. AFAIK Microsoft Internet
Explorer, MS Word, Notepad and many other applications do scrolling
via screen-to-screen blt. I think bitmaps form the most of the
traffic. As for Start menu, I thoght that RDP can cache icons, so that
first opening produces more bytes than subsequent ones.

Even if one sends "drawing commands", there is still need in bitmaps
transfer. And I don't know which way of obtaining that bitmaps is
better: framebuffer sharing (which discards possibility of taking
screen-2-screen blts into account) or creating separate bitmap for
each BitBlt.

Thanks, Eugene.


On 30 , 12:17, "Maxim S. Shatskih" <ma...@storagecraft.com> wrote:
> I think that VNC sends _drawing commands_ over wire, like X11 and RDP do,
> and not _the bitmaps of screen updated regions_, like the old pcAnywhere and
> the most newbie products.
>
> Sending commands is by far smarter. For instance, opening the Start menu in
> Server 2003 thru Remote Desktop produces only aroud 550 bytes of network
> traffic (measured).
>
> --
> Maxim Shatskih, Windows DDK MVP
> StorageCraft Corporation
> ma...@storagecraft.comhttp://www.storagecraft.com


Re: Mirror driver, screen-to-screen BitBlt and consistency. by Maxim

Maxim
Thu Aug 30 13:41:41 PDT 2007

> I know that. But if user-mode calls GDI32!BitBlt, drawing command will
> be "draw bitmap". And BitBlt is called often.

Many well-behaving apps and especially controls (most comctl32 I think) do
the following:

CreateCompatibleDC
CreateCompatibleBitmap
draw their icons and buttons to this compatible bitmap once
then, when there is a need to actually draw the icon, they do BitBlt from
compatible bitmap to the screen.

Windows shell does this itself since NT4 times.

Now just implement your driver in a way that the "compatible bitmap" is
also far from the drawing machine, just some "invisible screen" screen-side
(not host-side), and then _run BitBlt from compatible bitmap to screen as a
trivial wire command without sending bits over the wire!_.

This is how opening a Start menu costed only ~520 bytes of traffic.

>AFAIK Microsoft Internet
> Explorer, MS Word, Notepad and many other applications do scrolling
> via screen-to-screen blt.

Yes, ScrollDC and ScrollWindow work this way. So what? Over the wire, this
is only a trivial BitBlt command. The bits itself are not sent over the wire,
for what??? all bits for a scroll are on display-side and not host-side.

>I think bitmaps form the most of the
> traffic.

Yes. Running Photoshop over such a link is slow.

>As for Start menu, I thoght that RDP can cache icons

...and I described above how to support this caching.

> Even if one sends "drawing commands", there is still need in bitmaps
> transfer.

Surely, but if you will go the "screen updates" way - then the traffic will be
many times more. Just imagine the traffic for ScrollDC or TextOut.

--
Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
maxim@storagecraft.com
http://www.storagecraft.com


Re: Mirror driver, screen-to-screen BitBlt and consistency. by Eugene

Eugene
Thu Aug 30 23:46:31 PDT 2007

> I think that VNC sends _drawing commands_ over wire, like X11
> and RDP do,
> and not _the bitmaps of screen updated regions_, like the old
> pcAnywhere and
> the most newbie products.

It's not true. ALL variants of VNC use bitmaps and the queue of
updated rectanges. The VNC protocol itself doesn't have any means to
send drawing commands that makes it very easy to port to any platform
including handhelds, cell phones, etc. It's not the most efficient
protocol ever invented but the most popular for sure.

--
Sincerely,
Eugene Sukhodolin
www.demoforge.com


Re: Mirror driver, screen-to-screen BitBlt and consistency. by mofa

mofa
Sun Sep 02 23:10:03 PDT 2007

Thanks.

I know that VNC uses RFB protocol, so it doesn't operate "drawing
commands". I know that "drawing commands" are much faster than RFB.
I don't want to use RFB because my "screen updates" are not just
bitmaps. They will become "drawing commands" eventually. But at the
moment most drawing data is sent as bitmap.

My question was just how to obtain bitmap data from the framebuffer
and get the correct picture. I was speculating about this problem.
Maybe it is good idea to hold another "framebuffer" where drawing
occurs only when user-mode sends "flush" command to driver (e.g. via
ExtEscape). When driver gets "flush" command it copies bitmap data
from primary framebuffer into auxiliary one and writes bounding rects
to the "updates rects" buffer. This will guarantee data consistency.
But maybe someone knows a better way of doing that.

Eugene.