WinDbg & kernel-mode debugging of virtual machines

Are there any good white papers or other discussions about setting up WinDbg
for kernel-mode debugging of a driver within a virtual machine hosted within
VMware Workstation?

I'm getting started on some file system filter driver and/or file system
minifilter driver development. I have a build environment set up on Vista
Ultimate x64, and have a variety of VM guests for WinXP Pro SP2 x86, Vista
Business x86 & x64, and Win2K3 R2 x86 & x64. My intent was to use these VMs
as my test environment instead of having a separate physical computer with
multiple bootable partitions, etc....

Since kernel-mode debugging with WinDbg involves having a COM port or
FireWire connection between the debug host & the test system, I was looking
for some sort of virtual COM port that I could run under Vista that would
connect up to the named pipe that VMware Workstation can associate with a
COM port inside of a VM guest. Does such a thing exist?

Is anybody else successfully doing kernel-mode debugging of a driver on
Windows running as a VMware guest?


TIA,

Chuck
--
Chuck Chopp

ChuckChopp (at) rtfmcsi (dot) com http://www.rtfmcsi.com

RTFM Consulting Services Inc. 864 801 2795 voice & voicemail
103 Autumn Hill Road 864 801 2774 fax
Greer, SC 29651

Do not send me unsolicited commercial email.

Maxim
Wed Aug 29 09:16:29 CDT 2007

In virtual machine, set up the guest serial port to be a named pipe.

From WinDbg side:

"D:\Program Files\Debugging Tools for Windows\windbg.exe" -Q -v -k
com:port=\\.\pipe\vm,pipe

--
Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
maxim@storagecraft.com
http://www.storagecraft.com

"Chuck Chopp" <ChuckChopp@rtfmcsi.com> wrote in message
news:uKLP3ak6HHA.1188@TK2MSFTNGP04.phx.gbl...
> Are there any good white papers or other discussions about setting up WinDbg
> for kernel-mode debugging of a driver within a virtual machine hosted within
> VMware Workstation?
>
> I'm getting started on some file system filter driver and/or file system
> minifilter driver development. I have a build environment set up on Vista
> Ultimate x64, and have a variety of VM guests for WinXP Pro SP2 x86, Vista
> Business x86 & x64, and Win2K3 R2 x86 & x64. My intent was to use these VMs
> as my test environment instead of having a separate physical computer with
> multiple bootable partitions, etc....
>
> Since kernel-mode debugging with WinDbg involves having a COM port or
> FireWire connection between the debug host & the test system, I was looking
> for some sort of virtual COM port that I could run under Vista that would
> connect up to the named pipe that VMware Workstation can associate with a
> COM port inside of a VM guest. Does such a thing exist?
>
> Is anybody else successfully doing kernel-mode debugging of a driver on
> Windows running as a VMware guest?
>
>
> TIA,
>
> Chuck
> --
> Chuck Chopp
>
> ChuckChopp (at) rtfmcsi (dot) com http://www.rtfmcsi.com
>
> RTFM Consulting Services Inc. 864 801 2795 voice & voicemail
> 103 Autumn Hill Road 864 801 2774 fax
> Greer, SC 29651
>
> Do not send me unsolicited commercial email.

David
Wed Aug 29 11:03:31 CDT 2007

There is a help screen in VMWare on how to do it. Windbg also has comments
about using a VM target. It isn't that difficult if you read both sources
and just try it.

--
David J. Craig
Engineer, Sr. Staff Software Systems
Broadcom Corporation


"Chuck Chopp" <ChuckChopp@rtfmcsi.com> wrote in message
news:uKLP3ak6HHA.1188@TK2MSFTNGP04.phx.gbl...
> Are there any good white papers or other discussions about setting up
> WinDbg for kernel-mode debugging of a driver within a virtual machine
> hosted within VMware Workstation?
>
> I'm getting started on some file system filter driver and/or file system
> minifilter driver development. I have a build environment set up on Vista
> Ultimate x64, and have a variety of VM guests for WinXP Pro SP2 x86, Vista
> Business x86 & x64, and Win2K3 R2 x86 & x64. My intent was to use these
> VMs as my test environment instead of having a separate physical computer
> with multiple bootable partitions, etc....
>
> Since kernel-mode debugging with WinDbg involves having a COM port or
> FireWire connection between the debug host & the test system, I was
> looking for some sort of virtual COM port that I could run under Vista
> that would connect up to the named pipe that VMware Workstation can
> associate with a COM port inside of a VM guest. Does such a thing exist?
>
> Is anybody else successfully doing kernel-mode debugging of a driver on
> Windows running as a VMware guest?
>
>
> TIA,
>
> Chuck
> --
> Chuck Chopp
>
> ChuckChopp (at) rtfmcsi (dot) com http://www.rtfmcsi.com
>
> RTFM Consulting Services Inc. 864 801 2795 voice & voicemail
> 103 Autumn Hill Road 864 801 2774 fax
> Greer, SC 29651
>
> Do not send me unsolicited commercial email.

Maxim
Wed Aug 29 11:16:39 CDT 2007

Important detail: first start the virtual machine, and only then WinDbg.
Otherwise, it sometimes fails to connect (saw this on old versions of VMWare).

--
Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
maxim@storagecraft.com
http://www.storagecraft.com

"David J. Craig" <dave@yoshimuni.com> wrote in message
news:%23R5GmYl6HHA.2632@TK2MSFTNGP06.phx.gbl...
> There is a help screen in VMWare on how to do it. Windbg also has comments
> about using a VM target. It isn't that difficult if you read both sources
> and just try it.
>
> --
> David J. Craig
> Engineer, Sr. Staff Software Systems
> Broadcom Corporation
>
>
> "Chuck Chopp" <ChuckChopp@rtfmcsi.com> wrote in message
> news:uKLP3ak6HHA.1188@TK2MSFTNGP04.phx.gbl...
> > Are there any good white papers or other discussions about setting up
> > WinDbg for kernel-mode debugging of a driver within a virtual machine
> > hosted within VMware Workstation?
> >
> > I'm getting started on some file system filter driver and/or file system
> > minifilter driver development. I have a build environment set up on Vista
> > Ultimate x64, and have a variety of VM guests for WinXP Pro SP2 x86, Vista
> > Business x86 & x64, and Win2K3 R2 x86 & x64. My intent was to use these
> > VMs as my test environment instead of having a separate physical computer
> > with multiple bootable partitions, etc....
> >
> > Since kernel-mode debugging with WinDbg involves having a COM port or
> > FireWire connection between the debug host & the test system, I was
> > looking for some sort of virtual COM port that I could run under Vista
> > that would connect up to the named pipe that VMware Workstation can
> > associate with a COM port inside of a VM guest. Does such a thing exist?
> >
> > Is anybody else successfully doing kernel-mode debugging of a driver on
> > Windows running as a VMware guest?
> >
> >
> > TIA,
> >
> > Chuck
> > --
> > Chuck Chopp
> >
> > ChuckChopp (at) rtfmcsi (dot) com http://www.rtfmcsi.com
> >
> > RTFM Consulting Services Inc. 864 801 2795 voice & voicemail
> > 103 Autumn Hill Road 864 801 2774 fax
> > Greer, SC 29651
> >
> > Do not send me unsolicited commercial email.
>
>

Chuck
Wed Aug 29 11:51:34 CDT 2007

David J. Craig wrote:

> There is a help screen in VMWare on how to do it. Windbg also has comments
> about using a VM target. It isn't that difficult if you read both sources
> and just try it.

Thanks... my WinDbg installation was *stale* and was lacking some info. I
downloaded the latest version [would have needed it, anyway, for the x64
environment] and I see where it discusses the command line switches for
starting up the debugger with a named pipe as the communications method.


--
Chuck Chopp

ChuckChopp (at) rtfmcsi (dot) com http://www.rtfmcsi.com

RTFM Consulting Services Inc. 864 801 2795 voice & voicemail
103 Autumn Hill Road 864 801 2774 fax
Greer, SC 29651

Do not send me unsolicited commercial email.