What does the DDK compiler have that the Visual C++ 2005/8 doesn't?

I was just informed that it is impossible to use WPP in user-mode
applications built using Visual C++ because... "WPP is exclusively DDK
(see MSDN hits)".

Well, the above statement may be not 100% accurate since the following
MSDN link gives some hope that it might be possible to use WPP in
Visual C++ *if* the compiler is replaced by the DDK's:

http://msdn2.microsoft.com/en-us/library/bb500923.aspx

I said "hope" because even that tiny clue is missing some critical
information such as the actual command to execute in order to re-
configure Visual C++.

Assuming that it may be possible to overcome the "chain reaction" of a
whole slew of new problems stemming from re-configuring my
application's build environment to use the DDK compiler, I am now
curious to know:

What does the DDK compiler have that the Visual C++ 2005/2008 doesn't
have?

Is there a comparison table somewhere that would let me figure out
what are the side-effects of substituting one for another?

In particular, what in the WPP makes it usable only via the DDK
compiler? Is it really the *compiler* itself? or is it actually the
DDK environment (libraries, include files, environment variables,
etc.)?

Thanks,
Don

Ben
Wed Mar 19 11:32:40 CDT 2008

> In particular, what in the WPP makes it usable only via the DDK
> compiler? Is it really the *compiler* itself? or is it actually the
> DDK environment (libraries, include files, environment variables,
> etc.)?

I don't think the compiler has anything to do with it. It'll be the header
files defining the macros that are necessary.

But I found this sentence which clearly says WPP works in user-mode:

"For general information about event tracing for Windows, WPP software
tracing for user-mode applications, and the options available with WPP
software tracing, see the Windows SDK documentation."
http://msdn2.microsoft.com/en-us/library/ms793164.aspx

>
> Thanks,
> Don

Ian
Wed Mar 19 11:39:45 CDT 2008

On Wed, 19 Mar 2008 08:20:14 -0700 (PDT), 0dbell@gmail.com wrote:

>I was just informed that it is impossible to use WPP in user-mode
>applications built using Visual C++ because... "WPP is exclusively DDK
>(see MSDN hits)".

WPP is essentially a preprocessor that produces header files and
macros that boil down to Kernel mode APIs eg. WMITraceMessage. This
API does not exist in Win32.

Clearly it is just WMI so it is possible to write code equivalent to
WMITraceMessage etc. Nobody has yet. If you want it you will have to
do it yourself.

>
>Well, the above statement may be not 100% accurate since the following
>MSDN link gives some hope that it might be possible to use WPP in
>Visual C++ *if* the compiler is replaced by the DDK's:
>
>http://msdn2.microsoft.com/en-us/library/bb500923.aspx
>

This incomplete article is referring to compiling WDK under Visual C.

>I said "hope" because even that tiny clue is missing some critical
>information such as the actual command to execute in order to re-
>configure Visual C++.
>
>Assuming that it may be possible to overcome the "chain reaction" of a
>whole slew of new problems stemming from re-configuring my
>application's build environment to use the DDK compiler, I am now
>curious to know:
>
>What does the DDK compiler have that the Visual C++ 2005/2008 doesn't
>have?

Nothing. The compiler used by the WDK is sometimes identical. At the
moment I think Visual Studio is slightly newer version. Sometimes the
WDK version is slightly newer. It is sensible to keep the version of
the compiler with WDK. It is tested to work with it. Mixing tool
chains is hazardous. In the past Microsoft produced a version of
Visual C that could not link VxDs(Window 95 drivers) properly.

>
>Is there a comparison table somewhere that would let me figure out
>what are the side-effects of substituting one for another?
>
>In particular, what in the WPP makes it usable only via the DDK
>compiler? Is it really the *compiler* itself? or is it actually the
>DDK environment (libraries, include files, environment variables,
>etc.)?

WPP still will not work in user mode even with the WDK build
environment. You will recall the first function/macro used was
WPP_INIT_TRACING. Where do you expect to find the DriverObject and
RegistryPath in your WinMain?

0dbell
Wed Mar 19 13:15:42 CDT 2008

On Mar 19, 12:32 pm, "Ben Voigt [C++ MVP]" <r...@nospam.nospam> wrote:
> > In particular, what in the WPP makes it usable only via the DDK
> > compiler? Is it really the *compiler* itself? or is it actually the
> > DDK environment (libraries, include files, environment variables,
> > etc.)?
>
> I don't think the compiler has anything to do with it. It'll be the header
> files defining the macros that are necessary.
>
> But I found this sentence which clearly says WPP works in user-mode:
>
> "For general information about event tracing for Windows, WPP software
> tracing for user-mode applications, and the options available with WPP
> software tracing, see the Windows SDK documentation.
> "http://msdn2.microsoft.com/en-us/library/ms793164.aspx
>

Ben, thanks for your answer. Please note that "works in user-mode" may
mean "works in user-mode *drivers*", not win32 applications. See Ian
Blake's comment about WPP_INIT_TRACING: Where do you expect to find
the DriverObject and RegistryPath in your WinMain?

Don

Don
Wed Mar 19 13:30:11 CDT 2008

You have removed -km from RUN_WPP stuff have you not? Or for that matter
have you looked at the build log for a driver to see what is really invoked,
then do something similar as a command inside of VC++'s build


--
Don Burn (MVP, Windows DDK)
Windows 2k/XP/2k3 Filesystem and Driver Consulting
Website: http://www.windrvr.com
Blog: http://msmvps.com/blogs/WinDrvr
Remove StopSpam to reply


<0dbell@gmail.com> wrote in message
news:b433cc5d-7974-4868-87e1-33c56788c327@u72g2000hsf.googlegroups.com...
> On Mar 19, 12:32 pm, "Ben Voigt [C++ MVP]" <r...@nospam.nospam> wrote:
>> > In particular, what in the WPP makes it usable only via the DDK
>> > compiler? Is it really the *compiler* itself? or is it actually the
>> > DDK environment (libraries, include files, environment variables,
>> > etc.)?
>>
>> I don't think the compiler has anything to do with it. It'll be the
>> header
>> files defining the macros that are necessary.
>>
>> But I found this sentence which clearly says WPP works in user-mode:
>>
>> "For general information about event tracing for Windows, WPP software
>> tracing for user-mode applications, and the options available with WPP
>> software tracing, see the Windows SDK documentation.
>> "http://msdn2.microsoft.com/en-us/library/ms793164.aspx
>>
>
> Ben, thanks for your answer. Please note that "works in user-mode" may
> mean "works in user-mode *drivers*", not win32 applications. See Ian
> Blake's comment about WPP_INIT_TRACING: Where do you expect to find
> the DriverObject and RegistryPath in your WinMain?
>
> Don

0dbell
Wed Mar 19 13:48:12 CDT 2008

Ian, thanks for your detailed and educating answer.

On Mar 19, 12:39 pm, Ian Blake <NoNotMe@NotAnywhere> wrote:
>
> WPP is essentially a preprocessor that produces header files and
> macros that boil down to Kernel mode APIs eg. WMITraceMessage.
> This API does not exist in Win32.
>

Too bad it doesn't exist in Win32. It would have saved me a lot of
time re-inventing the wheel.

> Clearly it is just WMI so it is possible to write code equivalent to
> WMITraceMessage etc. Nobody has yet. If you want it you will have to
> do it yourself.
>

I /will/ do it myself. It is easier (for me) to start a sub-project
like this when I know that it doesn't exist already. :)

>
> Nothing. The compiler used by the WDK is sometimes identical.
> At the moment
> I think Visual Studio is slightly newer version. Sometimes the
> WDK version is slightly newer. It is sensible to keep the version of
> the compiler with WDK. It is tested to work with it. Mixing tool
> chains is hazardous. In the past Microsoft produced a version of
> Visual C that could not link VxDs(Window 95 drivers) properly.
>

Thanks for clarifying this issue.

Don

0dbell
Wed Mar 19 14:02:32 CDT 2008

On Mar 19, 2:30 pm, "Don Burn" <b...@stopspam.windrvr.com> wrote:
> You have removed -km from RUN_WPP stuff have you not? Or for that matter
> have you looked at the build log for a driver to see what is really invoked,
> then do something similar as a command inside of VC++'s build
>

Actually I didn't remove the -km from RUN_WPP (yet). I first wanted to
see how involved is making WPP work in VC++ and whether this will be a
pioneer work.

I could of course look at the tons of lines in a .tmh file and try to
mimick that. But right now this seems like a daunting task. All I
wanted, by using WPP in my application, is to save time in an attempt
to diagnose a performance issue. I didn't know that this could turn
out to be a project by itself. Oh well...

BTW, I read somewhere that there are many samples in the Platform SDK
demonstrating ETW (event tracing). I checked this and found out that
there really is only one sample (TraceDp) that demonstrates the trace
*providing* itself. That sample is extremely simple and does not
demonstrate the entire ETW API.

For example, There is a function called TraceMessageVa(). Except for
that reference page in MSDN (which leaves something to be desired) you
can't find any documentation or example referring to it. Even if you
just type TraceMessageVa in Google...

Thanks,
Don

Don
Wed Mar 19 13:58:31 CDT 2008

Sorry but this is wrong. These do not boil down to WMITraceMessage that is
a routine that is inserted into your driver source by the WPP preprocessor.
Second, there are equivalents for this stuff in user mode, look in the WDK
in the bin\wppconfig\rev1 directory you will see both km* and um* files and
the um files are the ones for user mode. Instead of Driver object you
supply the AppName to WPP_INIT_TRACING.

One of the original purposes of WPP tracing was to help tune SQL which is
certainly a user space application.


--
Don Burn (MVP, Windows DDK)
Windows 2k/XP/2k3 Filesystem and Driver Consulting
Website: http://www.windrvr.com
Blog: http://msmvps.com/blogs/WinDrvr
Remove StopSpam to reply


"Ian Blake" <NoNotMe@NotAnywhere> wrote in message
news:lre2u3hp7m4a2j5dv1au79e03j2srfff4q@4ax.com...
> On Wed, 19 Mar 2008 08:20:14 -0700 (PDT), 0dbell@gmail.com wrote:
>
>>I was just informed that it is impossible to use WPP in user-mode
>>applications built using Visual C++ because... "WPP is exclusively DDK
>>(see MSDN hits)".
>
> WPP is essentially a preprocessor that produces header files and
> macros that boil down to Kernel mode APIs eg. WMITraceMessage. This
> API does not exist in Win32.
>
> Clearly it is just WMI so it is possible to write code equivalent to
> WMITraceMessage etc. Nobody has yet. If you want it you will have to
> do it yourself.
>
>>
>>Well, the above statement may be not 100% accurate since the following
>>MSDN link gives some hope that it might be possible to use WPP in
>>Visual C++ *if* the compiler is replaced by the DDK's:
>>
>>http://msdn2.microsoft.com/en-us/library/bb500923.aspx
>>
>
> This incomplete article is referring to compiling WDK under Visual C.
>
>>I said "hope" because even that tiny clue is missing some critical
>>information such as the actual command to execute in order to re-
>>configure Visual C++.
>>
>>Assuming that it may be possible to overcome the "chain reaction" of a
>>whole slew of new problems stemming from re-configuring my
>>application's build environment to use the DDK compiler, I am now
>>curious to know:
>>
>>What does the DDK compiler have that the Visual C++ 2005/2008 doesn't
>>have?
>
> Nothing. The compiler used by the WDK is sometimes identical. At the
> moment I think Visual Studio is slightly newer version. Sometimes the
> WDK version is slightly newer. It is sensible to keep the version of
> the compiler with WDK. It is tested to work with it. Mixing tool
> chains is hazardous. In the past Microsoft produced a version of
> Visual C that could not link VxDs(Window 95 drivers) properly.
>
>>
>>Is there a comparison table somewhere that would let me figure out
>>what are the side-effects of substituting one for another?
>>
>>In particular, what in the WPP makes it usable only via the DDK
>>compiler? Is it really the *compiler* itself? or is it actually the
>>DDK environment (libraries, include files, environment variables,
>>etc.)?
>
> WPP still will not work in user mode even with the WDK build
> environment. You will recall the first function/macro used was
> WPP_INIT_TRACING. Where do you expect to find the DriverObject and
> RegistryPath in your WinMain?
>
>

Ben
Wed Mar 19 14:18:43 CDT 2008

>> But I found this sentence which clearly says WPP works in user-mode:
>>
>> "For general information about event tracing for Windows, WPP
>> software tracing for user-mode applications, and the options
>> available with WPP software tracing, see the Windows SDK
>> documentation.
>> "http://msdn2.microsoft.com/en-us/library/ms793164.aspx
>>
>
> Ben, thanks for your answer. Please note that "works in user-mode" may
> mean "works in user-mode *drivers*", not win32 applications. See Ian

I don't understand how you can interpret "WPP software tracing for user-mode
applications" to mean "user-mode drivers, not win32 applications".

0dbell
Wed Mar 19 14:47:29 CDT 2008

On Mar 19, 2:18=A0pm, "Ben Voigt [C++ MVP]" <r...@nospam.nospam> wrote:
>
> I don't understand how you can interpret "WPP software tracing for user-mo=
de
> applications" to mean "user-mode drivers, not win32 applications".
>

You are correct - the MSDN link that you brought indeed uses the word
"applications".

However, we all know that from time to time there may be good
intentions in documentation that are not always fullfilled in real-
life. For example, MSDN says that TraceEventInstance() is supported in
Windows XP, but Microsoft's dev team confirmed that TraceEventInstance
doesn't work correctly on XP:

http://tinyurl.com/yt8wqf

There may also be typos, just as in the link I mentioned in my first
post:

http://msdn2.microsoft.com/en-us/library/bb500923.aspx

That said, after reading Don Burn's reply I now believe that it is
possible to use WPP in user-mode applications, although there is no
trace (pun intended) that anyone has ever done so (except for the SQL
dev team, of course, but that doesn't help me much). :-)

Thanks,
Don

Ivan
Wed Mar 19 16:47:40 CDT 2008

Can you give a shot at those files ?


//------------------------------------------------------------- sources

TARGETNAME=testWPP
TARGETPATH=obj
TARGETTYPE=PROGRAM

BUFFER_OVERFLOW_CHECKS=NTDLL

TARGETLIBS=\
$(SDK_LIB_PATH)\ntdll.lib \
$(SDK_LIB_PATH)\kernel32.lib

_NT_TARGET_VERSION = $(_NT_TARGET_VERSION_WINXP)

USE_MSVCRT=1
USE_NATIVE_EH=ASYNC
USE_STL=1
STL_VER=70

UMENTRY=wmain
UMTYPE=console

INCLUDES=\
$(DDK_INC_PATH);

MSC_WARNING_LEVEL=/W3 /WX

C_DEFINES=$(C_DEFINES) -DUNICODE -D_UNICODE

SOURCES=\
testWPP.cpp

RUN_WPP=\
$(SOURCES)\
-func:TestWPPLog(LEVEL,MSG,...)

//------------------------------------------------------------- trace.h

#ifndef __trace_h__
#define __trace_h__

#define WPP_CONTROL_GUIDS \
WPP_DEFINE_CONTROL_GUID(TestWPPGuid,(11223344, AAAA, BBBB, CCCC,
DDFFFF112233), \
WPP_DEFINE_BIT(Error) \
WPP_DEFINE_BIT(Verbose))

// begin_wpp config
//
// FUNC TestWPPLog((FLAGS,LEVEL,MSG,...));
//
// end_wpp

#endif /* __trace_h__ */

//------------------------------------------------------------- testwpp.cpp


#include <nt.h>
#include <ntrtl.h>
#include <nturtl.h>
#define WIN32_STATUS
#include <windows.h>

#include <stdio.h>
#include <stdlib.h>
#include <strsafe.h>
#include <memory.h>

#include <memory>
#include <map>

#include<trace.h>

#include "testWPP.tmh"

class TestClass {
private:
BYTE Array[256];
public:
TestClass();
virtual ~TestClass();
};

TestClass::TestClass(){
TestWPPLog(Verbose,"%s",__FUNCTION__);
}

TestClass::~TestClass(){
TestWPPLog(Error,"%s",__FUNCTION__);
}


int __cdecl
wmain( __in int argc,
__in_ecount(argc) WCHAR ** argv)
/*++

--*/
{
WPP_INIT_TRACING(L"TestWPP");

TestClass * p = new TestClass;

delete p;

WPP_CLEANUP();

return 0;
}


--

--
This posting is provided "AS IS" with no warranties, and confers no rights.
Use of any included script samples are subject to the terms specified at
http://www.microsoft.com/info/cpyright.htm


<0dbell@gmail.com> wrote in message
news:e973f3f6-56bc-4068-9d05-e8374b8aa482@e60g2000hsh.googlegroups.com...
> Ian, thanks for your detailed and educating answer.
>
> On Mar 19, 12:39 pm, Ian Blake <NoNotMe@NotAnywhere> wrote:
>>
>> WPP is essentially a preprocessor that produces header files and
>> macros that boil down to Kernel mode APIs eg. WMITraceMessage.
>> This API does not exist in Win32.
>>
>
> Too bad it doesn't exist in Win32. It would have saved me a lot of
> time re-inventing the wheel.
>
>> Clearly it is just WMI so it is possible to write code equivalent to
>> WMITraceMessage etc. Nobody has yet. If you want it you will have to
>> do it yourself.
>>
>
> I /will/ do it myself. It is easier (for me) to start a sub-project
> like this when I know that it doesn't exist already. :)
>
>>
>> Nothing. The compiler used by the WDK is sometimes identical.
>> At the moment
>> I think Visual Studio is slightly newer version. Sometimes the
>> WDK version is slightly newer. It is sensible to keep the version of
>> the compiler with WDK. It is tested to work with it. Mixing tool
>> chains is hazardous. In the past Microsoft produced a version of
>> Visual C that could not link VxDs(Window 95 drivers) properly.
>>
>
> Thanks for clarifying this issue.
>
> Don

0dbell
Wed Mar 19 20:28:34 CDT 2008

On Mar 19, 4:47=A0pm, "Ivan Brugiolo [MSFT]"
<ivanb...@online.microsoft.com> wrote:
> Can you give a shot at those files ?
>

Ivan, thank you very much for your help. I believe that your example
will work - if the DDK is installed.

In my user-mode development environment the DDK (WDK) is not
installed. Two different machines are used for driver and application
development: one machine has WDK only, the other machine has PSDK
only.

I search the PSDK and found no trace of WPP_INIT_TRACING. Thus, I must
conclude that the DDK must be installed in order for your sample to
work.

At the moment I decided to work directly with raw ETW for my
application (mainly for project schedule reasons).

Thanks,
Don

Ivan
Wed Mar 19 20:59:47 CDT 2008

Yes, the example assumes the use of DDK/WDK.
I'm not aware of a PSDK only version of WPP.
If your development target were Vista, I would recomend
the use of the manifest compiler (mc.exe) to digest a Crimson-Event
manifest,
that will generate easy-to-use instrumentation macros that you
can scatter in your code in the same way you would do for WPP-tracing
macros.

Etw-WPP and Etw-Crimson events are the two mainstreams way of using Etw
in both user and kernel-mode, and, they are here to stay.

One other way to instrument the code in a WinXp+ compatible way
is to use Etw-Classic. The usage pattern wou