All,
Everytime our site is accessed the CPU usage for the w3wp.exe process climbs
to 95%+ and stays high for a long while, making the site extremely slow to
load and almost unusable.
The server uses W2K3 (SP1) and IIS6.
The site was built using Microsoft Content Management Server 2002 (SP1a) and
ASP.Net
The IIS settings are the default ones.
I've run DebugDiag and iisstate, but couldn't make much sense of the log file.
Could someone please help and point to the problem? The iisstate log file is
included below.
Thanks,
Ziad
-------------------------
Opened log file 'C:\iisstate\output\IISState-3916.log'
***********************
Starting new log output
IISState version 3.4
Wed Mar 15 13:20:21 2006
OS = Windows 2003 Server
Executable: w3wp.exe
PID = 3916
Note: Thread times are formatted as HH:MM:SS.ms
***********************
Thread ID: 0
System Thread ID: 77c
Kernel Time: 0:0:0.62
User Time: 0:0:0.62
Thread Status: Thread is in a WAIT state.
Thread Type: HTTP Compression Thread
# ChildEBP RetAddr
00 0006fc08 7c822124 ntdll!KiFastSystemCallRet
01 0006fc0c 77e6baa8 ntdll!NtWaitForSingleObject+0xc
02 0006fc7c 77e6ba12 kernel32!WaitForSingleObjectEx+0xac
03 0006fc90 5a36467a kernel32!WaitForSingleObject+0x12
04 0006fca0 5a366e63 w3dt!WP_CONTEXT::RunMainThreadLoop+0x10
05 0006fca8 5a3af41d w3dt!UlAtqStartListen+0x2d
06 0006fcb8 5a3bc259 w3core!W3_SERVER::StartListen+0xbd
07 0006ff0c 0100187c w3core!UlW3Start+0x26e
08 0006ff44 01001a23 w3wp!wmain+0x22a
09 0006ffc0 77e523cd w3wp!wmainCRTStartup+0x12b
0a 0006fff0 00000000 kernel32!BaseProcessStart+0x23
Thread ID: 1
System Thread ID: a68
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Status: Thread is in a WAIT state.
Thread Type: Other
# ChildEBP RetAddr
00 0097fea0 7c822114 ntdll!KiFastSystemCallRet
01 0097fea4 7c80e504 ntdll!NtWaitForMultipleObjects+0xc
02 0097ff48 7c80e933 ntdll!EtwpWaitForMultipleObjectsEx+0xf7
03 0097ffb8 77e66063 ntdll!EtwpEventPump+0x27f
04 0097ffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 2
System Thread ID: 2bc
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Type: Other
# ChildEBP RetAddr
00 009bff9c 7c821364 ntdll!KiFastSystemCallRet
01 009bffa0 7c81fe26 ntdll!NtDelayExecution+0xc
02 009bffb8 77e66063 ntdll!RtlpTimerThread+0x47
03 009bffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 3
System Thread ID: 298
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Type: Other
# ChildEBP RetAddr
00 009fff70 7c821bf4 ntdll!KiFastSystemCallRet
01 009fff74 7c83ad75 ntdll!NtRemoveIoCompletion+0xc
02 009fffb8 77e66063 ntdll!RtlpWorkerThread+0x3d
03 009fffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 4
System Thread ID: f40
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Type: Possible ASP page. Possible DCOM activity
Executing Page: Unable to locate ASP page
No remote call being made
# ChildEBP RetAddr
00 00a5fe18 7c821c54 ntdll!KiFastSystemCallRet
01 00a5fe1c 77c7538c ntdll!ZwReplyWaitReceivePortEx+0xc
02 00a5ff84 77c5778f RPCRT4!LRPC_ADDRESS::ReceiveLotsaCalls+0x198
03 00a5ff8c 77c5f7dd RPCRT4!RecvLotsaCallsWrapper+0xd
04 00a5ffac 77c5de88 RPCRT4!BaseCachedThreadRoutine+0x9d
05 00a5ffb8 77e66063 RPCRT4!ThreadStartRoutine+0x1b
06 00a5ffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 5
System Thread ID: fd4
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Type: Other
# ChildEBP RetAddr
00 00a9ff10 7c821364 ntdll!KiFastSystemCallRet
01 00a9ff14 77e42439 ntdll!NtDelayExecution+0xc
02 00a9ff7c 77e424b7 kernel32!SleepEx+0x68
03 00a9ff8c 776b22a0 kernel32!Sleep+0xf
04 00a9ff98 776b2307 ole32!CROIDTable::WorkerThreadLoop+0x14
05 00a9ffa8 77670000 ole32!CRpcThread::WorkerLoop+0x1e
06 00a9ffb8 77e66063 ole32!_imp__InstallApplication <PERF> (ole32+0x0)
07 00a9ffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 6
System Thread ID: dbc
Kernel Time: 0:0:0.15
User Time: 0:0:0.46
Thread Type: HTTP Listener
# ChildEBP RetAddr
00 00c5ff24 7c821bf4 ntdll!KiFastSystemCallRet
01 00c5ff28 77e6611a ntdll!NtRemoveIoCompletion+0xc
02 00c5ff54 5a30249e kernel32!GetQueuedCompletionStatus+0x29
03 00c5ff8c 5a3026bc W3TP!THREAD_POOL_DATA::ThreadPoolThread+0x33
04 00c5ffa0 5a301db9 W3TP!THREAD_POOL_DATA::ThreadPoolThread+0x24
05 00c5ffb8 77e66063 W3TP!THREAD_MANAGER::ThreadManagerThread+0x39
06 00c5ffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 7
System Thread ID: 860
Kernel Time: 0:0:0.31
User Time: 0:0:0.0
Thread Type: HTTP Listener
# ChildEBP RetAddr
00 00c9ff24 7c821bf4 ntdll!KiFastSystemCallRet
01 00c9ff28 77e6611a ntdll!NtRemoveIoCompletion+0xc
02 00c9ff54 5a30249e kernel32!GetQueuedCompletionStatus+0x29
03 00c9ff8c 5a3026bc W3TP!THREAD_POOL_DATA::ThreadPoolThread+0x33
04 00c9ffa0 5a301db9 W3TP!THREAD_POOL_DATA::ThreadPoolThread+0x24
05 00c9ffb8 77e66063 W3TP!THREAD_MANAGER::ThreadManagerThread+0x39
06 00c9ffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 8
System Thread ID: fdc
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Type: HTTP Listener
# ChildEBP RetAddr
00 00cdff24 7c821bf4 ntdll!KiFastSystemCallRet
01 00cdff28 77e6611a ntdll!NtRemoveIoCompletion+0xc
02 00cdff54 5a30249e kernel32!GetQueuedCompletionStatus+0x29
03 00cdff8c 5a3026bc W3TP!THREAD_POOL_DATA::ThreadPoolThread+0x33
04 00cdffa0 5a301db9 W3TP!THREAD_POOL_DATA::ThreadPoolThread+0x24
05 00cdffb8 77e66063 W3TP!THREAD_MANAGER::ThreadManagerThread+0x39
06 00cdffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 9
System Thread ID: cb4
Kernel Time: 0:0:0.171
User Time: 0:0:0.171
Thread Type: HTTP Listener
# ChildEBP RetAddr
00 00d1ff24 7c821bf4 ntdll!KiFastSystemCallRet
01 00d1ff28 77e6611a ntdll!NtRemoveIoCompletion+0xc
02 00d1ff54 5a30249e kernel32!GetQueuedCompletionStatus+0x29
03 00d1ff8c 5a3026bc W3TP!THREAD_POOL_DATA::ThreadPoolThread+0x33
04 00d1ffa0 5a301db9 W3TP!THREAD_POOL_DATA::ThreadPoolThread+0x24
05 00d1ffb8 77e66063 W3TP!THREAD_MANAGER::ThreadManagerThread+0x39
06 00d1ffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 10
System Thread ID: f34
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Status: Thread is in a WAIT state.
Thread Type: Other
# ChildEBP RetAddr
00 00ddfcec 7c822114 ntdll!KiFastSystemCallRet
01 00ddfcf0 7c83acfd ntdll!NtWaitForMultipleObjects+0xc
02 00ddffb8 77e66063 ntdll!RtlpWaitThread+0x161
03 00ddffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 11
System Thread ID: 460
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Status: Thread is in a WAIT state.
Thread Type: HTTP Compression Thread
# ChildEBP RetAddr
00 0181fa84 7c822124 ntdll!KiFastSystemCallRet
01 0181fa88 77e6baa8 ntdll!NtWaitForSingleObject+0xc
02 0181faf8 77e6ba12 kernel32!WaitForSingleObjectEx+0xac
03 0181fb0c 5a3b8147 kernel32!WaitForSingleObject+0x12
04 0181ffb8 77e66063 w3core!HTTP_COMPRESSION::CompressionThread+0x126
05 0181ffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 12
System Thread ID: fc4
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Type: Possible ASP page. Possible DCOM activity
Executing Page: Unable to locate ASP page
No remote call being made
# ChildEBP RetAddr
00 0185fe18 7c821c54 ntdll!KiFastSystemCallRet
01 0185fe1c 77c7538c ntdll!ZwReplyWaitReceivePortEx+0xc
02 0185ff84 77c5778f RPCRT4!LRPC_ADDRESS::ReceiveLotsaCalls+0x198
03 0185ff8c 77c5f7dd RPCRT4!RecvLotsaCallsWrapper+0xd
04 0185ffac 77c5de88 RPCRT4!BaseCachedThreadRoutine+0x9d
05 0185ffb8 77e66063 RPCRT4!ThreadStartRoutine+0x1b
06 0185ffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 13
System Thread ID: a54
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Type: Other
# ChildEBP RetAddr
00 0189ff8c 7c821364 ntdll!KiFastSystemCallRet
01 0189ff90 7c815267 ntdll!NtDelayExecution+0xc
02 0189ffb8 77e66063 ntdll!RtlpIOWorkerThread+0x3f
03 0189ffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 14
System Thread ID: e10
Kernel Time: 0:0:0.0
User Time: 0:0:0.15
Thread Status: Thread is in a WAIT state.
Thread Type: Idle ASP thread
# ChildEBP RetAddr
00 01e3fd00 7c822124 ntdll!KiFastSystemCallRet
01 01e3fd04 77e6baa8 ntdll!NtWaitForSingleObject+0xc
02 01e3fd74 77e6ba12 kernel32!WaitForSingleObjectEx+0xac
03 01e3fd88 75bb22b9 kernel32!WaitForSingleObject+0x12
04 01e3ffb8 77e66063 comsvcs!PingThread+0xf6
05 01e3ffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 15
System Thread ID: f78
Kernel Time: 0:0:23.515
User Time: 0:0:18.796
Thread Type: Other
# ChildEBP RetAddr
00 01eaf578 7c821204 ntdll!KiFastSystemCallRet
01 01eaf57c 77e6c3d1 ntdll!ZwCreateFile+0xc
02 01eaf618 77d0ed0c kernel32!CreateFileW+0x377
03 01eaf644 77d0ebdd OLEAUT32!ErrOpen+0x2a
04 01eaf660 77d0e452 OLEAUT32!FindTypeLib+0x51
05 01eaf708 77d0e601 OLEAUT32!LoadTypeLibEx+0xe7
06 01eaf71c 77d178af OLEAUT32!LoadTypeLib+0x12
07 01eafafc 77d17591 OLEAUT32!GetTypeInfoOfIID+0x371
08 01eafb1c 77788ec1 OLEAUT32!CUnivStubWrapper::Invoke+0x7c
09 01eafb60 77788e74 ole32!SyncStubInvoke+0x33
0a 01eafba8 7769b2b7 ole32!StubInvoke+0xa7
0b 01eafc80 7769b0b5 ole32!CCtxComChnl::ContextInvoke+0xe3
0c 01eafc9c 7769bd23 ole32!MTAInvoke+0x1a
0d 01eafcc8 77788da2 ole32!STAInvoke+0x48
0e 01eafcfc 77788c33 ole32!AppInvoke+0xa3
0f 01eafdd0 777890b3 ole32!ComInvokeWithLockAndIPID+0x2c5
10 01eafdf8 7769bc9c ole32!ComInvoke+0x9f
11 01eafe0c 7769bac0 ole32!ThreadDispatch+0x23
12 01eafe24 7739c3b7 ole32!ThreadWndProc+0xfe
13 01eafe50 7739c484 USER32!InternalCallWinProc+0x28
14 01eafec8 7739c73c USER32!UserCallWinProcCheckWow+0x151
15 01eaff30 7739c778 USER32!DispatchMessageWorker+0x327
16 01eaff40 7769501a USER32!DispatchMessageW+0xf
17 01eaff74 776cf35b ole32!CDllHost::STAWorkerLoop+0x5c
18 01eaff90 776cf2a3 ole32!CDllHost::WorkerThread+0xc8
19 01eaff98 776b2307 ole32!DLLHostThreadEntry+0xd
1a 01eaffac 776b2374 ole32!CRpcThread::WorkerLoop+0x1e
1b 01eaffb8 77e66063 ole32!CRpcThreadCache::RpcWorkerThreadEntry+0x20
1c 01eaffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 16
System Thread ID: f94
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Type: Managed Thread. Possible ASP.Net page or other .Net worker
succeeded
Loaded Son of Strike data table version 5 from
"C:\WINDOWS\Microsoft.NET\Framework\v1.1.4322\mscorwks.dll"
Thread 16
Not a managed thread.
Begin System Thread Information
# ChildEBP RetAddr
00 0222fec0 7c821bf4 ntdll!KiFastSystemCallRet
01 0222fec4 77e6611a ntdll!NtRemoveIoCompletion+0xc
02 0222fef0 792edc4b kernel32!GetQueuedCompletionStatus+0x29
03 0222ff24 792eccf3 mscorwks!ThreadpoolMgr::CompletionPortThreadStart+0x49
04 0222ffb8 77e66063 mscorwks!ThreadpoolMgr::intermediateThreadProc+0x44
05 0222ffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 17
System Thread ID: c9c
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Type: Managed Thread. Possible ASP.Net page or other .Net worker
Thread 17
Not a managed thread.
Begin System Thread Information
# ChildEBP RetAddr
00 0226fee0 7c821364 ntdll!KiFastSystemCallRet
01 0226fee4 77e42439 ntdll!NtDelayExecution+0xc
02 0226ff4c 77e424b7 kernel32!SleepEx+0x68
03 0226ff5c 792edf9e kernel32!Sleep+0xf
04 0226ffb8 77e66063 mscorwks!ThreadpoolMgr::GateThreadStart+0x54
05 0226ffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 18
System Thread ID: 848
Kernel Time: 0:0:27.953
User Time: 0:0:25.31
*** ERROR: Symbol file could not be found. Defaulted to export symbols for
C:\Program Files\Microsoft Content Management
Server\Server\bin\ResolutionObjectModel.dll -
Thread Status: Thread is in a WAIT state.
Thread Type: Possible ASP page. Possible DCOM activity
Executing Page: Unable to locate ASP page
DCOM call being made to Process ID: 19
Waiting on thread id: 26
# ChildEBP RetAddr
00 022ae7cc 7c822124 ntdll!KiFastSystemCallRet
01 022ae7d0 77e6baa8 ntdll!NtWaitForSingleObject+0xc
02 022ae840 77e6ba12 kernel32!WaitForSingleObjectEx+0xac
03 022ae854 776c54a7 kernel32!WaitForSingleObject+0x12
04 022ae870 77789905 ole32!GetToSTA+0x6f
05 022ae890 77787ed7 ole32!CRpcChannelBuffer::SwitchAptAndDispatchCall+0xcb
06 022ae970 77695349 ole32!CRpcChannelBuffer::SendReceive2+0xc1
07 022ae9dc 776952e1 ole32!CAptRpcChnl::SendReceive+0xab
08 022aea30 77ce127e ole32!CCtxComChnl::SendReceive+0x113
09 022aea4c 77ce13ca RPCRT4!NdrProxySendReceive+0x43
0a 022aee34 77ce11bd RPCRT4!NdrClientCall2+0x206
0b 022aee54 77c53a12 RPCRT4!ObjectStublessClient+0x8b
0c 022aee64 368243a0 RPCRT4!ObjectStubless+0xf
WARNING: Stack unwind informatio