I'm running Win2003 Server, with IIS 6.0.
I have an ASP.NET app that simply hangs (stops serving pages), with no error
and no errors in the event logs.
The only solution so far is to 'recycle' the DefaultApp Pool about every
hour.
The application is moderately used, about 20 logins/hour.
I've run both IISState and the CHAgent.
Neither seem to provide the information i'm looking for ... a cause, at
least a hint to where the problem might be.
I've included both the iisstate log file and the chagent log file (snipped).
I'm really hoping someone can lead me in the right direction
much thanks...
-bill
---------
Crash-041004-123843.log
---------
#Fields: date time src-ip src-port dst-ip dst-port cs-version cs-method
cs-url sc-status s-site s-reason
2004-10-04 19:21:39 132.239.3.83 3051 132.239.3.83 443 HTTP/1.1 GET
/MyHealth/Login.aspx?ReturnUrl=%2fMyHealth%2fDefault.aspx - 1
Connection_Abandoned_By_Inetinfo_Crash
2004-10-04 19:20:34 132.239.52.153 1242 132.239.3.83 443 HTTP/1.1 GET
/MyHealth/Login.aspx?ReturnUrl=%2fMyHealth%2fDefault.aspx - 1
Connection_Abandoned_By_Inetinfo_Crash
2004-10-04 19:20:34 132.239.52.153 1242 132.239.3.83 443 HTTP/1.1 GET
/MyHealth/Login.aspx?ReturnUrl=%2fMyHealth%2fDefault.aspx - 1
Connection_Abandoned_By_Inetinfo_Crash
2004-10-04 19:20:34 132.239.3.83 3029 132.239.3.83 443 HTTP/1.1 GET
/MyHealth/Login.aspx?ReturnUrl=%2fMyHealth%2fDefault.aspx - 1
Connection_Abandoned_By_Inetinfo_Crash
2004-10-04 19:20:34 132.239.3.83 3029 132.239.3.83 443 HTTP/1.1 GET
/MyHealth/Login.aspx?ReturnUrl=%2fMyHealth%2fDefault.aspx - 1
Connection_Abandoned_By_Inetinfo_Crash
2004-10-04 19:36:18 132.239.3.55 1253 132.239.3.83 443 HTTP/1.1 GET
/MyHealth/ 302 1 Request_Completed
2004-10-04 19:36:18 132.239.3.55 1253 132.239.3.83 443 HTTP/1.1 GET
/MyHealth/ 302 1 Request_Completed
2004-10-04 19:36:16 132.239.3.55 1252 132.239.3.83 80 HTTP/1.1 GET
/Pages/OnlineAppt.asp 200 1 Request_Completed
2004-10-04 19:36:16 132.239.3.55 1252 132.239.3.83 80 HTTP/1.1 GET
/Pages/OnlineAppt.asp 200 1 Request_Completed
2004-10-04 19:36:16 132.239.3.55 1249 132.239.3.83 80 HTTP/1.1 GET
/images_struct/ucsd_official.gif 200 1
---------
IISState-3616.log
---------
Opened log file 'C:\iisstate\output\IISState-3616.log'
***********************
Starting new log output
IISState version 3.3.1
Mon Oct 04 12:58:00 2004
OS = Windows 2003 Server
Executable: inetinfo.exe
PID = 3616
Note: Thread times are formatted as HH:MM:SS.ms
***********************
Thread ID: 0
System Thread ID: 47c
Kernel Time: 0:0:0.31
User Time: 0:0:0.0
Thread Type: Other
# ChildEBP RetAddr
00 0006f9a8 77f4303b SharedUserData!SystemCallStub+0x4
01 0006f9ac 77e4905d ntdll!NtReadFile+0xc
02 0006fa14 77db51f1 kernel32!ReadFile+0x16c
03 0006fa40 77db5297 ADVAPI32!ScGetPipeInput+0x28
04 0006fab0 77dfa7f1 ADVAPI32!ScDispatcherLoop+0x4c
05 0006fcec 01002655 ADVAPI32!StartServiceCtrlDispatcherA+0x91
06 0006fe1c 010027ea inetinfo!StartDispatchTable+0x214
07 0006ff44 01003160 inetinfo!main+0x104
08 0006ffc0 77e4f38c inetinfo!mainCRTStartup+0x12f
09 0006fff0 00000000 kernel32!BaseProcessStart+0x23
Thread ID: 1
System Thread ID: 6f8
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Type: Other
# ChildEBP RetAddr
00 0083fe18 77f43741 SharedUserData!SystemCallStub+0x4
01 0083fe1c 77e41817 ntdll!ZwWaitForSingleObject+0xc
02 0083fe8c 77e4168f kernel32!WaitForSingleObjectEx+0xac
03 0083fe9c 01002cf9 kernel32!WaitForSingleObject+0xf
04 0083ffb8 77e4a990 inetinfo!W3SVCThreadEntry+0x3b
05 0083ffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 2
System Thread ID: 86c
Kernel Time: 0:0:0.46
User Time: 0:0:0.265
Thread Type: Other
# ChildEBP RetAddr
00 0087fcc4 77f43741 SharedUserData!SystemCallStub+0x4
01 0087fcc8 77e41817 ntdll!ZwWaitForSingleObject+0xc
02 0087fd38 77e4168f kernel32!WaitForSingleObjectEx+0xac
03 0087fd48 649f24ac kernel32!WaitForSingleObject+0xf
04 0087fd70 010023b6 iisadmin!ServiceEntry+0x214
05 0087ffa8 77db571b inetinfo!InetinfoStartService+0x2a6
06 0087ffb8 77e4a990 ADVAPI32!ScSvcctrlThreadA+0xe
07 0087ffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 3
System Thread ID: ff8
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Type: Other
# ChildEBP RetAddr
00 00a7ff9c 77f4262b SharedUserData!SystemCallStub+0x4
01 00a7ffa0 77f6b5b2 ntdll!NtDelayExecution+0xc
02 00a7ffb8 77e4a990 ntdll!RtlpTimerThread+0x45
03 00a7ffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 4
System Thread ID: 758
Kernel Time: 0:0:0.0
User Time: 0:0:0.15
Thread Status: Thread is in a WAIT state.
Thread Type: Other
# ChildEBP RetAddr
00 00b8feb0 77f4372d SharedUserData!SystemCallStub+0x4
01 00b8feb4 77e41bfa ntdll!NtWaitForMultipleObjects+0xc
02 00b8ff5c 77e4b0e4 kernel32!WaitForMultipleObjectsEx+0x11a
03 00b8ff74 6e0b377a kernel32!WaitForMultipleObjects+0x17
04 00b8ffa0 6e0b6012 COADMIN!NOTIFY_CONTEXT::GetNextContext+0x68
05 00b8ffb8 77e4a990 COADMIN!NOTIFY_CONTEXT::NotifyThreadProc+0x62
06 00b8ffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 5
System Thread ID: e84
Kernel Time: 0:0:0.296
User Time: 0:0:0.703
Thread Type: Possible ASP page. Possible DCOM activity
Executing Page: ASP.dll symbols not found. Unable to locate ASP page.
Continuing with other analysis.
No remote call being made
# ChildEBP RetAddr
00 0144fe20 77f4313f SharedUserData!SystemCallStub+0x4
01 0144fe24 77c57b85 ntdll!NtReplyWaitReceivePortEx+0xc
02 0144ff8c 77c60829 RPCRT4!LRPC_ADDRESS::ReceiveLotsaCalls+0x193
03 0144ff90 77c60771 RPCRT4!RecvLotsaCallsWrapper+0x9
04 0144ffb0 77c60857 RPCRT4!BaseCachedThreadRoutine+0x9c
05 0144ffb8 77e4a990 RPCRT4!ThreadStartRoutine+0x17
06 0144ffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 6
System Thread ID: bec
Kernel Time: 0:0:0.93
User Time: 0:0:0.62
Thread Status: Thread is in a WAIT state.
Thread Type: Other
# ChildEBP RetAddr
00 0150fbc0 77f4372d SharedUserData!SystemCallStub+0x4
01 0150fbc4 77e41bfa ntdll!NtWaitForMultipleObjects+0xc
02 0150fc6c 77d076f5 kernel32!WaitForMultipleObjectsEx+0x11a
03 0150fcc8 77d077f5 USER32!RealMsgWaitForMultipleObjectsEx+0x13f
04 0150fce4 643f5723 USER32!MsgWaitForMultipleObjects+0x1d
05 0150fd30 6930d973 INFOCOMM!IIS_SERVICE::StartServiceOperation+0x22f
06 0150fd70 010023b6 ftpsvc2!ServiceEntry+0xac
07 0150ffa8 77db571b inetinfo!InetinfoStartService+0x2a6
08 0150ffb8 77e4a990 ADVAPI32!ScSvcctrlThreadA+0xe
09 0150ffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 7
System Thread ID: a38
Kernel Time: 0:0:0.234
User Time: 0:0:0.546
Thread Type: Possible ASP page. Possible DCOM activity
Executing Page: ASP.dll symbols not found. Unable to locate ASP page.
Continuing with other analysis.
No remote call being made
# ChildEBP RetAddr
00 016efe20 77f4313f SharedUserData!SystemCallStub+0x4
01 016efe24 77c57b85 ntdll!NtReplyWaitReceivePortEx+0xc
02 016eff8c 77c60829 RPCRT4!LRPC_ADDRESS::ReceiveLotsaCalls+0x193
03 016eff90 77c60771 RPCRT4!RecvLotsaCallsWrapper+0x9
04 016effb0 77c60857 RPCRT4!BaseCachedThreadRoutine+0x9c
05 016effb8 77e4a990 RPCRT4!ThreadStartRoutine+0x17
06 016effec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 8
System Thread ID: 770
Kernel Time: 0:0:0.78
User Time: 0:0:0.93
Thread Type: HTTP Listener
# ChildEBP RetAddr
00 017aff50 77f430c7 SharedUserData!SystemCallStub+0x4
01 017aff54 77e430bc ntdll!ZwRemoveIoCompletion+0xc
02 017aff80 63ec7235 kernel32!GetQueuedCompletionStatus+0x27
03 017affb8 77e4a990 ISATQ!AtqPoolThread+0x40
04 017affec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 9
System Thread ID: ea4
Kernel Time: 0:0:0.62
User Time: 0:0:0.0
Thread Type: HTTP Listener
# ChildEBP RetAddr
00 017eff50 77f430c7 SharedUserData!SystemCallStub+0x4
01 017eff54 77e430bc ntdll!ZwRemoveIoCompletion+0xc
02 017eff80 63ec7235 kernel32!GetQueuedCompletionStatus+0x27
03 017effb8 77e4a990 ISATQ!AtqPoolThread+0x40
04 017effec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 10
System Thread ID: 920
Kernel Time: 0:0:0.31
User Time: 0:0:0.0
Thread Status: Thread is in a WAIT state.
Thread Type: Other
# ChildEBP RetAddr
00 0182fcec 77f4372d SharedUserData!SystemCallStub+0x4
01 0182fcf0 77f75297 ntdll!NtWaitForMultipleObjects+0xc
02 0182ffb8 77e4a990 ntdll!RtlpWaitThread+0x158
03 0182ffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 11
System Thread ID: c40
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Type: Possible ASP page. Possible DCOM activity
Executing Page: ASP.dll symbols not found. Unable to locate ASP page.
Continuing with other analysis.
No remote call being made
# ChildEBP RetAddr
00 01c2fe20 77f4313f SharedUserData!SystemCallStub+0x4
01 01c2fe24 77c57b85 ntdll!NtReplyWaitReceivePortEx+0xc
02 01c2ff8c 77c60829 RPCRT4!LRPC_ADDRESS::ReceiveLotsaCalls+0x193
03 01c2ff90 77c60771 RPCRT4!RecvLotsaCallsWrapper+0x9
04 01c2ffb0 77c60857 RPCRT4!BaseCachedThreadRoutine+0x9c
05 01c2ffb8 77e4a990 RPCRT4!ThreadStartRoutine+0x17
06 01c2ffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 12
System Thread ID: eb0
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Type: Possible ASP page. Possible DCOM activity
Executing Page: ASP.dll symbols not found. Unable to locate ASP page.
Continuing with other analysis.
No remote call being made
# ChildEBP RetAddr
00 01c6feb4 77f430c7 SharedUserData!SystemCallStub+0x4
01 01c6feb8 77e430bc ntdll!ZwRemoveIoCompletion+0xc
02 01c6fee4 77c80bd1 kernel32!GetQueuedCompletionStatus+0x27
03 01c6ff20 77c80a78 RPCRT4!COMMON_ProcessCalls+0x9f
04 01c6ff8c 77c58159 RPCRT4!LOADABLE_TRANSPORT::ProcessIOEvents+0x115
05 01c6ff90 77c60771 RPCRT4!ProcessIOEventsWrapper+0x9
06 01c6ffb0 77c60857 RPCRT4!BaseCachedThreadRoutine+0x9c
07 01c6ffb8 77e4a990 RPCRT4!ThreadStartRoutine+0x17
08 01c6ffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 13
System Thread ID: 6e0
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Status: Thread is in a WAIT state.
Thread Type: Other
# ChildEBP RetAddr
00 01cafecc 77f4372d SharedUserData!SystemCallStub+0x4
01 01cafed0 77e41bfa ntdll!NtWaitForMultipleObjects+0xc
02 01caff78 6930882e kernel32!WaitForMultipleObjectsEx+0x11a
03 01caffb0 77f4308b ftpsvc2!PASV_ACCEPT_CONTEXT::AcceptThreadFunc+0x32
04 01caffb8 77e4a990 ntdll!NtRegisterThreadTerminatePort+0xc
05 01caffc4 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 14
System Thread ID: bc4
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Type: HTTP Listener
# ChildEBP RetAddr
00 01d2fe38 77f43741 SharedUserData!SystemCallStub+0x4
01 01d2fe3c 71b23ac3 ntdll!ZwWaitForSingleObject+0xc
02 01d2fe78 71b239d1 mswsock!SockWaitForSingleObject+0x19b
03 01d2ff3c 71c016c9 mswsock!WSPSelect+0x229
04 01d2ff8c 63ec4696 WS2_32!select+0xb9
05 01d2ffb4 63ec4700 ISATQ!ATQ_BMON_SET::BmonThreadFunc+0x22
06 01d2ffb8 77e4a990 ISATQ!BmonThreadFunc+0x9
07 01d2ffc4 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 15
System Thread ID: cbc
Kernel Time: 0:0:0.125
User Time: 0:0:0.578
Thread Type: Possible ASP page. Possible DCOM activity
Executing Page: ASP.dll symbols not found. Unable to locate ASP page.
Continuing with other analysis.
No remote call being made
# ChildEBP RetAddr
00 01d6fe20 77f4313f SharedUserData!SystemCallStub+0x4
01 01d6fe24 77c57b85 ntdll!NtReplyWaitReceivePortEx+0xc
02 01d6ff8c 77c60829 RPCRT4!LRPC_ADDRESS::ReceiveLotsaCalls+0x193
03 01d6ff90 77c60771 RPCRT4!RecvLotsaCallsWrapper+0x9
04 01d6ffb0 77c60857 RPCRT4!BaseCachedThreadRoutine+0x9c
05 01d6ffb8 77e4a990 RPCRT4!ThreadStartRoutine+0x17
06 01d6ffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 16
System Thread ID: c34
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Type: Other
# ChildEBP RetAddr
00 0148ff70 77f430c7 SharedUserData!SystemCallStub+0x4
01 0148ff74 77f7e6ae ntdll!ZwRemoveIoCompletion+0xc
02 0148ffb8 77e4a990 ntdll!RtlpWorkerThread+0x3b
03 0148ffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 17
System Thread ID: a54
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Type: Other
# ChildEBP RetAddr
00 00abff70 77f430c7 SharedUserData!SystemCallStub+0x4
01 00abff74 77f7e6ae ntdll!ZwRemoveIoCompletion+0xc
02 00abffb8 77e4a990 ntdll!RtlpWorkerThread+0x3b
03 00abffec 00000000 kernel32!BaseThreadStart+0x34
*****
Dump name is formatted as: PID-Timestamp.dmp
Creating C:\iisstate\output\3616-1096919913.dmp - mini user dump
*****
Closing open log file C:\iisstate\output\IISState-3616.log