IIS Hangs - requesting help analyzing iisstate.log

We have a Documentum application with the following oversimplified
architecture:

Browser --> ASP --> VB COM (wraps Java Documentum API) --> Java
Documentum API --> Documentum DocBroker --> Documentum DocBase (SQL
2000 DB)

When running very simple load tests with even 10 users just bouncing
through some links that do fairly common Documentum queries, IIS
hangs.

Here is what seems to be the offending thread from iisstate.log:

===========================================================
Thread ID: 32
System Thread ID: 964
Kernel Time: 0:0:0.93
User Time: 0:0:3.156
Thread Status: Thread is in a WAIT state.

IISState has detected that this thread has thrown an unhandled
exception
and is about to crash. IISState will now attempt to display the
original
faulting stack.


.cxr 7a8f68c
# ChildEBP RetAddr
00 07a8f960 6b052ffe msjava!CJavaThreadBlock::UnhijackThread_14+0x5
01 07a8f96c 6b0515cc msjava!CJavaThreadBlock::LeaveMCWorker+0x9
02 07a8f978 6b05159a msjava!CJavaThreadBlock::LeaveMC+0x21
03 07a8f988 6b052f49 msjava!CJavaThreadBlock::LeaveMCCompletely+0x15
04 07a8fa1c 6b051e4c msjava!CDeadPunkList::ReleaseAll+0x1a
05 07a8fa30 6b051e1b msjava!CJavaEngine::VMInvoke+0x2e
06 07a8fa44 6b082a7c msjava!MethodDesc::CallFromInterpreter+0x26
07 07a8fa54 6b0f0450 msjava!CJavaEngine::CallFromVMByNameAndType+0x29
08 07a8fa74 6b0f0229 msjava!JavaScript::SendBridgeEvent+0x2f
09 07a8fad0 6b0f122a msjava!JavaScript::ExecVMCallout+0x67
0a 07a8faec 6b0efd11 msjava!JavaScript::Close+0x5e
0b 07a8fb08 6b0f09fc msjava!JavaScript::Finalize+0xa0
0c 07a8fb18 6b0f1a98 msjava!JavaScript::NonDelegatingRelease+0x20
0d 07a8fb20 6b08f29d msjava!JavaScript::Release+0xd
0e 07a8fb28 6a9f93df msjava!CCW_UniversalRelease+0x1d
0f 07a8fb50 6a9e5caf MSVBVM60!RESDESCTBL::DestructItem+0x88
10 07a8fb70 6a9e5c65 MSVBVM60!RESDESCTBL::DestructModInst+0x43
11 07a8fb98 6a9e5a64 MSVBVM60!RcmDestroyModuleInstance+0x68
12 07a8fbbc 6a9e567c MSVBVM60!RcmSemiReset_606+0xfa
13 07a8fbd4 6a9e55d6 MSVBVM60!RcmShutDownProject+0x3f
14 07a8fbec 6a9e54d6 MSVBVM60!RcmResetProject+0x1b
15 07a8fe68 6a9e73bb MSVBVM60!EbResetProjectNormal+0xd5
16 07a8fe74 6a9e5352 MSVBVM60!EbResetProject+0x27
17 07a8fe8c 6a9e71dd MSVBVM60!CThreadPool::ResetProject+0x5a
18 07a8fea8 6aa58c57 MSVBVM60!CVBThreadAction::CleanupProjData+0x46
19 07a8feb8 6aa0eeed MSVBVM60!CThreadPool::CleanupAtDllDetach+0x14
1a 07a8fed0 77f86215 MSVBVM60!UserDllMain_38+0x39
1b 07a8fef8 77f8e4cd ntdll!LdrpCallInitRoutine+0x14
1c 07a8ff4c 7c59b89a ntdll!LdrShutdownThread+0xa3
1d 07a8ffb4 7c57b3b1 KERNEL32!ExitThread+0x53
1e 07a8ffec 00000000 KERNEL32!BaseThreadStart+0x81


**Current Stack State**

Other information: Thread is throwing an exception. Checking for known
issues.
*** WARNING: Unable to verify checksum for
C:\DOCUMENTUM\product\4.2\bin\DMCL40.dll
*** ERROR: Symbol file could not be found. Defaulted to export
symbols for C:\DOCUMENTUM\product\4.2\bin\DMCL40.dll -
Other information: Thread is throwing an exception. Checking for known
issues.
Thread Type: ASP
Executing Page: Unable to locate ASP page

# ChildEBP RetAddr
00 07a8f278 7c59be3b ntdll!NtWaitForSingleObject+0xb
01 07a8f588 6b0d682b KERNEL32!UnhandledExceptionFilter+0x2b5
02 07a8f598 6b11f44a msjava!SeveredExceptionFilter+0x36
03 07a8f5ac 77f8eb70 msjava!NativeStubExceptionHandler+0x1e
04 07a8f5d0 77f95160 ntdll!ExecuteHandler+0x26
05 07a8f658 77fa1606 ntdll!RtlDispatchException+0x76
06 07a8f6d4 77fcc882 ntdll!KiUserExceptionDispatcher+0xe
07 07a8fa1c 6b051e4c ntdll!RtlFreeHeap+0xb8
08 07a8fa74 6b0f0229 msjava!CJavaEngine::VMInvoke+0x2e
09 07a8fad0 6b0f122a msjava!JavaScript::ExecVMCallout+0x67
0a 07a8faec 6b0efd11 msjava!JavaScript::Close+0x5e
0b 07a8fb08 6b0f09fc msjava!JavaScript::Finalize+0xa0
0c 07a8fb18 6b0f1a98 msjava!JavaScript::NonDelegatingRelease+0x20
0d 07a8fb20 6b08f29d msjava!JavaScript::Release+0xd
0e 07a8fb28 6a9f93df msjava!CCW_UniversalRelease+0x1d
0f 07a8fb50 6a9e5caf MSVBVM60!RESDESCTBL::DestructItem+0x88
10 07a8fb70 6a9e5c65 MSVBVM60!RESDESCTBL::DestructModInst+0x43
11 07a8fb98 6a9e5a64 MSVBVM60!RcmDestroyModuleInstance+0x68
12 07a8fbbc 6a9e567c MSVBVM60!RcmSemiReset_606+0xfa
13 07a8fbd4 6a9e55d6 MSVBVM60!RcmShutDownProject+0x3f
14 07a8fbec 6a9e54d6 MSVBVM60!RcmResetProject+0x1b
15 07a8fe68 6a9e73bb MSVBVM60!EbResetProjectNormal+0xd5
16 07a8fe74 6a9e5352 MSVBVM60!EbResetProject+0x27
17 07a8fe8c 6a9e71dd MSVBVM60!CThreadPool::ResetProject+0x5a
18 07a8fea8 6aa58c57 MSVBVM60!CVBThreadAction::CleanupProjData+0x46
19 07a8feb8 6aa0eeed MSVBVM60!CThreadPool::CleanupAtDllDetach+0x14
1a 07a8fed0 77f86215 MSVBVM60!UserDllMain_38+0x39
1b 07a8fef8 77f8e4cd ntdll!LdrpCallInitRoutine+0x14
1c 07a8ff4c 7c59b89a ntdll!LdrShutdownThread+0xa3
1d 07a8ffb4 7c57b3b1 KERNEL32!ExitThread+0x53
1e 07a8ffec 00000000 KERNEL32!BaseThreadStart+0x81
================================================================

Can anyone shed light on this problem? Any help is greatly
appreciated!

Pat
Wed Jan 21 23:35:42 CST 2004

It means that it is failing inside the Java code. MSJava is the Java VM, so
if the Java code doesn't handle an exception, it will fail.

You will need to contact Documentum and see if they have an update. On the
good news side, your VB wrapper appears to be working fine.

Pat

"Noah Heldman" <nheldman@siliconspace.com> wrote in message
news:e20fa673.0401212008.478b1ec8@posting.google.com...
> We have a Documentum application with the following oversimplified
> architecture:
>
> Browser --> ASP --> VB COM (wraps Java Documentum API) --> Java
> Documentum API --> Documentum DocBroker --> Documentum DocBase (SQL
> 2000 DB)
>
> When running very simple load tests with even 10 users just bouncing
> through some links that do fairly common Documentum queries, IIS
> hangs.
>
> Here is what seems to be the offending thread from iisstate.log:
>
> ===========================================================
> Thread ID: 32
> System Thread ID: 964
> Kernel Time: 0:0:0.93
> User Time: 0:0:3.156
> Thread Status: Thread is in a WAIT state.
>
> IISState has detected that this thread has thrown an unhandled
> exception
> and is about to crash. IISState will now attempt to display the
> original
> faulting stack.
>
>
> .cxr 7a8f68c
> # ChildEBP RetAddr
> 00 07a8f960 6b052ffe msjava!CJavaThreadBlock::UnhijackThread_14+0x5
> 01 07a8f96c 6b0515cc msjava!CJavaThreadBlock::LeaveMCWorker+0x9
> 02 07a8f978 6b05159a msjava!CJavaThreadBlock::LeaveMC+0x21
> 03 07a8f988 6b052f49 msjava!CJavaThreadBlock::LeaveMCCompletely+0x15
> 04 07a8fa1c 6b051e4c msjava!CDeadPunkList::ReleaseAll+0x1a
> 05 07a8fa30 6b051e1b msjava!CJavaEngine::VMInvoke+0x2e
> 06 07a8fa44 6b082a7c msjava!MethodDesc::CallFromInterpreter+0x26
> 07 07a8fa54 6b0f0450 msjava!CJavaEngine::CallFromVMByNameAndType+0x29
> 08 07a8fa74 6b0f0229 msjava!JavaScript::SendBridgeEvent+0x2f
> 09 07a8fad0 6b0f122a msjava!JavaScript::ExecVMCallout+0x67
> 0a 07a8faec 6b0efd11 msjava!JavaScript::Close+0x5e
> 0b 07a8fb08 6b0f09fc msjava!JavaScript::Finalize+0xa0
> 0c 07a8fb18 6b0f1a98 msjava!JavaScript::NonDelegatingRelease+0x20
> 0d 07a8fb20 6b08f29d msjava!JavaScript::Release+0xd
> 0e 07a8fb28 6a9f93df msjava!CCW_UniversalRelease+0x1d
> 0f 07a8fb50 6a9e5caf MSVBVM60!RESDESCTBL::DestructItem+0x88
> 10 07a8fb70 6a9e5c65 MSVBVM60!RESDESCTBL::DestructModInst+0x43
> 11 07a8fb98 6a9e5a64 MSVBVM60!RcmDestroyModuleInstance+0x68
> 12 07a8fbbc 6a9e567c MSVBVM60!RcmSemiReset_606+0xfa
> 13 07a8fbd4 6a9e55d6 MSVBVM60!RcmShutDownProject+0x3f
> 14 07a8fbec 6a9e54d6 MSVBVM60!RcmResetProject+0x1b
> 15 07a8fe68 6a9e73bb MSVBVM60!EbResetProjectNormal+0xd5
> 16 07a8fe74 6a9e5352 MSVBVM60!EbResetProject+0x27
> 17 07a8fe8c 6a9e71dd MSVBVM60!CThreadPool::ResetProject+0x5a
> 18 07a8fea8 6aa58c57 MSVBVM60!CVBThreadAction::CleanupProjData+0x46
> 19 07a8feb8 6aa0eeed MSVBVM60!CThreadPool::CleanupAtDllDetach+0x14
> 1a 07a8fed0 77f86215 MSVBVM60!UserDllMain_38+0x39
> 1b 07a8fef8 77f8e4cd ntdll!LdrpCallInitRoutine+0x14
> 1c 07a8ff4c 7c59b89a ntdll!LdrShutdownThread+0xa3
> 1d 07a8ffb4 7c57b3b1 KERNEL32!ExitThread+0x53
> 1e 07a8ffec 00000000 KERNEL32!BaseThreadStart+0x81
>
>
> **Current Stack State**
>
> Other information: Thread is throwing an exception. Checking for known
> issues.
> *** WARNING: Unable to verify checksum for
> C:\DOCUMENTUM\product\4.2\bin\DMCL40.dll
> *** ERROR: Symbol file could not be found. Defaulted to export
> symbols for C:\DOCUMENTUM\product\4.2\bin\DMCL40.dll -
> Other information: Thread is throwing an exception. Checking for known
> issues.
> Thread Type: ASP
> Executing Page: Unable to locate ASP page
>
> # ChildEBP RetAddr
> 00 07a8f278 7c59be3b ntdll!NtWaitForSingleObject+0xb
> 01 07a8f588 6b0d682b KERNEL32!UnhandledExceptionFilter+0x2b5
> 02 07a8f598 6b11f44a msjava!SeveredExceptionFilter+0x36
> 03 07a8f5ac 77f8eb70 msjava!NativeStubExceptionHandler+0x1e
> 04 07a8f5d0 77f95160 ntdll!ExecuteHandler+0x26
> 05 07a8f658 77fa1606 ntdll!RtlDispatchException+0x76
> 06 07a8f6d4 77fcc882 ntdll!KiUserExceptionDispatcher+0xe
> 07 07a8fa1c 6b051e4c ntdll!RtlFreeHeap+0xb8
> 08 07a8fa74 6b0f0229 msjava!CJavaEngine::VMInvoke+0x2e
> 09 07a8fad0 6b0f122a msjava!JavaScript::ExecVMCallout+0x67
> 0a 07a8faec 6b0efd11 msjava!JavaScript::Close+0x5e
> 0b 07a8fb08 6b0f09fc msjava!JavaScript::Finalize+0xa0
> 0c 07a8fb18 6b0f1a98 msjava!JavaScript::NonDelegatingRelease+0x20
> 0d 07a8fb20 6b08f29d msjava!JavaScript::Release+0xd
> 0e 07a8fb28 6a9f93df msjava!CCW_UniversalRelease+0x1d
> 0f 07a8fb50 6a9e5caf MSVBVM60!RESDESCTBL::DestructItem+0x88
> 10 07a8fb70 6a9e5c65 MSVBVM60!RESDESCTBL::DestructModInst+0x43
> 11 07a8fb98 6a9e5a64 MSVBVM60!RcmDestroyModuleInstance+0x68
> 12 07a8fbbc 6a9e567c MSVBVM60!RcmSemiReset_606+0xfa
> 13 07a8fbd4 6a9e55d6 MSVBVM60!RcmShutDownProject+0x3f
> 14 07a8fbec 6a9e54d6 MSVBVM60!RcmResetProject+0x1b
> 15 07a8fe68 6a9e73bb MSVBVM60!EbResetProjectNormal+0xd5
> 16 07a8fe74 6a9e5352 MSVBVM60!EbResetProject+0x27
> 17 07a8fe8c 6a9e71dd MSVBVM60!CThreadPool::ResetProject+0x5a
> 18 07a8fea8 6aa58c57 MSVBVM60!CVBThreadAction::CleanupProjData+0x46
> 19 07a8feb8 6aa0eeed MSVBVM60!CThreadPool::CleanupAtDllDetach+0x14
> 1a 07a8fed0 77f86215 MSVBVM60!UserDllMain_38+0x39
> 1b 07a8fef8 77f8e4cd ntdll!LdrpCallInitRoutine+0x14
> 1c 07a8ff4c 7c59b89a ntdll!LdrShutdownThread+0xa3
> 1d 07a8ffb4 7c57b3b1 KERNEL32!ExitThread+0x53
> 1e 07a8ffec 00000000 KERNEL32!BaseThreadStart+0x81
> ================================================================
>
> Can anyone shed light on this problem? Any help is greatly
> appreciated!

Dev
Thu Jan 22 18:17:55 CST 2004

Noah,

If your server is crashing or hanging you might want to
download our ReJuvin8 tool. It's free for the first
15-days. It can keep your site running despite problems.

It's designed to detect problems then restart or reboot.
It can restart ANY programs, services, or processes in
precise order. It doesn't fix bugs but it does give you some
breathing room while you are finding them. ReJuvin8 also
gives you email and telephone alert messages and graphic
status reports. It runs as a Windows Service.

It can also test, alert, and report on URLs running anywhere
on any server on your net connection. So if you use multiple
servers you can use them to watch each other, too.

ReJuvin8 has both an automatic and a custom mode. In the
auto mode you make all the settings by pointing and
clicking. This will handle most applications.

The custom mode uses heartbeat synch files to get absolute
detection that key processes are working. Tools and
instructions are provided to help you do this. We also offer
customization as a contract service.

Charles


Get ReJuvin8 at: http://www.rejuvin8.net?ac=ch1

> "Noah Heldman" <nheldman@siliconspace.com> wrote in message
> news:e20fa673.0401212008.478b1ec8@posting.google.com...
> > We have a Documentum application with the following oversimplified
> > architecture:
> >
> > Browser --> ASP --> VB COM (wraps Java Documentum API) --> Java
> > Documentum API --> Documentum DocBroker --> Documentum DocBase (SQL
> > 2000 DB)
> >
> > When running very simple load tests with even 10 users just bouncing
> > through some links that do fairly common Documentum queries, IIS
> > hangs.
> >
> > Here is what seems to be the offending thread from iisstate.log:
> >
> > ===========================================================
> > Thread ID: 32
> > System Thread ID: 964
> > Kernel Time: 0:0:0.93
> > User Time: 0:0:3.156
> > Thread Status: Thread is in a WAIT state.
> >
> > IISState has detected that this thread has thrown an unhandled
> > exception
> > and is about to crash. IISState will now attempt to display the
> > original
> > faulting stack.
> >
> >
> > .cxr 7a8f68c
> > # ChildEBP RetAddr
> > 00 07a8f960 6b052ffe msjava!CJavaThreadBlock::UnhijackThread_14+0x5
> > 01 07a8f96c 6b0515cc msjava!CJavaThreadBlock::LeaveMCWorker+0x9
> > 02 07a8f978 6b05159a msjava!CJavaThreadBlock::LeaveMC+0x21
> > 03 07a8f988 6b052f49 msjava!CJavaThreadBlock::LeaveMCCompletely+0x15
> > 04 07a8fa1c 6b051e4c msjava!CDeadPunkList::ReleaseAll+0x1a
> > 05 07a8fa30 6b051e1b msjava!CJavaEngine::VMInvoke+0x2e
> > 06 07a8fa44 6b082a7c msjava!MethodDesc::CallFromInterpreter+0x26
> > 07 07a8fa54 6b0f0450 msjava!CJavaEngine::CallFromVMByNameAndType+0x29
> > 08 07a8fa74 6b0f0229 msjava!JavaScript::SendBridgeEvent+0x2f
> > 09 07a8fad0 6b0f122a msjava!JavaScript::ExecVMCallout+0x67
> > 0a 07a8faec 6b0efd11 msjava!JavaScript::Close+0x5e
> > 0b 07a8fb08 6b0f09fc msjava!JavaScript::Finalize+0xa0
> > 0c 07a8fb18 6b0f1a98 msjava!JavaScript::NonDelegatingRelease+0x20
> > 0d 07a8fb20 6b08f29d msjava!JavaScript::Release+0xd
> > 0e 07a8fb28 6a9f93df msjava!CCW_UniversalRelease+0x1d
> > 0f 07a8fb50 6a9e5caf MSVBVM60!RESDESCTBL::DestructItem+0x88
> > 10 07a8fb70 6a9e5c65 MSVBVM60!RESDESCTBL::DestructModInst+0x43
> > 11 07a8fb98 6a9e5a64 MSVBVM60!RcmDestroyModuleInstance+0x68
> > 12 07a8fbbc 6a9e567c MSVBVM60!RcmSemiReset_606+0xfa
> > 13 07a8fbd4 6a9e55d6 MSVBVM60!RcmShutDownProject+0x3f
> > 14 07a8fbec 6a9e54d6 MSVBVM60!RcmResetProject+0x1b
> > 15 07a8fe68 6a9e73bb MSVBVM60!EbResetProjectNormal+0xd5
> > 16 07a8fe74 6a9e5352 MSVBVM60!EbResetProject+0x27
> > 17 07a8fe8c 6a9e71dd MSVBVM60!CThreadPool::ResetProject+0x5a
> > 18 07a8fea8 6aa58c57 MSVBVM60!CVBThreadAction::CleanupProjData+0x46
> > 19 07a8feb8 6aa0eeed MSVBVM60!CThreadPool::CleanupAtDllDetach+0x14
> > 1a 07a8fed0 77f86215 MSVBVM60!UserDllMain_38+0x39
> > 1b 07a8fef8 77f8e4cd ntdll!LdrpCallInitRoutine+0x14
> > 1c 07a8ff4c 7c59b89a ntdll!LdrShutdownThread+0xa3
> > 1d 07a8ffb4 7c57b3b1 KERNEL32!ExitThread+0x53
> > 1e 07a8ffec 00000000 KERNEL32!BaseThreadStart+0x81
> >
> >
> > **Current Stack State**
> >
> > Other information: Thread is throwing an exception. Checking for known
> > issues.
> > *** WARNING: Unable to verify checksum for
> > C:\DOCUMENTUM\product\4.2\bin\DMCL40.dll
> > *** ERROR: Symbol file could not be found. Defaulted to export
> > symbols for C:\DOCUMENTUM\product\4.2\bin\DMCL40.dll -
> > Other information: Thread is throwing an exception. Checking for known
> > issues.
> > Thread Type: ASP
> > Executing Page: Unable to locate ASP page
> >
> > # ChildEBP RetAddr
> > 00 07a8f278 7c59be3b ntdll!NtWaitForSingleObject+0xb
> > 01 07a8f588 6b0d682b KERNEL32!UnhandledExceptionFilter+0x2b5
> > 02 07a8f598 6b11f44a msjava!SeveredExceptionFilter+0x36
> > 03 07a8f5ac 77f8eb70 msjava!NativeStubExceptionHandler+0x1e
> > 04 07a8f5d0 77f95160 ntdll!ExecuteHandler+0x26
> > 05 07a8f658 77fa1606 ntdll!RtlDispatchException+0x76
> > 06 07a8f6d4 77fcc882 ntdll!KiUserExceptionDispatcher+0xe
> > 07 07a8fa1c 6b051e4c ntdll!RtlFreeHeap+0xb8
> > 08 07a8fa74 6b0f0229 msjava!CJavaEngine::VMInvoke+0x2e
> > 09 07a8fad0 6b0f122a msjava!JavaScript::ExecVMCallout+0x67
> > 0a 07a8faec 6b0efd11 msjava!JavaScript::Close+0x5e
> > 0b 07a8fb08 6b0f09fc msjava!JavaScript::Finalize+0xa0
> > 0c 07a8fb18 6b0f1a98 msjava!JavaScript::NonDelegatingRelease+0x20
> > 0d 07a8fb20 6b08f29d msjava!JavaScript::Release+0xd
> > 0e 07a8fb28 6a9f93df msjava!CCW_UniversalRelease+0x1d
> > 0f 07a8fb50 6a9e5caf MSVBVM60!RESDESCTBL::DestructItem+0x88
> > 10 07a8fb70 6a9e5c65 MSVBVM60!RESDESCTBL::DestructModInst+0x43
> > 11 07a8fb98 6a9e5a64 MSVBVM60!RcmDestroyModuleInstance+0x68
> > 12 07a8fbbc 6a9e567c MSVBVM60!RcmSemiReset_606+0xfa
> > 13 07a8fbd4 6a9e55d6 MSVBVM60!RcmShutDownProject+0x3f
> > 14 07a8fbec 6a9e54d6 MSVBVM60!RcmResetProject+0x1b
> > 15 07a8fe68 6a9e73bb MSVBVM60!EbResetProjectNormal+0xd5
> > 16 07a8fe74 6a9e5352 MSVBVM60!EbResetProject+0x27
> > 17 07a8fe8c 6a9e71dd MSVBVM60!CThreadPool::ResetProject+0x5a
> > 18 07a8fea8 6aa58c57 MSVBVM60!CVBThreadAction::CleanupProjData+0x46
> > 19 07a8feb8 6aa0eeed MSVBVM60!CThreadPool::CleanupAtDllDetach+0x14
> > 1a 07a8fed0 77f86215 MSVBVM60!UserDllMain_38+0x39
> > 1b 07a8fef8 77f8e4cd ntdll!LdrpCallInitRoutine+0x14
> > 1c 07a8ff4c 7c59b89a ntdll!LdrShutdownThread+0xa3
> > 1d 07a8ffb4 7c57b3b1 KERNEL32!ExitThread+0x53
> > 1e 07a8ffec 00000000 KERNEL32!BaseThreadStart+0x81
> > ================================================================
> >
> > Can anyone shed light on this problem? Any help is greatly
> > appreciated!
>
>