MS IIS Memory Usage Growth

We have a web-application running in MS IIS 6.0 under MS Winsows 2003 with
all latest updates installed. The application is a COM-server, exposed for
users via ASP pages. During a stress-test we encounter the following problem.
Dllhost memory usage increasingly grows and never lowers down to its initial
value after all sessions closed. The COM-server itself does not have memory
leaks, we know this because we monitor memory usage in our application. The
COM-object gets successfully destroyed and clears its memory each time a
request is done. As for Dllhost, it seems as if it never frees up all memory
back. We tried to change session timeouts and even to run the application in
the sessionless mode. Nothing helps. Our aim is to process approx. 10
requests per second (for beginning). Normally, if 10 new users are logged at
the moment T and get some resources allocated for their requests, IIS must
clear up all resources, which were used for 10 existing users logged in at
the moment T-<some-timeout>. If this is true, then the following behaviour is
anticipated. After the application test started (M0), Dllhost starts to eat
memory to some, possibly high, value because of new requests. Once the
timeout period elapsed since M0, the memory usage must stabilize: new memory
allocations are equal to the memory which is freed up in every moment.

So, what is the timeout for the sessionless mode?

If sessions are used, what additional time should pass after session timeout
before IIS actually frees up the session memory?

We tried to use sessions with 5 minutes timeout, but memory remains in use
after 5 minutes.

What memory overheads per session are normally charged by Dllhost per
asp-page display?

What data is stored in the memory, used by Dllhost?

How should we set up IIS in order to force memory cleaning exactly by timeout?

Currently IIS is ran out of physical memory (1Gb) in 10 minutes after the
test start and begins to swap, so that performance decreases tremendously.

I've read about some W3SVC parameters in the registry, which might possibly
help us to change IIS caching policy, but it was about static files, whereas
we host mostly dynamic asp-pages.

Thanks in advance.

David
Tue May 02 07:16:13 CDT 2006

If you are running IIS6 in Worker Process Isolation Mode, anything going on
within dllhost.exe has nothing to do with IIS and everything to do with your
code or server configuration.

BTW, you do expect snappy ASP performance, which means that ASP page
compilations (templates) are cached, regardless if "sessions" are closed
(because you'd expect User1 and User2 to have snappy ASP performance against
the same ASP page regardless if User1's session timed out). And besides, who
said that a session that "times out" has its memory DEALLOCATED? That is a
bad concept for long-running server-side processed, BTW.


Also, realize that for long-running processes, clearing away and
deallocating memory after every request is a BAD strategy because it
fragments memory, which drives up memory allocation pressure. Your best bet
is to allocate memory at start up and then re-use that memory through
request processing and deallocate once on shutdown.

At this point, it is not clear to me how your code nor memory management
strategy survives stress.

Here is my counterpoint: I can tell you that we stress ASP with sessions,
COM object creation/invocation, and optionally with mixture of accesing
other IIS resources, continuously for at least 3 weeks under 100% CPU load
and hundreds of concurrent users WITHOUT encountering any resource problems
(like RAM, Handles, Threads, Disk, etc) on *default* configuration of very
modest Windows Server 2003 machines (just 512MB or 1024MB RAM, P4-2.6Ghz
single or dual proc, 80GB IDE HDD machines).

We presume that customers would have better hardware, so if IIS6 can survive
lower-end resources, it can surely survive given more resources.

--
//David
IIS
http://blogs.msdn.com/David.Wang
This posting is provided "AS IS" with no warranties, and confers no rights.
//

"UserAgent" <UserAgent@discussions.microsoft.com> wrote in message
news:770FFD08-6472-48BA-AD83-4B238375B985@microsoft.com...
> We have a web-application running in MS IIS 6.0 under MS Winsows 2003 with
> all latest updates installed. The application is a COM-server, exposed for
> users via ASP pages. During a stress-test we encounter the following
> problem.
> Dllhost memory usage increasingly grows and never lowers down to its
> initial
> value after all sessions closed. The COM-server itself does not have
> memory
> leaks, we know this because we monitor memory usage in our application.
> The
> COM-object gets successfully destroyed and clears its memory each time a
> request is done. As for Dllhost, it seems as if it never frees up all
> memory
> back. We tried to change session timeouts and even to run the application
> in
> the sessionless mode. Nothing helps. Our aim is to process approx. 10
> requests per second (for beginning). Normally, if 10 new users are logged
> at
> the moment T and get some resources allocated for their requests, IIS must
> clear up all resources, which were used for 10 existing users logged in at
> the moment T-<some-timeout>. If this is true, then the following behaviour
> is
> anticipated. After the application test started (M0), Dllhost starts to
> eat
> memory to some, possibly high, value because of new requests. Once the
> timeout period elapsed since M0, the memory usage must stabilize: new
> memory
> allocations are equal to the memory which is freed up in every moment.
>
> So, what is the timeout for the sessionless mode?
>
> If sessions are used, what additional time should pass after session
> timeout
> before IIS actually frees up the session memory?
>
> We tried to use sessions with 5 minutes timeout, but memory remains in use
> after 5 minutes.
>
> What memory overheads per session are normally charged by Dllhost per
> asp-page display?
>
> What data is stored in the memory, used by Dllhost?
>
> How should we set up IIS in order to force memory cleaning exactly by
> timeout?
>
> Currently IIS is ran out of physical memory (1Gb) in 10 minutes after the
> test start and begins to swap, so that performance decreases tremendously.
>
> I've read about some W3SVC parameters in the registry, which might
> possibly
> help us to change IIS caching policy, but it was about static files,
> whereas
> we host mostly dynamic asp-pages.
>
> Thanks in advance.

UserAgent
Tue May 02 16:08:02 CDT 2006

"David Wang [Msft]" wrote:

>If you are running IIS6 in Worker Process Isolation Mode, anything going on
>within dllhost.exe has nothing to do with IIS and everything to do with your
>code or server configuration.

Well, perhaps I were not quite clear. Actually I treat dllhost as a part of
IIS. Dllhost itself is a wrapping black box for us. This is not our
application
who creates and destroys session objects and other internal objects, and it
doesn't matter whether it is IIS or dllhost who provides the asp-application
with the objects - in either case there is a lot of external actions performed
around the application and are out of our control (for example, how can we
conditionally prevent an application from creating a new session object under
some circumstances?). Indeed, these actions seem to be a subject to
optimization
via server configuration, but this is a part of the question. I'd really
appreciate any recommendations.

>BTW, you do expect snappy ASP performance, which means that ASP page
>compilations (templates) are cached, regardless if "sessions" are closed
>(because you'd expect User1 and User2 to have snappy ASP performance against
>the same ASP page regardless if User1's session timed out).

Do you mean that the Enable buffering option may increase memory usage
irretraceably? Then this is a bad design, IMHO.

>And besides, who said that a session that "times out" has its memory
> DEALLOCATED?

Hm. What is the sense to have a session object which lives forever, even
after it becomes useless? I'd assume that a closed or timed out session
MUST clear its memory. Otherwise, I'm not sure I understand how such an
application can work in production mode. Let us put aside the question
about what memory manager is used and how it works and prevents memory
fragmentation. The fact is that we need to clean up memory at some
practicable moment if we don't want it to go down the drain shortly.

> That is a bad concept for long-running server-side processed, BTW.

What exactly? So, what IIS (or dllhost?) does with a session object of
timed out session? After all, what's the meaning of the session timeout,
which we can configure in the Application Configuration of IIS?

>Also, realize that for long-running processes, clearing away and
>deallocating memory after every request is a BAD strategy because it
>fragments memory, which drives up memory allocation pressure. Your best bet
>is to allocate memory at start up and then re-use that memory through
>request processing and deallocate once on shutdown.

Yes, we know this. Perhaps, I were again not quite correct (why - I'll
explain below). Actually I've ment a session-wide clean-up with
a logic somewhat like in the following example.

// processrequestexample.asp

var Obj;
Obj = Session.Contents("WorkObj");
if(!Obj)
{
Obj = Server.CreateObject("XServer.XIntf");
Session.Contents("WorkObj") = Obj;
}
req = Request.QueryString("request");
Response.Write(Obj.Process(req));

// global.asa

function Session_OnEnd()
{
var Obj = Session("WorkObj");
if (Obj) { delete Obj; Session("WorkObj") = null; }
Session.Contents.RemoveAll();
}

The reason why I wrote about clean-up after each request was that
we decided to disable sessions temporarily (for testing purposes),
so that each request indeed created a new com-object and then
destroyed it, but this is not a normal logic of the application.
Even with the session-wide clean-up memory usage insreases steadly.

>At this point, it is not clear to me how your code nor memory management
>strategy survives stress.

With the mentioned load, it does only for approx. 10 minutes. The question is
why even after the stress-test is stopped the memory is not get freed
in reasonable time. Our objects are destroyed, and fragments may be packed.
I don't know how can we optimize the ASP's memory manager which performs
Server.CreateObject and delete. I think this is the IIS's guts which can
not be altered by an asp application. Am I right?

>Here is my counterpoint: I can tell you that we stress ASP with sessions,
>COM object creation/invocation, and optionally with mixture of accesing
>other IIS resources, continuously for at least 3 weeks under 100% CPU load
>and hundreds of concurrent users WITHOUT encountering any resource problems
>(like RAM, Handles, Threads, Disk, etc) on *default* configuration of very
>modest Windows Server 2003 machines (just 512MB or 1024MB RAM, P4-2.6Ghz
>single or dual proc, 80GB IDE HDD machines).

There is another nuance which may be worth mentioning. The method Process
of the com-object creates sockets itself and performs http-requests to
some outer servers, after which the obtained data is processed and written
into asp's response. So, I guess that the sockets may eat memory on their own
internally, and do not free it promptly (it seems they obey the logic you
like:
do not free resources after usage but wait if they will be again in demand,
suddendly; I don't remember what is a default timeout for internal sockets'
buffers and where to change this in the registry).

>We presume that customers would have better hardware, so if IIS6 can survive
>lower-end resources, it can surely survive given more resources.

In fact, the com-object utilizes at most 500K per session. So, 1Gb could
be enough for 2000 sessions, if the memory overheads are constant and
predictable.
As for dllhost, I'd like to know how much memory does it allocate itself and
for what purposes, in which events, under what circumstances?

Thanks in advance.