IIS and File Replication

Hi,

I am currently creating/testing a small web farm consisting of two Win2k3
Standard Servers that will hopefully end up hosting services to a number of
small to medium-sized websites for clients. (probably a hundred or so - there
are no sites that are overly strenuous on bandwidth or resources.)

Both servers are AD DCs running mirrored IIS configurations (metabases
synced using IIsCnfg.vbs), NLB to provide load balancing and a DFS/replicated
"Domains" folder of all the content/logs of the websites.

I have a few questions for anyone that has maybe set up a configuration like
this? (Advice and hints would be very much welcomed too!)

1) Is it really good idea to replicate IIS log files? Would the replication
service be able to handle the potential problems of two different log files
on each server as visitors hit each one? (The servers are connected over a
local 1Gbit connection - there are two NICs in each server, one for NLB one
for general local network traffic) Stats generated from these log files are
essential for analysis so this definitely needs to be implemented somehow.

2) Network load balancing includes the affinity setting - we have a number
of classic ASP web apps using Session variables that could have issues with a
web farm environment. Is the affinity setting (in Single mode) in NLB
reliable enough to keep a web user sticky to their initial server for a
reasonable amount of time?

3) Performance-wise! What are other people's thoughts/ideas for this
implementation? I have read a number of white papers and documentation
available and it appears that the above solution is going in the right
direction for what we want to achieve?

Many thanks,
Regards,
Stewart.

Tom
Tue Oct 11 07:44:33 CDT 2005

"Stewart" <Stewart@discussions.microsoft.com> wrote in message
news:7DF2991A-30EF-4760-87B0-24A076E7803F@microsoft.com...
> 1) Is it really good idea to replicate IIS log files? Would the
> replication
> service be able to handle the potential problems of two different log
> files
> on each server as visitors hit each one? (The servers are connected over a
> local 1Gbit connection - there are two NICs in each server, one for NLB
> one
> for general local network traffic) Stats generated from these log files
> are
> essential for analysis so this definitely needs to be implemented somehow.

I don't understand why you would do this.

--
Tom Kaminski IIS MVP
http://www.microsoft.com/windowsserver2003/community/centers/iis/
http://mvp.support.microsoft.com/
http://www.iistoolshed.com/ - tools, scripts, and utilities for running IIS

Stewart
Tue Oct 11 08:25:03 CDT 2005

Hi,

How do you mean?

We use a webhosting control panel (Helm from Web Host Automation) as a
front-end for our customers to be able to log in and change their
settings/passwords etc (for their mail, IIS, FTP etc) - this resides on and
is only accessed from one of the servers (it is not on both servers as it
does not really need to redundant and it would get quite complicated being
run separately from each server as the metabase changes and other
configuration file changes for other services would need to be synchronised
between both servers.).

For every domain you add to the system in the web hosting control panel,
Helm automatically creates and configures (using each program's API) the
necessary DNS records, mail accounts with the mail server and the IIS site.
Helm then creates the website users (for IIS and FTP) and assigns their
permissions to the folder structure. In a default configuration, the Helm
"domain store" would be:

C:/Domains
- domain1.com
- db (for database files - Access etc)
- logs (holding all the IIS logfiles for the website)
- wwwroot (holding the web-accessible part of the site)
- domain2.com
- db
- logs
- wwwroot
...And so on for every domain hosted on the server.

At the moment, the setup replicates this "C:/Domains" folder across both
servers - this works fine. The main problem I can possibly see is that as the
two versions of the same website are being accessed from either server, the
log files for that same site are going to be changed on their respective
server. My worries are 1) This is going to possibly cause a lot of traffic
across the network (although the changes being made to each file are usually
miniscule amounts of data - a few lines of ASCII) and what happens if:

- Server1 serves a web page to WebUser1 and logs the request to the website
log on Server1.
- At the same time (or just before the replication service has a chance to
update Server2 with the changes just made on Server1), Server2 serves a web
page to WebUser2 and logs the request to the website log on Server2.

- Now there are two different log files - the replication service can't just
copy one over the other as log file entries will be lost.

How would the replication service deal with this kind of situation? That is
why I ask whether it is even a good idea to use replication on log files?

Web logs are analysed both a statistics program to provide graphical stats
to customers - this is an essential feature that can't be got rid of.

Regards,
Stewart.

"Tom Kaminski [MVP]" wrote:

> "Stewart" <Stewart@discussions.microsoft.com> wrote in message
> news:7DF2991A-30EF-4760-87B0-24A076E7803F@microsoft.com...
> > 1) Is it really good idea to replicate IIS log files? Would the
> > replication
> > service be able to handle the potential problems of two different log
> > files
> > on each server as visitors hit each one? (The servers are connected over a
> > local 1Gbit connection - there are two NICs in each server, one for NLB
> > one
> > for general local network traffic) Stats generated from these log files
> > are
> > essential for analysis so this definitely needs to be implemented somehow.
>
> I don't understand why you would do this.
>
> --
> Tom Kaminski IIS MVP
> http://www.microsoft.com/windowsserver2003/community/centers/iis/
> http://mvp.support.microsoft.com/
> http://www.iistoolshed.com/ - tools, scripts, and utilities for running IIS
>
>
>

Tom
Tue Oct 11 08:36:35 CDT 2005

"Stewart" <Stewart@discussions.microsoft.com> wrote in message
news:B72AD4A6-83AF-47E9-913A-C7931331FA54@microsoft.com...
> - Now there are two different log files - the replication service can't
> just
> copy one over the other as log file entries will be lost.

Right!

> How would the replication service deal with this kind of situation? That
> is
> why I ask whether it is even a good idea to use replication on log files?

No - you have to use a log analyzer that's capable of dealing with multiple
log files. The log files from each server are different and need to be
analyzed in aggregate.

--
Tom Kaminski IIS MVP
http://www.microsoft.com/windowsserver2003/community/centers/iis/
http://mvp.support.microsoft.com/
http://www.iistoolshed.com/ - tools, scripts, and utilities for running IIS

Stewart
Tue Oct 11 09:30:21 CDT 2005

Thanks!

I have just looked around for possible options, including analysers that
look at multiple log files and log file mergers
(http://www.bitstrike.com/weblogmixer/index.php) - these look like viable
solutions, I will research into them fully.

Just one more question, am I right in thinking that I can exclude log files
(.log) from being replicated by the replication service under C:/Domains?

Regards,
Stewart.


"Tom Kaminski [MVP]" wrote:

> "Stewart" <Stewart@discussions.microsoft.com> wrote in message
> news:B72AD4A6-83AF-47E9-913A-C7931331FA54@microsoft.com...
> > - Now there are two different log files - the replication service can't
> > just
> > copy one over the other as log file entries will be lost.
>
> Right!
>
> > How would the replication service deal with this kind of situation? That
> > is
> > why I ask whether it is even a good idea to use replication on log files?
>
> No - you have to use a log analyzer that's capable of dealing with multiple
> log files. The log files from each server are different and need to be
> analyzed in aggregate.
>
> --
> Tom Kaminski IIS MVP
> http://www.microsoft.com/windowsserver2003/community/centers/iis/
> http://mvp.support.microsoft.com/
> http://www.iistoolshed.com/ - tools, scripts, and utilities for running IIS
>
>
>

Kristofer
Tue Oct 11 12:20:08 CDT 2005

Yes, you do not want IIS log files to be replicated because of the issues
you have mentioned (unless the replication service is able to analyse what
has changed in both files and keep both changes).

Instead the log analyser must be able to analyse log files from multiple
sources.



--
Regards,
Kristofer Gafvert (IIS MVP)
http://www.gafvert.info/iis/ - IIS Related Info


Stewart wrote:

>Thanks!
>
>I have just looked around for possible options, including analysers that
>look at multiple log files and log file mergers
>(http://www.bitstrike.com/weblogmixer/index.php) - these look like viable
>solutions, I will research into them fully.
>
>Just one more question, am I right in thinking that I can exclude log files
>(.log) from being replicated by the replication service under C:/Domains?
>
>Regards,
>Stewart.
>
>
>"Tom Kaminski [MVP]" wrote:
>
>>"Stewart" <Stewart@discussions.microsoft.com> wrote in message
>>news:B72AD4A6-83AF-47E9-913A-C7931331FA54@microsoft.com...
>>>- Now there are two different log files - the replication service can't
>>>just
>>>copy one over the other as log file entries will be lost.
>>
>>Right!
>>
>>>How would the replication service deal with this kind of situation? That
>>>is
>>>why I ask whether it is even a good idea to use replication on log files?
>>
>>No - you have to use a log analyzer that's capable of dealing with
>>multiple
>>log files. The log files from each server are different and need to be
>>analyzed in aggregate.
>>
>>--
>>Tom Kaminski IIS MVP
>>http://www.microsoft.com/windowsserver2003/community/centers/iis/
>>http://mvp.support.microsoft.com/
>>http://www.iistoolshed.com/ - tools, scripts, and utilities for running
>>IIS
>>
>>
>>