What tool can I use for IIS v5 that will allow me to save/view HTTP
request headers?

I'm having a problem with msnbot and IIS giving status codes of 406
for .html files and I need to know the accept type header that msnbot
is sending when it tries to retrieve .html files.

Obviously I dont want to save headers for all clients so I'd like to
be able to do some kind of selective screening based on the user-agent
header.

Re: How to view/save headers from certain clients by Chris

Chris
Thu Aug 04 14:23:42 CDT 2005

On Thu, 04 Aug 2005 11:14:43 -0400, Clyde <b26440510@DELETEyahoo.com>
wrote:

I do not know of a product that can do this for you - but you could
depending on traffic implement a packet sniffer and have that capture
packets and then view then later.

Is it a very busy site? Does MSN Bot always appear around the same
time?

See this article http://iisfaq.com/Default.aspx?tabid=2976 for some
help with the ethereal product which is freely downloadable.

The details in the article are now a few versions behind but the basis
of it should be fine.

I know it has some powerful filtering techniques and you may be able
to filter on a specific bit of text but I am not sure on this.

You may be better to do this with display filters after you capture
some data.

Cheers

Chris

Chris Crowe [IIS MVP]

>What tool can I use for IIS v5 that will allow me to save/view HTTP
>request headers?
>
>I'm having a problem with msnbot and IIS giving status codes of 406
>for .html files and I need to know the accept type header that msnbot
>is sending when it tries to retrieve .html files.
>
>Obviously I dont want to save headers for all clients so I'd like to
>be able to do some kind of selective screening based on the user-agent
>header.

Re: How to view/save headers from certain clients by Clyde

Clyde
Thu Aug 04 16:26:32 CDT 2005

I redirected the index.html page to a JSP page that saves the headers
to a file and then issues a redirect to index.htm (a copy of
index.html), preventing a redirect deadlock.

MSNBot visited today and the header accept type info is what I
reported in another recent thread (MSNBOT causes 406 error),
text/html, text/plain, application/*

Don't know why msnbot and IIS can't play nice with html files. If
anyone has any ideas, let me know.



"Chris Crowe [IIS MVP]" <IISMVP2005@iisfaq.homeip.net> wrote:

>On Thu, 04 Aug 2005 11:14:43 -0400, Clyde <b26440510@DELETEyahoo.com>
>wrote:
>
>I do not know of a product that can do this for you - but you could
>depending on traffic implement a packet sniffer and have that capture
>packets and then view then later.
>
>Is it a very busy site? Does MSN Bot always appear around the same
>time?
>
>See this article http://iisfaq.com/Default.aspx?tabid=2976 for some
>help with the ethereal product which is freely downloadable.
>
>The details in the article are now a few versions behind but the basis
>of it should be fine.
>
>I know it has some powerful filtering techniques and you may be able
>to filter on a specific bit of text but I am not sure on this.
>
>You may be better to do this with display filters after you capture
>some data.
>
>Cheers
>
>Chris
>
>Chris Crowe [IIS MVP]
>
>>What tool can I use for IIS v5 that will allow me to save/view HTTP
>>request headers?
>>
>>I'm having a problem with msnbot and IIS giving status codes of 406
>>for .html files and I need to know the accept type header that msnbot
>>is sending when it tries to retrieve .html files.
>>
>>Obviously I dont want to save headers for all clients so I'd like to
>>be able to do some kind of selective screening based on the user-agent
>>header.