Disabling chunking in HTTP 1.1

Hi,

I'm not sure if this is the right place to post this but when
searching for the exception I got this group came up the most.

I'm making a HttpWebRequest to get data from a web page but am getting
the following error

System.Net.WebException: The server committed a protocol violation.
Section=ResponseBody Detail=Response chunk format is invalid
at System.Net.HttpWebRequest.GetResponse()

I've looked in the groups and the recomended fix for this error is to
set

<system.net>
<settings>
<httpWebRequest useUnsafeHeaderParsing="true" />
</settings>
</system.net>

in the App.config but this isnt working for me, playing around I find
that if I set the version in the HttpWebRequest to 1.0 like so

HttpWebRequest hwr = (HttpWebRequest)
HttpWebRequest.Create(url.Text);
hwr.ProtocolVersion = new System.Version(1,0);

then everything runs fine, running a packet sniffer I see this is
becuase I'm not recieving the data chunked, is there a header I can
set so I can carry on using 1.1 but not get chunked data as this is
obviously whats causing the problem and I dont really want to use http
1.0.

Failing that does anyone know what is causing this problem?

Joerg
Wed Feb 14 13:05:51 CST 2007

Thus wrote joseph_gallagher@hotmail.com,

> Hi,
>
> I'm not sure if this is the right place to post this but when
> searching for the exception I got this group came up the most.

It's the right place :-)

> I'm making a HttpWebRequest to get data from a web page but am getting
> the following error
>
> System.Net.WebException: The server committed a protocol violation.
> Section=ResponseBody Detail=Response chunk format is invalid
> at System.Net.HttpWebRequest.GetResponse()
> I've looked in the groups and the recomended fix for this error is to
> set
>
> <system.net>
> <settings>
> <httpWebRequest useUnsafeHeaderParsing="true" />
> </settings>
> </system.net>
> in the App.config but this isnt working for me, playing around I find
> that if I set the version in the HttpWebRequest to 1.0 like so

That would be a severe bug on the server-side. Is this a public site? Or
is there a web proxy in between?

> HttpWebRequest hwr = (HttpWebRequest)
> HttpWebRequest.Create(url.Text);
> hwr.ProtocolVersion = new System.Version(1,0);
> then everything runs fine, running a packet sniffer I see this is
> becuase I'm not recieving the data chunked, is there a header I can
> set so I can carry on using 1.1 but not get chunked data as this is
> obviously whats causing the problem and I dont really want to use http
> 1.0.

Understandable. But it's the server's decision whether to use chunking or
not. A HTTP 1.1 client must support chunking, so there's no way to avoid
chunking without reverting to HTTP 1.0.

> Failing that does anyone know what is causing this problem?

Not without having a look at the actual traffic.

Cheers,
--
Joerg Jooss
news-reply@joergjooss.de

joseph_gallagher
Thu Feb 15 05:35:52 CST 2007

Thanks for the reply Joerg, it is a public server and I'm not going
through any proxies so it is strange, its also strange that it only
happens randomly but quiet frequently, around 50% of the time, some
sample code that calls the page in question is

[STAThread]
public static void Main() {
string url = "http://site.sports.betfair.com/menu/
LoadMenuNodesAction.do?
sReturnPath=parent.frames['menu']&method=getMenuEvents&menuNodeId=11589568&strArrayName=allSkeletonArray&iParentID=11589568&layerName=allMarketsTreeContainer&strMenuPathArrayName=allPathArray&menuPathLayer=menuParents1&locale=en_GB";
for(int i = 0 ; i < 10 ; i++) {
HttpWebRequest hwr = (HttpWebRequest)
HttpWebRequest.Create(url);
try {
using(WebResponse wr = hwr.GetResponse()) {
using(Stream s = wr.GetResponseStream()) {
using(StreamReader sr = new
StreamReader(s)) {
Console.WriteLine(sr.ReadToEnd());
}
}
}
} catch(WebException we) {
Console.WriteLine(we.ToString());
}
Console.ReadLine();
}
}

Running a packet sniffer I can see that the correct text is been
returned but I'm not sure if it is been chunked correctly, its also
strange that the url works 100% in IE.

Thanks for any help, if you dont have time to look at it further dont
worry about it, usuing 1.0 is an acceptable alternative as its a call
that is only used occasionaly, I'm more just curious than anything
else.

Joe.

Alan
Thu Feb 15 08:06:28 CST 2007

In article news:1171539351.985007.227360@j27g2000cwj.googlegroups.com,
joseph_gallagher@hotmail.com wrote:
> Thanks for the reply Joerg, it is a public server and I'm not going
> through any proxies so it is strange, its also strange that it only
> happens randomly but quiet frequently, around 50% of the time, some
> sample code that calls the page in question is
>
Seems to work ok always for me, at least I've not repro'd any error in
the number of times that I've run it... Maybe someone else will find
different. There's no chance there's a transparent proxy in your path,
see if there's an Via header in the response for instance.

If you've a sniffer trace of case where it fails can you upload that
somewhere and we can have a look to see if there's something obviously
wrong with the response.

The format of chunked is relatively simple { n n n CR LF {data of size
nnn} CR LF } for each chunk, with the end marked with a zero length
chunk, and the length 'nnn' is an ascii formatted number
so the first trace I have has this response:

HTTP/1.1 200 ....
...
Flags: ... BRefresh=onCRLF
CRLF
65CRLF
{65 bytes of content}CRLF
289CRLF
{289 bytes of content}CRLF
0CRLF
CRLF

Seems ok -- should do as it was a successful download. (Not too sure
about the very last CRLF, but...)
--
Alan J. McFarlane
http://www.alanjmcf.me.uk/
Please follow-up in the newsgroup for the benefit of all.