Adodb.Stream - Problem download big files

Hi!


I want to use ASP to download big files using ADODB.STREAM. It works very
fine with files smaller than 80 MB.
On the Webserver I can see that memory allocation and the process w3wp is
running. After some time (more or less 2 minutes) I get a response timeout.

Here is the code:

Server.ScriptTimeout = 30000
Response.Buffer = True

Response.Clear
Response.Expires = 0
Response.ContentType = "Download-File"
Response.AddHeader "Content-Disposition","attachment; filename=" & sfile

Set oStream = Server.CreateObject("ADODB.Stream")
oStream.Type = adTypeBinary
oStream.Open
oStream.LoadFromFile(sfile)
Response.AddHeader "Content-Length", oStream.Size ' -- Schönheit
Response.CharSet = "UTF-8"

For i = 0 To oStream.Size
i = i + 128000
Response.BinaryWrite(oStream.Read(128000))
Response.Flush

Next

oStream.Close
Set oStream = Nothing
Response.Flush
Response.End

Do I have to change something in my code - or perhaps a general setting in
IIS / the metabase?

Many thanks in advance



Juan

Anthony
Thu Jul 03 08:33:27 CDT 2008

"Juan" <Juan@discussions.microsoft.com> wrote in message
news:719A642C-56CF-49D0-9453-B75FC406E666@microsoft.com...
> Hi!
>
>
> I want to use ASP to download big files using ADODB.STREAM. It works very
> fine with files smaller than 80 MB.
> On the Webserver I can see that memory allocation and the process w3wp is
> running. After some time (more or less 2 minutes) I get a response
timeout.
>
> Here is the code:
>
> Server.ScriptTimeout = 30000

30,000 seconds is a long script time out ;)

> Response.Buffer = True

Change to False, you don't want to buffer anyway (yes I know you are
flushing but this is simpler).

>
> Response.Clear
> Response.Expires = 0
> Response.ContentType = "Download-File"

This is not a valid content type if you don't know what the content type is
use the fallback type: "application/octet-stream"

> Response.AddHeader "Content-Disposition","attachment; filename=" & sfile
>
> Set oStream = Server.CreateObject("ADODB.Stream")
> oStream.Type = adTypeBinary
> oStream.Open
> oStream.LoadFromFile(sfile)
> Response.AddHeader "Content-Length", oStream.Size ' -- Schönheit

Don't add the Content-Length header its a duplication, ASP adds that for
you.

> Response.CharSet = "UTF-8"

How come you know the character encoding but don't know the content type?
Consider deleting this line or setting the content-type to the correct text
type.


>
> For i = 0 To oStream.Size
> i = i + 128000
> Response.BinaryWrite(oStream.Read(128000))
> Response.Flush

No need for a flush when buffering is off.

>
> Next
>
> oStream.Close
> Set oStream = Nothing
> Response.Flush
> Response.End

Again no need for a flush and .End is draconian, only use it if you know
something bad will happen if you don't. Even then design out the something
bad rather than use .End.

>
> Do I have to change something in my code - or perhaps a general setting in
> IIS / the metabase?
>

The response timeout is a clientside thing however normally response
timeouts related to the time it takes to get the first chunk of data not the
overall send time.



--
Anthony Jones - MVP ASP/ASP.NET

Daniel
Thu Jul 03 11:14:31 CDT 2008

Anthony wrote on Thu, 3 Jul 2008 14:33:27 +0100:

> "Juan" <Juan@discussions.microsoft.com> wrote in message
> news:719A642C-56CF-49D0-9453-B75FC406E666@microsoft.com...
>> Hi!


>> I want to use ASP to download big files using ADODB.STREAM. It works
>> very fine with files smaller than 80 MB.
>> On the Webserver I can see that memory allocation and the process
>> w3wp is running. After some time (more or less 2 minutes) I get a
>> response
> timeout.


>> Set oStream = Server.CreateObject("ADODB.Stream")
>> oStream.Type = adTypeBinary oStream.Open oStream.LoadFromFile(sfile)
>> Response.AddHeader "Content-Length", oStream.Size ' -- Schönheit

> Don't add the Content-Length header its a duplication, ASP adds that
> for you.

If the stream is being written out in chunks (which it is in the For Next
loop), how does the ASP engine know what the final size will be?

--
Dan

Daniel
Thu Jul 03 11:22:58 CDT 2008

Juan wrote on Thu, 3 Jul 2008 06:00:01 -0700:

> Hi!


> I want to use ASP to download big files using ADODB.STREAM. It works
> very fine with files smaller than 80 MB.
> On the Webserver I can see that memory allocation and the process w3wp
> is running. After some time (more or less 2 minutes) I get a response
> timeout.

> Here is the code:

> Server.ScriptTimeout = 30000
> Response.Buffer = True

> Response.Clear
> Response.Expires = 0
> Response.ContentType = "Download-File"
> Response.AddHeader "Content-Disposition","attachment; filename=" &
> sfile

> Set oStream = Server.CreateObject("ADODB.Stream")
> oStream.Type = adTypeBinary oStream.Open oStream.LoadFromFile(sfile)
> Response.AddHeader "Content-Length", oStream.Size ' -- Schönheit
> Response.CharSet = "UTF-8"

> For i = 0 To oStream.Size
> i = i + 128000
> Response.BinaryWrite(oStream.Read(128000))
> Response.Flush
> Next

This looks odd - you're looping from 0 to the size of the stream, yet you're
pushing out 128000 bytes at a time. I think you should be using

For i = 0 To oStream.Size Step 128000


Also, in your loop what happens when you reach the end of the stream?


This is how I handle download from one of my applications



Set oStream = Server.CreateObject("ADODB.Stream")
Call oStream.Open()
oStream.Type = 1
call oStream.LoadFromFile(strFilename)

iDownload = 1
If lcase(right(strfilename,4)) = ".pdf" then
Response.ContentType = "application/pdf"
else
Response.ContentType = "application/octet-stream"
end if
Response.AddHeader "Content-Disposition", "filename=" &
strfilename & ";"

Response.Buffer = False

'stream out the file in chunks
Do While Not (oStream.EOS)
Response.BinaryWrite oStream.Read(1024 * 256)
Loop

oStream.Close
Set oStream = Nothing


this reads the file into the stream (just like you already have), sets an
appropriate ContentType (as Anthony points out the one you use is not a
recognised MIME type), turns off buffering, and then writes out the stream
in 256kB chunks until there is nothing left to write to out (using
oStream.EOS to determine if the stream is at the end).

You really do need buffering turned off - it's entirely possible that the
80MB file is going over the defined ASP buffer limit and so you're getting
an ASP error thrown into the data, and that causes what appears to be a
timeout but is actually due to ASP terminating the script because it has
errored. So far with the above code I've had no trouble with files larger
than the defined ASP buffer limit, but I haven't tried it with anything as
large as your file.

--
Dan

Daniel
Thu Jul 03 11:31:59 CDT 2008

Daniel wrote to Anthony Jones on Thu, 3 Jul 2008 17:14:31 +0100:

> Anthony wrote on Thu, 3 Jul 2008 14:33:27 +0100:

>> "Juan" <Juan@discussions.microsoft.com> wrote in message
>> news:719A642C-56CF-49D0-9453-B75FC406E666@microsoft.com...
>>> Hi!


>>> I want to use ASP to download big files using ADODB.STREAM. It works
>>> very fine with files smaller than 80 MB.
>>> On the Webserver I can see that memory allocation and the process
>>> w3wp is running. After some time (more or less 2 minutes) I get a
>>> response
>> timeout.


>>> Set oStream = Server.CreateObject("ADODB.Stream")
>>> oStream.Type = adTypeBinary oStream.Open oStream.LoadFromFile(sfile)
>>> Response.AddHeader "Content-Length", oStream.Size ' -- Schönheit

>> Don't add the Content-Length header its a duplication, ASP adds that
>> for you.

> If the stream is being written out in chunks (which it is in the For
> Next loop), how does the ASP engine know what the final size will be?

Actually, I've just realised that the code I use myself doesn't set the
Content-Length header either. ASP doesn't add the Content-Length header
itself either as it has no idea what the file size is when it starts
outputting it, here's an example of the ASP headers from a download from my application:

HTTP/1.1 200 OK
Cache-Control: private
Date: Thu, 03 Jul 2008 16:26:53 GMT
Transfer-Encoding: chunked
Content-Type: application/pdf
Server: Microsoft-IIS/6.0
Content-Disposition: filename=Certifications_at_a_Glance.pdf;


This just means that you don't get the status bar in IE, for instance,
showing the download progress because the final size is not known. Adding
the Content-Length header allows IE (and any other browser that will show a
status bar) to give a progress with a final size value. I tested adding the
content length header to my own application and the headers changed to:

HTTP/1.1 200 OK
Cache-Control: private
Date: Thu, 03 Jul 2008 16:29:46 GMT
Transfer-Encoding: chunked
Content-Length: 1890660
Content-Type: application/pdf
Server: Microsoft-IIS/6.0
Content-Disposition: filename=Certifications_at_a_Glance.pdf;

and appeared to have no detrimental effect on the download.

--
Dan

Daniel
Thu Jul 03 11:37:28 CDT 2008

Daniel wrote to Juan on Thu, 3 Jul 2008 17:22:58 +0100:

> Juan wrote on Thu, 3 Jul 2008 06:00:01 -0700:

>> Hi!


>> I want to use ASP to download big files using ADODB.STREAM. It works
>> very fine with files smaller than 80 MB.
>> On the Webserver I can see that memory allocation and the process
>> w3wp is running. After some time (more or less 2 minutes) I get a
>> response timeout.

>> Here is the code:

>> Server.ScriptTimeout = 30000
>> Response.Buffer = True

>> Response.Clear
>> Response.Expires = 0
>> Response.ContentType = "Download-File"
>> Response.AddHeader "Content-Disposition","attachment; filename=" &
>> sfile

>> Set oStream = Server.CreateObject("ADODB.Stream")
>> oStream.Type = adTypeBinary oStream.Open
>> oStream.LoadFromFile(sfile)
>> Response.AddHeader "Content-Length", oStream.Size ' -- Schönheit
>> Response.CharSet = "UTF-8"

>> For i = 0 To oStream.Size i = i + 128000
>> Response.BinaryWrite(oStream.Read(128000))
>> Response.Flush
>> Next

> This looks odd - you're looping from 0 to the size of the stream, yet
> you're pushing out 128000 bytes at a time. I think you should be using

I actually had another idea as to why it might be "timing out" - for 80MB of
data, your loop is running 83886080 times (80 * 1024 * 1024). If you are
using 128kB chunks, it only needs 666 loops to send 80MB. Check your event
logs and see if you have any ASP or IIS events pointing to the process being
terminated.

--
Dan

Anthony
Thu Jul 03 16:08:09 CDT 2008

"Daniel Crichton" <msnews@worldofspack.com> wrote in message
news:eyCiVsS3IHA.2348@TK2MSFTNGP06.phx.gbl...
> Daniel wrote to Juan on Thu, 3 Jul 2008 17:22:58 +0100:
>
> > Juan wrote on Thu, 3 Jul 2008 06:00:01 -0700:
>
> >> Hi!
>
>
> >> I want to use ASP to download big files using ADODB.STREAM. It works
> >> very fine with files smaller than 80 MB.
> >> On the Webserver I can see that memory allocation and the process
> >> w3wp is running. After some time (more or less 2 minutes) I get a
> >> response timeout.
>
> >> Here is the code:
>
> >> Server.ScriptTimeout = 30000
> >> Response.Buffer = True
>
> >> Response.Clear
> >> Response.Expires = 0
> >> Response.ContentType = "Download-File"
> >> Response.AddHeader "Content-Disposition","attachment; filename=" &
> >> sfile
>
> >> Set oStream = Server.CreateObject("ADODB.Stream")
> >> oStream.Type = adTypeBinary oStream.Open
> >> oStream.LoadFromFile(sfile)
> >> Response.AddHeader "Content-Length", oStream.Size ' -- Schönheit
> >> Response.CharSet = "UTF-8"
>
> >> For i = 0 To oStream.Size i = i + 128000
> >> Response.BinaryWrite(oStream.Read(128000))
> >> Response.Flush
> >> Next
>
> > This looks odd - you're looping from 0 to the size of the stream, yet
> > you're pushing out 128000 bytes at a time. I think you should be using
>
> I actually had another idea as to why it might be "timing out" - for 80MB
of
> data, your loop is running 83886080 times (80 * 1024 * 1024). If you are
> using 128kB chunks, it only needs 666 loops to send 80MB. Check your event
> logs and see if you have any ASP or IIS events pointing to the process
being
> terminated.
>

The loop is incrementing 128001 every iteration so potentially it may not
send all the file

--
Anthony Jones - MVP ASP/ASP.NET

Anthony
Thu Jul 03 16:17:13 CDT 2008

"Daniel Crichton" <msnews@worldofspack.com> wrote in message
news:eJHPSpS3IHA.1428@TK2MSFTNGP06.phx.gbl...
> Daniel wrote to Anthony Jones on Thu, 3 Jul 2008 17:14:31 +0100:
>
> > Anthony wrote on Thu, 3 Jul 2008 14:33:27 +0100:
>
> >> "Juan" <Juan@discussions.microsoft.com> wrote in message
> >> news:719A642C-56CF-49D0-9453-B75FC406E666@microsoft.com...
> >>> Hi!
>
>
> >>> I want to use ASP to download big files using ADODB.STREAM. It works
> >>> very fine with files smaller than 80 MB.
> >>> On the Webserver I can see that memory allocation and the process
> >>> w3wp is running. After some time (more or less 2 minutes) I get a
> >>> response
> >> timeout.
>
>
> >>> Set oStream = Server.CreateObject("ADODB.Stream")
> >>> oStream.Type = adTypeBinary oStream.Open oStream.LoadFromFile(sfile)
> >>> Response.AddHeader "Content-Length", oStream.Size ' -- Schönheit
>
> >> Don't add the Content-Length header its a duplication, ASP adds that
> >> for you.
>
> > If the stream is being written out in chunks (which it is in the For
> > Next loop), how does the ASP engine know what the final size will be?
>
> Actually, I've just realised that the code I use myself doesn't set the
> Content-Length header either. ASP doesn't add the Content-Length header
> itself either as it has no idea what the file size is when it starts
> outputting it, here's an example of the ASP headers from a download from
my application:
>
> HTTP/1.1 200 OK
> Cache-Control: private
> Date: Thu, 03 Jul 2008 16:26:53 GMT
> Transfer-Encoding: chunked
> Content-Type: application/pdf
> Server: Microsoft-IIS/6.0
> Content-Disposition: filename=Certifications_at_a_Glance.pdf;
>
>
> This just means that you don't get the status bar in IE, for instance,
> showing the download progress because the final size is not known. Adding
> the Content-Length header allows IE (and any other browser that will show
a
> status bar) to give a progress with a final size value. I tested adding
the
> content length header to my own application and the headers changed to:
>
> HTTP/1.1 200 OK
> Cache-Control: private
> Date: Thu, 03 Jul 2008 16:29:46 GMT
> Transfer-Encoding: chunked
> Content-Length: 1890660
> Content-Type: application/pdf
> Server: Microsoft-IIS/6.0
> Content-Disposition: filename=Certifications_at_a_Glance.pdf;
>
> and appeared to have no detrimental effect on the download.
>

You're right it doesn't have any detrimental affect on IE in fact it serves
a useful purpose. However it is a strictly a breach of the protocol See
http://www.w3.org/Protocols/rfc2616/rfc2616-sec4.html#sec4.4 para 3.
However the protocol does then indicate that if it is Content-Length is
present it should be ignored in this case.


--
Anthony Jones - MVP ASP/ASP.NET

Anthony
Thu Jul 03 16:20:21 CDT 2008

"Juan" <Juan@discussions.microsoft.com> wrote in message
news:719A642C-56CF-49D0-9453-B75FC406E666@microsoft.com...
> Hi!
>
>
> I want to use ASP to download big files using ADODB.STREAM. It works very
> fine with files smaller than 80 MB.
> On the Webserver I can see that memory allocation and the process w3wp is
> running. After some time (more or less 2 minutes) I get a response
timeout.
>
> Here is the code:
>
> Server.ScriptTimeout = 30000
> Response.Buffer = True
>
> Response.Clear
> Response.Expires = 0
> Response.ContentType = "Download-File"
> Response.AddHeader "Content-Disposition","attachment; filename=" & sfile
>
> Set oStream = Server.CreateObject("ADODB.Stream")
> oStream.Type = adTypeBinary
> oStream.Open
> oStream.LoadFromFile(sfile)
> Response.AddHeader "Content-Length", oStream.Size ' -- Schönheit
> Response.CharSet = "UTF-8"
>
> For i = 0 To oStream.Size
> i = i + 128000
> Response.BinaryWrite(oStream.Read(128000))
> Response.Flush
>
> Next
>
> oStream.Close
> Set oStream = Nothing
> Response.Flush
> Response.End
>
> Do I have to change something in my code - or perhaps a general setting in
> IIS / the metabase?
>
> Many thanks in advance
>


FYI this is the routine I use for sending a file to response.

Sub SendFileToResponse(FilePath, FileName)

Const clChunkSize = 1048576 ' 1MB

Dim oStream, i
Response.Buffer = False

Response.ContentType = "application/octet-stream"
Response.AddHeader "Content-Disposition", _
"attachment; Filename=" & FileName

Set oStream = Server.CreateObject("ADODB.Stream")
oStream.Type = 1 ' Binary
oStream.Open
oStream.LoadFromFile FilePath

For i = 1 To oStream.Size \ clChunkSize
Response.BinaryWrite oStream.Read(clChunkSize)
Next
If (oStream.Size Mod clChunkSize) <> 0 Then
Response.BinaryWrite oStream.Read(oStream.Size Mod clChunkSize)
End If
oStream.Close

End Sub

1MB buffer is a bit on the big side though 128K is a good choice.

--
Anthony Jones - MVP ASP/ASP.NET

Evertjan
Fri Jul 04 01:59:50 CDT 2008

Anthony Jones wrote on 03 jul 2008 in
microsoft.public.inetserver.asp.general:

> FYI this is the routine I use for sending a file to response.
>
> Sub SendFileToResponse(FilePath, FileName)
>
> Const clChunkSize = 1048576 ' 1MB
>
> Dim oStream, i
> Response.Buffer = False
>
> Response.ContentType = "application/octet-stream"
> Response.AddHeader "Content-Disposition", _
> "attachment; Filename=" & FileName
>
> Set oStream = Server.CreateObject("ADODB.Stream")
> oStream.Type = 1 ' Binary
> oStream.Open
> oStream.LoadFromFile FilePath
>
> For i = 1 To oStream.Size \ clChunkSize
> Response.BinaryWrite oStream.Read(clChunkSize)
> Next
> If (oStream.Size Mod clChunkSize) <> 0 Then
> Response.BinaryWrite oStream.Read(oStream.Size Mod clChunkSize)
> End If

Would an error be given if an empty chunk or is sent?
And if the requested chunk size is larger than the remainder?

If not I would do:

For i = 0 To oStream.Size \ clChunkSize
Response.BinaryWrite oStream.Read(clChunkSize)
Next

> oStream.Close
>
> End Sub
>
> 1MB buffer is a bit on the big side though 128K is a good choice.
>



--
Evertjan.
The Netherlands.
(Please change the x'es to dots in my emailaddress)

AlhambraEidosKiquenet
Tue Jul 08 08:04:00 CDT 2008

Mister, any solution about it ??

any sample code that works fine ?? please

thanks in advance...

--
http://www.alhambra-eidos.es/web2005/index.html
www.kiquenet.net
http://www.setbb.com/putainformatica/viewtopic.php?p=843
www.trabajobasura.com/solusoft



"Juan" wrote:

> Hi!
>
>
> I want to use ASP to download big files using ADODB.STREAM. It works very
> fine with files smaller than 80 MB.
> On the Webserver I can see that memory allocation and the process w3wp is
> running. After some time (more or less 2 minutes) I get a response timeout.
>
> Here is the code:
>
> Server.ScriptTimeout = 30000
> Response.Buffer = True
>
> Response.Clear
> Response.Expires = 0
> Response.ContentType = "Download-File"
> Response.AddHeader "Content-Disposition","attachment; filename=" & sfile
>
> Set oStream = Server.CreateObject("ADODB.Stream")
> oStream.Type = adTypeBinary
> oStream.Open
> oStream.LoadFromFile(sfile)
> Response.AddHeader "Content-Length", oStream.Size ' -- Schönheit
> Response.CharSet = "UTF-8"
>
> For i = 0 To oStream.Size
> i = i + 128000
> Response.BinaryWrite(oStream.Read(128000))
> Response.Flush
>
> Next
>
> oStream.Close
> Set oStream = Nothing
> Response.Flush
> Response.End
>
> Do I have to change something in my code - or perhaps a general setting in
> IIS / the metabase?
>
> Many thanks in advance
>
>
>
> Juan