Any point in pre-creating a file for write performance?

Hi!

I'm creating a data accusition system which needs to dump data to disk,
basically as fast as possible.

I've used aligned buffers (VirtualAlloc) and overlapped unbuffered IO
so far. The write speed is good, probably as good as it will get on my
system (~35 MB/sec).

I sort of expected to see a difference in speed when overwriting an
already created file versus writing to a new file (in the favor of the
already created file) because the old one would already be 'laid out'
on disk. Is this the case? I dont see a difference, but I got a lot of
space on my HD. Would it matter if the disk was almost full and badly
fragmented?

Thanks!

Jim
Tue Feb 07 15:30:19 CST 2006

"SIS" <scammers_u_wont_get_me@hotmail.com> wrote in message
news:1139347485.799415.168910@g47g2000cwa.googlegroups.com...
> Hi!
>
> I'm creating a data accusition system which needs to dump data to disk,
> basically as fast as possible.
>
> I've used aligned buffers (VirtualAlloc) and overlapped unbuffered IO
> so far. The write speed is good, probably as good as it will get on my
> system (~35 MB/sec).
>
> I sort of expected to see a difference in speed when overwriting an
> already created file versus writing to a new file (in the favor of the
> already created file) because the old one would already be 'laid out'
> on disk. Is this the case? I dont see a difference, but I got a lot of
> space on my HD. Would it matter if the disk was almost full and badly
> fragmented?
>
> Thanks!
>

Depends on the OS, but in reality writing over an existing file should be a
little slower, since the OS has to delete the existing file first.

The time taken to delete a file depends on the OS and the size of the file.

Generally to write over an existing file the OS simply deletes the existing
file, then creates a new file with the same name.

SIS
Thu Feb 09 07:00:45 CST 2006

Thanks Jim! I can agree if you recreate the file at opening, but if you
open an existing file for writing and do a partial write, I think the
rest of the file should remain unchanged? (i.e. it does not get
deleted)

Anyway, I have searched a bit around and found the following articles
that touches into this topic.

http://groups.google.com/group/comp.os.ms-windows.programmer.win32/browse_thread/thread/b0131b5638cb2f0e/6ac1c760cbcb4edb?lnk=st&q=WriteFile+SetFilePointer+SetEndOfFile+blocks&rnum=4&hl=en#6ac1c760cbcb4edb
http://groups.google.com/group/comp.os.ms-windows.programmer.win32/browse_thread/thread/588c74e9d9276020/b4e20485fcfd084f?lnk=st&q=WriteFile+SetFilePointer+SetEndOfFile&rnum=2&hl=en#b4e20485fcfd084f

Carl
Thu Feb 09 08:53:02 CST 2006

Jim Langston wrote:
>
> Depends on the OS, but in reality writing over an existing file
> should be a little slower, since the OS has to delete the existing
> file first.
> The time taken to delete a file depends on the OS and the size of the
> file.
> Generally to write over an existing file the OS simply deletes the
> existing file, then creates a new file with the same name.

Not so. That's only the case if you open the file with CREATE_ALWAYS.
Otherwise, if you open an existing file and start to write to it, writes
will overwrite the existing extents of the file. Note that if the file is
stored on a compressed (or encrypted) volume, then writing to an existing
file is almost certain to be slower as writing to such files always entils
allocation of new extents and may require desompressing/decrypting the
existing content if the file isn't being written in compressed extent blocks
(16K, IIRC).

-cd

Carl
Thu Feb 09 08:54:19 CST 2006

SIS wrote:
> Hi!
>
> I'm creating a data accusition system which needs to dump data to
> disk, basically as fast as possible.
>
> I've used aligned buffers (VirtualAlloc) and overlapped unbuffered IO
> so far. The write speed is good, probably as good as it will get on my
> system (~35 MB/sec).
>
> I sort of expected to see a difference in speed when overwriting an
> already created file versus writing to a new file (in the favor of the
> already created file) because the old one would already be 'laid out'
> on disk. Is this the case? I dont see a difference, but I got a lot of
> space on my HD. Would it matter if the disk was almost full and badly
> fragmented?

I woulnd't expect to see a big difference, but on a highly fragmented disk
you are indeed likely to see more effect.

-cd