I am writing a C/C++ application that opens a set of files and reads/writes
to these files in chunks/blocks called pages.
There are two specific types of files used to store different sets of
information. There are data files used to store the logical data records
and index files used to store index references to the logical data
records in the data file.
The data file pagesize is determined by whether its storing variable or
equal size records. For this discussion, we can ignore variable length
data files and assume that data file pagesize is equal to a record size.
For the index files, the pagesize is equal to the disk's block size and
the index engine uses this to determine the number of index entries
that can exist on a single page, etc.
NOTE: The application that is handling the access to these files is
running multiple threads, so critical sections will be used to
aid in the reader/writer paradigm.
Now, if I want to be able to prevent unnecessary disk reads by implementing
some form of a page cache, is it worthwhile to spend time on developing
this "cache" functionality myself or utilize the win32 memory mapped file
logic to handle this?
By allowing the operating system to handle this, it means that I do not
have to worry about it and that I just access the file as if it were
memory pointers; however the limitation is that if the file needs to be
grown because a new record is to be added, then the file has to be
locked, unmapped, grown, remapped, and unlocked for the next operation
to take place.
By handling this myself, I then have to read the block of information
from disk, store it in memory and be able to access the memory buffer
when a thread references the page and occationally write dirty memory
buffers to disk. But this gives me the flexibility of growing the
files as needed without having to unmap/remap the file.
Both cases have their pros/cons and I was curious what others thought
would be the right avenue to go.
Thanks in advance!
Chris