Willy
Sun Mar 05 06:40:49 CST 2006
"Very worse", compared to what? your program can never run to an end on a 32
bit system where you only have access to 2GB of user space per process.
The free user space on a 32Bit system after the CLR, framework code and data
has been loaded is less than 1.8GB, so you memory requirements exceed
largely the available memory. Note that your data requirements will double
when converting the text file 'byte' oriented data into XML 'string'
oriented data, add to that the XML overhead and you will understand that you
simply can't hold all this data in memory on a 32 bit system.
So your options are:
- Split the input file into smaller portions, or
- Store the input data in a relational database, or
- Move to a 64 bit OS and add another 2GB of RAM.
Willy.
"Venkatachalam" <Venkatachalam@discussions.microsoft.com> wrote in message
news:EB65109B-A44F-43DD-A693-48EC1DF00419@microsoft.com...
| All the data has to be in the momory, as there are very complex relation
| involved among the tables.
|
| If I split the file to read, then the performance will go ver very worst.
|
| And also the server has 4 Intel Xeon CPU 3.00GHz processor.
|
| With regards
| Venkat
|
| "Jon Skeet [C# MVP]" wrote:
|
| > Venkatachalam <Venkatachalam@discussions.microsoft.com> wrote:
| > > In my application I have text(flat) file as input and I have to
generate
| > > an XML file. The maximum input text file size can be 900MB and
gererated xml
| > > may result 2+ GB.
| >
| > Do you really need to hold all of that in memory at the same time? It
| > sounds like it would be much more likely to work if you could process
| > the file writing out data as you went, even if you need to process the
| > input file a few times.
| >
| > <snip>
| >
| > > When the application process a 500MB file, it throws
OutOfMemoryException.
| > > In the task manager, the utilised memory when the system throws
| > > OutOfMemoryException is 2.15 GB.
| >
| > I believe that without some extra support (which I don't know the
| > details of) a single process can't go above 2GB.
| >
| > > But in my desktop, for the surprise, the system has 512 MB and when I
load
| > > several application, the process memory reached 1GB without any
| > > OutOfMemoryException issues.
| > >
| > > Can you one help me in this regard.
| >
| > That's your desktop swapping memory out to disk. Note that when this
| > happens, anything which tries to use that virtual memory will slow down
| > hugely.
| >
| > --
| > Jon Skeet - <skeet@pobox.com>
| >
http://www.pobox.com/~skeet Blog:
http://www.msmvps.com/jon.skeet
| > If replying to the group, please do not mail me too
| >