Using Comma Separated Values to de-serialize a DataSet

Subject: Using Comma Separated Values to de-serialize a DataSet

The DataSet.ReadXml() speed under Netcf1.0 is slow. It results in our App
taking over 60 seconds to start. I have talked to many newsgroups, researched
articles, etc to find a way to improve this time. The best answer to date is
to upgrade to Netcf2.0. I am in the process of doing this but there may be
problems with updating the physical device.

One other possibility was mentioned on a newsgroup. That is to use Comma
Separated Values rather then Xml to load the DataSet. Does anyone know how
that would be done and if any code snippets or examples exist.

Thanks.

--
John Olbert

Philip
Wed Mar 01 17:35:43 CST 2006

You might want to consider using TSV format instead. It is easier to parse,
and you can give the files a txt extension, which sometimes helps.

"John Olbert" <someone@snet.net> wrote in message
news:5889C557-65A4-448D-A10D-439BCCBC8338@microsoft.com...
> Subject: Using Comma Separated Values to de-serialize a DataSet
>
> The DataSet.ReadXml() speed under Netcf1.0 is slow. It results in our App
> taking over 60 seconds to start. I have talked to many newsgroups,
> researched
> articles, etc to find a way to improve this time. The best answer to date
> is
> to upgrade to Netcf2.0. I am in the process of doing this but there may be
> problems with updating the physical device.
>
> One other possibility was mentioned on a newsgroup. That is to use Comma
> Separated Values rather then Xml to load the DataSet. Does anyone know how
> that would be done and if any code snippets or examples exist.
>
> Thanks.
>
> --
> John Olbert
>

Ginny
Wed Mar 01 20:16:00 CST 2006

John,

Look at the TextDataAdapter on www.opennetcf.org. Depending on the device
and the particular data, CSV can be twice as fast and half as bulky as XML.
But on Windows Mobile 5.0 devices running CF 2.0 that I've tested, the
performance gap between XML and CSV isn't so dramatic (or even apparent in
some cases.)

--
Ginny Caughey
.NET Compact Framework MVP


"John Olbert" <someone@snet.net> wrote in message
news:5889C557-65A4-448D-A10D-439BCCBC8338@microsoft.com...
> Subject: Using Comma Separated Values to de-serialize a DataSet
>
> The DataSet.ReadXml() speed under Netcf1.0 is slow. It results in our App
> taking over 60 seconds to start. I have talked to many newsgroups,
> researched
> articles, etc to find a way to improve this time. The best answer to date
> is
> to upgrade to Netcf2.0. I am in the process of doing this but there may be
> problems with updating the physical device.
>
> One other possibility was mentioned on a newsgroup. That is to use Comma
> Separated Values rather then Xml to load the DataSet. Does anyone know how
> that would be done and if any code snippets or examples exist.
>
> Thanks.
>
> --
> John Olbert
>

someone
Wed Mar 01 20:23:27 CST 2006

Sorry, TSV stands for?

Also any code snippets or samples on de-serializing a DataSet from a TSV.

Thanks.

--
John Olbert



"Philip Sheard" wrote:

> You might want to consider using TSV format instead. It is easier to parse,
> and you can give the files a txt extension, which sometimes helps.
>
> "John Olbert" <someone@snet.net> wrote in message
> news:5889C557-65A4-448D-A10D-439BCCBC8338@microsoft.com...
> > Subject: Using Comma Separated Values to de-serialize a DataSet
> >
> > The DataSet.ReadXml() speed under Netcf1.0 is slow. It results in our App
> > taking over 60 seconds to start. I have talked to many newsgroups,
> > researched
> > articles, etc to find a way to improve this time. The best answer to date
> > is
> > to upgrade to Netcf2.0. I am in the process of doing this but there may be
> > problems with updating the physical device.
> >
> > One other possibility was mentioned on a newsgroup. That is to use Comma
> > Separated Values rather then Xml to load the DataSet. Does anyone know how
> > that would be done and if any code snippets or examples exist.
> >
> > Thanks.
> >
> > --
> > John Olbert
> >
>
>
>

Paul
Thu Mar 02 10:22:48 CST 2006

I'm not 100% sure, but I'm guessing Tab-separated Values.

Paul T.

"John Olbert" <someone@snet.net> wrote in message
news:93883877-9B68-43EF-B066-EE5946CEE303@microsoft.com...
> Sorry, TSV stands for?
>
> Also any code snippets or samples on de-serializing a DataSet from a TSV.
>
> Thanks.
>
> --
> John Olbert
>
>
>
> "Philip Sheard" wrote:
>
>> You might want to consider using TSV format instead. It is easier to
>> parse,
>> and you can give the files a txt extension, which sometimes helps.
>>
>> "John Olbert" <someone@snet.net> wrote in message
>> news:5889C557-65A4-448D-A10D-439BCCBC8338@microsoft.com...
>> > Subject: Using Comma Separated Values to de-serialize a DataSet
>> >
>> > The DataSet.ReadXml() speed under Netcf1.0 is slow. It results in our
>> > App
>> > taking over 60 seconds to start. I have talked to many newsgroups,
>> > researched
>> > articles, etc to find a way to improve this time. The best answer to
>> > date
>> > is
>> > to upgrade to Netcf2.0. I am in the process of doing this but there may
>> > be
>> > problems with updating the physical device.
>> >
>> > One other possibility was mentioned on a newsgroup. That is to use
>> > Comma
>> > Separated Values rather then Xml to load the DataSet. Does anyone know
>> > how
>> > that would be done and if any code snippets or examples exist.
>> >
>> > Thanks.
>> >
>> > --
>> > John Olbert
>> >
>>
>>
>>

Philip
Thu Mar 02 11:11:31 CST 2006

There is an official standard for TSV format, but none for CSV. CSV just
happens to be more prevalent.

"Paul G. Tobey [eMVP]" <ptobey no spam AT no instrument no spam DOT com>
wrote in message news:e1nNLWhPGHA.2696@TK2MSFTNGP14.phx.gbl...
> I'm not 100% sure, but I'm guessing Tab-separated Values.
>
> Paul T.
>
> "John Olbert" <someone@snet.net> wrote in message
> news:93883877-9B68-43EF-B066-EE5946CEE303@microsoft.com...
>> Sorry, TSV stands for?
>>
>> Also any code snippets or samples on de-serializing a DataSet from a TSV.
>>
>> Thanks.
>>
>> --
>> John Olbert
>>
>>
>>
>> "Philip Sheard" wrote:
>>
>>> You might want to consider using TSV format instead. It is easier to
>>> parse,
>>> and you can give the files a txt extension, which sometimes helps.
>>>
>>> "John Olbert" <someone@snet.net> wrote in message
>>> news:5889C557-65A4-448D-A10D-439BCCBC8338@microsoft.com...
>>> > Subject: Using Comma Separated Values to de-serialize a DataSet
>>> >
>>> > The DataSet.ReadXml() speed under Netcf1.0 is slow. It results in our
>>> > App
>>> > taking over 60 seconds to start. I have talked to many newsgroups,
>>> > researched
>>> > articles, etc to find a way to improve this time. The best answer to
>>> > date
>>> > is
>>> > to upgrade to Netcf2.0. I am in the process of doing this but there
>>> > may be
>>> > problems with updating the physical device.
>>> >
>>> > One other possibility was mentioned on a newsgroup. That is to use
>>> > Comma
>>> > Separated Values rather then Xml to load the DataSet. Does anyone know
>>> > how
>>> > that would be done and if any code snippets or examples exist.
>>> >
>>> > Thanks.
>>> >
>>> > --
>>> > John Olbert
>>> >
>>>
>>>
>>>
>
>

someone
Thu Mar 02 16:30:06 CST 2006

That is a shame. We are seeing load times of 8 to 12 seconds on a Xml file of
24kb being read into a DataSet via ReadXml. I was trying to find a way to get
that down to about 1 to 2 seconds. We do a number of reads and the end result
is that the app takes over 60 seconds to load up.

Thanks.

--
John Olbert



"Ginny Caughey [MVP]" wrote:

> John,
>
> Look at the TextDataAdapter on www.opennetcf.org. Depending on the device
> and the particular data, CSV can be twice as fast and half as bulky as XML.
> But on Windows Mobile 5.0 devices running CF 2.0 that I've tested, the
> performance gap between XML and CSV isn't so dramatic (or even apparent in
> some cases.)
>
> --
> Ginny Caughey
> ..NET Compact Framework MVP
>
>
> "John Olbert" <someone@snet.net> wrote in message
> news:5889C557-65A4-448D-A10D-439BCCBC8338@microsoft.com...
> > Subject: Using Comma Separated Values to de-serialize a DataSet
> >
> > The DataSet.ReadXml() speed under Netcf1.0 is slow. It results in our App
> > taking over 60 seconds to start. I have talked to many newsgroups,
> > researched
> > articles, etc to find a way to improve this time. The best answer to date
> > is
> > to upgrade to Netcf2.0. I am in the process of doing this but there may be
> > problems with updating the physical device.
> >
> > One other possibility was mentioned on a newsgroup. That is to use Comma
> > Separated Values rather then Xml to load the DataSet. Does anyone know how
> > that would be done and if any code snippets or examples exist.
> >
> > Thanks.
> >
> > --
> > John Olbert
> >
>
>
>

Ginny
Thu Mar 02 16:54:41 CST 2006

John,

I doubt you'll get it down to 1 or 2 seconds, but depending on the data and
the device, you might get it down to 4-6 seconds with the
OpenNETCF.TextDataAdapter and a csv file. ReadXml is generally faster with
CF 2 also - maybe twice as fast in my testing but your mileage may vary.

If you can't get the load time down much, be sure to have a nice splash
screen for the users to look at. ;-)

--
Ginny Caughey
.NET Compact Framework MVP


"John Olbert" <someone@snet.net> wrote in message
news:880B08B2-B380-45CC-9318-CEC4D5FA6301@microsoft.com...
> That is a shame. We are seeing load times of 8 to 12 seconds on a Xml file
> of
> 24kb being read into a DataSet via ReadXml. I was trying to find a way to
> get
> that down to about 1 to 2 seconds. We do a number of reads and the end
> result
> is that the app takes over 60 seconds to load up.
>
> Thanks.
>
> --
> John Olbert
>
>
>
> "Ginny Caughey [MVP]" wrote:
>
>> John,
>>
>> Look at the TextDataAdapter on www.opennetcf.org. Depending on the device
>> and the particular data, CSV can be twice as fast and half as bulky as
>> XML.
>> But on Windows Mobile 5.0 devices running CF 2.0 that I've tested, the
>> performance gap between XML and CSV isn't so dramatic (or even apparent
>> in
>> some cases.)
>>
>> --
>> Ginny Caughey
>> ..NET Compact Framework MVP
>>
>>
>> "John Olbert" <someone@snet.net> wrote in message
>> news:5889C557-65A4-448D-A10D-439BCCBC8338@microsoft.com...
>> > Subject: Using Comma Separated Values to de-serialize a DataSet
>> >
>> > The DataSet.ReadXml() speed under Netcf1.0 is slow. It results in our
>> > App
>> > taking over 60 seconds to start. I have talked to many newsgroups,
>> > researched
>> > articles, etc to find a way to improve this time. The best answer to
>> > date
>> > is
>> > to upgrade to Netcf2.0. I am in the process of doing this but there may
>> > be
>> > problems with updating the physical device.
>> >
>> > One other possibility was mentioned on a newsgroup. That is to use
>> > Comma
>> > Separated Values rather then Xml to load the DataSet. Does anyone know
>> > how
>> > that would be done and if any code snippets or examples exist.
>> >
>> > Thanks.
>> >
>> > --
>> > John Olbert
>> >
>>
>>
>>

Philip
Thu Mar 02 18:24:47 CST 2006

Our TSV import rogers through about 330 KB in 39 seconds, *and* inserts
about 3,000 rows into SQL tables. That is on a fairly humble Acer n30. No
C++, or tricks with fruit. Just bog standard VB.

With TSV, I can parse a file thus:

Dim f() As String
Dim a() As String
Dim s As String

With New StreamReader("\My Documents\data.txt")
s = .ReadToEnd
.Close
End With

s = Replace(s, vbCrLf, ChrW(1))
s = Replace(s, vbLf, vbCrLf)
f = Split(s, ChrW(1))

for each s in f
a = Split(s, vbTab)
next

You are talking speed here. And I can edit the file in Pocket Word if I want
to. Try doing that with XML. Or CSV for that matter.

"John Olbert" <someone@snet.net> wrote in message
news:880B08B2-B380-45CC-9318-CEC4D5FA6301@microsoft.com...
> That is a shame. We are seeing load times of 8 to 12 seconds on a Xml file
> of
> 24kb being read into a DataSet via ReadXml. I was trying to find a way to
> get
> that down to about 1 to 2 seconds. We do a number of reads and the end
> result
> is that the app takes over 60 seconds to load up.
>
> Thanks.
>
> --
> John Olbert
>
>
>
> "Ginny Caughey [MVP]" wrote:
>
>> John,
>>
>> Look at the TextDataAdapter on www.opennetcf.org. Depending on the device
>> and the particular data, CSV can be twice as fast and half as bulky as
>> XML.
>> But on Windows Mobile 5.0 devices running CF 2.0 that I've tested, the
>> performance gap between XML and CSV isn't so dramatic (or even apparent
>> in
>> some cases.)
>>
>> --
>> Ginny Caughey
>> ..NET Compact Framework MVP
>>
>>
>> "John Olbert" <someone@snet.net> wrote in message
>> news:5889C557-65A4-448D-A10D-439BCCBC8338@microsoft.com...
>> > Subject: Using Comma Separated Values to de-serialize a DataSet
>> >
>> > The DataSet.ReadXml() speed under Netcf1.0 is slow. It results in our
>> > App
>> > taking over 60 seconds to start. I have talked to many newsgroups,
>> > researched
>> > articles, etc to find a way to improve this time. The best answer to
>> > date
>> > is
>> > to upgrade to Netcf2.0. I am in the process of doing this but there may
>> > be
>> > problems with updating the physical device.
>> >
>> > One other possibility was mentioned on a newsgroup. That is to use
>> > Comma
>> > Separated Values rather then Xml to load the DataSet. Does anyone know
>> > how
>> > that would be done and if any code snippets or examples exist.
>> >
>> > Thanks.
>> >
>> > --
>> > John Olbert
>> >
>>
>>
>>

Ilya
Fri Mar 03 19:35:36 CST 2006

XML loading time depends on what are you loading and how you do that.



Please see this on how to speed it up:



http://groups.google.com/group/microsoft.public.dotnet.framework.compactframework/msg/6f6a9cbdff962bad



Also note first load includes JITing time which is about 3-5 seconds.


--
Best regards,

Ilya

This posting is provided "AS IS" with no warranties, and confers no rights.

*** Want to find answers instantly? Here's how... ***

1. Go to
http://groups-beta.google.com/group/microsoft.public.dotnet.framework.compactframework?hl=en
2. Type your question in the text box near "Search this group" button.
3. Hit "Search this group" button.
4. Read answer(s).

"John Olbert" <someone@snet.net> wrote in message
news:880B08B2-B380-45CC-9318-CEC4D5FA6301@microsoft.com...
> That is a shame. We are seeing load times of 8 to 12 seconds on a Xml file
> of
> 24kb being read into a DataSet via ReadXml. I was trying to find a way to
> get
> that down to about 1 to 2 seconds. We do a number of reads and the end
> result
> is that the app takes over 60 seconds to load up.
>
> Thanks.
>
> --
> John Olbert
>
>
>
> "Ginny Caughey [MVP]" wrote:
>
>> John,
>>
>> Look at the TextDataAdapter on www.opennetcf.org. Depending on the device
>> and the particular data, CSV can be twice as fast and half as bulky as
>> XML.
>> But on Windows Mobile 5.0 devices running CF 2.0 that I've tested, the
>> performance gap between XML and CSV isn't so dramatic (or even apparent
>> in
>> some cases.)
>>
>> --
>> Ginny Caughey
>> ..NET Compact Framework MVP
>>
>>
>> "John Olbert" <someone@snet.net> wrote in message
>> news:5889C557-65A4-448D-A10D-439BCCBC8338@microsoft.com...
>> > Subject: Using Comma Separated Values to de-serialize a DataSet
>> >
>> > The DataSet.ReadXml() speed under Netcf1.0 is slow. It results in our
>> > App
>> > taking over 60 seconds to start. I have talked to many newsgroups,
>> > researched
>> > articles, etc to find a way to improve this time. The best answer to
>> > date
>> > is
>> > to upgrade to Netcf2.0. I am in the process of doing this but there may
>> > be
>> > problems with updating the physical device.
>> >
>> > One other possibility was mentioned on a newsgroup. That is to use
>> > Comma
>> > Separated Values rather then Xml to load the DataSet. Does anyone know
>> > how
>> > that would be done and if any code snippets or examples exist.
>> >
>> > Thanks.
>> >
>> > --
>> > John Olbert
>> >
>>
>>
>>