File format for a new application

(I know it's a seond post, but I think I explain it better this way)
--

On my own I'm writting an application for the general public.
(a bit like word, except it has nothing to do with word ;-)

So far I used an embeded database engine to store the data (SQLite), I like
the concept because this way I could load/delete/updata whatever data I need
incrementally on demand. And I could also easily perform search.

What gets me more and more concerned is that creating a new (empty) file
(with just about 10 table, 30 index & triggers) takes up to 7 seconds!
Very bad user experience.

I'm considering my options and will gladly takes advice as I'm a bit daunted
by the work involve just to test.

- I was thinking to try an other embeded database engine, such as Firefox.
- I was thinking to use tar-gzipped serialized file (think XML serialization
should be better as it would be probably be better at version resistance
(but worst perf?)), but I was wondering about the wisdom of having all my
data in memory...
- any other ideas?

Lloyd
Wed Aug 31 05:51:38 CDT 2005

> - I was thinking to try an other embeded database engine, such as Firefox.
Oops...
I mean Firebird, of course!

MoriCristian
Wed Aug 31 21:35:01 CDT 2005

Hi!
I really don't know how SQLite works, but can't you keep a template file and
use it to create clones for eache new project? In this way it will only take
the copy time, and I suppose that it is less than 7 seconds

Regards
Cristian Mori

"Lloyd Dupont" wrote:

> (I know it's a seond post, but I think I explain it better this way)
> --
>
> On my own I'm writting an application for the general public.
> (a bit like word, except it has nothing to do with word ;-)
>
> So far I used an embeded database engine to store the data (SQLite), I like
> the concept because this way I could load/delete/updata whatever data I need
> incrementally on demand. And I could also easily perform search.
>
> What gets me more and more concerned is that creating a new (empty) file
> (with just about 10 table, 30 index & triggers) takes up to 7 seconds!
> Very bad user experience.
>
> I'm considering my options and will gladly takes advice as I'm a bit daunted
> by the work involve just to test.
>
> - I was thinking to try an other embeded database engine, such as Firefox.
> - I was thinking to use tar-gzipped serialized file (think XML serialization
> should be better as it would be probably be better at version resistance
> (but worst perf?)), but I was wondering about the wisdom of having all my
> data in memory...
> - any other ideas?
>
>
>

Damien
Thu Sep 01 02:45:33 CDT 2005

Lloyd Dupont wrote:
> (I know it's a seond post, but I think I explain it better this way)
> --
>
> On my own I'm writting an application for the general public.
> (a bit like word, except it has nothing to do with word ;-)
>
> So far I used an embeded database engine to store the data (SQLite), I like
> the concept because this way I could load/delete/updata whatever data I need
> incrementally on demand. And I could also easily perform search.
>
> What gets me more and more concerned is that creating a new (empty) file
> (with just about 10 table, 30 index & triggers) takes up to 7 seconds!
> Very bad user experience.
>
[snip rest]
Could you not have your File->New start a new thread which does the
initialisation, while your UI just says "Okay, I've done that, what
next?". Even if you need to interact immediately, surely the addition
of indexes can happen seperately. How fast would it be if those were
done by a background thread instead of immediately?

Damien

drh
Thu Sep 01 05:27:41 CDT 2005

Lloyd Dupont wrote:
>
> What gets me more and more concerned is that creating a new (empty) file
> (with just about 10 table, 30 index & triggers) takes up to 7 seconds!
> Very bad user experience.
>

Your are probably processing each CREATE as a separate transaction.
It will go *much* faster if you do all the creating as a single
transaction but wrapping all your CREATE statements inside a single
pair of BEGIN and COMMIT statements.

Lloyd
Thu Sep 01 06:36:54 CDT 2005

great idea, I'll try that, thanks!

--
"A preoccupation with the next world pretty clearly signals an inability to
cope credibly with this one."

<drh@hwaci.com> wrote in message
news:1125570461.965506.157340@g47g2000cwa.googlegroups.com...
> Lloyd Dupont wrote:
>>
>> What gets me more and more concerned is that creating a new (empty) file
>> (with just about 10 table, 30 index & triggers) takes up to 7 seconds!
>> Very bad user experience.
>>
>
> Your are probably processing each CREATE as a separate transaction.
> It will go *much* faster if you do all the creating as a single
> transaction but wrapping all your CREATE statements inside a single
> pair of BEGIN and COMMIT statements.
>