Hi all,

I'm looking for some advice please. I am using a list to hold a
catalogue of products of which there are about 5,000. One of the
fields in the list is a block of XHTML which can be 10-50K.

I need access to this information from a server not running Sharepoint
and want to work out the best option. The pattern of list updates is
about 10 new records added each day. Updates and deletion from the
catalogue list are rare.

1- Direct access to the list in its native form in the SP database -
not recommended or supported so I understand.
2- Use the SP web services. I can see these as being useful for
smaller lists but would they really be practical for fetching this
volume of data? A bit slow perhaps
3- Create a table in a second non-SP database which has columns that
match the fields of the catalogue list. Implement event handler on any
change event on the catalogue list to write to the database table. I
should always have a up to date version of the list data in a database
schema to access directly by my chosen means.

I would be interested to hear any comments on these, or any better
ideas that I have overlooked.

Many thanks,
Andrew

Re: export or replication of list data by galvin

galvin
Thu Mar 13 20:18:14 CDT 2008

On Mar 13, 6:42=A0pm, Andrew <az...@hotmail.com> wrote:
> Hi all,
>
> I'm looking for some advice please. I am using a list to hold a
> catalogue of products of which there are about 5,000. One of the
> fields in the list is a block of XHTML which can be 10-50K.
>
> I need access to this information from a server not running Sharepoint
> and want to work out the best option. The pattern of list updates is
> about 10 new records added each day. Updates and deletion from the
> catalogue list are rare.
>
> 1- Direct access to the list in its native form in the SP database -
> not recommended or supported so I understand.
> 2- Use the SP web services. I can see these as being useful for
> smaller lists but would they really be practical for fetching this
> volume of data? A bit slow perhaps
> 3- =A0Create a table in a second non-SP database which has columns that
> match the fields of the catalogue list. Implement event handler on any
> change event on the catalogue list to write to the database table. I
> should always have a up to date version of the list data in a database
> schema to access directly by my chosen means.
>
> I would be interested to hear any comments on these, or any better
> ideas that I have overlooked.
>
> Many thanks,
> Andrew

I think that if you read the database directly, you're OK. There are
many risks, but if your environment is such that you can manage them,
then I would at least consider it. I would never insert data
directly, however.

That said, I'd still have a look at the web service interface. It may
not be as slow as you think. The data has to travel the wire
eventually whether it's directly via SQL or indirectly via a web
service. In the scheme of thing, the web service isn't going to add a
lot of overhead. However, if the provided web services are slow, you
can always write your own efficient facade using the object model.

--Paul Galvin, Conchango
RSS @ http://feeds.feedburner.com/PaulGalvinsSharepointSpace
Web site @ http://paulgalvin.spaces.live.com

Re: export or replication of list data by Andrew

Andrew
Sun Mar 16 16:45:40 CDT 2008

On Mar 14, 1:18=A0am, "galvin.p...@gmail.com" <galvin.p...@gmail.com>
wrote:
> On Mar 13, 6:42=A0pm, Andrew <az...@hotmail.com> wrote:
>
>
>
>
>
> > Hi all,
>
> > I'm looking for some advice please. I am using a list to hold a
> > catalogue of products of which there are about 5,000. One of the
> > fields in the list is a block of XHTML which can be 10-50K.
>
> > I need access to this information from a server not running Sharepoint
> > and want to work out the best option. The pattern of list updates is
> > about 10 new records added each day. Updates and deletion from the
> > catalogue list are rare.
>
> > 1- Direct access to the list in its native form in the SP database -
> > not recommended or supported so I understand.
> > 2- Use the SP web services. I can see these as being useful for
> > smaller lists but would they really be practical for fetching this
> > volume of data? A bit slow perhaps
> > 3- =A0Create a table in a second non-SP database which has columns that
> > match the fields of the catalogue list. Implement event handler on any
> > change event on the catalogue list to write to the database table. I
> > should always have a up to date version of the list data in a database
> > schema to access directly by my chosen means.
>
> > I would be interested to hear any comments on these, or any better
> > ideas that I have overlooked.
>
> > Many thanks,
> > Andrew
>
> I think that if you read the database directly, you're OK. =A0There are
> many risks, but if your environment is such that you can manage them,
> then I would at least consider it. =A0I would never insert data
> directly, however.
>
> That said, I'd still have a look at the web service interface. =A0It may
> not be as slow as you think. =A0The data has to travel the wire
> eventually whether it's directly via SQL or indirectly via a web
> service. =A0In the scheme of thing, the web service isn't going to add a
> lot of overhead. =A0However, if the provided web services are slow, you
> can always write your own efficient facade using the object model.
>
> --Paul Galvin, Conchango
> =A0 RSS @http://feeds.feedburner.com/PaulGalvinsSharepointSpace
> =A0 Web site @http://paulgalvin.spaces.live.com- Hide quoted text -
>
> - Show quoted text -

Thanks for the response Paul.

I delved into the SP schema a bit and found the AllUserData table as
the most likely store of my list data. The schema itself scared me off
using it as it looks like a set number of generically named columns
e.g. nvarchar1,.. nvarchar8 in blocks of 8. I just feel a bit
uncomfortable with it.

I have 2 main types of data and I am going to employ the facade for
the first (lower volume) simpler type. I only need to fetch this into
another tool so it's a simple dump.

For the second, high volume, I am going to investigate the listener on
the add/update/delete list events. I need to be able to perform more
intricate DB join work on this data.

Thanks,
Andrew