I'm trying to create an xml document with values from a database, but have
the following problem.

Many of the items in the database contain characters from the extended
charcter set (I think that is the correct phrase!) and when i try to create
my xml it throws an error unless i have the server.urlencode statement

dim DelegateList,dqt,dtd,DelegateListElement
dqt = chr(34)
Set DelegateList = Server.Createobject("MSXML.DOMDocument")
Set dtd = DelegateList.createProcessingInstruction("xml", "version=" & dqt
& "1.0" & dqt )
DelegateList.appendChild dtd
Set DelegateListElement = DelegateList.createElement("delegate_details")
DelegateList.appendChild DelegateListElement

'some database stuff

Do Until adoRS.EOF
vCOUNT=vCOUNT+1
set recElement = DelegateList.createElement("delegate")
for i = 0 to adoRS.Fields.Count - 1
recElement.setAttribute adoRS(i).name, server.urlencode(adoRS(i).value)
DelegateList.lastChild.appendChild recElement
next
adoRS.MoveNext
Loop

If I don't have server.urlencode then it immediately doesn't work, but when
I do have server.urlencode it seems to translate some of the characters
incorrectly. For example "Ö" becomes %D6 but it should become %C3%96

Could anyone please steer me in the correct direction.

Many thanks

Dave

Re: creating xml document by Anthony

Anthony
Fri Nov 10 16:12:48 CST 2006


"Dave" <david@nospam.co.uk> wrote in message
news:jb15h.11951$hK2.3552@newsfe3-win.ntli.net...
> I'm trying to create an xml document with values from a database, but have
> the following problem.
>
> Many of the items in the database contain characters from the extended
> charcter set (I think that is the correct phrase!) and when i try to
create
> my xml it throws an error unless i have the server.urlencode statement
>
> dim DelegateList,dqt,dtd,DelegateListElement
> dqt = chr(34)
> Set DelegateList = Server.Createobject("MSXML.DOMDocument")
> Set dtd = DelegateList.createProcessingInstruction("xml", "version=" &
dqt
> & "1.0" & dqt )
> DelegateList.appendChild dtd
> Set DelegateListElement = DelegateList.createElement("delegate_details")
> DelegateList.appendChild DelegateListElement
>
> 'some database stuff
>
> Do Until adoRS.EOF
> vCOUNT=vCOUNT+1
> set recElement = DelegateList.createElement("delegate")
> for i = 0 to adoRS.Fields.Count - 1
> recElement.setAttribute adoRS(i).name,
server.urlencode(adoRS(i).value)
> DelegateList.lastChild.appendChild recElement
> next
> adoRS.MoveNext
> Loop
>
> If I don't have server.urlencode then it immediately doesn't work, but
when
> I do have server.urlencode it seems to translate some of the characters
> incorrectly. For example "Ö" becomes %D6 but it should become %C3%96
>
> Could anyone please steer me in the correct direction.

Firstly thing stop using urlencode. The XML will encode characters
correctly.

Other than that we need to see what you do with the DOM once you've built
it?
Is this in ASP?


>
> Many thanks
>
> Dave
>
>



Re: creating xml document by Dave

Dave
Mon Nov 13 04:12:23 CST 2006

Thanks for the reply Anthony

I'll explain a little more, this is indeed an asp page that I use as
middleware between the database and flash.

This is a list of names that is in the database that I need to get into
flash.

So I use response.write to send the values back to flash.

If I take out the server.urlencode statement then the asp page has an error
immediately saying "An invalid character was found in text content"

Do you have any suggestions?

Many thanks

Dave

"Anthony Jones" <Ant@yadayadayada.com> wrote in message
news:ui2BdVRBHHA.4568@TK2MSFTNGP04.phx.gbl...
>
> "Dave" <david@nospam.co.uk> wrote in message
> news:jb15h.11951$hK2.3552@newsfe3-win.ntli.net...
>> I'm trying to create an xml document with values from a database, but
>> have
>> the following problem.
>>
>> Many of the items in the database contain characters from the extended
>> charcter set (I think that is the correct phrase!) and when i try to
> create
>> my xml it throws an error unless i have the server.urlencode statement
>>
>> dim DelegateList,dqt,dtd,DelegateListElement
>> dqt = chr(34)
>> Set DelegateList = Server.Createobject("MSXML.DOMDocument")
>> Set dtd = DelegateList.createProcessingInstruction("xml", "version=" &
> dqt
>> & "1.0" & dqt )
>> DelegateList.appendChild dtd
>> Set DelegateListElement =
>> DelegateList.createElement("delegate_details")
>> DelegateList.appendChild DelegateListElement
>>
>> 'some database stuff
>>
>> Do Until adoRS.EOF
>> vCOUNT=vCOUNT+1
>> set recElement = DelegateList.createElement("delegate")
>> for i = 0 to adoRS.Fields.Count - 1
>> recElement.setAttribute adoRS(i).name,
> server.urlencode(adoRS(i).value)
>> DelegateList.lastChild.appendChild recElement
>> next
>> adoRS.MoveNext
>> Loop
>>
>> If I don't have server.urlencode then it immediately doesn't work, but
> when
>> I do have server.urlencode it seems to translate some of the characters
>> incorrectly. For example "Ö" becomes %D6 but it should become %C3%96
>>
>> Could anyone please steer me in the correct direction.
>
> Firstly thing stop using urlencode. The XML will encode characters
> correctly.
>
> Other than that we need to see what you do with the DOM once you've built
> it?
> Is this in ASP?
>
>
>>
>> Many thanks
>>
>> Dave
>>
>>
>
>



Re: creating xml document by Anthony

Anthony
Mon Nov 13 05:52:09 CST 2006


"Dave" <david@nospam.co.uk> wrote in message
news:byX5h.4972$371.4886@newsfe5-win.ntli.net...
> Thanks for the reply Anthony
>
> I'll explain a little more, this is indeed an asp page that I use as
> middleware between the database and flash.
>
> This is a list of names that is in the database that I need to get into
> flash.
>
> So I use response.write to send the values back to flash.
>
> If I take out the server.urlencode statement then the asp page has an
error
> immediately saying "An invalid character was found in text content"
>
> Do you have any suggestions?
>

Ok it's getting muddier.

Let me make guess.

Flash is expecting XML.
You build XML via a DOM.
You then use Response.Write DelegateList.XML to send the content to the
client.

Have I guessed correctly? If so do this instead:-

Response.ContentType = "text/xml"
Response.CharSet = "UTF-8"
DelegateList.Save Response




> Many thanks
>
> Dave
>
> "Anthony Jones" <Ant@yadayadayada.com> wrote in message
> news:ui2BdVRBHHA.4568@TK2MSFTNGP04.phx.gbl...
> >
> > "Dave" <david@nospam.co.uk> wrote in message
> > news:jb15h.11951$hK2.3552@newsfe3-win.ntli.net...
> >> I'm trying to create an xml document with values from a database, but
> >> have
> >> the following problem.
> >>
> >> Many of the items in the database contain characters from the extended
> >> charcter set (I think that is the correct phrase!) and when i try to
> > create
> >> my xml it throws an error unless i have the server.urlencode statement
> >>
> >> dim DelegateList,dqt,dtd,DelegateListElement
> >> dqt = chr(34)
> >> Set DelegateList = Server.Createobject("MSXML.DOMDocument")
> >> Set dtd = DelegateList.createProcessingInstruction("xml", "version="
&
> > dqt
> >> & "1.0" & dqt )
> >> DelegateList.appendChild dtd
> >> Set DelegateListElement =
> >> DelegateList.createElement("delegate_details")
> >> DelegateList.appendChild DelegateListElement
> >>
> >> 'some database stuff
> >>
> >> Do Until adoRS.EOF
> >> vCOUNT=vCOUNT+1
> >> set recElement = DelegateList.createElement("delegate")
> >> for i = 0 to adoRS.Fields.Count - 1
> >> recElement.setAttribute adoRS(i).name,
> > server.urlencode(adoRS(i).value)
> >> DelegateList.lastChild.appendChild recElement
> >> next
> >> adoRS.MoveNext
> >> Loop
> >>
> >> If I don't have server.urlencode then it immediately doesn't work, but
> > when
> >> I do have server.urlencode it seems to translate some of the characters
> >> incorrectly. For example "Ö" becomes %D6 but it should become %C3%96
> >>
> >> Could anyone please steer me in the correct direction.
> >
> > Firstly thing stop using urlencode. The XML will encode characters
> > correctly.
> >
> > Other than that we need to see what you do with the DOM once you've
built
> > it?
> > Is this in ASP?
> >
> >
> >>
> >> Many thanks
> >>
> >> Dave
> >>
> >>
> >
> >
>
>



Re: creating xml document by Dave

Dave
Mon Nov 13 06:07:09 CST 2006

It works!!!! Thank you very much Anthony, you have got me out of a BIG hole
there!

Could you please just explain what you have done differently, and why
server.urlencode was the wrong way to go, and why response.write does not
work in this case? That would be much appreciated as I'm still learning in
this area!

What does DelegateList.Save Response actually do? Is this similar to
response.write?

Thanks

Dave

"Anthony Jones" <Ant@yadayadayada.com> wrote in message
news:eDLpnoxBHHA.4892@TK2MSFTNGP04.phx.gbl...
>
> "Dave" <david@nospam.co.uk> wrote in message
> news:byX5h.4972$371.4886@newsfe5-win.ntli.net...
>> Thanks for the reply Anthony
>>
>> I'll explain a little more, this is indeed an asp page that I use as
>> middleware between the database and flash.
>>
>> This is a list of names that is in the database that I need to get into
>> flash.
>>
>> So I use response.write to send the values back to flash.
>>
>> If I take out the server.urlencode statement then the asp page has an
> error
>> immediately saying "An invalid character was found in text content"
>>
>> Do you have any suggestions?
>>
>
> Ok it's getting muddier.
>
> Let me make guess.
>
> Flash is expecting XML.
> You build XML via a DOM.
> You then use Response.Write DelegateList.XML to send the content to the
> client.
>
> Have I guessed correctly? If so do this instead:-
>
> Response.ContentType = "text/xml"
> Response.CharSet = "UTF-8"
> DelegateList.Save Response
>
>
>
>
>> Many thanks
>>
>> Dave
>>
>> "Anthony Jones" <Ant@yadayadayada.com> wrote in message
>> news:ui2BdVRBHHA.4568@TK2MSFTNGP04.phx.gbl...
>> >
>> > "Dave" <david@nospam.co.uk> wrote in message
>> > news:jb15h.11951$hK2.3552@newsfe3-win.ntli.net...
>> >> I'm trying to create an xml document with values from a database, but
>> >> have
>> >> the following problem.
>> >>
>> >> Many of the items in the database contain characters from the extended
>> >> charcter set (I think that is the correct phrase!) and when i try to
>> > create
>> >> my xml it throws an error unless i have the server.urlencode statement
>> >>
>> >> dim DelegateList,dqt,dtd,DelegateListElement
>> >> dqt = chr(34)
>> >> Set DelegateList = Server.Createobject("MSXML.DOMDocument")
>> >> Set dtd = DelegateList.createProcessingInstruction("xml", "version="
> &
>> > dqt
>> >> & "1.0" & dqt )
>> >> DelegateList.appendChild dtd
>> >> Set DelegateListElement =
>> >> DelegateList.createElement("delegate_details")
>> >> DelegateList.appendChild DelegateListElement
>> >>
>> >> 'some database stuff
>> >>
>> >> Do Until adoRS.EOF
>> >> vCOUNT=vCOUNT+1
>> >> set recElement = DelegateList.createElement("delegate")
>> >> for i = 0 to adoRS.Fields.Count - 1
>> >> recElement.setAttribute adoRS(i).name,
>> > server.urlencode(adoRS(i).value)
>> >> DelegateList.lastChild.appendChild recElement
>> >> next
>> >> adoRS.MoveNext
>> >> Loop
>> >>
>> >> If I don't have server.urlencode then it immediately doesn't work, but
>> > when
>> >> I do have server.urlencode it seems to translate some of the
>> >> characters
>> >> incorrectly. For example "Ö" becomes %D6 but it should become %C3%96
>> >>
>> >> Could anyone please steer me in the correct direction.
>> >
>> > Firstly thing stop using urlencode. The XML will encode characters
>> > correctly.
>> >
>> > Other than that we need to see what you do with the DOM once you've
> built
>> > it?
>> > Is this in ASP?
>> >
>> >
>> >>
>> >> Many thanks
>> >>
>> >> Dave
>> >>
>> >>
>> >
>> >
>>
>>
>
>



Re: creating xml document by Anthony

Anthony
Mon Nov 13 11:22:12 CST 2006


"Dave" <david@nospam.co.uk> wrote in message
news:NdZ5h.17148$Zy3.8387@newsfe2-win.ntli.net...
> It works!!!! Thank you very much Anthony, you have got me out of a BIG
hole
> there!
>
> Could you please just explain what you have done differently, and why
> server.urlencode was the wrong way to go, and why response.write does not
> work in this case? That would be much appreciated as I'm still learning in
> this area!
>
> What does DelegateList.Save Response actually do? Is this similar to
> response.write?
>
> Thanks
>
> Dave
>

The purpose of URLEncode is to ensure characters that are invalid or out of
the allowable range of a URL are encoded correctly. This is designed to be
used when building URLs since the encoding it uses is only understood by
something expecting to receive a URL. Hence you should only user URLEncode
when building strings for src and href attributes.

In your code there are three different character sets to consider.

Firstly the unicode character set (also known on Windows machines as
codepage 65000). This is the native character set used by VBScript and COM
objects. It is a common set of characters that is able represent pretty
much every character or every language on the planet and then some. It can
do this because each character occupies 2 bytes instead of just 1.

Hence this method:- adoRS(i).value would be returning a unicode string.

The second character set to consider is UTF-8 (also known on Windows
machines as codepage 65001). This is the default character set used by XML.
An XML parser (such as the one built into flash) when loading a file or
receiving a download stream should assume it is receiving UTF-8 unless the
<?xml declare says otherwise. UTF-8 is in fact the same set as unicode but
it use a variable length encoding. The first 128 characters are identical
to ASCII and occupy 1 byte most english strings will look identical whether
encoded as ASCII or UTF-8. The larger the Unicode value a character needs
the longer the UTF-8 encoding becomes. For example the British £ requires 2
bytes in UTF-8.

The third character set to consider in this case is Windows-1252 (also known
on Windows machines as codepage 1252). This is the standard system codepage
that a machine installed with english settings will be using. It is a
single byte character set where the first 128 characters are identical to
ASCII (hence for most english string again indistinguishable from UTF-8 or
ASCII). The upper 128 characters contain other 'Latin' characters in common
use in the West. The system code page is the code page used by ASP.

When you call Response.Write you asking ASP to convert the Unicode string
you are providing to the current codepage encoding then insert the result of
the conversion in to the output buffer. The current codepage is controlled
by the Session.Codepage and defaults to the system code page. If therefore
you were to examine the value of Session.Codepage you will find it is set to
1252. IIS6 has an additional Response.Codepage that is effective during the
current response only (whereas session.codepage when set affects the whole
session).

Armed with that information what is happening here:-

Response.Write DelegateList.XML

??

DelegateList.XML is a string property on a COM object. It will return a
Unicode string which is the only type of string VBScript understands.

Now Response.Write will encode the provided string to codepage 1252 because
that is the current codepage for the session (or response). If the unicode
string contains a character not available in the Windows-1252 set one of two
things can happen. Either it will error or it will send ? instead.

Even if all characters are in the 1252 set there is still a problem. For
example the British £ is character 163 in the 1252 set and that's how the
Response.Write would encode it. But the XML parser is expecting UTF-8
encoding which (since £ isn't in the ASCII set) is expecting a different 2
byte encoding. Hence the stream is corrupt.

Now take a look at this:-

DelegateList.Save Response

Normally one pass Filename as a string to the Save method of an XML DOM.
This would cause the XML DOM to save it's contents to a file. It will do
this using the encoding specified on the <?xml declare or if that isn't
present use the default UTF-8 encoding.

The Save method can also take a COM object that implements the IStream
interface. The IStream interface allows content to be serialised from one
media to another. The Response object in ASP implements the IStream
interface so it is a valid object to pass to the Save method. The DOM will
do the same thing it does when saving to a file but instead of writing bytes
to disk it will write bytes to the provided IStream. The Response object
will simply copy the bytes received to the output buffer, it will not
attempt any code page conversion.

So using Save causes UTF-8 encoding to be sent to the Flash client which is
what it was expecting.

These lines:-

Response.ContentType = "text/xml"
Response.CharSet = "UTF-8"

are just belts and braces in this case but it is good practice when
generating output from ASP to specify exactly what it is you are sending to
the client.

HTH,

Anthony







Re: creating xml document by Dave

Dave
Mon Nov 13 12:34:52 CST 2006

That's some really useful information, thanks for taking the time to explain
things in such detail, it's always a steep learning curve!

Thanks

Dave

"Anthony Jones" <Ant@yadayadayada.com> wrote in message
news:u3jSDh0BHHA.4292@TK2MSFTNGP02.phx.gbl...
>
> "Dave" <david@nospam.co.uk> wrote in message
> news:NdZ5h.17148$Zy3.8387@newsfe2-win.ntli.net...
>> It works!!!! Thank you very much Anthony, you have got me out of a BIG
> hole
>> there!
>>
>> Could you please just explain what you have done differently, and why
>> server.urlencode was the wrong way to go, and why response.write does not
>> work in this case? That would be much appreciated as I'm still learning
>> in
>> this area!
>>
>> What does DelegateList.Save Response actually do? Is this similar to
>> response.write?
>>
>> Thanks
>>
>> Dave
>>
>
> The purpose of URLEncode is to ensure characters that are invalid or out
> of
> the allowable range of a URL are encoded correctly. This is designed to
> be
> used when building URLs since the encoding it uses is only understood by
> something expecting to receive a URL. Hence you should only user
> URLEncode
> when building strings for src and href attributes.
>
> In your code there are three different character sets to consider.
>
> Firstly the unicode character set (also known on Windows machines as
> codepage 65000). This is the native character set used by VBScript and
> COM
> objects. It is a common set of characters that is able represent pretty
> much every character or every language on the planet and then some. It
> can
> do this because each character occupies 2 bytes instead of just 1.
>
> Hence this method:- adoRS(i).value would be returning a unicode string.
>
> The second character set to consider is UTF-8 (also known on Windows
> machines as codepage 65001). This is the default character set used by
> XML.
> An XML parser (such as the one built into flash) when loading a file or
> receiving a download stream should assume it is receiving UTF-8 unless the
> <?xml declare says otherwise. UTF-8 is in fact the same set as unicode
> but
> it use a variable length encoding. The first 128 characters are identical
> to ASCII and occupy 1 byte most english strings will look identical
> whether
> encoded as ASCII or UTF-8. The larger the Unicode value a character needs
> the longer the UTF-8 encoding becomes. For example the British £ requires
> 2
> bytes in UTF-8.
>
> The third character set to consider in this case is Windows-1252 (also
> known
> on Windows machines as codepage 1252). This is the standard system
> codepage
> that a machine installed with english settings will be using. It is a
> single byte character set where the first 128 characters are identical to
> ASCII (hence for most english string again indistinguishable from UTF-8 or
> ASCII). The upper 128 characters contain other 'Latin' characters in
> common
> use in the West. The system code page is the code page used by ASP.
>
> When you call Response.Write you asking ASP to convert the Unicode string
> you are providing to the current codepage encoding then insert the result
> of
> the conversion in to the output buffer. The current codepage is
> controlled
> by the Session.Codepage and defaults to the system code page. If
> therefore
> you were to examine the value of Session.Codepage you will find it is set
> to
> 1252. IIS6 has an additional Response.Codepage that is effective during
> the
> current response only (whereas session.codepage when set affects the whole
> session).
>
> Armed with that information what is happening here:-
>
> Response.Write DelegateList.XML
>
> ??
>
> DelegateList.XML is a string property on a COM object. It will return a
> Unicode string which is the only type of string VBScript understands.
>
> Now Response.Write will encode the provided string to codepage 1252
> because
> that is the current codepage for the session (or response). If the
> unicode
> string contains a character not available in the Windows-1252 set one of
> two
> things can happen. Either it will error or it will send ? instead.
>
> Even if all characters are in the 1252 set there is still a problem. For
> example the British £ is character 163 in the 1252 set and that's how the
> Response.Write would encode it. But the XML parser is expecting UTF-8
> encoding which (since £ isn't in the ASCII set) is expecting a different 2
> byte encoding. Hence the stream is corrupt.
>
> Now take a look at this:-
>
> DelegateList.Save Response
>
> Normally one pass Filename as a string to the Save method of an XML DOM.
> This would cause the XML DOM to save it's contents to a file. It will do
> this using the encoding specified on the <?xml declare or if that isn't
> present use the default UTF-8 encoding.
>
> The Save method can also take a COM object that implements the IStream
> interface. The IStream interface allows content to be serialised from one
> media to another. The Response object in ASP implements the IStream
> interface so it is a valid object to pass to the Save method. The DOM
> will
> do the same thing it does when saving to a file but instead of writing
> bytes
> to disk it will write bytes to the provided IStream. The Response object
> will simply copy the bytes received to the output buffer, it will not
> attempt any code page conversion.
>
> So using Save causes UTF-8 encoding to be sent to the Flash client which
> is
> what it was expecting.
>
> These lines:-
>
> Response.ContentType = "text/xml"
> Response.CharSet = "UTF-8"
>
> are just belts and braces in this case but it is good practice when
> generating output from ASP to specify exactly what it is you are sending
> to
> the client.
>
> HTH,
>
> Anthony
>
>
>
>
>
>