String conversion problem (encoders)

Hi to all,

can anyone tells me what is the correct way from the framework
to convert OEM characters to Ansi (and the opposite) ??

I mean, i am building an SMS infrastructure and one part of
it is to send sms through the help of Nokia PC Connectivity SDK.
The problem is that when the user enter Greek characters in a
textbox, they reach as empty chars in the phone,
so i have to convert them to Ansi (i guess) before sending them.

I tried the code below (taken from online help) but it gives me "??? ?????"
in a Greek string. I know i am doing something wrong with the encoders
but i still cannot find it.

Does anyone can help ??


Thanks a lot!


Anthony Boudouvas

Jon
Wed Oct 22 08:38:46 CDT 2003

Anthony Boudouvas <anthonyb@mediatrel.com> wrote:
> can anyone tells me what is the correct way from the framework
> to convert OEM characters to Ansi (and the opposite) ??

Yup - you use the appropriate Encoding instance. See
http://www.pobox.com/~skeet/csharp/unicode.html

> I mean, i am building an SMS infrastructure and one part of
> it is to send sms through the help of Nokia PC Connectivity SDK.
> The problem is that when the user enter Greek characters in a
> textbox, they reach as empty chars in the phone,
> so i have to convert them to Ansi (i guess) before sending them.

I'm not sure that guessing is going to be good enough here - does the
SDK not say what it's expecting?

--
Jon Skeet - <skeet@pobox.com>
http://www.pobox.com/~skeet
If replying to the group, please do not mail me too

Anthony
Wed Oct 22 09:46:09 CDT 2003

Surely it does, it expecting a string...But, for English characters work
like
a charm, with the Greek characters is that i have the problem,
so i have to find a way to convert.

The guess is that as we write sms to a phone, we write non-unicode
characters,
i suppose ansi. So i have to convert to ansi (or other ?) to pass to it.
I tried several combinations with no luck...

Any other idea?

anthonyb



"Jon Skeet [C# MVP]" <skeet@pobox.com> wrote in message
news:MPG.1a0094e9cb04c539898ee@msnews.microsoft.com...
> Anthony Boudouvas <anthonyb@mediatrel.com> wrote:
> > can anyone tells me what is the correct way from the framework
> > to convert OEM characters to Ansi (and the opposite) ??
>
> Yup - you use the appropriate Encoding instance. See
> http://www.pobox.com/~skeet/csharp/unicode.html
>
> > I mean, i am building an SMS infrastructure and one part of
> > it is to send sms through the help of Nokia PC Connectivity SDK.
> > The problem is that when the user enter Greek characters in a
> > textbox, they reach as empty chars in the phone,
> > so i have to convert them to Ansi (i guess) before sending them.
>
> I'm not sure that guessing is going to be good enough here - does the
> SDK not say what it's expecting?
>
> --
> Jon Skeet - <skeet@pobox.com>
> http://www.pobox.com/~skeet
> If replying to the group, please do not mail me too

Jon
Wed Oct 22 09:07:24 CDT 2003

Anthony Boudouvas <anthonyb@mediatrel.com> wrote:
> Surely it does, it expecting a string...

Is this a .NET SDK, or a Win32 SDK? If it's a .NET SDK, the encoding
shouldn't come into it.

> But, for English characters work
> like a charm, with the Greek characters is that i have the problem,
> so i have to find a way to convert.
>
> The guess is that as we write sms to a phone, we write non-unicode
> characters, i suppose ansi.

But ANSI isn't a single character encoding to start with - there are
lots of different versions.

> So i have to convert to ansi (or other ?) to pass to it.
> I tried several combinations with no luck...
>
> Any other idea?

Are you absolutely sure that you've got the right characters in your
string to start with (before encoding it)? I suggest you print out the
integer value of each character and check against
http://www.unicode.org

--
Jon Skeet - <skeet@pobox.com>
http://www.pobox.com/~skeet
If replying to the group, please do not mail me too

Anthony
Wed Oct 22 10:25:07 CDT 2003

> Is this a .NET SDK, or a Win32 SDK? If it's a .NET SDK, the encoding
> shouldn't come into it.

It is a Win32 SDK.


> Are you absolutely sure that you've got the right characters in your
> string to start with (before encoding it)? I suggest you print out the
> integer value of each character and check against

I use a code like this, (i read it from the Online Help):


Encoding ascii = Encoding.Ascii; // apparently this is not Ascii so i tried
the others with no result...
Encoding unicode = Encoding.Unicode;

// Convert the string into a byte[].
byte[] unicodeBytes = unicode.GetBytes(text);

// Perform the conversion from one encoding to the other.
byte[] asciiBytes = Encoding.Convert(unicode, ascii, unicodeBytes);

// Convert the new byte[] into a char[] and then into a string.
// This is a slightly different approach to converting to illustrate
// the use of GetCharCount/GetChars.

char[] asciiChars = new char[ascii.GetCharCount(asciiBytes, 0,
asciiBytes.Length)];
ascii.GetChars(asciiBytes, 0, asciiBytes.Length, asciiChars, 0);
string asciiString = new string(asciiChars);
return (asciiString);


So, the returned string is either ???????? or empty characters, depending on
the encoder i use.

Still no luck...


anthonyb

Jon
Wed Oct 22 09:35:03 CDT 2003

Anthony Boudouvas <anthonyb@mediatrel.com> wrote:
> > Is this a .NET SDK, or a Win32 SDK? If it's a .NET SDK, the encoding
> > shouldn't come into it.
>
> It is a Win32 SDK.

Right. In that case you need to find out *exactly* what kind of string
it's expecting - what does the API say? Is the API online anywhere for
us to look at it? I see there are various downloadable documents for it
- what version are you using?

> > Are you absolutely sure that you've got the right characters in your
> > string to start with (before encoding it)? I suggest you print out the
> > integer value of each character and check against
>
> I use a code like this, (i read it from the Online Help):

<snip>

That's not good - you're basically converting a string to another
string, which is very unlikely to do what you want. If you can pass a
string into the SDK, then encodings don't come into it (on your side).

--
Jon Skeet - <skeet@pobox.com>
http://www.pobox.com/~skeet
If replying to the group, please do not mail me too

Anthony
Wed Oct 22 11:03:49 CDT 2003


> Right. In that case you need to find out *exactly* what kind of string
> it's expecting - what does the API say? Is the API online anywhere for
> us to look at it? I see there are various downloadable documents for it
> - what version are you using?

I am using the Nokia PC Connectivity SDK 3.0 and the documents with it
described the function that we need to call to send sms but does not
particularly says what implications may be stand up with non-english
characters...

> That's not good - you're basically converting a string to another
> string, which is very unlikely to do what you want. If you can pass a
> string into the SDK, then encodings don't come into it (on your side).

I somehow try to pass the phone a string that it will like,
ana apparently, this it was my first attemp to achiev that.
I also saw that for example the Greek L "?" is reported by windows
as ascii 923...

I am working on it to find a solution in this direction now...

Can you think of something on it ?

A, i appreciate a lot your effort to help me Jon,
a sincerely Thanks a Lot!



anthonyb

Jon
Wed Oct 22 10:49:56 CDT 2003

Anthony Boudouvas <anthonyb@mediatrel.com> wrote:
>
> > Right. In that case you need to find out *exactly* what kind of string
> > it's expecting - what does the API say? Is the API online anywhere for
> > us to look at it? I see there are various downloadable documents for it
> > - what version are you using?
>
> I am using the Nokia PC Connectivity SDK 3.0 and the documents with it
> described the function that we need to call to send sms but does not
> particularly says what implications may be stand up with non-english
> characters...

Which function are you using? How *exactly* is it described? How are
you calling it? (If you're using P/Invoke, please show the function
delcaration.)

If you're using UserDataText, have you also set UserDataFormat to
UNICODE_16_BIT before setting UserDataText? I'm not entirely sure what
the DATA_8_BIT setting is for, but I think I'd use Unicode just for
sure. Note that that means the SMS can only be half as long.

> > That's not good - you're basically converting a string to another
> > string, which is very unlikely to do what you want. If you can pass a
> > string into the SDK, then encodings don't come into it (on your side).
>
> I somehow try to pass the phone a string that it will like,
> ana apparently, this it was my first attemp to achiev that.
> I also saw that for example the Greek L "?" is reported by windows
> as ascii 923...

I'm sure it's not, as ASCII 923 doesn't exist - ASCII only goes from 0
to 127. Where are you getting that number from?

> I am working on it to find a solution in this direction now...
>
> Can you think of something on it ?
>
> A, i appreciate a lot your effort to help me Jon,
> a sincerely Thanks a Lot!

No problem.

--
Jon Skeet - <skeet@pobox.com>
http://www.pobox.com/~skeet
If replying to the group, please do not mail me too

Anthony
Thu Oct 23 03:38:07 CDT 2003

> > I am using the Nokia PC Connectivity SDK 3.0 and the documents with it
> > described the function that we need to call to send sms but does not
> > particularly says what implications may be stand up with non-english
> > characters...
>
> Which function are you using? How *exactly* is it described? How are
> you calling it? (If you're using P/Invoke, please show the function
> delcaration.)

Not, it has com wrappers so i just create a reference to the typelibs of it
and
use the object methods.


> > I somehow try to pass the phone a string that it will like,
> > ana apparently, this it was my first attemp to achiev that.
> > I also saw that for example the Greek L "?" is reported by windows
> > as ascii 923...
>
> I'm sure it's not, as ASCII 923 doesn't exist - ASCII only goes from 0
> to 127. Where are you getting that number from?


Well...my mistake not an Ascii exactly, but the code page number for this
Greek letter is 923, as i get of:
Console.WriteLine(Convert.ToString((char) 923));


I have made an application of http server and the characters that i get
(the Greek ones) are between 913-947.
(irrelevant, just for you to un derstand what i am talking about...)


anthonyb

Jon
Thu Oct 23 03:25:27 CDT 2003

Anthony Boudouvas <anthonyb@mediatrel.com> wrote:
> > Which function are you using? How *exactly* is it described? How are
> > you calling it? (If you're using P/Invoke, please show the function
> > delcaration.)
>
> Not, it has com wrappers so i just create a reference to the typelibs of it
> and use the object methods.

Right.

So, what about my other question - are you setting the UserDataFormat
property to UNICODE_16_BIT first?

> > I'm sure it's not, as ASCII 923 doesn't exist - ASCII only goes from 0
> > to 127. Where are you getting that number from?
>
> Well...my mistake not an Ascii exactly, but the code page number for this
> Greek letter is 923, as i get of:
> Console.WriteLine(Convert.ToString((char) 923));

I think you're still very confused - what do you mean exactly by "code
page number"? What you *really* want to do is look at www.unicode.org
and check what the *Unicode* values for the characters you want are.

--
Jon Skeet - <skeet@pobox.com>
http://www.pobox.com/~skeet
If replying to the group, please do not mail me too

Anthony
Thu Oct 23 05:03:42 CDT 2003

> So, what about my other question - are you setting the UserDataFormat
> property to UNICODE_16_BIT first?

Jon,

there is no UserDataFormat property in any object in NokiaCLMessaging lib.
As i saw at the pdsf's of sdk, there are two ways to do something usefull
with it:
One is NokiaCLMessaging and the other is SMS3ASUITELIB.

I uses the first because the majority of the vb samples that came with the
sdk,
use this as well, especially for managin contacts/sms/calendar.


> > > I'm sure it's not, as ASCII 923 doesn't exist - ASCII only goes from 0
> > > to 127. Where are you getting that number from?
> >
> > Well...my mistake not an Ascii exactly, but the code page number for
this
> > Greek letter is 923, as i get of:
> > Console.WriteLine(Convert.ToString((char) 923));
>
> I think you're still very confused - what do you mean exactly by "code
> page number"? What you *really* want to do is look at www.unicode.org
> and check what the *Unicode* values for the characters you want are.

The Unicode value of the Greek L, is 039B, or in Decimal, 923 as i told you.
I knew that the value was correct from another project i have made,
the problem is making nokia 6210 understand it...


anthonyb

Jon
Thu Oct 23 04:29:15 CDT 2003

Anthony Boudouvas <anthonyb@mediatrel.com> wrote:
> > So, what about my other question - are you setting the UserDataFormat
> > property to UNICODE_16_BIT first?
>
> there is no UserDataFormat property in any object in NokiaCLMessaging lib.
> As i saw at the pdsf's of sdk, there are two ways to do something usefull
> with it:
> One is NokiaCLMessaging and the other is SMS3ASUITELIB.
>
> I uses the first because the majority of the vb samples that came with the
> sdk, use this as well, especially for managin contacts/sms/calendar.

Right. Well, looking for the same kind of info, it seems there's the
DataCodingScheme property of IGSMParameters - and if you set *that* to
8, it should be right for Unicode.

That's just from reading the docs though - whether or not it'll work
just like that, I don't know...

> > I think you're still very confused - what do you mean exactly by "code
> > page number"? What you *really* want to do is look at www.unicode.org
> > and check what the *Unicode* values for the characters you want are.
>
> The Unicode value of the Greek L, is 039B, or in Decimal, 923 as i told you.

Well, you told me it was a "code page number", which isn't the same
thing as Unicode at all. Not being precise about what you mean is the
best way to get things wrong when it comes to encodings. Anyway...

> I knew that the value was correct from another project i have made,
> the problem is making nokia 6210 understand it...

Right.

--
Jon Skeet - <skeet@pobox.com>
http://www.pobox.com/~skeet
If replying to the group, please do not mail me too

Anthony
Thu Oct 23 05:48:02 CDT 2003

> Right. Well, looking for the same kind of info, it seems there's the
> DataCodingScheme property of IGSMParameters - and if you set *that* to
> 8, it should be right for Unicode.
>
> That's just from reading the docs though - whether or not it'll work
> just like that, I don't know...

I made another example, using the SMS3ASuiteLib lIBRARY.

I set the UserDataFormat to USER_DATA_FORMAT_UNICODE_16BIT
and it is the only constant here that makes the program crash...

I have already set the DataCodingScheme (in the example that i use the
NokiaCLMessaging lib)
and it does not work too...It just send empty characters...

Jon, thanks a lot for your help, i have to say it another one time...


anthonyb

Jon
Thu Oct 23 05:28:20 CDT 2003

Anthony Boudouvas <anthonyb@mediatrel.com> wrote:
> > Right. Well, looking for the same kind of info, it seems there's the
> > DataCodingScheme property of IGSMParameters - and if you set *that* to
> > 8, it should be right for Unicode.
> >
> > That's just from reading the docs though - whether or not it'll work
> > just like that, I don't know...
>
> I made another example, using the SMS3ASuiteLib lIBRARY.
>
> I set the UserDataFormat to USER_DATA_FORMAT_UNICODE_16BIT
> and it is the only constant here that makes the program crash...

Ah - well that's a pain :(

> I have already set the DataCodingScheme (in the example that i use the
> NokiaCLMessaging lib)
> and it does not work too...It just send empty characters...

Did you set the DataCodingScheme before or after you set the message?

> Jon, thanks a lot for your help, i have to say it another one time...

No worries. However, it sounds like we've nearly exhausted all the
possibilities. Is there an online forum for discussion of the SDK? It
sounds like the kind of thing that other people may have run into -
maybe not using .NET, but I don't think that's terribly relevant.

--
Jon Skeet - <skeet@pobox.com>
http://www.pobox.com/~skeet
If replying to the group, please do not mail me too

Anthony
Thu Oct 23 06:42:28 CDT 2003

> Ah - well that's a pain :(

And you know where... :)


> > I have already set the DataCodingScheme (in the example that i use the
> > NokiaCLMessaging lib)
> > and it does not work too...It just send empty characters...
>
> Did you set the DataCodingScheme before or after you set the message?

I set it before and i tried some different coding schemes values i saw in
the manual,
like 240,41..247. None of them works.


> No worries. However, it sounds like we've nearly exhausted all the
> possibilities. Is there an online forum for discussion of the SDK? It
> sounds like the kind of thing that other people may have run into -
> maybe not using .NET, but I don't think that's terribly relevant.


I post last night a question to forum.nokia.com/support but as i saw today,
the answers there are very rare from nokia engineers...

So, i will give it another 2-3 hours of trying and then i will just not use
Greek
characters, i will replace them with English, a bit of ackward display but
the only that currently works...

A, something that will make your head spin:
I look at a sample that can read sms from phone/sim, so i selected one
that contain Greek characters and see if the SDK can actually transfer to a
Windows App.
That app, reads the data from an sms and it displays them to text boxes.

In the message text it displays all the Greek characters that have an Ascii
representation,
like A,E,H,I etc. and all other Greek characters as null values. (i didn't
expected anything better...)

The funny thing is that the message title (the first10-12 letters of each
message on 6210)
which, of course, is Greek, is displayed correctly...


Thanks again!

Jon
Thu Oct 23 06:01:26 CDT 2003

Anthony Boudouvas <anthonyb@mediatrel.com> wrote:

<snip>

> The funny thing is that the message title (the first10-12 letters of each
> message on 6210)
> which, of course, is Greek, is displayed correctly...

Bizarre. This is now all sounding sufficiently weird that I think I've
given all the help that'll actually be useful, I'm afraid :(

--
Jon Skeet - <skeet@pobox.com>
http://www.pobox.com/~skeet
If replying to the group, please do not mail me too