Developer warning: file handles can become un-usable after power OFF/ON

On WM2003 (and WM2003-SE), when you open a file located on a storage card,
the file handle become un-usable after power OFF/ON (i.e. power
suspend/resume).

This problem is described here in more details:

http://www.pocketpcthoughts.com/forums/viewtopic.php?t=27614

If you app opens files on a storage card (or in the application home folder,
which can be located on the Storage Card), you should be aware of this
issue, and implement the necessary workarounds, if needed. If you don't,
you application can mis-behave after the user presses the Power button, or
it can cause data corruption.

This issue affects some (maybe all) Pocket PC 2003, including the Toshiba
e800 series.

I believe that this problem also affects Smartphone 2003 devices, although
not all Smartphones are capable of suspending (The Samsung i600 does suspend
when closed, if no earphone jacked is used).

Michael
Wed May 05 09:45:03 CDT 2004

File handles become un-usable if you yank the card, too (while device is
still powered on). So you need to handle that type of situation anyway.

Is there any notification? That is, if you use SHChangeNotifyRegister and
the card is pulled, you would get WM_FILECHANGEINFO/SHCNE_MEDIAREMOVED
message if the user pulled the card out. If your app is using storage card
files, then it already is (or should be) registering for this notification,
and properly handling it.. WM_DEVICECHANGE also works on some devices, but
I found SHChangeNotifyRegister is more reliable.

Anyway, do you get that same notification, or any other notification, on
suspend/resume? That would be the real problem, if no notification was
sent. Otherwise use the same handling you have for
WM_FILECHANGEINFO/SHCNE_MEDIAREMOVED or WM_DEVICECHANGE.
--

Michael Salamone [eMVP]
Entrek Software, Inc.
www.entrek.com



"The PocketTV Team" <do-not-reply-by-email@pockettv.com> wrote in message
news:eqo$%23vnMEHA.3292@TK2MSFTNGP11.phx.gbl...
> On WM2003 (and WM2003-SE), when you open a file located on a storage card,
> the file handle become un-usable after power OFF/ON (i.e. power
> suspend/resume).
>
> This problem is described here in more details:
>
> http://www.pocketpcthoughts.com/forums/viewtopic.php?t=27614
>
> If you app opens files on a storage card (or in the application home
folder,
> which can be located on the Storage Card), you should be aware of this
> issue, and implement the necessary workarounds, if needed. If you don't,
> you application can mis-behave after the user presses the Power button, or
> it can cause data corruption.
>
> This issue affects some (maybe all) Pocket PC 2003, including the Toshiba
> e800 series.
>
> I believe that this problem also affects Smartphone 2003 devices, although
> not all Smartphones are capable of suspending (The Samsung i600 does
suspend
> when closed, if no earphone jacked is used).
>
>

The
Wed May 05 17:12:38 CDT 2004

"Michael J. Salamone [eMVP]" <mikesa#at#entrek#dot#com> wrote in message
news:efynO%23qMEHA.2244@TK2MSFTNGP10.phx.gbl...
> File handles become un-usable if you yank the card, too (while device is
> still powered on). So you need to handle that type of situation anyway.

Yes, of course, but that's different. That'a a fatal situation: the file
becomes permanently un-usable because the card has been removed.

But that's not what the problem is about, here. The problem here is that
case where power is suspended/resumed, without removing any removanle
storage.


> Is there any notification? That is, if you use SHChangeNotifyRegister and
> the card is pulled, you would get WM_FILECHANGEINFO/SHCNE_MEDIAREMOVED
> message if the user pulled the card out. If your app is using storage
card
> files, then it already is (or should be) registering for this
notification,
> and properly handling it.. WM_DEVICECHANGE also works on some devices,
but
> I found SHChangeNotifyRegister is more reliable.

Again, that's not the question. The problem does not involve removing any
card.'

And it makes it very complicated for apps to deal with re-opening files each
time an error occurs because of this problem.

In fact MSFT is now looking seriously at this issue, since it also breaks
most of MSFT's own apps. The only one that correctly deals with re-opening
files on removable storage after power suspend/resume is WMP. All the ather
MSFT apps break in this case.


> Anyway, do you get that same notification, or any other notification, on
> suspend/resume? That would be the real problem, if no notification was
> sent. Otherwise use the same handling you have for
> WM_FILECHANGEINFO/SHCNE_MEDIAREMOVED or WM_DEVICECHANGE.

Well, ReadFile returns an arror, so you get notified, in a way. But that's
not what is supposed to happen.

In WinCE 3.0 (and earlier), all handles to storage card files were remounted
correctly and automatically, sometimes after a small delay during which
ReadFile would return ERROR_ACCESS_DENIED, ERROR_NOT_READY or
ERROR_DEVICE_NOT_AVAILABLE. so you just needed to retry, as described here:

http://support.microsoft.com/default.aspx?scid=kb;en-us;811693

But now, in WinCE 4.2, you can retry as much as you want, the handles are
never remounted.

That's causing all sorts of troubles when people power off their devices,
while running apps have files opened on Storage Cards.

And I don't even want to think of the disaster if some running apps are
themself installed on the Storage Cards!


> --
>
> Michael Salamone [eMVP]
> Entrek Software, Inc.
> www.entrek.com
>
>
>
> "The PocketTV Team" <do-not-reply-by-email@pockettv.com> wrote in message
> news:eqo$%23vnMEHA.3292@TK2MSFTNGP11.phx.gbl...
> > On WM2003 (and WM2003-SE), when you open a file located on a storage
card,
> > the file handle become un-usable after power OFF/ON (i.e. power
> > suspend/resume).
> >
> > This problem is described here in more details:
> >
> > http://www.pocketpcthoughts.com/forums/viewtopic.php?t=27614
> >
> > If you app opens files on a storage card (or in the application home
> folder,
> > which can be located on the Storage Card), you should be aware of this
> > issue, and implement the necessary workarounds, if needed. If you
don't,
> > you application can mis-behave after the user presses the Power button,
or
> > it can cause data corruption.
> >
> > This issue affects some (maybe all) Pocket PC 2003, including the
Toshiba
> > e800 series.
> >
> > I believe that this problem also affects Smartphone 2003 devices,
although
> > not all Smartphones are capable of suspending (The Samsung i600 does
> suspend
> > when closed, if no earphone jacked is used).
> >
> >
>
>

hel
Wed May 05 18:25:41 CDT 2004

TPT [Wed, 5 May 2004 15:12:38 -0700]:
>And I don't even want to think of the disaster if some running apps are
>themself installed on the Storage Cards!

You won't have to think, I'll tell you...

software: iPlay 4.6 (in a directory)
hardware: iPAQ 2215 (why not)

I copied the whole directory to an SD card. Started it.
It's playing an .mp4 tune, also on the SD. Power (suspend).
...Okay, now, as I type, I'll move a hand over there to
Power (resume) and as I do it and stop wasting bytes here,

(resume)

All is fine. Play continues from where it was (it's a 1 MB
.EXE so there's a lot of opportunity to demand-load pages).

So, it's not a problem there, at least not on this 2215 (I've
done the same on a 4155, too). Even my alarm works from storage
card while suspended (how many apps can be started on a storage
card like that?). So, this part of the sky is still up, and
holding firm.


As you should already know, SD access won't work the first
three or so seconds after power has resumed. Are you taking
this into account when it comes to the "file handles not
valid after suspend" thread from earlier? I'm not at ALL
surprised that MS apps fail here (LOL) [sqlce is a master
of corruption, for example, despite MS denials for years].
I'm sure most will if they a reading and get a read error.
I think FEW will stick around retrying for 3 or more secs.

--
40th Floor - Software @ http://40th.com/
iPlay : the ultimate audio player for iPAQs
mp3, ogg, mp4, m4a, aac, wav, and then some

The
Wed May 05 19:07:24 CDT 2004

> All is fine. Play continues from where it was (it's a 1 MB
> .EXE so there's a lot of opportunity to demand-load pages).

> So, it's not a problem there,

Yes, that's probably because iPlay, just like WMP, PocketTV and BetaPlayer,
are capable of dealing with the problem, i.e. they attempt to re-open the
file if ReadFile returns an error.

> As you should already know, SD access won't work the first
> three or so seconds after power has resumed.

Yes, that's true on Pocket PC 2003 and 2002.

This is what is documented here:

http://support.microsoft.com/default.aspx?scid=kb;en-us;811693

But on most WM2003 and WM2003-SE devices, apparently the file handles are
not remounted correctly.

And this new behavior, if "by design" not only is undocumented, but it also
breaks a number of apps, including several from MSFT.

> Are you taking
> this into account when it comes to the "file handles not
> valid after suspend" thread from earlier?

Yes. You should read my article there:
http://www.pocketpcthoughts.com/forums/viewtopic.php?t=27614

It describes the problem.

The problem is not that you'll get a temporary error while the filesystem is
not yet remounted. The problem is that the handles are never correctly
remounted.

> I'm not at ALL
> surprised that MS apps fail here (LOL) [sqlce is a master
> of corruption, for example, despite MS denials for years].
> I'm sure most will if they a reading and get a read error.
> I think FEW will stick around retrying for 3 or more secs.

What even more interesting is that some MSFT people say: "It's by design",
while other people at MSFT don't agree with that! Definitely, it's a big
step back from Pocket PC 2002, where at least, after a delay, the handle
were re-mounted correctly.

Im my opinion, the ideal behavior should be:

After a power suspend/resume, a filesystem call involving a handle to a file
located on a removable storage should:

- block until the driver determines if the removable storage has been left
unchanged in the device, or if it has been removed, or modified while
suspended.

- then, return an error if the driver determines that the removable storage
has been removed, or has been modified while the device was suspended (e.g.
SD card moved to camera, photos taken, then moved back in device).

- and if the driver determines that the removable storage is still there and
un-modified, the handles should be remounted automatically and the call
should perform normally, as if nothing happened (except for the longer delay
before returning).

This is, IMHO, what should happen. This would allows the power
suspend/resume to be virtually transparent to all applications.

In WinCE 3.0, this was *almost* working like that. The only caveat was that
the application had to do the retries until timeout ( see
http://support.microsoft.com/default.aspx?scid=kb;en-us;811693 ).

The
Wed May 05 19:11:16 CDT 2004

> All is fine. Play continues from where it was (it's a 1 MB
> .EXE so there's a lot of opportunity to demand-load pages).
>
> So, it's not a problem there

Sorry, I did not catch was you were saying, i got it now:

Ok, then, at least the file mapping on SD cards seems to be immune to this
problem. It's a good thing.

But many apps open private data files located in their installation folder
(e.g. games open data files to load their skins and bitmaps, etc).

If the application home folder is on the SD card, then the handles of all
those file will be broken after a power suspend/resume.

Not good!

The
Wed May 05 19:20:42 CDT 2004

> All is fine. Play continues from where it was (it's a 1 MB
> .EXE so there's a lot of opportunity to demand-load pages).
>
> So, it's not a problem there, at least not on this 2215

In fact everything is not as rosie, at least on the e800.

I was able to cause an app installed on the SD card to quit brutally by
suspending/resuming the device while the app was running, even though the
app had no data file opened on the SD card.

Apparently if a page-fault (i.e. demand-load) occurs before the SD card is
re-mounted after suspend/resume, then the app will quit.

hel
Wed May 05 20:07:15 CDT 2004

Is this sort of stuff in the ce 4.2 kernel source that's
floating around? No way to know for sure unless you look
at the code to see what's going on.

>Sorry, I did not catch was you were saying, i got it now:

I knew you wouldn't, so I made the point twice. You
must have jump down to the last parapraph and skimmed
the meat at the top.


>If the application home folder is on the SD card, then the handles of all
>those file will be broken after a power suspend/resume.

Well, let's see how the help file goes: The help file I use is
open (via fopen) on first use. It's an external txt file that's
read on demand; its file handle stays open until the app exits.

And... the sky is still up! So, that works as expected. Your
theory seems to have cracks since this file is on the SD, it was
open before power suspend, and it continues to accept reads after
power resume. The only difference I see here is I'm using an
fopen handle, not CreateFile which I use for non-text stuff.

So, back to trying a CreateFile handle...giving it a good time
to sleep... yup, skipped, so that is failing, but I only retry
for a max of 1 second, and only do that for dealing with the
3800 which was known to fail SD reads for no reason at all.

--
40th Floor - Software @ http://40th.com/
iPlay : the ultimate audio player for iPAQs
mp3, ogg, mp4, m4a, aac, wav, and then some

The
Wed May 05 20:17:04 CDT 2004

> And... the sky is still up! So, that works as expected. Your
> theory seems to have cracks

Apparently it's no a theory. Several people at MSFT have been able to
reproduce all the issues I described, and they are working on these issues.

In the case you describe, you use a high-level library (fopen) which is
known to do some buffering. So it's impossible to know what is pre-buffered
and when ReadFile occurs.

If you use ReadFile, you can easiely reproduce the issue.

In any case, it is now an open case at MSFT and they are aware that there
are real problems there.

I thought some developers may be interested to know about these problems,
since it may affect them. I'm not trying to proove anything to you. MSFT
managed to duplicate those problems, that's what's important.

> dealing with the 3800 which was known to fail SD reads for no reason at
all

do you know what error code is return in this case on the 3800 ?

The
Wed May 05 20:18:55 CDT 2004

> And... the sky is still up! So, that works as expected.

According to MSFT, it may behave differendly on several devices, since some
drivers are involved.

It is possible that the handles do not get broken on the device you are
testing with.

hel
Wed May 05 20:42:46 CDT 2004

TPT [Wed, 5 May 2004 18:17:04 -0700]:
>In the case you describe, you use a high-level library (fopen) which is
>known to do some buffering. So it's impossible to know what is pre-buffered

Well, yes, I thought of that. The file is 47 KB and I read
the first entry before suspend, and after suspend the last
entry, just about 47 KB away. I don't know if it buffers
47 KB (the whole file). I could always setvbuff it to
nothing, but maybe you'd be more interested in doing that?
I would think CE's runtime is not buffering the whole thing;
it'd be a waste to buffer that much.

>In any case, it is now an open case at MSFT and they are aware that there
>are real problems there.

Case #955

>I thought some developers may be interested to know about these problems,
>since it may affect them. I'm not trying to proove anything to you. MSFT
>managed to duplicate those problems, that's what's important.

But you still have to deal with it. There are 100s of thousands of
these out there, and all but a handful will never be fixed. On
that depends on MSFT ever doing a retro-fix. And yes, I think if
you make the claim, that yes, you should prove it. You do have,
you know, a propensity to claim the sky is falling with, it seems,
everything you come across, new or old.

>do you know what error code is return in this case on the 3800 ?

It could return several. Check for those under 31 or so (old time
DOS hardware error codes). If you want to test it, pull a card
out while reading and put it back.

--
40th Floor - Software @ http://40th.com/
iPlay : the ultimate audio player for iPAQs
mp3, ogg, mp4, m4a, aac, wav, and then some

hel
Wed May 05 20:46:01 CDT 2004


TPT [Wed, 5 May 2004 18:18:55 -0700]:
>According to MSFT, it may behave differendly on several devices, since some
>drivers are involved.

I read that as, "it's not our problem". And didn't you say some
of these same people laughed at this, saying it was "be design"?

>It is possible that the handles do not get broken on the device you are
>testing with.

These things have problems, and they will continue to have problems.
No Jordi guys around to fix it once and for all. As a matter of
certainty, the next OS release will be the worst yet! What'll you
do then?

--
40th Floor - Software @ http://40th.com/
iPlay : the ultimate audio player for iPAQs
mp3, ogg, mp4, m4a, aac, wav, and then some

The
Wed May 05 20:57:24 CDT 2004

> But you still have to deal with it.

that's exactly the reason why I posted this article on pocketpcthoughts.com
.

apparently a large number of developers (including people at MSFT doing the
bundle apps) are not aware of this problem.

i never said the sky was falling, i said there is a problem, it is real, and
it can affect the apps that people are developing.

that's all. anything wrong with informing the Pocket PC developers community
?

The
Wed May 05 20:59:07 CDT 2004

<hel@40th.com> wrote in message
news:uMTyjvwMEHA.3556@TK2MSFTNGP09.phx.gbl...
>
> TPT [Wed, 5 May 2004 18:18:55 -0700]:
> >According to MSFT, it may behave differendly on several devices, since
some
> >drivers are involved.
>
> I read that as, "it's not our problem". And didn't you say some
> of these same people laughed at this, saying it was "be design"?

They did, but they don't say that anymore at MSFT.

Now they look at the issue seriously.

> >It is possible that the handles do not get broken on the device you are
> >testing with.
>
> These things have problems, and they will continue to have problems.
> No Jordi guys around to fix it once and for all. As a matter of
> certainty, the next OS release will be the worst yet! What'll you
> do then?

Inform people of problems if MSFT does not do a good job.

hel
Wed May 05 21:13:30 CDT 2004

>> But you still have to deal with it.

TPT[Wed, 5 May 2004 18:57:24 -0700]:
> that's exactly the reason why I posted this article on pocketpcthoughts.com

There have been many problems related to coming out of
suspend over the years. This is yet another.

>i never said the sky was falling, i said there is a problem, it is real, and
>it can affect the apps that people are developing.

The end of the world is coming... is the tone you set. It may
seem like it to you, but it's more of the same (at least to me).

>that's all. anything wrong with informing the Pocket PC developers community

OK. So, what are you going to do about it? Anything? You
can't wait for a fix, if it ever comes (for 2003) because it
won't do any good. So, right now, what are you going to do?
Or have you already?

--
40th Floor - Software @ http://40th.com/
iPlay : the ultimate audio player for iPAQs
mp3, ogg, mp4, m4a, aac, wav, and then some

The
Wed May 05 21:27:29 CDT 2004

> OK. So, what are you going to do about it? Anything?

As I said, inform people of the problem.

> You
> can't wait for a fix, if it ever comes (for 2003) because it
> won't do any good.

I'm not waiting for a fix any time soon, but maybe this will be fixed in
WinCE 5.0.

> So, right now, what are you going to do?
> Or have you already?

As I said in a previous post in this thread, we already have implemented a
workaround, but the workaround consisting in trying to re-open the file
based on the pathname are not very good, since you can end-up re-opening
another file that have the same name.

but the problem is that most applications out there (including most MSFT
apps) don't have the workaround, and they all assume that file handles don't
break on suspend/resume.

yes, there are many problems with suspend/resume, e.g. audio drivers
breaking.

i think it's time MSFT wakes up to this problem and help OEM with QA testing
involving power suspend/resume.

The power button on a Pocket PC is probably the one that is used most,
suspending pewer is a very common thing people do all the time, and the
facts that is does not work well is a problem.

i think this issue has just been grossely overlooked by MSFT.

hel
Wed May 05 21:41:46 CDT 2004

TPT [Wed, 5 May 2004 19:27:29 -0700]:
>i think this issue has just been grossely overlooked by MSFT.

Well, I've got some real bad news for you. The rc=4319 from
ReadFile was not expected (not documented either) so it was
getting no retries at all --zero. When I made that check, and
increased the time for retries to 3.4 seconds (may not have
needed all that), the read works fine, as expected, even after
a suspend/resume.

So, on this iPAQ 2215, your theory is ... hogwash (just kidding,
hogs don't need washing). ReadFile does work after a suspend.
No sky falling. But, thanks anyway. Whodathough rc=4319 from
ReadFile?

If you're wondering, I did this three times, loading a 14 MB
file. After a second or so I suspend (after it's started
loading). After a few more seconds (10-20) I resume. Read
continues. Play continues. I haven't checked to see how
many retries it takes, but I think this is simply the SD
card not yet ready, which I first mentioned.

--
40th Floor - Software @ http://40th.com/
iPlay : the ultimate audio player for iPAQs
mp3, ogg, mp4, m4a, aac, wav, and then some

The
Wed May 05 22:36:22 CDT 2004

> Well, I've got some real bad news for you.

you mean good news.

> So, on this iPAQ 2215, your theory is ... hogwash

so your good news is that file handles get remounted correctly on the iPAQ
2215. good to know.

but that's not really usefull, since apps need to handle the worse case, and
the worse case is that it does not work on some devices.

and that's not a theory. try it on the Toshiba e800, it fails. in fact the
developers of BetaPlayer noticed that too, and had to work around the
problem. so no theory here. i never theorized anything, it was all
observed.

we also have some new (not yet announced) WM2003-SE devices, where it fails
too.

try it on all you other WM2003 / WM2003-SE devices (if you have some).

hel
Wed May 05 22:48:31 CDT 2004

TPT [Wed, 5 May 2004 20:36:22 -0700]:
>so your good news is that file handles get remounted correctly on the iPAQ
>2215. good to know.

Works fine on my iPAQ 4155, too.

>and that's not a theory. try it on the Toshiba e800, it fails.

Only that one that you're sure of? In that case, leave
a notice:

TOSHIBA users: Do not suspend while playing

> i never theorized anything, it was all observed.

But you should know Toshiba, or any particular device, is subject
to messing up spectaculary, but that's a OEM problem, not the
world's.

>we also have some new (not yet announced) WM2003-SE devices, where it fails

Right, your toshi with se on it. You talk about it, or whatever
it is, enough.

>try it on all you other WM2003 / WM2003-SE devices (if you have some).

I tell you what. You tell me what devices it fails on. So far
you say the e800. That's one. I've already got two that it
does work on. (You mean to tell me you don't have ANY iPAQs?
I suppose not or you would have known that this is not a common
problem.)

Here you go:

o Which devices do you know for a fact that it doesn't work on?

o Which devices do you know for a fact that it does work on?
(And I don't mean my two.)

--
40th Floor - Software @ http://40th.com/
iPlay : the ultimate audio player for iPAQs
mp3, ogg, mp4, m4a, aac, wav, and then some

The
Wed May 05 23:01:27 CDT 2004

> >we also have some new (not yet announced) WM2003-SE devices, where it
fails
>
> Right, your toshi with se on it.

No, those are no from Toshiba.

hel
Wed May 05 23:07:56 CDT 2004

>No, those are no from Toshiba.

That's not useful to know. Hey, didn't I see you on
the street corner with a sign in your hands saying
something like

Developer warning: file handles
can become un-usable after power
OFF/ON : REPENT YOUR SINS

Wasn't that you? Robe, sandles, grey beard?

--
40th Floor - Software @ http://40th.com/
iPlay : the ultimate audio player for iPAQs
mp3, ogg, mp4, m4a, aac, wav, and then some

The
Wed May 05 23:48:11 CDT 2004

> That's not useful to know.

Sorry we are under NDA.

I don't think any of your comments in your thread were particularly useful
either, but thanks for you interest in this issue.

bye.

hel
Thu May 06 08:17:49 CDT 2004

TPT [Wed, 5 May 2004 21:48:11 -0700]:
>Sorry we are under NDA.

OK, that's two attempted evasions. Here you go, answer these:

1. Which devices do you know for a fact that it doesn't work on?

2. Which devices do you know for a fact that it does work on?
(And I don't mean my two.)

If those look familiar, it's because I've asked them a few hours
ago. Why you always talk about your "NDA" devices I don't know,
seems like the point is you shouldn't talk about them at all,
but if you want to play that game, then answer this:

2. Which devices do you know for a fact that it does work on?
(And I don't mean my two.)

In other words, you say it doesn't work on your e800 (and no,
you haven't convinced me that it's an e800 -- from here it's
more likely your problem), but what device does it work on?
If you can't make it work on any device, then it's obvious
the problem is yours alone.

>I don't think any of your comments in your thread were particularly useful
>either, but thanks for you interest in this issue.

The problem is you do this at least once a month or so, and
it's always the end of the world, but the fact turns out to
be it's either, a) an old problem already fixed, or, b) your
Toshiba e800 doing something funky. I've yet to see anything
happen on your street corner that's applied to anyone but you.

And you still haven't proven anything at all, except that
your code doesn't work.

--
40th Floor - Software @ http://40th.com/
iPlay : the ultimate audio player for iPAQs
mp3, ogg, mp4, m4a, aac, wav, and then some

Chris
Thu May 06 09:31:48 CDT 2004

Based on the behavior that the two of you are seeing, I'm highly inclined to
agree with hel@... If you see differences in API behavior on different
devices running the same OS, then it's very unlikely that it's an OS bug or
error. It's far more likely to be a bug in the OAL/HAL that the OEM wrote,
and as such it's not Microsoft's responsibility.

Sure, developers should be aware of it, and I agree with you that the
information should be published. Kudos to you for doing that. I don't
however agree that Microsoft should fix it nor be responsible for making the
OEM fix it. The most they should do is make sure that a similar test is
included in the Pocket PC licensing process to hopefully let their OEMs
catch it in the future, but it's unreasonable to expect Microsoft to test
every API under every condition on every device that ships with their OS.

--
Chris Tacke, eMVP
Co-Founder and Advisory Board Member
www.OpenNETCF.org
---
Windows CE Product Manager
Applied Data Systems
www.applieddata.net

The
Thu May 06 14:59:06 CDT 2004

you can do a very simple test:

with file explorer, copy a large file (3MB or more) from a storage card to
the main memory.

suspend/resume (i.e. power off/on) the device while the copy is in progess.

in most cases this will cause the copy to fail.

The
Thu May 06 15:07:05 CDT 2004

> And you still haven't proven anything at all, except that
> your code doesn't work.

and you have proven that you are arrogant, that you are not someone people
would like to have a conversation with, and that you certainely will do
nothing to help the situation. i was not expecting you to.

you think the problem does not exist or is irrelevant. that is just your own
personal opinion. it's fine.

but there are several people at MSFT that are now looking at the problem.

what will they do about that ?

maybe they will end up providing better testing tools that will help OEM
make sure that they implement their drivers properly. maybe they will
document the behavior and make sure developers know about it and take the
appropriate steps.

certainely they will do something more useful than you about this issue.

hel
Thu May 06 16:23:44 CDT 2004

TPT [Thu, 6 May 2004 13:07:05 -0700]:
>and you have proven that you are arrogant, that you are not someone people

That's another non-answer. You aren't helping anybody
with your baseless assertions (but you sure have wasted
a a lot of time). When you're challenged to prove it, you
don't, can't, won't. Maybe next time you'll come up with
some real proof, instead of anecdotal piddle from your
"NDA" whatevertheyares.

>but there are several people at MSFT that are now looking at the problem.

Has anything you ever sent to "several people at MSFT"
ever gotten anywhere? I think they just want to get
you off their backs. haha

--
40th Floor - Software @ http://40th.com/
iPlay : the ultimate audio player for iPAQs
mp3, ogg, mp4, m4a, aac, wav, and then some

The
Thu May 06 16:41:41 CDT 2004

> Has anything you ever sent to "several people at MSFT"
> ever gotten anywhere?

Oh yes.

You can contact Reed Robison or Sam Henderson if you want details on all
those bugs they have logged in their internal database following my reports.

Do you want some the case numbers ?
SRX040407604895
SRX040407604847
SRX040407604954
SRX040408603548
SRX040407604797
SRX040407604386
SRX040407604217

Happy now ?

Or do you have yet another arrogant or agressive remark ?

You should meet Jacek, I'm sure you'll like him :)

hel
Thu May 06 17:25:32 CDT 2004

TPT [Thu, 6 May 2004 14:41:41 -0700]:
>Oh yes.

Oh, so you answer pointless questions but you
avoid answering those that are directly related
to your doomsaying? Here you go again for the
fourth time.

1. Which devices do you know for a fact that it doesn't work on?


2. Which devices do you know for a fact that it does work on?
(And I don't mean my two.)

If you can't get it to work on any device, then
the problem is obviously your code. Since you
don't answer, it's a good bet you know this.

Yeah, it'll blow over in a few days, but then
you'll be back doomsaying again soon enough.

--
40th Floor - Software @ http://40th.com/
iPlay : the ultimate audio player for iPAQs
mp3, ogg, mp4, m4a, aac, wav, and then some

The
Thu May 06 18:36:06 CDT 2004

> 1. Which devices do you know for a fact that it doesn't work on?

e800 and some new devices (not from Toshiba) running WM2003-SE. We will be
running more tests on other devices.

You can easiely test that if you want to, using the File Explorer: just
suspend/resume the device while copying a file from SD to Main Memory.

> 2. Which devices do you know for a fact that it does work on?
> (And I don't mean my two.)

All Pocket PC 2000 and 2002 remount the file handles correctly after power
suspend/resume, except for the caveat described in KB article 811693:
http://support.microsoft.com/default.aspx?scid=kb;en-us;811693

Unfortunately many apps (including the file explorer) do not deal with the
problem described in KB article 811693.

You are correct that the problem may be caused by some bad drivers.

However, the product group commented that it is "By Design" that on WM2003,
file handles to Storage Card files become invalid after a suspend/resume.

If they are correct, then it's bad.

If they are not correct, then OEM's should fix their drivers.

Also, the situation described in KB article 811693 is not very good.

Ideally, after a power suspend/resume, a filesystem call involving a handle
to a file
located on a removable storage should:

- block until the driver determines if the removable storage has been left
unchanged in the device, or if it has been removed, or modified while
suspended.

- then, return an error if the driver determines that the removable storage
has been removed, or has been modified while the device was suspended (e.g.
SD card moved to camera, photos taken, then moved back in device).

- and if the driver determines that the removable storage is still there and
un-modified, the handles should be remounted automatically and the call
should perform normally, as if nothing happened (except for the longer delay
before returning).

This is, IMHO, what should happen. This would allows the power
suspend/resume to be virtually transparent to all applications.


>
> If you can't get it to work on any device, then
> the problem is obviously your code. Since you
> don't answer, it's a good bet you know this.

Our code works fine on all devices included those that do not remount the
file handles correctly after power suspend/resume. That's because we deal
with the issue.

I already said that in a previous posting. Do you want to ask the question
again ? I'd be please to answer it a third time.

The
Thu May 06 21:01:11 CDT 2004

MSFT just confirmed that:

On WM2003, applications should not expect storage card file handles to be
remounted correctly after power suspend/resume (even if the storage card was
not removed from the device while suspended).

Developers, you need to be aware of that.

As a side effect, it means that applications installed on storage cards can
quit unexpectidly upon power-on, even if they don't access any file located
on a storage card.

Something to keep in mind, too.

hel
Thu May 06 21:24:37 CDT 2004

Well, after all you still didn't answer my very
simple question(s). Evasion #4 for you.

Here you go, try to answer it one device per line. I want
actual brands and models, not vague "all our 2002s work".
Do not include any 2002 models. This is strictly 2003.

1. Which devices do you know for a fact that it doesn't work on?


2. Which devices do you know for a fact that it does work on?
(And I don't mean my two.)

Maybe that's too hard for you to grasp? Here's what I mean.
If you asked me that I'd say this:

1. I don't know of any devices that "do not mount their
file handles" after a suspend resume. Both my 2003
devices have access to their already-opened handles
after delaying the well-known three seconds (by
retrying)

2. iPAQ 2215
iPAQ 4155

Now, if you, TPT (no idea which one you are), don't have
any devices for #2, that means you simply are not handling
the SD card (or CF) drive-not-ready problem. And if you
DO have devices for #2, you shouldn't be wasting my time.

--
40th Floor - Software @ http://40th.com/
iPlay : the ultimate audio player for iPAQs
mp3, ogg, mp4, m4a, aac, wav, and then some

hel
Thu May 06 22:04:11 CDT 2004

I think you made it up, reworded it to suit your...needs.
(The part in parens, for example -- I think that's all
yours.) I wouldn't be too surprised if the Toshiba e800
was messed up (or any particular device), but since neither
my iPAQ 2215 nor my iPAQ 4155 fail the way you say your
Toshiba does, I'll wait until I see something from someone
credible.

TPT [Thu, 6 May 2004 19:01:11 -0700]:
>MSFT just confirmed that:
>On WM2003, applications should not expect storage card file handles to be
>remounted correctly after power suspend/resume (even if the storage card was
>not removed from the device while suspended).
>Developers, you need to be aware of that.
>As a side effect, it means that applications installed on storage cards can
>quit unexpectidly upon power-on, even if they don't access any file located
>on a storage card.
>Something to keep in mind, too.

--
40th Floor - Software @ http://40th.com/
iPlay : the ultimate audio player for iPAQs
mp3, ogg, mp4, m4a, aac, wav, and then some

The
Thu May 06 22:13:04 CDT 2004

> 1. I don't know of any devices that "do not mount their
> file handles" after a suspend resume.

But you will find some pretty soon. But hopefully now you are aware of this
issue and your code will deal with it.

If your applications can riun on the e800, they should already handle this
case and re-open the file (not just re-try the read), if not, the storage
card audio file that you are playing will break upon suspend/resume.

The
Thu May 06 22:21:48 CDT 2004

<hel@40th.com> wrote in message
news:u5wc5$9MEHA.2976@TK2MSFTNGP10.phx.gbl...
> I think you made it up, reworded it to suit your...needs.

I made up nothing, and we have no related "need" (??? no idea what you mean
there).

> (The part in parens, for example -- I think that's all
> yours.)

No, that's what MSFT's official answer was. The case number is
SRX040407604217, if that's of any use.

> I'll wait until I see something from someone
> credible.

That's a good idea. And if you see that I was correct, then I expect
sincere appologies from you. Deal ?

hel
Thu May 06 22:33:13 CDT 2004

>> (The part in parens, for example -- I think that's all
>> yours.)

TPT [Thu, 6 May 2004 20:21:48 -0700]:
>No, that's what MSFT's official answer was. The case number is

Yeah, but see, I don't believe you anymore.
Nothing you have presented is proof. You
just make claims, that's it. Zero proof.
Nada. Zip. Upso. And you evade answering
questions that might help support your case.
It makes no sense for a reasonable person
to do that. I mean, I think you're looney!
--
40th Floor - Software @ http://40th.com/
iPlay : the ultimate audio player for iPAQs
mp3, ogg, mp4, m4a, aac, wav, and then some

The
Thu May 06 22:40:36 CDT 2004

> Nothing you have presented is proof.

You can contact MSFT if you are interested.

MSFT will probably publish a KB article about this issue, i.e. a
clarification regarding what applications can/should expect when
suspend/resume occures while they have open file handles on storage cards.

Then you will have the proof you want.

Untel then, since you don't believe anything I said, I recommand that you
consider that file handles are always re-mounted, and deal with customer
support on a case by case when this does not happen.

The
Thu May 06 22:54:49 CDT 2004

You may want to check this thread:

http://www.pocketpcthoughts.com/forums/viewtopic.php?t=27614&postdays=0&postorder=asc&start=10

Some other people have confirmed the problem on the iPaq 2215 with the new
1.10 ROM and on some iPaq 5550 (depends on the ROM).

I also know that picard (developer of BetaPlayer) confirmed the problem on
the e800. You can contact him on this forum if you don't trust me:
http://www.pocketpcthoughts.com/forums/viewtopic.php?t=27589&postdays=0&postorder=asc&start=0

But all those people probably lied, right ?

Kirk
Fri May 07 02:41:06 CDT 2004

> On WM2003 (and WM2003-SE), when you open a file located on a storage
> card, the file handle become un-usable after power OFF/ON (i.e. power
> suspend/resume).

If the opened file is a database (with ADO or OLEDB), what's happening ?

The
Fri May 07 03:25:55 CDT 2004

> If the opened file is a database (with ADO or OLEDB), what's happening ?

It depends if the database engine is smart enough to deal with the problem.

Most likely, it is not smart enough, and most likely, you will get data
corruption if you suspend the device while the database engine is modifying
the database.

But again, maybe I'm wrong.

Michael
Fri May 07 10:58:23 CDT 2004

Wasn't try to indicate it's not a problem. Was just proposing a "solution".
Re-use whatever handler you invoke when the card is yanked. It would be
nice if you could detect this situation based on a power-on event or other
event instead of ReadFile/WriteFile errors out w/ERROR_ACCESS_DENIED, but if
that's the way you have to detect it, so be it.

Re: possible corruption issues, you might try flushing every write.
--

Michael Salamone [eMVP]
Entrek Software, Inc.
www.entrek.com


"The PocketTV Team" <do-not-reply-by-email@pockettv.com> wrote in message
news:eqU7U4uMEHA.556@TK2MSFTNGP10.phx.gbl...
> "Michael J. Salamone [eMVP]" <mikesa#at#entrek#dot#com> wrote in message
> news:efynO%23qMEHA.2244@TK2MSFTNGP10.phx.gbl...
> > File handles become un-usable if you yank the card, too (while device is
> > still powered on). So you need to handle that type of situation anyway.
>
> Yes, of course, but that's different. That'a a fatal situation: the file
> becomes permanently un-usable because the card has been removed.
>
> But that's not what the problem is about, here. The problem here is that
> case where power is suspended/resumed, without removing any removanle
> storage.
>
>
> > Is there any notification? That is, if you use SHChangeNotifyRegister
and
> > the card is pulled, you would get WM_FILECHANGEINFO/SHCNE_MEDIAREMOVED
> > message if the user pulled the card out. If your app is using storage
> card
> > files, then it already is (or should be) registering for this
> notification,
> > and properly handling it.. WM_DEVICECHANGE also works on some devices,
> but
> > I found SHChangeNotifyRegister is more reliable.
>
> Again, that's not the question. The problem does not involve removing any
> card.'
>
> And it makes it very complicated for apps to deal with re-opening files
each
> time an error occurs because of this problem.
>
> In fact MSFT is now looking seriously at this issue, since it also breaks
> most of MSFT's own apps. The only one that correctly deals with
re-opening
> files on removable storage after power suspend/resume is WMP. All the
ather
> MSFT apps break in this case.
>
>
> > Anyway, do you get that same notification, or any other notification, on
> > suspend/resume? That would be the real problem, if no notification was
> > sent. Otherwise use the same handling you have for
> > WM_FILECHANGEINFO/SHCNE_MEDIAREMOVED or WM_DEVICECHANGE.
>
> Well, ReadFile returns an arror, so you get notified, in a way. But
that's
> not what is supposed to happen.
>
> In WinCE 3.0 (and earlier), all handles to storage card files were
remounted
> correctly and automatically, sometimes after a small delay during which
> ReadFile would return ERROR_ACCESS_DENIED, ERROR_NOT_READY or
> ERROR_DEVICE_NOT_AVAILABLE. so you just needed to retry, as described
here:
>
> http://support.microsoft.com/default.aspx?scid=kb;en-us;811693
>
> But now, in WinCE 4.2, you can retry as much as you want, the handles are
> never remounted.
>
> That's causing all sorts of troubles when people power off their devices,
> while running apps have files opened on Storage Cards.
>
> And I don't even want to think of the disaster if some running apps are
> themself installed on the Storage Cards!
>
>
> > --
> >
> > Michael Salamone [eMVP]
> > Entrek Software, Inc.
> > www.entrek.com
> >
> >
> >
> > "The PocketTV Team" <do-not-reply-by-email@pockettv.com> wrote in
message
> > news:eqo$%23vnMEHA.3292@TK2MSFTNGP11.phx.gbl...
> > > On WM2003 (and WM2003-SE), when you open a file located on a storage
> card,
> > > the file handle become un-usable after power OFF/ON (i.e. power
> > > suspend/resume).
> > >
> > > This problem is described here in more details:
> > >
> > > http://www.pocketpcthoughts.com/forums/viewtopic.php?t=27614
> > >
> > > If you app opens files on a storage card (or in the application home
> > folder,
> > > which can be located on the Storage Card), you should be aware of this
> > > issue, and implement the necessary workarounds, if needed. If you
> don't,
> > > you application can mis-behave after the user presses the Power
button,
> or
> > > it can cause data corruption.
> > >
> > > This issue affects some (maybe all) Pocket PC 2003, including the
> Toshiba
> > > e800 series.
> > >
> > > I believe that this problem also affects Smartphone 2003 devices,
> although
> > > not all Smartphones are capable of suspending (The Samsung i600 does
> > suspend
> > > when closed, if no earphone jacked is used).
> > >
> > >
> >
> >
>
>

KS
Fri May 07 11:08:21 CDT 2004

I've seen this problem even on PPC2002. I used OLEDB to connect to SQL
Server CE. If the device is suspended or cold booted while writing to the
database, I've seen it corrupt the .sdf file.

KS

"The PocketTV Team" <do-not-reply-by-email@pockettv.com> wrote in message
news:%23pg7qzANEHA.2244@tk2msftngp13.phx.gbl...
> > If the opened file is a database (with ADO or OLEDB), what's happening ?
>
> It depends if the database engine is smart enough to deal with the
problem.
>
> Most likely, it is not smart enough, and most likely, you will get data
> corruption if you suspend the device while the database engine is
modifying
> the database.
>
> But again, maybe I'm wrong.
>
>

The
Fri May 07 17:41:02 CDT 2004

"KS" <ks@blah.com> wrote in message
news:e7sLG2ENEHA.3452@TK2MSFTNGP10.phx.gbl...
> I've seen this problem even on PPC2002. I used OLEDB to connect to SQL
> Server CE. If the device is suspended or cold booted while writing to the
> database, I've seen it corrupt the .sdf file.

I'm not surprised.

The
Fri May 07 17:43:03 CDT 2004

> Re: possible corruption issues, you might try flushing every write.

corruption can happen even if you flush every write. it can happen if you
re-open a
file, thinking it's the same (e.g. because it has the same name and size),
but when in fact it is not the same (e.g. because the storage card was
removed and modified while the device was suspended).

or more simply, it can happen if the device is suspended after you write and
before you flush!

Michael
Sat May 08 10:01:14 CDT 2004

Sure there's a window the device can be suspend after a write and before a
flush, but it's a pretty small window - and you've greatly reduced the
possibility of becoming corrupt..

As for determining the same file or not, you can put a unique signature at
the beginning of your files and see if the signature is the same as the last
signature when you open. Not ideal, and won't work real well if it's a text
file, but it will help in many cases.

Again, just trying to give ideas on how to workaround - because what else
are you going to do?

--

Michael Salamone [eMVP]
Entrek Software, Inc.
www.entrek.com



"The PocketTV Team" <do-not-reply-by-email@pockettv.com> wrote in message
news:uBSIoSINEHA.3380@TK2MSFTNGP11.phx.gbl...
> > Re: possible corruption issues, you might try flushing every write.
>
> corruption can happen even if you flush every write. it can happen if you
> re-open a
> file, thinking it's the same (e.g. because it has the same name and size),
> but when in fact it is not the same (e.g. because the storage card was
> removed and modified while the device was suspended).
>
> or more simply, it can happen if the device is suspended after you write
and
> before you flush!
>
>

The
Sat May 08 16:05:04 CDT 2004

> Again, just trying to give ideas on how to workaround - because what else
> are you going to do?

Encourage MSFT to ask (or require) OEM's to make drivers that correctly
remount file handles in that case, as they do on all Pocket PC 2000 and
2002.

I know it's a long shot...

Reed
Fri Jun 11 10:52:41 CDT 2004

Folks,

I wanted to add some closure to this-- I've spent quite a bit of time
looking at it here across a number of devices and wanted to give you a
better answer. Developers *should* have logic to re-establish file handles
if needed. Essentially, this is what we do in a lot of MS apps to ensure
playback and functionality can continue if the handle is broken. That said,
the behavior does indeed vary across devices-- especially with
suspend/resume and removing/re-inserting SD cards. With the 2003 platform
forward, we decreased a timeout aimed at making the OS more aware of card
removal/insertion (it happens faster). This can also have the affect of
tearing down files handles faster in regards to suspend/resume, etc. This
is not the only factor involved here, but I believe it is responsible for
most of the variations you see around this issue. If you find yourself
bumping into this problem, you can try increasing this setting back up to
4096 if it is lower on a device where you see the problem.

HKEY_LOCAL_MACHINE\System\StorageManager
PNPUnloadDelay : 4096

If you do change this, make a record of what it was set to so you can change
it back if needed. I would also point out that it's better to handle the
scenario in your code. This is not typically a setting you should change
or alter but I wanted to share it for ISVs that bump into this and are not
able to quickly make changes to support a newer device that this might occur
on.

Hope this helps,
Reed Robison

This posting is provided "AS IS" with no warranties, and confers no rights.

Check out Mobility Development FAQs at
http://msdn.microsoft.com/mobility/prodtechinfo/devtools/netcf/FAQ/default.aspx

Use of included script samples are subject to the terms specified at
http://www.microsoft.com/info/cpyright.htm



"The PocketTV Team" <do-not-reply-by-email@pockettv.com> wrote in message
news:e8RB3E%23MEHA.2052@TK2MSFTNGP10.phx.gbl...
> > 1. I don't know of any devices that "do not mount their
> > file handles" after a suspend resume.
>
> But you will find some pretty soon. But hopefully now you are aware of
this
> issue and your code will deal with it.
>
> If your applications can riun on the e800, they should already handle this
> case and re-open the file (not just re-try the read), if not, the storage
> card audio file that you are playing will break upon suspend/resume.
>
>

hel
Fri Jun 11 11:27:01 CDT 2004

RR [Fri, 11 Jun 2004 11:52:41 -0400]:
> Developers *should* have logic to re-establish file handles if needed.

Based on what historical precedent? No one does. Posting a
"[you should do that because we don't know what we're doing here
most of the time]" is not going to get very far. What you need
to do is not do that unless the card is gone. Simple. If you
blew it, unblow it, don't tell me what I " *should* have " done.
*You* should have done it right. *You* should make it work.

But, still, none of my iPAQs exhibit this. Maybe those OEMs
using your code without checking it out are failing bigtime,
I don't know, but you get out what you put in.

> this is what we do in a lot of MS apps to ensure
> playback and functionality can continue if the handle is broken.

I noticed in 2003 WiMP finally doesn't restart playback when it's
been paused for 15 seconds. That was broken for what, three years?
Was what the reason for pause/restart playback that nonsense?
The point to this is, you guys are very slow fixing PPC problems,
and forcing you *should*s to everyone else is amateurish. You guys
really have 56 billion dollars cash in the bank now? Why not use
some of it to fix things, for real,