Special thanks to Rob Jonson of Hobbyist Software for contributing this article.

An unstable palm can drive you mad. It’s worse when you can’t pin down what application is causing the problem.

If you get crashes that seem random and that don’t seem to be related to specific actions, then this article might be able to help. I’ll explain how some of those crashes come about, and give some tips on how to track them down.

This type of crash applies to all Palm devices that offer NVFS memory. That’s all the Treos from the 650 onwards, the Life Drive,Tungsten E2,Tungsten T5,Tungsten X and Zire Z22 and probably more to come.

We’ll be using Palm Internals to peer inside your Treo, and Reset Doctor to manage the crash, and to help make those crashes happen more often! By forcing the crashes to happen more often, and making them easier to reproduce, you’ll be able to tell more quickly where the actual problem is!

A quick cache recap

I’ll be using the Treo 650 as an example here, but although the memory sizes are different, the same applies to all devices with NVFS.

So, your Treo 650 claims in it’s specification that it has 32MB (23MB user accessible) memory. This is true, but it isn’t the memory that you are used to having.

• You can’t use that 23MB to run an application.

• Applications that are running can’t grab chunks of that memory to use for calculations.
• It’s not at all like the memory that your Treo 600 had!

Actually, it’s almost exactly like a 32MB sd card that is built into your palm. I’ll refer to this as the NVFS memory from now.

Palms can’t run applications from SD cards though. You have to copy them to RAM, and run them there (your launcher does this for you - if it doesn’t, then get Initiate!). You’ve probably even noticed the delay when you launch an application from your SD card while your launcher copies it over.

The same limitation applies to the NVFS memory. You can’t run applications on it, and the system can’t use it for working memory to perform calculations etc. To do this sort of stuff, the palm has some normal RAM memory. On the Treo 650, it has about 12MB. This is the DBCache.

Applications that are running on your device just think they’re dealing with a normal device with 23MB of RAM. They expect to be able to access databases, make databases and use memory. This means that the system has to do some very fancy footwork to dynamically load databases and applications in to the DBCache when they are needed. Of course, you can’t put a quart in a pint pot, so with 23MB of NVFS and only 12MB of DBCache, the DBCache can get full. At this point the system has to clear out stuff that it doesn’t think is needed. This is where it can all go wrong.

You can read in more detail about the NVFS and DBCache at Hazleware.

This is all horribly complicated, but there are a couple of excellent reasons why Palm do it!

• Just like an SD card, your NVFS memory doesn’t need battery power to keep its data. This means you can run out of battery completely and you don’t lose your data.

• As the NVFS memory doesn’t need battery, the palm gets better battery life.
• NFVS memory is much cheaper to buy. This lets Palm give us more memory in our device whilst keeping the price down. Frankly, I’m surprised they haven’t given us a 1gig device yet!

Where it all goes wrong!

In my last article Inside Your Treo: Notifications, Background applications and your Blazing-Fast Treo! I explained how applications can register for notifications. When specific events happen, the operating system calls the application and gives it a chance to do stuff.

Here is how it can go wrong:

1. An application (we’ll call it badApp) registers for the ‘hvch’ notification. This is the notification that is sent when any virtual character is processed. You get virtual characters all the time; When you press a hard key, when the battery reaches a certain level, when an alarm goes off, etc.

2. badApp cares about speed, so it registers ‘the fast way‘. This means that it just tells the system to call a function at a certain memory address when the notification is triggered.

3. badApp was written years ago and doesn’t know about NVFS, so it doesn’t do anything more. It just sits and waits for notifications.

4. This all works fine for several hours while you read documents, play games and more.

5. At some point, the DBcache gets full.

6. The system looks through the DBcache and sees that no one is using badApp, so it copies any changes back to the NVFS and deletes it from the DBcache.

7. Almost immediately the system gets a virtual character, so it tries to run the code at the memory address that was registered with the notification. What’s there now is the web page you’re reading, so the system crashes.

8. If you check your crash log, you’ll see an error like “Emul68KMain.c, Line:403, illegal instruction xxxx at address xxxxxxxx”

Most applications now know about NVFS, so at step 3, they lock their code which stops the system from clearing them out of DBCache.

Cache Crashes - Are you getting them?

DBcache crashes are really hard to pin down. They may or may not cause the error type listed above, and you never really know when they’ll happen.

One easy way to check is to use Reset Doctor. It lets you push a button and clear your DBcache. If that causes a crash (or if you get one very shortly after clearing the DBcache) then you have a problem.

Reset Doctor also lets you automatically clear the DBcache to keep a certain amount of memory free. Reset Doctor clears out the cache after if needed after one application exits, and before the next starts. This means that DBcache clearing is unlikely to happen while an application is actually open (another thing that could potentially cause problems). More importantly, it can save when it clears the DBcache and when resets happen in a log. If your log shows that crashes are happening shortly after the DBcache is cleared, then that’s another sign you have this problem.

Cache Crashes - What you can do

It’s fairly easy to to find most applications that register for notifications ‘the fast way‘, and it’s easy to see whether they have locked themselves in place. here is what you do:

1) Run Palm Internals and tap on the ‘Notif’ button. You’ll see a list as in the image to the right.

*If you are in a really bad way - then even running Palm Internals will cause the cache to clear and trigger a reset. If this is the case, then you will just have to move applications (start with any you are suspicious of) to your sd card, do a reset and then try Palm Internals again. When you have it working, you can gradually move applications back and keep an eye out for problems.

2) Scroll through and make a list of any applications that have ‘m68k xxxx DBCache’ after their registration (these are applications that have registered ‘the fast way‘. My list would have Butler and badApp. This is your list of suspects.

3) Now select menu/Locked DBs. This shows the which databases are locked.

4) Check each of the applications on your ‘list of suspects’. If one of the applications does not have it’s code resource locked, then it will probably cause problems. Here, Butler is locked, but badApp is not.

5) Now select menu/Protected DBs. This shows the which databases are protected from deletion.

6) Check each of the applications on your ‘list of suspects’. This is a less important check, but applications should protect themselves so that they can know when the system wants to delete them. Again, Butler is protected, but badApp is not.

You now have a list of applications that look dodgy! Try deleting them (or just move them to the SD card). With any luck, your palm will now be more stable!

Once you have pinned down the troublesome app, then you can either live without it, or use an application like Resco Locker to lock it. As a side bonus, now you know which applications are problematic, you don’t need to lock any others! Overuse of Resco Locker quickly fills your cache and seems to cause problems of its own!

Is it that easy?

Hopefully it is! Unfortunately, there are other things that could be happening.

1. Your suspect list isn’t quite complete. Some applications hack directly into the system rather than using notifications. Anything that seems to do stuff in the background, but which doesn’t appear in the notifications list should be added to your suspects. LudusP is an example of an app that hacks into the system rather than using notifications (though I’m sure it doesn’t leave itself unprotected!).

2. Your suspect list isn’t quite complete. Some applications only register for notifications in specific circumstances. For example, an application might register the slow way for phone related notifications, but register ‘the fast way’ for key press notifications while calls are active.

3. Something else ? Who knows - Palm has a special way of surprising us…

And there are exceptions…

Life is never easy with Palm, but there are always interesting things for developers to do.

One trick to avoid having to lock your code is just to copy it into some feature memory. This is normal memory that the OS knows to leave alone.

This technique is used by Daniel Grobe Sachs and possibly others.

Unfortunately I don’t know of a way that you can check if an app is using this technique.

Related Links

Inside Your Treo: Notifications, Background applications and your Blazing-Fast Treo!

Hobbyist Software Programs

Hobbyist Software Forum

Palm Internals

Reset Doctor

Resco Locker

Hobbyist Software Website

Filed under: Editorials