---

Memory management on the Treo 650

The introduction of the NVFS (Non-Volatile File System) on the Treo 650 has had many benefits but also caused a few issues unique to the Treo 650.

What is the problem?

If you often get a "There is not enough memory available..." message (typically when you start Blazer) despite having a couple of MB of free memory, you are likely running into the hidden, secondary memory limit of the Treo 650 - the NVFS DBCache memory limit. Not everyone will run into this problem – it really depends on which apps you run and how much total memory they use. Note that this is similar to, but not the same as the Blazer internal web page cache.

This FAQ was written primarily for the following ROM updates (referred to as "earlier versions of the firmware" below)
SprintPCS 1.12 and 1.12a
Unlocked GSM 1.13
Cingular 1.15
Rogers 1.13
Verizon 1.03

If you are using any of the following updates, the good news is that Palm has made significant improvements to the DBCache manager which has resolved many of the issues described below (referred to as "later versions of the firmware" below). The rest of this article is still useful information to help your understanding of what is going on inside the Treo.
SprintPCS 1.13
Unlocked GSM 1.20
Cingular 1.17
Verizon 1.04
Alltel, Cellular South, Telus Mobility and other CDMA 1.04

What is NVFS?

There are two types of memory in the Treo 650

1. 32MB of fast DRAM (dynamic) RAM which is used for running apps and PalmOS
2. 32MB of slower NVFS (flash) RAM which is used for persistent storage of PalmOS and your data and apps

The Treo 650 was one of the first Palm devices to use an NVFS memory system (also used on the LifeDrive and Tungsten E2 and T5). The good thing is that it prevents data loss when you run out of battery power. However, there are some new things to deal with in the current implementation in PalmOS 5.4 (Garnet).

For a good overview of the NVFS implementation, read this articlefirst. From the Resco Backup page∞, this is how NVFS is used on the Treo 650:





As shown in this diagram from the PalmOne site,



The "User Data" memory block is where data and programs are stored when they are not in use. Because this block is NVFS/flash memory, all the information in this memory block is retained even if the battery runs empty or is removed.

The DBCache block is where running programs and data being accessed is stored. The DBCache memory can be thought of as a "staging" area between fast dynamic RAM and slower NVFS/flash memory. Since this block is DRAM, nothing written here is permanent until the cache manager writes it out to the "User Data" block which is in NVFS/flash memory.

The "Dynamic Heap + Misc" block is where apps and PalmOS store data temporarily while running. Because it's in DRAM, it is lost after a reset or if the battery empties or is removed.

One final reference regarding the PalmOS NVFS memory management system presents more detail on the different memory blocks and how they are used during the initial boot-up sequence. You can find the article written by TreoCentral user mmastrac here∞. It provides a very good insight into how this works and how you can increase the DBCache size by making a custom ROM. The actual process of making a custom ROM is covered by other articles in the Knowledge Base and various threads here and at TreoCentral.

The PalmOS cache manager is supposed to manage the DBCache memory block so that updated data gets written back to flash while yet allowing new data to come in (when a new app is launched or database is opened) without the cache getting full again. In addition, free memory can get fragmented as apps start, stop, or use memory. This fragmentation can limit the size of the largest contiguous chunk of free memory that an app can request.

Unfortunately, the PalmOS cache manager in the early versions of the firmware wasn't effective enough managing the DBCache block which resulted in these "There is not enough memory available..." messages. Typically, that simply means "there is not enough contiguous free DBCache memory available to run this app." The good news is the the cache manager in later versions of the firmware has resolved most of the issues.

What can I do about the problem?

There are six main options to workaround this problem.

1. Make sure you are running the latest revision of the firmware (see the table at the beginning of this article). PalmOne has made some efficiency improvements in how it allocates memory which results in more free memory (biggest benefit is for PIM data) but most importantly has made significant improvements in the cache manager which resolves most of the issues. PalmOS updates may be found here∞.
2. Move applications to an SD card and launch them from there. You can use the Palm built-in launcher or a 3rd party app like zLauncher, Initiate, PowerRun or LauncherX to do this. The result is that while launching apps will be slightly slower (since the files need to be copied to onboard memory first), you will have more free memory since the launched app no longer takes up memory when it’s not running. Note that some apps must remain in onboard memory such as those that sync to your PC/Mac or those while are always running in the background (e.g. Butler, TreoGuard, ProfileCare, etc) and generate alarms or respond to certain events (time of day, ringtone/call managers, etc).
3. Ensure you are running apps that were specifically designed to run on NVFS systems such as the Treo 650. Given that databases may be moved around when PalmOS is managing the DBCache area, some apps need to be updated to be compatible. While this may not help your “out of memory” problems, it will definitely help prevent other crashes.
4. If you can go most or all of the day before getting an “out of memory” message, the simplest approach might be to do a nightly soft reset (you may already do this in prep for nightly backups). The advantage is that a soft reset will automatically clear the db cache and allow you to start fresh where the db cache amount is very close to the amount of total free memory available. You can use apps like ProfileCare, mReset+LookAtMe, and others to schedule a nightly soft reset.
5. Use the dbCacheTool and Resco Locker apps. Although optional, it's recommended to also use PrefDoctor or SyncBeforeReset (see details below). Another useful utility is MemoryInfo which has some of the same capabilities as dbCacheTool.
6. Create a custom ROM since reducing the size of the ROM will increase the size of your DBCache area. For an explanation of why, see the article written by TreoCentral user mmastrac here∞. The actual process of making a custom ROM is covered by other articles in the Knowledge Base and various threads here and at TreoCentral.

What is dbCacheTool and how do I use it?

dbCacheTool is an app which clears the cache in order to make more room to load new data. Essentially, it does what the PalmOS built-in cache manager didn't do well on it’s own (in the earlier versions of the firmware). You can find dbCacheTool here∞and a rough (computer) translation of the readme (from Japanese) is here∞.

Some background on my Treo (when this article was written) for comparison with whatever you might have:


Here is a snapshot and notes on the dbCacheTool settings - my comments in ( ):






For my Treo, I couldn't get max alloc/free above 1.5MB anyway, so "max <" was set to 1MB. "free <" was set to 3MB as that allowed anything to run and yet minimized the number of times dbCacheTool runs and I had to wait for it to finish. Setting this value too high will cause dbCacheTool to run after just about every app and setting it too low would result in “out of memory” messages. You’ll need to experiment to see which settings work best for you. You can also set them to 0 to turn the check off completely but that defeats the purpose of the tool unless you use the dbCacheToolDA desktop accessory (see below).

The intent of the Apps->AppDetail settings seem to be to override the free < and max < settings on an app-by-app basis. However, I’ve never gotten them to work on the 650 (perhaps they work on the LifeDrive, T2 or E2 devices). On those screens, you'll see the following options:



Based on my interpretation of the readme and notes in this 1src forum thread∞ these options control when dbCacheTool will check the amount of available memory against “free <” and “max <” settings you’ve chosen:


Because backup operations touch every file in memory, they can fill up the cache quickly. To avoid this problem (assuming the Apps->AppDetail settings work), creating a backup app-specific setting would be useful to ensure the cache was cleared just before backups are run.

What is the dbCacheToolDA app?

The dbCacheToolDA app is a desktop accessory. You can use it with McPhling and other apps like it, but I use it with zLauncher. It allows me to have an icon for dbCacheTool in the bottom bar, so it only takes one press to get a small dialog showing amount of free memory and a "Clear" (now) button. Pressing the Clear button allows you to force a cache clear whenever you want or need to. If you don’t want to use the dbCacheTool Auto Clear function, this would be a way to clear the cache manually.

Is dbCacheTool a perfect solution?

Unfortunately no, it would be best to have the PalmOS cache manager perfected to avoid all other problems. In my experience, I saw two types of problems using dbCacheTool:

1. App-specific crashes - When dbCacheTool runs, it can move databases around in order to make more cache available. If another app didn’t expect that to happen, it may try to access that data using the original memory location which may no longer contain the expected data. I’ve seen this happen with Causerie (r3.1 and prior) and GoodLink (r4.0.0.59) when the “Clear at soft reset” option was checked. GoodLink and Causerie both reference runtime libraries which often get moved by dbCacheTool after a soft reset. The result is either a system crash or error messages indicating the app can no longer find certain runtime libraries. The solution for GoodLink was to turn off the “Clear at soft reset” option and use Resco Locker to lock Causerie into cache so it’s files won’t be moved.

2. Lost preferences - Fortunately this is rare, but since the PalmOS API doesn’t offer a standard way to clear the cache, dbCacheTool sometimes acts too aggressive in clearing the cache which can sometimes cause the Saved Preferences db to get corrupted. The only solution seems to be to restore the Saved Preferences.db file from a backup. There are other known causes of the “lost prefs” reset such as having “Enable Local Network Time” checked on Sprint/Verizon CDMA phones (in early versions of the firmware) and rogue Exchange Manager entries. Fix the former by unchecking the option and the latter by using Butler∞ or PrefDoctor∞to automatically fix those entries.

What is the MemoryInfo tool?

MemoryInfo is an app released by Palm as part of the SDK utilities package to members of the Developer's Plugged-In Program. It displays details of your current memory usage and also has some of the "cache clearing" functions of dbCacheTool. The app and forum thread is here∞. It's primarily useful to see exactly how your memory is being used and how much is available.


Similar to dbCacheTool, it also has options to clear the cache. The "Flush DBCache (MemFill)" option seems to be equivalent to the dbCacheTool "clear type: DB" setting and the "Flush DBCache (HotSync)" option is the dbCacheTool "clear type: sync" setting. "Flush DBCache (MemFill)" consistently frees more DB cache (for me at least).


The difference between MemoryInfo and dbCacheTool is that MemoryInfo comes directly from Palm (don't know if dbCacheTool did), dbCacheTool does an automatic cache clearing when needed (MemoryInfo is manual), and dbCacheTool has more options.

What is Resco Locker and why might I need it?

Resco Locker is a free utility which locks apps into dbCache so they can’t be moved by a cache clearing. From the Resco Locker page∞

How do I use Resco Locker?

In general, you should lock any apps that are constantly running in the background (e.g. TreoGuard, Fonts4OS5, KeyCaps, ringtone/call managers, etc) if they don’t automatically lock themselves into cache (Butler, Causerie (r3.2.3) and GoodLink are examples which lock themselves). Background apps are those which automatically respond to some type of event like the time of day, a hotsync operation, or a particular button or key press. Foreground apps are those you specifically need to launch yourself.

Another way to determine which apps runs in the background is to use the free PalmInternals∞utility to check the list of apps which register for hede notification. Finally, you may need to search the forums or contact the developer if you are not sure if an app should be locked or not. For example, VolumeCare and ProfileCare (from GoTreo Software∞) are two apps that run in the background but due to the way they are written, do not need to be locked.

Is it possible to lock too much with Resco Locker?

Apparently yes. Lock too much cache memory and it won’t be available for anything else so you might not be able to run some of your large apps. The more memory that's locked, the more often other apps are going to get swapped out of the cache. So I'd recommend only locking those apps that really need to stay in the background all the time and then only the ones where using Resco Locker seems to help. Resco Locker has a 4 MB app lock limit.

What are PrefDoctor and SyncBeforeReset and how do I use them?

PrefDoctor∞ is an effort to prevent unexpected resets where you've lost all your preferences. This app contains three features. The first automatically fixes rogue Exchange Manager entries (in fact, it's the same code as in Butler noted above and also a technique recommended by PalmSource), the second forces preferences to be saved every time an application runs, and the third allows you to restore the backup of the preferences database. As of the end of January 2006, this app was in beta. Results so far seem to indicate it has definitely helped some people reduce (but not eliminate) this type of reset. To add your comments, please visit this thread.

SyncBeforeReset is a slightly different approach in that it traps a call to the PalmOS reset function so that open databases (such as Saved Preferences.db) can be saved from the DBCache area to NVFS RAM before they get corrupted or the data is lost. This app was released on near the end of December 2005 and so far seems to be helping people, so it's another option to consider. To install it, you'll need two apps - SyncBeforeReset∞ and YAHM∞. To add your comments or see other's experience, please visit this thread∞.

Summary

This is a complicated issue, but hopefully this article will help share what information the Treo Community has figured out. We can only hope that PalmOne will continue enhancing PalmOS and improve the stability and usefulness of our favorite device – the do-everything, know-everything Treo.

As of the end of January 2006, indications from Cingular 1.17 ROM, Unlocked GSM 1.20 and Sprint 1.13 users was that they noticed much more effective automatic cache management in PalmOS without any of the apps noted above. It's safe to assume that these improvements will be included in future firmware updates for remaining carriers. To add your comments, please visit this thread.∞

Many thanks to the forum members here and at TreoCentral and the developers noted above for their individual contributions that helped form this article.

---

CategoryTreo650