This document is part of the Caché Installation Guide for OpenVMS. This document explains how you can calculate the best parameters for your system.
For optimal Caché performance, you need to calculate proper values for certain Caché system parameters. These values allow you to determine whether you need to adjust certain OpenVMS system parameters. The values you choose should minimize swapping and paging that require disk accesses, and thus improve system performance.
Review this section carefully and calculate the proper values for both your operating system and Caché before proceeding. Use the OpenVMS Parameter Calculator and the tables provided here to record the current and calculated values for your system parameters. You can then refer to these tables when you install Caché. After your system is running, you may need to adjust these values to gain optimal performance.
If you are not already familiar with the memory organization at your operating system level, read the overviews of process and physical memory allocation provided in the next two sections. You should also consult the appropriate system documentation. Then you can go on to the Calculating OpenVMS Parameters section.
OpenVMS Process Memory Organization
An OpenVMS process is any entity that can be scheduled. It may be interactive or run as a batch process. If the OpenVMS process creates subprocesses, that process and all its subprocesses together are called a job.
Many operating system tasks are OpenVMS processes. All processes, whether they are interactive, batch, or operating system-specific, share the structures that this section discusses. OpenVMS can run several processes concurrently. Depending on the size of your computer, OpenVMS could be running between 4 and 1,000 or more simultaneous processes.
Each process uses a share of physical memory, where it stores certain process-specific operating system information, any code the process is using, and any data that it uses or generates. For example, if a programmer edits a routine with an OpenVMS editor, OpenVMS loads the editor into memory, allocates space for the editor's buffers and variables and loads all or a portion of what is being edited into memory.
Shared Memory and Caché
Sometimes code or data is available to more than one process. The editor is one example of this type of code. Another example is the code for Caché and some of the data created with Caché.
Any piece of code or data that several processes can share is stored in a shared section of memory. Shared sections are created by the first user of a system, or during system initialization. They are then made available to subsequent users of that system rather than being duplicated and stored with each individual process.
Caché system code, global buffers, and routine buffers are stored in shared sections of memory.
How OpenVMS Measures Process Memory
OpenVMS measures the code and data area that a process requires in pages. Shared sections are also measured in pages, sometimes called shared or global pages.
On OpenVMS Alpha systems, page size is not fixed. Normally, it is 8192 bytes. Each page is divided into pagelets that are 512 bytes long.
Page Organization
Sometimes a process needs to access many more pages than can fit within the physical memory allocated to the process. When this situation arises, OpenVMS stores the extra pages on disk. The operating system keeps only the most active, or most recently used, pages in memory. The total of all the pages of a process, whether they are stored in memory or on disk, is called the virtual memory set of the process. The pages currently stored in memory are called the physical working set of the process, often called working set. The status of each page is kept in the process's working set list.
OpenVMS keeps track of free pages available to processes in a free page list. It also keeps a modified page list, which tracks pages that have been changed by a process and must be written to disk before being made available on the free page list.
Sometimes a process needs access to a page not currently in its physical working set. When this happens, OpenVMS searches two places for that page. First, OpenVMS looks in the modified and free page lists, because the page might still be available in physical memory. If the page does not appear on either of these lists, OpenVMS then retrieves the page from disk.
When OpenVMS must retrieve a page, the process has incurred a page fault. When the new page is in the modified or free page list, the page fault is called a soft page fault. When the page is only available on disk, the page fault is called a hard page fault. While OpenVMS retrieves the page from disk, the process becomes inactive. It remains in a page fault wait state until OpenVMS has successfully retrieved the necessary page. The length of time that this procedure requires varies according to the size of the disk, its rotation speed, and the load on the system.
When OpenVMS retrieves the new page, it places one of the current working set pages in the modified or free page list. OpenVMS chooses a page that the process has not used recently.
You can minimize soft page faults by setting the OpenVMS process parameters WSDEFAULT, WSQUOTA, and WSEXTENT appropriately for each process.
Note:
The Caché SETQUOTA utility lets you modify the default values of quota parameters for jobbed processes, including the network daemons and job servers. SETQUOTA can be set to run automatically during multiuser startup, or interactively after startup.
OpenVMS Physical Memory Allocation
In OpenVMS, you use the SYSGEN utility to set system limits on process memory use. You can then set process-by-process (user-by-user) limits with the AUTHORIZE utility.
The table below describes the OpenVMS system parameters that affect the memory available to a process's working set.
OpenVMS Working Set Parameters
Parameter Definition
WSDEFAULT  Initial size of the working set in 512-byte pagelets. 
WSQUOTA  Maximum guaranteed amount of physical memory, in pagelets, that OpenVMS allocates to a given process. WSQUOTA is limited by the value of the SYSGEN parameter WSMAX. If WSQUOTA is larger than WSMAX, the user only has access to the amount of memory in WSMAX. 
WSEXTENT  Indicates the maximum amount of physical memory which the process can own. The amount WSEXTENT less WSQUOTA is called “borrowed” memory, which is only allocated to the process if the system's current free list size exceeds the amount specified in the parameter BORROWLIM. 
WSINC  The number of pagelets by which the working set size of a process is increased when it needs to grow. 
WSDEC  The number of pagelets by which the working set size of a process is decreased when it needs to be reduced in size. 
BORROWLIM  Minimum number of pagelets that must be in the free page list before a process can borrow pages in WSINC amounts up to WSEXTENT pages. 
PFRATH (Page Fault Rate High)  Indicates the number of page faults that must occur within the period specified by AWSTIME, before OpenVMS makes an “automatic working set adjustment” to increase the working set by WSINC. 
PFRATL (Page Fault Rate Low)  If the number of page faults is less than this value, OpenVMS makes an “automatic working set adjustment” to decrease the working set by WSDEC, but not below the point set by AWSMIN. 
Note:
While parameters have units of pagelets, OpenVMS allocates memory in pages. For this reason, the parameter values you choose should be even multiples of the number of pagelets per page. OpenVMS automatically rounds these parameters if you do not follow this guideline.
Default Memory Allocation
The following steps summarize the algorithm OpenVMS uses to determine the amount of physical memory available to each process. Each capitalized term represents a system parameter that OpenVMS uses in allocating physical memory.
  1. The process begins executing an image. OpenVMS allocates to it the number of pagelets specified by the quota parameter WSDEFAULT.
    Page Allocation:
      0--->WSDEFAULT
  2. If the process has a higher page fault rate than PFRATH, OpenVMS allocates more pagelets to it. OpenVMS allocates these pagelets in increments of WSINC until the process has WSQUOTA total pagelets.
    Page Allocation:
      0--->WSDEFAULT-->WSINC-->WSINC-->WSQUOTA
  3. If the process continues to have a high page fault rate and the free list size exceeds BORROWLIM, OpenVMS will continue to allocate pagelets in WSINC blocks until the free list size is inadequate or WSEXTENT is reached.
    Page Allocation:
      0-->WSDEFAULT-->WSINC-->WSQUOTA-->WSINC-->WSEXTENT
  4. If the process's page fault rate drops below PFRATL, OpenVMS removes pages from the working set in increments of WSDEC until the page fault rate exceeds PFRATL.
  5. When the process exits the image, it loses these additional pages. For example, consider a monthly batch payroll job. When you run it in July, it begins execution at WSDEFAULT, and gains pages until it reaches WSEXTENT. When you run it in August, it again begins execution at the default value of WSDEFAULT.
See your OpenVMS documentation for more on the OpenVMS page allocation scheme.
Process Control of Memory Allocation
OpenVMS provides several facilities to alter working set parameters on a process-by-process or user-by-user basis. The table below summarizes these methods:
Working Set Control
Type of Process Method of Working Set Control
Users  The AUTHORIZE utility lets you change the default working set on a user-by-user basis. 
Interactive Processes  The DCL command SET WORKING_SET allows interactive processes to change working set parameters. 
Batch Processes  The SUBMIT command qualifiers for working sets allow batch processes to alter physical memory allocations. 
Batch Queues  You can use the DCL command INITIALIZE/QUEUE to alter working set parameters for batch queues. 
Keeping Memory Free for New Processes
Memory is finite. Every time a new process begins under OpenVMS, it takes WSDEFAULT pages away from the total number of available pages. The remaining memory is available for the free page list and for WSINC additions to each process's allocation.
Heavily-loaded systems can run so many concurrent processes that the number of pages remaining for use in the free page list becomes very small. OpenVMS includes a parameter called FREELIM that sets a lower limit for the number of pages in the free page list.
The size of the free and modified page lists directly affects the ratio of hard page faults to soft page faults. Hard page faults cause a process to experience a resource wait state, resulting in slow execution. For better performance, it is important to tune the system to minimize hard page faults. While soft faults are less expensive, an excessive soft page fault rate can also lead to poor performance.
Whenever OpenVMS detects that the size of the free page list falls below FREELIM, OpenVMS starts a mechanism to bring the free list size up to FREEGOAL. To achieve this, if all other mechanisms are inadequate, OpenVMS may move the entire working set of an inactive process out to disk. This procedure is called swapping. The process placed on disk is swapped out. When the process is brought back into memory, it is swapped in. Swapping has a detrimental effect on overall system performance.
Allocation for Shared Sections
OpenVMS uses a slightly different approach for allocating memory for shared code and data. The GBLSECTIONS parameter sets the number of shared sections to be allocated when the system is started. The GBLPAGES parameter sets the number of global page table entries. Every group of 128 page entries requires 4 bytes of resident memory. In addition, OpenVMS uses the GBLPAGFIL parameter to set the maximum number of page file blocks available for global pages.
How Caché Uses OpenVMS Memory
Caché uses both shared memory and memory private to each process when running on OpenVMS.
Balance Memory Locking and Paging
All Caché code is shared, and can be physically locked in memory. The same is true for the global buffer pool.
Routine buffers work similarly, though with a slight difference: routine buffers are shared, but you can specify how much of each routine buffer is locked into memory. If a routine is larger than the locked portion of a routine buffer, only a portion of the routine in the routine buffer is locked in memory. The remainder of the routine is loaded into unlocked physical memory.
Locking shared data and routines in memory allows better response time since memory access is quicker than disk access. As a result, the more globals and routines that are kept in memory, the better. Memory, however, is a finite resource. The more global and routine buffers that are allocated, the less memory is available for OpenVMS processes. When less memory is available, more OpenVMS paging occurs.
The goal, therefore, is to choose a number of global and routine buffers that lets you keep enough globals and routines in memory without negatively affecting OpenVMS paging. This document provides guidelines for selecting these values.
Use Non-Paged Dynamic Memory to Reduce Soft Page Faults
In OpenVMS, Caché allows you to locate the global buffers in a type of system memory called non-paged dynamic memory.
The advantage of placing global buffers in non-paged dynamic memory is that this type of memory is indexed via the OpenVMS working set list, not individual process's working set lists. This method reduces soft page faulting related to accessing global buffers. InterSystems recommends that you use non-paged dynamic memory for your global buffers.
Use Process-Private Space to Reduce Paging
Every Caché process maps to the shared memory sections, but it also has access to a private area of memory called process-private space. This private area includes variables, arrays, stacks, and other data structures that belong to a particular process.
A portion of this private area of memory may be locked into the process's working set to reduce paging. None of the private area, however, is physically locked in memory.
Calculating OpenVMS Parameters
The accompanying OpenVMS Parameter Calculator offers good starting values for both Caché and OpenVMS parameters. Later, if your system is not functioning as well as you would like, you can adjust these parameters to achieve optimal performance. The following sections describe the process of determining OpenVMS parameters:
You may want to print out the tables below so that you can record the relevant values.
Determine Parameter Calculator Input Values
This section helps you find appropriate values for the fields that control the number of processes and the number of global and routine buffers used by Caché. You need to determine these values, so that you can use the Parameter Calculator to compute both Caché and OpenVMS parameters accurately.
During installation you can set these and other Caché parameters, or you can retain default values. If you find later that your system needs tuning, you can adjust the values at any time using the Caché Configuration Manager. You need to restart Caché for parameter changes to take effect.
As you determine the values for these fields, record them in the Input to OpenVMS Parameter Calculator Table.
Determine Number of Processes
Before you can determine how many global and routine buffers Caché should allocate at startup, you must first determine the maximum number of Caché processes that will be running on your system at one time.
Every user in Caché constitutes one Caché process. Every Caché Job command creates a process. You should consider any process that appears on the %SS system status display, except the system processes Garbage Collector and Write daemon. As a rule of thumb, use the number of processes called for in your Caché license for this value; it cannot exceed your license limit.
Enter the value in the Number of Processes field in the parameter calculator.
Determine Number of Routine Buffers
Each routine buffer holds one and only one routine at a time. An OpenVMS Alpha system can have up to 65,535 routine buffers.
Many processes can share a routine buffer. Thus, when a process switches from one Caché routine to another, it merely accesses the other routine in another buffer. When no process is currently accessing a buffer, the buffer is returned to the pool of free buffers, and a new routine can be loaded into the empty buffer as it is required by another Caché process. If all buffers are occupied when Caché needs to load a routine, it chooses the least recently used (LRU) buffer, not the least frequently used (LFU) buffer. Allowing more buffers enhances performance.
The nature of the application influences the appropriate number of buffers. If many users access a small number of routines, a relatively small number of buffers will suffice for optimum response time. Conversely, an application with a large number of routines will benefit from a large number of buffers.
The parameter calculator starts with a default value for the number of routine buffers. This default is based upon a medium-sized site. You can enter a different value for this parameter in the Routine Buffer Pool field in the parameter calculator based on the specifics of your site.
Note:
If you prefer to enter a value in this field with an MB unit of measure, this value is used for the Routine Buffers (MB) field in the Calculation Result section and also in calculations involving routine buffers; it is not recalculated.
Determine Number of Global Buffers per Job
The parameter calculator assumes the global buffers are 8 KB in size. The calculator starts with recommended “rule of thumb” values for clustered and non-clustered systems.
If your system is memory-poor, you can reduce this value. Do not, however, decrease it below a multiple of 4 global buffers per process.
Using more global buffers will help the performance of most sites. You can use the statistics produced by the GLOSTAT utility to determine if adding more global buffers will reduce disk access and thereby improve performance.
Record Current OpenVMS Parameter Values
Every system has memory needs other than those for Caché. For example, there are memory requirements for FORTRAN and COBOL layered products as well as for OpenVMS itself. To analyze these requirements for your system, follow the procedures below to display the values of the indicated parameters prior to installing Caché.
Note:
If you are performing an upgrade, shut down Caché to get accurate numbers.
  1. Run the OpenVMS SYSGEN utility to display the current value of each parameter listed in the OpenVMS System Parameter Values Table.
    $ SET DEF SYS$SYSTEM
    $ RUN SYSGEN
    SYSGEN> USE CURRENT
    SYSGEN> SHOW parameter-name 
    
  2. Run the OpenVMS SYSGEN utility to display the current value of the process parameters listed in the OpenVMS Process Parameter Values Table below:
    $ SET DEF SYS$SYSTEM
    $ RUN SYSGEN
    SYSGEN> USE CURRENT
    SYSGEN> SHOW/PQL
    
    Alternatively, you may run the OpenVMS AUTHORIZE utility to display the current value of the user authorization file (UAF) parameters for each user account that uses Caché:
    $ SET DEF SYS$SYSTEM
    $ RUN AUTHORIZE
    UAF> show user-name
    
    You must then set the UAF records for all of the user accounts individually once the calculator returns the minimum recommendations for these values.
Use the Caché OpenVMS Parameter Calculator
This section describes how to use the Caché OpenVMS Parameter Calculator. This tool is an interactive HTML document that provides up-to-date calculations for the OpenVMS parameters required by Caché. The default values provide reasonable numbers for a medium sized installation. InterSystems experts are available to provide additional help configuring and tuning Caché for your site.
Perform the following steps to retrieve parameter values:
  1. Choose whether you are calculating parameters for a clustered system. If you select Yes, you receive both the cluster master values and values for any additional nodes; otherwise, the tool calculates values for a single node.
  2. As input to the OpenVMS Parameter Calculator use the values you determined in the previous section for the following fields:
    Input to OpenVMS Parameter Calculator
    Parameter User-supplied value
    Number of Processes   
    Routine Buffer Pool   
    Global Buffers per Job   
  3. Access the Caché OpenVMS Parameter Calculator to calculate the necessary parameter values.
  4. Fill in the values returned by the calculator in the appropriate tables below.
Analyze the Calculation Results
The results from the calculator are suggested values; treat them as a guideline. You may need to adjust settings in order to balance and maximize system performance, depending on your site's hardware and software configuration.
Update Caché Parameters
The following values are for Caché parameters that you can update during the installation or by using the Configuration Manager from a Windows client after you install Caché.
Output Caché Parameters
Parameter Calculated value
Number of Global Buffers   
Global Buffers (MB)   
Routine Buffers (MB)   
Update OpenVMS System Parameters
The system parameter values computed by the OpenVMS Parameter Calculator are the amounts you need to add to your current OpenVMS system in order to make it ready to run Caché. Caché will consume this much memory, so evaluate your system to make sure that there is the proper amount of memory available.
Use this table to help you combine the required amounts returned by the calculator with your current system values. Update these parameters by using the SET command of the OpenVMS SYSGEN utility.
OpenVMS System Parameter Values
Parameter Current value Required Caché amount Resulting value
BALSETCNT       
MAXPROCESSCNT       
GBLPAGES       
GBLPAGFIL       
GBLSECTIONS       
LOCKIDTBL       
RESHASHTBL       
SYSMWCNT       
Update Special OpenVMS Parameters
The parameter calculator provides a few special parameters used to configure Caché on your OpenVMS system:
Verify the values returned by the calculator match those of your OpenVMS system.
Special OpenVMS Parameter Values
Parameter Returned value
Resident Memory (MB)   
PRIORITY_OFFSET   
PE1   
Update OpenVMS Process Parameters
InterSystems recommends minimum values for some PQL process quota parameters. This is because Caché maintains a table, GJOBQ, which holds default values for many of these parameters. You must have authorized these values in order for Caché processes to be able to use the values in the GJOBQ table. You do not need to calculate values for these parameters. The minimum suggested values appear in the OpenVMS Parameter Calculator. Enter these values in the table below.
If your current value for any of these parameters is less than the recommended minimum value, you should adjust the values during installation using the SET command of the OpenVMS SYSGEN utility or the OpenVMS AUTHORIZE utility for each user account.
OpenVMS Process Parameter Values
Parameter Current value Recommended minimum value
ASTLM     
BIOLM     
BYTLM     
DIOLM     
FILLM     
PGFLQUOTA     
TQELM     
WSQUOTA     
WSEXTENT     
ENQLM     
Resident Memory Section
The Alpha OpenVMS platform makes use of the Resident Memory Section facility. All OpenVMS Caché users are encouraged to make use of this facility.
Alpha systems running OpenVMS v7.1 and later have a memory mechanism for allocating global sections (memory which can be shared between processes). The two features that Caché supports, memory resident global sections and shared page tables, are always used as a pair.
The advantages of using a memory-resident global section mapped via a shared page table for the global buffer pool are:
The drawback to using these structures is that if you want to increase the size of your global buffer pool, you may have to restart your OpenVMS system in order to reconfigure the amount of space reserved for the memory-resident global section. This space can be reserved when the system starts (via SYSMAN and AUTOGEN) or it can be allocated dynamically. If it is not reserved then there may be insufficient space available when the request is made and the request will fail. If Caché is asked to use a memory-resident global section and it cannot, it will allocate the global buffer pool out of a non?memory-resident global section.
Create a Named Resident Memory Section
To use these structures you need to do the following:
  1. Use the value returned by the OpenVMS Parameter Calculator for Resident Memory as the reserved memory size.
  2. Use SYSMAN to reserve a section of memory. You must supply a name for the named section. You can use any name that consists of alphanumeric characters (and the underscore) and is not longer than 43 characters. The syntax for SYSMAN is:
    MCR SYSMAN ! run sysman
    SYSMAN> RESERVED_MEMORY ADD "Resident_Memory_Name" - 
           /ALLOCATE/PAGE_TABLES/ZERO/SIZE=<size in MB>
    SYSMAN> EXIT
  3. Run AUTOGEN to process the Reserved Memory Registry data file which contains the information to establish your memory-resident global section. After the system is restarted the SYSMAN command displays the shared memory section you reserved:
    MCR SYSMAN ! run sysman
    SYSMAN> RESERVED_MEMORY SHOW
  4. Modify the Caché configuration files (config.def, cache.cpf, etc.) to use the reserved section. This is done by setting the reserved memory parameter in the [config] section so that it reads:
    useresidentmem=1[,<resident memory section name>]
    The <resident memory section name> is the name of the section you specified in SYSMAN when you reserved it (“Resident_Memory_Name” in the example above). If you name this section, it is required in the configuration file.
    Alternately, you can also modify the configuration files by using the Caché Configuration Manager on a Windows client:
    1. On the General tab, verify you have the appropriate OpenVMS configuration file open. If not, on the Advanced tab, click Files, select the correct configuration file, click Open, and then click OK.
    2. On the Advanced tab, expand the Memory properties.
    3. Update the Resident Memory property to Yes.
    4. Enter the name you reserved in the SYSMAN utility (“Resident_Memory_Name” in the example above) in the Reserved Memory Section property and click OK.
  5. Restart Caché. If it cannot use the reserved memory section, Caché displays a message and stores an error code in the SYSLOG.
If you use resident memory to map your global buffer pool, you may be able to reduce some of the system parameters, in particular, WSMAX. Typically Alpha OpenVMS comes with a WSMAX default value much larger than is required for running Caché.