VAX Professional, 1987 – A Debug Register Display Fix


A DEBUG Register Display Fix


Hunter Goatley

Western Kentucky University

This was my very first article published in Vax Professional magazine in early 1987. I wrote it when I was a senior at Western Kentucky University, and I was amazed that they agreed to publish an article I wrote.

As ancient as this is, I was encouraged by others to post all of my early articles for “historical” purposes, so here it is. The concepts described below are still applicable to the debugger in current versions of VMS.

With VMS v4.4, Digital once again changed the symbolic debugger for VMS. Most of the changes were enhancements brought on with the addition of the screen management routines (SMG$ routines). The register display is probably the most significant visual change. Under VMS v4.2, the register display occupied 4 lines and 80 columns, as shown in Figure 1. The new format (Figure 2) covers almost 1/4 of the screen, taking 10 lines and 40 columns. The problem with this format is that the registers will cover part of another display; this means that slow “software scrolling,” rather than fast”hardware scrolling,” must be used to update the screen.Because our terminals are Z29s (Zenith VT100-compatible) that are capable of doing a hardware scroll, I decided to see what I could do to solve the problem of software scrolls in the covered display.

To see the changes in the display formats, a program should first be assembled and linked with the debugger. Take your favorite MACRO program and execute the following DCL commands:


When the program is run, the debugger will take control and you will be prompted with “DBG> “, the debugger prompt. At this prompt, type SET MODE SCREEN. If you are on a video terminal, the screen will clear and three displays will be set up for you—INST, OUT, and prompt. The INST window should contain your program’s disassembled instructions. The OUT window will be used for all DEBUG output—lines stating the line stepped to, things EXAMINEd, etc.; the PROMPT for DEBUG commands, errors, and program output. Now use the DEBUG command STEP to single-step through a few lines of code. Notice how the display is scrolled up by hardware (with a software scroll, each line must be repainted in its new position by the software; with hardware scroll, the terminal scrolls the line up). Now type DISPLAY REG—DEBUG will place the register display in the upper right quarter of the screen, occluding half of the INST display. Now single-step through a few more instructions; notice how the INST display is repainted for each line to be scrolled. Since the display is now only 40 columns wide, a software scroll must be used because a hardware scroll would scroll all 80 columns, causing the register display to scroll, too. A software scroll is an ugly sight at any baud rate, but is even uglier at 2400 or slower.

The new capabilities of the debugger include the ability to expand or contract display windows. The physical constraints of the displays can be set with the SET DISPLAY debugger command. To create a register display positioned at line 5, column 1 and extending to line 16, column 40, the command would look like:


DEBUG position format is (starting-line, num-of-lines, starting-col, num-of-cols). NODYNAMIC specifies that new dimensions should not be calculated for the display if one changed his screen width from 80 to 132. The SIZE is the maximum number of lines the display can occupy. The display MYREG is made a REGISTER display.

I read the debugger manual looking for a way to change the register format. It seemed to me that Digital might give you a choice as to which format you want for the register display. Alas, I could find no such choice. It appeared that the only way that I could change the register display was to create several new register displays, shrinking the windows so that I only saw those registers I wanted to see in each window. The command to expand or contract a display is EXPAND, whose qualifiers determine how the display is changed. The four qualifiers for EXPAND are /UP,/DOWN,/LEFT,/RIGHT. To shrink the display defined above to a 3-line by 8-column window starting at line 7, column 24, the command would look like:


The negative values tell the debugger that you want to contract the display; to expand it, positive values would be supplied.

The DEBUG command MOVE allows you to move a display from one position to a new position relative from the first. To move MYREG up 5 lines and to the right 10 columns, the MOVE command would be


Armed with these commands, I set off to build the register display the way I wanted it. After studying the default register display, I decided that I would be able to create 6 different register displays, shrinking each one to only show 5 registers. By doing so, all of the registers and stack positions could be shown with 5 rows, 80 columns. Unfortunately, the condition codes would be left out in the cold. I decided that I would need two more of the displays, each showing only 2 of the condition codes. The problem: where were they going to go? I couldn’t do much about the way they were displayed—on a line along the bottom of the default display (unlike the 3-column format under v4.2). My solution was to overlay REG6 (which displays memory locations @SP+20 – @SP+36) with the two condition code displays. By issuing the command


that display was pushed beneath the condition codes; the memory locations can be viewed by typing DISPLAY/POP REG6.

Despite the amount of work this DEBUG command file must perform, the screen is displayed after about a second or so on our system—a time comparable to the delay when simply issuing SET MODE SCREEN; DISPLAY REG. It appears to me to take a little longer to single-step my way through a program, because the screen management routines must update 8 register displays instead of one; having the registers displayed in a format I like and having hardware scroll for my other three windows (INST, OUT, and PROMPT) makes it worth it to me.

R0:00000001 R4:00000000 R8: 7FFED052 AP:7FF5EFCC  @AP:00000006 @SP:00000000  N:0
R1:00000008 R5:00000000 R9: 7FFED25A FP:7FF5EF80   +4:7FFE6440  +4:08000000  Z:0
R2:00000000 R6:7FF5EC49 R10:7FFEDDD4 SP:7FF5EF80   +8:7FFA8ED3  +8:7FF5EFCC  V:0
R3:7FF5EF94 R7:8001E4DD R11:000004A0 PC:00000668  +12:7FFE640C +12:7FF5EFB8  C:0
                   Figure 1 (VMS v4.2 DEBUG register display)
- REG ---------------------------------
R0:00000000  R10:7FFEDDD4  @SP:00000000
R1:00000000  R11:7FFE33DC   +4:00000000
R2:00000000  AP :7FF43FCC   +8:7FF43FCC
R3:7FF43F94  FP :7FF43F84  +12:7FF43FB8
R4:00000000  SP :7FF43F84  +16:000016A7
R5:00000000  PC :00000CDF  +20:000013FF
R6:7FF43C49  @AP:00000006  +24:00000005
R7:8001E4DD   +4:7FFE6440  +28:00000CDD
R8:7FFED052   +8:7FF95BB7  +32:00000000
R9:7FFED25A  +12:7FFE640C  +36:00000001
N:0    Z:0     V:0    C:0  +40:0000000D

Figure 2  (VMS v4.4 DEBUG register display)
- REG1 -----+- REG2 -----+- REG3 ------+- REG4 ------+- REG5 ------+- CC1 ------
R0:00000000 |R5:00000000 |R10:7FFEDDD4 |PC :00000CDF |@SP:00000000 |N:0    Z:0
R1:00000000 |R6:7FF43C49 |R11:7FFE33DC |@AP:00000006 | +4:00000000 |
R2:00000000 |R7:8001E4DD |AP :7FF43FCC | +4:7FFE6440 | +8:7FF43FCC +- CC2 ------
R3:7FF43F94 |R8:7FFED052 |FP :7FF43F84 | +8:7FF95BB7 |+12:7FF43FB8 |V:0    C:0
R4:00000000 |R9:7FFED25A |SP :7FF43F84 |+12:7FFE640C |+16:000016A7 |

              Figure 3  (Display created by DBGINI.DBG)


The DEBUG command file

!  Hunter Goatley       July 9, 1986
!  DEBUG command procedure to set up the screen so that registers
!       are displayed in a "better" format than the default DEBUG
!       v4.4 format.
set mode screen
set display/nodynamic/size=15 REG1 at lh1 register
set display/nodynamic/size=15 REG2 at (1,11,14,40) register
set display/nodynamic/size=15 REG3 at (1,11,14,40) register
set display/nodynamic/size=15 REG4 at (1,11,28,40) register
set display/nodynamic/size=15 REG5 at (1,11,28,40) register
set display/nodynamic/size=15 REG6 at (1,11,42,40) register
set display/nodynamic/size=15 CC1  at (1,12,69,40) register
set display/nodynamic/size=15 CC2  at (1,12,54,40) register
define/key MINUS "Display/Pop REG6"/terminate/nolog
define/key KP9   "Display/Push REG6"/terminate/nolog
! registers 0-4
display REG1
expand/down:-6/right:-28 REG1
! registers 5-9
display REG2
expand/up:-5/down:-1/right:-28 REG2
move/up:5 REG2
! registers 10,11,AP,FP,SP
display REG3
expand/down:-6/left:-13/right:-14 REG3
! PC & @AP - @AP+12
display REG4
expand/up:-5/down:-1/left:-13/right:-14 REG4
move/up:5 REG4
! @SP - @SP+16
display REG5
expand/left:-27/down:-6 REG5
! @SP+20 - @SP+36
expand/up:-5/down:-1/left:-27 REG6
move/up:5 REG6
! Condition codes N & Z
display CC1
expand/up:-10/down:1/right:-28 CC1
move/up:10 CC1
! Condition codes V & C
display CC2
expand/up:-10/down:1/left:-15/right:-12 CC2
move/up:7 CC2
display/push REG6       !  Keep @SP+24 - @SP+40 hidden for now
display inst at (7,7,1,80)
display out  at (14,6,1,80)


 Posted by at 5:23 am