Read virtual AWS files on a PDP-11 in Basic-Plus summary

Like a lot of dinosaurs I like to relive my computer past. My first job which I sought was on a DEC PDP-11/70 running the RSTS/E OS. We received alot of IBM customer tapes which we would read/process and add to our database. So since I already had many AWS tapes from my Hercules MVS testing, I decided this would be a fun project to sink my teeth into. I had created these AWS tapes using Jay Moseley’s maketape. I gave it a filename of CUST-TAPE-1000.aws, so I knew there were 1000 records in this virtual tape by the way I named it…a decent amount for a test. I also had a “tape” with 1 million records…not necessary for a test. However I forgot what was the content of this tape. Once again I turned to Jay Moseley’s tapedump to get the info. From this I got…

$ ./tapedump INPUT: CUST-TAPE-1000.aws
TAPEDUMP v1.0 (BASIC) copyright Jay Moseley, CCP 2000
Processing AWSTAPE file: CUST-TAPE-1000.aws
Serial Number from VOLume 1 label: VOL001
File MVT Dataset RECFM BLKSIZE LRECL BlocksExpected BlocksRead
0001 CUSTOMER.FILE B    16,400   164             10         10

You can also add a switch to let you look at the actual records. With the detail you can easily see where fields begin and end.

OK I have the data. The next thing I tackled was converting EBCDIC to ASCII which is the PDP-11s native format. These routines were already established when I was hired. So I think I came up a fairly good way to do this in Basic-Plus.

Next I had to see if I could even process AWS tapes on the PDP-11. But before even that, I had to define some tape drives to RSTS/E. Which meant a SYSGEN…which I didn’t want to do. I was perfectly happy up to this point using the pre-built RSTS V7.0-07, for my RSTS/E Basic-Plus enjoyment.

I found very little on using tapes in general let alone AWS tapes. I finally saw a comment, here, that led me to believe simh (the PDP emulator) could work with AWS tapes…but no examples. The AWS format wasn’t even listed as an option in the simh book (PDF).

Here are a few records from the the EBCDIC AWS input tape using the tapedump program

Once you can read the tape data, using Basic-Plus…you read a block with Basic-Plus’s GET statement, and using FIELD statements I could break these fixed length blocked records apart. Because IBM doesn’t use a new line character such as CR or LF to separate records.

Here is some output (which I showed in a earlier post) from the Basic-Plus program. I defined Account #, Last and First Name, and wrote them to a text file.

Below is a few records out of 1000.

PIP CUSTRC.TXT
4313823732834469            Petgrave                    Fabian         
4313768567626482            Schouten                    Robert         
4313251303989456            Keling                      Rodger         
4313825573004227            Cypher                      Raul           
4313181873894986            Massingale                  Juliana        
4313087137943172            Caamal                      Cary           
4313196239656267            Diefendorf                  Leonor         
4313416983633720            Woodend                     Otis           
4313273914282238            Smith                       Weston         
4313843895349824            Kaleo                       Raymond        
4313935972736152            Medine                      Karyn          
4313160683191320            Stose                       Richard        
4313183036785135            Zoeller                     Ada            
4313776948680839            Mariacher                   Jon            
4313621226845482            Shaak                       Donny          

After a lot of trial and error (I started this adventure 5/23/2023 and came to a good ending on 5/31/2023, so 8 days) I was able to re-create what we did…back in the day. A lot of trial and error mostly because of the quirky way simh would sometimes behave. For example the SYSGEN (which is rather lengthy) failed twice in the same place, because the simulator wouldn’t allow me to drop out of the SYSGEN…which was necessary at that point. I had already done this a few times previously in the SYSGEN. I’d get Protection violations one day and not the next. This was the newest version of Open simh. Finally I tried the newest version of classic simh… and was able to complete the SYSGEN.

Pretty cool blending two old technologies, I was involved with. I actually originally created the fake customer records using Python. Then wrote them to an AWS tape using Jay Moseley’s maketape. I also read and sorted them in TK4- (MVS) so I know they’re a valid representation of MVS Customer records.

When we outgrew the PDP-11/70 we moved to VSE/VM on IBM hardware, so I’m very familiar (at least I was at one time) with that side also.

In hind-site it has been a very satisfying exercise. I’m glad I had already worked with (and saved some data) AWS tapes in Hercules/MVS. It saved me a lot of time.

RSTS/E Protection Violation…Final

After the earlier success I decided there is no reason why I need to set the privileged bit to 192 because I’m running from 1,2 so I reran it after a compile without setting the privileged bit to 192. And…it worked. Like you would expect it to. Setting the privileged bit to 192 would allow a non privileged account to run it by running $TAPE05.

So evidently RSTS/E was messing with me the last few days. When I said “I don’t know what to do” 2 days ago, it was because nothing was wrong.

Simulation stopped, PC: 041410 (SOB R1,41406)
sim> att tu0 -r -f aws CUST-TAPE-1000.aws
%SIM-INFO: TU0: unit is read only
%SIM-INFO: TU0: Tape Image 'CUST-TAPE-1000.aws' scanned as AWS format
sim> c

OLD TAPE05

Ready

COMPILE

Ready

DIR TAPE05.*
 Name .Ext    Size   Prot    Date       SY:[1,2]
TAPE05.BAS       6   < 60> 01-Jan-99
TAPE05.BAC      15C  <124> 01-Jan-99

Total of 21 blocks in 2 files in SY:[1,2]


Ready

ASSIGN MM0:

Ready

LIST
NONAME  12:03 AM        01-Jan-99

Ready

RUN TAPE05
 430           11 
 1020          11 

Ready

DIR CUSTRC.*
 Name .Ext    Size   Prot    Date       SY:[1,2]
CUSTRC.TXT     143   < 60> 01-Jan-99

Total of 143 blocks in 1 file in SY:[1,2]


Ready

RSTS/E Protection Violation…Pt 2

Well I don’t know what is different, but…no problems today. I was going to play with the MODE paremeter on the OPEN statement. And no difference in time…2.4 sec either way.

RUN TAPE05
 430           11 
 1020          11 

Ready

LIST
TAPE05  12:04 AM        01-Jan-99
?Execute only file

Ready

Error 11 on lines 430 and 1020 are acceptable end of file errors, on a read, that I trap for

Yesterday was getting the Protection Violation on an OPEN.

It’s a different program…but the OPEN is the same.

RSTS/E Protection Violation

Decided to see if compiling my Tape program makes it run any faster. Because it runs a little slow. I assume mostly because of converting each record from EBCDIC to ASCII. There’s only 1000 records. Anyway after compiling it, I get error 10 (on the OPEN statement)…?Protection violation. I need to research because off the top of my head I don’t remember how to change it or what change it to. PIP maybe?

I got the following info from the RSTS/E Programming Manual.

A privileged job can set the privileged bit in the protection code. The privileged bit, with the compiled flle pro-tection bit, designates a program as privileged .. Thus, any program with a protection code of < 192> (the sum of the privileged protection < 128> and the compiled file. protection <64> ) or greater denotes a privileged program. Both protection code values must be set for a program to have privilege.

Wait…I found this…

RSTS/E accepted the following statement. However I still got a Protection Violation when I try and run it.

NAME 'TAPE04.BAC' AS 'TAPE04.BAC<192>'

Ready
DIR TAPE04.*
 Name .Ext    Size   Prot    Date       SY:[1,2]
TAPE04.BAS       5   < 60> 01-Jan-99
TAPE04.BAC      15C  <192> 01-Jan-99

Total of 20 blocks in 2 files in SY:[1,2]

Ready

Quite honestly I’m running this in a 1, account (1,2 to be specific). So it seems strange that I have to set any protection code. The source program runs with no problem. What is so special about a compiled program?

So at this moment I don’t know what to do. Especially when you don’t find out too much when Googling an ancient OS. I already consulted the manuals and did exactly what they said.

Writing RSTS/E tapes

Currently I only have 1 valid MM RSTS/E tape drive…MM0: so although the 2nd attach below worked, the ASSIGN from RSTS/E…didn’t.

So I’d like to attach like the Output below.

Input : att tu0 -r -f aws CUST-TAPE-1000.aws
Output: att tu1 TAPE-OUT.TAP
sim> att tu0 -r -f aws CUST-TAPE-1000.aws
%SIM-INFO: TU0: unit is read only
%SIM-INFO: TU0: Tape Image 'CUST-TAPE-1000.aws' scanned as AWS format
sim> att tu1 TAPE-OUT.TAP
%SIM-INFO: TU1: Tape Image 'TAPE-OUT.TAP' scanned as SIMH format
sim> 
sim> 
sim> show tu
TU      Massbus adapter 1 (RHB), FORMATTER, TM03, 8 units
  TU0   attached to CUST-TAPE-1000.aws, read only, write locked, TE16
        AWS format, unlimited capacity
  TU1   attached to TAPE-OUT.TAP, write enabled, TE16
        SIMH format, unlimited capacity
  TU2   not attached, write enabled, TE16
        SIMH format, unlimited capacity
  TU3   not attached, write enabled, TE16
        SIMH format, unlimited capacity
  TU4   not attached, write enabled, TE16
        SIMH format, unlimited capacity
  TU5   not attached, write enabled, TE16
        SIMH format, unlimited capacity
  TU6   not attached, write enabled, TE16
        SIMH format, unlimited capacity
  TU7   not attached, write enabled, TE16
        SIMH format, unlimited capacity
sim> c

ASSIGN MM0:

Ready

ASSIGN MM1:
?Not a valid device

Ready

RSTS/E only knows about MM0:. I guess I didn’t SYSGEN enough TU drives!

However I tested writes to THE ONE DRIVE…MM0:

I can write to a virtual tape (which is really a local disk drive) like so…

sim> att tu0 TAPE-OUT.TAP

ASSIGN MM0:

540 OPEN "MM0:" FOR OUTPUT AS FILE #1, MODE 64%+16%
570 PRINT #1, ""
600 PRINT #1, "LINE",I% FOR I%=1 TO 20
1050 CLOSE #1

Here’s the output on the Linux host…

$ dir *.TAP -la
-rw-rw-r-- 1 bill bill 532 Jun  1 15:56 TAPE-OUT.TAP
$ cat TAPE-OUT.TAP 

LINE           1 
LINE           2 
LINE           3 
LINE           4 
LINE           5 
LINE           6 
LINE           7 
LINE           8 
LINE           9 
LINE           10 
LINE           11 
LINE           12 
LINE           13 
LINE           14 
LINE           15 
LINE           16 
LINE           17 
LINE           18 
LINE           19 
$ 

DEC disk drives

RK03 and RK05 – are compatible and can interchange packs. Each pack stores about 2.4MB

RK05 – a capacity of 1MB?

RK06 – a capacity of 14MB

RL01 – a capacity of 5.2MB

RL02 – a capacity of10.4MB

RK06 – 13.8 MB
RK07 – 27.6 MB

RM05 – The RM05 has a formatted storage data capacity of 256 megabytes

RA81 – capacity of 456 Mbytes

RA90 – The 1988-released held 1.2GB

Basic-Plus tape output formatted

Read EBCDIC AWS (Hercules format) tape. Recordsize 164, Blocksize 16400.

Don’t output every field. Only output selected fields (Account Number, Last Name, First Name).
We might output this way in order to sort by account # or Last name, First name.

I think the program is a decent but far from perfect representation of what we did…back in the day to process IBM customer tapes using Basic-Plus on a PDP-11/70. There are probably many improvements I would make if I didn’t have to restart RSTS/E between every program change/test. It was vary tedious. As I said my problems may entirely be my lack of simh knowledge.

It was a long road to get here. Starting with writing the EBCDIC conversion routine here. Then, I had to SYSGEN a new RSTS/E system to even add support for Tape drives.

I found very little detail on processing tape files on RSTS/E running on the simh emulator. And as far as Hercules AWS files all I found was a statement that simh could handle them. And the AWS format wasn’t even documented. I just guessed that it was an option based on what I read somewhere. So there was a lot of trial and error.

Below is a few records out of 1000.

PIP CUSTRC.TXT
4313823732834469            Petgrave                    Fabian         
4313768567626482            Schouten                    Robert         
4313251303989456            Keling                      Rodger         
4313825573004227            Cypher                      Raul           
4313181873894986            Massingale                  Juliana        
4313087137943172            Caamal                      Cary           
4313196239656267            Diefendorf                  Leonor         
4313416983633720            Woodend                     Otis           
4313273914282238            Smith                       Weston         
4313843895349824            Kaleo                       Raymond        
4313935972736152            Medine                      Karyn          
4313160683191320            Stose                       Richard        
4313183036785135            Zoeller                     Ada            
4313776948680839            Mariacher                   Jon            
4313621226845482            Shaak                       Donny          

Basic-Plus Tape output

Well I’m converting an AWS EBCDIC tape. But I’m not rewinding like I thought. Converting from EBCDIC to ASCII helped lot, to figure out what was happening. I also had the blocking factor wrong. I also wrote to a file…CUSTRC.TXT. Now I need to learn how to control the tape drive. I could open/read/close the file to bypass tape marks. But there must be a more elegant way. I don’t remember how we did it. I guess I should read the label anyway to make sure I have the right tape.

One of my biggest problems in this exercise is having to shutdown/reboot between every test. Otherwise unpredictable things might, and usually does happen. Perhaps it’s just my lack of knowledge of the correct simh commands. For example the afore mentioned “?Error text lookup failure” which led me to have to ungracefully restart RSTS/E.

Here are the 1st few records…

x
CUSTRC.TXT     352   < 60> 01-Jan-99

Total of 2932 blocks in 101 files in SY:[1,2]


Ready

PIP CUSTRC.TXT
 1            43138237328344695FMMr      Fabian         Colin          P
etgrave                         1999-08-012019-02-19028.701981 NW 514 Pl
      Glendale            CA91206 
 2            43137685676264822DMMr      Robert         Antonio        S
chouten                 DMD     1992-03-142013-02-14049.957458 NE 657 Rd
      Mesa                AZ85204 
 3            43132513039894562FM        Rodger         Drew           K
eling                           1985-06-282018-09-03032.144970 S 925 Ter

I can confirm they are correct. All the way to rec 1000…the last record!