Computer Log

While googling “Hercules AWS ” I see that Jay Mosley made some updates in July 2020 that I was unaware of. Changes seems mostly concerning using gnuCobol to provide a fullscreen interface…interesting.

These are timings for a large one million customer records, dataset. It follows the same process below. It just goes to show my reasoning why I like this method to move large datasets to the virtual mainframe.

Use maketape to create the virtual tape file!

bill@bill-MS-7B79:~/Mystuff/Mainframe/MVS/JCL/TAPEDASD$ time ./maketape INPUT: customer-million.txt VOLSER: VOL001 DATASET: CUSTOMER.FILE OUTPUT: TAPE01.AWS LRECL: 165 BLOCK: 100
MAKETAPE v1.1 (C) copyright Jay Moseley, CCP 2000
Processing input from: customer-million.txt
Wrote 10000 blocks to AWSTAPE file: TAPE01.AWS (Seq #0001 Dataset:CUSTOMER.FILE )

real 0m0.430s
user 0m0.343s
sys 0m0.088s
bill@bill-MS-7B79:~/Mystuff/Mainframe/MVS/JCL/TAPEDASD$

Converted one million records in under a second to AWS tape

Load the virtual tape file into a MVS dataset!

HHC01603I devinit 00c /home/bill/Mystuff/Mainframe/MVS/JCL/TAPEDASD/TAPEDASD.JCL
HHC01046I 0:000C COMM: device unbound from socket 3505
HHC00101I Thread id 7F7AF1F1B700, prio 0, name Socket device listener ended
HHC02245I 0:000C device initialized
21.13.01 JOB 771 $HASP100 TAPEDASD ON READER1 BILL
21.13.01 JOB 771 $HASP373 TAPEDASD STARTED – INIT 1 – CLASS A – SYS TK4-
21.13.01 JOB 771 IEF403I TAPEDASD – STARTED – TIME=21.13.01
21.13.01 JOB 771 IEF233A M 480,VOL001,,TAPEDASD,STEP01
HHC01603I devinit 480 /home/bill/Mystuff/Mainframe/MVS/JCL/TAPEDASD/TAPE01.AWS
HHC00221I 0:0480 Tape file /home/bill/Mystuff/Mainframe/MVS/JCL/TAPEDASD/TAPE01.AWS, type aws: format type AW
HHC02245I 0:0480 device initialized
HHC00201I 0:0480 Tape file /home/bill/Mystuff/Mainframe/MVS/JCL/TAPEDASD/TAPE01.AWS, type aws: tape closed
21.13.11 JOB 771 IEF234E K 480,VOL001,PVT,TAPEDASD,STEP01
21.13.11 JOB 771 IEF404I TAPEDASD – ENDED – TIME=21.13.11
21.13.11 JOB 771 $HASP395 TAPEDASD ENDED
21.13.11 $HASP309 INIT 1 INACTIVE * C=A
21.13.11 JOB 771 $HASP150 TAPEDASD ON PRINTER1 90 LINES
21.13.11 $HASP160 PRINTER1 INACTIVE – CLASS=A
21.13.11 JOB 771 $HASP250 TAPEDASD IS PURGED

One million records loaded to a MVS dataset in about 10 seconds (21:13:01 – 21:13:11)

Of course a custom program such as a Cobol program could have just processed the Virtual tape directly! That would actually probably be best, no need to load to MVS dataset…it’s already on disk, even though MVS thinks it’s a tape!

Previously, I talked about Hercules virtual tapes without being specific. I find this useful for getting large amounts of data to a MVS dataset. So here I find myself reviewing/documenting the procedures…again!

These below examples are using test data with a fixed length of 165 characters!

Getting the data

It seems I created some fixed length test records with various record counts in ~/Mystuff/COBOL/data/
Since I need to be able to view the files on Linux they have one extra byte at the end that the mainframe doesn’t need…the new line character. It would be hard to view a large file on Linux without this character as most editors are looking for it.
Mainframe files or datasets are best created as fixed length fields and lengths.
I use a Python program to create a lot of real looking test records. This program uses the mimesis package, so currently mimesis is installed in a virtual environment, and can be accessed as follows.
conda deactivate
cd ~/Mystuff/Python3/project1_env/
source bin/activate

Converting the data to a Virtual Tape file

Note: I moved Jay’s “C” programs (maketape and tapedump) to “/usr/local/bin” so I can run them anywhere!

Once the test or real data is created as a Linux ASCII file, it needs to be converted to a EBCDIC virtual tape file. I use Jay Mosley’s maketape program. See here to get maketape. Example below. The dollar sign is the Linux command line prompt.

Here’s how I’d use maketape to convert the linux file into a Hercules mainframe virtual tape dataset

$ maketape INPUT: customer.txt VOLSER: VOL001 DATASET: CUSTOMER.FILE OUTPUT: TAPE04.AWS LRECL: 165 BLOCK: 100
MAKETAPE v1.1 (C) copyright Jay Moseley, CCP 2000
Processing input from: customer.txt
Wrote 1001 blocks to AWSTAPE file: TAPE04.AWS (Seq #0001 Dataset:CUSTOMER.FILE )
$

Viewing Virtual tape info locally

tapedump is useful for virtual tapefiles that you may have forgot details about. It can provide useful information needed for your JCL, DD card. See the IEBGENER utility in the next section.

$ tapedump I: TAPE01.AWS
TAPEDUMP v1.0 (C) copyright Jay Moseley, CCP 2000
Processing AWSTAPE file: TAPE01.AWS
Serial Number from VOLume 1 label: VOL001

File MVT Dataset RECFM BLKSIZE LRECL BlocksExpected BlocksRead
0001 CUSTOMER.FILE F 16500 165 1001 1001
$

Apparently there exists a program called “hetmap” that gives more details than TAPEDUMP.

Processing the virtual tape on the mainframe

You can either use a custom program (for example COBOL or Assembler) on the mainframe. For example the select statements in the COBOL program in my case looks like this…

SELECT IN-FILE ASSIGN TO UT-S-INTAPE.
SELECT OUT-FILE ASSIGN TO DA-S-OUTDISK.

Or you can use the IBM utility IEBGENER to load this virtual tape file to disk. As far as these programs are concerned this file is a tape!

IEBGENER (upload) Jcl

//TAPEDASD JOB (12345678),BILL,CLASS=A,MSGCLASS=A,
// MSGLEVEL=(1,1),USER=HERC01,PASSWORD=CUL8TR
//STEP01 EXEC PGM=IEBGENER
//SYSPRINT DD SYSOUT=*
//SYSOUT DD SYSOUT=*
//SYSUT1 DD DSN=CUSTOMER.FILE,UNIT=TAPE,VOL=SER=VOL001,
// DISP=OLD,LABEL=(,SL)
//SYSUT2 DD DSN=CUST.MILLION.DATA,
// DISP=(NEW,CATLG,DELETE),
// SPACE=(CYL,(200,10),RLSE),
// UNIT=3350,
// VOL=SER=WRK142,
// DCB=(DSORG=PS,RECFM=FB,LRECL=165,BLKSIZE=16500)
//SYSIN DD *
GENERATE MAXFLDS=1
RECORD FIELD=(165)
/*

After submitting this Job it will wait for the tape
You should get a message from the console like so…

HHC01603I devinit 00c /home/bill/Mystuff/Mainframe/MVS/JCL/TAPEDASD/TAPEDASD.JCL
HHC01046I 0:000C COMM: device unbound from socket 3505
HHC00101I Thread id 7F65F81BC700, prio 0, name Socket device listener ended
HHC02245I 0:000C device initialized
16.31.23 JOB 769 $HASP100 TAPEDASD ON READER1 BILL
16.31.23 JOB 769 $HASP373 TAPEDASD STARTED – INIT 1 – CLASS A – SYS TK4-
16.31.23 JOB 769 IEF403I TAPEDASD – STARTED – TIME=16.31.23
16.31.23 JOB 769 *IEF233A M 480,VOL001,,TAPEDASD,STEP01
herc =====>

As you can see from the last message IEF233A, the job is looking for a tape on device 480

Which you supply by giving Hercules the following command
devinit 480 pathToVirtualTape/TAPE01.AWS

Some other useful Hercules tape commands

Check for tape drives:
/d u,tape
/v 480,online

You can use dslist in TSO (to easily look at the dataset)

So I started MVS in hercules mainframe emulator on Manjaro. Went to start a x3270 terminal. It did not exist…no problem I’ll just install it. So I installed it. But it said it required a restart. Hardly anything requires a restart. Ok I need to try and shutdown MVS manually. Found manual said something like shutting down bsppilot, and gave the command…it didn’t know what I was talking about. So reentered with leading /. That seemed to work. Couldn’t they just show that in the command? Of course the script wanted to know if I wanted to (S)top or (C)ontinue or something like that. So I type C…it didn’t like that! I type /C…it didn’t like that either!!! Anywho I seemingly got it to end…returned to hercules prompt. Restart…good! Start MVS…good! Start x3270, and I get this…

Just look at that wonderful menu and error message!