Featured

Blog?

About

This computer, which predated Apple & the TRS-80, was instrumental to the beginning of “personal computers” and resulted in the creation of Microsoft. This in turn resulted in Linus Torvalds wanting something better so he created Linux.

Fake customer program timeline

I’m bringing this up, because below, I talked about the population issue. And this program has become incredibly useful to me, in generating “real looking” test data. In either normal random record length, data terminated with a line feed, on Linux… or fixed length records, common on IBM mainframes. There was a time I would try and manually build records like Mickey Mouse and Herman Munster, and it wasn’t long before I ran out of ideas. It was very tedious!

According to comments in my Python program, I started writing this in Python 2 in 2014. Later in 2018, I rewrote it Julia for speed improvements. Then it appears I converted my Python 2 program to Python 3 and added the mimesis package in 2020 for a huge speed improvement.

I almost didn’t find the program using grep, in my Python directory, because I was searching for “*mimesis*” (which works for find) and not “mimesis”., I just needed to add the ‘-E’ switch!

Fake customer generator program Modification

A few months ago or maybe several years ago I generated some fake customer data, using a Python program I wrote. Then I wrote a program (actually a GnuCOBOL control break program) to print totals by state and realized the number of customers between states were about the same. So Wyoming had approx the same amount of fake “customers” as Florida. Because I assumed, I basically generate a random number between 1 and 50, then grab a state from that number. It’s something I never thought of at the time, but it became obvious when I created some reports. That’s crazy. So I need to do something about that.

Population.

Not so fast!

I’m smarter than I thought.

I need to re-run some tests, because I don’t generate random states, as I assumed, by a number between 1 and 50. I actually select city/state/zip at the same time by selecting random zip code from my zip code data base, which has 42,741 zip codes. So by that virtue, California has more zip codes than Wyoming, so there should be a more realistic spread by state. This is an old table, but still… the customer records generated should be more, realistically represented, than I thought they were.

I did a few SQL SELECTS such as:

 SELECT count(*) FROM ZipCodes WHERE state="CA";.

Here are the results:

CA 2678
NY 2233
FL 1470
WY  197

So the odds are higher that I would select a CA zip code than a WY zip code. Maybe, not perfect (Today FL has a slightly larger population than NY)… but quite a bit more realistic. And acceptable to me.

I am smarter than I thought

In the past, I actually created a one million record SQLite customer database generated from my fake customer Python program . And doing a few queries proves I’m doing something right

CA 62983
NY 52620
FL 34977
WY  4672

So out of one million “customers” there are 62,983 from CA and only 4,672 from WY!

Linux file output to screen

I had to look this up, because I was drawing a blank.

I wrote a command line program that was expecting two files… input and output. And I wanted the output to print on the screen instead of a file.

./ditto.jl customer-fixed.ebc /dev/stdout|more

So the screen… /dev/stdout, is a file in Linux.

Also it was kind of hard to create a query that returned what I wanted.

It kept returning opposite results of what I wanted. Pipe screen output to a file

Build GnuCOBOL 3.2

Among other changes, I saw an EBCDIC conversion compile switch, which piqued my interest.
Latest version 3.2, manual build to replace 3.1.2 package install

My old install notes helped alot. One error at the beginning was solved simply by removing spaces in a subdirectory. Switching from apt-get to apt solved some problems. It appears that mpir, didn’t need to be installed.

Old (before)

bill@bill-MS-7B79:~/MyStuff/COBOL/progs/cob05-2$ cobc --version
cobc (GnuCOBOL) 3.1.2.0
Copyright (C) 2020 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <https://gnu.org/licenses/gpl.html>
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
Written by Keisuke Nishida, Roger While, Ron Norman, Simon Sobisch, Edward Hart
Built     Sep 14 2021 19:23:38
Packaged  Dec 23 2020 12:04:58 UTC
C version "11.2.0"
bill@bill-MS-7B79:~/MyStuff/COBOL/progs/cob05-2$ cobc -x cob05.cob 
bill@bill-MS-7B79:~/MyStuff/COBOL/progs/cob05-2$ ./cob05
Program Start!
+05000
bill@bill-MS-7B79:~/MyStuff/COBOL/progs/cob05-2$ 
bill@bill-MS-7B79:~/Downloads/Computers/Languages/gnuCOBOL/ver3.2/gnucobol-3.2$ sudo make install > log5.txt
[sudo] password for bill:           
common.c:94:10: fatal error: gmp.h: No such file or directory
   94 | #include <gmp.h>
      |          ^~~~~~~
compilation terminated.
make[2]: *** [Makefile:614: common.lo] Error 1
make[1]: *** [Makefile:657: install-recursive] Error 1
make: *** [Makefile:966: install] Error 2
bill@bill-MS-7B79:~/Downloads/Computers/Languages/gnuCOBOL/ver3.2/gnucobol-3.2$

Odd, I usually get these errors before the install process.
This fixed the above install problem
apt install libgmp3-dev

bill@bill-MS-7B79:~$ cobc --version
cobc (GnuCOBOL) 3.2.0
Copyright (C) 2023 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <https://gnu.org/licenses/gpl.html>
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
Written by Keisuke Nishida, Roger While, Ron Norman, Simon Sobisch, Edward Hart
Built     Mar 24 2024 15:09:28
Packaged  Jul 28 2023 17:02:56 UTC
C version "11.4.0"
bill@bill-MS-7B79:~$ 

GnuCOBOL other EBCDIC/ASCII considerations

Take a deeper look at this. I used INSPECT for my conversion, should also look into TRANSFORM. According to what I’ve read about TRANSFORM…

This statement exists within GnuCOBOL to provide compatibility with COBOL programs written to pre-1985 standards. The TRANSFORM statement was made obsolete in the 1985 standard of COBOL, having been replaced by the CONVERTING clause of the INSPECT statement (see INSPECT). New programs should be coded to use INSPECT CONVERTING rather than TRANSFORM. 

Also consider COMP-x data. Perhaps…

https://docs.oracle.com/cd/E53645_01/artwb/docs12cr2/wbref/CobolConverter.html

https://sourceforge.net/p/gnucobol/discussion/cobol/thread/3935289e14/

New project idea… already done

It’s 4AM and I can’t sleep. I was watching a YouTube video about COBOL on the Commodore 128. And it got me thinking about GnuCOBOL. I started looking at some of my old GnuCOBOL programs. And I realized I never wrote one to read EBCDIC data. You know hypothetically, an IBM shop sends me file in EBCDIC. So I think that’s the plan. I found some EBCDIC files in /home/bill/MyStuff/Computer_Simulator_Emulator/Emulators/Mainframe/Hercules/Tape/awsutilb-whb/ that I created with Jay Mosley’s maketape, I checked them with tapedump. I think to be consistent, tapedump should’ve been named dumptape. You know maketape… dumptape!

Ok, I forgot, but I found (by grep) that I did write a EBCDIC to ASCII program in 2017. Called COB05. I even copied it to github, 5 years ago. It compiles fine, but the input file is missing so I’ll have to recreate it.

This program assumes, all fields are EBCDIC. So it wouldn’t work properly for fields like comp-3.

Adding up the input fields, in the COBOL program, gives me a record length of 134. And I see I have some other ASCII files of 133 which would actually be 134 if you add the LF (end of line) character. I’m sure I used my Python program, to create the customer file. According to my comments, I used the Linux ‘dd’ command to convert an ASCII file to EBCDIC. But I didn’t give specifics.

OK... this
dd if="customer-fixed-5000(133).txt" of=file.ebc conv=ebcdic

Copied the directory COB05 to COB05-2 to make some minor cosmetic changes, such as align the PIC statements. Also reformat the ASCII table, I don’t know how those continuations (&) ended up on the same line. It looks better, but apparently wasn’t needed.

           78 ASCII   value 
          "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ" &
          "0123456789 !" & '"' & "#$%&'()*+,-./:;<=>?@[\]^_`{|}~"&
          x'0A'.
           78 ASCII   value 
          "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ" &
          "0123456789 !" & 
          '"' & 
          "#$%&'()*+,-./:;<=>?@[\]^_`{|}~"&
          x'0A'.

Apparently I used Mystuff for one of my input directories back then, and now I use MyStuff, so I had to change 1 line in the program… and it worked.

I was concerned it would be slow, and it may be. But for 5000 records…

bill@bill-MS-7B79:~/MyStuff/COBOL/progs/cob05$ time ./cob05 
Program Start!
+05000

real    0m0.109s
user    0m0.049s
sys     0m0.061s
bill@bill-MS-7B79:~/MyStuff/COBOL/progs/cob05$

Long Linux startup time

Maybe should be titled sign-on time. I remember a few years ago Linux came up very fast, even before I had an SSD, which I do now. Actually Linux itself doesn’t take too long to get to the sign-on screen. But after, signing on, it seems long probably over 30 seconds… to get to home screen. And it’s still doing a few background tasks, after that. I should do some timings.