CMILLS.OCT HOST-INITIATED FILE TRANSFER WITH UNATTENDED PCS by CHARLES A. MILLS The author is a software designer and the president of Firesign Computer Company in San Francisco. Firesign has developed communication software for IBM systems since 1975. Background A recent survey in the computer trade press listed 150 examples of micro-to-host file-transfer software. The products ranged in price from $200,000 to as little as $29.95. Vendors of 3270 emulation adapters and software often include file transfer at no additional charge, an example being FT/TSO which Digital Communication Associates, Inc. (DCA) packages with Irma cards. These products work well for their intended users, often a manager in need of mainframe financial data for a PC spreadsheet. Why, then, would there be a need for yet another file transfer product? Firesign Computer Company, founded in 1975, is a custom software firm that develops communication software for IBM systems. Twice in the past, Firesign has contracted for projects in which the PC served as a slave to file transfer software running on a larger IBM system. These programs reversed the client-server relationship of the host and the PC, compared with the 150 products mentioned above. The third time Firesign signed a development contract of this nature, we recognized a wider need for host-initiated file transfer and used the project as a vehicle to develop a generalized product that would accomplish file transfer with unattended PCs. The product, called Outbound, has been successfully marketed for a little over a year now. The Software The software we developed consists of two main programs. The one visible to the user is a utility program which runs as a jobstep under MVS, driven by control statements from SYSIN. The control statements use a format familiar to users of VSAM Access Method Services (see Figure 1). Like IEBGENER or IEBCOPY, one end of the copy is an "anyname" DD statement, but the other end is a file on the PC named on the SESSION control statement. The utility jobstep becomes part of a job which may be submitted from TSO, but is more often a production job initiated by the shop's scheduler package. The utility's return codes, which follow normal MVS conventions, may be used to control subsequent processing. The next release of the product will write a keyed VSAM file recording the return code, record counts and other statistics for subsequent queries. The other major component of the package is a program for the personal computer. This package can be one of several programs, depending upon the connectivity hardware on the PC. The PC must be equipped with 3270 emulation hardware or software. Our initial implementation was for one of the most ubiquitous 3270 adapters, the Irma card. We have since extended our support to include the IBM 3278 emulation adapter and most of the other emulators from DCA, IBM, Attachmate and INS. The PC program monitors the 3270 "presentation space" (emulated screen, in plain English) through the emulator's Application Program Interface (API). The utility program communicates with the target PC by sending 3270 "screens" through VTAM to the Logical Unit (LU) named in the SESSION command. When the PC program recognizes a screen from its counterpart on the host, it intercepts the data and interprets it as a command: file open, read, write, close, etc. The PC program does the operation and returns the requested data or an error code to the host. The PC program runs as a Terminate and Stay Resident (TSR) or "background" program under PC-DOS. When a TSR executes, it appears to end immediately, and the DOS prompt reappears. By running as a TSR, the PC is kept available for word processing, spreadsheets or any other program, so actually files can be transferred with both unattended PCs and PCs which are in use for an unrelated task. We recommend to our customers that they start the program at PC IPL time (in other words, from the AUTOEXEC.BAT file) so that the PC is always ready when the mainframe requests a file transfer. Some TSRs have been plagued by compatibility problems with other PC programs. The problems seem to occur primarily because there TSRs must intercept every keystroke to check for their "pop-up" hot key. We do not have any need to intercept keystrokes, and have had no compatibility problems. Two Applications What makes a good application for host-initiated file transfer? The PC-initiated file transfer products work well for an application that is PC-driven. If the motivation for the file transfer is with the PC user, then that user will probably do a reliable job of initiating the file transfer on the PC. On the other hand, if the motivation is with the host, the PC user cannot always be counted on to initiate the file transfer when it is needed. What are the applications for which host-initiation makes sense? Our customers' applications fall into two categories: "production" data and software distribution. We originally developed the program to transfer payroll data from PCs which controlled time clocks. Before we wrote it, the client had not had a single week in which each of their pilot installations had completed the PC-initiated file transfer correctly! The package is also used by electric and gas utilities. The power companies use hand-held meter reading devices which dump their data into PCs. Our product uploads the meter data for the billing cycle and downloads route data for the next day. The other application is software distribution and backup. PC software may be uploaded from an information center PC into a host dataset and then broadcast to the appropriate PCs. A similar application is the scheduled backup of PC files. Our studies show a media cost of about 4 cents per megabyte for mainframe cartridge tape backup, as opposed to 62 cents to $2.64 for PC diskettes or streaming tapes. Benefits The major benefit of host-initiated file transfer is that it transmits files every day in the same way at the same time. There is a central control point, usually an automated scheduler, and a reduced dependency on field personnel, most of whom cannot appreciate the complexity of a mainframe data center. When an error occurs -- and errors always occur in teleprocessing -- we report it at the mainframe end, where there are trained specialists to resolve it, rather than at the PC end, where a frustrated PC user may simply power off the machine and go home. Another benefit is that it shifts the multi-megabyte network load of file transfer from daytime, when users commonly do file transfer, to nighttime, when it will not affect CICS response. For dial lines, telephone rates are also lower. The final major benefit is consistency of data. If a shop relies on PC users to backup files or to install programs from diskette, they may not do it regularly or on schedule. Internals The host utility program is a straightforward 370 Assembler program. The program opens a VTAM ACB and then uses the VTAM macros SIMLOGON and OPNDST to acquire an SNA session with each PC. The SNA Bind is built partially from hard coded values and partially from the default LOGMODE for the logical unit (PC). A file transfer program is considerably easier than a full- fledged teleprocessing monitor. We were able to use a straightforward design with minimum use of VTAM exit routines and a corresponding increase in robustness. Unlike a teleprocessing monitor, which must contend with hundreds of users hitting the Enter key in random order and the possibility of an error of any sort on any terminal at any time, a file transfer program like ours merely has to send and receive, checking each operation for errors. We chose to make the host utility reentrant, to minimize resource requirements during simultaneous transfers. To access individual members of partitioned datasets (PDS), we use a technique learned from the PF1 column in Technical Support (September 1987). Rather than using BPAM FIND, STOW, READ and WRITE and implementing our own deblocking and buffering, we use QSAM GET and PUT. To open a single member of a PDS, we use RDJFCB to read the Job File Control Block for the DD statement, insert the membername and issue OPEN TYPE=J. Two interesting considerations arise when doing wildcard copies from a PC to a PDS. The first is that PC file names are 11 characters (filename.ext) and membernames are only eight. We had to put a lot of thought into a mapping scheme that would be reasonably intuitive for the user. Secondly, PC filenames may contain characters such as underscore and hyphen which are not allowed in membernames. PC filenames may also begin with a digit. We have found that illegal membernames do not seem to cause a problem, so we print a warning and proceed with the copy. The names that contain invalid characters cannot be used in MVS JCL or utility control statements or typed into ISPF panels, but ISPF's selection lists handle them with no problem. MVS discards members whose names begin with numeric characters. Transparency (the ability to transmit all 256 possible bytes) is a difficult problem for a file transfer program which uses the 3270 datastream. A program may legally transmit only about 92 of the 256 possible characters as data. The remaining characters either have control meanings to the 3270 (for example x'1D' is Start Field), are used internally by the controller to represent attribute bytes, or simply have no reliable translation in the 3270 environment. A complication is that the datastream from the host is in EBCDIC, the PC utilizes ASCII, and the 3274 uses a unique character set internally which is neither ASCII nor EBCDIC. This means that each byte must be translated several times during its journey between the host and the PC. When the user specifies a file as "binary," some file transfer packages achieve transparency by sending the data in hexadecimal. That is, they transmit two printable characters for each file byte. For example, to transmit x'1D', a program sends the character "1" followed by the character "D". We chose a more economical scheme: we expand one, two or three untransmittable characters into 2, 3 or 4 legal bytes. The software packs the file bytes into 6 bits of the expanded bytes and sets the other 2 bits to make a legal character. The same routine which implements transparency also handles compression. It replaces sequences of identical bytes with a count and a single byte. On average, text files are compressed 30 to 50 percent. PC programs and other binary data typically expand 10 to 30 percent -- still better than doubling it by sending it in hex! The PC software is coded in Pascal. We use Microsoft Pascal, rather than the more popular Turbo Pascal, because it lets us link to Microsoft C libraries, which we need for the DCA emulator APIs. Most PC programs which support diverse devices, such as different printer models, consist of a main program which contains the basic logic and separate "driver" programs for each supported device. We chose instead to distribute a separate PC program for each 3270 emulator. This approach keeps the program small: under 26K for most emulators. Small size is critical for a TSR because it is always resident, occupying part of the limited 640K address space of PC-DOS. We organized the source code as a main module which implements all the file functions, and several emulator-specific modules. We linkedit the main module with an emulator module to produce the executable program for specific 3270 hardware or software. We could have chosen other protocols besides the 3270 datastream (called LU Type 2 in SNA). An obvious choice would have been Advanced-Program-To-Program-Communication (APPC) or LU 6.2. The advantages of APPC are its inherent transparency (i.e., all 256 possible characters may be transmitted) and transmission blocks may exceed the 2K size of a 3270 screen. The drawbacks are that APPC is not yet widely and reliably implemented on the PC, and many of our customers still use 3274s and older Irma cards which will not support APPC. Also, we felt that the benefits of APPC were greatest for customer-developed transaction programs which application staffs must port to successive generations of hardware platforms, rather than for file transfer software maintained by a tightly-knit group of systems programmers. We are considering APPC for a future version, when implementations are more widely available at our customers' sites. Other Approaches What are the other solutions to the problems of PC-initiated file transfer? Some shops simply live with the problems. Others install PC-initiated file transfer software which carries a license charge but is faster or easier to use, such as FT/Express, from DCA. Or a company can avoid the issue, distributing programs by mailing diskettes, rather than with file transfer software. Virtual disk systems, such as Arbiter from Tangram Systems Corporation, provide a good solution for PCs which require infrequent access to large host-oriented files. Virtual disk products allow a PC to reference a host disk extent as though it were a PC disk drive. Sending a file to this virtual disk is a simple DASD-to-DASD copy. Some customers choose to combine user-initiated file transfer with an extended control language such as Automator from Direct Technology Incorporated. These products will wake up at a specified time, sign on to the host and transmit files, all without human intervention. Of course, they can be no more reliable than the PC's battery operated clock and have little ability to deal with unanticipated transmission errors. Cooperative processing products such as Enter/3270 from Aspen Research, Inc. allow shops to develop cooperating PC and mainframe programs, which may be used for file transfer or other distributed applications. NetView Distribution Manager must be considered because it is part of IBM's strategic NetView family. It has comprehensive data management features, but its complexity, starting license price of over $100,000 and requirement for additional network hardware make it overkill for file transfer on most networks. Problems The product has been largely trouble-free and easy to install. Our most common problem during installation has been with VTAM definition. Installing one or more VTAM "APPLIDs" which identify the host utility to VTAM is not difficult, but it is sometimes the jurisdiction of an individual who may be absent or occupied at the time of the installation. Many customers are unaware of the implications of multiple domain (multiple host processor) VTAM networks. In a multi-domain network, terminals and PCs may be "owned" by a different MVS processor than the one on which an application is running. Shops usually define programs as "cross-domain" resources for the CPU with the terminals, so that the VTAM which owns the terminal can find the application when a user logs on. Our program's need to locate PCs reverses this requirement. We require that PC logical units be defined as cross-domain resources on the processor where the utility runs. This "backwards" cross-domain definition sometimes is confusing to customers. During mainframe development, our greatest problem was the "non- SNA" or "LU Type 0" protocols required for channel-attached 3174s and 3274s. Our past experience was entirely with remote telecommunications and our development data center has no channel-attached controllers which are available to us, so this was a problem area. Re-entrance is also difficult to test. MVS ignores the re- entrant attribute unless the program loads from an authorized library, and our data center is understandably wary of the security exposure in allowing testing from an authorized library. On the PC side, the wide variety of emulator offerings from IBM has proved bewildering, and the incompatibilities among IBM's almost compatible emulators has been frustrating. With most vendors, there is a support number that you can call and say, What are your product offerings? Where can I get documentation on the Application Programming Interface? With IBM there seems to be no single point of contact for development information. The sales force is helpful, considering that we are a small customer, but is (not surprisingly) oriented toward solutions for specific business needs, not toward providing an overview of IBM software. We hope that IBM's recently-announced Personal Communications/3270, which replaces several predecessor products, will bring more order to their offerings. Moving Ahead We are developing a VM version of the host utility. Although the functional porting of an MVS program to VM is straightforward, CMS users are accustomed to a different style of user interface, and this aspect of the conversion is a more subtle task. We have also decided to implement a header record to enable us to retain 11-character PC file names in a sequential mainframe file. We will also be able to maintain PC file dates, times and attributes. We are adding a command to print a PC directory listing on the host. We are also looking forward to APPC, and to adding more features to help customers control and manage their file transfers. /* 2866