Peeling the Onion of SFTP: Options for Securing z/OS File Transfers

January 1, 2012 (last updated: October 6, 2023)

The latest webinar in our Peeling the Onion series was on October 4th, 2023: Peeling the Onion! Exploring Mainframe File Transfer Methods & Best Options for Security. Visit our Secure FTP for z/OS Webinar page to watch our latest SFTP webinars.

Does your organization need a Secure FTP (SFTP) solution for the mainframe after becoming aware of the serious security shortcomings in the typical FTP setup? Our complete, secure mainframe FTP solution is a collaboration between SDS and SSH Communications Security: together, VitalSigns for FTP^™ and Tectia^® SSH provide unprecedented Secure FTP for the z/OS mainframe and beyond.

FTP (File Transfer Protocol) between a client and server, regardless of which solution is used for the file transfers, is potentially dangerous to your organization and should never be used without additional security measures and precautions. Sure, you can go overboard for your particular organization’s amount of file transfers and other security requirements, but that’s much better than underestimating the threat and not doing enough.

The VFTP-SSH Collaboration has become a leading option for organizations requiring absolutely secure FTP, especially across multiple platforms. VFTP-SSH may classify as going overboard and deemed excessive for many average small businesses; however, for an entity that has regulatory requirements within their industry or would face substantial costs if their FTP is compromised, the file transfer process is never too secure. Don’t believe it? Just find anyone who’s ever worked for a business that’s been breached, even if only in a minor way with no negligible impact, and ask them about the aftermath…

In truth, any number of costs, liabilities, penalties, and other unforeseeable, additional hardships could result from a compromised FTP setup, leaving, plenty of times, a different and financially-crippled, or even bankrupted, organization in its wake.

OK, let’s take stock of where we are: you’ve been sufficiently informed (translation of “informed”: warned and scared) about the horrible, yet predictable, FTP-related disaster awaiting and the resulting outcomes. Basically, this situation is rightfully scaring you and we’ve caught your attention, let’s get on with this and give you a few options to relieve all your newly acquired, FTP-related anxiety by securing your file transfers…

Due to security concerns, many of our customers have been told by the security and compliance decision makers in their organization to stop using their current FTP setup NOW! Now meant now – no more normal FTP transfers are allowed, starting right now! They’re told to use SFTP instead. And that’s about all the direction they’ve been given.

Often, that direction is imprecise. What their users want is security and encryption. But they don’t necessarily mean the particular security tool, common to Linux and Unix systems, that is named SFTP. For z/OS, there are other ways to secure file transfers, several others—maybe too many others.

An SDS customer recently explained that, the more he looked into securing FTP, the more it resembled an onion!

It had layer after layer, and the more he peeled them back, the smellier it became.

This is how he capped off the comparison between their securing FTP quandary with a stinky onion: It’s enough to make a grown z/OS admin cry!

That resonated with us and eventually lead to The Onion FTP Webinar and this Article…

My Goal for this Article: Help you Secure your z/OS FTP

My intent with this article is to briefly explain some ways to make FTP a secure resource in your environment, to give you a brief taste of the options.

In other words, let’s peel the secure FTP onion

Slice #1: FTP (File Transfer Protocol)

Figure 1 - FTP The File Transfer Protocol (FTP) is a common, well-known, and easy-to-use application for moving data. The big advantage of FTP is that it is already available in so many places: z/OS, Windows, Unix, Linux, you name it. Almost every system comes with FTP. The client and server are both simple to use.
FTP has been the de facto standard for so long that most implementations come with platform-appropriate extensions. The z/OS FTP client and server, for example, handle MVS datasets and ASCII to EBCDIC conversion without skipping a beat. Even JES support is available for submitting jobs and extracting reports.

So it’s too bad FTP deals with privacy and security so poorly. FTP was designed back when the electronic world was small and safe. It transmits clear text. Eavesdroppers can see user IDs, passwords, data files. In a closed, secure environment, FTP simplicity can save a lot of hassles, but that type of environment is more the exception than the rule.

Firewalls, a common precaution, let safe traffic pass. But FTP is troublesome with firewalls because it uses two connections, one for commands, one for data transfers. That second, data, connection, starts with a connection request from outside the firewall, from a server trying to reply to a client request (that’s active FTP). Or it starts with the client asking to connect to a previously unknown, ephemeral port chosen by the server and passed in reply to the client request (that’s passive FTP). In either case, it takes a special effort to tell a firewall it can trust that outside server. And it’s near impossible if the FTP commands are encrypted (see FTPS, below).

FTP has three other shortcomings worth noting.

Data compression is not native to the FTP protocol and is rarely available. Some implementations have a MODE Z extension that supplies compression, but you can’t count on it.

FTP does not provide any application-level verification of data integrity. It relies on the TCP checksum, a minimal check that is easily fooled.

FTP sends user IDs and passwords in clear text, as well as data. So if the FTP server requires authentication, your IDs and passwords may become public along with your data.

So simple FTP is useful, but only so long your use of it passes the “anonymity test:” Is it OK for the data to become public knowledge? Accuracy is not a big deal? Authentication isn’t necessary? If yes, yes, yes, then FTP makes good sense.

In the end, far too many secrets are being passed via FTP, and a review of what FTP users are moving into, and especially out of, your z/OS machines is probably a good idea.

Slice #2: FTPS (FTP over SSL)

Figure 2 - FTPS FTP over a secure socket layer (SSL) is a secure twist on native FTP. FTPS is already available in the z/OS arena, as part of the base FTP already installed. A bit of configuration work adds command and data encryption onto the simple tool we’ve come to know and love. The protocol is the same. The command syntax is the same. The z/OS-specific features are there: support for MVS datasets, ASCII-EBCDIC, JES.

FTPS accommodates x.509 certificates and Kerberos encryption. Outside services—VeriSign, Equifax, and others—can make sure clients/users are who they say they are. On z/OS, RACF key rings can manage the certificates.

What’s the downside?

Well, for one, remember simple FTP and firewalls? When commands are encrypted, a firewall can’t sniff the commands that identify the ports to be opened. So passive FTP can’t work.

For another thing, while FTPS is available on most platforms, it’s also less likely to be configured and active.

So when does FTPS make good sense?

FTP over SSL is a great candidate for data transfer between LPARs in your sysplex. In-house communications among z/OS and IBM i/Series systems is another good use of FTPS. By “in-house” I mean where there’s no firewall to block the connections. Other platforms might have FTPS available too, but the further apart the partners are, the more likely a firewall will need configuration for multi-connection traffic.

Slice #3: FTP through SSH Tunnel

Figure 3 - FTP over SSH Here’s an encryption mechanism that’s more firewall-friendly than SSL.

Secure Shell (SSH) is an encryption technology that provides confidentiality, authentication, and data integrity. SSH can encrypt most any kind of TCP traffic, including z/OS file transfers. The z/OS FTP clients and servers you already have in place can take easy advantage of it by sending FTP traffic through an “SSH tunnel.”

It works like this: For each FTP client or server, there is also an SSH client or server on the same side of the firewall, often on the same machine. Users work with FTP as they’re accustomed to, and batch jobs use the same FTP commands they always have. But FTP clients and servers pass their messages to the SSH software. SSH encrypts commands and data, makes the connections, and passes the messages across the network through the “SSH tunnel.” A similar FTP-SSH partnership receives the data at the other end.

SSH communications are more firewall-friendly than FTP because SSH works with a single connection, usually on port 22. One connection handles commands and data. One tunnel can multiplex multiple connections between multiple applications simultaneously. No one is requesting secondary connections from the wrong side of the firewall.

SSH compresses data as it encrypts. The reduced bandwidth requirement has some CPU cost, but that might be offloaded to a crypto processor, a ZAAP or ZIIP, or an IFL processor.

SSH also provides a more sophisticated checksum process. If data gets corrupted in transit, you can be confident that you will know about it.

If all the pieces are in place, SSH tunnels are a firewall-friendly solution that requires little or no modification of batch jobs.

But getting the pieces in place requires significant coordination among you and your partners. Both ends of every transmission need both FTP and SSH up and active. FTP clients are common enough. SSH clients and servers are common on Linux and Unix, where admins have come to rely on them. But those admins typically do not run FTP servers, because they never needed them before, and because it means opening another port and another risk. Your partners may resist loading a back-end FTP server.

Slice #4: SFTP (SSH Secure FTP)

Figure 4 - SFTP SFTP is a subsystem built into SSH clients and servers. While SSH encrypts data and manages connections, SFTP provides a command language similar to, but not the same as, FTP commands.

SFTP takes full advantage of SSH encryption, compression, and integrity checks. It is also semi-ubiquitous—almost always available on Linux and Unix, not on z/OS and Windows.

You can get SFTP implementations for free, but the free solutions have significant limitations. For a fee you can install SFTP with extensions for ASCII-EBCDIC translation, support for MVS datasets and the JES queue, and other z/OS-specific features. The ability to offload encryption work to a crypto processor can also be important.

Finally, be aware that SFTP command syntax is only similar to FTP’s. Users will have to be re-educated, and batch jobs will have to be revised.

SFTP might be a great option for you if you’re sending data to your Linux farm or to partners who use Linux or Unix.

Slice #5: FTP to SFTP Conversion

Figure 5 - FTP-SFTP conversion What if you’re so heavily invested in FTP that conversion to SFTP simply isn’t an option? It would mean re-writing way too many batch jobs?

Consider this trick for building a bridge from FTP to SFTP: Tell z/OS FTP clients to route messages through an SFTP proxy. The proxy translates FTP commands to SFTP commands and uses SSH to transport messages to an SSH/SFTP partner at the other end.

Not all FTP commands can be automatically converted to SFTP, but all the basic ones can.

Thus network traffic is secured by SFTP, but client jobs don’t need to be re-written. They’re still using FTP syntax.

You can implement this FTP-to-SFTP conversion by altering FTP client configuration, by coding simple changes in the jobs, or, most easily, by implementing software that controls FTP client configuration dynamically. Depending on the job, and depending on whether the remote end of the connection has SSH/SFTP, that controller can adjust FTP client configuration to use its SFTP proxy or take some other route.

The SFTP proxy for this solution is a SSH^® client from SSH Communications Security. The controller for the z/OS FTP client is the VFTP Client from SDS.

Slice #6: VPN (Virtual Private Network)

Figure 6 - VPN VPNs are not specifically file transfer tools. They encrypt everything passing between two trusted partners. VPNs are commonly set up to securely join two networks, or to join a single host to a network—like a home office to company headquarters.

Most VPNs provide data integrity as well as encryption. Some provide compression too. VPNs work at a low level of the IP stack, so applications don’t need any special accommodations for them.

But VPNs take a significant amount of installation and administration. You don’t set them up on the fly. You don’t employ them for ad-hoc connections. You build VPNs for high-volume, persistent partnerships over the long-term.

Lastly, understand that VPNs are generally managed by the network group. If that’s not you, the work gets offloaded, and maybe that’s good. Or maybe it’s out of your control, and that’s bad. It depends on your point of view and your place in your organization, I suppose.

But anyway, a VPN can be a great solution for the right problem, and it might well play an important part in an overall security strategy.

Slice #7: PGP Encryption (Encrypt data at rest)

Figure 7 - PGP Each in its own way, the above solutions do a great job of protecting data while it travels through the internet, but data in motion is not your only concern. It’s pretty clear, sadly, that most electronic fraud is the work of insiders, people to whom you’ve given intranet access.

The FTP to SSH partnerships I described above pass unencrypted data between the FTP and SSH clients/servers. And with a typical VPN, all encryption stops at the firewall. So let’s turn to a security solution that is different than all the others. It works inside your organization as well as out, no matter what your file-transfer method.

Encrypt your data at rest.

Encrypt files inside the organization, regardless of your file-transfer methods. Encrypt data before it walks away on a stolen laptop. Encrypt data before a bad employee copies it to a thumb drive. Encrypt data getting offloaded to archives (just keep track of the keys for later). Encrypt data locally before any file transfer, and know that it’s safe and encrypted even after it’s transferred beyond your control.

Storing and transferring encrypted files means that only people with the right credentials and keys can extract and read the data. It also means that any transport method will do. Plain, simple FTP becomes a valid choice again. It doesn’t matter that IDs and passwords for FTP sessions are in plain text. For encrypted data, anonymous FTP can be safe enough, no IDs and passwords necessary.

Encryption-before-transfer works for you even over networks you don’t control. Maybe your firewall admin is a little finicky or has other priorities other than yours. No problem that. You still have control over your data.

On z/OS, encrypting data at rest probably means using PGP® encryption. The PGP encryption protocol uses an open standard—see RFC 2440—that has survived more than a decade of intense review. It is widely used on every major platform, so it’s not hard to get cooperation from your partners. SDS provides an implementation, PGP Command Line 9 for IBM Mainframes, that includes compression and integrity checks.

SDS E-Business Server® is a PGP encryption product for z/OS data at rest. It may take a bit of overhead and staging, but once you’ve introduced it into batch jobs and processes, that step can be as transparent as anything else.

Conclusion

I hope secure file transfer isn’t quite as confusing as it was when we started.

You still face decisions, of course. None of the above solutions will likely meet all your needs. Each has its own benefits, advantages, complications, and limitations. Odds are that your environment and your partners require a mixture of security solutions.

Our technical staff here at SDS will gladly help you get a grip on your particular onion. Also, be sure to check out our unique VitalSigns for FTP and SSH Collaboration for a complete z/OS FTP solution.

Tim Full, technical services manager, SDS, Jan. 2012

Product Webinar