Unattended Server to Server File Transfers: Unix to Unix
This article applies to: Managed Servers
Use SSH (Secure Shell) keys and SCP (Secure Copy) to perform secure, unattended, server to server file transfers between Unix servers.
Setting Up ID Accounts
If necessary, request IDs using the Unix Account Request Form. For more information about accounts, see the Single Sign-On Overview.
ID Types and Audits
The ID types used in secure, unattended file transfers are complex in order to create an audit trail. Holding IDs and File Transfer IDs cannot be traced back to a single user so each type of ID is given limited functions.
- (Regular) User ID: Is traceable to a single user. Can log in, run processes, send files. Do not use for unattended file transfers.
- Holding ID: Is not traceable to a single user. Can run processes. Cannot log in or send files.
- File Transfer ID: Is not traceable to a single user. Can send files. Cannot log in or run processes.
Setting Up SSH Keys for File Transfer ID
Secure Shell (SSH) is a program used to provide encryption during file transfers using SCP or SFTP (Secure File Transfer Protocol). SSH can use public/private key pairs (with or without a password) to authenticate file transfers.
How does it work? First, you set up a public/private key pair. See the following procedure for steps. Then, when you are ready to transfer files, the File Transfer ID on the receiving server sends the public key to the source server. Using the public key, the source server sends an encrypted challenge back to the receiving server. The receiving server uses the private key to decrypt the challenge and respond to it.
Note: If one or more of the servers does not use OpenSSH, this won't work. For more information, see the Open SSH site. Please contact systems-support@cornell.edu if you need assistance.
Note: This procedure works only when using the default key file name .ssh/id_dsa. If necessary, change the file name.
-
Run the procedure using the Holding ID for which the keys are being generated. On the source server, as the Holding ID, generate the key pair. Enter:
- For Solaris: /opt/common/bin/ssh-keygen -t dsa
- For Linux: /usr/bin/ssh-keygen -t dsa
This generates a key pair in ~/.ssh. You will be prompted for a password. Click Enter to indicate you don't want to use a password. (Don't set up a password because in an unattended file transfer, no one will be available to enter the password. However, because the private key is not password-protected, be sure to take extra care to protect it. For example, don't copy the private key to other directories.)
-
Using the password for the File Transfer ID, copy the public key to the File Transfer ID account on the receiving server. Enter:
cd ~/.ssh
scp id_dsa.pub filetransferid@receiving server:.ssh/authorized_keys -
Note: This overwrites any keys already in the authorized keys file. If you need to transfer files from more than one Holding ID to this File Transfer ID, use your own UserID to transfer the keys and concatenate them together. Please contact systems-support@cornell.edu if you need assistance.)
You can now scp from the source server to the receiving server without answering a password request.
Transferring Files
- Use the Holding ID's cron job process to transfer the file from the source server to the receiving server.
-
You can use this sample script:
30 1 * * * /opt/common/bin/scp source_file filetransferid@receivingserver:/path/receiving_file > /path/transfer_error_log 2>&1
This cron entry in the Holding ID account will attempt to copy the source_file every night at 1:30 AM and record any errors that occur in the file /path/transfer_error_log. (This error file is on the source server.) You can add to the sample script if you need to record the transfer in a log file, rename the file with the date once the transfer is done, archive and remove old files, etc.
Comments?