Introduction

RSA can encrypt data to a maximum amount of your key size (2048 bits = 256 bytes) minus padding/header data (11 bytes for PKCS#1 v1.5 padding). As a result, it is often not possible to encrypt files with RSA directly. Also, RSA is not meant for this. Instead, we can encrypt a secret password (not shared with recipient) using recipient’s RSA public key, encrypt the large file using a key derived from this secret password and then send the encrypted secret password and encrypted file to the recipient.

NOTE: For this example, let’s assume that the recipient has generated a publickey and corresponding private.pem private key. Sender has received a copy of publickey

Sender will follow these steps:

  1. Generate a 256 (or any non-zero value) bit (32 byte) random password

     # Generate a 32 (or any non-zero value) byte random password in hex
 $ openssl rand -hex 32 -out randompassword
 $ cat randompassword
 28D42DC7FF8AEAE5BFB23B1771C8E2C5997A8ABF10011520D9047C4AA938F10A

    Since hex character occupies 4 bits, to generate 256 bits, we would need 64 hex characters (64 x 4 = 256)

  2. Encrypt your file with a random key derived from randompassword
     $ openssl enc -p -aes-256-cbc -salt -in foo.txt -out foo.enc -pass file:./randompassword
 salt=945B287F64A17C25
 key=D888EC68E573197CF770624AC5738193753FE8D3D8A6718DE4C8B15A0E726626
 iv =D2BC27B45EAAFA427005573DCE192FC7

 $ file foo*
 foo.enc: openssl enc\'d data with salted password
 foo.txt: ASCII text
    
 $ ls -la foo*
 -rw-r--r--  1 <username>  <group>  3040 Mar  6 16:48 foo.enc
 -rw-r--r--  1 <username>  <group>  3022 Mar  6 15:48 foo.txt

    Parameters explained

    • enc: Encoding with Ciphers
    • -p: Print the key, initialization vector and salt value (if used)
    • -aes-256-cbc: AES Encryption with a 256 bit key and CBC mode
    • -in: Input file name
    • -salt: Add a salt to password
    • -out: Output file name
    • -pass: Password source. Possible values for arg are pass:password or file:filename, where password is your password and filename is file containing the password.

      The key and IV (initialization vector) are derived from randompassword

  3. Encrypt your random password using recipient’s RSA public key

     $ openssl rsautl -encrypt -inkey publickey -pubin -in randompassword -out randompassword.encrypted
    
 $ file randompassword*
 randompassword:           ASCII text
 randompassword.encrypted: data
    
 $ ls -la randompassword*
 -rw-r--r--  1 <username>  <group>   65 Mar  6 15:38 randompassword
 -rw-r--r--  1 <username>  <group>  256 Mar  6 16:26 randompassword.encrypted

    Parameters explained

    • rsautl: Command used to sign, verify, encrypt and decrypt data using RSA algorithm
    • -encrypt: encrypt the input data using an RSA public key
    • -inkey: input key file
    • -pubin: input file is an RSA public key
    • -in: input filename to read data from
    • -out: output filename to write to
  4. Send both randompassword.encrypted and big-file.pdf.encrypted to the recipient

Recipient will follow these steps:

  • Decrypt the randompassword.encrypted using his RSA private key private.pem to obtain randompassword

      $ openssl rsautl -decrypt -inkey private.pem -in randompassword.encrypted -out randompassword.decrypted
  $ diff randompassword.decrypted randompassword
  $ cat randompassword
  28d42dc7ff8aeae5bfb23b1771c8e2c5997a8abf10011520d9047c4aa938f10a
  $ cat randompassword.decrypted
  28d42dc7ff8aeae5bfb23b1771c8e2c5997a8abf10011520d9047c4aa938f10a

  • Decrypt big-file.pdf.encrypted using randompassword (to derive the keying material for decryption)

      $ openssl enc -d -p -aes-256-cbc -salt -in foo.enc -out foo.dec -pass file:./randompassword.decrypted
  salt=945B287F64A17C25
  key=D888EC68E573197CF770624AC5738193753FE8D3D8A6718DE4C8B15A0E726626
  iv =D2BC27B45EAAFA427005573DCE192FC7
    
  $ diff foo.txt foo.dec
  $

    Notice that the salt, key and IV used are same for encryption and decryption.

  • You may use any non-zero password size
  • You may use any cipher listed by $ openssl list-cipher-algorithms for symmetric encryption

[Q] Some articles mention generating a 256 bit random key and not a password. What is this password and key derivation? Are you sure?

[A] Right! Some articles refer to the 256-bit random material as key which is misleading and creates confusion. It leads us to think that we will generate a 256 bit random key and OpenSSL will use it to perform a symmetric encryption. However, we are using a secret password (length is much shorter than the RSA key size) to derive a key. OpenSSL uses this password to derive a random key and IV. This key will be used for symmetric encryption. If you don’t believe me, scroll up and see if the secret password (32 bytes) and the key used are same (they’re not!)

References

Credits