How Does SSH Key Authentication Work?

At over twenty years old, SSH can be considered something of an old man in IT - given the pace of change. It's not often that the old ways are best in IT, but SSH is one of those rare pillars of consistency, providing security in a file transfer world where security threats, data exfiltration and spoofing are commonplace. So how has it remained so relevant? In this blog, I explain how SSH uses cryptographic key authentication to ensure that both client and server are who they say they are. 


What is SSH Key Authentication and Why it’s Used in Business?

To explain what SSH key authentication is, we need to take a step back and discuss the principle of authentication factors. In theory, when assessing the strength of an authentication requirement, the more factors required, the more difficult it is to gain unauthorised access. To give you an example, if a door requires a key and a combination lock to unlock it, then it is more secure than a door that only requires a key to unlock it. 

However, in IT, factors are associated with certain types of authentication. 

Single Factor: Something You Know Two-Factor: Something You Possess / Own Software Tokens – X.509 Certificates, SSH Private Keys
    • Password. 

    • PIN.

    • Secret Questions and Answers. 

    • OTP (One Time Passcode) – Google Authenticator or Microsoft MFA. 

    • Physical Tokens – YubiKeys. 

    • Keycards or fobs. 

    • Software Tokens – X.509 Certificates, SSH Private Keys.

    • Biometrics – Fingerprint, Voice Matching, Iris Scanning. 

    • Behaviour – Keystroke or Signature Analysis.

A simple username and password authentication process - which we are all very familiar with - is considered a single factor authentication. Adding a PIN to the authentication process is two elements that the user knows, which will make it harder to gain unauthorised access, but it is not a two-factor authentication and is not considered as secure as two-factor authentication. 

In the case of SSH, the protocol supports the use of something called an SSH key for authentication. Keys can be used instead of, or in addition to, a username and password combination - the latter being two-factor authentication and therefore the stronger option. The password is something you know and the key, which resides on the client, is something you own as it remains on that physical client. 

With this inherent strengthening of security, it is not uncommon for companies hosting Managed File Transfer (MFT) servers to mandate the use of SSH for connectivity and the use of keys for security. 


Public and Private Keys

Sounds good so far, right? 

Well, the mechanics of SSH keys are not the easiest to understand, and although the use of keys in MFT solutions is not uncommon, many people who work with MFT solutions find that they need to refer to articles such as this periodically to remind themselves of some of the intricacies. 

SSH keys use asymmetric encryption to generate a private and public key on the client. The public key, by virtue of its name, is public and can be securely shared with any server it wishes to authenticate against in the future. The private key remains with the client and is never shared, as this would expose the asymmetric encryption process to unauthorised authentication attempts. 

The process for correct and authorised authentication is as follows:

    1. Client sends its username to the server. 

    2. Client sends its password to the server which is verified (optional). 

    3. The server finds the public key associated with the username in its database, encrypts a message using it and sends it back to the client. 

    4. The server creates a hashed version of the unencrypted message and holds it temporarily. 

    5. The client decrypts the message using its private key and calculates a hash of the unencrypted message. 

    6. The client sends the hash back to the server and the hash is compared with that which the server has. 

If the hash matches, it proves that the public key was correct, and that the client is in possession of the corresponding private key. Authentication successful. 


How SSH Keys Work


Three Benefits of Using an SSH Key in Business

SSH key authentication offer several key benefits, some of which have been discussed already, but here are three examples. 

1. SSH keys offer not just authentication but also identification. This hasn't been explained in detail before because it's not technically authentication, but when an SSH connection is made, both client and server can check each other's keys to make sure they're connecting to the right parties. This helps to reduce the possibility of spoofing and man-in-the-middle attacks. 

2. SSH keys introduce a second-factor of authentication and therefore make the authentication process more secure. Two-factor authentication makes unauthorised authentication extremely difficult. Where passwords can be brute-forced, rainbow attacked or guessed, cryptographic keys are statistically impervious to these techniques. 

3. SSH keys don’t expire. This may not sound like a win but consider the challenges you may have experienced or witnessed in the past with SSL/TLS certificates expiring unexpectedly. SSH keys are secure and do not expire unless revoked or modified, giving organisations more control and fewer headaches when caught off guard. 


How to Generate a SSH Key Pair

As mentioned earlier, SSH keys can be used for both identification and authentication. However, to stay within the scope of this blog, we will focus on authentication. 

SSH authentication is the process by which a client is authenticated by a server. The server holds and uses the client's public key to encrypt a secret message. Therefore, we need to generate a private and public key pair on the client. 

How this is done depends on the software the client is using to connect to the server. I will use PuTTY as an example because it is a commonly used SSH client and is well known in IT circles. The PuTTY client comes with an additional utility called PuTTY Key Generator, which is used to generate key pairs for both PuTTY and other SSH clients.


How Does SSH Key Authentication Work?

We start by selecting the key algorithm and key size. There are a number of options available. In this example, RSA is used with a key size of 2048, but I would recommend using ECDSA or ED25519 with the largest key size if supported by the server you are authenticating against.

Asymmetric keys require the generation of random numbers, which PuTTY generates by capturing the cursor positions after asking the user to move the mouse. It's a bit geeky, and maybe unnecessary, but I love it!

Finally, you get your keys, but not before you can secure them with an optional password. The public key must be sent to the server owner for import and the private key imported into the client software.


SSH Key Management

There are three best practice rules I would recommend anyone keep ‘front of mind’ when using, creating or managing SSH keys.

1. Make use of strong algorithms and long key sizes. Server-enabled, as some servers don't support modern algorithms or long key sizes, especially if they haven't updated themselves. DSA should be avoided; RSA is widely used but is not as strong as elliptic curve algorithms such as ECDSA and ED25519. ECDSA can sometimes raise eyebrows due to its association with NIST and perceived problems with cryptographic backdoors, but unless you're transmitting state secrets or military information, this shouldn't be a concern.

2. Key should be rotated on a regular basis. How often is up to the client or server owner, but key rotation ensures that if keys are compromised or lost, their unauthorised use is limited to a certain period of time. Key rotation is not common where it is not required, as it is an overhead, but it is recommended.

3. Don't use keys in multiple places. It can be tempting, in the spirit of keeping things simple, to create a key pair and send the same public key to multiple servers. However, this multiple use of keys increases your attack surface and therefore your risk. Ideally, each server connection should have its own key pair so that if compromised, the impact is limited


Navigating SSH Key Challenges

SSH keys and key authentication are not the easiest of topics to get to grips with, and even those who have been working with MFT solutions for many years often do not have the best of understanding, leading to confusion and the creation of numerous support tickets. We welcome the opportunity to work more closely with our customers and help them with SSH key authentication but recognise that it can be a tricky subject. Here are three common challenges we see:


1. Key Mismatches

When connecting to a server, the client receives a warning that the key presented is not the same as the key presented on previous occasions. This warning attempts to prevent man-in-the-middle attacks by warning that something has changed. In most cases, the server owner has changed the SSH key on the server and has not notified his trading partners. We recommend that you confirm the key with the server owner and then update the client so that the new key is whitelisted.

SSH Key Mismatches

2. Further Authentication Required

This error and variants of it are extremely common. The authentication methods required are defined on the server and must therefore be matched in the client configuration. If key authentication is not enabled on the client, but is required on the server, you'll see a message like this.

SSH Key: Further Authentication Required

3. Server Refusing the Key

Similarly to the previous example, if no public key or the wrong public key has been provided to the server, authentication will fail. The key pair must be created on the client and the public key sent to the server to be associated with the username provided by the client. Is it possible that the server owner hasn't updated the public key with the one you supplied?

SSH Server Refusing the Key

SSH key authentication is not a well understood concept and continues to confuse people daily. This challenge is unlikely to diminish as SSH is a popular protocol for managed file transfer as it not only provides superior authentication compared to its contemporaries, but also provides an encrypted communications tunnel. These are all features that any organisation will want to use when transferring data between internal systems or with trading partners.

Both SSH and Pro2col are over twenty years old and in that time, we have helped countless customers to ensure that both are set up correctly and securely. Whether it is creating key pairs, troubleshooting connectivity, or asking about best practice, there is no one better placed to ensure that your business operates optimally. Get in touch with our team of experts today. 


About the Author

headshot of Christ Payne, Director of Strategic Alliances and Technical at Pro2col


Chris Payne is the Director of Strategic Alliances and Technical at Pro2col, with decades of experience in software management and Managed File Transfer solutions. Chris is not your traditional techie with server racks under the stairs. He advocates for a healthy work/life balance, saving his love for technology for work and developing his passion for craft beer at home. Chris is a qualified brewer and has on occasion whipped out the brewing equipment. He’ll be up for a Friday pint.

Find out more about Chris Payne.