My Techie-Thoughts

Passwords are great and a strong password is definitely the first line of defense against unauthorized access. But, if you think your Facebook account is important enough to warrant another layer of security so people can’t hack pictures you posted about your lunch, then surely you can understand why your Linux console could use another security wrapper too, right? Let’s add second factor authentication (2FA) to your console, su, sudo and SSH access all in just a few easy steps.

Probably the easiest to use and widely adopted form of multi-factor authentication (MFA) is what powers things like Google Authenticator and Authy: Time-based One-Time Passwords (TOTP). Simply put, these are 6 digit codes that are derrived from a shared secret and change every n-number of seconds. I doubt I need to explain these, you’re probably already familiar with them so let’s get to securing our consoles!

If you only want to secure SSH then skip to the Secure SSH with 2FA TOTP section and follow along with whatever options you want. Make sure to read to the end of that entire section so your clients are configured too. THEN, go back to the Get your TOTP code to activate the whole setup.

What we’ll be securing

By the end of this article, we will enable but not requre 2FA TOTP password authentication for console logins including su (switch-user) and sudo. There are also sections following that on how to enable 2FA for SSH logins.

A brief note before we continue: I am referring to these as 2FA TOTP passwords because, although we will be using a Google Authenticator module, any program that can process TOTP can be used to generate/lookup these passwords. This means Authy, Bitwarden, Keepass, Lastpass and a ton of other apps. So do not feel you must use Google Authenticator!

Install and setup Google Authenticator PAM

Linux authentication is generally handled via Pluggable Authentication Modules (PAM). We will be installing another module in addition to what you are already using. Namely, we will install the Google Authenticator PAM. Make sure you are logged in as root for the remainder of this article, it will just make your life easier not having to type ‘sudo’ constantly.

Install Google Authenticator module

Run the following, that’s it.

apt update
apt install -y libpam-google-authenticator

Configure PAM

To secure the console including su and sudo, we need to make a change to a PAM configuration file, specifically /etc/pam.d/common-auth. The comments with double-hashes (##) are for your information, comments with a single-hash (#) I’m actually recommending you put in your file as a good practice of annotating your changes. Finally, the unmarked line needs to be added to the end of the file.

## /etc/pam.d/common-auth
## add the following lines to the END of the file

# TOTP 2FA
auth    required    pam_google_authenticator.so nullok

You might notice that the last parameter listed is nullok. This means that 2FA will only be enforced for users that have set it up. Users that have not set up 2FA will not be affected, hence a ‘null’ 2FA is ‘ok’. If you want to require 2FA of everyone regardless if it’s setup or not, simply omit nullok.

That’s it as far as system configuration goes. Now we need to generate our codes.

Get your TOTP code

Hopefully the giant red box didn’t scare you too much… but seriously, take careful heed of that warning. Re-read it if you glossed over it. Ok, let’s keep going. This part is really simple, just execute this command and answer the questions on your screen.

google-authenticator

The first question is whether or not you want to use ‘time based’ codes. Answer *yes*. Next you will see a large scannable QR code and under that the secret key that you can add to any TOTP compatible password-manager program. Once you enter the key or scan the barcode, your client program should display the same verification code as is displayed on your screen. If they match, then you entered/scanned the code correctly. Finally, under that are your emergency scratch codes – please remember to save these!

You will then be asked if you want the program to update your .google_authenticator file. Say *yes*.

The next question is regarding whether to time-limit how often you can login using a TOTP code. While saying yes here will lower your attack surface, I usually say no since I often type my TOTP codes wrong and don’t want to wait 30 seconds to try again. This is totally up to you.

You are then asked a time-skew question. Read this and decide what you want to answer. I strongly suggest saying no here because your device/password-manager should never be more than 30 seconds out of sync with actual time!

This is similar to an earlier question but it doesn’t enforce a 30 second waiting period if you get your TOTP code wrong. Instead, it says that you can’t screw up more than 3 times in a 30 second time period. I find this very reasonable and usually say yes.

That’s it! You’re codes are setup. If you do a directory listing of your home directory, you’ll see you have a new .google_authenticator file. Make sure you keep this file safe and leave the permissions as they are (0400).

Secure SSH with 2FA TOTP

I hate to ruin the surprise, but SSH is automatically and inevitiably secured for any user that has their console secured with TOTP. Think about it, in securing the console, we required a TOTP in order to execute the login process and eventually dump to the console. In essence, we actually secured the login process not really just the console. Well, isn’t an SSH session just another type of login before opening a TTY console? That being said, there are reasons to specifically set up SSH 2FA TOTP such as separate requirements (i.e. mandatory on SSH, optional at console or vice-versa) or dealing with keys. So let’s go through all the options.

Basic password-only TOTP setup

The most basic and most logical SSH 2FA setup involves activating it for password-only connections. Passwords are great but, they are often not really that secure since people notoriously choose weak passwords. Thus, it makes sense to setup a TOTP option here. Also, as I mentioned before, if you secured your console using TOTP, then you’ve already secured SSH so we should set it up properly anyways.

Update PAM sshd configuration

We need to make a change to the PAM sshd configuration file located at /etc/pam.d/sshd. Open your text editor of choice and make the changes as shown below. The comments with double-hashes (##) are for your information, comments with a single-hash (#) I’m actually recommending you put in your file as a good practice of annotating your changes. Finally, the unmarked line needs to be added to the end of the file.

## /etc/pam.d/sshd
## add the following lines to the END of the file

# TOTP 2FA
auth    required    pam_google_authenticator.so nullok

You might notice that the last parameter listed is nullok. This means that 2FA will only be enforced for users that have set it up. Users that have not set up 2FA will not be affected, hence a ‘null’ 2FA is ‘ok’. If you want to require 2FA of everyone regardless if it’s setup or not, simply omit nullok.

IF you have also secured your console with 2FA then you need to make one more change to this file. Near the top, you will find a line referencing common-auth. That must be commented out as shown below:

## /etc/pam.d/sshd

## find this section of the file, usually near the top:
# Standard Un*x authentication
@include common-auth

## comment out the '@include' line entirely so it looks like this:
# Standard Un*x authentication
#@include common-auth

We do this because otherwise you will be prompted twice for your TOTP code. Once by common-auth for the actual login and then again by common-auth via this include statement in sshd. That’s just annoying.

Update sshd configuration

Since we made changes to the sshd PAM configuration file, we need to configure the sshd daemon to implement those changes as well. Here we will require a ‘challenge response’ for password logins only. The challenge will be a prompt asking for the TOTP password, the response would be the TOTP password – pretty clear, right?

This setup will NOT require TOTP passwords for ssh-certificate secured logins since they are not keyboard-interactive (i.e. no password is required, only the certificate). I’ll address how to add 2FA to certificate-based connections in the next section and why I don’t usually do it.

Open your sshd_config in your favourite text editor and make the following changes to enable ChallengeResponseAuthentication:

## /etc/ssh/sshd_config
## change the following line

# ChallengeResponseAuthentication no
ChallengeResponseAuthentication yes

To be safe, test your sshd configuration to make sure you didn’t spell anything wrong:

sshd -t

Assuming you get no error messages, you did everythink ok. Restart your sshd service (don’t worry, it will not terminate any existing connections).

systemctl restart sshd

For a basic ssh password-only setup, that’s it. Skip down to the client configuration settings section.

Add TOTP to SSH-certificate based connections

Before getting into this, I do want to give a little perspective on this one. I do not usually enable this since I think it’s overkill for most situations. Connecting with an SSH key is already very secure since the key is almost always at least 2048-bit and much more secure than any password a human could ever remember. In addition, you can password-protect this key which is already 2FA (you know the password, you have the key). Therefore, adding TOTP would be true MFA – 3-factor authorization. If you really think you need this level of protection, then by all means, here’s how you do it!

First, re-open your /etc/pam.d/sshd file and comment out the following line IF it’s in your configuration. Assuming it’s there, it should look like this when you’re done:

## /etc/pam.d/sshd
...
#auth    required        pam_unix.so no_warn try_first_pass

If you don’t have that line, then don’t worry about it. By default it is not present in Debian installations.

Now, let’s edit our sshd configuration at /etc/ssh/sshd_config and tell the daemon that you require two forms of authentication instead of just one (your key).

## /etc/ssh/sshd_config
## add the following line anywhere you'd like
AuthenticationMethods publickey,keyboard-interactive

Just make sure you keep the parameters in that order. Save the file. Test your sshd configuration and, assuming no errors, restart the service.

# test your configuration
sshd -t
# assuming no errors, restart the service to activate the configuration
systemctl restart sshd

That’s it. Make sure you read the next section on client configuration settings!

SSH client configuration settings

While settings will vary widely between SSH clients and platforms, given that you have in one way or another secured your SSH connections with TOTP, you need to let your SSH client know about this.

• If you secured password-only/console connections and you are not using an SSH-key, then you need to change your authentication method to keyboard-interactive or kbdi. If your client has ‘sub-methods’, then make sure you have pam,totp… listed in that order. These options tell your client to expect multiple prompts from the server and in the order of password then challenge question (TOTP).
• If you secured password-only/console connections and you are using an SSH-key, then you can keep your client settings the same. Namely, your authentication method remains publickey. TOTP is only enabled when you are using your password – in other words, if you use su or sudo during your console session.
• If you secured SSH-certificate connections, then you need to change your authentication method to publickey+keyboard-interactive or publickey+kbdi. If your client has ‘sub-methods’, then make sure you have pam,totp… listed in that order. These options tell your client to first authenticate using your public key and then move to interactive mode so you can supply the answer to a challenge question (TOTP).

If you leave your client set to the standard options, it likely will not understand the TOTP prompt and the server will log this as a failed connection. You can see if this is happening by checking the /var/log/auth.log file for mentions of pam_google_authenticator failures and immediate SSH connection closures.

Using the same TOTP code on multiple machines

If you’re like me and you administer several systems, it can be a pain having separate TOTP codes for each machine. I know it’s more secure, but for my personal networks and test networks, different codes for each machine is overkill. So how can we use the same code on multiple machines? Easy!

First, do the setup procedures on your other machines just as you did for this machine following this article. Install the Google Authenticator PAM, configure your PAM options and sshd, etc. However, do not run google-authenticator. Instead, copy the .google-authenticator file in your home directory to your home directory on your other machines. Boom! You’re done. The same TOTP code will work on any machine with that .google-authenticator file in your home directory.

Disabling 2FA TOTP

This is also pretty simple. It depends on how you have your PAM configuration setup, however. There are two options:

1. If you used the nullok parameter, then things are very simple. Just delete the .google-authenticator file in the home directory of any/all users for whom you want to turn off 2FA TOTP. If you want to re-enable 2FA, just re-run google-authenticator as that user.
2. If you omitted the nullok parameter, then you still follow the above step. However, you also have to add the nullok parameter to your /etc/pam.d/sshd and/or /etc/pam.d/common-auth configuration files. Alternatively, you could delete the entire pam_google_authenticator.so line altogether or comment it out in those files. Remember that omitting nullok means 2FA is mandatory, that’s why you need to take these additional steps versus option 1 above. To re-enable 2FA, you’d have to un-comment or re-add those lines in your PAM configuration files and then re-run google-authenticator as the appropriate user(s).

Final thoughts

Well, there you go! Nice and MFA secured now. Sorry that was a little long, but I found a lot of sites glossed over details about this setup such as mentioning that securing the console automatically secured SSH. That was a surprise to me, at least. I hope this helped you get things set up the way you want and you are breathing an extra sigh of relief knowing your machines are a little safer on the untamed internet.