Using hashes to protect passwords from casual observers and undetermined attackers

The first thing we might reasonably want to do is protect passwords from casual observers, such as technical staff who have access to passwords in the database. While your admins are probably people you trust, there's no reason to tempt them, right?

The easiest way to protect passwords from casual observers is to hash the passwords before storing them in the database. That way admins see only hex codes instead of plaintext passwords.

If you recall from the previous section, we mentioned that in general we can't compute the password from the hash. You might wonder then how we accomplish logins. The answer to that is to hash the user's submitted password when he logs in, and then to compare the submitted hash with the stored hash. Because different passwords will almost certainly have different hashes, we can conclude that the authentication was successful if and only if the two hashes match. Sounds great, right?

Not exactly. This scheme does in fact prevent the casual observer from accidentally seeing passwords, and it may even present the newbie attacker from recovering a password. But in jujutsu-like fashion, we can use the for-all-practical-purposes 1-1 nature of the hash function as a tool to defeat it. Let's look at an example, shall we?

Go to Google and search for "online md5 hash database". Here are a couple that I found. (Note that I don't have any control over these websites and in particular I can't promise that they won't have unsavory ads. These two don't appear to have that, but sometimes hacker websites do have that sort of thing.)
- http://md5.igrkio.info/md5-hash-database.html (this is a metasearch)
- http://gdataonline.com/seekhash.php
Copy the MD5 hash for 'friend', 'friends' or 'password' above.
Paste it into the hash database search field submit it.
Hey, you've recovered your password! Huh?!

So what's going on here? It turns out that there are various online hash databases—essentially lookup tables—that map hashes to the inputs that generated them. Helpful hackers around the world create such databases by precomputing the hashes associated with dictionary words and close variants (in multiple languages, even). Because different passwords map to different hashes, you can create a lookup table for any explicitly given set of passwords. If an attacker has such a hash database at his disposal, it's pretty easy to launch a so-called dictionary attack against a compromised password database. The attacker can either attack passwords one-by-one, or (more commonly), the attacker can simply look for hashes associated with common passwords ('password', 'qwerty', etc.) and do whatever he wants to do with them afterward.

You'll notice that we did not pick the hash for 'I pledge allegiance to the flag'. If you copy the MD5 hash for 'I pledge allegiance to the flag' into a hash databases, chances are that it won't be able to figure out what generated the hash. (And if it does, it's easy enough to manufacture hashes that won't generate a match.) That's because the hash database creator did not precompute a hash for that particular phrase. And we can use that little tidbit to make our hashing scheme more secure, as we're about to see.

« Previous | 1 | 2 | 3 | 4 | 5 | Next »

Comments (12)

Great article, very timely given the continuous stream of new's about security exploit's and problem's. Another item consideration might be to use per user random salt's. If you continue down this line of tutorials If i may suggest a topic, how to address API security, and also browser to server security.

Thanks

By Aaron Raddon on Sep 1, 2008 at 9:13 PM PDT

Hi Aaron, really nice to see you here. :-D Thanks for the nice words about the article. I agree that security articles are timely and I'm afraid they probably always will be.

Per-user random salts are a great idea, especially in the case where you are somehow able to keep the random salts separate from the passwords. If the passwords are compromised but the salts are not, having random salts will be much more effective than using a sequence-based PK, because the attacker won't know what needs to be hashed, and a brute-force approach would probably be infeasible if the number of salt bits is sufficiently high. I don't myself know best practices around keeping passwords and per-user salts separate--it seems to me that if you store the salts and the passwords in the same table then the attacker will typically have either both the password and the salt or neither--but I'd be interested to hear if somebody can suggest a best practice in this area.

Having said that, non-random salts (such as a numeric PK) are still a lot better than just a straight hash. This at least forces the attacker to create a new rainbow table for each user he wants to attack instead of being able to rely on a single generic rainbow table.

By Willie Wheeler on Sep 1, 2008 at 10:02 PM PDT

I did enjoy this article but perhaps it would better be entitled Storing Password Hashes Securely as you are in fact storing hashes and not passwords.

There are some situations which require storing passwords themselves. This is a very difficult problem I would love to see more written about.

By Richard Minerich on Sep 2, 2008 at 7:02 AM PDT

Hi Richard. I can see what you are saying. I agree that in some cases you want to store passwords themselves (or at least encrypted passwords that you can decrypt). That can happen for example if you need to use the password to authenticate into some other system. In that case you might consider using an actual cipher (two-way) to encrypt the password before storage instead of a hash function (one-way).

By Willie Wheeler on Sep 2, 2008 at 7:26 AM PDT

Nice article Willie!!! Can you share some best practices to implement "remember me" functionality? Do you think using a "salted hash" is a good enough solution to store passwords in cookies?

By Venugopal on Sep 2, 2008 at 8:48 AM PDT

Excellent article describing a subject that can be difficult for people to get there heads around.
Do you have any more advice about choosing a salt? Would you create a random string and store it with the user?

By Ben on Sep 2, 2008 at 9:33 AM PDT

Great article Willie. You've opened by eyes to the random salt concept. Thanks, Collin

By Collin on Sep 2, 2008 at 10:19 AM PDT

@Venu: Regarding remember-me, one suggestion would be to recognize the reality that it trades security for usability. It's really authenticating the browser (using a persistent cookie) instead of the user, which is sometimes OK and sometimes not. (For example, several users in the household can share a machine; several users may share a public machine.) So I would suggest that when someone remember-me's into your app, the safest assumption is to assume that the user is very possibly *not* the user who they appear to be. This assumption forces you to distinguish high- and low-risk functionality, and if the user attempts something high-risk, you force an actual login. Amazon does exactly this. A remember-me user can see product recommendations, but if he wants to buy something he has to log in.

For the actual mechanics (for instance, what tokens to use), there are multiple approaches. Here are two you might take a look at:

http://static.springframework.org/spring-security/site/apidocs/
org/springframework/security/ui/rememberme/TokenBasedRememberMeServices.html

and

http://static.springframework.org/spring-security/site/apidocs/
org/springframework/security/ui/rememberme/PersistentTokenBasedRememberMeServices.html

The first doesn't require a database, but it does involve storing the username in the cookie, which may be unacceptable in certain cases. (For example, if you are doing remember-me for a credit repair website, you may not want to expose usernames in cookies.) The second does require a database but it avoids the username problem. It is based on the article

http://jaspan.com/improved_persistent_login_cookie_best_practice

Hope that helps.

By Willie Wheeler on Sep 2, 2008 at 9:16 PM PDT

@Ben: See my response to Aaron, but I'll elaborate.

One approach is to use randomization to create secret salts. That makes brute forcing the hash orders of magnitude harder because your salt search space is now orders of magnitude larger. But this only works if the salts can be kept secret. If the attacker has the per-user salts (which seems likely if he already has the password database) then brute-forcing a hash with a random salt is no different than brute-forcing a hash with a nonrandom salt--one the salt is revealed, it makes no difference whether the salt is random or not. The salt is known; the search is over. :-)

If you can keep the salt(s) secret (either global salt or per-user, however you decide to do it), then your passwords are much more secure, since it's essentially like having two passwords, with at least one of them being very strong. But now you just have to figure out how you're going to manage to keep the salts secret. That is a challenge in its own right.

The other approach is to treat the salts as known rather than secrets. Though it's easier to crack this scheme, this approach is still useful because it foils attempts to use a precomputed rainbow table against your passwords. The attacker can still get at your passwords by building new rainbow tables, but now he has to work at it. :-) You're essentially creating a deterrent that will cause many attackers to look elsewhere, and such deterrents are very much a tool you can use to enhance security.

By Willie Wheeler on Sep 2, 2008 at 9:57 PM PDT

Just one more thought regarding the deterrent thing I just mentioned. I was at a security talk the other day and the presenter noted that attackers are very much ROI-driven. Deterrence adds security because it drives down the ROI.

By Willie Wheeler on Sep 2, 2008 at 10:12 PM PDT

when will your book Spring in practice come in Indian market ?

By punit singh on Nov 15, 2008 at 2:41 AM PST

@punit: Thanks for asking. I'm not sure how Manning handles international sales, so I can't speak to the Indian market. In terms of the book itself being available, the original estimate was June 2009, but I suspect that we will push that back a bit since the original schedule didn't account for Spring 3, and we obviously want to cover Spring 3.

By Willie Wheeler on Nov 17, 2008 at 1:10 AM PST

Using hashes to protect passwords from casual observers and undetermined attackers

Comments (12)

Post a comment

What's New?