The Illustrated Guide To Salting Passwords

This is Bob, the snail.

Bob just hacked your company’s authentication database, containing your customer’s secrets. Depending on how well you’ve secured those secrets, Bob either becomes a rich and happy snail, or he ends up getting nothing.

So let’s see what happens to Bob, based on how you stored your company’s secrets.

Level 0 – Plain Text

At level 0, it’s amateur hour. Your secrets are all stored in plain-text.

Your team is composed of literal monkeys who accidentally created a piece of software by mashing random buttons until they got a program that compiled.

Because you’ve stored all of the passwords in plain-text, Bob now has instant-access to all of your customer’s information. Not only that, since most people use the same password on several websites, he’s got access to those accounts as well.

Bob now gets to relax on the forest floor, while money showers down on him from all of the accounts he’s hacked.

Level 1 – Hashing

One step up from plain-text is basic hashing with SHA256. When you hash a string, you get another string as output. This output is always the same length, regardless of its input length.

Additionally, the hash function is irreversible, and doesn’t give you any useful indication about the input. In the example below, the SHA256 hash for “foo” and “fooo” are completely different, even though they are only one letter apart.

While hashes can’t be reversed, they can be cracked. All Bob has to do is bring out his trusty tool, the rainbow table.

You see, hashes have a problem, which is that they always produce the same output no matter what.

In other words, when I use SHA256 to hash “foo”, I always get “2c26b46b68ffc68ff99b453c1d30413413422d706483bfa0f98a5e886266e7ae” no matter what.

What Bob has done, is used something called a “rainbow table”, which is a table that contains input keys, and the hashed outputs for those keys (typically as chains of hashes).

Theoretically, if Bob had lots of storage space, Bob could generate all possible passwords up to 8 characters in length, hash every single of those passwords, and store them. Once Bob has assembled his rainbow table, Bob will be able to automatically crack any password hashed with SHA256 in the blink of an eye if it’s 8 characters or less.

Why? Because Bob has already pre-computed all the possible hashes for all 8 (or less) character passwords.

Realistically, Bob is bottlenecked by memory and computation times when it comes to how long his rainbow table can be. Let’s say passwords can contain alphanumerical characters (35 possibilities) and symbols that are located on a normal keyboard, like +, -, ~, \, etc. This gives us an additional 21 possibilities.

In reality, there are way more possible characters, but let’s say that there were 56 in theory as a lower bound. If Bob wants to generate all 5 character possibilities, Bob needs to generate 550,731,776 inputs, and hash all them.

Here’s a chart that shows how absurd the rate of growth is for rainbow tables (assuming 56 possibilities per character):

# characters# of generated inputs
156
3175,616
5550,731,776
896,717,311,574,016
10303,305,489,096,114,176 (303 quadrillion)
169,354,238,358,105,289,311,446,368,256 (9 octillion)

As you can see, rainbow tables really can’t extend very far. Eight characters is still feasible, but after that, it gets inordinately expensive. A 16-character rainbow table would be incredibly difficult to store, and would take an eternity to generate. Note that 56 characters is an incredibly low bound, since there are many other characters that can be used as well. You should expect it to be even more expensive, in practice, to generate a full rainbow table for all 1-8 character passwords.

Level 2 – Salt And Hash

Clearly, the problem with our previous approach was that we were hashing passwords, but these hashed passwords could be cracked by a rainbow table.

So how do we prevent rainbow tables from being used against us?

By using salts! A salt is just a simple string that, in many cases, is just appended to your password before it’s hashed. These salts are generally randomly generated and quite long (16+ characters), which means that our user’s password (8-16 characters) plus the salt (16 characters) should be at least 24 characters long (also note that every password should have a different salt).

This makes our unhashed password at minimum 24 characters, so this password can’t exist in a rainbow table because no one has the capability to generate all possible passwords up to 24 characters long.

We then hash this password, and store both the hashed + salted password, as well as the salt itself, in our database.

But what about our salt? It’s stored in plain-text!

The salt actually has to be stored in plain-text, because the way we verify a salted hash is that we take the user’s inputted password, append the salt, then hash it. If it matches what’s in our database, then it’s the correct password.

So how does all of this effect Bob? Well for starters, it makes the rainbow table incredibly useless. Although Bob has our salt, Bob can’t use his rainbow table against us, unless he re-generates an entirely new one, where each input has the salt appended to it.

For example, suppose our password was “foo”, and the salt was “saltyfish”. Without the salt, Bob would instantly be able to figure out the hash for “foo”. But with the salt, Bob now has to take all of his inputs, append “saltyfish” to them, and only then, will he find that “foosaltyfish” gives a hash match.

Once Bob finds the correct match, he removes the salt (“saltyfish”), leaving him with “foo”. And that’s our password! So Bob can still manage to hack our account.

But that was a really expensive operation. You don’t want to have to re-generate an entire rainbow table, because they’re often petabytes or larger in size. And the whole process of doing that only allowed you to hack one user! If Bob has to go through 300 million users, he would have to regenerate his rainbow table 300 million times, because every password has a different salt!

With your secrets safely salted, Bob has no choice but to find other unwitting victims to hack. Preferably, ones who haven’t salted and hashed their passwords.

Advertisement

How To Become An Elite Performer By Using Multi-Tenant Architecture

Multi-tenant architecture, also known as multi-tenancy, is the idea of having your users (also known as tenants) share a pool of resources on a single piece of hardware.

To understand multi-tenancy, imagine you had a single computer, running a single program. Multi-tenancy is like having multiple applications run on one computer, while single-tenancy is like buying a new computer each time you had to open a new application.

Let’s suppose opening a YouTube video on Google Chrome counted as one program (and that you couldn’t open a new tab on Google Chrome). This would be the result:

However, unlike new Google Chrome instances, every tenant’s data is logically separated, such that no tenant has access to another tenant’s data, unless they have permission to do so.

The two main reasons to use single-tenancy architectures is

  1. They’re really really easy to setup. For the most part, you usually just have to give each tenant their own private database. Also, if your tenant needs a specific kind of environment that no other tenant uses, this is also easy to do, because each tenant can have their own private environment. Suppose one tenant wants to use PostgreSQL, another one wants to use MySQL, and third wants to use OracleDB. Assuming you have an interface to handle different types of databases, that would be no problem at all, because each tenant has their own database.
  2. It’s “more secure” than multi-tenancy architectures. Proponents of single-tenancy architectures will say that since you’re sharing resources on a single piece of hardware, you might have huge security vulnerabilities. This is because your tenants are typically separated by virtual machines/containers, with these VMs/containers being on the same computer. It stands to reason then, that if someone finds a vulnerability in the hypervisor that you’re using, then they can escape the VM/container (also known as a hypervisor attack) and theoretically access every tenant’s data on that machine.

Note that point 2, while a potential concern, is not really much of a concern. There are very few organizations that care more about security than the NSA (National Security Agency), and even the NSA reports these kinds of shared-tenancy vulnerabilities are incredibly rare, and that there “have been no reported isolation compromises in any major cloud platform”. Should you be afraid of hypervisor attacks over the massive cost savings of a multi-tenancy architecture? Probably not.

Now that you understand the basics of multi-tenancy architectures, let’s talk about why elite performers use multi-tenancy architectures.

Elite Performers Are Lightning Fast

The five essential characteristics of cloud computing, according to Accelerate’s State of DevOps 2019 Report, are:

  1. On-demand self-service (allow the consumer to provision resources)
  2. Broad network access (works on most platforms)
  3. Resource pooling (multi-tenancy)
  4. Rapid elasticity (rapid scaling)
  5. Measured service (can control optimize, and report usages)

Of these five characteristics, resource pooling, rapid elasticity, and measured services are all directly related to, or made easier with multi-tenancy. In other words, just by using a multi-tenancy architecture, you should already have three of these five things checked off, assuming you are following generally good practices. On-demand self-service and broad network access can be equally easily done with either single-tenancy or multi-tenancy architectures.

It should come to no surprise, then, that “elite performers were 24 times more likely to have met all essential cloud characteristics than low performers”. So clearly, the elite performers are using multi-tenancy architectures, but what’s the material gain?

Image courtesy of Accelerate’s State of DevOps 2019

More deployments, faster deployments, faster recoveries, and fewer failures. And not just by small margins either, recovering from incidents 2604 times faster is like the difference between recovering from an issue in a week, and recovering in just under 4 minutes.

Elite Performers Are Worth Their Weight in Gold

So how much exactly are elite performers saving with multi-tenancy architectures? According to Accenture, their work with police departments in the UK to switch over to multi-tenancy architectures will save a projected estimate of $169 million pounds (~$220 million USD). And on top of that, by making this switch, they claim that “time and workload gains in incident reporting can be upwards of 60%”. While switching to a multi-tenancy architecture is not easy, those are huge savings, brought mostly in part due to centralized data.

In multi-tenancy architectures, every tenant shares the same database, which makes sharing data much easier. According to Accenture, the average state in the U.S. has “300 different record systems”. That means that in the U.S.’s law enforcement agencies, there should be a minimum of 15,000 different databases.

Imagine you had to fetch data from 15,000 different databases. You don’t have access to any of them, and they’re all potentially in different formats with no national standardization. How do you coordinate such a massive task? Sure, you could email every single enforcement agency and have them manually share their data with you, but that would take on the scale of months to years.

Now imagine that all law enforcement agencies shared one database, with a handful of different standardized schemas. If you wanted to access a list of known offenders in Texas and Ohio, and merge them with some other table, you can do so without having to first convert everything into a standardized format, because they’re guaranteed to be in the same format. This makes everything incredibly easy to handle, and best of all, you can access them on-demand because the data is centralized, so you can get them instantly if you have admin-level authorization.

Elite performers who use multi-tenancy architectures save money, and accelerate data sharing. With savings like these, elite performers are absolutely worth their weight in gold.

Conclusion

Elite performers use multi-tenancy architectures, and while it’s true that using a multi-tenancy architecture won’t automatically turn you into an elite performer overnight, it will certainly set you on the right path.

Following good cloud computing practices is essential to speeding up development times and reducing costs. If you are interested in learning more, I strongly advise you to read Accelerate’s State of DevOps 2019, which is sponsored by Google, Deloitte, Pivotal, and other great tech companies.

One of the best ways to improve at your craft is to just copy the best. Is it original and creative? Not at all. But is it effective? Absolutely.

How To Get Your Company Hacked In Record Time

DISCLAIMER: This article is satire, and should not be taken seriously. Although the methods listed here will indeed get your company hacked in record time, this article was written to highlight security vulnerabilities that programmers should avoid. The methods used in this article are strictly for educational purposes, and should not be used for malicious reasons.

These days, it can be a struggle to get your company hacked. Between all of the new security patches that programmers push out on a day-to-day basis, and all of the fancy tools that you can use to statically analyze your code, getting hacked can be a chore.

Fortunately, I’ve learned some highly-effective methods that will get your company hacked in record time — even if it’s a multi-billion dollar company with tens of thousands of employees.

Method #1 – Publicly Leak Your Secrets Into Your Code

I know what you’re thinking. This is too obvious, isn’t it? It’ll get caught in code review, or by code analyzers. Fortunately for you, this method is really effective. How effective, you ask?

It’s so effective, that it caused a massive data privacy breach for over 57 million users for a certain multi-billion dollar company. Don’t worry if you’re not a multi-billion dollar company, because it works even for small companies and small projects, as evidenced by the over 100,000 leaked secrets on GitHub.

Here’s how to do it.

STEP 1 – Add this snippet anywhere in your company’s code. Remember to purposely ignore any password, secret, or key-vault/secret manager that your company might have.


    private String secretKey = "COMPANY_SECRET_KEY_HERE";

STEP 2 – Upload this code to your company’s preferred git repository.

Here, the path diverges based on how your company handles code reviews. If your company does a code review and notices the secret, then they will ask you to change your code. Fortunately, there is still a way to leak your company’s secret keys, even if they ask you to change your code.

The trick is to push the code with the secret, remove it, then push again. If your company approves the PR without squashing, then the leaked secret can be obtained by checking out the commit where you had added the secret, but before you had removed it.

As seen in the image above, by squashing, the commit with the leaked secret disappears and all of it turns into one commit. But if there’s no squashing, and the full history is retained, then you can access the commit history to checkout the commit with the leaked secret.

Method #2 – “Hide” Company Secrets In Slightly Less Plain Sight

This one is almost identical to method #1. The difference is that instead of blatantly being in plain sight, you’ve put a flimsy barrier in-between. For example, you can publish a client library with your secrets inside. Since it’s a client library, users seemingly shouldn’t be able to view the source code to extract your secrets.

Let’s illustrate the scenario. You are a consumer for this client library, and you see the following class structure for one of the client classes:


public class ClassWithSecret {
    private String secret;
    .... (other fields and methods)
}

Sadly, since this is a client library, programmers can’t modify the source code. And since it’s a private field, other programmers won’t be able to read it. Looks fool-proof, doesn’t it? With no way to access the source code, we can have the users use our clients without providing any configuration.

Luckily for us, it’s actually easy to completely side-step this issue, even if the field is private. All you have to remember is that “private” fields are not actually private. They’re a formality for saying, “don’t try to access me, and if you try to do it the normal way, it’ll fail”.

To access private fields, you just have to use reflection, which is a form of meta-programming that will let you modify the runtime behavior of a program.


public class ClassWithSecret {
    private String secret;
}

// Assume we have a ClassWithSecret instance
ClassWithSecret classWithSecret = ....; 

try { 
    Field field = classWithSecret.getClass().getDeclaredField("secret");
    field.setAccessible(true);
    Object secretVal = field.get(classWithSecret);
    System.out.println("The secret was " + secretVal.toString());
} catch (Exception ex) {
    ...
}

By using reflection, we can just manually make the field accessible, and it will be as if it never had a private modifier.

The same thing applies to constants, readonly fields, and more. With reflection, you can override basically any behavior, so you can’t rely solely on language features to protect secrets or to guarantee specific behavior when it comes to client libraries.

In a different vein, another way that programmers often “hide” their secrets is through environmental variables, and one of the worst ways to do it is by putting your secrets into your .bashrc file. Although some may disagree, using environmental variables as a way to hold your secrets is generally bad practice.

However, putting your environmental variables into your .bashrc is absolutely catastrophically bad practice. The worst part is that the majority of online tutorials will actually tell you to do this (there’s a reason why people use key vaults and secret managers!)

So why does this make you susceptible to getting hacked? Because by putting your environmental variables into your .bashrc, you are injecting your environmental variables into every single process that runs on your user. If even a single one of these processes, or one of their child processes dies, they will almost certainly dump or log their environmental variables in some way, and once this happens, your secrets are now visible in your logs/log files. Boom. Your secrets are now in plain-sight.

For a more detailed explanation, check out what Diogo Monica, former security lead of Docker, has to say about this.

Method #3 – Forget About Getting Hacked Online, Get Hacked In-Person

Having your data leaked by online hackers is one thing, but what about getting hacked in-person?

It might sound strange, after all, when was the last time someone got hacked in-person? But getting hacked in-person is actually quite easy. All you have to do is scribble down your company’s passwords onto a piece of paper, or maybe a Post-it note.

Remember that the oldest surviving piece of paper, the Missal of Silos, still exists even after a thousand years, while the oldest known electronic computer, the Eniac Computer, has only existed for a measly 75 years. Therefore, a piece of paper is more durable than a computer, which is why you should be storing your passwords on Post-it notes, rather than something like LastPass.

Now that your password is in the physical realm, where it can safely survive for another thousand years under the right conditions, all you have to do is record a video of yourself with your password in the background. If a video is too difficult, you can alternatively take a photo instead.

Once your password has been leaked to the internet, your company will have successfully been hacked.

Conclusion

Getting hacked can be hard, but as long as you follow this guide, it’s as easy as 1-2-3. Just remember that there are more ways to get hacked than the ones listed above.

If you’re finding that getting hacked is just too easy, you can up the difficulty by using key vaults/secret managers, squashing commits in pull requests, and using static-code analyzers to check for leaked secrets. And most important of all, check your logs for secrets. Secrets should never show up in your logs, because logs are generally publicly visible (ie; in a dashboard or error tracking service).

Now that you know how to get hacked in record time, I challenge you to flip this article on its head, and see how long you can go without getting hacked. Good luck!