Why You Should Value Function Over Form With Window Managers

Many developers like to spend an excessive amount of time ricing their Linux distros, usually with window managers (WMs) like Awesome, i3wm, and monsterwm. Of course, window managers are often chosen because of their aesthetic, but in many cases, you should value function over form with window managers.

If you’re not familiar with window managers, they essentially split your screen into discrete sections, and assign windows into those sections. You’re free to resize these at any time so that certain windows can occupy more space, and depending on which window manager you choose, you can also stack windows on top of each other, place them in a tabbed view, and switch between multiple different screens. This, combined with the use of workspaces, allows you to conveniently jump back and forth between different sets of windows in an instant.

A screen-capture of i3wm, a popular window manager.

The image above is a bare-basic i3wm setup, with Terminator as the terminal and the standard i3statusbar at the bottom. Aside from modifying the status bar and setting Terminator to use solarized-dark colors, this is an un-riced setup. Typically, ricers will begin the ricing process by using Conky.


The Art of Displaying Tons of Irrelevant Information With Conky


Many ricers love to display an excessive amount of irrelevant information on their desktop as a way to make their screen look fancy and technical. The more scary and overwhelming it looks, the better. Usually, this is done by using Conky, a software whose purpose is to display information on your desktop. At first, this doesn’t sound particularly interesting, but if you’ve ever looked at a riced desktop, then you know first-hand how impressive Conky can make your desktop look.

Conky in action.

I know what you’re thinking — you’re probably thinking, “That’s amazing! I want my desktop to look like that too! Time to close this article and look up a ricing guide“. It’s a trap. Speaking from personal experience, ricing is a massive time-sink. Imagine you’re trying to make a website look fancy by using CSS. Now, swap out the CSS for Lua.

The great thing about making Conky scripts in Lua is that there are no selectors, meaning you need to constantly copy and paste the styling over and over again. For example,  suppose you have yellow text in Times New Roman, font 12. If you want to make ten separate sets of words with that exact styling, you will have to simply copy and paste the exact same styling ten times.

Here’s an example from the Conky setup listed above, which you can find on Conky’s official GitHub page at https://github.com/xyphanajay/conky/blob/master/.conkyrc1

conky.text = [[
${font DejaVu Sans Mono:size=14}${alignc}${time %I:%M:%S}
${font Impact:size=10}${alignc}${time %A, %B %e, %Y}
${font Entopia:size=12}${color orange}CALENDAR ${hr 2}$color
${font DejaVu Sans Mono:size=9}${execpi 1800 DA=`date +%_d`; cal | sed s/"\(^\|[^0-9]\)$DA"'\b'/'\1${color orange}'"$DA"'$color'/}
${font Entopia:bold:size=12}${color red}FILE SYSTEM ${hr 2}${font Noto sans:size=8}
#${offset 4}${color}dev ${alignr}FREE     USED
${offset 4}${color}root (${fs_type /}) ${color yellow}${alignr}${fs_free /} ${fs_used /}
${offset 4}${color yellow}${fs_size /} ${color}${fs_bar 4 /}
${offset 4}${color FFFDE2}home (${fs_type /home}) ${color yellow}${alignr}${fs_free /home/} ${fs_used /home/}
${offset 4}${color yellow}${fs_size /home/} $color${fs_bar 4 /home/}
${offset 4}${color FFFDE2}sda5 (${fs_type /run/media/senpai/6EC832DEC832A3ED/}) ${color yellow}${alignr}${fs_free /run/media/senpai/6EC832DEC832A3ED/} ${fs_used /run/media/senpai/6EC832DEC832A3ED/}
${offset 4}${color yellow}${fs_size /run/media/senpai/6EC832DEC832A3ED/} $color${fs_bar 4 /run/media/senpai/6EC832DEC832A3ED/}
${offset 4}${color FFFDE2}sda6 (${fs_type /run/media/senpai/6EC832DEC832A3ED/}) ${color yellow}${alignr}${fs_free /run/media/senpai/C208F88708F87BAB/} ${fs_used /run/media/senpai/C208F88708F87BAB/}
${offset 4}${color yellow}${fs_size /run/media/senpai/C208F88708F87BAB/} $color${fs_bar 4 /run/media/senpai/C208F88708F87BAB/}
${font Entopia:bold:size=12}${color green}CPU ${hr 2}
${offset 4}${color black}${cpugraph F600AA 5000a0}
${offset 4}${font DejaVu Sans Mono:size=9}${color white}CPU: $cpu% ${color red}${cpubar 6}
${font Entopia:bold:size=12}${color 00FFD0}Network ${hr 2}  
${color black}${downspeedgraph enp8s0 32,80 ff0000 0000ff}${color black}${upspeedgraph enp8s0 32,80 0000ff ff0000}
$color${font DejaVu Sans Mono:size=8}▼ ${downspeed enp8s0}${alignc}${color green} IPv6${alignr}${color}▲ ${upspeed enp8s0}
${color black}${downspeedgraph wlp10s0f0 32,80 ff0000 0000ff}${color black}${upspeedgraph wlp10s0f0 32,80 0000ff ff0000}
$color${font DejaVu Sans Mono:size=8} ▼ ${downspeed wlp10s0f0}${alignc}${color orange} ${wireless_essid wlp10s0f0}${alignr}${color}▲ ${upspeed wlp10s0f0}
${font Entopia:bold:size=12}${color F600AA}Disk I/O ${hr 2}
${alignc}${font}${color white}SSD vs HDD $mpd_name
${color black}${diskiograph /dev/sda 32,80 a0af00 00110f}${diskiograph /dev/sdb 32,80 f0000f 0f0f00}
${font DejaVu Sans Mono:size=8}${color white}   ${diskio /dev/sda}${alignr}${diskio /dev/sdb}
]]

It’s not exactly the prettiest.

But there’s more. With Conky, your desktop doesn’t act like a normal web page, it acts more like a canvas where everything is practically absolutely positioned. Finally, because you’ll likely be too lazy to learn Lua and Conky’s many nuances just for the sake of ricing, you’ll likely resort to to copying and pasting snippets from all over the internet, and with each copy and pasted snippet you add, your configuration becomes even more hacky and unreadable.


If You Value Your Screen Real Estate, Don’t Rice Your Desktop


Ricing your desktop almost always results in massive amounts of frustration, wasted time, and tons of exerted effort. The fun of showing off your setup only lasts for a few brief moments. After that, your riced desktop is basically useless or obstructive, because you’ll either have your Conky setup overlay the desktop, meaning it will be blocked out when you have a window over it, or overlay all of your windows, which means it’s permanently visible.

If you choose to have Conky only on your desktop, it will be 100% covered by whatever window you have open, as a window manager will consume as much desktop space as possible. Since you’ll almost always have tons of terminals or programs running on most of your workspaces, you’ll almost never see your desktop. Once the novelty wears off, your desktop will feel as ordinary as before. If you decide to automate everything, which you likely will do, you’ll have programs automatically opened on all your separate workspaces anyway.

If you choose to have Conky overlay all of your windows, you will quickly realize how annoying it is to have all of that extraneous information plastered over your screen at all times. In most cases, it’s distracting and reduces the amount of visible space on your desktop. If you choose this option, you basically lose anywhere from 20-40% of your screen’s real estate. Assuming that the code you’re working on is limited to 80 characters per line, you won’t be able to split your windows vertically because that would reduce each window’s real estate to only ~30% of the screen, so 80 character lines won’t even fit on your screen.

 

It might be cool to have something like the Conky animation above on your desktop, but if it’s eating up tons of precious desktop real estate, then it’s not worth it. With an animation like this, you won’t be able to read code, debuggers, or terminal/console logs without having your window in full screen. It goes without saying that there’s no difference between using a window manager and a standard desktop environment like GNOME if you’re only going to view things in full-screen mode..

Ricing your desktop might look pretty, but in my experience, it has always turned out to be a massive waste of time, either because the novelty wore off or because it eventually became annoying. The most efficient way to use a window manager is to use it for what it’s made for — maximizing screen real estate. By using 100% of your screen, leaving no blank spots, you’ll be able to maximize the amount of information you can view at once.

Incidentally, by ricing your desktop, you’ll likely either reduce the amount of information you can view at once, or you’ll end up plastering large amounts of useless information on your desktop, both of which will end up reducing your productivity. Should you decide to use a window manager, remember to value function over form, because at the end of the day, computers are tools, not decorations.

Advertisements

Front-End Color Schemes for the Blind, Colorblind, and Creatively Challenged

Picking colors for a website or GUI can be difficult. However, for blind, colorblind, and creatively challenged people, choosing colors can be nearly impossible. Most people will resort to using vague descriptions, like “Blue is a cool color”, or “Use green to describe liveliness and nature!”.

The fact is, they’re not wrong. But this method of choosing colors is not effective for people who physically cannot see color, or for people who severely lack creativity or artistic ability. For these people, we need a new method; one that can scientifically and objectively determine color combinations that might be visually appealing.

In this article, I will be using pictures. I realize the irony of using pictures when the target audience for this blog post will be people who are unable to see or effectively understand color. However,  I still want to make sure this post is inclusive of non-colorblind people who are looking for a scientific way to choose colors.

In one go, we’ll be covering both of these scenarios at once :

  1. You already have some sort of photo or theme that you are basing the rest of the color scheme around.
  2. .You don’t have any material, meaning you have a white slate. You haven’t chosen a color scheme, and you need help choosing a color.

If You Already Have A Photo or Theme


If you already have a photo, then the first step is to figure out what the dominant color is. This should be fairly simple, but if your photo has multiple varying colors, then you will need to use the average color of the photo.

The average color of the photo is defined by this simple algorithm :

totalRed = 0
totalGreen = 0
totalBlue = 0
numPixels = 0
for every pixel in the image
    totalRed += red in pixel
    totalGreen += green in pixel
    totalBlue += blue in pixel
    numPixels += 1
averageColor = [totalRed / numPixels,
                totalGreen / numPixels,
                totalBlue / numPixels]

Now that you have the average color, we will be basing our entire color scheme around saturation variances of that color.

For example, let’s take a look at cover of my machine learning book, The Mostly Mathless Guide To TensorFlow.

Cover

If you get the average color of this book cover, you’ll find that the average color is a gray-ish blue.

2018-09-14_23-01

Although, in this cover, you can easily tell that the dominant color is blue. When you are easily able to determine the dominant color by a quick glance, there is no need to compute the average color of a photo. However, if you cannot see color, or if the dominant color cannot be easily determined, then an average color algorithm can be very helpful.

The only caveat is that you cannot use the average color if your photo or theme is rainbow or has too many varying colors. For example, here is a rainbow photo.

download

And if we look at the average color, we will get a color like this :

2018-09-14_23-05

In general, if you mix together all of the colors, you will get a murky brown. So it’s no surprise that getting the average color of a rainbow photo will give you an ugly brown-red-ish color.


But What If I Don’t Have a Photo?


If you don’t have a photo, don’t worry! You can determine an ideal color by using the following guidelines from one of Cornell’s art classes, which you can find here.

In general, colors can have both a positive and a negative effect. The most obvious one is red.

Red is often used to evoke feelings of power, strength, energy, and vitality. However, red can also symbolize anger, blood, and violence. Usually red is too aggressive of a color to use as a dominant color, and if it is used as a dominant color, it should be paired with copious amounts of white. Red is often a very good color to use, provided you don’t over-use it.

Blue represents calmness, tranquility, elegance, coolness, and knowledge. On the flip side, it could also represent depression and sadness. Blue is a very safe color to use in your color scheme. In fact, if you are developing software, blue is an excellent color to choose because it disrupts the sleep cycle of the user, causing them to use your product for longer periods of time without getting tired (according to Scientific American).

Yellow is upbeat, cheerful, and optimistic. It’s extremely bright, and will catch anyone’s eye. The downside is that if you use too much yellow, your customers will be annoyed because yellow has very high contrast. Other than that, there are basically no downsides to yellow, and no other wrong ways to misinterpret yellow.

Red, blue, and yellow are the ideal colors, and the remaining colors are mostly niche colors. A study of the top 100 company logos shows that 38% of them have red, 37% have blue, 24% have yellow, and the rest are either black, white, or have negligible amounts of the other colors.

Green represents life, vitality, growth, and money. Especially money, since the dollar bill and dollar sign are green. Green is also associated with toxicity, but realistically, green is a very hard color to misinterpret. Aside from companies that are focused on finance, agriculture, or the environment, there is n’t too much incentive to use green.

The remaining colors are extraordinarily niche.

Pink is for companies that want to focus on women, especially for cosmetics or clothes. Other than that, don’t use pink.

Purple is for games, fantasy, royalty, and dreaming/sleeping. It’s a very niche and rare color, although Yahoo! and Twitch both use purple for their color schemes.

Orange is a friendly color, and suggests accessibility and openness. Use orange when your company is related to oranges, for obvious reasons. However, the negative side is that orange can be associated with ordinariness and being over-accessible.

In general, if you’re unsure of which color to pick, you really can’t go wrong with one of red, blue, or yellow. 


White and Black


Before we dive into discussing about colors that compliment and match each other, we first need to talk about white and black. Use white and black generously with any color you choose.

In general, prefer white over black unless you are going for a night/dark theme. White will go with any color you choose, except yellow. White and yellow is a nightmare color scheme. When using yellow, you must have contrast, otherwise the yellow will blend in with the white, making your website or logo hard to read.

The other forbidden combination is red with black as the dominant color. There are very few scenarios that red and black will look good in. Unless you are specifically going for a gothic, dark, vampiric, scary, or satanic theme, I strongly advise against choosing red and black. This is why all four books of the Twilight vampire series are in black and red.

af9334ce61da3c9abb1257201211060f--twilight-new-moon-twilight-series


Tints and Shades


Let’s first talk about tints and shades of a color. This is the easiest way to guarantee that you won’t mess up a color combination — by using the exact same color, but changing its saturation or lightness. It gives a very comfortable feel, and is the easiest design to do.  This is a principle that follows in line with material design.

Pick literally any color you want, and then use a (usually) darker version of it to add variety to your color scheme, although a lighter version is also acceptable.

Here are some examples :

Blue
2018-09-14_22-10

Red
2018-09-14_22-11

Yellow
2018-09-14_22-12


Complementary Colors


Complementary colors are polar opposites of each other. You use complementary colors when you want to create extremely high contrast. On a web page, avoid using too many complementary colors together, or your users will feel uncomfortable due to the high contrast.

All algorithms can be found online. A complementary color is just two colors that, when combined together, form black or white.

Purple – Complement, Yellow

2018-09-14_22-21

Green – Complement, Red

2018-09-14_22-22

Blue – Complement, Orange

2018-09-14_22-22_1


Analogous Colors


Analogous colors are colors that are next to each other on the color wheel. Analogous colors are very comfortable, and give a feeling of calmness and tranquility. They look beautiful when placed next to each other for this reason.

However, from experience, analogous colors work best when you use light variations of the colors, rather than dark variations. When going for a calm feeling, dark colors give too much expression and contrast, which is generally the opposite of what you want.

To generate a complementary color, simply pick a color, and then the two colors next to it on the color wheel.

Light Red-Orange – Analogous Colors are Light Orange and Pink-Red 

2018-09-14_22-27.png

Light Green – Analogous Colors Are Green-Teal and Green-Yellow

2018-09-14_22-28

Lavender (Blue) – Analogous Colors Are Royal Purple and Bright Teal

2018-09-14_22-29

Bright Yellow – Analogous Colors Are Green-Yellow and Orange

2018-09-14_22-31.png

In general, analogous colors are harmonious, as you can (or maybe can’t) see.


Split Complementary Colors


If you want to use complementary colors, but you feel that the contrast is too strong, use split complementary colors instead. A split complementary color is formed when you take a color and find its complement. Then, instead of choosing the complementary color, you take the two analogous colors for that complementary color instead.

This gives you a similar high-contrast effect as a complementary color, but it is harder to mess up because the colors will have significantly less contrast. On the color wheel, this will usually look like a very thin isosceles triangle.

Bright Yellow – Split Complementary Colors Are Hot Pink and Lavender

2018-09-14_22-35

Green – Split Complementary Colors Are Red-Orange and Hot Pink

2018-09-14_22-36

Royal Blue – Split Complementary Colors Are Mustard Yellow and Orange

2018-09-14_22-37

Red – Split Complementary Colors Are Green and Blue 

2018-09-14_22-39


Triad Colors


Triad colors are what you would get if you took a color wheel and made an equilateral triangle. The colors are perfectly spaced from each other, which gives you a vibrant feel. Unfortunately, it does not give you a harmonious feel, so use one color dominantly, and the other two as supporting colors.

Use the other two colors sparingly for best effect, because these colors will generally be quite close to the split complementary colors. The only difference is that they are more spaced apart, which gives them less contrast.

Blue – Triadic Colors Are Yellow and Red

2018-09-14_22-42.png

Green – Triadic Colors Are Orange and Purple

2018-09-14_22-43

Since triadic colors are equidistantly spaced, there are essentially only two examples. If you pick yellow as your dominant colors, you will get blue and red. If you choose red as the dominant color, you will get yellow and blue.


Tetradic Colors


You get tetradic colors when you pick two colors next to each other on the color wheel, and then pick the two complementary colors of those two colors. You will end up with a rectangle. The effect is that you get high-contrast, but with a variety in colors, as you now have four colors instead of two.

Because of the high contrast, you should generally pick one or two colors (assuming the two colors are NOT complementary colors) and using the other two colors sparingly.

Having a tetradic color scheme is significantly harder and more complex than the other color schemes. As a result, you may find it difficult to get a decent ratio for each color, causing either an ugly or overly high-contrast design.

Purple (Tetradic Colors Are Yellow, Green, and Red)

2018-09-14_22-50

Red (Tetradic Colors Are Green, Blue, and Orange)

2018-09-14_22-51

Orange (Tetradic Colors Are Blue, Purple, and Yellow)

2018-09-14_22-55


Conclusion


We’ve gone over many different color schemes, and the one that works best for you may vary. In general, the easiest color scheme will be analogous colors, split complementary colors, or easiest of all, using tints and shades of a specific color. That’s not to say that the other color palettes are bad — rather, they are exceptionally useful. However, if you are an absolute beginner at picking colors, go for one of the three choices listed above.

If you need something more flexible, the other color palettes are there for you. As with anything involving creativity, the best way to discover what works and what doesn’t is to try it out. Blindly choosing random colors is a poor way to design graphics, but with this approach, it is easy to create multiple designs and have other people critique them.

So what are you waiting for? Get out there and start playing around with your new-found color powers!

Simply Explained : Multithreading

John and Mary both walk into work. Let’s consider both John and Mary to be a “thread”. We’ll define a “thread” to simply be a sequence of consecutive actions. In this case, let’s say that John is a “thread”, and Mary is also a “thread”, because they can both execute actions in parallel. If John is working on a spreadsheet, Mary does not have to wait for John to finish before she can do her own tasks. They are both independent entities.

Now, John loves donuts. Fortunately for John, his company always stocks donuts in the company fridge. For John, his thought process is quite simple, and looks like this :

if John sees at least one donut in the fridge
    then John washes his hands
    then John grabs a donut and eats it

And this is fine and dandy. When John is alone, this course of logic always works. He sees a donut in the fridge, he washes his hands, and he eats it. It always works without fail.

However, one day, Mary walks into the room. And Mary also likes donuts. When John looks into the fridge and sees that there is exactly one donut left, he hurriedly goes to wash his hands. However, when he returns, he finds that the donut is gone! But that shouldn’t have been possible. John had checked that at least one donut existed before he decided to wash his hands to eat it!

So what happened? It turns out that Mary had taken the donut in the time that John had spent washing his hands. This is the problem with multithreading, which is the act of programming with more than one thread.

 


The Problem with Multithreading


While it may be true that John’s logic always succeeds when he is alone (single-threaded), the same cannot be said when Mary is with him (multithreaded). This is because when objects are shared between multiple threads, multiple different threads are able to modify the object at the same time. In this case, the fridge is the object, and both John and Mary are able to modify the contents of the fridge at any time.

This is a huge problem, because now, functions that had been unit tested or proven to work on a single thread no longer work when multithreaded. As shown above, doing secure checks like putting in defensive if statements is entirely ineffective, because the object can be modified right after your if statement is completed due to the fact that multithreading allows everything to be done in parallel.

So how can John prevent Mary from modifying the contents of the fridge until he is done using it? The answer is fairly simple — when John accesses the fridge, he needs to make sure that he is the only one who has control of it until he is done whatever he is doing. There are multiple ways to do this, but the most common is to use something called a “lock”.

This new multithreaded system with locks has three simple rules :

1. Whoever controls the lock will have sole control over the object for as long as they possess the lock.
2. An object should only have one lock.
3. Any thread waiting on a locked object can either throw an exception or wait for it to be unlocked. In which case, it will be passed by some priority order (usually first come first serve, like a queue).

Let’s observe why these three rules are true.

 


Rule Number One


“Whoever controls the lock will have sole control over the object for as long as they possess the lock.”

The first rule is true because that is how a lock is defined. But for a more intuitive approach, you can think of the lock as “binding” itself to its owner. As long as the owner of the lock does not relinquish control, the object will forever be locked. This also means that if a thread controlling a locked object gets stuck in an infinite loop, that locked object will be locked forever.

Of course, while you could manually unlock the object, there is no way to determine if an object will be locked forever, because you can’t (in general) determine whether or not a thread is stuck in an infinite loop (see Halting problem).

 


Rule Number Two


“An object should only have one lock.”

Suppose an object could have two separate locks that give complete access over the object. That means that two different threads can access the object at the same time. Unfortunately, this is essentially no different than not having a lock at all, because now the object can be freely edited at any time by anyone. We are back to the exact same problem that we initially started with!

This means that an object must have one and only one lock, and only one owner for that lock.

 


Rule Number Three


“Any thread waiting on a locked object can either throw an exception or wait for it to be unlocked. In which case, it will be passed by some priority order (usually first come first serve, like a queue).”

In general, it is bad practice to have your code throw an exception when it attempts to access a locked object. The more conventional approach is to create a queue for that locked object, like a wait list. This is so that if three threads wanted to access an object, they would access the object in the order that they called the lock.

For example, the execution order might look like this :


1. Thread 1 calls for the object. It is not locked, so Thread 1 gains control of the lock.
2. Thread 2 calls for the object. It is locked. Thread 2 is put on the queue (wait list)
3. Thread 1 performs some operation using the object.
4. Thread 1 relinquishes control of the lock.
5. Thread 3 calls for the object. It is unlocked at this very exact moment, but because there is a queue for this object, the lock goes to the first element in the queue, which is Thread 2. Thread 3 is now put on the queue.
6. Thread 2 performs some operation using the object.
7. Thread 2 relinquishes control of the lock.
8. Thread 3 automatically gains control of the lock, as Thread 3 is next on the queue.
9. Thread 3 performs some operation using the object.
10. Thread 3 relinquishes control of the lock.


As you can see, when multiple threads want access to a locked object, they are accessed in first-come-first-serve order. However, this does not necessarily mean that Thread 2 just has to twiddle its thumb and do nothing. Thread 2 does not have to be blocked while it waits — it can do other computations while it waits for its access to the locked object.

This is important for when the desired operation could take a long time. For example, imagine if Thread 1 wanted access to an object controlling the webcam for your computer. Thread 1 could be using this object for a long time, and if Thread 2 and Thread 3 both sat idly doing nothing, we would experience a severe loss in performance.

 


Conclusion


In this article, you’ve learned that multithreading is difficult because if two threads access an object at the same time, there’s no guarantee that the object will give you the desired behavior. To fix this problem, you simply have to use a lock, which will prevent other threads from accessing the object. When a thread is done performing operations with a locked object, it relinquishes the lock, and the next thread gets control of the lock.

In the next Simply Explained post, I’ll be talking about Futures, which is a programming construct used to obtain a value that does not yet exist (for example, because the object you wanted to use to get that value was locked). It allows you to write non-blocking code, because after you create a Future, your thread can do other tasks until the Future makes a callback, telling you that the value now exists.

Happy coding, and enjoy your new-found multithreading powers.

Singletons are Glorified Globals

You’re working on a snippet of code, and out of the blue, you happen to need a class of which you should only have one instance of, and which needs to be referenced by other classes.

Sounds like a job for a singleton! Or is it?


You want to use a singleton to store state


Realistically speaking, if you’re using a singleton solely to store state, you are doing it wrong. What you’ve made isn’t a singleton — what you’ve made is a bunch of glorified globals. Using singletons to store state seems appealing at first, because you want to avoid using a global and every programmer knows that globals are evil. But using a singleton in this case is just abstracting the globals one layer back so that you can feel happy about your code not having globals.

Furthermore, having globals will make your code unpredictable and bug-prone. This becomes exponentially worse as the number of dependencies in your singleton increases. If you are using a singleton because you need its global properties, it’s because you are not taking advantage of dependency injection (DI) / inversion of control (IoC). You should be passing the state around with IoC, not by creating a giant global and passing around the global fields.


Singletons make your code painfully difficult to test and debug


Singletons can’t be easily tested when they are integrated with other classes. For every class that uses a singleton, you will have to manually mock the singleton and have it return the desired test values. On top of this, singletons are notoriously difficult to debug when multi-threading is involved. Because your singleton is a major dependence for several classes, not only is your code tightly coupled, but you will also suffer from hard-to-find multi-threading bugs due to the uncertain nature of globals.


You want to use a singleton to avoid repeating an expensive IO action


Occasionally, you may be tempted to use a singleton to perform an expensive IO action exactly once, and then store the results. An incredibly common example of this is using a singleton for a database connection. But doing so in addition to having multi-threading will cause massive headaches unless your connection is guaranteed to be thread-safe.

In general, database concurrency will not be easy to implement if you are using a singleton for the connection. What you really want is a database connection pool. By caching the connection, you avoid having to repeatedly close and open new connections, which is expensive. As an added bonus, if you use a connection pool, you simply won’t need to use a singleton.


You want to pass data around, but you don’t need to modify the data


You just need a data transfer object. No need for a singleton here. And worse, giving your singleton access to methods that can modify the data makes your singleton a god object. It knows how to do everything, knows all of the implementation details, and is likely coupled to basically everything, violating almost every software development principle.

Worse yet, using a singleton to provide context is a fatal mistake. If a singleton provides context to all the other classes, then that means every class that interacts with the singleton theoretically has access to all the states/contexts in your program.


You are using a singleton to create a logger


Actually, this is really one of the few acceptable use for a singleton. Why? Because a logger does not pass around data to other classes, provides no context, and there is generally minimal coupling between the logger and the classes that require the logger. All the logger needs to know is that given some log request or string, it should output the log as a file or to the console.

As you can see, a logger will provide nothing to classes that require it — there is nothing to grab from the logger. Therefore, it’s impossible to use the logger as a glorified global container. And best of all, loggers are incredibly easy to test due to how simple they are. These properties make loggers an excellent choice for a singleton.

In the future, you’ll probably find a scenario where you’re considering using a singleton for any of the reasons above. But hopefully, you’ll now realize that singletons are not the answer — inversion of control is.

Checking Whether or Not an Ad Is a Tide Ad with Keras And NumPy

Note : If you’re interested in machine learning, you can get a copy of my E-book, “The Mostly Mathless Guide to TensorFlow Machine Learning” by clicking HERE

There’s a bunch of kids running around with a Coca-Cola in their hands. But hold
on — look at their clothes! They’re so clean and white. Too clean, almost. Could this be a Tide ad?

Machine learning to the rescue! In this article, I’ll be showing you how to use TensorFlow, a machine learning library, to predict whether or not an ad is a Tide ad.

Prerequisites


This tutorial will be using Linux. You can probably do it on Windows too, but you may have to change some things. Here are the things you will need :

1. Python 3
2. TensorFlow (pip3 install tensorflow)
3. Keras (pip3 install keras)
4. ffmpeg (sudo apt-get install ffmpeg)
5. h5py (pip install h5py)
6. HDF5 (sudo apt-get install libhdf5-serial-dev)
7. Pillow (pip3 install pillow)
8. NumPy (pip3 install numpy)

Although VirtualEnv is not required, it is suggested that you use VirtualEnv to prevent any conflicts / version mistakes between Python 2 and Python 3.

Also, you can find all the code and bash scripts here : https://github.com/HenryDangPRG/TideAdIdentifier

Getting Started


First, we need to describe what our neural network will do. In this case, our neural network will will take one image as input, and tell you whether or not that image belongs to a Tide ad or not. Using ffmpeg, we can split a video into its frames to input an entire video into the neural network as well, and if over 50% of the frames in a video are classified as “Tide ads”, then we will consider it to be a Tide ad.

Next, we need data for our neural network to train on. The data will be a large set of .png images that we will get from slicing a video into individual pictures. I will not provide the videos as a download here, so you will need to find the 1 minute 45 second video of all the SuperBowl Tide ads, as well as 5 minutes worth of non-Tide ads. Also, the two videos should have the same size dimensions so that the images that come out are all the same size.

Once you obtain these two videos, convert them into .avi format and use ffmpeg to split them into its constituent frames. I’ve created a simple Bash script that will do the splitting process automatically for you, as long as you name the Tide ad video “tide.avi” and the non-Tide ad video “non_tide.avi”.

You can find the script here : https://github.com/HenryDangPRG/TideAdIdentifier/blob/master/generate_data.sh

The script above will take the two videos, and split 5 frames per second of the video, each frame being 512 x 288, into two separate folders. You can choose to do this on your own as well, but in this tutorial, as a convention, all Tide ad pictures will be in a directory called “tide_ads”, and all non-Tide ad pictures will be in a directory called “non_tide_ads”.

We’ll have to do the same with the test data, and the prediction data, and these are the bash scripts for those:

https://github.com/HenryDangPRG/TideAdIdentifier/blob/master/generate_test.sh
https://github.com/HenryDangPRG/TideAdIdentifier/blob/master/generate_predictions.sh

For the predictions, you can input any video format as the argument for the Bash script, but .avi is suggested for consistency.

NOTE : Remember to use chmod +x BASH_SCRIPT_NAME.sh on all of the Bash scripts so that you can execute them!

Creating A Convolutional Neural Network


Although this analogy is not perfect, you can think of a neural network as a group of students in a classroom who are all shouting out an answer. In this classroom, students are trying to determine whether or not a single image is from a Tide ad. Some students have a louder voice, so their “vote” for an answer counts more. A neural network can be thought of as thousands of students, all shouting different answers. The loudest answer gets passed to the next classroom, and those students discuss the answer (with their answers being modified by the previous classroom’s answer, perhaps by peer pressure), until we reach the very last classroom, where the loudest answer is the answer for the neural network.

201802081619561000

A convolutional neural network (CNN) is similar in that the students are still looking at an image, but they are only looking at a piece of the image. When they finish analyzing this image, they pass it to the next classroom, but the next classroom gets an even tinier piece of the original image. And so on and so forth, with each next classroom’s image getting smaller and smaller. When they’re done, a vote is outputted.

This is different in that in a normal neural network, the students vote for the entire image at once, but in a CNN, they each only vote for a piece of the image.

Now that you know the basics, let’s jump into the code.

Making The Magic Happen


First, let’s define the parameters for our CNN.

from keras.preprocessing.image import ImageDataGenerator
from keras.models import Sequential
from keras.layers import Conv2D, MaxPooling2D
from keras.layers import Activation, Dropout, Flatten, Dense
from keras import backend as K

# Could use larger dimensions, but will make training
# times much much longer
img_width, img_height = (128, 72)

train_dir = 'train_data'
test_dir = 'test_data'

num_train_samples = 4000
num_test_samples = 2000
epochs = 20
batch_size = 8

Each image will be shrunk to 128 x 72 pixels. Although we could go smaller, we would risk losing too much information. Larger could be better, but the larger the image dimensions are, the longer it will take for the CNN to train.

Next, we’ll have to specify in our CNN whether the color channels are first or last. Usually, they are first, though (at least, for png files). Note that an image could have just one color channel if it was grayscale, but in this case, we will only be using color images. We have to specify these because Keras will reduce our images into NumPy arrays, and the ordering matters.

# If data is formatted to have the channels first,
# then stick the RGB channels in front, else put them
# at the end.
if K.image_data_format() == 'channels_first':
    input_shape = (3, img_width, img_height)
else:
    input_shape = (img_width, img_height, 3)

Now that our CNN knows how many color channels (3 means RGB) our pictures have, as well as the dimensions of the image, we can apply three layers of convolution -> RELu -> max pooling.

The convolution step is essentially taking a tiny matrix and multiplying it to sections of the original matrix. When this is done, the result of each multiplication is recorded into a new matrix. The result is that we get a smaller matrix, which is called a feature map, containing unique features of the image that the machine can then process.

Next, we will need to apply ReLu, or Rectified Linear Unit. ReLu is incredibly simple. If the number is negative, make it zero, otherwise, leave it alone. This is because for a neural network, a negative value doesn’t really offer much information in the context of an image.

Imagine if we were detecting whether or not an image has dark blue lines. A value of zero in the feature map just means that there are no dark blue lines, while a positive value means that there might be a dark blue line. If so, then a negative value has no useful meaning, and can just be set to zero. ReLu also makes computations easier because zeroes are incredibly easy to deal with.

Finally, we apply max pooling, which takes subsections of a matrix and extracts the highest value from that subsection. This will shrink the matrix,reducing computation times, as well as giving us the most important parts of the matrix.

For our Tide ad predictor, we’ll use three layers of convolution, ReLu, and max pooling. When we’re done, we’ll put it all together with a fully connected layer.

# First convolution layer
model = Sequential()
model.add(Conv2D(32, (3, 3), input_shape=input_shape))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))

# Second convolution layer
model.add(Conv2D(32, (3, 3)))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))

# Third convolution layer
model.add(Conv2D(64, (3, 3)))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))

# Convolution is done, so make the fully connected layer
model.add(Flatten())
model.add(Dense(64))
model.add(Activation('relu'))

Then, we apply dropout. Dropout, in our classroom analogy, is like duct taping certain students so that they cannot vote. By using dropout, every student needs to learn to give the right answer, rather than depending on the “smart” students to give the right answers. This gives us an extra layer of redundancy and prevents the loudest students from always overpowering the quiet students.

201802081702131000

In the context of neural networks, it stops overfitting, which is when the neural network becomes too accustomed to the training data and fails to generalize for data that it hasn’t seen before. This can be caused when certain neurons have their weights modified too much by the previous layer’s weights, especially when one neuron has an abnormally high weight. Dropout makes it so that the influence of any one neuron (or group of neurons) is significantly reduced.

From here, everything is pretty self-explanatory. We create some extra data points by slightly modifying the original images, and then plug those into our model. Finally, we train the data and save the completed CNN model.

# perform random transformations so that the
# data is more varied
train_datagen = ImageDataGenerator(
    rescale=1. / 255,
    shear_range=0.2,
    zoom_range=0.2,
    horizontal_flip=True)

test_datagen = ImageDataGenerator(rescale=1. / 255)

# make extra training data by modifying original training images
train_generator = train_datagen.flow_from_directory(
    train_dir,
    target_size=(img_width, img_height),
    batch_size=batch_size,
    class_mode='binary')

# make extra test data by modifying original test images
validation_generator = test_datagen.flow_from_directory(
    test_dir,
    target_size=(img_width, img_height),
    batch_size=batch_size,
    class_mode='binary')

# Train the CNN
model.fit_generator(
    train_generator,
    steps_per_epoch=num_train_samples // batch_size,
    epochs=epochs,
    validation_data=validation_generator,
    validation_steps=num_test_samples // batch_size)

# Saved CNN model for use with predictions
model.save('saved_model.h5')

If you don’t want to spend the time finding videos and training the CNN, the trained model is available in the GitHub repository here : https://github.com/HenryDangPRG/TideAdIdentifier

Predicting Whether a Video Is a Tide Ad


The prediction part is fairly trivial. All we need to do is take a video, turn it into image frames, turn those image frames into NumPy arrays, and then feed them into the trained CNN.

The video splitting part is done by generate_predictions.sh. All this Python code below does is feed those image frames into the CNN. If more than half of the frames aren’t Tide ads, then we can conclude that the video probably isn’t a Tide ad. (note that in the prediction array, a value of 1 means it is NOT a Tide ad, and a value of 0 means that it IS a tide ad)

import numpy as np
import subprocess
from keras.preprocessing import image
from keras.models import load_model

def img_to_array(file_name):
    loaded_img = image.load_img(file_name, target_size=(img_width, img_height))
    img_array = image.img_to_array(loaded_img)
    img_array = np.expand_dims(img_array, axis=0)
    return img_array

if __name__ == "__main__":
    directory = "predictions/"
    num_images = int(subprocess.getoutput("ls predictions | wc -l"))
    img_width, img_height = (128, 72)

    # load the saved model, and use it for prediction
    model = load_model("saved_model.h5")
    images = []

    for i in range(1, num_images+1):
        # Need up to 6 leading zeroes for formatting
        i_ = "{:0>7}".format(i)
        next_img = img_to_array(directory + "prediction-" + str(i_) + ".png")
        images.append(next_img)  
    images = np.vstack(images) 
    prediction = model.predict(images) 
    percent_tide = sum(prediction)[0] * 100 / num_images

    print("There is a " + str(percent_tide) + "% chance this is not a Tide ad.")

    # If more than half the images are NOT Tide ads
    if(sum(prediction) >= num_images / 2):
        print("It can be concluded that this is NOT a Tide ad!")
    else:
        print("It can be concluded that this IS a Tide ad!")

Conclusion


How well does this work? Well, not too well. This neural network uses a pitifully small data set, and was trained for very few epochs, so it makes sense that the results are not particularly accurate.

Inputting in Pepsi’s 2018 SuperBowl ad gives the following result :

There is a 45.833333333333336% chance this is not a Tide ad.
It can be concluded that this IS a Tide ad!

And inputting in Skittle’s 2018 SuperBowl ad gives this result :

There is a 53.4675615212528% chance this is not a Tide ad.
It can be concluded that this is NOT a Tide ad!

So, does this make every SuperBowl ad a Tide ad? Our neural network seems to think it does. Almost.

Everything in Haskell Is a Thunk

There are a lot of misconceptions about Haskell.

Some people think everything in Haskell is a function. Others think that Haskell is just an implementation of Church’s lambda calculus. But both of these are false.

In reality, everything in Haskell is a thunk.

A thunk is a value that has not been evaluated yet. If everything in Haskell is a thunk, then that means that nothing in Haskell is evaluated unless it has to be evaluated.

From a non-functional programmer’s perspective, a thunk may initially seem to be a useless feature. To understand why thunks are useful, consider the following Python code.

x = 5 / 0

Traceback (most recent call last):
  File "", line 1, in 
ZeroDivisionError: division by zero

Unlike Haskell, Python evaluates the results immediately, which means that Python will crash as soon as an error occurs.

On the other hand, Haskell is different. In fact, you can string together dozens of lines of code, all of which contain an error, and no error will appear until one of those lines get evaluated.

let x = 3 `div` 0
let y = x

-- Can even divide x and y!
let z = x `div` y

let stringX = show x

-- Evaluate stringX, which
-- evaluates show x, which evaluates x
-- which finally evaluates 3 `div` 0, causing
-- the error to occur
putStrLn stringX

In this example, no matter what, Haskell will not give an error until something is evaluated. A common misconception is that a function call will always force the results to be evaluated, but that is simply not true.

In the example above, z is assigned the value of x divided by y, and yet there is no error. It is only when the value of stringX is evaluated via putStrLn (which prints a String) that Haskell throws an error.

One way to imagine thunks is to think of every value and function as being wrapped in a box. No one is allowed to peek into the box until the order is given. For all Haskell knows, the box could have anything in it, and Haskell wouldn’t care. As long as there are no type conflicts, Haskell will happily pass the box along.

In the code above, Haskell is fine with dividing two errors (both x and y, when evaluated, will give a division by zero error). However, it is only because x and y are both thunks, and Haskell doesn’t actually know the true values of x and y.

This gives Haskell the benefit of aggressively optimizing code, allowing for massive performance boosts.

Imagine if Haskell had an expensive function, F, that takes in one parameter, takes 10 seconds to run, and returns the evaluated value 5. Say that this same function also existed in Python.

If we were to run F ten times, we would find that Haskell only takes about 10 seconds to finish, while Python would take 100 seconds. Here, Haskell’s functional properties and lazy evaluation allows it to outperform Python.

In Python, functions are allowed to have side-effects. This means that if a Python function is run ten times, with the same input, it can give ten different results. On the other hand, for Haskell, if a function is run ten times, with the same input, it will always give the same result. This means that if there are multiple copies of the same function, called on the same input, Haskell can be certain that those results will all be the same. This is known as referential transparency, and it’s a feature that exists in most functional programming languages.

This property, combined with Haskell’s lazy evaluation, allows Haskell to memoize function calls. Now, if Haskell sees multiple copies of this expensive function called on the exact same input, it will simply evaluate one of the function calls, cache the result, and replace every future function call of F on that same input with the cached result.

What About Infinite Lists?

One consequence of everything being a thunk in Haskell is that Haskell is able to create infinite lists, and pass infinite lists around. In other words, Haskell is able to process and manipulate infinite lists as if they weren’t infinite, because they aren’t. Although the lists are technically infinite, Haskell only takes the elements that it wants.

For example,

let x = [1..]
let y = [1..]

x ++ y
-- concatenate all of x with all of y
-- to infinity
[1, 1, 2, 2, 3, 3, 4, 4, 5, 5 ...]

Here, x is stored to the infinite list from 1 to infinity, and y is stored to the exact same infinite list. Haskell is perfectly fine with storing these infinite data structures, because Haskell does not actually evaluate these structures until it needs to. When they are finally evaluated, they perform exactly like an infinite list — because that is exactly what x and y are.

In Haskell, treating everything as a thunk allows big performance boosts, and also gives the programmer the ability to store infinite data structures. So the next time someone says, “Everything in Haskell is X”, gently remind yourself that unless X is a thunk, they’re most likely wrong.

Common Boat Anchors in Programming and How to Avoid Them

A boat anchor is a metaphor that describes something that is obsolete or useless. In programming, boat anchors are everywhere. If you aren’t careful, boat anchors can ruin your productivity and cause confusion. Luckily, boat anchors are easy to prevent and correct if you look out for them. With just a little bit of foresight, you too can preserve your sanity.

TODO comments


Everyone’s used it. A seemingly innocent TODO comment. No harm done, since it’s just a little reminder for yourself right?

Wrong.

TODO comments are not only useless, but a productivity sink as well. Firstly, the “TODO” that you gave for yourself usually never gets done, as it’s buried in some function or class that you probably won’t even revisit for another week or two. Even if you do see the TODO, you’ll most likely ignore it as you’ll be too focused on your current task.

Secondly, TODO comments are bad because they waste the reader’s time. TODO comments belong in a Trello board or some other organizational system, not in your code base. No one wants to waste time reading about something that you should have/want to have done, but never got around to.

Commented Code


In this day and age, if your organization uses GitHub or some other form of version control, commented code is absolutely useless. This isn’t referring to commenting your code with useful English messages, but rather commenting out chunks of runnable code.

Commented code is essentially noise for the reader. If you push commented code to your repository because you think you might “need it later”, then you are ignoring the key purpose of version control.

Version control saves code. If you ever need that snippet of code that you deleted two months ago, it’s easily accessible. Don’t comment out code, or you’ll end up with hundreds and hundreds of obsolete lines of code.

Uncommented Code That Is Unused


Imagine you’ve written a utility class that calculates mortgages for you.

Suddenly, you’re told that the feature is scrapped, but that it might make a reappearance in the future.

Upon hearing that it might be used again in the future, you decide to leave the utility class in the code, since you might “need it later”. Once again, this goes back to the idea of version control. If it’s not relevant, and if it’s never used or referenced, then it doesn’t belong in your repository. Delete it.

Obsolete Documentation


Let’s say you have a nifty little method that method that divides every number in a list by 4.

def list_divide(list, dividend):
  """
  Divides every number in a list by a
  dividend and returns a new list with 
  the resulting numbers.
  """
  ret_list = []
  
  for num in list:
    ret_list.append(num/dividend)
  
  return ret_list

Suddenly, you remember about division by zero and quickly add a check to your function.

def list_divide(list, dividend):
  """
  Divides every number in a list by a
  dividend and returns a new list with 
  the resulting numbers.
  """

  if dividend == 0:
    print("Dividend must not be zero.")
    return

  ret_list = []
  
  for num in list:
    ret_list.append(num/dividend)
  
  return ret_list

Unfortunately, since you didn’t update the documentation, the docstring is now a lie. It doesn’t tell the user what the function actually does. There is no reference to division by zero in the docstring, causing the documentation to become a boat anchor.

Outdated documentation causes all sorts of issues for users. Luckily, this issue is easy to resolve. As a rule of thumb, if you add additional functionality to a function, document it. Even if you don’t like documentation, or if you think it’s tedious and pointless, do it anyway. You’ll save other programmers from headaches and wasted time.

Adding Ticket Numbers as Comments


Although not extremely common, some people feel the need to compulsively comment every change they make with a ticket number.

For example, you might see something like this :

#Added foobar function to resolve ticket 53
def foobar(num):
  #Fixes ticket 281
  if num == 0:
    print("Hello World!")
  elif num == 3:
    #Added because of ticket 571
    print(num/9)
  #Added else because of ticket 1528
  else:
    #Due to ticket 4713, changed to 'Goodbye World!' 
    print("Goodbye World!")

Before long, everything in your code will have a comment regarding a ticket.

If you are guilty of doing this, stop it. No one cares what ticket number the fix was in. Make the fix, commit, and push. All of the comments about tickets and issue numbers are utterly pointless and end up as unnecessary noise to the reader.

If you feel that the change is so strange or quirky that you absolutely must give some sort of background information, then write a comment. Making the reader search up a specific ticket to understand the code is a gigantic time sink, especially since most tickets have multiple comments. No one wants to read through all that text. Summarize it in a single comment and move on.

Storing Variables For No Reason


It’s a common practice for developers to do something along the lines of this :

def foobar(num):
  return num * 13

#Doing this
num = foobar(32)
print(num)

#As opposed to
print(foobar(32))

In this case, as long as “num” is not set to some wildly long function call, there is no reason to store it as a variable. In this case, foobar(32) is short, so it should be directly inputted into another function, rather than being set to a variable, and then inputted into the function.

Assigning it to a variable when the functional call is short tells the reader, “The variable num is important. It will be reused later in some way somewhere down the line.”

Not only that, but unnecessarily creating variables adds extraneous code for no good reason.

Conclusion


Boat anchors are everywhere in code. If you can avoid them, you will make not only your life easier, but everyone else’s as well. No one wants to waste time reading excessive code. Reading code is already difficult. Making your peers read more code than necessary hurts their productivity and makes coding an unpleasant experience.

So be a good person. Prune off the excess and correct the outdated. If you find any excessive code, then remove it. If the documentation is inaccurate or outdated, fix it. It might be a thankless job, but know that you’ll have made the world just a little brighter in the process.