Singularity: Multi-threading for dummies

Finally, I'm back for another update. Sorry about the wait, but I think this one will be nice and informative.

I've been working on getting Singularity's new version ready for use, and it's now almost at a point where I can use it as my daily driver. Today, I'm going to discuss how I've changed Singularity to feel faster and smoother by moving most of the work it does to separate threads.

Disclaimer:

the following post is meant as a simple (ish) introduction to event-driven applications and multi-threading for non-programmers. This is not a comprehensive guide, and I smooth over many details. It's certainly not a good introduction for developers looking to learn multi-threading, so please keep that in mind while reading this post.

 

The Problem

In the previous version of Singularity, I'd noticed some performance issues. Updates took a fairly long time to run, and even just starting up the application could cause a few seconds of unresponsiveness as it loaded up the initial data. One of my goals for the new version was to eliminate any unresponsiveness, and make everything just a bit faster and smoother.

However, the new version was much, much worse. The initial lag was gone, but updates took much longer and continued to leave the application frozen. Even just scrolling quickly could cause lag!

But why? Let's look deeper...

 

Where Do Lockups Come From?

Let's step back for a moment and discuss unresponsiveness in general. In many older, or poorly-coded applications, some actions can cause the application to freeze temporarily. Nothing will happen when you click on buttons or type, and the display might even display other windows on top of it! See below for an example:

Image courtesy of Wikipedia

If you don't write code, you might find this perplexing, but the reason is quite simple. It all comes back to how an application "flows".

Most modern applications are event-driven. Don't run away, it'll make sense in a moment! They run using a main loop, which is a fancy way to say that the do the same common set of actions over and over again, forever. A simple window that does nothing might look like this:

Note: UI means "user interface". It's the stuff that is shown to the user.

Every time the loop begins, it will do something like this:

1. Check for input. This could be clicking, moving the mouse, pushing buttons on your keyboard, or anything else that the programmer wants to know about.
2. When it gets the input that the programmer wants, it will do something immediately. This is why our application is event-driven: It waits for an event to happen, like getting input, then responds to the event.
3. If it needs to, it redraws itself to the screen. This usually only happens if something has changed, like a switch being flipped or an animation playing.

As you can see, the main loop is fairly simple: Our window waits for something to happen, redraws its contents, then repeats. Even trying to close the window is just another event: If we don't tell it to close, it won't.


So, what happens when we add a cool button to our window? When we click the button, we'll do something. It doesn't really matter what. Let's go back to our flowchart:

Maybe it plays an airhorn sound?

Notice how pushing the button and performing the action happen before the loop repeats? Most code works in a linear fashion: It can only do one thing at a time. Pushing the button stops our application from looking for input or redrawing itself until it's done working.

Next, we'll make our button check 100 sources for updates when pressed. After all, we need our news! Here's what happens:

This could take a while!

This could easily take a few minutes, and the whole time our application can do nothing else. Moving your mouse to another button and clicking will do nothing. And how could it? It's busy updating, and it can't check to see that you clicked until it's done. Even adding a progress bar wont help--It can't draw any changes to the bar until it's done with the updates.

Our users will be furious! How can we solve this problem?

First Pass: Networking

So, that's basic explanation of the problem that was facing Singularity. The updates were taking longer for two reasons: I was making some extra checks to improve the cleanup feature, and I was speeding up my updates in the old version using a method that I'll explain in another post. What I did to help alleviate the problem was create a separate thread for running updates.

What's a Thread?

Remember when I said that most code works in a linear fashion? Every action has to happen in order, as if it was following a line. Or in this case, a thread. And that's exactly what threads are: A thread can be thought of a list of instructions for the computer to follow. A single-threaded application is an application like the one above, that does one thing at a time. A multi-threaded application creates new threads, allowing it to do several things at the same time.

Still following me? They say a picture is worth a thousand words, so let's make our application check for updates in a second thread:

Note the branch after start update, and how the two lines merge back together after running the updates.

When we push our button now, all the main thread (the thread with our main loop) has to do is tell the update thread to start checking. While the update thread does the heavy lifting, our main thread can get input, redraw itself, and act responsive and alive while it works. We win!

However, it's not perfect. As soon as the updates finish loading, the window freezes again. It has to process and save the new updates to our database, and we can't tell our update thread to do it. I have several reasons for this, but to keep things simple I'm not going to get into it. It suffices to say that we should avoid having our update loader save our updates.

How do we get around this issue?

 

Second Pass: Database IO

 If you've been observant, you may have noticed that our update thread is also a loop. After sending the updates back to the main thread, it goes back to waiting. We can do something similar with our database too: By making it wait for requests (input), we can have it react to requests that we make.

Just like the update thread, we can make a database thread and ask it to save our updates. Now our main thread will have almost nothing keeping it from its important duty:

Note how the database sends nothing back. It doesn't need to since it's just saving, but we could always return something like the number of updates saved.

One advantage of this design is that our database can take any sort of request that we make, and all database interactions no longer stop our application from running smoothly.

 

The Results

This is an excellent start, although it doesn't solve all of our problems. Our application should never freeze or stutter anymore, but it's still slow as molasses when running updates. We also won't be able to use our database while saving updates, so any actions that involve the database will simply be delayed until the save is done. Our users won't think the application is broken, but they'll soon learn to hate that little progress bar in the corner.

To fix this, we can add more threads for handling more updates at a time, or give database requests priority levels (both of which I've done, although the priorities are still a work-in-progress), both of which will make things feel a bit faster and smoother. Ultimately, there are limits to how much you can optimize an application, but there's a lot that can be done to make things feel silky-smooth.