Six queues and a reserved worker
How LightPhotos loads images in the background, told as the problems that shaped it. Six queues and a worker that refuses the most common job can look like too much. But I added each part because something broke visibly without it.
Your photo is number 101. It waits behind 100 thumbnails that you scrolled past.
One queue, several workers
LightPhotos decodes images on a few background threads, which I call workers, so that the app stays responsive while they work. My first version had one queue and a handful of workers, first in, first out.
It worked until I scrolled. A scroll queues a hundred thumbnails. A photo that you open then goes to the end of the queue, behind thumbnails that are no longer on screen, and the app looks frozen.
Some jobs are more important
So I split the one queue into six. I ordered them by what you look at now, not by how quick a job is. Camera details (metadata) come before thumbnails because the details panel is next to the open photo, so you would notice a delay there. A thumbnail two rows down is not on the screen yet.
The first position in the queue is not sufficient
With six queues, the freeze was shorter, but it did not go away. Priority only decides which waiting job starts next. If all eight workers picked up thumbnails one millisecond before you double-clicked, your photo is first in line, but no worker is free to take it.
The usual solution is to interrupt a running job and give its worker to the urgent one. I could not, because the decode happens inside someone else's library, which has no way to know that I want it to stop. So I kept one worker aside that never takes thumbnails. That took a few lines, and it is what actually got the photo through. On a computer with one CPU core, I turn this off, because reserving the only worker would leave the grid with nothing.
A worker that can refuse jobs broke the wake-up signal
Workers sleep when the queue is empty. When the app adds a job, it wakes one worker. This standard method works only if any sleeping worker can take any job. My reserved worker cannot: it can refuse a job.
Wake one: the job stays in the queue until an unrelated event wakes a worker.
Markers for work that no worker started
The loader keeps "already requested" markers, so it does not queue the same photo five times. It clears each marker when the job is complete. In the browser version, the worker pool does not start any threads at all, so nothing was ever going to finish those jobs or clear the markers.
No error message appeared. Instead, the app woke sixty times a second for the rest of the session to check on work that did not exist, and drained your laptop battery.
My fix is a rule: each marker must have an owner that clears it. When the app queues a job, it gets one of three answers: accepted, no workers exist, or the worker pool is broken. The code marks the job as in progress only for the first answer. I now use this rule for each "loading" flag.
In a browser, workers can fail
A newly started browser worker (a Web Worker) discards any message sent to it before its script has finished loading. So each worker now says when it is ready. If one stays quiet for ten seconds, I treat it as failed. I replace a failed worker up to three times. Then I mark that slot as permanently gone.
"Permanently gone" must stay separate from "not ready yet", because mixing them up caused a real hang. Export waits while the number of running jobs is at or above capacity, and capacity counts only ready workers. A slot stuck halfway through replacement looked not ready, so capacity dropped to zero and the export progress bar waited forever.
What made this design easy to test
Parallel code usually has a predictable core: the rules about which worker runs which job, and in which order. The timing around it changes from run to run. I keep the rules in a plain data structure, so I test them without real threads, in microseconds, with the same result every time. One test adds one job of each type in reverse priority order and checks the order that comes out. Another test checks that the reserved worker skips thumbnails.