Never load the whole folder
A folder of 8,889 photos used 1.9 GB of memory, and then the app crashed. The fix is the idea I reuse most in the code. I made the same mistake four times, each time in a different part of the app.
8,889 photos, every photo loaded: 8,889 textures, 1.3 GB of video memory, 1.9 GB in total. Then the OS kills the app.
What I learned
- Look for loops over everything the user gave you. They grow with the number of photos a person owns.
- Size memory by what you must keep at the same time, not by the number of photos in a job.
- Write the calculation next to the number. A person who later reads only "32" must know what can make it wrong.
- Check old numbers again when things change. Three of these four bugs were correct when I wrote them.
The crash
A wedding photographer gives you a card with 8,889 photos. You open the folder in LightPhotos, and the app crashes. My code to fill the grid came down to this:
for each photo in folder:
request a thumbnail
upload it to the GPU as a texture That loop is fine for twenty photos. With 8,889 photos, it asks the graphics card to hold 8,889 textures, about 1.3 GB of video memory, which pushes the total memory use of the app to about 1.9 GB. Then the operating system kills the app. The loop assumes that a folder is small.
The fix, the first time
A large monitor shows about forty thumbnails at once. Nobody needs the other 8,849 yet. Now the grid draws only the rows that you scrolled to, and requests thumbnails for those rows plus a few extra rows above and below, so that scrolling stays smooth. Web developers call this list virtualisation.
The fix, the second time
After a thumbnail loads, I keep it in a memory cache that holds up to a fixed number of thumbnails. I picked that number when thumbnails were small. Later, I made thumbnails a fixed 512 pixels on the long side and about 700 KB each, roughly seven times the old cost. Nothing crashed or logged an error. The limit was still a valid number, but it no longer meant what I intended.
So I derived the limit again from what it should have depended on from the start: how many thumbnails must be in memory at the same time. That is the visible grid plus three extra rows above and below, about 180 cells, so the limit is now 256. A lower limit makes the cache evict thumbnails right after it loads them, which is worse than no cache.
The fix, the third time
Auto Adjust analyses each selected photo to calculate its adjustments. My first version requested a thumbnail for every selected photo at the start, because it would need all of them eventually. With 20,000 photos at roughly 0.4 MB each, I asked the cache to hold about 8 GB inside a browser tab that is limited to 4 GB and never gives memory back.
20,000 selected, window of 32: about 13 MB.
The analysis of one photo takes about 8 ms, and it must run on the main thread of the app, so I cannot split it across CPU cores. Loading a thumbnail from an uncached RAW file takes 137 ms in the slow cases (95 out of 100 loads are faster). The loading must stay ahead of the analysis even in that worst case, so the window is the slowest load divided by one analysis.
I assumed that the thumbnail loads made the batch slow. I was wrong. The analysis takes about fifty-five times longer in wall-clock time, because it cannot run in parallel. Faster loading would not help at all.
The fix, the fourth time
The large view loads a photo at a size that fits the window, so in principle a one-pixel resize means a new load. Now the requested size rounds up to a multiple of 512 pixels, so a drag of the window edge triggers only a few reloads, not one for every frame. It is the same idea, applied to a value that changes smoothly (the window size) instead of a list: if an expensive cache depends on a value like that, I round the value off.