Navidrome: Fixing Unicode Path Issues

Hey there, fellow music lovers and tech enthusiasts! Today, we're diving deep into a rather specific, yet potentially frustrating, issue that can pop up when managing your digital music library with Navidrome. You know, that awesome self-hosted music server that lets you access your tunes from anywhere? Well, sometimes, even the best software can run into quirks, especially when dealing with the wonderfully complex world of file paths and characters. Specifically, we're going to tackle a bug related to Unicode normalization when comparing file paths in Navidrome. Don't worry if "Unicode normalization" sounds a bit intimidating; we'll break it down and explore why this matters and how it can be resolved. Our goal is to ensure your music library stays accurate, even when dealing with those fancy characters in your file names and paths.

Understanding the Unicode Normalization Conundrum

So, what exactly is Unicode normalization and why is it causing headaches for Navidrome users? Think of it this way: the same character can sometimes be represented in multiple ways using Unicode. For example, the character "é" can be represented as a single precomposed character, or it can be a combination of a regular "e" followed by a combining acute accent. To a human, they look identical, but to a computer, they are technically different byte sequences. This is where the "normalization" comes in. Unicode normalization is the process of converting these different representations into a single, standard form. This ensures that when you compare two strings that look the same, your computer sees them as the same, too.

This issue often surfaces during file migrations or when moving libraries between different file systems. Different operating systems and file systems (like NTFS, exFAT, APFS, ext4) can handle Unicode characters in slightly different ways, leading to these variations in representation. For instance, as reported by a user, moving a library from an exFAT filesystem (common on external drives) back to an ext4 filesystem (common on Linux) caused this problem. Navidrome, when scanning the library, encountered file paths that, while visually identical, had different underlying Unicode representations. This caused Navidrome to see them as new files instead of recognizing them as the same existing ones. The result? Songs were marked as missing, and duplicate entries were created in the database, which is definitely not what we want.

In the provided example, the sqlite query clearly shows two different hexadecimal representations for what should be the same file path: Bran Van 3000/Rosé/09 Stand Up.mp3. One shows RosC3A9 (which is the UTF-8 encoding for the precomposed 'é'), while the other shows RoseCC81 (which is the UTF-8 encoding for 'e' followed by the combining acute accent '´'). Navidrome, not performing proper Unicode normalization during its comparison, treated these as two distinct paths. This is a critical behavior to address because it directly impacts the integrity and manageability of your music library within the application. It highlights the importance of consistent handling of character encoding across different systems and applications.

The Impact on Your Music Library: Missing Songs and Duplicates

When Unicode normalization goes awry, the consequences for your music library can be quite disruptive. The most immediate and noticeable effect is that Navidrome starts flagging songs as missing. If you've recently moved your music files, perhaps to a new drive or after a system backup, and you notice a significant chunk of your library suddenly disappears from Navidrome's interface, this could be the culprit. It's like your music has vanished into thin air!

But it gets worse. Navidrome doesn't just mark them as missing; it then proceeds to create new entries for these