Immich Mobile Backup: Resolving Re-upload Loops After Deletion

by Alex Johnson 63 views

Understanding the Immich Re-upload Loop

When managing digital assets, especially photos and videos, efficient backup and deduplication processes are crucial. Immich, a modern self-hosted photo and video backup solution, offers robust features for organizing and storing your media. However, users have reported a persistent issue: after deleting duplicate uploads, they reappear after 30 days, creating an annoying re-upload loop. This article explores the causes of this problem, its impact, and potential solutions to ensure a smoother experience with Immich.

The core of the issue lies in how Immich handles deleted duplicates, particularly when integrating internal upload libraries with external libraries. The typical scenario involves merging duplicates, keeping the asset from an external library, and deleting the duplicate from the upload library. While this seems straightforward, the interaction with mobile backup clients introduces a critical flaw.

When a user deletes a duplicate from the upload library, it initially goes to the Trash. After 30 days, Immich permanently removes it. However, the mobile client, constantly checking the device library, detects that the file is no longer on the server. Consequently, the backup logic interprets this as a “missing” file and re-uploads it. This action recreates the duplicate, forcing the user to repeat the merging process, only for the cycle to repeat after another 30 days. This continuous loop undermines the efficiency of Immich and frustrates users seeking a clean, organized media library.

This problem arises because Immich does not retain the hash of uploads that users intentionally delete. From the mobile client's perspective, any missing file must be resynced. There is no mechanism to flag a file as “Intentionally excluded — do not re-upload this asset in the future.” Addressing this requires a fundamental change in how Immich handles deleted files, ensuring that the system remembers and respects user-initiated deletions.

The Technical Details: Why the Loop Occurs

To fully grasp the issue, it’s essential to understand the technical aspects of how Immich manages files and interacts with mobile clients. Immich uses hashing algorithms to identify duplicate files. When a file is uploaded, Immich calculates its hash and compares it against existing hashes in the database. If a match is found, the system recognizes the file as a duplicate. This deduplication process is designed to save storage space and reduce redundancy.

However, the current implementation lacks a crucial feature: remembering intentionally deleted hashes. When a user manually deletes a file or merges duplicates and chooses to delete one of the copies, Immich removes the file but does not record its hash as “excluded.” As a result, when the mobile client scans the device’s media library, it sees that the file is not present on the server and initiates a re-upload. This re-upload creates a new entry in Immich, leading to the reappearance of the duplicate.

The problem is compounded by the 30-day trash retention policy. This delay means that the system does not immediately recognize the deletion as permanent, and the mobile client continues to see the file as missing. Only after the file is permanently removed from the trash does the re-upload occur, creating a recurring cycle that is both resource-intensive and frustrating for users. The absence of a mechanism to mark files as intentionally excluded is the root cause of this re-upload loop.

Proposed Solutions and Expected Behavior

To resolve this issue, Immich needs to implement a system that remembers hashes of files explicitly deleted by the user. This would prevent backup clients from re-uploading these files and ensure that user-initiated deletions are respected. Several user-facing options could help facilitate this behavior and improve the overall user experience.

Immich should:

  • Keep a record of hashes explicitly deleted by the user.
  • Prevent backup clients from re-uploading these files.
  • Allow users to override this behavior if needed.
  • Optionally show a warning if the user tries to upload such a file manually.

When deleting an asset or merging duplicates, the following options could be presented to the user:

  • Delete permanently and prevent future uploads of this file.
  • Delete, but allow future uploads normally.

Additionally, when a user manually uploads a file whose hash is excluded, Immich should display a warning message, such as:

“This file was previously excluded. Import anyway?”

These enhancements would give users greater control over their media library and prevent the annoying re-upload loop. By implementing these features, Immich can ensure that user actions are respected and that the system behaves in a predictable and efficient manner. This will lead to a more satisfying and reliable user experience.

Impact of the Issue

The re-upload loop has several negative impacts on users, including:

  • Preventing the selection of an external library asset as the canonical version.
  • Hindering the deletion of internal duplicates.
  • Making mobile backup unusable due to infinite loops.

This issue is distinct from general deduplication efforts and necessitates server-side tracking of excluded hashes. Without this tracking, users are trapped in a cycle of merging, deleting, and re-uploading, which wastes time, storage space, and network bandwidth. The problem is particularly acute for users who rely on external libraries for their primary media storage, as they are forced to repeatedly manage duplicates that should have been permanently removed.

The continuous re-uploading also impacts server performance. Each re-upload consumes server resources, including CPU, memory, and disk I/O. Over time, this can degrade the overall performance of the Immich server, especially for users with large media libraries. Addressing this issue is therefore crucial not only for improving the user experience but also for maintaining the efficiency and stability of the Immich platform.

Technical Environment and Configuration

The re-upload loop has been observed in various environments and configurations. For example, one user reported the issue on an Immich server running on Ubuntu 24.04.3 LTS, with Immich Server version v2.2.3 and Mobile App version 2.2.3 build.235. The user's setup includes Docker Compose, with configurations for the Immich server, machine learning component, Redis, and PostgreSQL database.

The docker-compose.yml file defines the services required to run Immich, including the versions of the images, volumes, and network configurations. The .env file contains environment variables such as the upload location, external location, database credentials, and Immich version. These configurations are essential for ensuring that Immich runs correctly and that the data is stored and managed efficiently.

Here’s an example of the Docker Compose configuration:

name: immich
services:
 immich-server:
 container_name: immich_server
 image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release}
 volumes:
 - ${UPLOAD_LOCATION}:/data
 - ${EXTERNAL_LOCATION}:/external:ro
 - /etc/localtime:/etc/localtime:ro
 env_file:
 - .env
 ports:
 - '2283:2283'
 depends_on:
 - redis
 - database
 restart: always
 networks:
 - proxy

 immich-machine-learning:
 container_name: immich_machine_learning
 image: ghcr.io/immich-app/immich-machine-learning:${IMMICH_VERSION:-release}
 volumes:
 - model-cache:/cache
 env_file:
 - .env
 restart: always
 networks:
 - proxy

 redis:
 container_name: immich_redis
 image: docker.io/valkey/valkey:8
 restart: always
 networks:
 - proxy

 database:
 container_name: immich_postgres
 image: ghcr.io/immich-app/postgres:14-vectorchord0.4.3-pgvectors0.2.0
 environment:
 POSTGRES_PASSWORD: ${DB_PASSWORD}
 POSTGRES_USER: ${DB_USERNAME}
 POSTGRES_DB: ${DB_DATABASE_NAME}
 volumes:
 - ${DB_DATA_LOCATION}:/var/lib/postgresql/data
 restart: always
 networks:
 - proxy

volumes:
 model-cache:
networks:
 proxy:
 external: true

And here’s a snippet from the .env file:

# You can find documentation for all the supported env variables at https://docs.immich.app/install/environment-variables

# The location where your uploaded files are stored
UPLOAD_LOCATION=/mnt/Volume_103/immich
EXTERNAL_LOCATION=/mnt/Volume_103/import

# The location where your database files are stored. Network shares are not supported for the database
DB_DATA_LOCATION=./postgres

# To set a timezone, uncomment the next line and change Etc/UTC to a TZ identifier from this list: https://en.wikipedia.org/wiki/List_of_tz_database_time_zones#List
# TZ=Etc/UTC
TZ=Europe/Rome

# The Immich version to use. You can pin this to a specific version like "v2.1.0"
IMMICH_VERSION=v2

# Connection secret for postgres. You should change it to a random password
# Please use only the characters `A-Za-z0-9`, without special characters or spaces
DB_PASSWORD=hidden

# The values below this line do not need to be changed
###################################################################################
DB_USERNAME=postgres
DB_DATABASE_NAME=immich

#IMMICH_LOG_LEVEL=debug

These configurations highlight the complexity of setting up and managing an Immich server. Addressing the re-upload loop requires careful consideration of these configurations and their impact on the overall system behavior.

Steps to Reproduce the Issue

To reproduce the re-upload loop, follow these steps:

  1. Enable mobile backup/sync on iOS or Android for a user.
  2. Ensure the device contains a photo that also exists in an external library indexed by Immich (same file content).
  3. Allow the mobile app to upload the photo to Immich (or upload it manually to the upload library).
  4. Trigger a duplicate detection/merge in the UI so that one asset points to the upload library file and another to the external library file.
  5. In the merge dialog, choose to keep the external library asset and delete the upload library asset (which goes to the Trash).
  6. Wait until the Trash retention period expires (or manually trigger permanent deletion, if available) so that the upload library asset is completely removed from the server.
  7. Open the mobile app and let it run the backup/sync again.
  8. Observe that the mobile client re-uploads the same photo, Immich creates a new upload library asset, and the duplicate reappears, even though the external library asset already exists.
  9. Repeat the merge and deletion and observe that the same re-upload behavior happens after every Trash expiration, effectively creating an infinite loop.

By following these steps, users can reliably reproduce the issue and verify that the proposed solutions effectively address the re-upload loop.

Conclusion

The Immich re-upload loop is a significant issue that affects user experience and server performance. By understanding the root cause of the problem and implementing the proposed solutions, Immich can provide a more reliable and efficient media management platform. Server-side tracking of excluded hashes, combined with user-friendly options for managing deleted files, will prevent the re-upload loop and ensure that user-initiated deletions are respected. Addressing this issue is crucial for maintaining the integrity of the media library and providing a seamless backup and deduplication experience.

For more information on how Immich works, you can check the official documentation on their website.