[Feature] Automatic creation of albums in external libraries #4279

dharapvj · 2023-09-22T18:06:20Z

dharapvj
Sep 22, 2023

The bug

Erstwhile --import option from immich CLI also supported an -al option due to which the folders from external library getting imported would also create albums.

Now with new External Library feature, photos are getting imported natively but they are not creating Albums.

The OS that Immich Server is running on

podman on ubuntu

Version of Immich Server

v1.79.0

Version of Immich Mobile App

v1.79.0

Platform with the issue

Server
Web
Mobile

Your docker-compose.yml content

Same as docker-compose.yaml on immich github.

Your .env content

No special configuration relevant to this bug

Reproduction steps

1. Add external library as volume in docker-compose file
2. Add external library in user -> account settings
3. Let jobs finish all imports
4. Check albums page to see that there are no albums created

Additional information

No response

Gatherix · 2023-09-22T19:03:57Z

Gatherix
Sep 22, 2023

This doesn't seem like a bug, though it would be a much appreciated feature

0 replies

dharapvj · 2023-09-22T19:08:25Z

dharapvj
Sep 22, 2023
Author

i thought of it as a bug because I felt External library feature as a "natural progression" of import feature in CLI and that feature offered album creation.

0 replies

wechsler42 · 2023-09-23T18:43:58Z

wechsler42
Sep 23, 2023

Well, I second that. Automatic creation of albums from external libraries would be really a nice feature .

0 replies

jrasm91 · 2023-09-23T18:46:20Z

jrasm91
Sep 23, 2023
Maintainer

Definitely not a bug, since it was never designed to do that in the first place. A feature request is more appropriate though. There are actually a lot of library enhancement requests, so it would be good to make a list and include this one as well.

0 replies

swap83 · 2023-09-25T17:40:07Z

swap83
Sep 25, 2023

#4186

0 replies

alvistar · 2023-09-26T10:34:53Z

alvistar
Sep 26, 2023

Hello,
I had the same issue. For quick and dirty workaround I wrote a python script to create albums from file structure

https://github.com/alvistar/immich-albums

Main issue is that the script needs also access directly to postgres as API to lookup an asset by oiriginal path is not exposed in the controller.

I am sharing hoping this can help somehow and eventually this functionality will be integrated in CLI.

10 replies

alvistar Oct 5, 2023

Hello, sorry was just for lack of time...

#4352

It should be very trivial to implement, but as you know, doing a proper patch with test, pr, etc takes some time.

etnoy Nov 14, 2023
Collaborator

This is now competed, feel free to use the new search api implemented in #4931

etnoy Nov 22, 2023
Collaborator

And with the 1.88 release, this new API is available for everyone

alexanderhuzar Aug 10, 2024

I for the life of me can't get this script working. No matter what I do it ends up with:
"can't open file '/app/immich_folder_album_creator.py': [Errno 2] No such file or directory".

pin147 Aug 12, 2024

Try this one. An updated version that can also be automated.
https://github.com/Salvoxia/immich-folder-album-creator

etnoy · 2023-09-26T11:20:14Z

etnoy
Sep 26, 2023
Collaborator

Hello, I had the same issue. For quick and dirty workaround I wrote a python script to create albums from file structure

https://github.com/alvistar/immich-albums

Main issue is that the script needs also access directly to postgres as API to lookup an asset by oiriginal path is not exposed in the controller.

I am sharing hoping this can help somehow and eventually this functionality will be integrated in CLI.

that looks very nice. Maybe you can open a feature request to expose that API? We absolutely want to support external API access for relevant usecases :)

0 replies

PJGitHub9 · 2023-09-26T19:38:01Z

PJGitHub9
Sep 26, 2023

Yeah same issue here. When i try use the import flag, it asks me if i want to upload all the images which i dont want to do.

0 replies

cristiangauma · 2023-10-05T07:17:43Z

cristiangauma
Oct 5, 2023

For me, this is the last step before migrating to Immich. With this feature, the amount of people that will migrate to Immich will be really huge :). Can't wait to see this implemented!

0 replies

etnoy · 2023-10-05T07:22:49Z

etnoy
Oct 5, 2023
Collaborator

Can you give a more detailed explanation of how this feature would work?

14 replies

nikolajhendel Nov 1, 2023

maybe the most flexible option is to use metadata tags (folder?) which can then be used to search based on metadata and manually create an album?
Looking at how photoprism has "done it" there is a folder view (not metadata as such) which allows you to browse - select all photos and manually create an album.

A solution like this takes away any direct coupling of folders and albums - but makes it very easy to manually create albums based on folder structure.
What would probably help is a metadata viewer in immich, as right now it only looks like you can search (and have to know your metadata tags) - I may be wrong, as I'm new to using/trying Immich.

alextran1502 Nov 2, 2023
Maintainer

@nikolajhendel

and have to know your metadata tags

We use contextual search (CLIP), so it doesn't actually generate a tag but an array of numbers which will be compared to with the encoded search phase you put in.

etnoy Nov 8, 2023
Collaborator

maybe this can be done in a one-shot command, possibly from the (new) cli. Click a button and it'll create albums but without any two-way linking

andyxpert Nov 8, 2023

And a job to allow for easy triggering library creation (one way from external library)

mooglestiltzkin Nov 22, 2023

just wanted to point out, with photoprism, they can create albums by browsing folders then select those pics and create an album for that. that is very fast DESPITE not being an auto album creation. At the very least u can browse existing folders where pics may have already been put into and create albums from that.

whereas immich, it puts them ALL into one single search to mix up all the pics despite being physically in separate folders from the start. Then expects you to create an album by selecting images 1 by 1 from all that jumbled up mess.

i'm no expert but that just seems to be counter productive :{

tsemachh · 2023-10-18T19:15:23Z

tsemachh
Oct 18, 2023

I see it's on discord in alex todo
Will be happy to see it soon.

0 replies

maltokyo · 2023-11-04T15:01:50Z

maltokyo
Nov 4, 2023

I would also completely migrate to immich if we had this. All of my photos are on the hard drive in a directory called "Photos", curated like this, where of course "Album_N" is a manually input description of the event.

Photos/YYYYMM Album_1
Photos/YYYYMM Album_2
Photos/YYYYMM Album_3

I have 1000s of such directories, curated over the years, as I am sure most avid photographers have... It would be great if we could use the info in a directory structure that is already there!

0 replies

davidacampos · 2023-11-17T21:41:59Z

davidacampos
Nov 17, 2023

Had the same issue as the rest... existing collection with many folders but now showing "flat" in Immich, so I created a quick and dirty JavaScript snippet that helped me create those.

How it works?

Gets all existing assets (photos/videos). This might take a while, in my case the generated JSON was over 100MB.
Goes through each one of them, and if they have the originalPath value (I assume that's only for external? I don't have non-external assets) extract the top folder from it and add it to our list of albums to create.
With the list of top folders, create albums and include the assets as needed.

For me this is good enough as it's a one-time thing. It currently doesn't handle re-running the script (i.e. removing autogenerated albums and recreating them) meaning you will need to remove the auto-generated albums to start from scratch (is there tagging for albums in Immich? this could help to tag remove them easily).

To run, navigate to Immich's web app in a browser (tested in Chrome), and copy&paste it to the "console" and run it. Make sure to change as needed under the PARAMETERS section of the snippet:

(async function () {

    //** PARAMETERS */
    /***********************************************************/
    let immichHost = "https://IMMICH_HOST";
    let rootPath = "/mnt/pictures/"; // This will be used as the "root" folder... meaning albums will be created based on folders directly under "root" (e.g. /mnt/pictures/ChristmasHoliday will end up as album "ChristmasHoliday")
    let foldersToExclude = []; // Optional, this is to exclude folders from been created as albums
    let albumGenerationLimit = 5; // Max number of albums auto-generated. Need to change to something like 9999 to create all albumos
    /***********************************************************/

    let assets = await (await fetch(immichHost + "/api/asset", {
        "body": null,
        "method": "GET",
        "mode": "cors",
        "credentials": "include"
    })).json();

    let albumsWithAssets = {};
    assets.forEach(asset => {
        if (asset.originalPath) {
            let folder = asset.originalPath.replace(rootPath, "").split("/")[0];

            if (foldersToExclude.includes(folder) == false) {
                if (!albumsWithAssets[folder]) {
                    albumsWithAssets[folder] = {
                        assets: []
                    }
                }

                albumsWithAssets[folder].assets.push(asset.id);
            }
        }
    });

    Object.keys(albumsWithAssets).forEach(async (folder, index) => {

        let body = {
            albumName: folder,
            assetIds: albumsWithAssets[folder].assets
        }

        if (index < albumGenerationLimit) { // Limits the number of albums auto-generated, mainly for testing purposes
            setTimeout(async function () {
                await fetch(immichHost + "/api/album", {
                    "headers": {
                        "accept": "application/json, text/plain, */*",
                        "content-type": "application/json",
                    },
                    "body": JSON.stringify(body),
                    "method": "POST",
                    "mode": "cors",
                    "credentials": "include"
                });
                console.log("Created album [" + index + "/" + Object.keys(albumsWithAssets).length + "] for: " + folder);
            }, index * 250); // Space out each album creation by 250ms to not hit all at once
        }

    });

})()

6 replies

go-vegan Nov 23, 2023

Works amazingly well! Thank you!

koen81 Mar 8, 2024

Curious if this (amazing and helpful) script is still working for version 1.97?

I tried it and it creates just max 3 albums (while I did specify let albumGenerationLimit = 999

Output looks like;

Created album [0/3] for:
Created album [2/3] for:
Created album [1/3] for:

Kind regards

davidacampos Mar 10, 2024

It should still work. As a matter of fact, I just published an updated version as a gist and added an option to be able to re-run the script (i.e. it removes existing folders and create them again).

I just tried it with 1.98.1 and it works.

Is your rootPath parameter correct? In my case my shared folder is mounted in /mnt/pictures/ so my folders are something like /mnt/pictures/FolderA. Do you have more than 3 folders at that level (the script doesn't create albums for sub-folders, just per folders at the root level)?

koen81 Mar 16, 2024

Thanks for getting back to me, so I have added 3 external libraries added to Immich:

photos_shared (/mnt/media/photos_shared) contains 101 directories
photos_koen (first name :)) (/mnt/media/photos_koen excluded mobile_upload as that has a different structure) contains 110 directories
photos_mobile upload: (/mnt/media/photos_koen/mobile_upload)

I wanted to run the script for each of the different external libraries (perhaps that's my mistake here, as it creates 3 directories, which corresponds to the number of external libraries I have);

So my settings in the script have been;

let immichHost = "http://192.168.1.XX:XXXX";
let rootPath = "/mnt/media/photos_shared/";
let foldersToExclude = [];
let albumGenerationLimit = 999;

Sorry for the late reply.

aronmgv Jun 1, 2024

Just wondering if somebody tried to extend the script to consider nested folders as well? So we would have separate album for each leaf folder and name would be the whole "path":

@cars/Kamiq/2021
@cars/Kamiq/2022
@cars/Kodiaq/2023
@cars/Kodiaq/2024
@skiing/Apls/2023/January
@skiing/Apls/2023/February

and so.. thanks!

mooglestiltzkin · 2023-11-22T14:51:21Z

mooglestiltzkin
Nov 22, 2023

@davidacampos this actually worked for me.

i went to here
http://my immich ip:2283/photos

then press F12 in browser, this opens up console.

I copied your code to note++ to edit.

i edited the following

let immichHost = "http://my immich lan ip:2283";

let rootPath = "/mnt/media/";

let albumGenerationLimit = 999;

fyi for reference my docker compose volumes for pics are like this

- /share/CACHEDEV4_DATA/Storage/Pictures/Belle:/mnt/media/Belle:ro

then i copied the edited code into the console, then ran it. and it successfully created the albums.

Ty you so much. hopefully the devs add this into the immich app gui as an action button or something. it's so simple to use.

*update

based on the maintainers advice, i opted to remove the quote above (which had the full script) for every1s safety as a precaution. if you want to see the original, refer to the original poster davidacampos who posted above.

what i did leave though is the parts i modified. very minimal edits and you can see for yourself they don't do anything by itself. i only commented based on his code what i changed to get it to work for me.

5 replies

alextran1502 Nov 22, 2023
Maintainer

Just general information for you. Not relating to Immich.

It is not wise to post random javascript snippets to your browser's console, especially if you don't understand what the code does. This is an easy way to get hacked.

mooglestiltzkin Nov 22, 2023

correct. and ty for the warning. i'm no pro coder, but i can read the code somewhat, and it seemed pretty benign (plz dont take my word for this, like i said i'm no expert). this is the very first thing i did before i considered trying (but yes you are absolutely correct, normally you should not do this especially if you don't understand the code as to what it is doing). and that's why i wanted to confirm for others it actually did what it purported to do.

maybe others who know more about code can also confirm the code is not malicious and does what it claims. i tested myself and worked for me.

i also tested browsing the albums using a different device using the immich android app (no i did not enter any code on that device). albums created from before are working there too just fine.

AverTry Feb 18, 2024

Thanks for the walkthrough, I had a few problems though as for the host with /photos causes the script to fail.
worked fine after that.

thorsten-hehn Oct 3, 2024

@davidacampos this actually worked for me.

i went to here http://my immich ip:2283/photos

then press F12 in browser, this opens up console.

I copied your code to note++ to edit.

i edited the following

let immichHost = "http://my immich lan ip:2283";

let rootPath = "/mnt/media/";

let albumGenerationLimit = 999;

fyi for reference my docker compose volumes for pics are like this

- /share/CACHEDEV4_DATA/Storage/Pictures/Belle:/mnt/media/Belle:ro

then i copied the edited code into the console, then ran it. and it successfully created the albums.

Ty you so much. hopefully the devs add this into the immich app gui as an action button or something. it's so simple to use.

*update

based on the maintainers advice, i opted to remove the quote above (which had the full script) for every1s safety as a precaution. if you want to see the original, refer to the original poster davidacampos who posted above.

what i did leave though is the parts i modified. very minimal edits and you can see for yourself they don't do anything by itself. i only commented based on his code what i changed to get it to work for me.

Thanks @mooglestiltzkin for the guide. Without it, I would have failed since I did not not what @davidacampos meant by "console".

mooglestiltzkin Oct 3, 2024

Thanks @mooglestiltzkin for the guide. Without it, I would have failed since I did not not what @davidacampos meant by "console".

no problem.

i removed the code just in case.

since i last posted, seems others have come up with other solutions, with their own githubs. but i'm not sure if their code is legit or not. wish there was an official solution :/ less worry xd.

But yes i did test the code previously it did work, but i don't know if malicious stuff got added or not. i'm not a coder :{

alvistar · 2023-11-23T09:58:38Z

alvistar
Nov 23, 2023

Hi all,
I have updated https://github.com/alvistar/immich-albums python script to remove database requirement and use the new api

It still needs some love regarding documentation but the script is working.

Thanks everybody,
a.

13 replies

floco Dec 4, 2023

Just came to say a big thanks to alvistar. Worked great for me for hundreds of albums.
Even made me realize that my immich scan was incomplete (as it creates in this case empty albums). And it's really fast when on the host running docker/immich and directly connected to the immich-server without any reverse proxy. Don't forget the /api or it won't work. Also don't forget to specify the path the script need to runs in (in my case "." / current directory).

poetry shell
cd /srv/sdb1/sharedrive/Photos/Events
im --api-key aaUgx59myapikeySAY67VU5WQ --api-host http://localhost:2283/api --replace-path /DATA/Gallery/Events --original-path /srv/sdb1/sharedrive/Photos/Events --recursive .

It's also easy to restart from scratch with a new immich user (in case of) by using a find command to remove all .album dot files like so: find /srv/sdb1/sharedrive/Photos/Events -name '.album' -type f -delete

The automated albums creation was the key thing missing from the previous/deprecated external library import. And that script fills the gap beautifully until this makes it to immich natively.

amit-handa Dec 4, 2023

thank you much for the script. It works for me as well. unfortunately, the albums created are of meh folders. can I remove the existing albums and restart (without removing the media) ?

maltokyo Dec 4, 2023

meh folders?

amit-handa Dec 4, 2023

sorry, meh folders => I meant there were spurious folders (having no images) in the path. and script created an album for each folder

amit-handa Dec 11, 2023

update: in the weekend. I did deep dive. after album creation, I updated one of the files but the album thumbnail were not updated :( filed a possible issue

alvistar · 2023-11-28T19:32:59Z

alvistar
Nov 28, 2023

Hi, From directory on host computer. You can mount them read-only in docker, but the script should be run on host. Eventually I could track folder->albumId externally in a db, but for easy implementation I chose to store in .album in the same folder structure. From: johndoe0815 ***@***.***> Reply to: immich-app/immich ***@***.***> Date: Tuesday, 28 November 2023 at 20:29 To: immich-app/immich ***@***.***> Cc: alvistar ***@***.***>, Comment ***@***.***> Subject: Re: [immich-app/immich] [Feature] Automatic creation of albums in external libraries (Discussion #4279) Where does it store these .album files, I didn't get that yet. My external library is so far mounted read-only, as also suggested by immich. — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you commented.Message ID: ***@***.***>

0 replies

kladderadeng · 2023-12-10T12:55:53Z

kladderadeng
Dec 10, 2023

I would love to see this. I have a curated library of thousands of folders I imported into Immich. I was even able to adjust Immich's Storage Template to fit to my existing library naming schema. So currently Immich, on import, creates Directories that match my existing Library as soon as I create an Album with the pictures. Having the feature also the other way round, Immich creating an Album if it finds a folder named according to the Storage Template (which includes an {{album}}, that would be fantastic :-)

0 replies

astronaute77 · 2024-01-15T19:36:39Z

astronaute77
Jan 15, 2024

I just stumbled upon this issue while trying to import an external library.
As a user, I’ve spend a lot of time to organize the folder structure thinking it was going to be imported as Albums.

So, my vote goes to implementing folder support for Albums, either nested (preferable) or non-nested.

0 replies

pin147 · 2024-01-18T01:29:39Z

pin147
Jan 18, 2024

If you have, like somebody mentioned above, hundreds and thousands of folders, maybe you don't want an album for every single folder. I think a simple folder view would be sufficient - check #5409
Not sure how albums are stored in the database, but it might add some extra unnecessary weight. I am only guessing that a folder view would be much cheaper to access the photos.
Pretty much every other major open-source photo management software comes with this feature (apart from the product-that-must-not-be-named). I am curious to know the reason why Alex decided not to implement folder view when the external library feature was implemented.

0 replies

ghost · 2024-02-22T20:31:49Z

ghost
Feb 22, 2024

Hello,

after hours of reading my 2 cents for a scripting solution in PowerShell. Maybe later a full module, for maintaining ( bulk action ) and customizing the awesome IMMICH by API.

COPY OF README.MD

Immich ( https://immich.app/ ) is absolutely great, but has weaknesses with mass processing of external libraries. A lot of people always have the same questions with GIT and very few have Python knowledge or Java Script to debug or customize the code snippets posted in various comments. Me neither ;)

But besides SQL, PowerShell is my second native language, so thanks to the perfect Api I was able to help myself quickly. I would like to share this with you. The connoisseur sees the beginnings of a module. I decided against it, because module customizations could cause problems for users. Well, I tried to keep it simple. Therefore no synopsis, clean ReadMe or other. ( or error handling, shame on me )

Requirements are PowerShell 5.1 ( tested on Win10 & 11 ). Maybe Core on Linux works too, that's it. Customize the _param.ps1 in advance and then execute the scripts directly in the root, private are only functions.

Currently there are two scripts that I really needed and these functions were asked again and again in this discussion.

album by folder name (external library) --> here even with freely selectable folder depth, not only root (I always missed that in the examples)
clean up the default library with bulk applications

Should there really be an interest for more ( the apo is powerful ), we could talk about an official PowerShell module >> psIMMICH :)

PS: Please use the testAsset variable for testing. I have tested everything up and down in my 70K asset environment but you never know. And the ApiKey and Address in Code, it doesnt matter, local only :P

GIT REPO psIMMICH

0 replies

cole21771 · 2024-02-26T18:35:30Z

cole21771
Feb 26, 2024

In the meantime, is there anyway to use the search feature to filter folder name?

Also: does anybody have any information on what metadata searches capable of? I’ve never seen it return a single result for me and the docs don’t say anything other than that metadata is possible

3 replies

ghost Feb 26, 2024

The way I see it, every asset has the OriginalPath property, which contains exactly the path that I and most others use to create the albums... and u want to search in. However, this is apparently only accessible via the Api and cannot be read in the WebFrontend, unfortunately. I've already stumbled across this. Just make it searchable as metadata. M:nameXY

cole21771 Feb 26, 2024

@TwiggyG2K23 the web shows the path in the details of each image, so why wouldn’t it be readable by the website??

ghost Feb 27, 2024

yeah, but noch searchable

sdht0 · 2024-03-24T17:16:07Z

sdht0
Mar 24, 2024

I think this would be an useful feature addition to immich.

With external libraries, there are so many edge cases. What happens on a rescan? What if you want to rename an album? What if you delete an album? Etc etc

We start by keeping it very simple. External library based albums will be read only. Any changes that users would want to do: renaming/deleting albums, moving photos between albums, etc will have to be done outside immich and then initiate a rescan. Immich can then faithfully reset its state to whatever is in the external folders.

I suppose there's still the question of allowing users to mix external and internal photos. For instance, creating albums that contains photos from both phone uploads and external libraries, or allowing shared users to add their photos to an external library album. Perhaps we can simply disallow these to start with.

1 reply

sdht0 Mar 24, 2024

Sounds like I'm basically thinking of #5409 haha.

MrSimmo · 2024-04-16T07:12:33Z

MrSimmo
Apr 16, 2024

The web code from @davidacampos works brilliantly for me. Noting I did need to set the variables as he recommended (host, path, total albums to 9999).

This would be a much needed feature to build in imo.

Option in settings to auto create albums from either metadata/json or folder structure.
When library is scanned initially (or when a subsequent change scan is initiated) it follows (1) and then performs the relevant action.

0 replies

This comment has been minimized.

Sign in to view

[Feature] Automatic creation of albums in external libraries #4279

The bug

The OS that Immich Server is running on

Version of Immich Server

Version of Immich Mobile App

Platform with the issue

Your docker-compose.yml content

Your .env content

Reproduction steps

Additional information

Replies: 25 comments · 52 replies

dharapvj Sep 22, 2023 Author

jrasm91 Sep 23, 2023 Maintainer

etnoy Nov 14, 2023 Collaborator

etnoy Nov 22, 2023 Collaborator

etnoy Sep 26, 2023 Collaborator

etnoy Oct 5, 2023 Collaborator

alextran1502 Nov 2, 2023 Maintainer

etnoy Nov 8, 2023 Collaborator

alextran1502 Nov 22, 2023 Maintainer

COPY OF README.MD

This comment has been minimized.

This comment has been minimized.

Replies: 25 comments 52 replies

dharapvj
Sep 22, 2023
Author

jrasm91
Sep 23, 2023
Maintainer

etnoy Nov 14, 2023
Collaborator

etnoy Nov 22, 2023
Collaborator

etnoy
Sep 26, 2023
Collaborator

etnoy
Oct 5, 2023
Collaborator

alextran1502 Nov 2, 2023
Maintainer

etnoy Nov 8, 2023
Collaborator

alextran1502 Nov 22, 2023
Maintainer