Duplicate Key error

[mkrzywonski]

May 25 08:19:38 nc docker-compose[166603]: photonix | IMPORTED /data/photos/Slides/Batch001-005.jpg
May 25 08:19:38 nc docker-compose[166603]: photonix | duplicate key value violates unique constraint "photos_tag_library_id_name_type_source_7da37b6b_uniq"
May 25 08:19:38 nc docker-compose[166603]: photonix | DETAIL: Key (library_id, name, type, source)=(a94928fa-3b6a-469a-9878-c2e7f31d7e1c, Color Slide Film, G, ) already exists.
May 25 08:19:38 nc docker-compose[166603]: photonix |
May 25 08:19:38 nc docker-compose[166603]: photonix | 2021-05-25 13:19:38,237 INFO exited: rescan_photos_periodically (exit status 0; not expected)
May 25 08:19:39 nc docker-compose[166603]: photonix | 2021-05-25 13:19:39,238 INFO gave up: rescan_photos_periodically entered FATAL state, too many start retries too quickly

[Idly41]

I do have the same error. I just intsalled Photonix and added some RAW-Pics for testing purposes to the photos folder. It starts to import 7 out of around 50 pictures but fails generating the thumbnails (black squares).

The logs show:

IMPORTED  /data/photos/20080925_piemont_0009.dng
duplicate key value violates unique constraint "photos_tag_library_id_name_type_source_7da37b6b_uniq"
DETAIL:  Key (library_id, name, type, source)=(57c35895-c6d7-48ce-b04b-53908e0d2d1f, Europe, G, ) already exists.

2021-05-26 09:23:14,098 INFO exited: rescan_photos_periodically (exit status 0; not expected)
2021-05-26 09:23:15,100 INFO gave up: rescan_photos_periodically entered FATAL state, too many start retries too quickly

Exception in thread Thread-3:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/threading.py", line 932, in _bootstrap_inner
    self.run()
  File "/usr/local/lib/python3.8/threading.py", line 870, in run
    self._target(*self._args, **self._kwargs)
  File "/srv/photonix/photos/management/commands/thumbnail_processor.py", line 22, in worker
    generate_thumbnails_for_photo(task.subject_id, task)
  File "/srv/photonix/photos/utils/thumbnails.py", line 38, in generate_thumbnails_for_photo
    get_thumbnail(photo=photo, width=thumbnail[0], height=thumbnail[1], crop=thumbnail[2], quality=thumbnail[3], force_regenerate=True, force_accurate=thumbnail[5])
  File "/srv/photonix/photos/utils/thumbnails.py", line 73, in get_thumbnail
    output_path = get_thumbnail_path(photo_file.id, width, height, crop, quality)
AttributeError: 'NoneType' object has no attribute 'id'

And the Postgres-Logs:

2021-05-26 09:22:09.294 UTC [58] ERROR:  duplicate key value violates unique constraint "photos_tag_library_id_name_type_source_7da37b6b_uniq"
2021-05-26 09:22:09.294 UTC [58] DETAIL:  Key (library_id, name, type, source)=(57c35895-c6d7-48ce-b04b-53908e0d2d1f, Europe, G, ) already exists.
2021-05-26 09:22:09.294 UTC [58] STATEMENT:  INSERT INTO "photos_tag" ("id", "created_at", "updated_at", "library_id", "name", "parent_id", "type", "source", "ordering") VALUES ('96144189-c4a0-4ba9-8b58-ea69cc42bd6e'::uuid, '2021-05-26T09:22:09.293187+00:00'::timestamptz, '2021-05-26T09:22:09.293187+00:00'::timestamptz, '57c35895-c6d7-48ce-b04b-53908e0d2d1f'::uuid, 'Europe', NULL, 'G', '', NULL)

[damianmoore]

Thanks for reporting this. Looks like a race condition when creating Tags. We have mitigation for this in photonix/classifiers/runners.py::get_or_create_tag() but maybe there is a problem with it.

[Linecutterx]

I have exactly the same issue.

2021-06-30 21:06:06.552 UTC [564] ERROR:  duplicate key value violates unique constraint "photos_tag_library_id_name_type_source_7da37b6b_uniq"
2021-06-30 21:06:06.552 UTC [564] DETAIL:  Key (library_id, name, type, source)=(1726c579-befd-488a-baf9-714981c11d73, George, G, ) already exists.
2021-06-30 21:06:06.552 UTC [564] STATEMENT:  INSERT INTO "photos_tag" ("id", "created_at", "updated_at", "library_id", "name", "parent_id", "type", "source", "ordering") VALUES ('67369a42-9749-4848-bf7a-2d389e6b7f9c'::uuid, '2021-06-30T21:06:06.551367+00:00'::timestamptz, '2021-06-30T21:06:06.551367+00:00'::timestamptz, '1726c579-befd-488a-baf9-714981c11d73'::uuid, 'George', NULL, 'G', '', NULL)

Could the key issue be caused by the '' which is the entry for the source (& common between all of these log snippets)?

[damianmoore]

Thanks for these comments. I think I understand this a bit more. I agree @Linecutterx, the Tag's source should be H (Human) rather than the empty string ``. These tags are of type=G (Generic) and happening on import so I assume these are tags that have been added to EXIF data from other programs. @GyanP, could you take a look?

[Linecutterx]

This is an unfortunate deal breaker for me, in that I can't get more than a few of my photos into photonix :(

They come from a Nikon SLR, processed from RAW through the Nikon software & with GPS coordinates added via DigiKam. I occasionally add face tags via DigiKam (which are tagged regions IIRC). They have unique file names.
Everything that's not just a standard jig should be within the standard EXIF data. I think!
I don't have that many different names for the face tags (the same people have their photo taken more than once!), which might be the issue.

[damianmoore]

@GyanP I'm removing this task from your list as community member @Red-F has fixed it in PR #304. Merged as of c8d6123 and release v0.18.0.

[rickysarraf]

Not sure if this fix applies to all the exceptions reported in this bug.

I rebuilt the images and tested. I still get this exception

photonix    | 2021-07-30 08:29:15,431 ERROR    Internal Server Error: /thumbnailer/photo/3840x3840_contain_q75/2e648ae9-4377-44e8-a8e0-a7530ebfcc5b/
photonix    | Traceback (most recent call last):
photonix    |   File "/usr/local/lib/python3.8/site-packages/django/core/handlers/exception.py", line 47, in inner
photonix    |     response = get_response(request)
photonix    |   File "/usr/local/lib/python3.8/site-packages/django/core/handlers/base.py", line 181, in _get_response
photonix    |     response = wrapped_callback(request, *callback_args, **callback_kwargs)
photonix    |   File "/srv/photonix/photos/views.py", line 34, in thumbnailer
photonix    |     path = get_thumbnail(photo_file=photo_file_id, photo=photo_id, width=width, height=height, crop=crop, quality=quality, return_type='url', force_accurate=force_accurate)
photonix    |   File "/srv/photonix/photos/utils/thumbnails.py", line 72, in get_thumbnail
photonix    |     output_path = get_thumbnail_path(photo_file.id, width, height, crop, quality)
photonix    | AttributeError: 'NoneType' object has no attribute 'id'
photonix    | 2021-07-30 08:29:15,432 ERROR    "GET /thumbnailer/photo/3840x3840_contain_q75/2e648ae9-4377-44e8-a8e0-a7530ebfcc5b/ HTTP/1.1" 500 73023
photonix    | 2021/07/30 08:29:15 [info] 67#67: *27 client 10.42.0.24 closed keepalive connection

[Linecutterx]

I had errors mentioning permissions & promise.py in my case. Deleting the database etc & starting from scratch seems to have fixed those. Did you do that?

[rickysarraf]

I had errors mentioning permissions & promise.py in my case. Deleting the database etc & starting from scratch seems to have fixed those. Did you do that?

Dear @Linecutterx

THank you so much. After wiping out the existing database, it is all working proper. Though it is okay, given that I just faced the scan problem in the beginning. Was this issue uncovered after the full scan (15k photos) and if then a user has to follow this workaround, that'd be frustrating.

But I guess this is okay for now. Photnoix is in its early phase of development and so far looks very very promising. 🙏