Skip to content

annif docker container failed to run #784

Closed
@s0rin

Description

Using annif_app as in https://github.com/NatLibFi/Annif/blob/main/docker-compose.yml, the annif docker container was running a for while until pulling recently the image quay.io/natlibfi/annif:latest, since then it fails with the following error:

# docker logs --follow docker_annif
[2024-04-05 12:02:34 +0000] [1] [INFO] Starting gunicorn 21.2.0
[2024-04-05 12:02:34 +0000] [1] [INFO] Listening at: http://0.0.0.0:8000 (1)
[2024-04-05 12:02:34 +0000] [1] [INFO] Using worker: sync
[2024-04-05 12:02:34 +0000] [7] [INFO] Booting worker with pid: 7
/usr/local/lib/python3.10/site-packages/sklearn/base.py:376: InconsistentVersionWarning: Trying to unpickle estimator TfidfTransformer from version 1.3.2 when using version                                                               1.4.1.post1. This might lead to breaking code or invalid results. Use at your own risk. For more info please refer to:
https://scikit-learn.org/stable/model_persistence.html#security-maintainability-limitations
  warnings.warn(
/usr/local/lib/python3.10/site-packages/sklearn/base.py:376: InconsistentVersionWarning: Trying to unpickle estimator TfidfVectorizer from version 1.3.2 when using version 1                                                              .4.1.post1. This might lead to breaking code or invalid results. Use at your own risk. For more info please refer to:
https://scikit-learn.org/stable/model_persistence.html#security-maintainability-limitations
  warnings.warn(
/usr/local/lib/python3.10/site-packages/sklearn/base.py:376: InconsistentVersionWarning: Trying to unpickle estimator TfidfTransformer from version 1.3.2 when using version                                                               1.4.1.post1. This might lead to breaking code or invalid results. Use at your own risk. For more info please refer to:
https://scikit-learn.org/stable/model_persistence.html#security-maintainability-limitations
  warnings.warn(
/usr/local/lib/python3.10/site-packages/sklearn/base.py:376: InconsistentVersionWarning: Trying to unpickle estimator TfidfVectorizer from version 1.3.2 when using version 1                                                              .4.1.post1. This might lead to breaking code or invalid results. Use at your own risk. For more info please refer to:
https://scikit-learn.org/stable/model_persistence.html#security-maintainability-limitations
  warnings.warn(
WARNING:annif:Couldn't initialize backend 'fasttext': model data/projects/hogwarts/fasttext-model not found
WARNING:annif:Couldn't initialize backend 'tfidf': vectorizer file 'data/projects/tfidf-fi/vectorizer' not found
WARNING:annif:Couldn't initialize backend 'tfidf': vectorizer file 'data/projects/tfidf-sv/vectorizer' not found
WARNING:annif:Couldn't initialize backend 'tfidf': vectorizer file 'data/projects/tfidf-en/vectorizer' not found
WARNING:annif:Couldn't initialize backend 'fasttext': model data/projects/fasttext-fi/fasttext-model not found
WARNING:annif:Couldn't initialize backend 'fasttext': model data/projects/fasttext-sv/fasttext-model not found
WARNING:annif:Couldn't initialize backend 'fasttext': model data/projects/fasttext-en/fasttext-model not found
/usr/local/lib/python3.10/site-packages/sklearn/base.py:376: InconsistentVersionWarning: Trying to unpickle estimator CountVectorizer from version 1.3.2 when using version 1                                                              .4.1.post1. This might lead to breaking code or invalid results. Use at your own risk. For more info please refer to:
https://scikit-learn.org/stable/model_persistence.html#security-maintainability-limitations
  warnings.warn(
/usr/local/lib/python3.10/site-packages/sklearn/base.py:376: InconsistentVersionWarning: Trying to unpickle estimator DecisionTreeClassifier from version 1.3.2 when using ve                                                              rsion 1.4.1.post1. This might lead to breaking code or invalid results. Use at your own risk. For more info please refer to:
https://scikit-learn.org/stable/model_persistence.html#security-maintainability-limitations
  warnings.warn(
/usr/local/lib/python3.10/site-packages/sklearn/base.py:376: InconsistentVersionWarning: Trying to unpickle estimator BaggingClassifier from version 1.3.2 when using version                                                               1.4.1.post1. This might lead to breaking code or invalid results. Use at your own risk. For more info please refer to:
https://scikit-learn.org/stable/model_persistence.html#security-maintainability-limitations
  warnings.warn(
2024-04-05 12:02:44.113014: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critica                                                              l operations.
To enable the following instructions: AVX2 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
2024-04-05 12:02:52.370582: W external/local_tsl/tsl/framework/cpu_allocator_impl.cc:83] Allocation of 28799200 exceeds 10% of free system memory.
2024-04-05 12:02:52.470843: W external/local_tsl/tsl/framework/cpu_allocator_impl.cc:83] Allocation of 28799200 exceeds 10% of free system memory.
2024-04-05 12:02:52.497902: W external/local_tsl/tsl/framework/cpu_allocator_impl.cc:83] Allocation of 28799200 exceeds 10% of free system memory.
2024-04-05 12:02:52.603658: W external/local_tsl/tsl/framework/cpu_allocator_impl.cc:83] Allocation of 14399600 exceeds 10% of free system memory.
2024-04-05 12:02:52.642776: W external/local_tsl/tsl/framework/cpu_allocator_impl.cc:83] Allocation of 28799200 exceeds 10% of free system memory.
[2024-04-05 12:02:54 +0000] [7] [ERROR] Exception in worker process
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/gunicorn/arbiter.py", line 609, in spawn_worker
    worker.init_process()
  File "/usr/local/lib/python3.10/site-packages/gunicorn/workers/base.py", line 134, in init_process
    self.load_wsgi()
  File "/usr/local/lib/python3.10/site-packages/gunicorn/workers/base.py", line 146, in load_wsgi
    self.wsgi = self.app.wsgi()
  File "/usr/local/lib/python3.10/site-packages/gunicorn/app/base.py", line 67, in wsgi
    self.callable = self.load()
  File "/usr/local/lib/python3.10/site-packages/gunicorn/app/wsgiapp.py", line 58, in load
    return self.load_wsgiapp()
  File "/usr/local/lib/python3.10/site-packages/gunicorn/app/wsgiapp.py", line 48, in load_wsgiapp
    return util.import_app(self.app_uri)
  File "/usr/local/lib/python3.10/site-packages/gunicorn/util.py", line 424, in import_app
    app = app(*args, **kwargs)
  File "/Annif/annif/__init__.py", line 59, in create_app
    annif.registry.initialize_projects(cxapp.app)
  File "/Annif/annif/registry.py", line 114, in initialize_projects
    app.annif_registry = AnnifRegistry(projects_config_path, datadir, init_projects)
  File "/Annif/annif/registry.py", line 41, in __init__
    project.initialize()
  File "/Annif/annif/project.py", line 132, in initialize
    self._initialize_backend(parallel)
  File "/Annif/annif/project.py", line 116, in _initialize_backend
    self.backend.initialize(parallel)
  File "/Annif/annif/backend/nn_ensemble.py", line 132, in initialize
    self._model = load_model(
  File "/usr/local/lib/python3.10/site-packages/keras/src/saving/saving_api.py", line 254, in load_model
    return saving_lib.load_model(
  File "/usr/local/lib/python3.10/site-packages/keras/src/saving/saving_lib.py", line 281, in load_model
    raise e
  File "/usr/local/lib/python3.10/site-packages/keras/src/saving/saving_lib.py", line 269, in load_model
    _load_state(
  File "/usr/local/lib/python3.10/site-packages/keras/src/saving/saving_lib.py", line 466, in _load_state
    _load_container_state(
  File "/usr/local/lib/python3.10/site-packages/keras/src/saving/saving_lib.py", line 534, in _load_container_state
    _load_state(
  File "/usr/local/lib/python3.10/site-packages/keras/src/saving/saving_lib.py", line 435, in _load_state
    trackable.load_own_variables(weights_store.get(inner_path))
  File "/usr/local/lib/python3.10/site-packages/keras/src/engine/base_layer.py", line 3531, in load_own_variables
    raise ValueError(
ValueError: Layer 'dense' expected 2 variables, but received 0 variables during loading. Expected: ['dense/kernel:0', 'dense/bias:0']
[2024-04-05 12:02:54 +0000] [7] [INFO] Worker exiting (pid: 7)
[2024-04-05 12:02:57 +0000] [1] [ERROR] Worker (pid:7) exited with code 3
[2024-04-05 12:02:57 +0000] [1] [ERROR] Shutting down: Master
[2024-04-05 12:02:57 +0000] [1] [ERROR] Reason: Worker failed to boot.

Metadata

Assignees

Labels

Type

No type

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions