Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Update CVEs not working, GitFetchOperator and smtp-password typed in airflow-logs #499

Open
tobfel opened this issue Jan 9, 2025 · 2 comments
Open

Update CVEs not working, GitFetchOperator and smtp-password typed in airflow-logs #499

tobfel opened this issue Jan 9, 2025 · 2 comments

Comments

@tobfel
Copy link

tobfel commented Jan 9, 2025

Specifications

  • OpenCVE version: v2
  • Python version: docker included
  • Platform: docker/debian

I have some problems with the auto-update of the cve's -

In airflow is see this error on all "GitFetchOperator" executed.

552f4ac9a6ae
*** Could not read served logs: [Errno -2] Name or service not known

Any hint why this is not working?

when i start the DAG manual via the web interface "Trigger DAG" i was wondering the smtp-server password is logged?!

Hint: the password contains a "%" char.

5156e85dc3e5
*** Found logs served from host http://5156e85dc3e5:8793/log/dag_id=opencve/run_id=manual__2025-01-09T12:21:54.394124+00:00/task_id=cves.fetch_vulnrichment/attempt=1.log
[2025-01-09, 12:21:56 UTC] {taskinstance.py:1979} INFO - Dependencies all met for dep_context=non-requeueable deps ti=<TaskInstance: opencve.cves.fetch_vulnrichment manual__2025-01-09T12:21:54.394124+00:00 [queued]>
[2025-01-09, 12:21:56 UTC] {taskinstance.py:1979} INFO - Dependencies all met for dep_context=requeueable deps ti=<TaskInstance: opencve.cves.fetch_vulnrichment manual__2025-01-09T12:21:54.394124+00:00 [queued]>
[2025-01-09, 12:21:56 UTC] {taskinstance.py:2193} INFO - Starting attempt 1 of 1
[2025-01-09, 12:21:56 UTC] {taskinstance.py:2217} INFO - Executing <Task(GitFetchOperator): cves.fetch_vulnrichment> on 2025-01-09 12:21:54.394124+00:00
[2025-01-09, 12:21:56 UTC] {standard_task_runner.py:60} INFO - Started process 95 to run task
[2025-01-09, 12:21:56 UTC] {standard_task_runner.py:87} INFO - Running: ['airflow', 'tasks', 'run', 'opencve', 'cves.fetch_vulnrichment', 'manual__2025-01-09T12:21:54.394124+00:00', '--job-id', '192', '--raw', '--subdir', 'DAGS_FOLDER/opencve_dag.py', '--cfg-path', '/tmp/tmpabdmjwak']
[2025-01-09, 12:21:56 UTC] {standard_task_runner.py:88} INFO - Job 192: Subtask cves.fetch_vulnrichment
[2025-01-09, 12:21:56 UTC] {standard_task_runner.py:107} ERROR - Failed to execute job 192 for task cves.fetch_vulnrichment (invalid interpolation syntax in **'<smtp-password>'** at position 8; 95)
[2025-01-09, 12:21:56 UTC] {local_task_job_runner.py:234} INFO - Task exited with return code 1
[2025-01-09, 12:21:56 UTC] {taskinstance.py:3312} INFO - 0 downstream tasks scheduled from follow-on schedule check

Thanks.
@ncrocfer
Copy link
Member

ncrocfer commented Jan 9, 2025

If I understand well you have 2 problems in this issue:

  1. for the first one it seems the Airflow logs can't be read from the webserver. Did you change something from the official installation and its docker-compose file?

  2. this is the first time I see this error [2025-01-09, 12:21:56 UTC] {standard_task_runner.py:107} ERROR - Failed to execute job 192 for task cves.fetch_vulnrichment (invalid interpolation syntax in **'<smtp-password>'** at position 8; 95)

And I'm a bit surprised to see in the cves.fetch_vulnrichment as it's only a git task. Maybe, but I'm really not sure, you changed something to the initial OpenCVE setup (so the first bug), and because of it the git pull is blocked. So Airflow tries to send you an email but fails because of an issue on your SMTP settings. But this is just guess :/

@tobfel
Copy link
Author

tobfel commented Jan 9, 2025

If I understand well you have 2 problems in this issue:

  1. for the first one it seems the Airflow logs can't be read from the webserver. Did you change something from the official installation and its docker-compose file?

yes, the passwords in .env/config, postgres 15->postgres 17, and i put traefik (with http-auth) in front of the nginx (changed port from 80->9080) and the airflow-webserver. for this i disabled the exposed ports from the two containers. i will give it a try with the default install.

  1. this is the first time I see this error [2025-01-09, 12:21:56 UTC] {standard_task_runner.py:107} ERROR - Failed to execute job 192 for task cves.fetch_vulnrichment (invalid interpolation syntax in **'<smtp-password>'** at position 8; 95)

And I'm a bit surprised to see in the cves.fetch_vulnrichment as it's only a git task. Maybe, but I'm really not sure, you changed something to the initial OpenCVE setup (so the first bug), and because of it the git pull is blocked. So Airflow tries to send you an email but fails because of an issue on your SMTP settings. But this is just guess :/

That was also my guess, that i put the password in a wrong var. but i have double checked that. also the emails (registration, email-confirmation) are working.

Sadly i do a "./install.sh -r latest" to have the new "email-test-dag" in airflow and so all my customized configs was getting overriden. :-( Any hints who to do an update? is a "docker compose pull" enough?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@ncrocfer @tobfel