Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

「Forbidden (403) CSRF verification failed. Request aborted」in Django administration page #6516

Open
Pan-will opened this issue Jul 19, 2023 · 17 comments

Comments

@Pan-will
Copy link

My actions before raising this issue

I used K8S to deploy all the services that cvat depends on in the cluster. The deployment is as follows:
image

Then, I went to the backend-server pod and created a superuser account with the command python3 ~/manage.py createsuperuser

Next, I went back to the Web page, logged in the superuser account with the browser, and entered the background management page, as shown below:
image

Steps to Reproduce (for bugs)

In the Django administration page, all my Post requests are getting errors like this:
image

Possible Solution

After Google, I found a workable solution:
doccano/doccano#1820

Your Environment

As mentioned above, I used the official ymal file to deploy the various services in the K8S cluster. We haven't used Django before, so we don't know how to add annotations to our code.
I was looking for a way to fix this by adding an environment variable to the backend-server deploy file.

Thanks for thinking about this!!

@Zanz2
Copy link

Zanz2 commented Jul 19, 2023

I encountered the same error, but only on file uploads. PR that would (likely) fix this: #6322

@Pan-will
Copy link
Author

I encountered the same error, but only on file uploads. PR that would (likely) fix this: #6322

Thank you very much for your solution!
My upload SDK does not have this issue. Due to the addition of some authentication logic, I wrapped the official upload tool and my new code to provide upload service in the form of SDK.

@Pan-will
Copy link
Author

Pan-will commented Aug 4, 2023

Depending on the result of the query, the CSRF question requires the domain name to be configured as an environment variable.
But I'm not sure what environment variables should be configured for the backend-server container.
Later, Google learned that it should add its own new domain name in CORS_ORIGIN_WHITELIST, so the following figure shows three environment variables configured, but to no effect:
image

image

@Zanz2
Copy link

Zanz2 commented Aug 7, 2023

As far as i know those settings are only applied when you use the development mode, not the production mode? Assuming a normal "vanilla" helm install:

  1. You run helm install, it uses the normal cvat docker image, this one has the DJANGO_CONFIGURATION env variable set to production, not development. https://github.com/opencv/cvat/blob/a43477af8eea946cf6498a6b1d12888840925bcc/Dockerfile#L76
  2. The manage.py file then uses "production" since the DJANGO_CONFIGURATION variable is set (otherwise if it wasnt it would use "development") https://github.com/opencv/cvat/blob/a43477af8eea946cf6498a6b1d12888840925bcc/manage.py#L12
  3. You then use the "production.py" file and not the "development.py" file that you linked, this one has no such configuration options ( https://github.com/opencv/cvat/blob/develop/cvat/settings/production.py )

So if you would like to use the development.py options, you have to add a ENV variable to your helm chart called DJANGO_CONFIGURATION with the value of "development".

@baudneo
Copy link

baudneo commented Sep 26, 2023

I have CVAT behind a cloudflared tunnel and am getting the 403 forbidden CSRF on django admin page POST. The provided fix says to use the CSRF_TRUSTED_ORIGINS env var but, Adding it to all the containers doesnt seem to solve the issue.

export CSRF_TRUSTED_ORIGINS="htttp://myserversIP,http://some other ip"

in the docker-compose.yml where do I add the ENV var?

    environment:
      CSRF_TRUSTED_ORIGINS: ${CSRF_TRUSTED_ORIGINS:-}

I put it in cvat-server, cvat-ui, cvat-utils and I am still getting CSRF 403 forbidden

@geometrikal
Copy link

@baudneo have you found a fix for this yet? I'm running into same thing

@baudneo
Copy link

baudneo commented Nov 9, 2023

@geometrikal not yet. I took it down and hosted it locally so I could actually import datasets and annotations.

One fix I was thinking of is adding the CVAT_HOST env var to every container that is running cvat-server image. The CVAT_HOST fix works in the cvat-server container but, several of the worker containers are based on cvat-server image.

My guess is the worker containers need that env var so they know what to trust, that's just my.guess though.

@kelbstf
Copy link

kelbstf commented Jan 18, 2024

Hello,
i am facing the exact same POST error in the administration page, when attempting to "SAVE".
But am hosting it locally, nevertheless having the problem.
I reproduced all methods that have been reported to help others so far, based on tuning "CSRF_TRUSTED_ORIGINS" with the accessing IP addresses, but the problem persists.

@baudneo: do you have an idea why going local solved the problem for you?

Just in case it matters: our CVAT is installed exactly according to the official "Quick installation guide", running on 8080 with CVAT_HOST set to the LAN IP, behind a reverse proxy that terminates HTTPS.

I tried different stable release including the latest and then the latest dev, but see no difference.

Any input would be great, since currently i seem to be stuck.

@baudneo
Copy link

baudneo commented Jan 18, 2024

Behind a proxy = no good. Accessing direct local is what worked for me. Good luck getting help from cvat team.

@kelbstf
Copy link

kelbstf commented Jan 18, 2024

@baudneo: ahh, i see, thank you for letting me know!
I would be glad to get it working with "direct 443" at least in order to make it available soon.
But on the long, a proxy would be important because we have to host many different applications, and since it's docker based...
It's weired that only POSTs seem to be affected, but that's apparently a side effect of the underlying security concept.

@baudneo
Copy link

baudneo commented Jan 18, 2024

@kelbstf I did a quick deep dive and couldn't get it figured out. My situation was using cloudflared proxy for public access and I could not get it to work.

My guess is that when the tus or whatever lib it is gets init'd, it needs an option to disable CORS or configure it for CVAT_HOST. At the time I was super busy so couldn't take the time to fix the issue and now I've moved on to other software.

IIRC, it was issue after issue after issue and I had gotten fed up. Cvat team doest reply to non trivial issues and when they do, they usually say "it works on our cloud hosted service, you should stop self hosting and subscribe to our cloud service" instead of doing any meaningful troubleshooting and fixing.

@kelbstf
Copy link

kelbstf commented Jan 18, 2024

@baudneo thank you for sharing your experiences.
So it's not just me and i get a perspective.
I am totally new to this software, but "issue after issue after issue" is exactly my experience so far. I have quite a lot of experience with implementing a variety of web applications behind reverse proxies, but here i feel i get lost in a maze of fractions of potential reasons and fixes, sometimes even for the same problem, so it's difficult to understand what is going wrong actually. Apparently the integration of django, traefik and CVAT has not a consolidated configuration pattern for the self hosted use case.

Since this is just about getting able to login and creating users, and uploading files seems to be the next challenge, i wonder how this will behave when it comes to the real meat of this application later on.

I still hope someone would share a brief guide, because this application is actually the primary choice for our users.
If someone out there managed to get this super generic 90% use case (HTTPS terminsation, reverse proxy, institutional certificates) implemented in a robust fashion and can share the wisdom, our users would be really happy.

I keep digging and will call back if i find a solution.

@Zanz2
Copy link

Zanz2 commented Mar 25, 2024

I got it to work by pinning the cvat-server image to v2.4.5 in the helm-chart file. (tag attribute of cvat/server). However I can imagine that for this to work you have to checkout your project to an older commit (around where v2.4.5 got released).

@marc31
Copy link

marc31 commented Apr 26, 2024

@kelbstf This is the solution I came up with. Although it may not be perfect, it's functional.

  1. Clone the CVAT repository
  2. Create a docker-compose.override.yml file. This file allows you to extend or override settings from the original docker-compose.yml.
services:
  cvat_server:
    volumes:
       # Resolve CRSF token error by mounting production.py
      - ./production.py:/home/django/cvat/settings/production.py:ro
    environment:
      # Make CVAT_HOST accessible INSIDE the Docker container
      CVAT_HOST: ${CVAT_HOST:-localhost}
  1. Duplicate the production.py file located at ./cvat/settings/production.py into a new file named ./production.py.
  2. Append the following line to the end of the ./production.py file:
CSRF_TRUSTED_ORIGINS = [origin.rstrip('/') for origin in os.getenv('CSRF_TRUSTED_ORIGINS', f'http://{CVAT_HOST},https://{CVAT_HOST}').split(',')]
  1. Once everything is set up, export the CVAT_HOST variable with your desired domain and bring up the Docker containers:
export CVAT_HOST="mydomain.com"
docker compose up -d

@alif-munim
Copy link

@kelbstf This is the solution I came up with. Although it may not be perfect, it's functional.

  1. Clone the CVAT repository
  2. Create a docker-compose.override.yml file. This file allows you to extend or override settings from the original docker-compose.yml.
services:
  cvat_server:
    volumes:
       # Resolve CRSF token error by mounting production.py
      - ./production.py:/home/django/cvat/settings/production.py:ro
    environment:
      # Make CVAT_HOST accessible INSIDE the Docker container
      CVAT_HOST: ${CVAT_HOST:-localhost}
  1. Duplicate the production.py file located at ./cvat/settings/production.py into a new file named ./production.py.
  2. Append the following line to the end of the ./production.py file:
CSRF_TRUSTED_ORIGINS = [origin.rstrip('/') for origin in os.getenv('CSRF_TRUSTED_ORIGINS', f'http://{CVAT_HOST},https://{CVAT_HOST}').split(',')]
  1. Once everything is set up, export the CVAT_HOST variable with your desired domain and bring up the Docker containers:
export CVAT_HOST="mydomain.com"
docker compose up -d

This was the only thing that worked after hours of searching - thank you so much!

@marc31
Copy link

marc31 commented Jul 31, 2024

I have not tried it yet, but take a look at: #6322 (comment)

@hiseulgi
Copy link

hiseulgi commented Aug 7, 2024

@kelbstf This is the solution I came up with. Although it may not be perfect, it's functional.

  1. Clone the CVAT repository
  2. Create a docker-compose.override.yml file. This file allows you to extend or override settings from the original docker-compose.yml.
services:
  cvat_server:
    volumes:
       # Resolve CRSF token error by mounting production.py
      - ./production.py:/home/django/cvat/settings/production.py:ro
    environment:
      # Make CVAT_HOST accessible INSIDE the Docker container
      CVAT_HOST: ${CVAT_HOST:-localhost}
  1. Duplicate the production.py file located at ./cvat/settings/production.py into a new file named ./production.py.
  2. Append the following line to the end of the ./production.py file:
CSRF_TRUSTED_ORIGINS = [origin.rstrip('/') for origin in os.getenv('CSRF_TRUSTED_ORIGINS', f'http://{CVAT_HOST},https://{CVAT_HOST}').split(',')]
  1. Once everything is set up, export the CVAT_HOST variable with your desired domain and bring up the Docker containers:
export CVAT_HOST="mydomain.com"
docker compose up -d

Thank you so much for this solution!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

8 participants