@AlmightyYakob - the current redirector does a few things that only a server can do. e.g. https://github.com/dandi/redirector/blob/master/serve.py#L132 (search for other HEAD requests and also the sitemap generation). if this is something that netlify could do, we can simply make netlfiy be the canonical source for dandiarchive.org and staging.dandiarchive.org.

would you be able to look into whether netlify can do those things?

if netlify cannot do that we can copy over this change to the redirector and serve the compiled js from the redirector. netlify can continue to do deployment previews, but it won't serve gui.dandiarchive.org.

we will want to integrate the redirector deployment into terraform and perhaps rewrite that server with fastapi instead of sanic.

@AlmightyYakob - the current redirector does a few things that only a server can do. e.g. https://github.com/dandi/redirector/blob/master/serve.py#L132 (search for other HEAD requests and also the sitemap generation). if this is something that netlify could do, we can simply make netlfiy be the canonical source for dandiarchive.org and staging.dandiarchive.org.

would you be able to look into whether netlify can do those things?

if netlify cannot do that we can copy over this change to the redirector and serve the compiled js from the redirector. netlify can continue to do deployment previews, but it won't serve gui.dandiarchive.org.

we will want to integrate the redirector deployment into terraform and perhaps rewrite that server with fastapi instead of sanic.

I don't think Netlify would be able to do this, since to my knowledge it can't do programmatic redirection, just that defined in the Netlify.toml. However, I'd urge strongly against trying to replace Netlify with the redirector for serving the GUI, as Netlify provides us a lot of benefits, deploy previews just being one of them.

@AlmightyYakob - the current redirector does a few things that only a server can do. e.g. https://github.com/dandi/redirector/blob/master/serve.py#L132 (search for other HEAD requests and also the sitemap generation). if this is something that netlify could do, we can simply make netlfiy be the canonical source for dandiarchive.org and staging.dandiarchive.org.
would you be able to look into whether netlify can do those things?
if netlify cannot do that we can copy over this change to the redirector and serve the compiled js from the redirector. netlify can continue to do deployment previews, but it won't serve gui.dandiarchive.org.
we will want to integrate the redirector deployment into terraform and perhaps rewrite that server with fastapi instead of sanic.

I don't think Netlify would be able to do this, since to my knowledge it can't do programmatic redirection, just that defined in the Netlify.toml. However, I'd urge strongly against trying to replace Netlify with the redirector for serving the GUI, as Netlify provides us a lot of benefits, deploy previews just being one of them.

I agree with Jake on replacing Netlify with the redirector. We would be embracing wholesale many known and unknown costs by doing so. I'm planning to pull together a meeting where we can really dig into the problems we're facing so we can develop the right solution here.

i wouldn't replace netlify with the redirector for anything other than serving the instance project. if netlify pushes the compiled js somewhere, we would simply use that. or even use netlify as a proxy. we want to continue having it do deploy previews.

the main goal here is to remove gui.dandiarchive.org as a target. so only that bit would be removed from netlify.

let's go through a few options:

1. keep netlify and let it serve from dandiarchive.org as the base url:

this would require understanding that any server side requests being done by the redirector has to come from the api server (which is ok). we would need to ensure that we turn on all the pieces of server side rendering in netlify such that our pages get indexed, etc.,. i believe there was an issue to this effect.

what we lose in this mode is to respond to anything at dandiarchiv.org, since it becomes a client side app. we need to ensure that the CLI understands this and only makes requests to the api. i.e. anyone developing tools has to understand that the web app is not a server, and needs to know what the server is.

2. keep redirector and simply serve the netlify compiled code.

allows us to provide responses to requests and we simply serve the js code for the instance.

i'm ok with implementing 1, but conveying that tools should be built against api will be important.

we would need to ensure that we turn on all the pieces of server side rendering in netlify such that our pages get indexed, etc.,. i believe there was an issue to this effect.

As of now I'd say this is out of scope, since currently our GUI is a Singe Page App (SPA), and thus doesn't really play nice with search engines (since mainly JS is returned from netlify, which is then rendered and injected into the html). The topic of SEO and search engine indexing is a separate discussion IMO, as it would be a much larger change.

As of now I'd say this is out of scope, since currently our GUI is a Singe Page App (SPA), and thus doesn't really play nice with search engines (since mainly JS is returned from netlify, which is then rendered and injected into the html).

there is a separate issue open about this. netlify has settings that automatically help with search that can be enabled. but the current role of the sitemap generator is related to search and is done by the redirector service at present.

i.e. anyone developing tools has to understand that the web app is not a server, and needs to know what the server is.

agree, and let's keep https://dandiarchive.org/server-info providing that information so in principle client could "ask" a server about what API server to talk to. BUT also /api could be proxied (or redirected?) to https://api.dandiarchive.org/api .

Overall -- in addition to #997 (comment) I still see a possible 3. in making "redirector" into a reverse proxy to "serve" at least the web UI from its respective underlying netflify, and otherwise not seen/visited directly by the users (don't know yet if possible to enforce redirection for direct access to https://gui.dandiarchive.org to the reverse proxy - should probably be possible).

I think removing the hash from the URL is a UX improvement and should be done now regardless.

how will existing urls in papers that have the # be redirected?

how will existing urls in papers that have the # be redirected?

Wouldn't this just be a simple update to the redirector, to remove the # from the url to redirect to?

Wouldn't this just be a simple update to the redirector, to remove the # from the url to redirect to?

the redirector serves the domain dandiarchive.org not gui.dandiarchive.org. the # is only in the gui urls that people have copied and pasted into their papers.

the redirector serves the domain dandiarchive.org not gui.dandiarchive.org. the # is only in the gui urls that people have copied and pasted into their papers.

Oh, I didn't realize people were using the GUI URL with the # included. I thought the whole point of the redirector was so people could use stable dandiset URLs and not need to use GUI URLs directly.. It might be possible to check for that on the client.

I thought the whole point of the redirector was so people could use stable dandiset URLs and not need to use GUI URLs directly.

that was indeed the hope, the reality is people prefer copying from browser url bar rather than the share icon.

that was indeed the hope, the reality is people prefer copying from browser url bar rather than the share icon.

Did some quick experimenting with the UI, and I think I could include a change to this PR that would automatically redirect those old URLs to the correct ones.

I think I could include a change to this PR that would automatically redirect those old URLs to the correct ones.

that would be great.

@satra 07dc0f0 contains the change I mentioned. I'd encourage you to play around with the deploy preview to make sure it redirects things as you expect. It's pretty simple but from what I can tell it seems to perform exactly as expected.

I can tell it seems to perform exactly as expected.

yup. seems to do the right thing.

🚀 PR was released in v0.2.8 🚀