Hi @Kludex
Having looked at some of the discussions and PRs, I would like to contribute as well 🙂

Let me share how I can imagine a solution to the problem and I would appreciate it if you could give me some feedback regarding my suggestions and whether I got the idea of the project right 😄

First, as discussed above, it would be great to implement it as a web app with an SQLite DB (for simplicity and cost efficiency). FastApi with SQLAlchemy could be used. Alternatively, Streamlit could be a good idea because it could ease creating visualizations, though limiting the general UI capabilities. With FastAPI, HTMX can be used on the presentation layer. When it comes to visualizations, packages like Plotly, Bokeh, or ipyvizzu offer support for rendering graphs to HTML so they can get embedded in the templates.

The question is - what kind of visualization do you envision on the page?

Things get trickier when it comes to actually creating the index of the projects. It turns out it is not so obvious to get the dependencies employed by the Python projects. There are a lot of ways one could provide dependencies - either via a requirements file, pyproject.toml, setup.cfg, and so on ... (https://discuss.python.org/t/list-dependencies-of-a-package/12341) The tools exist to parse various formats of dependencies (https://github.com/nexB/dparse2/tree/main) but they are quite limited and do not support most of the formats. Do you have any ideas on how dependencies could be discovered efficiently? Alternatively, we could use a tool like https://pypi.org/project/third-party-imports/ to find the third-party imports - less efficient but might work well

The GitHub Actions could be used to run the periodic scraping of the repositories, as it has been suggested earlier.
I believe the GitHub Search should really be enough. Take a look at the following query: https://github.com/search?q=stars%3A%3E3+path%3A*.py+fastapi+language%3APython&type=&ref=advsearch&l=Python. Does https://sourcegraph.com/search allow to show the dependencies of the projects? Do you think Github Search would end up throttling the app? Which search engine would you prefer?

Please, let me know what you think and I would be happy to create a PR 🙂

Looking for maintainers #16

Description

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions