This project is a Scrapy-based web scraping application named "sreality", designed to scrape real estate listings from the website "www.sreality.cz". It extracts various details about the properties, such as ID, name, price, and location, and stores this information in a PostgreSQL database.
This project requires Python 3.6 or newer. Additionally, it necessitates a PostgreSQL database or Docker for database containerization.
To set up the project, follow these steps:
- Clone the repository to your local machine.
- Ensure you have Python installed.
- Install the required Python packages using the command:
pip install -r requirements.txt
. - Copy the
.env.example
file, rename it to.env
, and update the variables with your PostgreSQL database credentials. - Optionally, you can use the provided
Dockerfile
anddocker-compose.yml
files to set up a PostgreSQL container with Docker. To do this, run the command:docker-compose up -d
.
To run the scraper:
- Navigate to the project directory.
- Run the spider using the command:
scrapy crawl sreality
.
The spider will get sreality data and store them in the PostgreSQL database.
Contributions are welcome. Please follow the standard fork and pull request workflow.
This project is distributed under the MIT License. See the LICENSE
file for more details.
For any queries or further information, please contact the project maintainer at [me@karelmaly.com].