This is a simple event-driven scheduler that consits of 2 parts:
- go-job-runner: a simple job runner that triggers worker by sending a message to a queue
- go-job-worker: a simple worker that receives a message from a queue and executes a job
- Docker
- Docker Compose
- Run
docker-compose up -d
to start the scheduler - Open
http://localhost:4001/swagger/index.html
to see the swagger documentation - Register worker with name
random
- Copy worker id from the response
- Launch a worker with id from step 4
- Copy job id from the response
- Check job statuses by job id from step 6
Scheduler consists of 2 parts: job runner and job worker.
- Job runner is responsible for triggering workers by sending messages to a queue.
- Job worker is responsible for receiving messages from a queue and executing jobs.
- RabbitMQ is used as a message broker.
- PostgreSQL is used as a database.
- Flyway is used for database migrations.
- Swagger is used for API documentation.
To scale the scheduler, we can run multiple instances of job runner and job worker. To balance HTTP requests between job runners, we can use a load balancer. We can run as many instances of job worker as we need. Multiple workers will be able to receive messages from a queue and execute jobs. RabbitMQ will balance messages between different workers.