Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow auto-configured HikariDataSource to participate in checkpoint-restore #36422

Conversation

christophstrobl
Copy link
Member

This PR allows to suspend and gracefully shut down the connection pool of a HikariDataSource within the container lifecycle phases.

@spring-projects-issues spring-projects-issues added the status: waiting-for-triage An issue we've not yet triaged label Jul 17, 2023
Copy link
Member

@wilkinsona wilkinsona left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the PR, @christophstrobl. I'm still not sure about the reflection, but ignoring that for the time being I think there are a few refinements that could be made. WDYT?

if (hikariDataSource.getHikariPoolMXBean() instanceof HikariPool pool) {

Field closeConnectionExecutor = ReflectionUtils.findField(HikariPool.class, "closeConnectionExecutor");
if (closeConnectionExecutor != null) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the field's not found something's gone really wrong. I think it would be better to fail hard.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The idea was to be very defensive here, and fall back to checking total connections if for whatever reason HikariPool does not meet the expectations.

Copy link
Member

@wilkinsona wilkinsona Jul 19, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If Hikari doesn't meet expectations then I think we should just give up as we have no way of knowing that our approach is valid. We'll know, through tests, that it works with the version of Hikari in dependency management. If someone overrides that version and Hikari has changed incompatibility I think we'd be better to fail fast. That'll make the problem much easier to diagnose rather than dealing with a checkpoint failure and trying to figure out why it occurred.


ReflectionUtils.makeAccessible(closeConnectionExecutor);
Object field = ReflectionUtils.getField(closeConnectionExecutor, pool);
if (field instanceof ThreadPoolExecutor executor) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Won't this always be a ThreadPoolExecutor?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same defensive strategy as with the field above.
I looked up the history of HikaryPool and it seems that part did not change too much in the past. Will take care of this.

}
}
}
if (this.hasOpenConnections == null) {
Copy link
Member

@wilkinsona wilkinsona Jul 19, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wondering about thread safety here. Ideally hasOpenConnections would be final.

@wilkinsona wilkinsona added the status: waiting-for-internal-feedback An issue that needs input from a member or another Spring Team label Jul 19, 2023
This commit allows to suspend and gracefully shut down the connection pool of a HikariDataSource within the container lifecycle phases.
This commit makes sure to fail hard when assumptions related to the HikariDataSource are not met to perform lifecycle mgmt.
Since the pool may not have been initialized when creating the lifecycle some checks are delayed an will be performed when accessing a managed lifecycle phase.
@christophstrobl christophstrobl force-pushed the hikariCP/hikari-lifecycle branch from 750217a to 201ac53 Compare July 19, 2023 12:17
@christophstrobl
Copy link
Member Author

@wilkinsona rebased and now failing hard when conditions are not met. I had to delay the reflective lookup to pass all tests. So initially there'll only be a field and type check, but the HikariPool instance check of HikariPoolMXBean will happen later. Also made sure to use DataSourceUnwrapper to get the target of a proxy/delegate instance.

@wilkinsona
Copy link
Member

Thanks for the updates, @christophstrobl. We discussed this today and decided that we're happy with the reflection as it'll only come into effect when checkpoint-restore is being used. Would you like to take the PR out of draft status?

@christophstrobl christophstrobl marked this pull request as ready for review July 24, 2023 18:43
@wilkinsona wilkinsona changed the title Add lifecycle bean for managing HikariCP Add lifecycle bean for managing HikariCP for checkpoint-restore Jul 25, 2023
@wilkinsona wilkinsona added theme: checkpoint-restore Issues related to coordinated restore at checkpoint and removed status: waiting-for-triage An issue we've not yet triaged status: waiting-for-internal-feedback An issue that needs input from a member or another Spring Team labels Jul 25, 2023
@wilkinsona wilkinsona added this to the 3.2.x milestone Jul 25, 2023
@wilkinsona wilkinsona added the type: enhancement A general enhancement label Jul 25, 2023
@wilkinsona wilkinsona self-assigned this Jul 25, 2023
@wilkinsona wilkinsona changed the title Add lifecycle bean for managing HikariCP for checkpoint-restore Make HikariDataSource participate in checkpoint-restore Jul 25, 2023
@wilkinsona wilkinsona modified the milestones: 3.2.x, 3.2.0-M2 Jul 25, 2023
@wilkinsona
Copy link
Member

Thanks very much, @christophstrobl.

@wilkinsona wilkinsona changed the title Make HikariDataSource participate in checkpoint-restore Allow HikariDataSource to participate in checkpoint-restore Jul 26, 2023
@wilkinsona wilkinsona changed the title Allow HikariDataSource to participate in checkpoint-restore Allow auto-configured HikariDataSource to participate in checkpoint-restore Jul 26, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
theme: checkpoint-restore Issues related to coordinated restore at checkpoint type: enhancement A general enhancement
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants