Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: remove default backup target settings #3414

Merged
merged 3 commits into from
Dec 27, 2024

Conversation

derekbit
Copy link
Member

@derekbit derekbit commented Dec 26, 2024

Which issue(s) this PR fixes:

Issue longhorn/longhorn#10043

What this PR does / why we need it:

Special notes for your reviewer:

Additional documentation or context

Copy link

coderabbitai bot commented Dec 26, 2024

Walkthrough

This pull request introduces changes to multiple files related to backup target management in the Longhorn system. The modifications include adding a new function OwnerIDFromBackupTarget in the API forwarder, updating router handling for backup target actions, removing the syncDefaultBackupTarget method from the setting controller, and enhancing backup target resource synchronization in the datastore. The changes aim to refactor how backup targets are handled, validated, and configured across different components of the system.

Changes

File Change Summary
api/forwarder.go Added OwnerIDFromBackupTarget function to retrieve owner ID for backup targets
api/router.go Updated backup target actions to use new OwnerIDFromBackupTarget function for routing
controller/setting_controller.go Removed syncDefaultBackupTarget method and updated syncNonDangerZoneSettingsForManagedComponents
datastore/longhorn.go Added syncDefaultBackupTargetResourceWithCustomizedSettings method and updated UpdateCustomizedSettings
types/setting.go Added DefaultBackupstorePollInterval constant, updated GetCustomizedDefaultSettings method
webhook/resources/backuptarget/validator.go Enhanced error handling in validateCredentialSecret method
webhook/resources/systembackup/validator.go Modified Create method validation for system backup

Assessment against linked issues

Objective Addressed Explanation
Default backup target should remain configured and not be cleared (10043)
Shift in how backup targets are managed

Possibly related PRs

  • fix(backup): enqueue volumes with correct names #3413: The changes in this PR involve the enqueueVolumesForBackupVolume method, which is related to backup target handling, similar to the new OwnerIDFromBackupTarget function introduced in the main PR. Both PRs enhance the management of backup targets within the Longhorn system.

Suggested reviewers

  • derekbit
  • c3y1huang

Tip

CodeRabbit's docstrings feature is now available as part of our Early Access Program! Simply use the command @coderabbitai generate docstrings to have CodeRabbit automatically generate docstrings for your pull request. We would love to hear your feedback on Discord.


Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

❤️ Share
🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai generate docstrings to generate docstrings for this PR. (Beta)
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

@mantissahz mantissahz requested a review from a team December 26, 2024 09:17
@mantissahz mantissahz self-assigned this Dec 26, 2024
@mantissahz mantissahz marked this pull request as ready for review December 27, 2024 01:56
Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

🧹 Nitpick comments (7)
types/setting.go (4)

34-35: Add doc comment for newly introduced constant
It's helpful to document that 300 seconds is the default interval for polling the backup store to make it more intuitive for future maintainers.


1559-1559: Consider adding a docstring
It's a good practice to describe the purpose and expected behavior of GetCustomizedDefaultSettings in a docstring, especially given its central role in customizing settings.


1562-1563: Initialize the map in a single line
Combining line 1562 and 1563 for map initialization can reduce verbosity, e.g., defaultBackupTargetSettings := map[string]string{}.


1571-1577: Improve readability for setting checks
Rather than chaining multiple || expressions, consider a short set membership check. This becomes more maintainable if new backup-target-related settings are introduced later.

datastore/longhorn.go (2)

9-9: Initialize a random seed
If randomness is used in scheduling or selection logic, consider calling rand.Seed(time.Now().UnixNano()) once at startup to avoid deterministic outcomes.


78-81: Add error context logging
Including more detailed logging (e.g., error message or relevant fields) upon failure of syncDefaultBackupTargetResourceWithCustomizedSettings can help diagnose issues.

webhook/resources/backuptarget/validator.go (1)

156-166: Refine whitespace checks
This switch statement thoroughly guards against leading/trailing spaces or newlines. If multiple whitespace issues exist in one key, consider combining them into a single, more descriptive error message for clarity.

📜 Review details

Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between b4418eb and c06a98e.

📒 Files selected for processing (7)
  • api/forwarder.go (1 hunks)
  • api/router.go (1 hunks)
  • controller/setting_controller.go (0 hunks)
  • datastore/longhorn.go (4 hunks)
  • types/setting.go (4 hunks)
  • webhook/resources/backuptarget/validator.go (1 hunks)
  • webhook/resources/systembackup/validator.go (1 hunks)
💤 Files with no reviewable changes (1)
  • controller/setting_controller.go
🔇 Additional comments (7)
types/setting.go (2)

1566-1566: Check caller for potential nil references
Ensure the caller properly handles the scenario where both returned maps are nil when err is non-nil, to avoid potential runtime errors.


1627-1627: Confirm return order
Verify that the caller consistently expects (defaultSettings, defaultBackupTargetSettings, err) in this exact order to avoid potential misuse.

datastore/longhorn.go (2)

59-59: Validate non-nil configMap
Ensure defaultSettingCM is non-nil before invoking GetCustomizedDefaultSettings; otherwise, a nil pointer could occur.


218-280: Test coverage suggestion
This new function correctly creates or updates the default BackupTarget resource. Consider adding unit tests covering edge cases such as invalid poll intervals, missing URL/credential, or partial overrides.

webhook/resources/systembackup/validator.go (1)

57-60: Validate backup target availability
Good addition ensuring that a system backup isn't created if the BackupTarget is not available. Consider clarifying the error message by indicating that backup target availability is required for system backups.

api/forwarder.go (1)

45-57: Return an error when BackupTarget is missing
Mirror the logic in OwnerIDFromVolume by returning a non-nil error if the requested backup target does not exist, rather than returning an empty string. This can prevent ambiguous handling downstream.

api/router.go (1)

129-130: Ensure correctness of the new forwarding approach.

The actions "backupTargetSync" and "backupTargetUpdate" now rely on OwnerIDFromBackupTarget(s.m) to determine the node's HTTP address. This approach looks more consistent with the rest of the “owner ID” logic, but please confirm:

  1. Error-handling: That OwnerIDFromBackupTarget gracefully handles cases where the backup target is missing or not found.
  2. Coverage: That relevant unit tests or integration tests are updated to verify this change in routing logic.

Would you like me to generate a shell script to locate all tests referencing these actions and confirm their coverage?

@derekbit
Copy link
Member Author

@mantissahz remember to add ticket number to #3414 (comment) and each commit.

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

🧹 Nitpick comments (1)
datastore/longhorn.go (1)

218-279: Enhance error handling in syncDefaultBackupTargetResourceWithCustomizedSettings

The function handles backup target synchronization well, but could benefit from some improvements:

  1. Consider adding validation for pollInterval value range
  2. Consider adding logging for important state transitions
 func (s *DataStore) syncDefaultBackupTargetResourceWithCustomizedSettings(customizedDefaultBackupTargetSettings map[string]string) error {
+    logrus.Debugf("Syncing default backup target with settings: %v", customizedDefaultBackupTargetSettings)
+
     backupTarget := &longhorn.BackupTarget{
         ObjectMeta: metav1.ObjectMeta{
             Name: types.DefaultBackupTargetName,
@@ -238,6 +240,9 @@
     if pollIntervalStr != "" {
         pollIntervalAsInt, err := strconv.ParseInt(pollIntervalStr, 10, 64)
         if err != nil {
+            return errors.Wrapf(err, "invalid poll interval value %v", pollIntervalStr)
+        }
+        if pollIntervalAsInt < 0 {
             return err
         }
         backupTarget.Spec.PollInterval = metav1.Duration{Duration: time.Duration(pollIntervalAsInt) * time.Second}
📜 Review details

Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between c06a98e and 7efd8ed.

📒 Files selected for processing (7)
  • api/forwarder.go (1 hunks)
  • api/router.go (1 hunks)
  • controller/setting_controller.go (0 hunks)
  • datastore/longhorn.go (4 hunks)
  • types/setting.go (4 hunks)
  • webhook/resources/backuptarget/validator.go (1 hunks)
  • webhook/resources/systembackup/validator.go (1 hunks)
💤 Files with no reviewable changes (1)
  • controller/setting_controller.go
🚧 Files skipped from review as they are similar to previous changes (4)
  • webhook/resources/backuptarget/validator.go
  • api/forwarder.go
  • webhook/resources/systembackup/validator.go
  • api/router.go
🔇 Additional comments (4)
types/setting.go (3)

34-35: LGTM: DefaultBackupstorePollInterval constant definition

The constant is properly defined using time.Duration for backup store polling interval.


140-145: LGTM: Backup target settings definition

The code properly documents that these settings are used in the ConfigMap to update the default BackupTarget resource, with clear comments explaining their purpose.


Line range hint 1559-1627: Verify error handling in GetCustomizedDefaultSettings

The function now returns an additional map for backup target settings. While the implementation looks correct, we should verify that all callers handle the new return value properly.

✅ Verification successful

Let me check the usage of the returned backup target settings.


The new return value is properly handled by all callers

The verification shows that GetCustomizedDefaultSettings is only called in datastore/longhorn.go and the returned customizedDefaultBackupTargetSettings is correctly used in syncDefaultBackupTargetResourceWithCustomizedSettings to initialize the backup target resource with customized settings for:

  • BackupTargetURL
  • CredentialSecret
  • PollInterval

The error handling is also properly implemented throughout the call chain.

🏁 Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Search for all callers of GetCustomizedDefaultSettings to verify they handle the new return value
rg "GetCustomizedDefaultSettings\(" -A 3

Length of output: 646


Script:

#!/bin/bash
# Check how customizedDefaultBackupTargetSettings is used in datastore/longhorn.go
rg "customizedDefaultBackupTargetSettings" -A 10 datastore/longhorn.go

Length of output: 2107

datastore/longhorn.go (1)

78-81: LGTM: UpdateCustomizedSettings integration

The integration of backup target sync into UpdateCustomizedSettings is clean and follows proper error handling patterns.

@derekbit
Copy link
Member Author

derekbit commented Dec 27, 2024

LGTM, but need to wait for the results of regression and robot tests (including upgrade test)

@mantissahz
Copy link
Contributor

e2e local run: (test_backup.robot)

==============================================================================
Tests
==============================================================================
Tests.Regression
==============================================================================
Tests.Regression.Test Backup :: Backup Test Cases
==============================================================================
...

Test Incremental Restore :: Test restore from disaster recovery vo... | PASS |
------------------------------------------------------------------------------
Tests.Regression.Test Backup :: Backup Test Cases                     | PASS |
1 test, 1 passed, 0 failed
==============================================================================
Tests.Regression                                                      | PASS |
1 test, 1 passed, 0 failed
==============================================================================
Tests                                                                 | PASS |
1 test, 1 passed, 0 failed
==============================================================================
==============================================================================
Tests
==============================================================================
Tests.Regression
==============================================================================
Tests.Regression.Test Backup :: Backup Test Cases
==============================================================================
...

Test Backup Volume List :: Test Backup Volume List We want to make... | PASS |
------------------------------------------------------------------------------
Tests.Regression.Test Backup :: Backup Test Cases                     | PASS |
1 test, 1 passed, 0 failed
==============================================================================
Tests.Regression                                                      | PASS |
1 test, 1 passed, 0 failed
==============================================================================
Tests                                                                 | PASS |
1 test, 1 passed, 0 failed
==============================================================================

@derekbit
Copy link
Member Author

innobead
innobead previously approved these changes Dec 27, 2024
Copy link
Member

@innobead innobead left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

@@ -140,12 +137,16 @@ const (
SettingNameDefaultMinNumberOfBackingImageCopies = SettingName("default-min-number-of-backing-image-copies")
SettingNameBackupExecutionTimeout = SettingName("backup-execution-timeout")
SettingNameRWXVolumeFastFailover = SettingName("rwx-volume-fast-failover")
// These three backup target settings are used in the "longhorn-default-setting" ConfigMap
Copy link
Member

@innobead innobead Dec 27, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To prevent any confusion, we might need to decouple these settings from global settings in chart values.yaml when working on a PR on longhorn repo.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@@ -358,24 +355,6 @@ var (
SettingNameRWXVolumeFastFailover: SettingDefinitionRWXVolumeFastFailover,
}

SettingDefinitionBackupTarget = SettingDefinition{
DisplayName: "Default Backup Target",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remember to update doc accordingly.

}

// won't accept partially valid result
for name, value := range defaultSettings {
if name == string(SettingNameBackupTarget) ||
name == string(SettingNameBackupTargetCredentialSecret) ||
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: Some util libs can be considered for this kind of general operation.
https://github.com/samber/lo

COLDTURNIP
COLDTURNIP previously approved these changes Dec 27, 2024
Copy link
Member Author

@derekbit derekbit left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

🧹 Nitpick comments (3)
types/setting.go (1)

Line range hint 1559-1627: Consider adding error handling for invalid poll interval string

The function correctly separates backup target settings but should validate the poll interval string format before returning it.

Consider adding validation like:

 if pollIntervalStr != "" {
+    if _, err := strconv.ParseInt(pollIntervalStr, 10, 64); err != nil {
+        return nil, nil, fmt.Errorf("invalid poll interval format %v: %v", pollIntervalStr, err)
+    }
 }
datastore/longhorn.go (2)

238-244: Add validation for poll interval range

The code should validate that the poll interval is within a reasonable range.

Consider adding range validation:

 if pollIntervalStr != "" {
     pollIntervalAsInt, err := strconv.ParseInt(pollIntervalStr, 10, 64)
     if err != nil {
         return err
     }
+    if pollIntervalAsInt < 0 {
+        return fmt.Errorf("poll interval %v must be non-negative", pollIntervalAsInt)
+    }
     backupTarget.Spec.PollInterval = metav1.Duration{Duration: time.Duration(pollIntervalAsInt) * time.Second}
 }

272-279: Consider adding logging for backup target updates

Adding debug logs would help track backup target configuration changes.

Consider adding:

 if reflect.DeepEqual(existingBackupTarget.Spec, backupTarget.Spec) {
+    logrus.Debugf("No changes detected for backup target %v", backupTarget.Name)
     return nil
 }
+logrus.Infof("Updating backup target %v", backupTarget.Name)
📜 Review details

Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 7efd8ed and ec206a5.

📒 Files selected for processing (7)
  • api/forwarder.go (1 hunks)
  • api/router.go (1 hunks)
  • controller/setting_controller.go (0 hunks)
  • datastore/longhorn.go (4 hunks)
  • types/setting.go (4 hunks)
  • webhook/resources/backuptarget/validator.go (1 hunks)
  • webhook/resources/systembackup/validator.go (1 hunks)
💤 Files with no reviewable changes (1)
  • controller/setting_controller.go
🚧 Files skipped from review as they are similar to previous changes (4)
  • api/forwarder.go
  • webhook/resources/backuptarget/validator.go
  • api/router.go
  • webhook/resources/systembackup/validator.go
🔇 Additional comments (4)
types/setting.go (2)

34-35: LGTM: DefaultBackupstorePollInterval constant is well defined

The constant is properly defined with a clear type and reasonable default value of 300 seconds (5 minutes).


140-145: LGTM: Backup target settings are properly documented

The backup target settings are well documented with clear comments explaining their purpose and usage in the ConfigMap.

datastore/longhorn.go (2)

218-237: LGTM: Default backup target initialization is well structured

The initialization code properly sets up the default backup target with appropriate defaults and handles customized settings.


258-267: LGTM: Fallback to existing values is properly handled

The code correctly falls back to existing values when customized settings are not provided.

derekbit and others added 3 commits December 27, 2024 17:06
ref: longhorn/longhorn 5411, 10043

Signed-off-by: Derek Su <derek.su@suse.com>
And add missing changes when fixing 7159.

ref: longhorn/longhorn 5411, 7159, 10043

Signed-off-by: James Lu <james.lu@suse.com>
instead of `NodeHasDefaultEngineImage` because we should allow to
modify a backup target without the `default` engine image.

ref: longhorn/longhorn 5411, 10043

Signed-off-by: James Lu <james.lu@suse.com>
Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

🧹 Nitpick comments (1)
datastore/longhorn.go (1)

218-279: Add validation for poll interval value range

The function handles backup target synchronization well, but should validate that the poll interval value is within a reasonable range to prevent potential issues with very short or very long intervals.

 if pollIntervalStr != "" {
     pollIntervalAsInt, err := strconv.ParseInt(pollIntervalStr, 10, 64)
     if err != nil {
         return err
     }
+    // Add validation for reasonable poll interval range (e.g., 30s to 24h)
+    if pollIntervalAsInt < 30 || pollIntervalAsInt > 86400 {
+        return fmt.Errorf("poll interval must be between 30 and 86400 seconds")
+    }
     backupTarget.Spec.PollInterval = metav1.Duration{Duration: time.Duration(pollIntervalAsInt) * time.Second}
 }
📜 Review details

Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between ec206a5 and fecf6d7.

📒 Files selected for processing (7)
  • api/forwarder.go (1 hunks)
  • api/router.go (1 hunks)
  • controller/setting_controller.go (0 hunks)
  • datastore/longhorn.go (4 hunks)
  • types/setting.go (4 hunks)
  • webhook/resources/backuptarget/validator.go (1 hunks)
  • webhook/resources/systembackup/validator.go (1 hunks)
💤 Files with no reviewable changes (1)
  • controller/setting_controller.go
🚧 Files skipped from review as they are similar to previous changes (4)
  • api/forwarder.go
  • webhook/resources/backuptarget/validator.go
  • webhook/resources/systembackup/validator.go
  • api/router.go
🔇 Additional comments (3)
types/setting.go (2)

140-145: LGTM: Clear documentation for backup target settings

The backup target settings are well-documented with clear comments explaining their purpose and usage in the ConfigMap.


Line range hint 1559-1627: LGTM: Well-structured settings handling with proper validation

The function properly separates general settings from backup target settings and includes appropriate validation. The error handling and default value management are implemented correctly.

datastore/longhorn.go (1)

Line range hint 59-81: LGTM: Clean integration of backup target settings

The changes properly integrate the new backup target settings handling into the existing UpdateCustomizedSettings flow.

@derekbit derekbit merged commit 0ac13db into longhorn:master Dec 27, 2024
9 checks passed
@derekbit
Copy link
Member Author

@mergify backport v1.8.x

Copy link

mergify bot commented Dec 27, 2024

backport v1.8.x

✅ Backports have been created

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants