Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Merged by Bors] - fix: recover from invalid segments #2909

Closed
wants to merge 15 commits into from
Closed

[Merged by Bors] - fix: recover from invalid segments #2909

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
15 commits
Select commit Hold shift + click to select a range
e965b1f
more cleanup in validator
sehz Dec 28, 2022
9e883de
remove unnecessary method
sehz Dec 28, 2022
d7212ad
use proper api to get endoffset
sehz Dec 29, 2022
2812888
introduce default validation to simplify
sehz Dec 29, 2022
45ddb85
change header decoding to use anyhow
sehz Dec 29, 2022
216ccbd
remove skip error and verbose flag from validator
sehz Dec 29, 2022
cae2f18
add last file position and batch position
sehz Dec 29, 2022
2308062
update
sehz Dec 29, 2022
f07a969
remove unused error for validation
sehz Dec 29, 2022
e4a380b
implement repair for batch decoding error
sehz Dec 29, 2022
ec59bfa
update test for repairing replica segment
sehz Dec 29, 2022
848251a
add more info for header error
sehz Dec 29, 2022
541c278
add cli tools to get replica info
sehz Dec 29, 2022
a61d6a5
ensure hw is consistent with leo
sehz Dec 30, 2022
07220a0
restore fail fast for integration test
sehz Dec 30, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
change header decoding to use anyhow
  • Loading branch information
@sehz
sehz committed Dec 29, 2022
commit 45ddb854faabae22e6340c9d74ce8207a8c03f56
22 changes: 12 additions & 10 deletions crates/fluvio-storage/src/batch.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ use std::fmt::Debug;
use std::marker::PhantomData;
use std::path::Path;

use fluvio_protocol::record::BatchHeader;
use tracing::instrument;
use tracing::trace;
use tracing::debug;
Expand All @@ -22,17 +23,15 @@ use crate::file::FileBytesIterator;
#[derive(Debug, thiserror::Error)]
/// Outer batch representation
/// It's either sucessfully decoded into actual batch or not enough bytes to decode
pub enum BatchHeaderError<R> {
#[error(transparent)]
Io(#[from] IoError),
pub enum BatchHeaderError {
#[error("Not Enough Header{actual_len} {expected_len}")]
NotEnoughHeader {
actual_len: usize,
expected_len: usize,
},
#[error("Not Enough Content {actual_len} {expected_len}")]
NotEnoughContent {
header: Batch<R>, // decoded header
header: BatchHeader,
actual_len: usize,
expected_len: usize,
},
Expand All @@ -57,7 +56,7 @@ where
#[instrument(skip(file))]
pub(crate) async fn read_from<S: StorageBytesIterator>(
file: &mut S,
) -> Result<Option<FileBatchPos<R>>, BatchHeaderError<R>> {
) -> Result<Option<FileBatchPos<R>>> {
let pos = file.get_pos();
trace!(pos, "reading from pos");
let bytes = match file.read_bytes(BATCH_FILE_HEADER_SIZE as u32).await? {
Expand All @@ -81,7 +80,8 @@ where
return Err(BatchHeaderError::NotEnoughHeader {
actual_len: read_len,
expected_len: BATCH_FILE_HEADER_SIZE,
});
}
.into());
}

let mut cursor = Cursor::new(bytes);
Expand Down Expand Up @@ -110,10 +110,11 @@ where
Some(bytes) => bytes,
None => {
return Err(BatchHeaderError::NotEnoughContent {
header: batch,
header: batch.header,
actual_len: 0,
expected_len: content_len,
})
}
.into())
}
};

Expand All @@ -128,10 +129,11 @@ where

if read_len < content_len {
return Err(BatchHeaderError::NotEnoughContent {
header: batch,
header: batch.header,
actual_len: read_len,
expected_len: content_len,
});
}
.into());
}

let mut cursor = Cursor::new(bytes);
Expand Down
4 changes: 2 additions & 2 deletions crates/fluvio-storage/src/validator.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -310,7 +310,7 @@ mod tests {
let validator = LogValidator::default_validate::<LogIndex>(&log_path, None, false, false)
.await
.expect("validate");
assert_eq!(validator.last_valid_offset, BASE_OFFSET);
assert_eq!(validator.leo(), BASE_OFFSET);
}

#[fluvio_future::test]
Expand Down Expand Up @@ -348,7 +348,7 @@ mod tests {
let validator = LogValidator::default_validate::<LogIndex>(&log_path, None, false, true)
.await
.expect("validate");
assert_eq!(validator.last_valid_offset, BASE_OFFSET + 5);
assert_eq!(validator.leo(), BASE_OFFSET + 5);
}

#[fluvio_future::test]
Expand Down