Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

修正快照的接口问题 #8

Merged
merged 68 commits into from
Jan 13, 2021
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
68 commits
Select commit Hold shift + click to select a range
06f9219
增加错误基类、简单的记录接口
kaaass Jan 1, 2021
e9fd13f
增加.gitignore
kaaass Jan 1, 2021
7e3e2b4
完成记录的mock
kaaass Jan 1, 2021
f6edd17
增加记录接口的简易单元测试
kaaass Jan 1, 2021
910fc57
Merge branch 'dev' into feature-mock
kaaass Jan 1, 2021
481bea3
引入Slf4j、log4j、jbbp外部库
kaaass Jan 1, 2021
eb3d19c
[+]add table mock and unit test
KveinAxel Jan 1, 2021
50a8ad6
Merge branch 'feature-mock' into feature-table
KveinAxel Jan 2, 2021
1776095
Merge pull request #2 from kaaass/feature-table
KveinAxel Jan 2, 2021
d8aa315
增加Travis配置
kaaass Jan 2, 2021
c89d9c0
Merge remote-tracking branch 'origin/feature-mock' into feature-mock
kaaass Jan 2, 2021
a2d5fa2
fix: transform java14 to java11
KveinAxel Jan 2, 2021
2491186
更新Gradle配置以允许Scan ToS
kaaass Jan 2, 2021
b1b8e33
删除冗余Travis配置
kaaass Jan 2, 2021
f843d2f
README增加Travis Badge
kaaass Jan 2, 2021
8211faf
增加UTF-8 gradle配置
kaaass Jan 8, 2021
2e50aba
对于数据项管理的mock和测试部分
xwinks Jan 8, 2021
0e6bddf
修改测试用例的文件名
KveinAxel Jan 8, 2021
9240fe7
index mock
DctorWei1314 Jan 8, 2021
845c7f9
增加事务的Mock
Criterionist Jan 8, 2021
37a86d8
Merge remote-tracking branch 'origin/feature-mock' into feature-mock
Criterionist Jan 8, 2021
642c52e
testPage
xiaoxuanNLP Jan 9, 2021
96fc8dd
Merge branch 'feature-mock' of https://github.com/kaaass/rumbase_java…
xiaoxuanNLP Jan 9, 2021
2797554
index mock plus test
DctorWei1314 Jan 9, 2021
3079b1f
Merge remote-tracking branch 'origin/feature-mock' into feature-mock
DctorWei1314 Jan 9, 2021
be73a7d
对于数据项管理的mock和测试部分的修改
xwinks Jan 9, 2021
aa6faa7
Merge remote-tracking branch 'origin/feature-mock' into feature-mock
xwinks Jan 9, 2021
35f6267
testPage1.01
xiaoxuanNLP Jan 9, 2021
015b1d1
Merge remote-tracking branch 'origin/feature-mock' into feature-mock
xiaoxuanNLP Jan 9, 2021
4d046cc
dataitem模块修改建议
kaaass Jan 9, 2021
934e336
Merge remote-tracking branch 'origin/feature-mock' into feature-mock
kaaass Jan 9, 2021
4f79206
dataitem模块修改建议
kaaass Jan 9, 2021
dd92f0e
Merge remote-tracking branch 'origin/feature-mock' into feature-mock
Criterionist Jan 9, 2021
2f0874e
修正编码风格
Criterionist Jan 9, 2021
2c4cbbb
对于数据项管理的mock和测试部分
xwinks Jan 9, 2021
0242ee6
对于数据项管理的mock和测试部分
xwinks Jan 9, 2021
fd25691
对于数据项管理的mock和测试部分
xwinks Jan 9, 2021
6029610
Index审计、修改建议
kaaass Jan 9, 2021
03f8b09
Merge remote-tracking branch 'origin/feature-mock' into feature-mock
kaaass Jan 9, 2021
019036d
增加JBBP的使用示例
kaaass Jan 9, 2021
5656c6f
page模块审计、修改建议
kaaass Jan 9, 2021
4c3e3c2
删除table模块,更改record模块
kaaass Jan 9, 2021
c381c2e
recovery模块审计
kaaass Jan 9, 2021
1fc4b5c
transaction模块审计
kaaass Jan 9, 2021
c2b5606
对于日志恢复的接口和测试
xwinks Jan 9, 2021
69845d0
完善事务模块接口
Criterionist Jan 9, 2021
4974a28
Merge remote-tracking branch 'origin/feature-mock' into feature-mock
Criterionist Jan 9, 2021
6511024
testPage1.02
xiaoxuanNLP Jan 9, 2021
e590bf2
Merge remote-tracking branch 'origin/feature-mock' into feature-mock
xiaoxuanNLP Jan 9, 2021
7e6ccd9
testPage1.03
xiaoxuanNLP Jan 10, 2021
e50c917
复审recovery,tx
kaaass Jan 10, 2021
795ecef
格式化代码
kaaass Jan 10, 2021
26f1626
更新README
kaaass Jan 10, 2021
8fefaa0
Merge branch 'dev' into feature-mock
kaaass Jan 10, 2021
ecbe27f
修改异常名称
xwinks Jan 11, 2021
7fe37c4
修正编译失败
kaaass Jan 11, 2021
f8c5bff
Merge branch 'dev' into feature-mock
kaaass Jan 12, 2021
3aa86e3
表创建、表头解析以及页头解析
xwinks Jan 13, 2021
1cebb4a
Merge branch 'feature-dataitem' into feature-mock
xwinks Jan 13, 2021
476e115
Merge remote-tracking branch 'origin/feature-mock' into feature-mock
xwinks Jan 13, 2021
78f7433
索引审计修改
DctorWei1314 Jan 13, 2021
39870f8
Merge remote-tracking branch 'origin/feature-mock' into feature-mock
DctorWei1314 Jan 13, 2021
5a84ca0
RumbaseException增加错误原因
kaaass Jan 13, 2021
f6a5fe0
Merge to dev
kaaass Jan 13, 2021
98da699
完善事务模块接口,增加事务模块测试用例
Criterionist Jan 13, 2021
668121e
修正事务接口
Criterionist Jan 13, 2021
a46b8c0
Merge remote-tracking branch 'origin/dev' into feature-mock
kaaass Jan 13, 2021
25930c7
修正事务接口
Criterionist Jan 13, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
testPage1.03
  • Loading branch information
xiaoxuanNLP committed Jan 10, 2021
commit 7e6ccd9ee5dae704a2cb99107d7636e5442e4ce3
4 changes: 2 additions & 2 deletions src/main/java/net/kaaass/rumbase/page/Page.java
Original file line number Diff line number Diff line change
Expand Up @@ -66,7 +66,7 @@ default InputStream getData() {
* 将字节数据写入页
*
* @param data 待写入字节数据
* @throws PageException TODO
* @throws PageException 若写入的数据超出页的范围则抛出异常
*/
default void writeData(byte[] data) throws PageException {
patchData(0, data);
Expand All @@ -77,7 +77,7 @@ default void writeData(byte[] data) throws PageException {
*
* @param offset 页内偏移值,以字节为单位
* @param data 待写入数据
* @throws PageException TODO
* @throws PageException 若写入的数据超出页的范围则抛出异常
*/
void patchData(int offset, byte[] data) throws PageException;

Expand Down
Empty file.
6 changes: 3 additions & 3 deletions src/main/java/net/kaaass/rumbase/page/PageManager.java
Original file line number Diff line number Diff line change
Expand Up @@ -13,11 +13,11 @@ public class PageManager {
public static int BYTE_BUFFER_SIZE = 1024 * 4 * PAGE_NUM;

/**
* TODO 文档
* 取数据库文件生成文件管理的对象
*
* @param filepath
* @param filepath 每个表文件路径
* @return
* @throws FileException
* @throws FileException 若文件不存在则创建,创建过程中出现错误会抛出错误
*/
public static PageStorage fromFile(String filepath) throws FileException {
return new MockPageStorage(filepath);
Expand Down
3 changes: 1 addition & 2 deletions src/main/java/net/kaaass/rumbase/page/PageStorage.java
Original file line number Diff line number Diff line change
Expand Up @@ -13,9 +13,8 @@ public interface PageStorage {
* 获取该页存储中的某一页
* @param pageId 页号
* @return 该页页对象
* @throws FileException TODO 为什么要抛出FileException?此步骤应该隐藏文件接口的任何细节
*/
Page get(long pageId) throws FileException;
Page get(long pageId);

/**
* 将页存储中的所有脏页写回文件
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@
/**
* E9002 文件异常
* <p>
* E9002-1 创建文件失败 TODO 不一致
* E9002-1 回写数据偏移与大小之和超过规定
*
* @author XuanLaoYee
*/
Expand Down
23 changes: 1 addition & 22 deletions src/main/java/net/kaaass/rumbase/page/mock/MockPage.java
Original file line number Diff line number Diff line change
Expand Up @@ -16,20 +16,17 @@ public class MockPage implements Page {
boolean dirty;
int pinned = 0;
String filepath;
private final ReentrantLock lock;

MockPage(byte[] data, long pageId, String filepath) {
this.data = data;
this.pageId = pageId;
this.lock = new ReentrantLock();
this.dirty = false;
this.filepath = filepath;
}

@Override
public byte[] getDataBytes() {
synchronized (this) {
pin(); // TODO 错误使用
return data;
}
}
Expand All @@ -43,25 +40,7 @@ public void patchData(int offset, byte[] data) throws PageException {
}

@Override
public void flush() throws FileException {
File file = new File(this.filepath);
try {
RandomAccessFile out = new RandomAccessFile(file, "rw");
try {
out.seek((PageManager.FILE_HEAD_SIZE + this.pageId) * PageManager.PAGE_SIZE);
}catch(Exception e){
throw new FileExeception(4);
}
try {
out.write(data);
} catch (Exception e) {
throw new FileException(2);
}
out.close();
} catch (Exception e) {
throw new FileException(3);
}
}
public void flush() throws FileException { }

@Override
public void pin() {
Expand Down
45 changes: 12 additions & 33 deletions src/main/java/net/kaaass/rumbase/page/mock/MockPageStorage.java
Original file line number Diff line number Diff line change
Expand Up @@ -21,61 +21,40 @@ public class MockPageStorage implements PageStorage {

public MockPageStorage(String filepath) throws FileException {
this.filepath = filepath;
File file = new File(filepath);
pageMap = new HashMap<>();
if (!file.exists()) {
try {
file.createNewFile();
OutputStream os = null;
try {
os = new FileOutputStream(file, true);
byte[] data = new byte[PageManager.PAGE_SIZE];
for (int i = 0; i < 10; i++) {
os.write(data, 0, data.length);
os.flush();
}
} catch (Exception e) {
throw new FileException(2);
}
} catch (Exception e) {
throw new FileException(1);
this.fakeFile = new byte[1024 * 4 * 20];
for (int i = 0; i < 20; i++) {
for (int j = 0; j < 1024 * 4; j++) {
this.fakeFile[1024 * 4 * i + j] = (byte)i;
}
}
}

@Override
public Page get(long pageId) throws FileException {
File file = new File(filepath);
public Page get(long pageId) {
//文件会预留5页作为文件头
try {
FileInputStream in = new FileInputStream(file);
byte[] data = new byte[PageManager.PAGE_SIZE];
try {
in.skip((pageId + PageManager.FILE_HEAD_SIZE) * PageManager.PAGE_SIZE);
int readNumber = in.read(data);
if (readNumber < PageManager.PAGE_SIZE) {
throw new FileException(4);
}
} catch (Exception e) {
throw new FileException(4);
}
System.arraycopy(fakeFile, (int) (pageId + PageManager.FILE_HEAD_SIZE)*PageManager.PAGE_SIZE, data, 0, data.length);
Integer tmpId = (int) pageId;
if(pageMap.containsKey(tmpId)){
if (pageMap.containsKey(tmpId)) {
return pageMap.get(tmpId);
}
Page page = new MockPage(data, pageId, this.filepath);
pageMap.put(tmpId, page);
return page;
} catch (Exception e) {
throw new FileException(3);
e.printStackTrace();
}
return null;
}

@Override
public void flush() {
//TODO
}
private Map<Integer,Page> pageMap;

private Map<Integer, Page> pageMap;

private String filepath;
private byte[] fakeFile;
}
37 changes: 19 additions & 18 deletions src/test/java/net/kaaass/rumbase/page/PageStorageTest.java
Original file line number Diff line number Diff line change
Expand Up @@ -8,31 +8,32 @@
import static org.junit.Assert.assertArrayEquals;

/**
* TODO 文档
* TODO Mock的测试不应该和文件有关,本测试仅仅测试接口对上层的意义。而物理文件对上层不可见
* 文件管理测试
*
* @author XuanLaoYee
* @see net.kaaass.rumbase.page.PageStorage
*/
public class PageStorageTest extends TestCase {
public void testGet() {
try {
PageStorage pc = PageManager.fromFile("testFile");
Page p0 = pc.get(0);
p0 = pc.get(0);
Page p3 = pc.get(3);

File file = new File("testFile");
FileInputStream in = new FileInputStream(file);
in.skip((PageManager.FILE_HEAD_SIZE) * PageManager.PAGE_SIZE);
byte[] data0 = new byte[PageManager.PAGE_SIZE];
in.read(data0);
in.close();
assertArrayEquals(data0, p0.getDataBytes());
in = new FileInputStream(file);
in.skip((3 + PageManager.FILE_HEAD_SIZE) * PageManager.PAGE_SIZE);
byte[] data3 = new byte[PageManager.PAGE_SIZE];
in.skip((PageManager.FILE_HEAD_SIZE) * PageManager.PAGE_SIZE);
in.read(data3);
assertArrayEquals(data3, p3.getDataBytes());

byte[] data0 = new byte[1024 * 4];
byte[] data3 = new byte[1024 * 4];
p0.getData().read(data0);
p3.getData().read(data3);

byte[] testData0 = new byte[1024 * 4];
for (int j = 0; j < 1024 * 4; j++) {
testData0[j] = (byte)5;
}
byte[] testData3 = new byte[1024 * 4];
for (int j = 0; j < 1024 * 4; j++) {
testData3[j] = (byte)8;
}
assertArrayEquals(testData0,data0);
assertArrayEquals(testData3,data3);

} catch (Exception e) {
e.printStackTrace();
Expand Down
48 changes: 14 additions & 34 deletions src/test/java/net/kaaass/rumbase/page/PageTest.java
Original file line number Diff line number Diff line change
Expand Up @@ -15,28 +15,26 @@
*/
public class PageTest extends TestCase {

// TODO Mock的测试不应该和文件有关,本测试仅仅测试接口对上层的意义。而物理文件对上层不可见
public void testGetData() {
try {
PageStorage pc = PageManager.fromFile("testFile");
Page p0 = pc.get(0);
Page p3 = pc.get(3);
byte[] data0 = new byte[1024 * 4];
byte[] data3 = new byte[1024 * 4];
p0.getData().read(data0);
p3.getData().read(data3);
byte[] testData0 = new byte[1024 * 4];
for (int j = 0; j < 1024 * 4; j++) {
testData0[j] = (byte)5;
}
byte[] testData3 = new byte[1024 * 4];
for (int j = 0; j < 1024 * 4; j++) {
testData3[j] = (byte)8;
}
assertArrayEquals(testData0,data0);
assertArrayEquals(testData3,data3);

File file = new File("testFile");
FileInputStream in0 = new FileInputStream(file);
in0.skip((PageManager.FILE_HEAD_SIZE) * PageManager.PAGE_SIZE);
byte[] data0 = new byte[PageManager.PAGE_SIZE];
int readNumber1 = in0.read(data0);
in0.close();
assertArrayEquals(data0, p0.getDataBytes());
assertEquals(PageManager.PAGE_SIZE, readNumber1);
FileInputStream in3 = new FileInputStream(file);
in3.skip((3 + PageManager.FILE_HEAD_SIZE) * PageManager.PAGE_SIZE);
byte[] data3 = new byte[PageManager.PAGE_SIZE];
int readNumber3 = in3.read(data3);
assertArrayEquals(data3, p3.getDataBytes());
assertEquals(PageManager.PAGE_SIZE, readNumber3);
in3.close();
} catch (Exception e) {
e.printStackTrace();
}
Expand Down Expand Up @@ -82,22 +80,4 @@ public void testPatchData() {
e.printStackTrace();
}
}

public void testFlush() {
byte[] data = new byte[PageManager.PAGE_SIZE];
for (int i = 0; i < data.length; i++) {
data[i] = (byte) (i % 120);
}

try {
PageStorage pc = PageManager.fromFile("testFile");
Page p0 = pc.get(0);
p0.writeData(data);
p0.flush();
p0 = pc.get(0);
assertArrayEquals(data, p0.getDataBytes());
} catch (Exception e) {
e.printStackTrace();
}
}
}