Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Serialize the PackedAtlas fields using protocol buffer format #107

Merged
merged 91 commits into from
May 24, 2018
+4,009 −108
Merged

Serialize the PackedAtlas fields using protocol buffer format #107

Changes from 1 commit
Commits
Show all changes
91 commits
Select commit Hold shift + click to select a range
90eb8db
Moved all the naive proto atlas files to a subfolder.
lucaspcram Apr 5, 2018
21a6118
Reorganized proto-related code into new packages.
lucaspcram Apr 5, 2018
a31bc4d
Initial prototyping for the ProtoSerializable interface design strate…
lucaspcram Apr 5, 2018
0d5c2a3
Added transient qualifier to adapter in LongArray.
lucaspcram Apr 5, 2018
418d13b
Of course my one-line commit breaks checkstyle.
lucaspcram Apr 5, 2018
184b64e
Integrated some changes from PR notes.
lucaspcram Apr 5, 2018
f18c5a6
Added support for names to ProtoLongArray
lucaspcram Apr 5, 2018
e98d08d
Fixed adapter test
lucaspcram Apr 5, 2018
cd6ab6e
Added code to handle case where LongArray has a null name.
lucaspcram Apr 5, 2018
aba6ba2
Fixed an abbreviation
lucaspcram Apr 6, 2018
716b1f9
Implemented LongToLongMap serialization
lucaspcram Apr 6, 2018
f8ccb61
Implemented proto code for LongArrayOfArrays
lucaspcram Apr 6, 2018
539a103
Added checks to ensure the provided types for serialization fit withi…
lucaspcram Apr 6, 2018
85ee302
Implemented IntegerArrayOfArrays serialization
lucaspcram Apr 6, 2018
2b163e5
Modified test cases to run faster.
lucaspcram Apr 9, 2018
563374b
Implemented ByteArrayOfArrays serialization.
lucaspcram Apr 9, 2018
74c162a
Directory refactoring of proto files.
lucaspcram Apr 9, 2018
7b8c48f
Implemented AtlasMetaData serialization
lucaspcram Apr 9, 2018
9df1ddc
Added hashCode implementation to AtlasMetaData and AtlasSize to appea…
lucaspcram Apr 9, 2018
53fbbe1
Implemented serialization for the dreaded LongToLongMultiMap
lucaspcram Apr 9, 2018
a5c17a2
Implemented serialization for IntegerDictionary. See commit comments …
lucaspcram Apr 10, 2018
3affca4
Removed extra unnecessary field in IntegerDictionary
lucaspcram Apr 10, 2018
39c0ba9
Fixed wrong Assert import in ProtoIntegerStringDictionaryAdapterTest
lucaspcram Apr 10, 2018
0288db4
Changed size method of IntegerDictionary to use index instead of hash…
lucaspcram Apr 10, 2018
4ae2bdc
Changed an exception check in add() to more closely match the excepti…
lucaspcram Apr 11, 2018
952967d
Did some refactoring surrounding the IntegerArrayOfArrays
lucaspcram Apr 12, 2018
16d5cee
Implemented serialization for PackedTagStore
lucaspcram Apr 12, 2018
5ab2bab
Added an author tag for classes I changed. So people know who to comp…
lucaspcram Apr 12, 2018
1f6d637
Made some performance tweaks relating to block size in the ProtoLongA…
lucaspcram Apr 13, 2018
7b277a2
Implemented serialization for PolyLineArray
lucaspcram Apr 13, 2018
bdff212
Implemented serialization for PolygonArray
lucaspcram Apr 13, 2018
c31628f
Added default equals and hashCode implementations to LargeArray and L…
lucaspcram Apr 13, 2018
4445035
Added equals and hashCode implementation to ByteArrayOfArrays
lucaspcram Apr 13, 2018
6b33650
Updated test for ProtoIntegerArrayOfArraysAdapter
lucaspcram Apr 13, 2018
9a2ed9f
Fixed annoying bug with IntegerArrayOfArrays converter
lucaspcram Apr 13, 2018
457e93b
Added serialization handler code to Atlas, BareAtlas, and PackedAtlas…
lucaspcram Apr 13, 2018
9c223c0
Implemented equals and hashCode for IntegerDictionary
lucaspcram Apr 16, 2018
c416534
Implemented equals and hashCode for LongArray.
lucaspcram Apr 16, 2018
d4793c8
Implemented equals and hashCode for LongArrayOfArrays.
lucaspcram Apr 16, 2018
6e67b2e
Implemented equals and hashCode for LongToLongMap.
lucaspcram Apr 16, 2018
1ed1adc
Implemented equals and hashCode for LongToLongMultiMap.
lucaspcram Apr 16, 2018
e2e98ac
Change to Optional unpacking per PR comment
lucaspcram Apr 16, 2018
e8a9e68
Finished equals() and hashCode() for the rest of the packed types.
lucaspcram Apr 16, 2018
18e0dd9
More changes per PR notes.
lucaspcram Apr 16, 2018
4fcb2f6
Added author tags for blame purposes.
lucaspcram Apr 16, 2018
2d030d5
Implemented a Builder class for AtlasSize to make constructor easier …
lucaspcram Apr 17, 2018
a9207ae
Fixed spotless.
lucaspcram Apr 17, 2018
bacf1c5
Formatting fixes in PackedAtlas
lucaspcram Apr 17, 2018
9cf09ad
Added a full suite test set for the new proto code.
lucaspcram Apr 17, 2018
45a683f
Implemented lots of PR tips and nits.
lucaspcram Apr 17, 2018
0c80984
One more nit change.
lucaspcram Apr 17, 2018
852eda4
Some updates to author tags.
lucaspcram Apr 17, 2018
0e90c4c
More tests added
lucaspcram Apr 17, 2018
6893fb8
More author tag cleanup
lucaspcram Apr 18, 2018
ce6f86d
Implemented nullary constructors for PackedAtlas array types. These a…
lucaspcram Apr 18, 2018
cc6a3e6
Changed hardcoded field names in PackedAtlasSerializer
lucaspcram Apr 18, 2018
f552e60
Refactored name of default serialization enum value.
lucaspcram Apr 18, 2018
dfcac11
Implemented PackedAtlas deserialization. You can now do full serializ…
lucaspcram Apr 19, 2018
b738ebc
Added test that uses AtlasDelta for comparisons
lucaspcram Apr 19, 2018
109d5b8
Refactored ProtoIntegerStringDictionaryAdapter to use reflection.
lucaspcram Apr 20, 2018
53957be
Added some tests to expose known bugs with the adapters. From now on,…
lucaspcram Apr 20, 2018
55f3f62
More tests, now for PackedTagStore
lucaspcram Apr 20, 2018
2c2bcbb
Fixed hashCode in IntegerDictionary to handle null entries gracefully
lucaspcram Apr 20, 2018
5107fbb
Added the proposed null sentinel
lucaspcram Apr 26, 2018
d381fcc
Implemented null sentinel fix. Empty tags should now work properly.
lucaspcram Apr 26, 2018
7f65eb9
Removed bounds calls before atlas loading
lucaspcram Apr 26, 2018
2ba285e
Addressed some PR comments
lucaspcram Apr 26, 2018
469a0a7
Tweak to a unit test to see travis results
lucaspcram Apr 26, 2018
f3730f1
Change to build.gradle to see better test log output
lucaspcram Apr 26, 2018
436bb11
Tweaked a sanity check in LargeArray to be more permissive
lucaspcram Apr 26, 2018
3439fc2
Updated exception message from last commit
lucaspcram Apr 26, 2018
4f283ec
Implemented null handling for AtlasMetaData
lucaspcram Apr 26, 2018
5feb8a4
Handling null fields in LongToLongMap
lucaspcram Apr 26, 2018
0d0bc37
Implemented null handling for LongArrayOfArrays adapter
lucaspcram Apr 27, 2018
dac87c2
Implemented null handling for LongToLongMultiMap adapter
lucaspcram Apr 27, 2018
c171849
Implemented null handling for PolyLineArray adapter
lucaspcram Apr 27, 2018
7fa3d56
Implemented null handling for PolygonArray adapter
lucaspcram Apr 27, 2018
a0ae049
Implemented null handling for ByteArrayOfArrays and IntegerArrayOfArr…
lucaspcram Apr 27, 2018
e91a3b9
Some refactors in the ProtoAtlas code.
lucaspcram Apr 30, 2018
c75b7c8
Updated spotless. This is a case where Eclipse save-actions do not al…
lucaspcram Apr 30, 2018
8094820
Fixed size hacks in LongToLongMap adapters
lucaspcram Apr 30, 2018
4210f44
Small refactor to ternary logic in LongToLong*Map adapters
lucaspcram Apr 30, 2018
3a69f03
Fixed comment typo
lucaspcram Apr 30, 2018
6ea5661
Fixed array allocation code in some of the adapters. This should prev…
lucaspcram Apr 30, 2018
8d513c7
Implemented auto detection of serialized format.
lucaspcram May 1, 2018
480fe1f
Added checks to the adapters that verify array size
lucaspcram May 1, 2018
e42306e
Refactored loop syntax and comments.
lucaspcram May 1, 2018
59ea1a8
Fixed atlas format autodetection.
lucaspcram May 2, 2018
755a1fd
Set default serialization to Java. Ready for merge.
lucaspcram May 2, 2018
89eb523
Changed default serialization scheme to protobuf.
lucaspcram May 10, 2018
b22df1c
Fixed known bug with transient format fields in PackedAtlas. Changed …
lucaspcram May 14, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Directory refactoring of proto files.
@lucaspcram
lucaspcram committed May 7, 2018
commit 74c162acd840e2c57e39a0d70e60990d0063f5fa
4 changes: 2 additions & 2 deletions src/main/proto/protoatlas/Area.proto → src/main/proto/Area.proto
View file
Original file line number Diff line number Diff line change
@@ -5,8 +5,8 @@ option java_outer_classname = "ProtoAreaWrapper";

package org.openstreetmap.atlas.proto;

import "protoatlas/Location.proto";
import "protoatlas/Tag.proto";
import "Location.proto";
import "Tag.proto";

message ProtoArea {
optional int64 id = 1;
12 changes: 6 additions & 6 deletions ...ain/proto/protoatlas/AtlasContainer.proto → src/main/proto/AtlasContainer.proto
View file
Original file line number Diff line number Diff line change
@@ -5,12 +5,12 @@ option java_outer_classname = "ProtoAtlasContainerWrapper";

package org.openstreetmap.atlas.proto;

import "protoatlas/Point.proto";
import "protoatlas/Line.proto";
import "protoatlas/Area.proto";
import "protoatlas/Node.proto";
import "protoatlas/Edge.proto";
import "protoatlas/Relation.proto";
import "Point.proto";
import "Line.proto";
import "Area.proto";
import "Node.proto";
import "Edge.proto";
import "Relation.proto";

/*
* NOTE: the 'repeated' directive is cpmpiled into Java as an ArrayList.
4 changes: 2 additions & 2 deletions src/main/proto/protoatlas/Edge.proto → src/main/proto/Edge.proto
View file
Original file line number Diff line number Diff line change
@@ -5,8 +5,8 @@ option java_outer_classname = "ProtoEdgeWrapper";

package org.openstreetmap.atlas.proto;

import "protoatlas/Location.proto";
import "protoatlas/Tag.proto";
import "Location.proto";
import "Tag.proto";

message ProtoEdge {
optional int64 id = 1;
4 changes: 2 additions & 2 deletions src/main/proto/protoatlas/Line.proto → src/main/proto/Line.proto
View file
Original file line number Diff line number Diff line change
@@ -5,8 +5,8 @@ option java_outer_classname = "ProtoLineWrapper";

package org.openstreetmap.atlas.proto;

import "protoatlas/Location.proto";
import "protoatlas/Tag.proto";
import "Location.proto";
import "Tag.proto";

message ProtoLine {
optional int64 id = 1;
2 changes: 1 addition & 1 deletion src/main/proto/protoatlas/Location.proto → src/main/proto/Location.proto
View file
Original file line number Diff line number Diff line change
@@ -5,7 +5,7 @@ option java_outer_classname = "ProtoLocationWrapper";

package org.openstreetmap.atlas.proto;

import "protoatlas/Tag.proto";
import "Tag.proto";

message ProtoLocation {
optional int32 latitude = 1;
4 changes: 2 additions & 2 deletions src/main/proto/protoatlas/Node.proto → src/main/proto/Node.proto
View file
Original file line number Diff line number Diff line change
@@ -5,8 +5,8 @@ option java_outer_classname = "ProtoNodeWrapper";

package org.openstreetmap.atlas.proto;

import "protoatlas/Location.proto";
import "protoatlas/Tag.proto";
import "Location.proto";
import "Tag.proto";

message ProtoNode {
optional int64 id = 1;
4 changes: 2 additions & 2 deletions src/main/proto/protoatlas/Point.proto → src/main/proto/Point.proto
View file
Original file line number Diff line number Diff line change
@@ -5,8 +5,8 @@ option java_outer_classname = "ProtoPointWrapper";

package org.openstreetmap.atlas.proto;

import "protoatlas/Location.proto";
import "protoatlas/Tag.proto";
import "Location.proto";
import "Tag.proto";

message ProtoPoint {
optional int64 id = 1;
0 ...in/proto/packedatlas/ProtoByteArray.proto → src/main/proto/ProtoByteArray.proto
View file
File renamed without changes.
2 changes: 1 addition & 1 deletion .../packedatlas/ProtoByteArrayOfArrays.proto → src/main/proto/ProtoByteArrayOfArrays.proto
View file
Original file line number Diff line number Diff line change
@@ -5,7 +5,7 @@ option java_outer_classname = "ProtoByteArrayOfArraysWrapper";

package org.openstreetmap.atlas.proto;

import "packedatlas/ProtoByteArray.proto";
import "ProtoByteArray.proto";

message ProtoByteArrayOfArrays {
optional string name = 1;
0 ...proto/packedatlas/ProtoIntegerArray.proto → src/main/proto/ProtoIntegerArray.proto
View file
File renamed without changes.
2 changes: 1 addition & 1 deletion ...ckedatlas/ProtoIntegerArrayOfArrays.proto → ...ain/proto/ProtoIntegerArrayOfArrays.proto
View file
Original file line number Diff line number Diff line change
@@ -5,7 +5,7 @@ option java_outer_classname = "ProtoIntegerArrayOfArraysWrapper";

package org.openstreetmap.atlas.proto;

import "packedatlas/ProtoIntegerArray.proto";
import "ProtoIntegerArray.proto";

message ProtoIntegerArrayOfArrays {
optional string name = 1;
0 ...in/proto/packedatlas/ProtoLongArray.proto → src/main/proto/ProtoLongArray.proto
View file
File renamed without changes.
2 changes: 1 addition & 1 deletion .../packedatlas/ProtoLongArrayOfArrays.proto → src/main/proto/ProtoLongArrayOfArrays.proto
View file
Original file line number Diff line number Diff line change
@@ -5,7 +5,7 @@ option java_outer_classname = "ProtoLongArrayOfArraysWrapper";

package org.openstreetmap.atlas.proto;

import "packedatlas/ProtoLongArray.proto";
import "ProtoLongArray.proto";

message ProtoLongArrayOfArrays {
optional string name = 1;
2 changes: 1 addition & 1 deletion ...roto/packedatlas/ProtoLongToLongMap.proto → src/main/proto/ProtoLongToLongMap.proto
View file
Original file line number Diff line number Diff line change
@@ -5,7 +5,7 @@ option java_outer_classname = "ProtoLongToLongMapWrapper";

package org.openstreetmap.atlas.proto;

import "packedatlas/ProtoLongArray.proto";
import "ProtoLongArray.proto";

message ProtoLongToLongMap {
optional string name = 1;
2 changes: 1 addition & 1 deletion src/main/proto/protoatlas/Relation.proto → src/main/proto/Relation.proto
View file
Original file line number Diff line number Diff line change
@@ -5,7 +5,7 @@ option java_outer_classname = "ProtoRelationWrapper";

package org.openstreetmap.atlas.proto;

import "protoatlas/Tag.proto";
import "Tag.proto";

message ProtoRelation {
optional int64 id = 1;
0 src/main/proto/protoatlas/Tag.proto → src/main/proto/Tag.proto
View file
File renamed without changes.