Update binary log proto, add document explaining it

pythias · Feb 10, 2016 · f8c6545 · f8c6545
1 parent 44de6a1
commit f8c6545
Show file tree

Hide file tree

Showing 2 changed files with 81 additions and 5 deletions.
diff --git a/doc/binary-logging.md b/doc/binary-logging.md
@@ -0,0 +1,52 @@
+# Binary Logging
+
+## Format
+
+The log format is described in [this proto file](src/proto/grpc/binary_log/v1alpha/log.proto). It is intended that multiple parts of the call will be logged in separate files, and then correlated by analysis tools using the rpc\_id.
+
+## API
+
+The binary logger will be a separate library from gRPC, in each language that we support. The user will need to explicitly call into the library to generate logs. The following API is an example of what it will approximately look like in C++:
+
+```c++
+// The context provides the method_name, deadline, peer, and metadata contents.
+// direction = CLIENT_SEND
+LogRequestHeaders(ClientContext context);
+// direction = SERVER_RECV
+LogRequestHeaders(ServerContext context);
+
+// The context provides the metadata contents
+// direction = CLIENT_RECV
+LogResponseHeaders(ClientContext context);
+// direction = SERVER_SEND
+LogResponseHeaders(ServerContext context);
+
+// The context provides the metadata contents
+// direction = CLIENT_RECV
+LogStatus(ClientContext context, grpc_status_code code, string details);
+// direction = SERVER_SEND
+LogStatus(ServerContext context, grpc_status_code code, string details);
+
+// The context provides the user data contents
+// direction = CLIENT_SEND
+LogUserData(ClientContext context);
+// direction = SERVER_SEND
+LogUserData(ServerContext context);
+
+// direction = CLIENT_SEND
+LogRequestMessage(ClientContext context, uint32_t length, T message);
+// direction = SERVER_RECV
+LogRequestMessage(ServerContext context, uint32_t length, T message);
+// direction = CLIENT_RECV
+LogResponseMessage(ClientContext context, uint32_t length, T message);
+// direction = SERVER_SEND
+LogResponseMessage(ServerContext context, uint32_t length, T message);
+```
+
+In all of those cases, the `rpc_id` is provided by the context, and each combination of method and context argument type implies a single direction, as noted in the comments.
+
+For the message log functions, the `length` argument indicates the length of the complete message, and the `message` argument may be only part of the complete message, stripped of sensitive material and/or shortened for efficiency.
+
+## Language differences
+
+In other languages, more or less data will need to be passed explicitly as separate arguments. In some languages, for example, the metadata will be separate from the context-like object and will need to be passed as a separate argument.
diff --git a/src/proto/grpc/binary_log/v1alpha/log.proto b/src/proto/grpc/binary_log/v1alpha/log.proto
@@ -45,25 +45,41 @@ message KeyValuePair {
   string value;
 }
 
+// Any sort of metadata that may be sent in either direction during a call
 message Metadata {
-  uint32 rpc_id = 1;
+  // Cryptographically unique identifier, generated on the client and sent
+  // to the server.
+  uint64 rpc_id = 1;
+  // Timestamp of logging the metadata
   google.protobuf.Timestamp timestamp = 2;
   Direction direction = 3;
+  // The actual metadata that is being sent
   repeated KeyValuePair metadata = 4;
 
+  // Initial metadata sent by the client to initiate a request
   message ClientInitialMetadata {
+    // The full method name that is being called
     string method_name = 1;
-    google.protobuf.Timestamp timestamp = 2;
+    // The call's deadline
+    google.protobuf.Timestamp deadline = 2;
+    // The address of the connected peer
+    string peer = 3;
   }
 
+  // Arbitrary key/value pairs specified by the user that are not sent over
+  // the network but are nonetheless useful to log
   message UserData {
   }
 
+  // Initial metadata response sent by the server after accepting the request
   message ServerInitialMetadata {
   }
 
+  // Status sent by the server when closing the call on the server side
   message ServerStatus {
+    // The status code
     uint32 code = 1;
+    // The status details
     string details = 2;
   }
 
@@ -75,10 +91,18 @@ message Metadata {
   }
 }
 
+// A message that is sent during a call
 message Message {
+  // Cryptographically unique identifier, generated on the client and sent
+  // to the server.
   uint32 rpc_id = 1;
-  uint32 sequence_number = 2;
+  // The sequence number of the message. Messages sent by the client and by the
+  // server should have independently incrementing sequence numbers.
+  uint64 sequence_number = 2;
   Direction direction = 3;
-  uint64 length;
-  bytes data;
+  // The length of the complete message.
+  uint32 length = 4;
+  // The contents of the message. May be a prefix instead of the complete
+  // message.
+  bytes data = 5;
 }