Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

logger: adjust logger to receive logs blobs #1172

Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
40a3a79
codegen: removed the otel layer generation
iulianbarbu Aug 22, 2023
1339888
proto: updated logger proto file
iulianbarbu Aug 22, 2023
7615495
logger: add store_logs RPC impl
iulianbarbu Aug 22, 2023
12fea55
logger: fixed tests
iulianbarbu Aug 23, 2023
6f7c1da
logger: decreased the visibility for Log type
iulianbarbu Aug 23, 2023
616fe2b
logger: remove the unwrap
iulianbarbu Aug 23, 2023
a9c0bd5
proto/logger: differentiate between a fetched/stored log items
iulianbarbu Aug 23, 2023
4c46e57
logger: simplify tests
iulianbarbu Aug 24, 2023
c813a61
runtime: removed tracing/otel dependencies
iulianbarbu Aug 24, 2023
2411767
runtime: removed the tracing dependencies
iulianbarbu Aug 24, 2023
3bae7ab
codegen: cleanup tracing layers and tests
iulianbarbu Aug 24, 2023
9e0fb45
runtime/next: fix compilation
iulianbarbu Aug 24, 2023
fc255d6
logger: remove claim verification on store_logs
iulianbarbu Aug 24, 2023
b1ab984
logger: implemented from_stored for StoredLogItem convertion to Log
iulianbarbu Aug 24, 2023
093e223
codegen: readd a missing import
iulianbarbu Aug 24, 2023
35f737e
codegen: fix tests
iulianbarbu Aug 24, 2023
ceaa9ff
addressed P review
iulianbarbu Aug 24, 2023
2659387
logger: removed integration test comment
iulianbarbu Aug 24, 2023
bf9c9b8
logger/tests: remove the DeploymentPush claim scope
iulianbarbu Aug 24, 2023
5fb7053
address orhun feedback
iulianbarbu Aug 24, 2023
ae860f5
logger: fixed error message typo
iulianbarbu Aug 24, 2023
67781d4
address Johan review
iulianbarbu Aug 24, 2023
53e7369
logger/proto: remove dedup protobuf defs
iulianbarbu Aug 24, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
logger: fixed tests
  • Loading branch information
iulianbarbu committed Aug 23, 2023
commit 12fea556717f4b8b99a79a266cb3eb346b5b4332
16 changes: 13 additions & 3 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

5 changes: 1 addition & 4 deletions logger/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -25,8 +25,7 @@ tokio = { workspace = true, features = ["rt-multi-thread"] }
tokio-stream = { workspace = true }
tonic = { workspace = true }
tracing = { workspace = true }
tracing-subscriber = { workspace = true, features = ["default"] }
tracing-opentelemetry = { workspace = true }
tracing-subscriber = { workspace = true, features = ["default", "json"] }
iulianbarbu marked this conversation as resolved.
Show resolved Hide resolved

[dependencies.shuttle-common]
workspace = true
Expand All @@ -40,5 +39,3 @@ portpicker = { workspace = true }
pretty_assertions = { workspace = true }
serde_json = { workspace = true }
shuttle-common-tests = { workspace = true }
opentelemetry-otlp = { workspace = true }
opentelemetry = { workspace = true }
5 changes: 2 additions & 3 deletions logger/migrations/0000_init.sql
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
CREATE TABLE IF NOT EXISTS logs (
deployment_id TEXT, -- The deployment that this log line pertains to.
shuttle_service_name TEXT, -- The shuttle service which created this log.
timestamp INTEGER, -- Unix epoch timestamp.
level INTEGER, -- The log level
fields TEXT -- Log fields object.
tx_timestamp INTEGER, -- Unix epoch timestamp.
iulianbarbu marked this conversation as resolved.
Show resolved Hide resolved
data BLOB -- Log fields object.
);
8 changes: 4 additions & 4 deletions logger/src/dal.rs
Original file line number Diff line number Diff line change
Expand Up @@ -120,10 +120,10 @@ impl Dal for Sqlite {

#[derive(Clone, Debug, FromRow)]
pub struct Log {
pub(crate) deployment_id: String,
pub(crate) shuttle_service_name: String,
pub(crate) tx_timestamp: DateTime<Utc>,
pub(crate) data: Vec<u8>,
pub deployment_id: String,
pub shuttle_service_name: String,
pub tx_timestamp: DateTime<Utc>,
pub data: Vec<u8>,
}

impl From<Log> for LogItem {
Expand Down
4 changes: 2 additions & 2 deletions logger/src/lib.rs
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
use async_broadcast::Sender;
use async_trait::async_trait;
use chrono::{DateTime, NaiveDateTime, Utc};
use dal::{Dal, DalError, Log};
use dal::{Dal, DalError};
use shuttle_common::{backends::auth::VerifyClaim, claims::Scope};
use shuttle_proto::logger::{
logger_server::Logger, LogItem, LogsRequest, LogsResponse, StoreLogsRequest, StoreLogsResponse,
Expand All @@ -14,6 +14,7 @@ use tonic::{Request, Response, Status};
pub mod args;
mod dal;

pub use dal::Log;
pub use dal::Sqlite;

/// A wrapper to capture any error possible with this service
Expand Down Expand Up @@ -62,7 +63,6 @@ where

let request = request.into_inner();
let logs = request.logs;

if !logs.is_empty() {
_ = self
.logs_tx
Expand Down
1 change: 0 additions & 1 deletion logger/src/main.rs
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
use std::time::Duration;

use clap::Parser;
use opentelemetry_proto::tonic::collector::trace::v1::trace_service_server::TraceServiceServer;
use shuttle_common::backends::{
auth::{AuthPublicKey, JwtAuthenticationLayer},
tracing::{setup_tracing, ExtractPropagationLayer},
Expand Down
Loading