Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG]: Some events are being lost and not recorded in Firebase. #5063

Open
seanlip opened this issue Jun 19, 2023 · 0 comments
Open

[BUG]: Some events are being lost and not recorded in Firebase. #5063

seanlip opened this issue Jun 19, 2023 · 0 comments
Labels
bug End user-perceivable behaviors which are not desirable. Impact: Low Low perceived user impact (e.g. edge cases). Work: Medium The means to find the solution is clear, but it isn't at good-first-issue level yet.

Comments

@seanlip
Copy link
Member

seanlip commented Jun 19, 2023

Describe the bug
When analyzing data from the Kenya research study, we found discrepancies between the data that was supposed to be logged from the phone and the data received in Firebase. In particular, data was completely missing for some stretches of time. We suspect that some events are being lost.

From further analysis (by comparing data received from Firebase to data logged on the apps), I estimate the event loss rate to be about 17%. This means that 1 in 6 events, on average, are being lost. However, note that, based on the data we have:

  • The loss happens for stretches of events (not interspersed random events). I.e. lost events tend to happen in contiguous batches.
  • The event recorded immediately prior to loss happens shortly before the first event in the 'lost batch', but the event recorded immediately after loss often happens quite a long time after the 'lost batch'. This suggests that perhaps the "not finished last batch" is not being sent.
  • In addition to "last parts of batches aren't sent", I also saw some cases where entire batches were not sent.
  • There is at least one instance, however, where the event recorded immediately after loss happens 4 seconds after the last event that was lost. Interestingly, the lost events are all of the same type (open/close revision card). Here is the relevant segment (timestamps in seconds):
    • 314: submit_answer_context
    • 314: end_card_context
    • 315: start_card_context
    • 339: submit_answer_context
    • 357: exit_exploration_context
    • 364: open_revision_tab
    • Lost events start here
    • 368: open_revision_card
    • 370: close_revision_card
    • 372: open_revision_card
    • 381: open_revision_card
    • 387: open_revision_card
    • 392: open_revision_card
    • 398: open_revision_card
    • 401: open_revision_card
    • 402: close_revision_card
    • Lost events end here
    • 406: open_lessons_tab
    • 407: open_revision_tab
    • 411: open_lessons_tab

To Reproduce
Unfortunately, we don't have clear reproduction steps. The phone were connected to the Internet for a few hours each day, but were mostly offline.

Expected behavior
All events sent by the phone should be recorded in Firebase.

Environment

  • App version (you can get this through system app settings or via the admin controls menu in-app): 0.10-alpha_kenya-31b05b1e78
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug End user-perceivable behaviors which are not desirable. Impact: Low Low perceived user impact (e.g. edge cases). Work: Medium The means to find the solution is clear, but it isn't at good-first-issue level yet.
Development

No branches or pull requests

2 participants