Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

dbsp: test using window to implement SQL now. #2015

Merged
merged 1 commit into from
Jul 10, 2024
Merged

dbsp: test using window to implement SQL now. #2015

merged 1 commit into from
Jul 10, 2024

Conversation

ryzhyk
Copy link
Contributor

@ryzhyk ryzhyk commented Jul 10, 2024 •
edited
Loading

Added a test that illustrates how to efficiently implement the following SQL using the DBSP window operator:

SELECT
  user,
  COUNT(*)
FROM transactions
WHERE transactions.time >= now() - 1000
GROUP BY user

Is this a user-visible change (yes/no): ___

dbsp: test using window to implement SQL now.
8417f7c 
Added a test that illustrates how to efficiently implement the following
SQL using `window`:

```sql
SELECT
  user,
  COUNT(*)
FROM transactions
WHERE transactions.time >= now() - 1000
GROUP BY user
```

Signed-off-by: Leonid Ryzhyk <leonid@feldera.com>
@ryzhyk ryzhyk added the DBSP core Related to the core DBSP library label Jul 10, 2024
@ryzhyk ryzhyk requested a review from mihaibudiu July 10, 2024 07:04
@mihaibudiu
Copy link
Collaborator

The SQL program should also include transactions.time <= now()

mihaibudiu
mihaibudiu approved these changes Jul 10, 2024
View reviewed changes
Copy link
Collaborator

@mihaibudiu mihaibudiu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not sure how easily it will be to generate code like this from the compiler. We should think about a general optimization for this discovering a pattern that can be rewritten as a window, that can be applied generally.

@ryzhyk
Copy link
Contributor Author

ryzhyk commented Jul 10, 2024

I am not sure how easily it will be to generate code like this from the compiler. We should think about a general optimization for this discovering a pattern that can be rewritten as a window, that can be applied generally.

yeah, the general pattern I think is a filter with a monotonically growing bound on a collection. But yeah, it's not trivial to detect.

@ryzhyk ryzhyk merged commit 74d37a5 into main Jul 10, 2024
5 checks passed
@ryzhyk ryzhyk deleted the dbsp_now branch July 10, 2024 18:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@mihaibudiu mihaibudiu mihaibudiu approved these changes

Assignees
No one assigned
Labels
DBSP core Related to the core DBSP library
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@ryzhyk @mihaibudiu