Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor download route to support multiple file formats #272

Merged
merged 31 commits into from
Dec 6, 2024

Conversation

devansh-m12
Copy link
Contributor

@devansh-m12 devansh-m12 commented Dec 6, 2024

fixes #268

Added for evaluation. @dinmukhamedm, could you share a sample CSV for the dataset? The sample dataset I'm currently using is: https://gist.github.com/dinmukhamedm/f90286065c3201e6c75fc7d2e3fda704.

Screen.Recording.2024-12-06.at.3.31.37.PM.mov

Important

Refactor download routes and components to support multiple file formats (JSON, CSV) with format validation and selection.

  • Routes:
    • Refactor download/route.ts to download/[format]/route.ts for datasets and evaluations.
    • Add DownloadFormat enum to validate supported formats (JSON for datasets, JSON and CSV for evaluations).
    • Return 400 error for unsupported formats.
  • Components:
    • Update DownloadButton in download-button.tsx to support format selection via dropdown.
    • Modify dataset.tsx and evaluation.tsx to use updated DownloadButton with supportedFormats prop.
  • Misc:
    • Remove fileFormat prop from DownloadButton and replace with supportedFormats array.

This description was created by Ellipsis for e70804f. It will automatically update as commits are pushed.

dinmukhamedm and others added 28 commits October 29, 2024 21:00
Readme fixes and global colors
Update landing and welcome email, contrib experience, minor fixes
Improve onboarding, fix eval page placeholder
Explicit mapping in labels + frontend improvements
add claude-3-5-haiku to pipeline and prices (lmnr-ai#174)
fix evaluations query in labeling queue (lmnr-ai#176)
Middleware auth, traces page efficiency
improve UI for workspaces + internal fixes
Store label values sent from association properties
* initial work to compare evals

* remove unnecessary div

* design

---------

Co-authored-by: Din <dinmukhamed.mailibay@gmail.com>
disable all clickhouse queries for non-full builds (lmnr-ai#213)
update models in pipeline templates to 4o family (lmnr-ai#215)
Fix realtime and manual spans
Semantic search API, blogs, minor fixes
update rust deps and fix blog images (lmnr-ai#246)
Allow disabling tracing, search by spans
Copy link
Member

@dinmukhamedm dinmukhamedm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left a few small comments.

For the dataset, you don't need a sample CSV. Let's do it this way for now – just JSON.stringify three fields: data, target, metadata. With empty strings if there's nothing. We can think of the format later, when we have use cases for this.

@devansh-m12
Copy link
Contributor Author

@dinmukhamedm added requested changes and here the ss for dataset download
Screenshot 2024-12-06 at 11 21 20 PM

@devansh-m12 devansh-m12 marked this pull request as ready for review December 6, 2024 17:52
Copy link
Contributor

@ellipsis-dev ellipsis-dev bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 Looks good to me! Reviewed everything up to e70804f in 1 minute and 58 seconds

More details
  • Looked at 253 lines of code in 5 files
  • Skipped 0 files when reviewing.
  • Skipped posting 3 drafted comments based on config settings.
1. frontend/app/api/projects/[projectId]/datasets/[datasetId]/download/[format]/route.ts:7
  • Draft comment:
    To fully resolve the issue, support for CSV format should be added here, similar to the implementation in the evaluations download route.
  • Reason this comment was not posted:
    Decided after close inspection that this draft comment was likely wrong and/or not actionable:
    The comment is suggesting a feature enhancement rather than pointing out a necessary change or issue with the current code. The current implementation is correct for supporting JSON format, and the comment does not indicate a problem with the existing code. It is more of a suggestion for future improvement, which does not align with the rules for review comments.
    The comment could be seen as a valid suggestion for future improvement, but it does not address a current issue or necessary change in the code. It might be useful for planning but not for the current review.
    While the suggestion could be useful, it does not align with the criteria for necessary code changes in the current review. It is more of a feature request than a code review comment.
    The comment should be deleted as it suggests a feature enhancement rather than addressing a necessary code change.
2. frontend/components/dataset/dataset.tsx:191
  • Draft comment:
    To fully resolve the issue, add 'csv' to the supportedFormats array to allow downloading datasets in CSV format as well.
  • Reason this comment was not posted:
    Decided after close inspection that this draft comment was likely wrong and/or not actionable:
    The comment suggests a feature enhancement rather than addressing a specific issue with the current code. The change in the diff was to convert the format from a string to an array, which might be a preparatory step for supporting multiple formats in the future. However, the comment does not indicate a necessary change or a bug fix, just a suggestion for additional functionality.
    I might be missing the context of whether supporting CSV is a requirement or a planned feature. Without this context, it's hard to determine if the comment is actionable or necessary.
    The comment does not indicate a bug or a necessary change, just a suggestion for additional functionality. Without evidence that CSV support is required, the comment is speculative.
    The comment should be deleted as it suggests a feature enhancement rather than addressing a specific issue with the current code.
3. frontend/app/api/projects/[projectId]/evaluations/[evaluationId]/download/[format]/route.ts:7
  • Draft comment:
    Consider consolidating with the existing DownloadFormat enum and extending it to support CSV format.

  • enum DownloadFormat (route.ts)

  • Reason this comment was not posted:
    Decided after close inspection that this draft comment was likely wrong and/or not actionable:
    The comment suggests a refactor that doesn't seem necessary because the current enum already supports both formats needed. The evidence provided shows that the other enum is not as comprehensive as the one in the current file. Therefore, the comment does not seem to require any code change.
    I might be missing the broader context of how enums are used across the codebase, but based on the provided evidence, the comment seems unnecessary.
    The evidence provided is specific to the enums in question, and it shows that the current implementation is already more comprehensive. Therefore, the comment is not useful.
    The comment is not useful because the current enum already supports the necessary formats, and the suggested consolidation would not add any value.

Workflow ID: wflow_Vmxdd29vig7gxFfQ


You can customize Ellipsis with 👍 / 👎 feedback, review rules, user-specific overrides, quiet mode, and more.

@devansh-m12
Copy link
Contributor Author

here @dinmukhamedm the sample csv for dataset after download

test-92686b44-9ba6-426d-9521-b3cda93a74f7 (1).csv

Copy link
Member

@dinmukhamedm dinmukhamedm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just in case, can you make sure you run pnpm lint:fix and pnpm build. If that changes any files, commit the changes too. LGTM otherwise

@dinmukhamedm dinmukhamedm changed the base branch from main to dev December 6, 2024 18:53
@dinmukhamedm dinmukhamedm merged commit cc82ccb into lmnr-ai:dev Dec 6, 2024
1 check passed
@devansh-m12 devansh-m12 deleted the download-formates branch December 12, 2024 18:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Enable download in both formats: CSV and JSON
3 participants