Fixed Several Exceptions in Workflow Usage #12075

ndhjm · 2024-12-25T08:58:09Z

This pull request includes several changes across multiple files to improve functionality, fix bugs, and enhance code readability. The most important changes include the addition of a new package in the Dockerfile, a fix for file name encoding in HTTP headers, and various bug fixes and improvements in the workflow nodes.

Resolves #12074

Improvements and Bug Fixes:

Dockerfile Update:
- Added libreoffice to the list of packages installed in the Dockerfile to support additional functionalities.
File Name Encoding:
- Fixed the encoding of file names in the Content-Disposition header to handle special characters properly by using urllib.parse.quote.
Tool Parameter Type Fix:
- Corrected a typo in the ToolParameterType class by changing systme-files to system-files.
Encoding Detection:
- Added chardet to detect file encoding dynamically when extracting text from CSV files, improving robustness against various encodings. [1] [2]
Equality Check Fix:
- Fixed the equality check in the _is function to use == instead of is for string comparison.This pull request includes several changes across multiple files to improve functionality and fix bugs. The most important changes include adding a new package to the Dockerfile, fixing a typo in an enum, enhancing file handling in the image preview controller, improving text extraction from CSV files, and updating the question classifier node to handle different query types.

Improvements to Dockerfile:

Added libreoffice installation to the Dockerfile to support additional functionalities.

Bug Fixes:

Fixed a typo in the ToolParameterType enum from "systme-files" to "system-files".
Corrected the _is function to use == instead of is for string comparison in list_operator/node.py.

Enhancements:

Enhanced file handling in the image_preview.py controller by safely encoding filenames for the Content-Disposition header.
Improved text extraction from CSV files by detecting and using the correct file encoding with the chardet library. [1] [2]

Updates to Question Classifier Node:

Updated the question_classifier_node.py to handle different query types, including converting non-string types to JSON strings. [1] [2]This pull request includes several changes to the api module, focusing on enhancements, bug fixes, and improvements to the codebase. The most important changes include adding a new package to the Dockerfile, fixing a typo in an enumeration value, improving file handling in the image_preview controller, and updating text extraction and comparison logic.

Enhancements:

api/Dockerfile: Added libreoffice to the list of packages installed during the Docker image build process.

Bug fixes:

api/core/tools/entities/tool_entities.py: Corrected a typo in the ToolParameterType enumeration value from "systme-files" to "system-files".
api/controllers/files/image_preview.py: Improved the handling of file names in the Content-Disposition header by encoding the filename to ensure it is safely transmitted.

Codebase improvements:

api/core/workflow/nodes/document_extractor/node.py: Added chardet for detecting file encoding when extracting text from CSV files, enhancing the robustness of text extraction. [1] [2]
api/core/workflow/nodes/list_operator/node.py: Fixed the comparison logic in the _is function to use == instead of is, ensuring correct functionality.# Summary

Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. List any dependencies that are required for this change.

Tip

Close issue syntax: Fixes #<issue number> or Resolves #<issue number>, see documentation for more details.

Screenshots

Before	After
...	...

Checklist

Important

Please review the checklist below before submitting your pull request.

This change requires a documentation update, included: Dify Document
I understand that this PR may be closed in case there was no previous discussion or issues. (This doesn't apply to typos!)
I've added a test for each change that was introduced, and I tried as much as possible to make a single atomic change.
I've updated the documentation accordingly.
I ran dev/reformat(backend) and cd web && npx lint-staged(frontend) to appease the lint gods

…ilar exceptions

yihong0618 · 2024-12-25T13:40:35Z

very clean Pull Request message learned that, thank you for the content

api/core/tools/entities/tool_entities.py

This reverts commit 2f0823d.

ndhjm · 2024-12-27T14:12:27Z

I have standardized my code format and look forward to review

ndhjm · 2025-01-02T07:06:13Z

I apologize for missing some file format changes. I have revised my code format again and successfully ran the 'ruff format --check' command. I look forward to another review. @yihong0618 @laipz8200

ndhjm · 2025-01-08T05:52:58Z

hello @laipz8200 🌚

laipz8200 · 2025-01-08T08:05:35Z

api/models/model.py

@@ -1045,7 +1045,7 @@ def message_files(self):
            files.append(file)

        result = [
-            {"belongs_to": message_file.belongs_to, **file.to_dict()}
+            {"belongs_to": message_file.belongs_to, "upload_file_id": file.related_id, **file.to_dict()}


Please don't include this field in the returned result, to fix the 'more like this' function, you can query MessageFile manually and build the File object by using file_factory.build_from_message_files in

dify/api/core/app/apps/completion/app_generator.py

Line 227 in c135967

def generate_more_like_this(

laipz8200 · 2025-01-08T08:07:37Z

api/controllers/files/image_preview.py

@@ -71,7 +71,10 @@ def get(self, file_id):
        if upload_file.size > 0:
            response.headers["Content-Length"] = str(upload_file.size)
        if args["as_attachment"]:
-            response.headers["Content-Disposition"] = f"attachment; filename={upload_file.name}"
+            from urllib.parse import quote


A lazy load for this module is not necessary, put it on the top of the file is better.

laipz8200 · 2025-01-08T08:10:37Z

api/core/workflow/nodes/question_classifier/question_classifier_node.py

@@ -46,6 +47,13 @@ def _run(self):
        # extract variables
        variable = variable_pool.get(node_data.query_variable_selector) if node_data.query_variable_selector else None
        query = variable.value if variable else None
+        if query:


We've no plan to support files in the Question Classifier node. If you just want to convert the input to string, you can use variable.text directly.

193658 added 7 commits December 25, 2024 14:36

fix:misspelling

2f0823d

fix:filenames were not properly encoded

b9bd8c2

fix:after the TextGeneration function parses the file, click more sim…

be2acd1

…ilar exceptions

fix:DocumentExtractorNode uploaded the csv file with garbled characters

5849f0b

fix:DocumentExtractorNode uploaded the ppt file exceptions

bb7e602

fix:The Is condition of the ListOperationNode is abnormal

21b6a9b

fix: QuestionClassifierNode using a type other than string

3b67bbb

dosubot bot added size:M This PR changes 30-99 lines, ignoring generated files. 🐞 bug Something isn't working 💪 enhancement New feature or request labels Dec 25, 2024

yihong0618 reviewed Dec 25, 2024

View reviewed changes

api/core/tools/entities/tool_entities.py Outdated Show resolved Hide resolved

Revert "fix:misspelling"

e2244ff

This reverts commit 2f0823d.

crazywoola requested a review from laipz8200 December 27, 2024 12:22

style: Format code

dc57f5e

dosubot bot added size:S This PR changes 10-29 lines, ignoring generated files. and removed size:M This PR changes 30-99 lines, ignoring generated files. labels Dec 27, 2024

style: Format code

4d26074

laipz8200 requested changes Jan 8, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fixed Several Exceptions in Workflow Usage #12075

Fixed Several Exceptions in Workflow Usage #12075

ndhjm commented Dec 25, 2024

yihong0618 commented Dec 25, 2024

ndhjm commented Dec 27, 2024

ndhjm commented Jan 2, 2025 •

edited

Loading

ndhjm commented Jan 8, 2025

laipz8200 Jan 8, 2025

laipz8200 Jan 8, 2025 •

edited

Loading

laipz8200 Jan 8, 2025

Fixed Several Exceptions in Workflow Usage #12075

Are you sure you want to change the base?

Fixed Several Exceptions in Workflow Usage #12075

Conversation

ndhjm commented Dec 25, 2024

Improvements and Bug Fixes:

Improvements to Dockerfile:

Bug Fixes:

Enhancements:

Updates to Question Classifier Node:

Screenshots

Checklist

yihong0618 commented Dec 25, 2024

ndhjm commented Dec 27, 2024

ndhjm commented Jan 2, 2025 • edited Loading

ndhjm commented Jan 8, 2025

laipz8200 Jan 8, 2025

Choose a reason for hiding this comment

laipz8200 Jan 8, 2025 • edited Loading

Choose a reason for hiding this comment

laipz8200 Jan 8, 2025

Choose a reason for hiding this comment

ndhjm commented Jan 2, 2025 •

edited

Loading

laipz8200 Jan 8, 2025 •

edited

Loading