I found a similar unsolved discussion regarding the error "Response output is missing or does not contain embeddings" ^[1]. Additionally, there are related issues that might provide some insights:

1. An issue titled "upload document embedding error & After deleting the failed document, the knowledge base will still retain the content of the failed document" suggests ensuring that the document size does not exceed the limit and verifying the ETL configuration. It also provides a method to manually clean up database entries if automatic deletion fails ^[2].

2. Another issue, "psycopg2.errors.UndefinedTable: relation 'embedding_vector_index_0799a0c6_ca9a_418e_9e33_900236a74af4_nod' does not exist," suggests checking the Docker Compose configuration and ensuring that migrations are enabled and properly executed ^[3].

For your specific case, ensure that the document size is within the allowed limits and check the configuration settings related to the embedding process. If the problem persists, reviewing the logs for any additional error messages might provide more clues.

To continue talking to Dosu, mention @dosu.

I I'm running into this problem as well，on dify 0.9.1

I got the same problem with the version 0.10.1. In the sentry log, it shows that the api key is invalid.

This issue might be related to API key encryption and storage?

This appears to be caused by using encrypted credentials directly as API key

I got the same problem with the version 0.10.1. In the sentry log, it shows that the api key is invalid.
...
This appears to be caused by using encrypted credentials directly as API key

I reviewed the source code, and during the process of obtaining the API key, it first retrieves historical decrypted results from the Redis cache. The stored data in Redis consists of the encrypted results.

It seems that this piece of code did not properly modify the credentials, resulting in the direct use of the encrypted values from the database:

https://github.com/langgenius/dify/blob/0.10.1/api/core/provider_manager.py#L588-L595

I obtained the encrypted credentials data and private key from Redis in the environment which has the error and simulated the decryption process locally. An exception was thrown on this line of code:

https://github.com/langgenius/dify/blob/main/api/libs/rsa.py#L76

Error message is: Incorrect decryption.

I conducted further debugging and found that the failure was caused by the invalid is 255 on this line of code:

https://github.com/langgenius/dify/blob/main/api/libs/gmpy2_pkcs10aep_cipher.py#L200

I further examined the encryption and decryption process and ultimately discovered that the error was due to the RSA public key stored in the db being inconsistent with the public key retrieved from the private key file. However, I am currently unsure why this inconsistency exists.

Perhaps there was an issue with updating the public key when generating the new private key.

From the code, it appears that the public key in the db is only written once during tenant creation. If the private key for the same tenant is regenerated afterward, it will lead to an inconsistency. It seems there are no other methods in the code to update the public key.

One SOLUTION is to use the command provided by Dify to reset the key pair, but be aware that this will invalidate the existing model authentication and require reconfiguration of all providers.

https://github.com/langgenius/dify/blob/main/api/commands.py#L98-L115

In api container, flask reset-encrypt-key-pair

I have submitted a Pull Request(#10210) for your review. Thank you! @crazywoola

Hi, @yinzih. I'm Dosu, and I'm helping the Dify team manage their backlog. I'm marking this issue as stale.

Issue Summary

• You reported an error during the embedding process in Dify version 0.10.2 on the cloud platform.
• The error was related to missing or incomplete embeddings in the response output.
• User shisaru292 identified a potential issue with API key encryption and proposed a solution.
• A Pull Request (feat: add check for key matching on decryption failure #10210) was submitted to address the public key inconsistency.

Next Steps

• Please confirm if this issue is still relevant to the latest version of Dify.
• If the issue is resolved or no longer relevant, it will be automatically closed in 15 days unless further comments are made.

Thank you for your understanding and contribution!