Summary

I enhanced error logging with traceback throughout the application, updated dependencies for unstructured and langchain core packages, adjusted Dockerfiles to pre-download NLTK data and disable Unstructured analytics, and removed the version specification from docker-compose.yaml.

• Added detailed error logging with traceback in main.py to aid in debugging and error tracking.
• Updated unstructured and langchain core packages to the latest versions in requirements.txt and requirements.lite.txt.
• Modified Dockerfile and Dockerfile.lite to download NLTK data during build time, preventing unstructured from downloading packages at runtime.
• Set the NLTK_DATA environment variable to /app/nltk_data in Dockerfiles for consistent NLTK data path.
• Disabled Unstructured analytics by setting SCARF_NO_ANALYTICS=true in Dockerfiles.
• Removed the version specification from docker-compose.yaml to utilize the default version.

Relevant GitHub references:

• Use native ntlk download Unstructured-IO/unstructured#3796
• [BUG] punkt_tab breaking change nltk/nltk#3293

Change Type

• Bug fix (non-breaking change which fixes an issue)

Testing

I tested the changes by:

• Running the application to ensure it operates correctly with the updated dependencies.
• Triggering exceptions to confirm that error logging with traceback works as expected.
• Building Docker images using the updated Dockerfiles and verifying that NLTK data is downloaded during the build process.
• Running the application using the updated docker-compose.yaml to ensure there are no Docker-related issues.
• Checking the application logs to confirm that Unstructured analytics is disabled.

Test Configuration:

• Operating System: Ubuntu 22.04 LTS
• Docker version: 23.0.6
• Docker Compose version: 2.17.3
• Python version: 3.11

Checklist

• My code adheres to this project's style guidelines
• I have performed a self-review of my own code
• I have commented in any complex areas of my code
• I have made pertinent documentation changes
• My changes do not introduce new warnings
• Local unit tests pass with my changes