This update improves diagnostic capabilities in the transcript worker by adding boot-time verification for the TRANSCRIPT_CALLBACK_SECRET and incorporating job logging for request outcomes. Including the callback response body in error reports will help resolve 401 unauthorized errors by identifying missing or misconfigured secrets.

Fixed an issue where using ':' in custom job IDs caused BullMQ to reject enqueues, resulting in 500 errors. Switched to using '_' as a separator, which is compatible with the alphanumeric format of our IDs, and implemented better error reporting by returning JSON responses instead of generic error pages.

We have introduced a robust queue-based architecture for handling bulk transcript generation, replacing synchronous processes with a scalable BullMQ-driven pipeline. This new system handles the full lifecycle—fetching render-ready HTML, generating PDFs via the shared browser pool, and performing direct S3 uploads—with automatic reporting back to SparkMentis. This ensures reliable, fail-safe processing, even for large batches.

Strengthened infrastructure security by adding an entrypoint script to correctly manage /tmp/jobs directory permissions before executing as a non-root user. Also introduced a deploy helper script for Ubuntu environments to streamline deployment processes. These changes prevent potential permission issues in the job runner and simplify container management.

We fixed SELinux access denials for sibling containers by adding the :z relabel option to host bind mounts for /tmp/jobs, ensuring execution containers have necessary write permissions. Additionally, we updated the G++ configuration to support C++20, allowing for modern C++ syntax in submitted code. These changes improve reliability for execution environments on restricted environments like AWS EC2.

Added BullBoard to the application to provide a visual interface for monitoring background job queues, accessible on port 3001. The integration utilizes @bull-board/api and @bull-board/express, and includes necessary updates to the Docker setup and port mapping to ensure proper access. This will significantly improve observability and debugging for our queue-based tasks.

We have introduced a robust code-executor service designed for secure execution of user-submitted code in isolated Docker environments. This service leverages BullMQ for scalable task queuing and currently supports 8 major programming languages. Comprehensive deployment guides and API runbooks have been added to streamline integration and management.