Go jobs were previously failing due to read-only filesystem issues when attempting to create build directories in /tmp, and slow cold-start times due to recompilation of the standard library. I've updated the runner to use a custom Docker image with a pre-compiled build cache and switched from go run to a explicit go build step, which significantly improves both reliability and performance. Go programs now start execution in under a second.

Resolved an issue where code execution jobs would hang indefinitely due to /tmp directory cleanup by moving job storage to /var/lib/code-executor/jobs. Additionally, replaced the deprecated openjdk:21-slim Docker image with eclipse-temurin:21-jdk-alpine to restore Java language support. These changes ensure robust job lifecycle management and reliable environment configuration for supported languages.

This update introduces robust support for SQL (MariaDB) and PHP website projects to the SparkMentis executor. We've added secure, containerized execution environments using tmpfs for temporary storage, alongside increased code size limits to handle larger datasets and multi-file project structures. These changes enable more comprehensive coding assessments, particularly for database queries and interactive web development tasks.

This update improves diagnostic capabilities in the transcript worker by adding boot-time verification for the TRANSCRIPT_CALLBACK_SECRET and incorporating job logging for request outcomes. Including the callback response body in error reports will help resolve 401 unauthorized errors by identifying missing or misconfigured secrets.

Fixed an issue where using ':' in custom job IDs caused BullMQ to reject enqueues, resulting in 500 errors. Switched to using '_' as a separator, which is compatible with the alphanumeric format of our IDs, and implemented better error reporting by returning JSON responses instead of generic error pages.

We have introduced a robust queue-based architecture for handling bulk transcript generation, replacing synchronous processes with a scalable BullMQ-driven pipeline. This new system handles the full lifecycle—fetching render-ready HTML, generating PDFs via the shared browser pool, and performing direct S3 uploads—with automatic reporting back to SparkMentis. This ensures reliable, fail-safe processing, even for large batches.

Strengthened infrastructure security by adding an entrypoint script to correctly manage /tmp/jobs directory permissions before executing as a non-root user. Also introduced a deploy helper script for Ubuntu environments to streamline deployment processes. These changes prevent potential permission issues in the job runner and simplify container management.

We fixed SELinux access denials for sibling containers by adding the :z relabel option to host bind mounts for /tmp/jobs, ensuring execution containers have necessary write permissions. Additionally, we updated the G++ configuration to support C++20, allowing for modern C++ syntax in submitted code. These changes improve reliability for execution environments on restricted environments like AWS EC2.

Added BullBoard to the application to provide a visual interface for monitoring background job queues, accessible on port 3001. The integration utilizes @bull-board/api and @bull-board/express, and includes necessary updates to the Docker setup and port mapping to ensure proper access. This will significantly improve observability and debugging for our queue-based tasks.

We have introduced a robust code-executor service designed for secure execution of user-submitted code in isolated Docker environments. This service leverages BullMQ for scalable task queuing and currently supports 8 major programming languages. Comprehensive deployment guides and API runbooks have been added to streamline integration and management.