The accelerating convergence of cloud computing infrastructures with advanced manufacturing and service-oriented digital ecosystems has produced an unprecedented demand for intelligent task scheduling, energy-aware resource allocation, and adaptive queuing mechanisms. Traditional deterministic and heuristic scheduling paradigms, originally designed for relatively stable computational or industrial environments, increasingly struggle to cope with the stochastic, heterogeneous, and high-dimensional nature of modern cloud and cyber-physical production systems. Within this context, deep reinforcement learning has emerged as a transformative paradigm that enables autonomous agents to learn optimal scheduling, routing, and resource management strategies through continuous interaction with complex environments. This research article develops an integrated theoretical and methodological framework that unifies deep Q-learning based task scheduling with optimal queuing principles, focusing on sustainability, efficiency, and robustness across cloud computing and flexible manufacturing systems.

Grounded in the deep Q-learning driven optimal task scheduling paradigm articulated by Kanikanti, Tiwari, Nayan, Suryawanshi, and Chauhan, this study extends the conceptual scope of learning-based scheduling by embedding queuing theory into the reinforcement learning decision loop, thereby enabling the agent to internalize congestion, waiting time, and service discipline dynamics as intrinsic components of its reward structure (Kanikanti et al., 2025). Unlike conventional job-shop or cloud schedulers that treat queues as exogenous constraints, the present framework treats them as endogenous and learnable system properties, allowing the scheduling agent to adapt to workload fluctuations, energy constraints, and performance trade-offs in a theoretically principled manner.

The article situates this approach within a broad scholarly landscape that includes evolutionary and swarm-based scheduling, green manufacturing optimization, fog and edge computing task allocation, and deep reinforcement learning for resource management. Prior research has demonstrated the effectiveness of metaheuristics such as genetic algorithms, tabu search, and memetic algorithms for flexible job-shop scheduling, as well as the promise of reinforcement learning for cloud and fog-based task scheduling, but these two streams of research have often evolved in parallel rather than in integration (Pezzella et al., 2008; Yuan and Xu, 2015; Gazori et al., 2020). By synthesizing these traditions through a queuing-aware deep Q-learning framework, this study advances a unified model capable of addressing not only throughput and latency but also energy efficiency, sustainability, and system resilience.

Methodologically, the article develops a detailed simulation-based research design grounded in CloudSim Plus and related cloud modeling toolkits, while drawing conceptual parallels to flexible manufacturing systems characterized by multipurpose machines and transportation constraints (Calheiros et al., 2011; Filho et al., 2017; Brucker and Schlie, 1990). Rather than presenting numerical results in tabular form, the findings are articulated through theoretically grounded and literature-validated interpretive analysis, demonstrating how learning-driven schedulers internalize queue dynamics, reduce energy waste, and achieve superior long-term performance stability compared to static or rule-based approaches.

The discussion section engages deeply with theoretical debates surrounding function approximation, stability, and exploration-exploitation trade-offs in deep reinforcement learning, incorporating insights from foundational work on deep Q-networks and actor-critic architectures (Mnih et al., 2015; Fujimoto et al., 2018). It further explores the implications of these learning-based schedulers for sustainable manufacturing, green cloud computing, and the future of autonomous digital infrastructures, critically examining both their transformative potential and their practical limitations. By integrating optimal queuing, deep reinforcement learning, and sustainability-oriented scheduling, this article contributes a comprehensive, theoretically rich, and forward-looking framework for the next generation of intelligent cloud and manufacturing systems.