Abstract:
The emergence of sixth-generation (۶G) wireless networks and the rapid expansion of cyber-physical systems demand reliable support for mission-critical and time-

sensitive applications. In these networks, conventional performance metrics such as throughput and delay are insufficient for continuously evaluating information

freshness at the receiver. Although the Age of Information (AoI) has been introduced as a time-centric metric to address this limitation, its statistical distribution in

operational environments experiences considerable fluctuations due to physical channel dynamics, energy constraints, and spatial interference. The predominant reliance

of existing literature on optimizing the expected value of AoI leaves systems vulnerable to rare but high-cost events in the distribution tail. Furthermore, the dependence of

current risk-sensitive methods on a priori analytical models or full-episode collection in trajectory-based algorithms hinders their online implementation in time-varying

wireless environments.
To address these challenges, this dissertation presents a proposed integrated approach for radio resource management and AoI optimization that simultaneously builds

upon three foundational pillars: model-free learning, risk-awareness, and non-trajectory-based updates. Within this framework, the need for prior knowledge of

environmental transition probabilities is eliminated, tail-distribution fluctuations are controlled via risk measures including mean-variance, exponential utility, and

Conditional Value at Risk (CVaR), and the learning process prevents data staleness during training by replacing trajectory-based methods with step-by-step, incremental

stochastic approximations across multiple timescales.

To evaluate its effectiveness and generalizability, the proposed integrated approach is implemented across three network scales of increasing complexity. First, for a point-

to-point link over millimeter-wave (mmWave) channels with energy harvesting constraints, a two-timescale Variance-Adjusted Q-Learning (VAQL) algorithm is developed.

Second, the problem is extended to a centralized multi-user network assisted by Reconfigurable Intelligent Surfaces (RIS) under Non-Orthogonal Multiple Access (NOMA)

and Rate-Splitting Multiple Access (RSMA) protocols, where an Ultra-Fresh Risk-Sensitive Deep Deterministic Policy Gradient (UF-RS-DDPG) algorithm featuring a dual-critic

structure is designed to isolate and mitigate individual user variance. Third, dense and decentralized Internet of Things (IoT) networks with Grant-Free NOMA (GF-NOMA)

are formulated as a Partially Observable Stochastic Game (POSG), and a Risk-Constrained Multi-Agent DDPG (RCMADDPG) algorithm is proposed within a Centralized

Training with Decentralized Execution (CTDE) paradigm, incorporating state augmentation and four-timescale primal-dual Lagrangian updates to satisfy CVaR constraints.

Theoretical convergence analyses and simulation results demonstrate that, compared to risk-neutral baselines, the proposed algorithms reduce the dispersion and tail

percentiles of the AoI distribution with a controllable trade-off in the average age, providing an effective framework for maintaining stability in time-critical applications.