Adaptive Online Convex Optimization: A Survey of Algorithms, Theory, and Modern Applications

Abstract

Amid the exponential growth of streaming data and rising demands for real-time decision-making, Online Convex Optimization (OCO) has emerged as a foundational framework for sequential data processing in dynamic environments. This survey presents a systematic review of recent evolutionary and adaptive OCO strategies, offering a detailed taxonomy that classifies algorithms according to their constraint-handling mechanisms and environmental feedback. The analysis first examines Constrained OCO, elucidating the trade-offs between computational efficiency and theoretical guarantees across projection-based methods, projection-free Frank–Wolfe variants, and general convex optimization approaches. It then explores the Unconstrained OCO landscape, emphasizing the shift from parameter-dependent methods to fully adaptive, parameter-free algorithms capable of handling unknown comparator norms and gradient scales. Furthermore, the study synthesizes state-of-the-art applications in power systems, network communication, and quantitative finance, bridging theoretical OCO models with robust engineering solutions. The paper concludes by outlining critical open challenges and future research directions, such as the integration of OCO with deep learning, non-convex optimization, and robustness against adversarial corruptions in data-intensive scenarios.

1. Introduction

With the rapid advancement of information technology and big data, especially in fields like cloud computing, the Internet of Things, and artificial intelligence, the rate of data generation shows an explosive growth trend. According to statistics, the global data volume is expected to reach 175 Zettabytes (ZB) by 2025, far surpassing the capabilities of traditional data storage and processing, while daily data volumes on internet platforms and social media continue to exhibit exponential growth [1]. For example, on social platforms, millions of images, videos, and comments are generated every minute; on e-commerce platforms, the number of transactions and package dispatches per second will also increase significantly. This rapid growth of big data poses unprecedented challenges for traditional data analysis methods, making efficient extraction, processing, and utilization of key information from massive data a core issue in the current technological field. In this context, machine learning, as a core technology of artificial intelligence, has gradually become a key tool to address the challenges of the big data era. By constructing mathematical models, machine learning continuously accumulates experience from data and optimizes decisions, enabling autonomous learning and prediction without explicit programming. With the rapid development of advanced methods like deep learning and transfer learning, machine learning approaches can not only handle more complex and diverse data but also achieve more precise and efficient analysis in areas such as real-time data streams, recommendation systems, and intelligent decision-making.

As data scales expand unprecedentedly, machine learning research faces new problems and challenges. Machine learning can typically be divided into two broad categories: Batch Learning and Online Learning [2]. Traditional machine learning methods mainly rely on batch learning, which is based on the entire pre-stored dataset. The data is divided into training and test sets; the model is first trained on the training set and then evaluated on the test set. Batch learning algorithms are widely used for classification, regression, and other problems, performing excellently in many traditional scenarios. However, in the big data era, training models with batch learning algorithms requires significant computational resources, and whenever the dataset changes, the model must be retrained. This drawback makes batch learning challenging in dynamic environments, failing to meet the needs of high-frequency scenarios requiring real-time responses, such as autonomous driving and financial trading. Unlike batch learning, online learning does not require storing all data. It processes new data incrementally, updating model parameters instantly. As new data arrives, online learning algorithms adjust the model through real-time learning without relying on all previous data. This incremental updating allows online learning algorithms to efficiently handle large-scale and dynamically changing data streams. More importantly, online learning can provide real-time feedback and adjust its decisions, enabling it to adapt to rapidly changing environments, thus holding immense application potential in real-time systems.

A fundamental challenge in online learning lies in the efficient update of models upon the arrival of streaming data, necessitating the application of robust optimization theories to guarantee algorithmic stability and convergence [3]. Online Convex Optimization provides a powerful framework to address this challenge by modeling the sequential decision-making process. In this framework, a learner interacts with the environment over a sequence of rounds $t=1,\dots ,T$. In each round t, the learner selects a decision vector ${\mathbf{x}}_t$ from a known convex feasible set $\mathcal{K}\subseteq {\mathbb{R}}^d$. Subsequently, the environment reveals a convex loss function $f_t:\mathcal{K}\to \mathbb{R}$, and the learner suffers a loss $f_t\left({\mathbf{x}}_t\right)$. The ultimate goal of the learner is to minimize the cumulative loss over the horizon T. The performance of an OCO algorithm is typically measured by Regret, defined as the difference between the cumulative loss incurred by the learner and that of the best fixed decision in hindsight. Mathematically, the regret $R_T$ is expressed as:

$$R_T=\sum _{t=1}^T{f}_t\left({\mathbf{x}}_t\right)-\underset{{\mathbf{x}}^{*}\in \mathcal{K}}{min}\sum _{t=1}^T{f}_t\left({\mathbf{x}}^{*}\right).$$

(1)

Intuitively, this metric quantifies the learning capability of the algorithm by measuring how quickly the average loss converges to that of the optimal static strategy. A sublinear regret implies that the average performance gap vanishes as the time horizon approaches infinity, indicating that the learner successfully adapts to the environment and performs asymptotically as well as the best fixed decision in hindsight. Minimizing regret implies that the online algorithm performs asymptotically as well as the optimal offline strategy. To ensure the tractability of this optimization and the existence of a global optimum, the loss functions $f_t$ are typically assumed to be convex, satisfying the inequality:

$$f_t\left(\mathbf{y}\right)\ge f_t\left(\mathbf{x}\right)+\langle \nabla f_t\left(\mathbf{x}\right),\mathbf{y}-\mathbf{x}\rangle ,\forall \mathbf{x},\mathbf{y}\in \mathcal{K}.$$

(2)

Adaptive algorithms represent a sophisticated class of optimization methodologies capable of dynamically modulating hyperparameters in response to real-time feedback. In the context of Online Convex Optimization, the integration of adaptive strategies is pivotal for maintaining robust performance within non-stationary environments [3]. Unlike traditional online algorithms that rely on pre-determined, static learning rates—often leading to suboptimal convergence or oscillation—adaptive algorithms optimize the learning trajectory by leveraging historical data characteristics. By automatically rescaling step sizes based on accumulated gradient information, these methods ensure appropriate parameter updates at each iteration. This mechanism not only accelerates convergence and enhances algorithmic stability but also minimizes regret in dynamic settings. Consequently, adaptive strategies empower OCO frameworks to effectively navigate complex, time-varying landscapes, thereby significantly augmenting their practical utility.

Despite the rapid proliferation of Online Convex Optimization research, a systematic survey focusing specifically on the evolution of adaptive strategies and their constraint-handling mechanisms remains absent. To bridge this gap, this paper provides a comprehensive review organized as follows: Section 3 establishes the foundation of Constrained OCO, systematically taxonomizing algorithms into projection-based, projection-free, and general convex optimization frameworks to analyze the critical trade-off between regret guarantees and computational complexity. Section 4 pivots to the Unconstrained OCO setting, detailing the paradigmatic transition from parameter-dependent methods to fully adaptive, parameter-free learning capable of navigating unknown environments. Section 5 bridges theory and practice by examining state-of-the-art applications in power systems, network communication, and quantitative finance. Finally, Section 6 identifies critical challenges and future directions, including the integration with deep learning, robustness against uncertainty, and scalability in large-scale data scenarios. Appendix A provides a detailed categorical index of all 75 publications reviewed in this study. Through this detailed analysis, we aim to provide a valuable perspective and guidance for further research in adaptive OCO.

While foundational surveys like Hazan [4] have comprehensively established the theoretical baselines of OCO—primarily focusing on static regret minimization in standard convex settings—the field has significantly evolved towards handling non-stationary environments and complex physical constraints. Unlike existing reviews that often treat constrained optimization and adaptive learning in isolation, this survey systematically bridges these domains. To the best of our knowledge, this is the first comprehensive review to focus specifically on the evolution of adaptive strategies alongside constraint-handling mechanisms.

• A Novel Taxonomy of Constrained OCO: We provide a structured classification of constrained algorithms based on their computational complexity and constraint interaction. Distinct from general overviews, we deeply analyze the trade-offs between Projection-based, Projection-free, and General Convex Optimization methods, specifically highlighting recent breakthroughs in minimizing Cumulative Constraint Violation for safety-critical systems.
• Systematic Review of Parameter-free Learning: We trace the paradigm shift in Unconstrained OCO from parameter-dependent methods to fully adaptive, parameter-free algorithms. We synthesize key theoretical frameworks, including Reward–Regret Duality and Coin Betting, which are essential for deploying OCO in the wild where environmental parameters are unknown.
• Integration of Modern Engineering Applications: Bridging the gap between theory and practice, we review state-of-the-art applications in Power Systems, Network Communication, and Quantitative Finance. A distinguishing feature of this survey is its focus on the most recent literature (covering significant works from 2024 to early 2026), addressing emerging challenges such as renewable energy uncertainty and adversarial robustness in distributed networks.

2. Methods

This review followed a systematic literature review (SLR) methodology, conducted in accordance with standard guidelines to ensure transparency and reproducibility. The process consisted of three main phases: search strategy planning, study selection based on rigorous criteria, and systematic classification and analysis.

2.1. Search Strategy and Data Sources

To construct a comprehensive bibliography of Adaptive Online Convex Optimization, we executed a structured search protocol covering the period from the inception of the field (early 2000s) through September 2025. We utilized four primary academic databases to capture both foundational theory and cutting-edge engineering applications:

• IEEE Xplore: Focused on engineering applications in control systems and signal processing.
• Web of Science: Used for broad interdisciplinary coverage.
• Google Scholar: Employed for broad keyword matching.
• arXiv: Specifically targeting the categories cs.LG (Machine Learning) and math.OC (Optimization and Control) to identify high-quality preprints and recent developments.

The search strategy employed Boolean logic to combine core concepts with specific algorithmic attributes. The search terms were designed to filter for studies addressing dynamic environments and constraint handling:

• Primary Topics: “Online Convex Optimization” OR “OCO”.
• Algorithmic Qualifiers: AND (“Adaptive” OR “Projection-free” OR “Parameter-free”).
• Performance Metrics: AND (“Regret Analysis” OR “Constraint Violation”).

2.2. Study Selection and Eligibility Criteria

The initial search yielded a broad corpus of literature spanning two decades. To ensure full reproducibility, we report the exact protocols executed as follows. The search was concluded on 30 September 2025, using the specific Boolean query: (“Online Convex Optimization” OR “OCO”) AND (“Adaptive” OR “Projection-free” OR “Parameter-free”) AND (“Regret” OR “Constraint Violation”) applied to metadata across all databases. The selection process followed a strict PRISMA-compliant flow: The initial search identified a total of 315 records. Following automated deduplication via EndNote and manual verification, 92 duplicate records were removed. We then screened the titles and abstracts of the remaining 223 unique records, excluding 118 that were outside the scope of adaptive or constrained optimization. The full texts of the remaining 105 articles were assessed for eligibility, resulting in the exclusion of 31 articles (due to lack of theoretical bounds or being non-English). This process yielded the final 74 studies included in this review. To ensure the survey maintains high algorithmic rigor and theoretical depth, the raw search results were subjected to a stringent multi-stage screening process. This process was designed not only to filter for relevance but also to prioritize studies that have substantially advanced the theoretical boundaries of OCO.

The selection strategy prioritized seminal contributions published in premier high-impact venues, ensuring the reliability and influence of the surveyed algorithms. Our final dataset predominantly comprises studies from top-tier machine learning conferences, specifically the International Conference on Machine Learning (ICML), Advances in Neural Information Processing Systems (NeurIPS), Conference on Learning Theory (COLT), and AAAI Conference on Artificial Intelligence. In parallel, we incorporated rigorous theoretical analyses from leading journals, including the Journal of Machine Learning Research (JMLR) and IEEE Transactions series (e.g., Automatic Control, Power Systems, and Networking).

Crucially, to capture the state-of-the-art in rapidly evolving subfields, we explicitly targeted recent breakthroughs that address complex open problems such as feedback delay, adversarial constraints, and non-stationary environments. Consequently, a significant portion of the selected literature consists of publications and high-quality preprints from 2024 and 2025. These recent works were instrumental in analyzing emerging trends in projection-free optimization and robustness against heavy-tailed noise. The formal eligibility criteria applied during the full-text assessment are defined as follows:

• Inclusion Criteria (IC):

  –

  IC1 (Venue Quality): The study must be peer-reviewed and published in the aforementioned top-tier conferences or reputable journals, or be a highly cited preprint representing a significant recent advancement.

  –

  IC2 (Algorithmic Novelty): The article must propose novel adaptive algorithms or provide new theoretical analyses within the domains of Constrained OCO (e.g., projection-free methods), Unconstrained OCO (e.g., parameter-free learning), or apply these frameworks to complex engineering systems (Power Systems, Networks, Finance).

  –

  IC3 (Theoretical Completeness): The study must provide formal mathematical guarantees, specifically establishing sublinear regret bounds (static, dynamic, or adaptive) or constraining cumulative violations.

• Exclusion Criteria (EC):

  –

  EC1 (Lack of Rigor): Articles that relied solely on heuristic methods without providing theoretical convergence analysis or formal regret bounds were excluded to preserve the review’s focus on mathematical provability.

  –

  EC2 (Language and Accessibility): Publications not written in English or not publicly accessible were excluded from the analysis.

While prioritizing top-tier venues guarantees the theoretical reliability of the surveyed algorithms, we acknowledge that this criterion introduces a selection bias towards fundamental theory over niche applications. Specifically, valid contributions appearing in specialized regional conferences or lower-impact engineering journals might be systematically excluded. To assess the impact of this exclusion, we conducted a random sampling of 30 papers from non-indexed venues during the screening phase. The analysis revealed that the majority of these excluded studies applied existing OGD or OMD methods to specific use cases without proposing novel adaptive mechanisms or improved regret bounds. Consequently, we maintain that our selection criteria effectively capture the core evolutionary trajectory of Adaptive OCO, ensuring that the review focuses on algorithmic breakthroughs and generalizable theoretical advancements rather than exhaustive application enumeration.

2.3. Data Extraction and Taxonomy Construction

The systematic selection process yielded a final dataset of 75 distinct publications. We conducted a comprehensive full-text analysis to extract key algorithmic properties, distinguishing features, and convergence guarantees. To rigorously synthesize the evolution of the field, we constructed a hierarchical taxonomy that delineates algorithms based on their interaction mechanisms, computational complexity, and adaptability to environmental uncertainty. This classification framework is organized into three primary research streams:

(1) Constrained OCO Algorithms. The first dimension of our taxonomy classifies algorithms according to their constraint-handling mechanisms and per-round computational overhead.

• Projection-based Methods: This category encompasses algorithms that enforce feasibility via Euclidean projections or generalized Bregman projections, which are statistically optimal but computationally intensive for complex sets.
• Projection-free Architectures: To address the bottleneck of high-dimensional projections, we identified methods utilizing Linear Optimization Oracles (LOO) and Frank–Wolfe variants, which reduce complexity by solving linear subproblems.
• General Convex Optimization: This class includes approaches that solve full convex subproblems to handle intricate constraints.

(2) Unconstrained OCO Algorithms. The second dimension captures the paradigm shift in unconstrained optimization, classifying studies by their adaptability to unknown environmental parameters. We distinguish between Parameter-dependent methods, which necessitate prior knowledge of comparator norms ($\left|\right|u\left|\right|$) or gradient bounds (G) to tune learning rates, and Parameter-free algorithms. The latter represents fully adaptive strategies utilizing advanced frameworks, such as Reward–Regret Duality and Coin Betting, to achieve optimal regret without manual tuning.

(3) Modern Application Domains. Finally, we extracted use-cases that bridge theoretical OCO models with robust engineering solutions. The analysis focuses on three critical sectors: Power Systems, specifically regarding renewable integration and microgrid dispatch under uncertainty; Network Communication, covering dynamic resource allocation and delay-tolerant optimization; and Quantitative Finance, examining portfolio management strategies and Online Newton Step implementations.

2.4. Temporal and Venue Distribution

The temporal distribution of the 75 included references delineates the historical trajectory and current maturation of the Online Convex Optimization landscape. While the survey is anchored in seminal foundational works from the early 1990s and 2000s. A quantitative analysis indicates that a predominant portion of the selected literature is concentrated in the period from 2016 to 2025. This temporal clustering corresponds to the community’s transition from solving static, full-information problems to addressing non-stationary environments and complex constraint landscapes. Notably, the inclusion of a substantial corpus of literature from 2024 and 2025 underscores the rapid and ongoing evolution of the field. These most recent contributions, including multiple preprints and conference papers published in early 2025, are pivotal in addressing emerging challenges.

In terms of venue distribution, the selected studies consistently appear in high-impact publication channels, ensuring the review’s alignment with the highest standards of algorithmic rigor. The bibliography heavily features proceedings from premier machine learning conferences (e.g., ICML, NeurIPS, AAAI) and authoritative journals in control and optimization (e.g., IEEE Transactions on Automatic Control, Journal of Machine Learning Research). This distribution reflects the interdisciplinary nature of modern OCO research, bridging theoretical computer science with robust engineering applications.

3. Constrained Online Convex Optimization Algorithms

Constrained Online Convex Optimization (COCO) represents a pivotal paradigm within the broader OCO framework [4]. In this setting, the problem is modeled as a repeated game between a learner and an adversarial environment over a horizon T. At each round t, the learner first commits to a decision ${\mathbf{x}}_t$ from a fixed closed convex set $\mathcal{X}$. Subsequently, the adversary reveals a convex cost function $f_t(·)$ and a set of convex constraint functions $g_t(·)$. Crucially, since the constraint functions are disclosed only after the decision is made (i.e., post-hoc), the decision ${\mathbf{x}}_t$ may inadvertently violate the instantaneous constraints. Consequently, the objective of a COCO algorithm is twofold: (i) to minimize Regret, defined as the difference between the cumulative cost of the algorithm and that of the best fixed decision in hindsight that satisfies all constraints; and (ii) to minimize Cumulative Constraint Violation (CCV), quantified as ${\sum }_{t=1}^T{\left[g_t\left({\mathbf{x}}_t\right)\right]}_{+}$, where ${\left[z\right]}_{+}=max\{0,z\}$. In practical terms, this metric represents the total magnitude of safety breaches accumulated over the operation period. Unlike hard constraints that demand strictly feasible solutions at every step, minimizing this cumulative quantity treats safety as a long-term budget. This perspective allows the system to tolerate minor transient violations caused by unexpected environmental fluctuations provided that the constraints are satisfied in the long-term average sense. Based on the per-iteration computational complexity, existing COCO algorithms can be taxonomized into three distinct categories. The first category comprises projection-based methods. These approaches typically reduce the computational burden to one gradient evaluation and one Euclidean projection onto $\mathcal{X}$ per round. While they achieve statistically optimal bounds for both regret and CCV, the projection operation can become a computational bottleneck when the geometry of $\mathcal{X}$ is complex [5]. The second category consists of projection-free algorithms, which aim to circumvent the projection bottleneck by substituting it with a computationally cheaper Linear Programming (LP) oracle. These methods are generally the most computationally efficient but often at the expense of suboptimal statistical performance [6]. The third category involves general convex optimization-based approaches, which necessitate solving a full constrained optimization subproblem at each iteration, rendering them the most computationally expensive [7].

Table 1. Development of Constrained Online Convex Optimization Algorithms.

Reference	Regret Bound	Cumulative Violation	Per-Round Complexity	Constraints
Mahdavi (2012) [8]	$O\left(\sqrt{T}\right)$	$O\left(T^{3/4}\right)$	Euclidean Projection	Time-invariant Constraints
Jenatton (2016) [9]	$O\left(T^{max(\beta ,1-\beta )}\right)$	$O\left(T^{1-\beta /2}\right)$	Euclidean Projection	Time-invariant Constraints
Sun (2017) [10]	$O\left(\sqrt{T}\right)$	$O\left(T^{3/4}\right)$	Bregman Projection	—
Neely & Yu (2017) [11]	$O\left(\sqrt{T}\right)$	$O\left(\sqrt{T}\right)$	General Convex Optimization	Slater Condition
Yu (2017) [12]	$O\left(\sqrt{T}\right)$	$O\left(\sqrt{T}\right)$	Euclidean Projection	Slater Condition, Stochastic Constraints
Yuan & Lamperski (2018) [13]	$O\left(T^{max(\beta ,1-\beta )}\right)$	$O\left(T^{1-\beta /2}\right)$	Euclidean Projection	Time-invariant Constraints
Yu & Neely (2020) [14]	$O\left(\sqrt{T}\right)$	$O\left(1\right)$	General Convex Optimization	Slater Condition, Time-invariant Constraints
Yi (2021) [15]	$O\left(T^{max(\beta ,1-\beta )}\right)$	$O\left(T^{(1-\beta )/2}\right)$	General Convex Optimization	Time-invariant Constraints
Guo (2022) [16]	$O\left(\sqrt{T}\right)$	$O\left(T^{3/4}\right)$	General Convex Optimization	—
Guo (2022) [16]	$O(log(T\left)\right)$	$O\left(\sqrt{TlogT}\right)$	General Convex Optimization	Strongly Convex Loss Function
Yi (2022) [17]	$O\left(T^{\beta }\right)$	$O\left(T^{1-\beta /2}\right)$	Euclidean Projection	Strongly Convex Loss Function
Yi (2023) [18]	$O\left(T^{max(\beta ,1-\beta )}\right)$	$O\left(T^{1-\beta /2}\right)$	General Convex Optimization	—
Yi (2023) [18]	$O(log(T\left)\right)$	$O\left(\sqrt{TlogT}\right)$	General Convex Optimization	Strongly Convex Loss Function
Sinha & Vaze (2024) [19]	$O\left(\sqrt{T}\right)$	$O\left(\sqrt{TlogT}\right)$	Euclidean Projection	—
Sinha & Vaze (2024) [19]	$O(log(T\left)\right)$	$O\left(\sqrt{TlogT}\right)$	Euclidean Projection	Strongly Convex Loss Function
Garber & Kretzu (2024) [20]	$O\left(T^{3/4}\right)$	$O\left(T^{7/8}\right)$	Linear Programming	—
Garber & Kretzu (2024) [20]	$O\left(T^{3/4}\right)$	$O\left(T^{7/8}\right)$	General Convex Optimization	—
Sarkar(2025) [6]	$O\left(T^{3/4}\right)$	$\tilde{O}\left(T^{3/4}\right)$	Linear Programming	—
Hutchinson(2025) [5]	$O\left(\sqrt{T}\right)$	$\tilde{O}\left(\sqrt{T}\right)$	Euclidean Projection	Slater Condition, Time-invariant Constraints

Technical Remarks: Notation: $\tilde{O}(·)$ suppresses polylogarithmic factors. Assumptions: Bounds assume Full-Information with adversarial losses unless stated as “Stochastic”. Lagrangian methods imply Slater’s Condition. Constraint Types: “Cumulative Violation” targets Long-term Constraints (∑g_t ≤ 0), while squared violations penalize deviations more heavily than hinge loss.

summarizes the recent advancements in constrained online convex optimization algorithms.

3.1. Projection-Based Algorithms

Projection-based algorithms constitute a pivotal class of methods within the Constrained Online Convex Optimization (COCO) framework. A defining characteristic of this paradigm is the enforcement of a projection operation immediately following each parameter update, ensuring that the subsequent decision vector $x_{t+1}$ strictly resides within the feasible decision set $\mathcal{X}$. These algorithms are predominantly implemented using Euclidean projection or Bregman projection.

3.1.1. Euclidean Projection-Based Methods

Projection-based algorithms constitute a pivotal class of methods within the Constrained Online Convex Optimization framework. A defining characteristic of this paradigm is the enforcement of a projection operation immediately following each parameter update, ensuring that the subsequent decision vector $x_{t+1}$ strictly resides within the feasible decision set $\mathcal{X}$. These algorithms are predominantly realized through Euclidean projection or Bregman projection.

The archetypal algorithm grounded in Euclidean projection is Online Gradient Descent (OGD). Fundamentally, OGD operates by computing the gradient $\nabla f_t\left(x_t\right)$ of the loss function $f_t$ at the current decision point $x_t$ in each round. It then updates the decision variable along the negative gradient direction and maps the result back onto the feasible set $\mathcal{K}$ via the Euclidean projection operator ${\mathsf{\Pi }}_{\mathcal{K}}$, thereby guaranteeing the feasibility of the next decision $x_{t+1}$. The update rule is formally expressed as $x_{t+1}={\mathsf{\Pi }}_{\mathcal{K}}(x_t-{\eta }_t\nabla f_t\left(x_t\right))$ [21]. In terms of theoretical guarantees, OGD achieves a regret bound of $O\left(\sqrt{T}\right)$ for general convex loss functions, while attaining a superior bound of $O(logT)$ under strongly convex conditions.

However, a significant computational bottleneck of the standard OGD algorithm lies in the projection step ${\mathsf{\Pi }}_{\mathcal{K}}$. When the feasible set $\mathcal{K}$ exhibits complex structures—such as those defined by a multitude of constraints, non-linear boundaries, or specific geometries like the semi-definite cone—calculating the projection becomes computationally prohibitive, potentially constituting a non-trivial optimization sub-problem in its own right. This challenge has catalyzed research into alternative constraint-handling mechanisms aimed at improving efficiency. Strategies such as relaxing strict per-round feasibility requirements or modifying the target projection set have emerged, paving the way for the various algorithms discussed in the subsequent sections.

Mahdavi (2012) pioneered the investigation of Online Convex Optimization with long-term constraints [8]. To circumvent the computationally prohibitive projection operations onto the complex constraint set $\mathcal{K}$ at every iteration, their approach permits instantaneous constraint violations, provided that the constraints are satisfied in the long-term cumulative sense. By introducing Lagrange multipliers $\lambda$, they proposed a novel algorithm where the updates for the primal variable $x_t$ and the dual variable ${\lambda }_t$ involve a simplified projection step:

$$x_{t+1}={\mathsf{\Pi }}_{\mathcal{B}}(x_t-\eta {\nabla }_x{\mathcal{L}}_t(x_t,{\lambda }_t)),$$

(3)

where ${\mathsf{\Pi }}_{\mathcal{B}}$ denotes the projection onto a simple Euclidean ball $\mathcal{B}$. This operation is typically much more efficient than projecting onto the original complex set $\mathcal{K}$. Consequently, the algorithm achieves an $O\left(\sqrt{T}\right)$ regret bound and an $O\left(T^{3/4}\right)$ cumulative constraint violation bound.

Building upon Mahdavi’s framework, Jenatton et al. (2016) continued the exploration of OCO with long-term constraints by proposing a novel adaptive Online Gradient Descent algorithm [9]. This method utilizes a saddle-point formulation of the problem based on the Lagrangian ${\mathcal{L}}_t(x,\lambda )\triangleq f_t\left(x\right)+\lambda g\left(x\right)-{\displaystyle \frac{{\theta }_t}{2}}{\lambda }^2$, performing alternating primal gradient descent and dual gradient ascent. The update rule for $x_t$ retains the projection step: $x_{t+1}={\mathsf{\Pi }}_{\mathcal{B}}(x_t-{\eta }_t{\nabla }_x{\mathcal{L}}_t(x_t,{\lambda }_t))$. Distinct from Mahdavi (2012) [8], both the step size ${\eta }_t$ and the regularization parameter ${\theta }_t$ adapt over time t. The algorithm introduces a user-defined trade-off parameter $\beta \in (0,1)$, achieving a regret bound of $O\left(T^{max(\beta ,1-\beta )}\right)$ and a cumulative violation bound of $O\left(T^{1-\beta /2}\right)$. This allows for a tunable trade-off between regret and violation. For instance, $\beta =2/3$ yields $O\left(T^{2/3}\right)$ for both metrics, while $\beta =1/2$ recovers Mahdavi’s bounds of $O\left(T^{1/2}\right)$ regret and $O\left(T^{3/4}\right)$ violation. This theoretical result highlights a fundamental trade-off between optimality and feasibility where strictly improving the constraint satisfaction often necessitates more conservative decisions. Consequently, this conservatism can prevent the algorithm from fully exploiting profitable opportunities, thereby increasing the regret. For engineers, this implies that the choice of algorithm must be carefully tailored to whether the specific application prioritizes maximizing operational returns or strictly adhering to safety limits.

Yu (2017) addressed the OCO problem under stochastic constraints, where constraint functions $g_k(x;\omega \left(t\right))$ are generated i.i.d. from an unknown distribution at each round t and revealed only after the decision $x_t$ is made [12]. The decision $x_t$ is required to belong to a known fixed set ${\mathcal{X}}_0$, with the objective of achieving sub-linear regret and sub-linear expected constraint violation relative to the optimal fixed decision $x^{*}$ that satisfies the expected constraints ${\mathbb{E}}_{\omega }\left[g_k(x;\omega )\right]\le 0$. Yu proposed an algorithm based on a virtual queue $Q_k\left(t\right)$, the core of which minimizes a “drift-plus-penalty” bound. The update rule is defined as a minimization problem solvable via a simple projection: $x_{t+1}={\mathsf{\Pi }}_{{\mathcal{X}}_0}(x_t-{\displaystyle \frac{1}{2\alpha }}{d}_t)$ [22]. For stochastic constraint problems, this algorithm achieves $O\left(\sqrt{T}\right)$ expected regret and expected violation. For long-term constraint problems, it simultaneously attains $O\left(\sqrt{T}\right)$ regret and cumulative violation under the Slater condition.

Yuan and Lamperski (2018) [13] focused on a stricter form of cumulative constraints, explicitly penalizing positive constraint violations ${\left[g\left(x_t\right)\right]}_{+}=max\{0,g\left(x_t\right)\}$. The goal is to minimize the cumulative squared violations ${\sum }_{t=1}^T{\left({\left[g\left(x_t\right)\right]}_{+}\right)}^2$. This form penalizes large single violations heavily, and strict satisfaction requires that valid violations cannot cancel out existing ones [13]. The algorithm modifies the augmented Lagrangian function in Mahdavi by replacing $g\left(x\right)$ with ${\left[g\left(x\right)\right]}_{+}$. The primal update remains a projected gradient descent—namely, $x_{t+1}={\mathsf{\Pi }}_{\mathcal{B}}(x_t-\eta {\nabla }_x{\mathcal{L}}_t(x_t,{\lambda }_t))$, while the dual update adopts a closed-form solution ${\lambda }_{t+1}={\displaystyle \frac{{\left[g\left(x_{t+1}\right)\right]}_{+}}{\sigma \eta }}$. For convex loss functions, this achieves the same regret $O\left(T^{max(\beta ,1-\beta )}\right)$ as Jenatton (2016) [9], while bounding the cumulative squared violation by $\sum {\left({\left[g\left(x_t\right)\right]}_{+}\right)}^2\le O\left(T^{1-\beta }\right)$ and the cumulative positive violation by $\sum {\left[g\left(x_t\right)\right]}_{+}\le O\left(T^{1-\beta /2}\right)$. Under strongly convex loss functions, the proposed algorithm attains $O(logT)$ regret and $O\left(\sqrt{TlogT}\right)$ cumulative positive violation.

Sinha and Vaze (2024) [19] aimed to resolve a long-standing open problem: Is it possible to simultaneously achieve $O\left(\sqrt{T}\right)$ regret and $\tilde{O}\left(\sqrt{T}\right)$ cumulative violation without restrictive assumptions like the Slater condition? This paper proposes a concise and efficient one-stage algorithm centered on constructing a surrogate loss function ${\widehat{f}}_t\left(x\right):={\nu }^T{\tilde{f}}_t\left(x\right)+{\mathsf{\Phi }}^{\prime }\left(Q\left(t\right)\right){\overline{g}}_t\left(x\right)$. This function combines the original loss with a constraint violation term weighted by the derivative of a Lyapunov function $\mathsf{\Phi }$. This surrogate sequence is then fed into a standard adaptive OCO algorithm (AdaGrad) for optimization. The update is specifically $x_{t+1}={\mathsf{\Pi }}_{\mathcal{X}}(x_t-{\eta }_t{\nabla }_t)$ [19]. This method affirmatively answers the open problem, achieving $O\left(\sqrt{T}\right)$ regret and $O\left(\sqrt{TlogT}\right)$ cumulative violation for convex loss functions.

Finally, Hutchinson and Alizadeh (2025) targeted a stricter objective than cumulative constraint satisfaction: stronger round-by-round constraint satisfaction; i.e., $g\left(x_t\right)\le 0$ for all t [5]. The algorithm combines Online Gradient Descent with a Polyak feasibility step. After each OGD step, the algorithm applies a secondary gradient step based on the information of the constraint function g. The step size is calculated according to the Polyak step size rule, using the first-order approximation of g at $x_t$. The final update rule is:

$$x_{t+1}={\mathsf{\Pi }}_{\kappa \mathbb{B}}\left(y_t-{\displaystyle \frac{{[g_t+s_t^{\top }(y_t-x_t)+\rho ]}_{+}}{\parallel s_t{\parallel }^2}}{s}_t\right),$$

(4)

where $y_t$ is the intermediate point after the OGD step, $\rho$ is a constraint tightening parameter, and ${\mathsf{\Pi }}_{\kappa \mathbb{B}}$ denotes projection onto a Euclidean ball of radius R. Given a strictly feasible point and initialization at that point, the algorithm guarantees $O\left(\sqrt{T}\right)$ regret and satisfied constraints $g\left(x_t\right)\le 0$ for every round. If a strictly feasible point is unknown or the algorithm starts from an arbitrary point, it maintains $O\left(\sqrt{T}\right)$ regret and achieves constraint satisfaction after $O(logT)$ rounds.

The evolutionary trajectory of Euclidean projection-based algorithms for Online Constrained Convex Optimization is fundamentally characterized by the continuous refinement of the trade-off between computational efficiency and theoretical performance guarantees. To mitigate the high computational costs associated with projecting onto complex constraint sets, pioneering studies introduced the concept of long-term constraints, simplifying the projection operation to elementary geometric bodies that encapsulate the original feasible domain. Subsequent developments extended this paradigm to stochastic constraint scenarios, emphasizing more rigorous metrics for constraint violation while predominantly retaining projections onto simple enclosing sets.

Recent scholarship has ventured into increasingly complex distributed environments, pursuing stronger instantaneous (round-by-round) constraint satisfaction and striving to eliminate restrictive assumptions to achieve tighter performance bounds. Throughout this evolution, the role and target of the projection operator have undergone significant transformation: shifting from a mere mechanism for computational simplification to a standard projection onto the primal decision set within advanced frameworks to approach optimality, or persisting as an auxiliary instrument for achieving specific constraint objectives. In the realm of future algorithm design, exploring optimized methodologies for projection set selection represents a critical avenue for enhancing system performance.

3.1.2. Bregman Projection-Based Methods

Methods based on Bregman projections constitute a cornerstone of the Online Convex Optimization landscape, tracing their origins to the Mirror Descent (MD) algorithm introduced by Nemirovski and Yudin (1984) [23]. Viewed as a generalization of gradient descent, the core philosophy of MD lies in eschewing standard Euclidean distance during the projection step in favor of a generalized distance metric—the Bregman divergence $B_{\mathcal{R}}$ (or $D_{\mathcal{R}}$)—induced by a strongly convex function $\mathcal{R}$, referred to as the regularization function or mirror map. This conceptual framework was subsequently extended to the online learning domain by Zinkevich (2003) as the Online Mirror Descent (OMD) algorithm [24]. OMD operates as a generalized online gradient descent where the update step is typically executed in the “dual space” mapped by $\nabla \mathcal{R}$, followed by a projection back onto the primal feasible set $\mathcal{K}$ via the minimization of the Bregman divergence $B_{\mathcal{R}}$. This methodological approach is also identified in the subsequent literature as the Regularized Follow the Leader (RFTL) algorithm. Notably, Hazan and Kale (2008) formally established that RFTL is equivalent to the “lazy” variant of OMD under linear loss functions [25]. The paramount advantage of the OMD framework is its inherent flexibility: by selecting distinct regularization functions $\mathcal{R}$, the Bregman divergence can be tailored to align with the underlying geometry of the problem structure. For instance, when the decision set $\mathcal{K}$ is a probability simplex, selecting the negative entropy $\mathcal{R}\left(x\right)=\sum x_{i}log{x}_i$ as the regularizer yields the Kullback–Leibler (KL) divergence as the corresponding Bregman divergence. In this setting, the OMD algorithm effectively evolves into the renowned Multiplicative Weights or Hedge algorithm.

Orabona et al. (2014) [26] further broadened the OMD framework by proposing a generalized Online Mirror Descent algorithm. This extension accommodates time-varying regularizers, thereby enabling the framework to not only encompass the majority of first-order algorithms but also provide a unified analysis for second-order algorithms [26]. Concurrently, Hazan and Kale (2014) leveraged Bregman projection techniques to address batch stochastic strongly convex optimization problems [27]. Their proposed algorithms, EPOCH-GD and RANDOM-STEP-GD, are both predicated on Bregman updates and projection operations. For instance, EPOCH-GD executes standard stochastic OMD steps within each epoch, incorporating the Bregman projection $x_{t+1}^k=arg{min}_{x\in \mathcal{K}}\left\{B_{\mathcal{R}}(x,y_{t+1}^k)\right\}$. These algorithms attain an optimal convergence rate of $O\left({\displaystyle \frac{1}{T}}\right)$, which surpasses the $O\left({\displaystyle \frac{logT}{T}}\right)$ rate typically derived from standard online-to-batch conversion techniques.

Furthermore, Bregman projections have demonstrated robust capabilities within distributed and dynamic environments. Shahrampour and Jadbabaie (2016) introduced a decentralized OMD variant tailored for dynamic settings, wherein the core mechanism involves each agent executing a mirror descent step that balances local gradient information ${\nabla }_{i,t}$ and consensus information $y_{i,t}$ from neighbors by minimizing the Bregman divergence $D_{\mathcal{R}}$ [28]. In the realm of constrained optimization, Sun et al. (2016) developed a primal-dual meta-algorithm for adversarial contextual bandits with risk constraints [29]. This approach introduces a Lagrange multiplier $\lambda$ to handle time-varying convex constraints $f_t\left(x\right)\le 0$, where the primal update is an OMD step, $x_{t+1}=arg{min}_{x\in X}{D}_R(x,{\tilde{x}}_{t+1})$. This methodology ensures a sublinear accumulation of constraint violations while maintaining low regret. Addressing OCO with energy harvesting constraints, Asgari and Neely (2023) [30] proposed a novel Bregman algorithm distinguished by an ingenious decomposition strategy: the decision vector $X_t$ is decomposed into a magnitude $A_t$ and a direction $P_t$. Innovatively, it employs two distinct types of Bregman divergences—using KL divergence to update the directional vector $P_t$ on the probability simplex and a quadratic divergence for the magnitude $A_t$ [30]. This decomposition strategy significantly mitigates the dependency of the regret bound on the decision dimension, improving it from $O\left(\sqrt{n}\right)$ to $O\left(\sqrt{logn}\right)$. Most recently, Bregman projection methods have been further applied to distributed online convex optimization, specifically targeting online saddle point problems. Zhang et al. (2025) proposed the Distributed Online Stochastic Mirror Descent algorithm and its multi-consensus variants [31]. These algorithms utilize Bregman divergence as a generalized distance metric, where each agent solves a subproblem minimizing Bregman divergence during the update step, thereby solving distributed saddle point problems in non-Euclidean spaces.

Bregman projection methods fundamentally replace the standard Euclidean distance with Bregman divergence, thereby endowing algorithms with the flexibility to adapt to the underlying geometry of the problem. This framework has evolved from offline optimization to the realm of Online Convex Optimization (OCO), progressing further to incorporate time-varying regularizers. This evolution has facilitated successful applications across diverse scenarios, including distributed optimization in dynamic environments, online programming subject to time-varying constraints, and online saddle point problems. Notably, Bregman projection techniques not only achieve the optimal $O\left({\displaystyle \frac{1}{T}}\right)$ convergence rate for stochastic strongly convex optimization but also offer a sophisticated mechanism for handling complex resource constraints through the hybrid utilization of distinct divergence types. Looking forward, a pivotal research trajectory lies in the design of advanced Bregman divergences—potentially through the mixing or adaptive modulation of regularization functions—to address online learning problems characterized by more stringent constraints and intricate structures, such as those found in distributed, dynamic, and adversarial settings. Furthermore, leveraging the intrinsic geometric properties of Bregman projections to develop algorithms with enhanced efficiency and robustness—specifically those capable of auto-adaptation to unknown environmental parameters and problem structures—remains a critical frontier in this domain.

It is critical to distinguish the operational contexts of Euclidean and Bregman projection methods to understand the evolution of OCO. Euclidean projection, used in standard Online Gradient Descent, assumes a uniform geometry where the constraint set is isotropic, such as a sphere or a box. While it offers simplicity and optimal $O\left(\sqrt{T}\right)$ regret for these specific shapes, its performance degrades when the feasible set exhibits complex geometry like the probability simplex or a matrix trace norm. In contrast, Bregman projection methods, exemplified by Online Mirror Descent, generalize this framework by replacing Euclidean distance with a divergence tailored to the geometry of the constraint set. For instance, using the Kullback–Leibler divergence on a simplex allows the algorithm to depend logarithmically rather than linearly on the dimension, yielding the Multiplicative Weights Update algorithm. This geometric adaptation is the fundamental reason why Bregman methods often outperform standard Euclidean methods in high-dimensional sparse settings despite their shared reliance on projection steps. However, both approaches ultimately face a computational bottleneck when the projection step itself becomes an expensive quadratic optimization problem, a limitation that directly motivated the development of the projection-free algorithms discussed in the subsequent section.

3.2. Projection-Free Algorithms

A central challenge within the realm of Online Convex Optimization (OCO) resides in the efficient management of complex feasible sets $\mathcal{K}$. Conventional online algorithms, such as Online Gradient Descent (OGD) or Online Mirror Descent (OMD), predominantly hinge upon a projection step. Specifically, at each iteration t, the algorithm first updates the decision variable $x_t$ based on the gradient (or subgradient) $g_t$ of the loss function $f_t$, subsequently mapping the updated point back into the feasible region $\mathcal{K}$ via the projection operator ${\mathsf{\Pi }}_{\mathcal{K}}(·)$:

$$x_{t+1}={\mathsf{\Pi }}_{\mathcal{K}}(x_t-{\eta }_t{g}_t)$$

(5)

The projection operation ${\mathsf{\Pi }}_{\mathcal{K}}\left(y\right)=arg{min}_{x\in \mathcal{K}}{\parallel x-y\parallel }^2$ constitutes a quadratic optimization problem in its own right. When the structure of $\mathcal{K}$ is intricate—for instance, a high-dimensional polyhedron or a positive semidefinite matrix cone—the computational overhead of this projection becomes prohibitive, often emerging as the primary computational bottleneck in each iterative round. To circumvent these computationally expensive projection operations, a critical avenue of research focuses on Projection-Free algorithms. Pioneering work in this domain was established by Hazan and Kale (2012) [32]. The fundamental philosophy involves abandoning the projection operator in favor of a Linear Optimization Oracle (LOO) to interact with the feasible set $\mathcal{K}$. Given a convex feasible set $\mathcal{K}$ and a cost vector c, the LOO traverses the entire set $\mathcal{K}$ to return a vertex v that minimizes the inner product $\langle c,v\rangle$, formally defined as:

$$v\leftarrow \mathrm{LOO}(\mathcal{K},c)=arg\underset{x\in \mathcal{K}}{min}\langle c,x\rangle$$

(6)

Algorithms predicated on the LOO are essentially online variants of the Frank-Wolfe algorithm, often termed Online Frank–Wolfe (OFW). This approach updates the decision variable by forming a convex combination of the current point $x_t$ and the steepest descent vertex $v_t$ returned by the LOO:

$$x_{t+1}=(1-{\eta }_t)x_t+{\eta }_t{v}_t$$

(7)

Since both $x_t$ and $v_t$ reside within the convex set $\mathcal{K}$, their convex combination $x_{t+1}$ is guaranteed to remain within $\mathcal{K}$. This geometric property ingeniously preserves feasibility, thereby completely obviating the need for projection. In numerous practical scenarios, the computational complexity of the LOO is significantly lower than that of projection. For instance, if $\mathcal{K}$ is a polyhedron, the LOO necessitates solving a Linear Programming (LP) problem, whereas the projection algorithm requires solving a Quadratic Programming (QP) problem. Similarly, if $\mathcal{K}$ represents an $L_1$-ball, the LOO simply requires identifying the basis vector with the largest absolute value and opposing sign—a process with $O\left(d\right)$ complexity—whereas the projection algorithm typically entails a complexity of $O(dlogd)$.

Following the seminal success of the LOO paradigm established by Hazan and Kale (2012) [32], algorithms predicated on Linear Optimization Oracles have garnered significant scholarly attention. Addressing the more challenging bandit feedback setting, Chen et al. (2019) introduced the first computationally efficient Projection-Free Bandit Convex Optimization (PF-BCO) algorithm applicable to general convex constraint sets [33]. Grounded in the Follow-the-Regularized-Leader (FTRL) framework, this algorithm synthesizes Frank–Wolfe principles, smoothing techniques, and one-point gradient estimators to navigate bandit feedback, thereby achieving an expected static regret of $O\left(T^{4/5}\right)$. Subsequently, Garber and Kretzu (2020) proposed the Block Bandit Conditional Gradient Method (BBCGM), which leverages a pivotal block-decomposition strategy to refine the regret bound for projection-free BCO to $O\left(T^{3/4}\right)$ [34]. This bound matches the optimal rate for the full-information setting known at the time, while similarly requiring only $O\left(T\right)$ expected LOO invocations. The blocking technique facilitates a superior trade-off when employing conditional gradient methods to approximately solve the underlying RFTL subproblems. Recognizing the inherent limitations of FTL-based methodologies regarding adaptive regret, Garber and Kretzu (2022) explored novel avenues grounded in Online Gradient Descent (OGD) [35]. By integrating novel and efficient infeasible projection techniques, they designed projection-free algorithms that operate independently of the FTL framework. In the LOO setting, their LOO-BOGD algorithm achieves an adaptive regret of $O\left(T^{3/4}\right)$ across both full-information and bandit environments. Wang et al. (2024) further extended projection-free OCO to non-stationary environments, focusing on general dynamic regret and strongly adaptive regret [36]. Through the construction of POLD and POLA meta-algorithms, they improved the dynamic regret to $O\left(T^{3/4}{(1+P_T)}^{1/4}\right)$ and achieved a strongly adaptive regret of $\tilde{O}\left({\tau }^{3/4}\right)$, where $\tau$ denotes the interval length. Most recently, Garber and Kretzu (2024) addressed a significantly more complex problem within OCO, wherein actions must satisfy not only a fixed, projection-complex hard constraint but also a sequence of time-varying soft constraints in an average sense [20]. To this end, they proposed the inaugural projection-free algorithmic framework for this setting. Predicated on Drift-plus-Penalty (DP) and Primal-Dual (PD) methodologies, their approach achieves adaptive and non-adaptive regret bounds of $\tilde{O}\left(T^{3/4}\right)$, alongside a cumulative constraint violation bound of $O\left(T^{7/8}\right)$ in the full-information setting, with a total LOO complexity of $O\left(T\right)$, a summary of additional algorithms is shown in

Table 2. Development of Projection-free Algorithms in Online Convex Optimization.

Algorithm	Type	Feedback	Regret Metric	Regret Bound	Main Contribution
OFW (2012) [37]	Based on FTL	Full Info	Static	$O\left(T^{3/4}\right)$	First systematic proposal of projection-free OCO concept and the LOO-based OFW algorithm.
PF-BCO (2020) [38]	Based on FTRL	Bandit	Static	$O\left(T^{4/5}\right)$	Successfully applied projection-free methods to bandit feedback scenarios.
BBCGM (2020) [34]	Based on FTRL	Bandit	Static	$O\left(T^{3/4}\right)$	Significantly improved static regret bound for projection-free BCO via blocking techniques.
BOGD (2022) [35]	Based on OGD	Full Info	Adaptive	$O\left(T^{3/4}\right)$	Provided strong adaptive regret guarantees for projection-free OCO for the first time.
POLD/A (2024) [36]	Based on OGD	Full Info	Dynamic, Adaptive	$O\left(T^{3/4}{(1+P_T)}^{1/4}\right)$, $\tilde{O}\left({\tau }^{3/4}\right)$	First to handle non-stationary environments in projection-free OCO, achieving excellent general dynamic and strongly adaptive regret bounds.
DP/PD (2024) [39]	Based on OGD	Full Info	Static, Adaptive	$\tilde{O}\left(T^{3/4}\right)$	First to solve time-varying soft-constrained OCO problems under projection-free framework, distinguishing between hard and soft constraints.

Technical Remarks: Feedback: “Full Info” uses gradient $\nabla f_t$; “Bandit” uses only function value $f_t$ (zero-order). Regret: Bandit bounds refer to Expected Regret $\mathbb{E}\left[R_T\right]$. Scalability: “Projection-free” relies on Linear Optimization Oracle (LOO), avoiding $O\left(d^3\right)$ projection costs, ideal for high-dim matrix tasks despite slower $O\left(T^{3/4}\right)$ rates.

.

3.3. Algorithms Based on General Convex Optimization

In the realm of OCO, particularly when grappling with intricate constraint landscapes, a pivotal and increasingly prevalent methodological paradigm involves the per-round resolution of a newly constructed, generic convex optimization problem. This approach diverges fundamentally from traditional projection-based or projection-free methodologies by fostering a more flexible integration of gradient information, constraint penalties, and regularization terms. Yi et al. (2021) adopted this strategy to address rigorous metrics of cumulative constraint violation [15]. In their framework, the algorithm minimizes a composite objective function at each iteration, dynamically synthesizing gradient information from the preceding step $\langle \partial f_{t-1}\left(x_{t-1}\right),x\rangle$, a convex constraint penalty term predicated on an auxiliary variable $\langle {\widehat{q}}_{t-1},{\left[g\left(x\right)\right]}_{+}\rangle$, and a quadratic regularization term $\parallel x-x_{t-1}{\parallel }^2$.

Subsequently, the RECOO algorithm proposed by Guo et al. (2022) leveraged a comparable methodology to manage hard constraints [16]. The crux of RECOO lies in its recursive correction decision process, which necessitates solving a distinct $argmin$ problem at every step t. The objective function is similarly constructed as a dynamic convex function combining a gradient-based linear term, a rectified constraint penalty term $Q(t-1){\widehat{g}}_{t-1}^{+}\left(x\right)$, and a quadratic regularizer ${\alpha }_{t-1}{\parallel x-x_{t-1}\parallel }^2$. Furthermore, in the context of distributed environments, Yi et al. (2023) introduced a novel algorithm that intrinsically mandates the solution of a generic convex optimization problem [18]. The core update mechanism is formulated as an $argmin$ operation involving a composite convex function ${\alpha }_t\langle x,{\omega }_{i,t+1}\rangle +{\alpha }_t{r}_i\left(x\right)+D_{\psi }(x,x_{i,t})$, comprising a linear component, a potentially non-trivial general convex function $r_i\left(x\right)$, and a Bregman divergence term. Barring the special case where $r_i\left(x\right)$ is zero and the Bregman divergence reduces to the squared Euclidean distance, this step constitutes a generic convex optimization problem.

Recent scholarship in OCO has coalesced around the challenges posed by time-varying and adversarial constraints, widely adopting the stricter cumulative constraint violation metric $\sum g_t^{+}$ as a primary performance benchmark. A dominant algorithmic trend has emerged: a departure from reliance on simple projections in favor of solving either a newly constructed generic convex optimization problem or, via projection-free methods, a linear optimization problem. A promising frontier for future research involves the deeper exploitation of problem-specific structural attributes—such as the curvature of loss or constraint functions—to engineer next-generation algorithms capable of achieving accelerated convergence rates and minimized constraint violations.

In summary, the selection of constrained algorithms is fundamentally governed by the trade-off between statistical optimality and computational efficiency. Projection-based methods remain the standard for achieving optimal regret bounds and are best suited for scenarios where the constraint set is simple, like Euclidean balls. Conversely, for high-dimensional problems with complex geometries, projection-free architectures utilizing Linear Optimization Oracles offer a critical computational advantage by avoiding expensive quadratic projections, albeit sometimes at the cost of weaker dynamic regret guarantees. Furthermore, for safety-critical applications requiring strict adherence to limitations, general convex optimization frameworks have emerged as the necessary solution, shifting the focus from mere feasibility to the rigorous control of Cumulative Constraint Violations.

4. Unconstrained Online Optimization Algorithms

Conventional research in OCO typically predicates on the assumption that the learner’s decision space $\mathcal{W}$ is a bounded convex set. However, a multitude of practical applications—ranging from online prediction and classification to regression and portfolio management—naturally operate within unconstrained decision spaces, such as the entire Euclidean space ${\mathbb{R}}^n$ or a Hilbert space $\mathcal{H}$. The unconstrained setting introduces unique theoretical challenges for OCO. Standard algorithms, such as Online Gradient Descent with fixed learning rates, perform optimally under bounded settings but falter in unconstrained environments. Specifically, to guarantee sublinear regret with respect to an arbitrary comparator u, these algorithms typically necessitate prior knowledge of the norm of the comparator $\parallel u\parallel$ or a bound G on the gradient norms to tune the learning rate. Absent such priors, classical algorithms fail to ensure sublinear regret.

To circumvent this reliance on prior knowledge, the research community has pivoted towards adaptive or parameter-free algorithms capable of automatically adapting to the unknown comparator norm $\parallel u\parallel$ and the gradient norm bound G. Pioneering this direction, Streeter and McMahan (2012) introduced innovative unconstrained OCO algorithms that, without any prior information on $R={\parallel u\parallel }_2$, achieve near-optimal regret bounds for any $u\in {\mathbb{R}}^n$, typically of the form $\mathcal{O}(\parallel u{\parallel }_2\sqrt{Tlog\left(\right(1+{\parallel u\parallel }_2\left)\sqrt{T}\right)})$ [40]. The crux of these algorithms lies in the dynamic adjustment of the learning rate based on historical gradient information. A defining characteristic is their ability to maintain constant regret with respect to the origin ($u=0$). The theoretical framework of Reward–Regret Duality has proven instrumental in this domain, transforming the regret minimization problem into the maximization of a specific form of reward. By designing appropriate potential functions $F_T$ and ensuring the algorithm’s cumulative reward is lower-bounded by $F_T(\sum g_t)$, one can derive regret bounds involving the Fenchel conjugate $F_T^{*}$. McMahan (2014) further synthesized this framework with Minimax analysis, employing Normal distribution approximation to design algorithms yielding refined regret bounds of the form $\mathcal{O}(\parallel u\parallel \sqrt{Tlog\parallel u\parallel \sqrt{T}})$, which matches the optimal bound when T is known [41]. Subsequently, Orabona (2016) proposed a framework grounded in the Coin Betting metaphor, viewing the OCO problem as a process of maximizing wealth through sequential betting [42]. By leveraging optimal betting strategies such as the Krichevsky–Trofimov estimator, researchers can derive simple yet parameter-free OCO algorithms that attain optimal regret guarantees.

While the aforementioned parameter-free algorithms successfully obviate the need for a known $\parallel u\parallel$, they predominantly assume bounded gradients or, at a minimum, knowledge of a gradient bound G. The problem becomes significantly more formidable when G is also unknown. Cutkosky (2017) formally proved an impossibility result: algorithms attempting to simultaneously adapt to unknown $\parallel u\parallel$ and unknown G may incur exponential regret penalties, rendering sublinear regret guarantees unattainable [43]. To navigate the potential exponential penalty associated with an unknown Lipschitz constant G, Cutkosky (2019) [44] and Mhammedi (2020) [45] proposed compromise regret bounds containing an $\mathcal{O}(\parallel u{\parallel }^{3}G)$ term. This class of bounds exhibits a near-linear dependence on $\parallel u\parallel$ but degrades to a cubic dependence when $\parallel u\parallel$ is large, representing a necessary trade-off to accommodate unknown G while preserving the dominant $\tilde{\mathcal{O}}\left(\sqrt{T}\right)$ dependence. This balance is typically achieved through gradient clipping or proximal hints, where the former restricts the growth of the iterate $w_t$ to control errors, albeit at the cost of additional regret $\mathcal{O}(\parallel u{\parallel }^{3}G)$ for comparators outside the effective range. While polynomial, this penalty is vastly superior to exponential growth.

Recently, Cutkosky and Mhammedi (2024) introduced a novel trade-off strategy offering a regret term of $\mathcal{O}(\parallel u{\parallel }^2+G^2)$ for unknown G [46]. Compared to the prior $\mathcal{O}(\parallel u{\parallel }^{3}G)$ bound, this new result accepts a quadratic dependence on G to avoid the cubic dependence on $\parallel u\parallel$. This trade-off is advantageous when the comparator bound G is not excessively large relative to $\parallel u\parallel \sqrt{T}$, as it mitigates the substantial penalty incurred when $\parallel u\parallel$ is large. Intuitively, when $G\ge \parallel u\parallel \sqrt{T}$ or $\parallel u\parallel \ge G\sqrt{T}$, the ideal term $\tilde{\mathcal{O}}(\parallel u\parallel G\sqrt{T})$ already approaches linear regret, rendering the overhead of $\mathcal{O}(\parallel u{\parallel }^2+G^2)$ relatively negligible. Conversely, in the ideal scenario where regret is sublinear, provided G is not exceptionally large, the penalty of $\mathcal{O}(\parallel u{\parallel }^2+G^2)$ is generally smaller than $\mathcal{O}(\parallel u{\parallel }^{3}G)$.

Having partially resolved the dual challenges of adapting to unknown $\parallel u\parallel$ and unknown G, research on unconstrained OCO has not stagnated. Instead, the focus has shifted towards extending the OCO framework to enhance algorithmic robustness and address more complex, realistic scenarios, such as learning with noisy gradients or under dynamic environments. Handling noisy gradients in unconstrained OCO is particularly challenging because many parameter-free algorithms are highly sensitive to the scale of the gradients. Unbounded noise ${\xi }_t$ can lead to unbounded observed gradients ${\tilde{g}}_t$, potentially violating algorithmic assumptions and causing divergence or extreme regret. Even when noise is bounded or light-tailed, the problem remains non-trivial, as unconstrained algorithms may generate iterates $w_t$ that grow exponentially to accommodate large comparators u, thereby amplifying the impact of noise and rendering standard concentration inequalities inapplicable. Addressing sub-exponential noise, Orabona (2019) [47] and Hoeven (2018) [48] extended the Coin Betting and potential function frameworks, respectively. Their algorithms ensure a lower bound on expected rewards rather than non-negativity and incorporate the noise variance ${\sigma }^2$ into the potential function design. Hoeven (2018) [48] further exploited the symmetry of noise distributions (akin to Laplace noise in local differential privacy) to achieve adaptive regret bounds without knowing noise parameters, ultimately attaining optimal expected regret bounds of $\mathbb{E}\left[R_T\left(u\right)\right]=\tilde{\mathcal{O}}(\parallel u\parallel \sqrt{(G^2+{\sigma }^2)T})$ or data-dependent bounds $\tilde{\mathcal{O}}(\parallel u\parallel \sqrt{\sum \parallel {\tilde{g}}_t{\parallel }^2})$.

For the more arduous setting of heavy-tailed noise, Zhang and Cutkosky (2022) employed gradient clipping techniques to transform heavy-tailed gradients into sub-exponential ones with bounded bias [49]. Although this bounds the gradients ${\widehat{g}}_t$, it introduces bias. To overcome the failure of concentration inequalities caused by the bias and the large norm of iterates $w_t$, they designed novel regularizers capable of offsetting the bias and the variance relative to the iterates, ultimately achieving optimal $\tilde{\mathcal{O}}(\parallel u\parallel T^{1/p})$ regret with high probability. Most recently, Zhang and Cutkosky (2025) investigated the even more challenging scenario where gradient information ${\tilde{g}}_t$ may be subject to adversarial corruption, making no assumptions about the nature of the corruption [50]. By integrating gradient clipping with a generalized regularization strategy, they proposed algorithms capable of achieving $\mathcal{O}(\parallel u\parallel G(\sqrt{T}+k))$ regret when G is known (where k measures the corruption level), while requiring an extra cost when G is unknown. Robustness in unconstrained settings is notably difficult, as even minor corruptions can trigger explosive growth in iterates $w_t$, leading to massive regret.

The evolution of Unconstrained Online Convex Optimization has traversed a trajectory from relying on prior knowledge to pursuing fully adaptive and parameter-free methodologies. Through the introduction of powerful theoretical tools such as Reward–Regret Duality, Coin Betting, and Black-Box Reductions, researchers have successfully designed algorithms that achieve near-optimal regret bounds under unknown comparator norms and, to a significant extent, unknown gradient norms. In recent years, these foundational ideas have been further extrapolated to encompass more intricate and realistic settings, including feedback with noise and dynamic environments. Currently, the field of Unconstrained OCO presents numerous avenues for exploration. Key directions include: finding better trade-offs for unknown G; designing algorithms that auto-adapt to noise parameters or regularization parameters; achieving data-dependent bounds in broader contexts; extending high-probability guarantees; further enhancing robustness against adversarial corruption; and improving the computational efficiency of complex algorithms. Ultimately, bridging the performance gap between online learning and stochastic optimization in both theory and practice remains a central objective.

The evolution of unconstrained OCO represents a decisive paradigm shift from manual parameter tuning to fully automated adaptivity. While traditional methods struggle without prior knowledge of comparator norms or gradient bounds, modern parameter-free algorithms, leveraging frameworks such as Reward–Regret Duality and Coin Betting, successfully achieve near-optimal regret in black-box settings. Current research frontiers are now extending these adaptive capabilities to ensure algorithmic robustness, specifically aiming to maintain stability in the face of heavy-tailed noise and adversarial gradient corruptions where standard concentration inequalities may fail.

5. Applications of Adaptive Online Convex Optimization

5.1. Applications in Power and Energy Systems

In the domain of power and energy systems, the Online Convex Optimization framework offers robust theoretical and algorithmic underpinnings for navigating the high stochasticity of renewable energy sources, demand-side fluctuations, and evolving market dynamics [51]. Traditional power system operations have historically relied on the accurate forecasting of future loads, renewable generation outputs, and market prices [52]. However, such deterministic forecasting becomes increasingly precarious and often unreliable under high penetration levels of renewables and within complex market mechanisms. OCO is intrinsically suited for these scenarios, which are characterized by sequential decision-making under incomplete information and unknown future states. Within this framework, the operation of a power system is modeled as an online game process. In each decision period t, the system operator must determine a decision variable $x_t$ prior to acquiring the complete information for that period. This decision $x_t$ is strictly confined within a convex feasible region defined by physical constraints, such as generator ramping limits, energy storage capacity, and transmission line flow limits. Only after the decision is executed does the environment reveal the cost function for the current period. This cost function, $f_t$, encapsulates all uncertainty factors observed at that instant, such as real-time load demand, actual renewable generation, and market clearing prices. Typically, $f_t$ is modeled as a convex function, exemplifying scenarios like quadratic power generation costs or linear electricity procurement costs [53].

In the context of Microgrids and Source-Grid-Load-Storage collaborative scenarios, which face the rigorous challenges of low predictability, high volatility, and deep coupling, OCO is naturally predisposed for online scheduling due to its non-anticipatory nature. Recent scholarship has substantively deepened this application [54]. On one hand, by integrating multi-timescale energy storage and renewable uncertainty into a unified OCO framework, and employing hindsight optimal trajectories for training, researchers have realized online strategies that are asymptotically optimal in the long run while remaining robust to short-term perturbations. On the other hand, by modeling scenarios such as Peer-to-Peer (P2P) energy trading and shared energy storage as Distributed OCO problems, recent works have utilized joint metrics of regret and constraint violation to provide convergence and feasibility guarantees under privacy preservation and communication bandwidth constraints [55]. This trajectory echoes the broader trend of online optimization amidst high renewable penetration: as traditional offline scheduling struggles to cope with rapid variability and uncertainty, the online paradigm—represented by OCO—has emerged as a predominant methodology for multi-timescale scheduling and control due to its inherent robustness.

5.2. Applications in Network Optimization and Communication

In the realm of network optimization and communications, the OCO framework constitutes a formidable theoretical apparatus for addressing dynamic and stochastic resource allocation problems [56]. This paradigm effectively models network management processes—such as wireless spectrum allocation, data center traffic scheduling, and network slicing—as sequential games. In each time slot, the network controller is compelled to make resource allocation decisions under incomplete information, strictly adhering to convex constraints defined by capacity, power, and interference limits [57]. It is only after the decision is executed that the environment discloses the realized feedback for that slot, specifically the actual loss functions and the constraint functions. Given the highly non-stationary nature of communication environments, conventional regret analysis anchored on static optimal solutions often proves inadequate. Consequently, research has pivoted towards performance metrics that better align with engineering realities. The first is Dynamic Regret, which measures the performance deviation of the online algorithm from the hindsight optimal dynamic trajectory, thereby precisely quantifying the algorithm’s capability to track time-varying optima. The second is Cumulative Constraint Violation, a metric that permits transient violations while guaranteeing long-term average satisfaction, offering provable assurances for critical Service Level Agreements (SLAs) such as throughput, energy efficiency, and delay bounds.

Furthermore, communication systems frequently grapple with challenges such as feedback latency and bandwidth-constrained observations [7]. These realities necessitate the rigorous integration of delayed feedback mechanisms into online algorithms. A substantive body of work has established a unified methodology for the “black-box” reduction of non-delayed algorithms into delay-tolerant variants, effectively treating delay as an additive penalty term within the regret analysis. In scenarios restricted to zero-order observations—where only function values are observable—Bandit-OCO frameworks are employed. These approaches construct unbiased gradient estimators via one-point or two-point sampling and integrate time-varying constraints through primal-dual schemes. This methodology theoretically preserves sublinear regret guarantees while maintaining engineered control over overhead, a capability that has been empirically validated in domains such as IoT resource management and WLAN proportional fairness problems [58].

5.3. Applications in Quantitative Finance and Portfolio Management

In the domain of quantitative finance and portfolio management, the OCO framework constitutes a canonical and robust non-stochastic game-theoretic framework [59,60]. This paradigm models the investment process as a sequential game: at each trading epoch, the investor must allocate a portfolio vector within the probability simplex constraints, after which the market reveals the asset returns for that period. In this setting, the preeminent loss function is the negative logarithmic loss. This choice is theoretically grounded in its direct correspondence to maximizing the asymptotic geometric growth rate, aligning perfectly with long-term compounding objectives while retaining a favorable convex structure [61].

A pivotal mathematical attribute in this domain is the property of exp-concavity inherent to the logarithmic loss function [62]. Leveraging this curvature allows algorithms to transcend the standard performance of first-order methods by employing second-order methods, such as the Online Newton Step (ONS) [63]. Consequently, this reduces the cumulative regret bound significantly from $O\left(\sqrt{T}\right)$ to the logarithmic order $O(logT)$, a feat achievable on the simplex and other norm-constrained sets.

Regarding performance evaluation, OCO strategies are typically benchmarked against the hindsight Best Constant Rebalanced Portfolio (BCRP). The primary algorithmic objective is to minimize the regret relative to this static optimal strategy. Cover’s seminal work on “Universal Portfolios” first established theoretical asymptotic optimality without reliance on statistical assumptions about the market generation process. Subsequently, the Multiplicative Update algorithm proposed by Helmbold et al. provided a computationally and storage-efficient implementation for this purpose [64].

The selection of OCO algorithms is strictly contingent upon information availability. In the full-information setting, where the prices of all underlying assets are observable, Online Newton or Step-based algorithms are the preferred methodologies [65]. However, in more restrictive environments—such as the Bandit-OCO setting where the investor observes only the aggregate return of their own portfolio, or scenarios plagued by feedback latency and missing data—recourse to zeroth-order methods is necessary [66]. These approaches typically construct unbiased stochastic gradient estimators via one-point or two-point function evaluations. Even when integrated with finance-specific constraints like transaction costs or risk limits, these methods preserve sublinear regret guarantees while maintaining algorithmic simplicity and implementation viability.

5.4. Applications in Emerging Frontiers

Beyond traditional engineering domains, the OCO framework has become increasingly pivotal in modern artificial intelligence applications, particularly within recommendation systems where data dimensionality is vast. In this context, the problem is often modeled as online matrix completion where the objective is to recommend items by learning a low-rank preference matrix. The primary challenge lies in the trace-norm constraint used to induce low-rank structures, as the standard Euclidean projection requires performing a Singular Value Decomposition (SVD) at every iteration. This operation is computationally prohibitive for large-scale platforms. Consequently, projection-free algorithms such as the Online Frank–Wolfe method have become the standard solution in this domain. By replacing the expensive SVD with a linear approximation that only requires computing the top singular vector, these methods achieve a scalable balance between recommendation accuracy and real-time responsiveness.

Furthermore, the principles of OCO are fundamentally reshaping control strategies in robotics and the optimization of foundation models. In robotics, specifically for Vision-Language-Action (VLA) models and embodied AI, the environment is inherently dynamic and unpredictable [67]. Constrained OCO provides a rigorous framework for Model Predictive Control (MPC) where safety constraints, such as collision avoidance, must be satisfied in an online fashion. The metric of cumulative constraint violation is particularly relevant here as it allows for the theoretical certification of safety over a trajectory. Simultaneously, in the realm of foundation models, the adaptive learning rates derived from OCO theory, such as those in AdaGrad and Adam, remain the bedrock of training Large Language Models. Recent research is further exploring the connection between OCO and Reinforcement Learning from Human Feedback (RLHF), modeling the alignment process as an online game where the model must regret-minimize against a human reward function, thereby extending OCO from convex optimization to the optimization of deep, non-convex networks [68].

5.5. Synthesis: Bridging Applications with Theoretical Frameworks

To facilitate the selection of appropriate algorithms for practitioners, we synthesize the distinct theoretical characteristics of the discussed application domains, mapping them back to the algorithmic taxonomies established in Section 4 and Section 5.

In the domain of Power and Energy Systems, the decision geometry is typically characterized by box constraints and ramp constraints. While geometrically simple, these boundaries are physically strict, presenting a primary challenge rooted in their safety-critical nature. This domain usually operates under a Full Information feedback model where costs, such as fuel consumption, are revealed ex-post, though often subject to stochastic noise from renewable generation. Consequently, Constrained OCO methods utilizing Lagrangian or penalty formulations emerge as the optimal choice. Specifically, algorithms that minimize Cumulative Constraint Violation are preferred over standard projection methods to rigorously manage the safety budget while optimizing operational costs.

Conversely, Network Optimization presents a different structural profile where constraints are often coupled over time—such as queue stability and average latency buffers—distinguishing them from static geometric sets. The environment is frequently restricted to Bandit Feedback, where only scalar throughput is observed, or Delayed Feedback due to network latency. To address these conditions, Drift-plus-Penalty frameworks and Bandit Convex Optimization algorithms provide the best fit. These methods naturally handle long-term coupled constraints through virtual queue stability and operate efficiently without full gradient observations.

In the context of Quantitative Finance, the decision set is strictly defined by the Probability Simplex ($\sum x_i=1,x_i\ge 0$) representing portfolio weights. This domain operates within a classic Full Information setting where market returns for all assets are publicly observable after each round. Here, the Online Newton Step (ONS) and Exponentiated Gradient (EG) methods are theoretically superior. Since the logarithmic loss function inherent to finance is exp-concave, these second-order or entropic-regularized methods can achieve the optimal logarithmic regret rate $O(logT)$, significantly outperforming the $O\left(\sqrt{T}\right)$ rate of standard Gradient Descent [63,64].

Finally, within emerging frontiers such as Robotics and AI, constraints involve complex, often non-convex geometries like collision-free corridors or high-dimensional trace-norms found in recommendation systems. Feedback in these scenarios is often Black-box or based on human preference, lacking explicit gradient information. Therefore, Projection-free variants and Parameter-free adaptive methods become essential. The former circumvents computationally prohibitive projections such as SVD, while the latter adapts to the unknown scales inherent in black-box AI environments.

6. Challenges and Future Directions

6.1. Algorithm Generalization and Adaptivity

The field of OCO is currently undergoing a critical paradigm shift from classical static environments to dynamic and unknown territories. Traditional OCO algorithms are predominantly engineered to minimize static regret, benchmarking performance against a single fixed decision in hindsight [24]. However, this metric often proves inadequate in non-stationary environments where the optimal decision drifts over time. Consequently, a pivotal future direction lies in the development of algorithms capable of minimizing dynamic regret, thereby endowing agents with the capability to effectively track time-varying optimal sequences [69].

This evolution not only necessitates resilience against adversarial changes but also catalyzes the demand for adaptive algorithms. Ideally, such algorithms should operate without prior knowledge of the environment’s nature—whether stochastic or adversarial. They must exhibit universality: guaranteeing $O\left(\sqrt{T}\right)$ regret in adversarial settings while automatically accelerating convergence to $O(logT)$ or even $O\left(1\right)$ when the environment manifests stochasticity or strong convexity. The ultimate frontier of this generalization is the extension of theoretical guarantees from convex settings to the more challenging domain of Online Non-Convex Optimization (ONCO) [70]. This leap is fundamental for providing solid theoretical foundations for the streaming training of deep learning models and the design of adaptive optimizers.

6.2. Computational Efficiency and Large-Scale Data

In terms of computational efficiency and large-scale data, OCO faces rigorous tests as it transitions from theoretical constructs to large-scale engineering implementations. With the explosive growth of data dimensionality and velocity, the computational bottlenecks of traditional algorithms have become increasingly conspicuous. Many classic algorithms that achieve optimal regret—particularly second-order methods like the Online Newton Step [63]—incur prohibitive costs: $O\left(d^2\right)$ for storage and $O\left(d^3\right)$ for matrix inversion per iteration (where d is the dimension). Such complexity is rendered untenable in high-dimensional streaming scenarios. Therefore, future research must prioritize scalability, spawning several key research avenues. First, there is a pressing need for the development of low-complexity algorithms, such as efficient first-order methods or those leveraging sparsity and randomization (Stochastic OCO) to approximate gradients or Hessian matrices. Second, the field is increasingly pivoting towards Distributed OCO, aiming to design algorithms that facilitate collaborative learning under constraints of limited communication, asynchronous updates, or privacy preservation when data is physically dispersed or massive in scale [71]. Finally, significant attention is being directed towards Projection-Free paradigms—exemplified by methods like Online Frank–Wolfe —which circumvent expensive Euclidean projections onto complex convex sets via computationally cheaper Linear Optimization Oracles, thereby significantly enhancing the computational feasibility of OCO for large-scale constrained problems [37].

A robust assessment of existing algorithms reveals a distinct divergence between theoretical elegance and practical scalability which dictates their suitability for implementation. While second-order methods such as the Online Newton Step achieve the theoretically optimal logarithmic regret, their practical utility is severely limited by the requirement to compute and invert a Hessian matrix at every step. This operation incurs a computational cost cubic in the dimension, rendering such methods largely theoretical and non-scalable for high-dimensional applications where the feature space exceeds a few thousand dimensions. Consequently, for large-scale systems prevalent in big data scenarios, first-order methods remain the pragmatic choice despite their slower convergence rates. Furthermore, when the constraint set involves complex geometries like the trace norm ball used in recommender systems, even standard gradient descent becomes computationally prohibitive due to the Singular Value Decomposition required for projection. In these specific contexts, projection-free algorithms such as Online Frank–Wolfe emerge as the truly scalable solutions. Although they often yield a suboptimal regret rate compared to projection-based counterparts, their linear per-iteration complexity makes them indispensable for real-time decision-making where computational latency is the primary bottleneck.

6.3. Integration with Deep Learning

The synergistic integration of OCO and deep learning represents one of the most vibrant frontiers in modern AI. This intersection not only presents formidable theoretical challenges but also offers profound insights for algorithm design. The training processes of deep learning, particularly those predicated on Stochastic Gradient Descent (SGD) and its variants (e.g., Adam [72]), are intrinsically sequential online learning processes. The OCO framework—especially its derivatives in Online Mirror Descent and Regularized Follow the Leader—provides a core theoretical toolkit for elucidating why current adaptive optimizers converge efficiently.

However, a fundamental discrepancy remains: the cornerstone of OCO theory is convexity, whereas the loss landscapes of deep neural networks are highly non-convex. Consequently, future research aims to generalize OCO’s regret analysis tools from the convex domain to the non-convex realm, shifting the objective from guaranteeing convergence to global optima towards providing theoretical assurances for finding high-quality local optima or saddle points. Furthermore, this integration has given rise to Online Deep Learning (ODL), which investigates how to efficiently update deep models in continuous streaming scenarios [73]. This requires algorithms to handle non-stationary data distributions and rapidly adapt to new patterns—tasks for which OCO serves as a natural and powerful framework.

6.4. Algorithm Robustness and Uncertainty

Regarding robustness and uncertainty, OCO is striving to break the idealized assumption of perfect information inherent in classical models to address the pervasive feedback uncertainties of the real world. While traditional OCO assumes immediate, accurate, and complete feedback post-decision, future inquiries must focus on algorithmic robustness in more stringent, information-limited scenarios. Key directions include handling Bandit Feedback, where the algorithm observes only the function value rather than the gradient at each step [66]—a critical capability in black-box optimization or communication-constrained bandwidth settings. Furthermore, research must address imperfect feedback scenarios characterized by delayed, missing, or noisy data, requiring algorithms to maintain sublinear regret guarantees despite incomplete or asynchronous information [74]. Additionally, addressing constraint uncertainty remains vital, where algorithms must not only manage uncertainty in loss functions but also navigate time-varying constraints or constraints revealed only after decisions are made. To this end, a central objective is the design of robust algorithms capable of providing long-term viability guarantees [8]. These algorithms should tolerate transient constraint violations while theoretically ensuring that cumulative violations remain bounded, thereby robustly satisfying system-level average performance requirements even in the face of adversarial perturbations or stochastic noise.

7. Conclusions

This survey has provided a comprehensive examination of the evolutionary landscape of Online Convex Optimization, tracing its trajectory from foundational static frameworks to adaptive, dynamic, and robust methodologies suited for the big data era. We systematically categorized the algorithmic advancements into constrained and unconstrained paradigms, revealing a fundamental shift in design philosophy: the transition from relying on prior environmental knowledge to achieving fully adaptive, parameter-free learning.

In the realm of Constrained OCO, we synthesized the trade-offs between computational efficiency and theoretical guarantees. While projection-based methods (e.g., OGD, Bregman projections) continue to offer optimal regret bounds, the emergence of projection-free algorithms (e.g., Online Frank–Wolfe) and general convex optimization approaches has successfully addressed the computational bottlenecks inherent in high-dimensional and complex constraint sets. Crucially, the focus has expanded from minimizing regret alone to simultaneously controlling cumulative constraint violations, a necessity for safety-critical applications. In the Unconstrained domain, our review highlights the breakthrough of parameter-free algorithms. By leveraging frameworks such as Reward–Regret Duality and Coin Betting, recent research has overcome the dependency on unknown comparator norms and gradient scales, enabling robust performance in unpredictable environments. We further demonstrated the practical vitality of these theoretical constructs through their transformative applications in power systems, network optimization, and quantitative finance, where they effectively manage stochasticity and non-stationarity.

Looking forward, the field of OCO stands at a pivotal juncture. The integration of OCO principles with Deep Learning—particularly in bridging the gap between convex theory and non-convex loss landscapes—remains a nascent yet promising frontier. Furthermore, ensuring algorithmic robustness against adversarial corruption and heavy-tailed noise in distributed, projection-free settings will be essential for the next generation of resilient AI systems. Ultimately, the continued evolution of OCO promises not only to refine the theoretical boundaries of online learning but also to provide the scalable, adaptive engines required for future intelligent systems.