A Second look at Exponential and Cosine Step Sizes: Simplicity, Adaptivity, and Performance

Keyphrases

• Adaptivity 100%
• Noise Level 100%
• Cosine 100%
• Hyperparameters 100%
• Stochastic Gradient Descent 50%
• Convergence Rate 50%
• Optimal Performance 50%
• Nonconvex Functions 50%
• Large-scale Machine Learning 50%
• Stochastic Gradient 50%
• Machine Learning Models 50%
• Almost Optimal 50%
• Deep Learning Architectures 50%
• Adaptive Step Size 50%

Computer Science

• Gradient Descent 100%
• Machine Learning 100%
• Convex Function 100%
• Optimal Performance 100%
• Convergence Rate 100%
• Deep Learning 100%

Mathematics

• Step Size 100%
• Cosine 100%
• Stochastics 25%
• Polynomial 12%
• Proving Convergence 12%
• Convergence Rate 12%
• Deep Learning 12%
• Convex Function 12%