New Theory And Algorithms For Convex Optimization With Non-Standard Structures