Mathematical Formalisms and Data-Driven Approaches for Coarse-Graining of Multi-Scale Finite Element Discretizations