Tensors (aka multiway arrays) can be instrumental in revealing latent correlations residing in high dimensional spaces. Despite their applicability to a broad range of applications in machine learning, speech recognition, and imaging, inconsistencies between tensor and matrix algebra have been complicating their broader utility. Researchers seeking to overcome those discrepancies have introduced several different candidate extensions, each introducing unique advantages and challenges. In this talk, we review some of the common tensor definitions, discuss their limitations, and introduce our tensor product framework which permits the elegant extension of linear algebraic concepts and algorithms to tensors. Following introduction of fundamental tensor operations, we discuss in further depth tensor decompositions and in particular the tensor SVD (t-SVD) and its randomized variant, which can be computed efficiently in parallel. We present details of the t-SVD, theoretical results, and provide numerical results that show the promise of our approach for compression and analysis of operators and datasets, highlighting examples such as facial recognition and model reduction.