The ALICE collaboration presents the first measurement of the dijet invariant mass spectra of charged-particle jets in pp and p-Pb collisions at $\sqrt{s_{\rm NN}} = 5.02$ TeV. Charged particles in the mid-pseudorapidity region, $|η| <~ 0.9$, are clustered into jets using the anti-$k_{\rm T}$ algorithm with a resolution parameter $R = 0.4$. The leading and subleading jets are required to have a transverse momentum $p_{\rm T} > 20$ GeV$/c$ and to be contained within $\left|η_{\rm jet}\right| <~ 0.5$. The dijet invariant mass spectrum and the nuclear modification factor $R_{\rm pA}$ are presented in the low-mass region of 75 to 150 GeV/$c^2$. The nuclear modification factor for charged-particle dijet invariant mass is consistent with unity. This is in line with previous small-system jet studies. Comparisons with Monte Carlo simulations suggest that the low-mass region is sensitive to anti-shadowing effects on parton densities in the nucleus, however, the expected signal is subtle and below the present experimental sensitivity.