Charged jet production cross sections in p-Pb collisions at $\sqrt{s_{\rm NN}} = 5.02$ TeV measured with the ALICE detector at the LHC are presented. Using the anti-$k_{\rm T}$ algorithm, jets have been reconstructed in the central rapidity region from charged particles with resolution parameters $R = 0.2$ and $R = 0.4$. The reconstructed jets have been corrected for detector effects and the underlying event background. To calculate the nuclear modification factor, $R_{\rm pPb}$, of charged jets in p-Pb collisions, a pp reference was constructed by scaling previously measured charged jet spectra at $\sqrt{s} = 7$ TeV. In the transverse momentum range $20 \le p_{\rm T,ch\ jet} \le 120$ GeV/$c$, $R_{\rm pPb}$ is found to be consistent with unity, indicating the absence of strong nuclear matter effects on jet production. Major modifications to the radial jet structure are probed via the ratio of jet production cross sections reconstructed with the two different resolution parameters. This ratio is found to be similar to the measurement in pp collisions at $\sqrt{s} = 7$ TeV and to the expectations from PYTHIA pp simulations and NLO pQCD calculations at $\sqrt{s_{\rm NN}} = 5.02$ TeV.