We explore the possibilities enabled by the spatiotemporal modulation of graphene's conductivity to realize magnetic-free isolators at terahertz and infrared frequencies. To this purpose, graphene is loaded with periodically distributed gates that are time-modulated. First, we investigate plasmonic isolators based on various mechanisms such as asymmetric bandgaps and interband photonic transitions and we demonstrate isolation levels over 30 dB using realistic biasing schemes. To lessen the dependence on high-quality graphene able to support surface plasmons with low damping, we then introduce a hybrid photonic platform based on spatiotemporally modulated graphene coupled to high-Q modes propagating on dielectric waveguides. We exploit transversal Fabry-Perot resonances appearing due to the finite-width of the waveguide to significantly boost graphene/waveguide interactions and to achieve isolation levels over 50 dB in compact structures modulated with low biasing voltages. The resulting platform is CMOS-compatible, exhibits an overall loss below 4 dB, and is robust against graphene imperfections. We also put forward a theoretical framework based on coupled-mode theory and on solving the eigenstates of the modulated structure that is in excellent agreement with full-wave numerical simulations, sheds light in the underlying physics that govern the proposed isolators, and speeds-up their analysis and design. We envision that the proposed technology will open new and efficient routes to realize integrated and siliconcompatible isolators, with wide range of applications in communications and photonic networks.