In many practical imaging scenarios, including computed tomography and magnetic resonance imaging (MRI), the goal is to reconstruct an image from few of its Fourier domain samples. Many state-of-the-art reconstruction techniques, such as total variation minimization, focus on discrete 'on-the-grid' modelling of the problem both in spatial domain and Fourier domain. While such discrete-to-discrete models allow for fast algorithms, they can also result in sub-optimal sampling rates and reconstruction artifacts due to model mismatch. Instead, we present a framework that allows for the recovery of a continuous domain 'off-the-grid' representation of piecewise constant images from the optimal number of Fourier samples. The main idea is to model the edge set of the image as the level-set curve of a continuous domain band-limited function. Sampling guarantees can be derived for this framework by investigating the algebraic geometry of these curves. Finally, we show how this model can be put into a robust and efficient optimization framework by posing signal recovery entirely in Fourier domain as a structured low-rank matrix completion problem, and demonstrate the benefits of this approach over standard discrete methods in the context of undersampled MRI reconstruction.