Functional MRI using blood oxygenation level-dependent (BOLD) contrast indirectly probes neuronal activity via evoked cerebral blood volume (CBV) and oxygenation changes. Thus, its spatio-temporal characteristics are determined by vascular physiology and MRI parameters. In this paper, we focus on the spatial distribution and time course of the fMRI signal and their magnetic field strength dependence. Even though much is still unknown, the following consistent picture is emerging: a) For high spatial resolution imaging, fMRI contrast-to-noise increases supra-linearly with field strength. b) The location and spacing of penetrating arteries and ascending veins in the cortical tissue are not correlated to cortical columns, imposing limitations on achievable point-spread function (PSF) in fMRI. c) Baseline CBV distribution may vary over cortical layers biasing fMRI signal to layers with high CBV values. d) The largest CBV change is in the tissue microvasculature, less in surface arteries and even less in pial veins. e) Venous CBV changes are only relevant for longer stimuli, and oxygenation changes are largest in post-capillary blood vessels. f) The balloon effect (i.e. slow recovery of CBV to baseline) is located in the tissue, consistent with the fact that the post-stimulus undershoot has narrower spatial PSF than the positive BOLD response. g) The onset time following stimulation has been found to be shortest in middle/lower layers, both in optical imaging and high-resolution fMRI, but we argue and demonstrate with simulations that varying signal latencies can also be caused by vascular properties and, therefore, may potentially not be interpreted as neural latencies. With simulations, we illustrate the field strength dependency of fMRI signal transients, such as the adaptation during stimulation, initial dip and the post-stimulus undershoot. In sum, vascular structure and function impose limitations on the achievable PSF of fMRI and give rise to complex fMRI transients, which contain time-varying amount of excitatory and inhibitory neuronal information. Nevertheless, non-invasive fMRI at ultra-high magnetic fields not only provides high contrast-to-noise but also an unprecedented detailed view on cognitive processes in the human brain.