Working memory has often been modeled and conceptualized as a kind of binary (bistable) memory switch, where stimuli turn on plateau-like persistent activity in subsets of cells, in line with many in vivo electrophysiological reports. A potentially related form of bistability, termed up- and down-states, has been studied with regard to its synaptic and ionic basis in vivo and in reduced cortical preparations. Also single cell mechanisms for producing bistability have been proposed and investigated in brain slices and computationally. Recently, however, it has been emphasized that clear plateau-like bistable activity is rather rare during working memory tasks, and that neurons exhibit a multitude of different temporally unfolding activity profiles and temporal structure within their spiking dynamics. Hence, working memory seems to be a highly dynamical neural process with yet unknown mappings from dynamical to computational properties. Empirical findings on ramping activity profiles and temporal structure will be reviewed, as well as neural models that attempt to account for it and its computational significance. Furthermore, recent in vivo, neural culture, and in vitro preparations will be discussed that offer new possibilities for studying the biophysical mechanisms underlying computational processes during working memory. These preparations have revealed additional evidence for temporal structure and spatio-temporally organized attractor states in cortical networks, as well as for specific computational properties that may characterize synaptic processing during high-activity states as during working memory. Together such findings may lay the foundations for highly dynamical theories of working memory based on biophysical principles.