Synaptic Basis of Cortical Persistent Activity: the Importance of NMDA Receptors to Working Memory

Xiao-Jing Wang

doi:10.1523/JNEUROSCI.19-21-09587.1999

. 1999 Nov 1;19(21):9587–9603. doi: 10.1523/JNEUROSCI.19-21-09587.1999

Synaptic Basis of Cortical Persistent Activity: the Importance of NMDA Receptors to Working Memory

Xiao-Jing Wang ¹

PMCID: PMC6782911 PMID: 10531461

Abstract

Delay-period activity of prefrontal cortical cells, the neural hallmark of working memory, is generally assumed to be sustained by reverberating synaptic excitation in the prefrontal cortical circuit. Previous model studies of working memory emphasized the high efficacy of recurrent synapses, but did not investigate the role of temporal synaptic dynamics. In this theoretical work, I show that biophysical properties of cortical synaptic transmission are important to the generation and stabilization of a network persistent state. This is especially the case when negative feedback mechanisms (such as spike-frequency adaptation, feedback shunting inhibition, and short-term depression of recurrent excitatory synapses) are included so that the neural firing rates are controlled within a physiological range (10–50 Hz), in spite of the exuberant recurrent excitation. Moreover, it is found that, to achieve a stable persistent state, recurrent excitatory synapses must be dominated by a slow component. If neuronal firings are asynchronous, the synaptic decay time constant needs to be comparable to that of the negative feedback; whereas in the case of partially synchronous dynamics, it needs to be comparable to a typical interspike interval (or oscillation period). Slow synaptic current kinetics also leads to the saturation of synaptic drive at high firing frequencies that contributes to rate control in a persistent state. For these reasons the slow NMDA receptor-mediated synaptic transmission is likely required for sustaining persistent network activity at low firing rates. This result suggests a critical role of the NMDA receptor channels in normal working memory function of the prefrontal cortex.

Keywords: working memory, prefrontal cortex, persistent activity, NMDA receptor, synaptic dynamics, short-term plasticity, rate control, synchronization, spiking neuron model

Working memory is a fundamental cognitive function, by virtue of which information can be actively retained for seconds and used in the brain (Baddeley, 1986; Fuster, 1988; Goldman-Rakic, 1995). Its neuronal correlate, delay-period activity, has been widely documented by unit recording studies of behaving monkeys (Fuster and Alexander, 1971; Kubota and Niki, 1971;Miyashita and Chang, 1988; Gnadt and Andersen, 1988; Funahashi et al., 1989; Miller et al., 1996; Chafee and Goldman-Rakic, 1998; Rainer et al., 1998; Romo et al., 1999). For example, in a visuospatial delayed-response experiment (Funahashi et al., 1989), the animal's delayed saccadic eye movement is guided by the short-term memory of a visual cue. Neurons in the dorsolateral prefrontal (PFC) cortex were found to display elevated firing activity during the entire delay period. This persistent activity is tuned to the spatial location of the cue in some cells, but not in other cells. Therefore, there are two distinct aspects of the mnemonic coding by the PFC cells: the persistent nature of the delay-period activity and the formation of the tuned “memory field”.

It is generally assumed that persistent activity is sustained by some kind of reverberating discharge within a recurrent neural network (Hebb, 1949; Amit, 1995). The characteristic horizontal connections found in the superficial layers II–III of the dorsolateral PFC may provide the anatomical substrate for such a recurrent circuit (Levitt et al., 1993; Kritzer and Goldman-Rakic, 1995). However, it remains unknown what are the realistic synaptic properties and circuit dynamics that are required for a robust network-induced persistent activity. Indeed, most previous model studies used simple firing-rate models (Wilson and Cowan, 1973; Amari, 1977; Zipser et al., 1993; Amit et al., 1994; Camperi and Wang, 1998; Moody et al., 1998). Amit and collaborators (Amit et al., 1990; Amit and Tsodyks 1991; Amit and Brunel, 1997) used leaky integrate-and-fire (LIF) spiking neurons but did not take into account realistic postsynaptic current time courses.

In this paper I present a network model of spiking neurons in which synapses are endowed with realistic gating kinetics, based on experimentally measured dynamical properties of cortical synapses. I will focus on how delay-period activity could be generated by neuronally plausible mechanisms; the issue of memory field formation will be addressed in a separate study. A main problem to be investigated is that of “rate control” for a persistent state: if a robust persistent activity necessitates strong recurrent excitatory connections, how can the network be prevented from runaway excitation in spite of the powerful positive feedback, so that neuronal firing rates are low and comparable to those of PFC cells (10–50 Hz)? Moreover, a persistent state may be destabilized because of network dynamics. For example, fast recurrent excitation followed by a slower negative feedback may lead to network instability and a collapse of the persistent state. It is shown that persistent states at low firing rates are usually stable only in the presence of sufficiently slow excitatory synapses of the NMDA type. Functional implications of these results for the role of NMDA receptors in the PFC working memory function are discussed.

MATERIALS AND METHODS

The leaky integrate-and-fire model. To simulate a local recurrent cortical network, I used a network model of leaky integrate-and-fire neurons (Tuckwell, 1988), with either all-to-all or sparse connectivity. Such a network can be viewed as a cortical cell assembly that stores a particular memory item. As a result of Hebbian learning, the internal excitatory recurrent connections are strong and homogeneous, whereas the interactions between this cell assembly and the rest of the circuit are relatively weak and are neglected.

The network model consists of two populations of neurons (N_e pyramidal cells and N_i inhibitory interneurons). Each pyramidal cell obeys the following equation:

C_{m} \frac{{dV}_{m}}{dt} = - I_{L} - I_{AHP} - I_{syn, ee} - I_{syn, ie} + I_{app}

Equation 1

\frac{d [{Ca}^{2 +}]}{dt} = α_{Ca} \sum_{j} δ (t - t_{j}) - [{Ca}^{2 +}] / τ_{Ca},

Equation 2

where C_m is the capacitance, I_app represents the afferent input, and the leak current I_L = g_L(V_m− V_L). I_AHP = g_AHP[Ca²⁺](V_m − V_K) describes a calcium-activated potassium current for spike-frequency adaptation. [Ca²⁺] is incremented by an amount α_Ca with each spike discharge, and decays with a time constant τ_Ca afterwards (cf. Treves, 1993; Y. H. Liu and X.-J. Wang, unpublished observations). I_syn,ee and I_syn,ie are the recurrent synaptic inputs from pyramidal cells and interneurons, respectively.

A spike is discharged each time V_m is driven to reach a firing voltage threshold V_th. Then V_m is reset to V_reset and stays there for an absolute refractory period τ_ref. The intrinsic parameters were calibrated based on the intracellular data of cortical pyramidal neurons (McCormick et al., 1985; Mason and Larkman, 1990; Troyer and Miller, 1997): C_m = 0.5 nF, g_L = 0.025 μS (so that the time constant τ_m = C_m/g_L = 20 msec); V_L = −70, V_th = −52, V_reset = −59 (in mV); τ_ref = 2 msec. The frequency–current curve of an isolated cell has a current threshold I_c = g_L(V_th − V_L) = 0.45 nA. For the adaptation current V_K = −85 mV, α_Ca= 0.2 μM, τ_Ca = 80 msec (Helmchen et al., 1996), and g_AHP will be specified in the text whenever it is not zero.

The interneuron model represents fast-spiking GABAergic cells that do not display spike-frequency adaptation (McCormick et al. 1985). Each interneuron obeys the equation:

C_{m} \frac{{dV}_{m}}{dt} = - I_{L} - I_{syn, ei} + I_{app},

Equation 3

which is similar to Equation 1, except that I_AHP is absent, and I_syn,ei is the recurrent synaptic input from pyramidal cells. Mutually inhibitory interactions among interneurons were not included. The parameter values for the interneurons are (cf.McCormick et al., 1985) C_m = 0.2 nF, g_L = 0.02 μS (τ_m= C_m/g_L = 10 msec); V_L = −65, V_th = −52, V_reset = −60 (in mV); τ_ref = 1 msec. The frequency–current curve of an isolated interneuron has a current threshold I_c = 0.26 nA.

Synaptic kinetics and short-term depression. The EPSC originating from a presynaptic pyramidal cell consists of two components, I_AMPA and I_NMDA. The AMPA receptor (AMPAR)-mediated current I_AMPA = g_AMPAs(V_m − V_E), with V_E = 0 mV. The gating variable s (the fraction of open channels) is described by two first-order kinetics:

\frac{dx}{dt} = φ (α_{x} \sum_{j} δ (t - t_{j}) - x / τ_{x});

Equation 4

\frac{ds}{dt} = φ (α_{s} x (1 - s) - s / τ_{s}),

Equation 5

where the sum is over presynaptic spike times. The scaling factor φ controls the speed of synaptic kinetics without affecting the steady state, φ = 1 unless specified otherwise. For the AMPAR channels, I used τ_x = 0.05 msec and τ_s = 2 msec (the time-to-peak is ∼0.2 msec); α_x = 1 (dimensionless), and α_s = 1 (in msec⁻¹). The NMDA receptor (NMDAR)-mediated current I_NMDA= g_NMDAs(V_m − V_E)/(1 + [Mg²⁺] exp(−0.062 V_m)/3.57) (Jahr and Stevens, 1990), with a voltage dependence controlled by the extracellular magnesium concentration [Mg²⁺] = 1.0 mm. The gating variable s obeys the same types of equations (Eqs. 4, 5), but with τ_x = 2 msec and τ_s = 80 msec (the time-to-peak is ≃8 msec).

This model of excitatory synapses was chosen for three reasons. First, it is based on a plausible kinetic scheme (Wang and Rinzel, 1992;Destexhe et al., 1994). In response to a presynaptic spike, the time course of s has a smooth rising phase and an exponential decay with time constant τ_E = τ_s/φ, that can be matched to the experimental data (Hestrin et al., 1990a; Lester et al., 1990). Second, there is temporal summation and, if the presynaptic firing frequency is high compared to 1/τ_E, s will saturate in the steady state (s ≤ 1) (Fig. 1). The saturation effect is much more significant for the slow NMDAR-mediated EPSC than for the fast AMPAR-mediated EPSC and has important implications for the network dynamical behavior. Finally, the model is sufficiently simple to allow detailed analysis of the network activity.

Fig. 1. — Temporal summation of the NMDAR-mediated EPSCs.A, NMDAR-mediated EPSCs elicited by four stimuli, when the membrane potential is clamped at −40 mV. *Top panel*, Data from a pyramidal neuron in CA1 of the rat hippocampus (redrawn fromHestrin et al., 1990b, with permission). The stimulus is at 25 Hz. Note the significant summation and saturation. These properties are mediated postsynaptically by the NMDARs, because they are absent in the non-NMDR-mediated EPSCs recorded in the same cell at −100 mV.Bottom panel, NMDAR-mediated EPSCs produced by the model synapse (Eqs. 4, 5); the stimulus is at 20 Hz. g_NMDA = 0.07; α_x = 1, τ_x = 2 msec; α_s = 0.3, τ_s = 120 msec. B, NMDAR-mediated EPSCs of the model synapse at various stimulus frequencies R. The EPSC amplitude decreases in time in each train, and its steady state is smaller at higher R. The average current saturates at high R. C, The ratio of the NMDAR-mediated EPSC in the steady state (ΔI_{NMDA, ss}) over its initial value (ΔI_NMDA,0), as function of the stimulus frequency. *Solid curve*, A(R) = 1/(1 + 0.025 ∗ R)², which fits well the simulation data; therefore ΔI_NMDA,ss ∼ 1/R² at high R. D, The average s_NMDA as function of stimulus frequency.Solid curve, s_NMDA = νR/(νR + 1), ν = α_xα_sτ_xτ_s.

The IPSC originating from an interneuron is assumed to be mediated by GABA_A receptors (GABA_ARs), I_GABA = g_GABAs(V_m− V_I), with V_I = V_L = −70 mV (“shunting inhibition”). The gating variable s obeys a simple first-order kinetics with saturation (Wang and Rinzel, 1992):

\frac{ds}{dt} = α_{I} \sum_{j} δ (t - t_{j}^{-}) (1 - s) - s / τ_{I},

Equation 6

with α_I = 0.9 and τ_I = 10 msec. The superscript in t_j⁻ indicates that the increment of s by a spike should be calculated using the value of s immediately before the spike on the right hand side of the equation, Δs = s(t_j⁺) − s(t_j⁻) = α_I(1 − s(t_j⁻)).

Most simulations were done with all-to-all connectivity. In that case a neuron receives synaptic inputs from all neurons in the network, and the summation of synaptic currents is normalized by the number of neurons N. Sparse connectivity was also considered (see Fig.11). There, the coupling between neurons is randomly assigned, with an average number of synapses per neuron M_syn(which is much smaller than N ), and the summation of synaptic currents is normalized by M_syn. The probability that a pair of neurons are connected in either direction is p = M_syn/N.

Fig. 11. — Stability of the persistent state in a sparse network with short-term depression (average number of synapses per neuron M_syn = 100 except for B).A, For each of the five active states in Figure10B, the network model is simulated, whereas the synaptic time constant τ_E is varied systematically. The minimal value of τ_E for which the persistent state was observed is plotted against the firing rate. Thus, the lower is the firing rate, the slower the synapses must be to sustain the network persistent activity. B, The required minimal τ_E is not sensitive to M_syn, as long as the latter is >100.C, D, An example with p_υ = 0.35 and R = 35 Hz. The initial condition for the network simulation was prescribed to be as close to the asynchronous state as possible. C, For τ_E = 49 msec, the fluctuations of the network activity as measured by R(t) grow in time, and eventually die out. *Bottom panel*, Histogram of the number of connections per neuron, centered at M_syn = 100. C, For τ_E = 50 msec, network fluctuations are damped out, and the persistent state is stabilized. *Bottom panel*, The neural firing rate is a linear function of the number of synaptic inputs and varies in a wide range (20–60 Hz) across the population.

In some of the model simulations, short-term depression was incorporated for the pyramid-to-pyramid recurrent excitatory synapses (Markram and Tsodyks, 1996; Abbott et al., 1997; C. M. Hempel, K. H. Hartman, X.-J. Wang, G. G. Turrigiano, and S. B. Nelson, unpublished observations). Short-term depression is assumed to be caused by transmitter vesicle depletion at the presynaptic terminals (Stevens and Wang, 1995). It is introduced into the synapse model as follows. The parameter α_x, which mimicks the amount of transmitter release per spike, is multiplied by a quantity D (the fraction of available vesicles). D obeys the dynamical equation (Abbott et al., 1997):

\frac{dD}{dt} = - p_{υ} D \sum_{j} δ (t - t_{j}^{-}) + (1 - D) / τ_{D} .

Equation 7

That is, D is reduced by a factor (1 − p_υ) for each spike, ΔD = D(t_j⁺) − D(t_j⁻) = −p_υD(t_j⁻), or D(t_j⁺) = (1 − p_υ)D(t_j⁻). It recovers toward 1 with time constant τ_D in the absence of stimuli. In a simple biophysical model of vesicle depletion in which the release probability is proportional to the number of available vesicles, p_υ is identified with the release probability per vesicle (Wang, 1999). I used τ_D = 500 msec and p_υ = 0–0.35.

Asynchronous States. In this work, persistent activity is assumed to be achieved by a bistability between a rest state and an active state of the network. We shall see that the persistent activity often occurs as an asynchronous network state, in which the discharges of neurons are spread out in time uniformly so that at any time there is a same fraction of neurons firing (Amit and Tsodyks, 1991; Abbott and van Vreeswijk, 1993; Gerstner, 1999). In the presence of the voltage dependence of the NMDAR channels, the nonlinear LIF model cannot be solved explicitly, and the analysis of the asynchronous states is intractable. However, as we shall see, none of our conclusions in this work depends on the voltage sensitivity of the NMDAR-activated conductance. Therefore, the calculations of the asynchronous state were done with [Mg²⁺] = 0.

The firing rates R_E and R_I of pyramidal cells and interneurons in an asynchronous state were calculated as follows. Let us denote the average synaptic drives by s_E and s_I. Each of the two is an average over neural population, and is constant in time for an asynchronous state. It is the same as the time average of each individual s(t) over a period 1/R. For s_E (Eqs. 4,5), an approximation is obtained by substituting ∑_jδ(t − t_j) with R. The steady state is:

x_{E} = α_{x} τ_{x} R_{E}; s_{E} = \frac{α_{s} τ_{s} x_{E}}{α_{s} τ_{s} x_{E} + 1} = \frac{ν R_{E}}{ν R_{E} + 1} .

Equation 8

where ν = α_xα_sτ_xτ_smsec⁻¹. This approximation is accurate when the synaptic current kinetics are sufficiently slow (Ermentrout, 1994), hence reasonable for the NMDAR channels (Fig.1D). On the other hand, it is also correct as long as saturation is negligible, which is the case for the fast AMPAR channels. The average s_Eonly depends on the product ν and is independent of the scaling factor φ. It becomes nonlinear in R_E at R_E ≥ 1/ν and saturates at R_E ≫ 1/ν. For AMPAR channels ν = 0.1 msec⁻¹, 1/ν = 10 kHz; so s_AMPA does not saturate at realistic firing rates, s_AMPA ≃ νR_E. For NMDAR channels ν = 160 msec⁻¹, 1/ν = 6.25 Hz, and s_NMDA is a highly nonlinear function of R_E.

With short-term depression (p_ν ≠ 0), the parameter α_x is multiplied by the steady-state value of D. It is worth noting that the amount of synaptic transmission is given by the value of D immediately preceding a spike (denoted by D_), and not the time average over a period. The steady state value of D_ is (Abbott et al., 1997; Wang, 1999):

D_= \frac{1 - e^{- 1 / R_{E} τ_{D}}}{1 - (1 - p_{υ}) e^{- 1 / R_{E} τ_{D}}} ≃ \frac{1}{p_{υ} τ_{D} R_{E} + 1},

Equation 9

where the approximation is obtained by ∑_jδ(t − t_j⁻) = R_E in Equation 7.

For the GABA_AR-activated synaptic drive, the average was calculated over a periodic firing pattern of rate R_I (compare Eq. 6):

s_{I} = \frac{R_{I} τ_{I} α_{I} (1 - e^{- 1 / R_{I} τ_{I}})}{1 - (1 - α_{I}) e^{- 1 / R_{I} τ_{I}}} .

Equation 10

At realistic firing rates, the steady-state approximation obtained by ∑_j(t − t_j⁻) = R_I in Equation 6, s_I = α_Iτ_IR_I/(α_Iτ_IR_I+ 1), is not accurate for the moderately slow IPSCs.

Given s_E(R_E), the equation for an interneuron is the same as that of a single LIF neuron,

C_{m} \frac{{dV}_{m}}{dt} = - g_{L} (V_{m} - V_{L}) - g_{syn, ei} s_{E} V_{m} + I_{app} = - {\tilde{g}}_{L} (V_{m} - V_{reset}) + \tilde{I},

Equation 11

with g̃_L = g_L + g_syn,eis_E, andĨ = I_app − g_L(V_reset − V_L) − g_syn,eis_EV_reset.

For a constant input current I_app, the firing rate is given by:

R_{I} = \{\begin{matrix} \frac{1}{τ_{ref} - \frac{C_{m}}{{\tilde{g}}_{L}} ln [1 - ({\tilde{g}}_{L} θ) / \tilde{I}]} & if \tilde{I} > {\tilde{g}}_{L} θ, \\ 0 & otherwise \end{matrix}

Equation 12

which is itself a function of s_E(R_E) (i.e. the interneurons are driven by recurrent excitation). Similarly, the voltage equation for a pyramidal cell can be solved for a constant input current. The adaptation current has a steady-state average I_AHP = g_AHP[Ca⁺²]_aυ(V_m− V_K), where [Ca⁺²]_aυ = α_Caτ_CaR_E according to Equation 2 with ∑_j δ(t − t_j) = R_E. The same formula in Equation 12 applies to R_E, except thatg̃_L = g_L + g_AHP[Ca⁺²]_aυ + g_syn,ees_E + g_syn,ies_I, andĨ = I − g_L(V_reset − V_L) − g_AHP[Ca⁺²]_aυ(V_reset− V_K) − g_syn,ees_EV_reset − g_syn,ies_I(V_reset − V_I).

In simulations, noise was added by including a random component I_λ = i_λs_λ in the external current, I_app = I₀+ I_λ; I₀ is a constant current, and I_λ is a stochastic synaptic current of the AMPA type. With a Poisson input train of rate λ, s_λ is incremented by 1 with each input and decays with a time constant τ_λ = 2 msec. At a high rate λ, this Poisson current is approximated by a Gaussian white noise with a mean μ = i_λλτ_λ and a variance ς² = i_λ²τ_λλ. Unless noted otherwise, i_λ = 0.06 nA and λ = 2500 Hz for pyramidal cells; and i_λ = 0.04 nA and λ = 2000 Hz for interneurons. Given a fixed μ = 0.06 × 2.5 × 2 = 0.3 nA, the mean input current to pyramidal cells I = I₀ + μ can be varied by changing I₀, whereas the noise amplitude remains the same.

In the presence of noise, the neural discharges are described by the first-passage times across the firing threshold (Ricciardi, 1977), instead of Equation 12. The expression for the firing rate is:

1 / R = τ_{ref} + \sqrt{π} {\tilde{τ}}_{m} \int_{C_{m} (V_{reset} - V_{ss}) / (\sqrt{{\tilde{τ}}_{m}} ς)}^{C_{m} (V_{th} - V_{ss}) / (\sqrt{{\tilde{τ}}_{m}} ς)} {dxe}^{x^{2}} (1 + erf (x)),

Equation 13

where er f(x) = (2/ $\sqrt{π}$ ) ∫₀^x exp (−x′²)dx′ is the error function, V_ss = I_eff/g̃_L andτ̃_m = C_m/g̃_L (g̃_Las given above). The effective current I_eff = I₀ + μ + g_LV_L for interneurons, and I_eff = I₀ + μ + g_LV_L + g_AHP[Ca⁺²]_aυV_K+ g_syn,ies_IV_I for pyramidal cells.

The neural firing rates of the asynchronous state were approximately computed in two steps. First, Equation 13 applied to R_I is a function of R_E, R_I = g(R_E). Then, Equation 13for R_E becomes self-consistent,

R_{E} = f (R_{E}, R_{I}) = f (R_{E}, g (R_{E})); R_{I} = g (R_{E}),

Equation 14

which is solved to yield R_E. Note that f is the input–output function of a LIF neuron, another way of writing Equation 14 is:

R_{E} = f (I_{tot}), where I_{tot} = I_{app} - I_{AHP} - I_{syn, ee} - I_{syn, ie},

Equation 15

where I_tot depends on I_app, R_E and R_I.

In numerical simulations, the initial condition can be prescribed to be near the asynchronous state, by assuming that the neural output patterns are periodic with the phases uniformly distributed in a time period [0, T = 1/R] (Abbott and van Vreeswijk, 1993). Note that in the presence of noise, the time course of the neural membrane potential is not exactly periodic. However, this initial condition should be close to the actual asynchronous state. If the latter is a stable attractor, with this initial condition the network should quickly converge to it.

Numerical integrations. The model was numerically integrated using a second order Runge–Kutta method, with an interpolation procedure to determine the spike times (Hansel et al., 1998). The time step dt = 0.02 − 0.05 msec. Typically I used N_e = 1000 and N_i = 200, some conclusions were checked with N_e= 5000 and N_i = 1000 (N_i/N_e = 20%).

In simulations, the network activity was measured by the instantaneous firing rate R_E(t) of the pyramidal cell population as follows. The time was divided into small bins (Δt = 1–10 msec were used). Then,

R_{E} (t) = \frac{total number of spikes from all pyramidal cells in (t, t + Δ t)}{N_{e} Δ t} .

For example, in an asynchronous state R_Ewould be constant in time and equals the firing rate of each individual cell. A coherent network oscillation would be reflected by a rhythmic time course of R_E(t).

RESULTS

NMDA receptor channels and persistent activity at low rates

Persistent activity is produced by an excitatory neural network, when the recurrent synapses are sufficiently strong. In Figure2A, the network is initially in a rest state. In response to a transient input pulse, neurons start to discharge spikes that activate recurrent synapses, which in turn elicit more spikes. This positive feedback loop between the spike firing and the recurrent synaptic drive leads to a self-sustained network activity, outlasting the input. In the persistent state, neurons fire spikes asynchronously in time: at any given moment there is always a fixed fraction of cells firing. Therefore, the synaptic drive to each cell is tonic (constant in time). Moreover, the average firing rate of neurons is ∼40 Hz, within the physiological range of the persistent activity of PFC cells during the delay period (Funahashi et al., 1989; Rainer et al., 1998). The network is turned off by a brief hyperpolarizing input, from the persistent state back to the rest state. In this simulation, the leak conductance g_L differs from cell to cell according to a Gaussian distribution. Cells with the smallest g_L values are the most excitable and display spontaneous firing in the rest state; whereas cells with the largest g_L values are the least excitable and only show transient responses to the input pulse but no persistent activity (Fig.2A).

Fig. 2. — Persistent active state in an excitatory neural network. A, Panels from *top* to *bottom*, membrane potentials of three cells, external input current, rastergram, and population firing rate. The network model is initially at rest. In response to a transient current pulse, the network is activated. After the termination of the input, neurons continue to discharge spikes asynchronously with an average firing rate of 40 Hz [R(t) is constant in time; see also the rastergram]. In this simulation, there is a Gaussian distribution of the leak conductance g_L across the cell population, with a mean of 0.025 μS and SD of 0.003 μS. Cells with the least g_L display spontaneous firing in the rest state (*Cells 1, 2*), whereas cells with the largest g_L do not show sustained firing in the network persistent state (*Cell 3*) (g_AMPA = 0.2; g_NMDA = 0.04; I = 0.3 nA).B, Bistability is a network phenomenon. During persistent activity, a neuron is hyperpolarized by a current pulse (with two different intensities) to a negative membrane potential, but at the end of the perturbation the firing activity resumes itself because of the massive synaptic drive from the network.

The bistability between the rest state and active state is a network phenomenon. As illustrated in Figure 2B, during persistent activity, a neuron can be temporally hyperpolarized by an applied current pulse, but its activity resumes itself immediately after the perturbation, because the firing of any single neuron is sustained by synaptic inputs from the circuit. Such a manipulation would be feasible experimentally only with intracellular recording from a behaving animal during a working memory task. The prediction is that if bistability is not a single cell property but is instead induced by the network circuit, a hyperpolarizing current pulse should be incapable of switching a neuron off from its persistent activity.

In model simulations, the NMDAR-mediated synaptic transmission was necessary to generate network persistent activity, at low firing rates such as in Figure 2. For the purpose of illustration, consider first the simplified situation of a perfectly synchronous network state in which all neurons behave exactly the same in time. Therefore, the population of identical excitatory neurons can be reduced to a single neuron endowed with an autapse (Fig.3A). Suppose that the synaptic transmission is of the NMDA type (decay time τ_E = 80 msec). The cell is switched onto a firing state by a transient input. At the end of input pulse, the NMDAR-mediated current decays slowly, and after the time span of an interspike interval (ISI), it remains large enough to trigger another spike, which in turn generates more EPSC. This process between the spiking and synaptic activation can continue indefinitely, provided that the decay of the NMDAR-mediated current is not too fast compared to a typical ISI, i.e. the τ_E/ISI ratio is sufficiently large. Otherwise, if the synaptic current generated by one spike decays back to zero before the next spike is triggered, the cell will return to the rest state instead. This is shown in Figure3B, where the synapse is now assumed to be of the AMPA type (τ_E = 2 msec). The peak AMPAR-mediated EPSC here is ∼10× that of the NMDAR-mediated EPSC in Figure3A, but it decays rapidly between spikes during the input pulse and does not give rise to persistent activity. Using a considerably stronger synaptic conductance, the AMPAR-mediated current can be large enough to generate a persistent activity, but at a very high firing rate, so that the τ_E/ISI ratio is again large (see below).

Fig. 3. — Tonic synaptic drive is required to sustain a persistent active state. A, A single neuron with an autapse of the NMDA type is excited from the rest to an active state that outlasts the transient input. The persistent firing is at 36 Hz. Note the tonic NMDAR-mediated current (g_NMDA = 0.1). B, If the synaptic current is mediated by the AMPARs (g_AMPA = 1.5), the synaptic current fluctuates rapidly between a maximum and zero. When it is zero, the cell does not receive synaptic drive any more; therefore the cell decays back to the rest state as soon as the input is withdrawn. Note the different scale for the synaptic current in A andB.

The above argument applies to the network, if the neural firing patterns are partially synchronous. For example, this can happen because of the interplay between rapid synaptic excitation and slower inhibition in a two-population network of pyramidal cells and interneurons (Fig. 4). Powerful AMPAR-activated synapses between pyramidal cells amplify the network activity, which is damped afterwards by recurrent inhibition, leading to synchronous network oscillations at ∼8 Hz (the oscillation frequency ranges from 8 to 65 Hz, when the pyramid-to-interneuron coupling strength is varied gradually). Note that the AMPAR-activated synaptic drive s_AMPA fluctuates between zero and a peak level during the oscillation. Without NMDAR channels, clearly this synchronous network state would not be self-sustained, because when s_AMPA is almost zero the network would have to collapse back to the rest state. The slow NMDAR-mediated current does not decay back to zero during the waning phases of the network oscillation. As a result, the tonic component of the NMDAR-activated synaptic drive s_NMDA can sustain a synchronous persistent state. Here, the requirement is that the oscillation period T must not be too long compared to the NMDAR channel decay time constant (τ_E/T must be large).

Fig. 4. — Slow NMDAR channels can sustain a persistent active state in which the network dynamics is partially synchronous. The network model consists of two (pyramid and interneuron) populations. The network is initially at rest and is switched to the active state by a transient input. Synchronous oscillations at 8 Hz are generated by the interplay between the fast recurrent AMPAR-activated excitation and slower feedback inhibition. Note that the pyramidal cell and interneuron populations show very small relative phase shift (*inset*). The AMPAR-activated synaptic drive s_AMPA phasically oscillates between zero and a maximum, whereas the NMDAR-activated synaptic drive s_NMDA remains at a significantly high level, which is sufficient to maintain the network activity (g_AMPA,ee = 0.7; g_NMDA,ee = 0.07; g_AMPA,ei = 0.2; g_NMDA,ei = 0.02; g_GABA = 0.1; I = 0.3 nA).

Therefore, by virtue of its temporal summation, NMDAR channels (but not AMPAR channels) can provide sufficient tonic drive to maintain a synchronous persistent state at low rates. On the other hand, if the persistent state is asynchronous, a tonic synaptic drive can be realized by a spatial summation over neurons. In the latter case, because the synaptic drive is constant in time regardless of the τ_E/ISI ratio, it would seem that the fast AMPAR channels alone might be sufficient to maintain a persistent network state at any firing rate. As we will see below, this is not the case because of the problem of rate control with the AMPAR channels.

Frequency–current relation of a bistable network

Persistent activity in our network model is realized as a bistability between a rest state and an active state, where the network can be switched on from the rest state by a transient stimulus and remains in the persistently active state afterwards. Consider for example the case where synaptic connections are mediated by the fast AMPARs. For a fixed synaptic coupling (g_AMPA = 1.05) and a given external drive (I = 0.3 nA), the neuronal firing rate of an asynchronous network is given by the nonlinear equation R = f(I_tot(R)) (Eq. 15in Materials and Methods). The function f is the neuronal input–output relation, and the total input I_totis a function of R caused by the recurrent synaptic interactions. When the left and right hand sides of the equation are plotted on a same graph, the solutions for R correspond to the intersection points of the two curves. As shown in Figure5A (top panel, solid curve), there are three states of different firing rates: a rest state (in which synapses are not activated), an active (persistent) state, and a middle state (which is always unstable, thus not observable in network simulations). The instability of a steady state can be intuitively understood as follows. When f(I_tot) < R, the total current acts to decrease firing, whereas when f(I_tot) > R the total current acts to increase firing. Therefore, if the rate R happens to be slightly higher than the middle steady state, f(I_tot) > R and R will increase further; whereas if R is lower than the middle state, f(I_tot) < R and R will decrease further. In either case the system will drift away, and the middle steady state is not stable against small perturbations.

The bistability occurs within a certain range of the I values (Fig. 5A, top panel). If the external drive is too small (I = 0.1), the combined external and recurrent drive is not sufficient to maintain a persistent state. On the other hand, if it is too large (I = 0.5), the rest state no longer exists. By plotting the steady states as function of I, an S-shaped frequency–current curve is obtained for a bistable asynchronous network (Fig. 5A, bottom panel). Let us denote by I_a and I_b the two I values delimiting the bistable range. I_a is the smallest I value for an active state, and I_b is the largest I value for the rest state. I_b ≃ 0.4 nA is close to the threshold current for an isolated neuron, because recurrent synapses are not activated in the rest state. The firing rate of the active state increases with I; the lowest possible rate corresponds to I_a, at the left-knee of the curve. In our example neuronal firing rates of persistent activity are above 110 Hz, much higher than those observed in the PFC neurons (10–50 Hz).

Can the firing rate of persistent activity be reduced by weaker recurrent synaptic connections? In Figure 5B are shown the frequency–current curves of the network at various coupling strengths (g_AMPA). We see that bistability becomes possible only with sufficiently strong g_AMPA. With larger g_AMPA, persistent state can be realized at smaller I (I_a shifts to the left), so the bistability range (I_b − I_a) is wider (the persistent state is more robust). On the other hand, the lowest firing rate of a persistent state (at I_a) dramatically increases with g_AMPA (Fig. 5B, filled square). Therefore, there is a tradeoff between the lowest firing rate possible and the robustness of the phenomenon: if we require that the bistable range be reasonably large (at least 0.1–0.3 nA, for example), the firing rate of a persistent state is always 100–200 Hz or higher. Furthermore, the stability of the active state is not guaranteed. Indeed, the persistent state close to I_a is usually not observed in direct simulations of the network model, presumably because it is not stable in the presence of noise. The stability issue will be discussed in more detail below, when negative feedback processes are included.

In contrast to the case with AMPAR-activated synaptic transmission, with only NMDAR-activated synaptic transmission, robust persistent states at low firing rates are possible (Fig. 5C,D). The bistable range increases nearly linearly with the NMDAR-activated conductance g_NMDA, whereas the lowest firing rate of the persistent state remains <40 Hz (Fig. 5D, filled square). The dramatically different input–output relations obtained with the AMPA- or NMDA-type synapses can be explained in terms of their respective gating kinetics. As shown in Figure 5C(top panel), for a given synaptic coupling (g_NMDA = 0.006) and external drive (I = 0.3 nA), the input–output relation f(R) saturates at low firing rates with NMDAR channels, in contrast to the case with AMPAR channels (Fig. 5A, top panel). This is because the dependence of f on R is via the synaptic drive s_E(R) (Eq. 8). The fast-decaying AMPAR channels do not accumulate over time, hence do not saturate except at very high firing rates (∼500 Hz). By contrast, the slowly decaying NMDAR-mediated current saturates at firing rates within the physiological range (Fig.1D). At >50 Hz or so s_NMDAbecomes independent of the input rate, so it can no longer be increased further to sustain higher firing rates. (The actual firing rate, which also depends on g_NMDA and the input I, can of course be >50 Hz.) For this reason, NMDAR (not AMPAR) channels are well suited to realize persistent states at low firing rates in a robust manner.

Negative feedback mechanisms for rate control

Can some negative feedback mechanisms be used to resolve the problem of rate control with the AMPAR channels alone? This question is addressed next, by considering consecutively spike-frequency adaptation, recurrent shunting inhibition, and short-term synaptic depression.

Spike-frequency adaptation

Spike-frequency adaptation, a common property of (“regular spiking”) cortical pyramidal neurons (McCormick et al., 1985; Mason and Larkman, 1990; Wang, 1998), is added to the model neuron by including an I_AHP. To assess the effects of I_AHP on a persistent state sustained by the AMPAR channels, the frequency–current curve is calculated for different g_AHP values (Fig.6A). For a fixed I the firing rate of the active state is reduced by g_AHP, (Fig. 6A, vertical dotted line). At the same time, however, the bistable range shrinks dramatically and eventually disappears with large g_AHP values (for g_AHP ≥ 0.005).

Fig. 6. — Effect of spike-frequency adaptation in an excitatory network (g_AMPA = 1.2).A, Frequency–current curves with different g_AHP values. For a given input current (e.g., I = 0.35 nA; *vertical dotted line*), the firing rate is decreased by increasing g_AHP. At the same time, the bistable range shrinks, and the bistability disappears when g_AHP is >0.005. *Dotted line*, g_AMPA = 0.99 and g_AHP = 0, which is superimposable with that of g_AMPA = 1.2 and g_AHP = 0.0025. The persistent state at reduced firing rate (e.g. *open circle* at I = 0.35 and g_AHP = 0.004) is unstable if the excitatory synapses are mediated by the fast AMPARs (see ).B, Adaptation induced network rhythmic bursting. When the asynchronous state is unstable and does not coexist with the rest state, the network displays synchronous burst firing patterns (with I = 0.45 and g_AHP = 0.01, indicated by a *cross* in A). Strong and fast recurrent excitation recruits neurons and accelerates neural discharges, until I_AHP grows sufficiently to terminate the burst. I_AHP then decays back to zero, and the cycle starts over again. Note that the neural firing is coherent at the onset of the burst, but desynchronizes within the burst (*inset*).

This effect of I_AHP is readily explained in term of a negative current that counterbalances the excitatory synaptic current. Suppose that the firing rate is given by the input–output relation R = f(I_tot) (Eq. 15), where I_tot = I_app − I_syn − I_AHP. The average membrane potential of a firing neuron is approximately half-way between V_reset and V_th, V_aυ ≃ (V_reset + V_th)/2 = −55.5 mV. Then, one has I_syn ≃ g_AMPAs_EV_aυ ≃ g_AMPAυV_aυR = −g̃_AMPAR, with s_E ≃ υR and g̃_AMPA = −g_AMPAυV_aυ. On the other hand, I_AHP ≈ g_AHP[Ca²⁺]_aυ (V_aυ − V_K) =g̃_AHPR, with [Ca²⁺]_aυ = α_Caτ_CaR andg̃_AHP = g_AHPα_Caτ_Ca(V_aυ− V_K). Taken together, we have

I_{tot} = I_{app} + ({\tilde{g}}_{AMPA} - {\tilde{g}}_{AHP}) R .

Equation 16

Therefore, the addition of I_AHP amounts to a subtractive reduction of the effective recurrent synaptic excitation. For example, if g_AMPA = 1.2 and g_AHP = 0.0025, g̃_AMPA = −g_AMPAνV_aυ = 6.66 andg̃_AHP = g_AHPα_Caτ_Ca(V_aυ− V_K) = 1.18. Thus,g̃_AMPA − g̃_AHP = 6.66 − 1.18 = 5.48. This is equivalent to a reduced g_AMPA value (g̃_AMPA− g̃_AHP)/(−νV_aυ) = 0.99 in the absence of I_AHP. Indeed, the frequency–current curve with g_AMPA = 1.2 and g_AHP = 0.0025 and that with g_AMPA = 0.99 and g_AHP = 0 are essentially superimposable (Fig. 6A).

Note that the specific form of this subtraction depends on the model details. For example, if I_AHP has the following functional form I_AHP = g_AHP[Ca²⁺]ⁿ/([Ca²⁺]ⁿ+ D_Kⁿ)(V_m − V_K), n > 1, then the subtractive term in Equation 16 will be nonlinear.

It is important to emphasize that the stability of an active state is not guaranteed. In the it is shown that the stability of an asynchronous state depends critically on the synaptic time constant. In fact, with the fast AMPAR-mediated synapses, any active state in the presence of an I_AHP is expected to be unstable if its firing rate is below the lowest possible firing rate of an active state with I_AHP = 0. This is true regardless whether the active state belongs to a bistable range or not. For example, at I = 0.45 nA and g_AHP = 0.01 there is a single state with R = 30 Hz (Fig. 6A, cross). As shown in Figure 6B, this asynchronous state is not stable. Instead, neurons fire synchronously repetitive bursts of spikes that alternate with quiescent phases in time, the network oscillation has a frequency of 3 Hz. Such rhythmic bursting has also been reported in other studies that are not related to persistent activity (van Vreeswijk and Hansel, 1997; G. B. Ermentrout, personal communication). Synchronous burst oscillation is a common phenomenon in neurons and networks, usually when a strong and rapid autocatalytic process is combined with a slower negative feedback (here, the recurrent AMPAR-activated synaptic excitation and the I_AHP). Clearly, because the fast AMPAR-activated synaptic drive goes to zero between the bursts, the network would have to collapse onto the rest state, if the latter existed. In other words, when the active state in a bistable range is unstable (Fig. 6A, open circle), it is not observable, and the only stable behavior is the rest state. From these results it is concluded that I_AHP cannot subserve as a rate control mechanism unless additional slow synaptic transmission is present, such as that mediated by the NMDARs.

Recurrent shunting inhibition

Synaptic shunting inhibition has been suggested as a rate control mechanism in the neocortex (Douglas et al., 1995). When a neuron is at rest, shunting inhibition does not produce a net hyperpolarizing current because its reversal potential V_I is close to the resting potential. Instead, it causes an increase in membrane conductance, which divides the excitatory synaptic current (Carandini and Heeger, 1994). However, as it was recently pointed out by Holt and Koch (1997), the situation is different when the cell is in a repetitively firing state. In that case, the spiking mechanism essentially clamps the average membrane potential roughly half way between V_reset and V_th, well above V_I(for example, V_I = −70 mV, whereas V_aυ = (V_reset + V_th)/2 = −55.5 mV), and the effect of inhibitory synapses is hyperpolarizing. For example, suppose that the model network is in a persistently active state, and each neuron receives a feedforward synaptic inhibition with a given input rate R_I. Then, this input is equivalent to a negative current I_GABA = g_GABAs_I(V_aυ − V_I), where the synaptic drive s_I as a function of R_I is given by Equation 10. Therefore, the addition of feedforward inhibition simply shifts a frequency–current curve to the right by the fixed amount I_GABA, without changing the range of network bistability or the lowest firing rate of a persistent state. This conclusion was confirmed by simulations (data not shown).

In the case of feedback synaptic inhibition, the firing of inhibitory interneurons is driven by pyramidal cells, and R_I is a function of R_E, R_I = g(R_E) (Fig.7C). In this case I_b remains the same, because g_GABA has no effect on the rest state. On the other hand, a larger I is needed to counterbalance I_GABA for the persistent activity (I_GABA shifts I_a to the right). Therefore the range of network bistability (I_b − I_a) is reduced. Note that, with increasing g_GABA, although the firing rate at a given I is reduced, the lowest possible rate of a persistent state (Fig. 7A, filled square) remains almost the same. Therefore, recurrent inhibition acts in a subtractive manner, in the sense that is produces a negative current that counterbalances the recurrent excitatory synaptic current. In terms of the firing rate equation R_E = f(I_tot), we have:

I_{tot} = I_{app} - I_{syn, ee} - I_{syn, ie} = I_{app} + {\tilde{g}}_{AMPA} R_{E} - {\tilde{g}}_{GABA} s_{I} (R_{I}),

Equation 17

with g̃_GABA = g_GABA(V_aυ − V_I) and R_I = g(R_E) (Eq. 14). The subtractive term is nonlinear in R_E.

Fig. 7. — Effect of feedback shunting inhibition.A, B, Frequency–current curves with different g_GABA values when isolated interneurons are near or well below the firing threshold, respectively (C).A, Stronger g_GABA reduces the bistable range and abolishes the persistent state. Note that the lowest firing rate of persistent activity (*filled square*) is hardly changed by inhibition. B, In this case, the portion of the frequency–current curve with R_E < 25 Hz is unaffected by recurrent inhibition. With sufficiently large g_GABA, bistability is preserved, and the active states have reasonably low firing rates (25–50 Hz).C, The firing rate R_I of interneurons as function of R_E for A andB (g_AMPA,ee = 1.2, g_AMPA,ei = 0.4; the Poisson input rate to interneurons is λ = 2500 Hz in A and 2000 Hz inB).

If inhibitory neurons are not near the firing threshold, they will fire spikes only when their excitatory drive is sufficiently strong, e.g. R_I = 0 unless R_E is above a critical value ∼25 Hz (Fig. 7C). As a result, the portion of the frequency–current curve of the pyramidal cell with R_E < 25 Hz (on the middle branch) cannot be altered by feedback inhibition. With sufficiently strong g_GABA, network bistability is always preserved, and the lowest firing rate of a persistent state remains 25 Hz (Fig. 7B). In this way, persistent activity with reasonably low firing rates becomes possible.

I also considered an additional effect that may be caused by shunting inhibition. Suppose that shunting inhibition produces an increase in membrane conductance along a dendritic cable of length L, between the excitatory synapses and the spike triggering zone. The effective characteristic cable length λ is then expected to decrease like λ ∼ (g_L + g_GABAs_I)^−1/2. To take into account the exponential attenuation of excitatory synaptic inputs along a passive cable, the excitatory conductance g_E should be multiplied by a factor ∼ exp(− L/λ) ∼ exp(− β(g_GABAs_I)^1/2), where β is given in terms of the cable properties (Abbott, 1991). This highly nonlinear effect was suggested to provide a solution to the high firing rate problem in neural networks (Abbott, 1991). When this effect is included in the model, persistent states with low firing rates can be obtained, the frequency–current curve of the pyramidal cell is similar to Figure 7B (data not shown).

In any case, when a persistent state with low firing rate is realized with synaptic inhibition, its stability still remains to be determined. In fact, such a state was never observed in the network simulations, if the recurrent excitation was mediated exclusively by the fast AMPARs. Again, intuitively, such an active state is expected to be unstable because of the interplay between a fast recurrent excitation and a slower negative feedback. This is shown mathematically in the . To illustrate this point by computer simulations, I used the scaling parameter φ for the synaptic kinetics (Eqs. 4, 5) to change systematically the EPSC gating rates, whereas the average synaptic drive s_E and the firing rate R_E remained the same. φ was varied so that τ_E = τ_s/φ was between 2 and 80 msec. Let us choose g_GABA = 0.03 and I = 0.34 nA, the persistent state has a firing rate of 33 Hz (Fig. 7B). As shown in Figure8, when the excitatory synapses are slow (τ_E = 80 msec; comparable to that of the NMDAR channels), a persistent state can be sustained in the network (Fig. 8A). Because of the slow synaptic build-up, the network firing activity gradually ramps up during the input pulse. Moreover, in contrast to partially synchronous activity of Figure 4, with slow synaptic excitation (in the absence of a fast component) the persistent state is asynchronous. When τ_E is sufficiently reduced, the network activity in the persistent state displays increasingly large temporal fluctuations (Fig.8B). If τ_E is decreased below a critical value (τ_E ≃ 18 msec), the persistent state becomes unstable, because synchronous fluctuations eventually bring the network too close to the rest state, and the activity terminates (Fig. 8C).

Fig. 8. — The low rate asynchronous state is not stable if excitatory synapses are too fast. The network model is simulated in the presence of strong recurrent inhibition. The speed of the excitatory synaptic kinetics is varied, whereas the steady–state synaptic drive and the mean firing rate are preserved. A, With τ_E = 80 msec, the network can be turned on to the persistent state with R_E ≃ 33 Hz. Note the slow ramping-up of R_E(t) during the transient stimulus, caused by the temporal summation of the slow synaptic current. B, With τ_E = 18 msec, the persistent state is still stable, but R_E(t) displays large fluctuations in time.C, With τ_E = 17 msec, the fluctuations eventually bring R_E(t) too close to zero, and the network returns to the rest state (same parameters as in Fig. 7B, with g_GABA = 0.03 and I = 0.34 nA).

To conclude, the effect of GABA_Asynaptic inhibition is largely subtractive rather than divisive in repetitively firing neurons. Therefore, the phenomenon of persistent activity becomes less robust and can be abolished completely by strong recurrent inhibition. Moreover, when a persistent state with low rate does exist, it cannot be stably maintained unless the excitatory synapses are sufficiently slow (the ratio τ_E/τ_I must not be too small).

Short-term synaptic depression

I now turn to short-term depression of the excitatory synapses as a rate control mechanism. A typical simulation result is shown in Figure 9. In the absence of short-term depression (the parameter p_υ = 0; see Materials and Methods), a persistent activity state has a firing rate close to 200 Hz (Fig. 9A). The addition of short-term depression (p_υ = 0.35) reduces the firing rate to ∼40 Hz, back to the physiological range of PFC neurons (Fig.9B). Note that, because of short-term depression, the neuronal firing shows an exponential decrease during the depolarizing input pulse (Fig. 9B, top and middle panels); and immediately after the pulse there is a trough in the neural activity during which time the synapses recover from depression (Fig.9B, bottom panel). In this simulation both fast AMPAR and slow NMDAR channels are included, and the dynamics is asynchronous in the persistent state.

Fig. 9. — Rate control by short-term synaptic depression (*STD*). A, Without STD the firing rate of the persistent state is typically high, as long as there is a substantial AMPAR-mediated component of the recurrent synaptic transmission.B, The addition of STD (p_υ = 0.3) significantly reduced the firing rate to ∼40 Hz, within the physiological range of PFC cells. Note that during the transient depolarizing pulse R(t) has a rapid peak, then decreases to a low steady state caused by STD (see D(t)). There is a trough in R(t) immediately after the input pulse, when D(t) recovers and reaches a steady state (g_AMPA = 0.7; g_NMDA = 0.07; I = 0.3 nA).

The frequency–current curve is calculated for different degrees of short-term depression (Fig.10A). In this case the undepressed AMPAR-mediated currents are so strong that with p_υ = 0 the firing rates of the persistent states are ∼500 Hz, near the neuronal saturation (data not shown). As we see in Figure 10A, short-term depression dramatically decreases the lowest firing rate of the active states (Fig. 10A, filled square). The range of bistability also shrinks (I_a shifts to the right) with increasingly strong short-term depression; but for some p_υ values this range remains reasonably large while the physiological firing rates are achieved. Short-term depression gives rise to synaptic saturation, which occurs at lower firing rates with larger p_υ (Fig.10B). Indeed, for AMPAR channels s_E ≃ νR_E. With short-term depression s_E = νR_E/(1 + p_υτ_DR_E) (Eq. 9). In terms of the firing rate equation R = f(I_tot), we have:

I_{tot} = I_{app} - I_{syn} = I_{app} + \frac{{\tilde{g}}_{AMPA} R_{E}}{1 + p_{υ} τ_{D} R_{E}} .

Equation 18

Therefore, the effect of short-term depression divides the amplitude of the excitatory synaptic drive. Unlike a subtractive mechanism (spike-frequency adaptation or recurrent inhibition), which is equally strong at all rates, a divisive mechanism affects high rates disproportionally. This leads to the flattening of the f(I_tot(R)) curve (Fig.10B). At high frequencies [R_E ≫ 1/(p_υτ_D)], the synaptic current becomes independent of the firing rate (Abbott et al., 1997). As a result, the positive feedback between firing and synaptic excitation has to stop at some firing rate, well below the neuronal saturation level (∼500 Hz).

Fig. 10. — Effect of short-term synaptic depression in an excitatory neural network. A, Frequency–current curves with p_υ = 0.15 to 0.35, by increment of 0.05. Short-term depression reduces the lowest firing rates of the active states (*filled square*), whereas the bistable range remains reasonably large. B, For a fixed input current (I = 0.3 nA) in A, the firing rate of the asynchronous state is given by R = f(R); or the intersections of f(R) with the diagonal line. Stronger short-term depression leads to saturation of the function f(R) at progressively lower firing rates, so that rate control is achieved for the persistent state. (g_AMPA = 8).

The dynamical stability of these asynchronous persistent states with short-term depression was checked in direct network simulations. Simulations were performed with both all-to-all and sparse couplings. In a sparse network, unlike an all-to-all network, the number of synaptic connections varies widely from cell to cell, with an average M_syn. One might expect that such heterogeneity would favor an asynchronous persistent state against instability and synchrony. In fact, I found that as long as M_synis not too small (≥100), the network behaves similarly with sparse or all-to-all coupling. This is true independent of the network size N_e. In other words, what matters is the absolute number of connections per neuron M_syn, not the connection probability p = M_syn/N_e. Similar to the case of spike-frequency adaptation or recurrent inhibition, it was found that fast AMPAR channels could not sustain such a low rate state, and that slower synapses were required (See for stability analysis). To be quantitative, for a given persistent state I varied the synaptic time constants systematically in network simulations by changing the scaling parameter φ (Eqs. 4, 5). This way, the smallest value of τ_E = τ_s/φ that was needed for the persistent state to be observable was determined. For example, consider the persistent states at I = 0.3 nA of Figure 10A, which have the firing rate ranged from 100 to 35 Hz as p_υ is varied from 0.15 to 0.35. The minimal τ_E required for the stability of each of these states is plotted as function of the firing rate R in Figure11A. The critical τ_E is larger with lower R, it also depends on the time constant of the depression process τ_D (see ).

Figure 11A was obtained with M_syn = 100 for a sparse network. The dependence on M_syn is shown in Figure11B, for R = 35 Hz. One observes that the minimal τ_E is not sensitive to M_syn, as long as M_syn ≥ 100. At very small M_syn, there is an abrupt increase of the required minimal τ_E, i.e. even slower synapses are needed to stabilize the active state. This is because, if a neuron receives a very small number of synaptic inputs, each at a low rate, the synaptic current must be long-lasting in order to produce a sustained tonic drive to the postsynaptic cell. Figure 11, Cand D, illustrates the network dynamical behavior for τ_E around the critical minimum for R = 35 Hz (M_syn = 100). In these simulations, the network was initially set to be very close to the asynchronous state (see Materials and Methods for the asynchronous initial condition). Below the critical value (Fig. 11C; τ_E = 49 msec), the synchronous state is unstable. The network activity fluctuates in time, and R oscillates with growing amplitude. When R gets close to zero, the synaptic excitation becomes too weak to bring the network back up again, and the network activity dies out (Fig. 11C). On the other hand, above the critical value (Fig.11D; τ_E = 50 msec), fluctuations of the network activity decay with time, and the asynchronous persistent state is stable. In this random and sparse network, the firing rate of a neuron is a linear function of its number of synaptic connections (Fig. 11D, bottom panel), and is widely distributed across the neural population (20–60 Hz).

To conclude, unlike spike-frequency adaptation or synaptic inhibition, short-term depression acts as a divisive mechanism for rate control. The resulting persistent states at low firing rates are not stable, unless τ_E is larger than a critical value, which depends on both the short-term depression time constant and the firing rate. For the firing rates in the physiological range of PFC cells, the required synaptic kinetics is much slower than that of the AMPAR channels.

DISCUSSION

The general finding of this work is that memory processes performed in strongly recurrent cortical circuits, such as delay-period activity, depend on the temporal dynamics as much as on the efficacy of recurrent synapses. Three main conclusions are: (1) the asynchronous dynamics is generally not stable in a fast recurrent excitation/slow negative feedback system; (2) slow NMDAR-activated synapses are powerful for maintaining a stable persistent activity at low firing rates; (3) short-term depression of excitatory synapses provides an efficient mechanism for rate control.

NMDA receptors and persistent activity

NMDAR channels were found to be crucial to persistent activity in the network model for two reasons. First, their slow gating kinetics naturally leads to synaptic saturation at low firing rates, as observed experimentally (Fig. 1A), thereby contributing to the rate control of network activity. This saturation of the steady-state response to repetitive stimulation should be distinguished from receptor saturation by a single vesicle of transmitter; the latter is not supported by recent data (Mainen et al., 1999) and is not assumed in the present model. Second, slow synapses usually suppress network instability and oscillations, but are also able to sustain a partially synchronized network dynamics realized by other (fast) mechanisms. The voltage dependence of gating kinetics represents another interesting feature of I_NMDAto be explored in the context of working memory processes (Lisman et al., 1998).

Is there experimental evidence for a critical role of NMDARs in delay-period activity of the prefrontal cortex? Scherzer et al. (1998)reported a much higher expression of the NMDAR subunit mRNAs in the prefrontal cortex than in other cortical areas (such as primary visual cortex) of the human brain; which raises the interesting question of whether this regional difference could be correlated with the conspicuous occurrence of persistent activity in the association cortices in contrast to sensory cortices. NMDARs have been demonstrated to contribute to synaptic transmission at intracortical connections of sensory cortices (Thomson et al., 1985; Larson-Prior et al., 1991;Armstrong-James et al., 1993; Thomson and Deuchars, 1995; Markram et al., 1997) and frontal cortex (Sutor and Hablitz, 1989; Hirsch and Crépel, 1990; Kang, 1995). In certain cortical area, this contribution may overwhelm that of AMPARs and dominate recurrent horizontal excitations (Fleidervish et al., 1998). More quantitative analysis of the NMDAR- and AMPAR-mediated synaptic currents in the PFC has been lacking, for both the monkey and the rodent. On the other hand, in behavioral experiments with rats performing a spatial delayed alternation task, systematical administration (Verma and Moghaddam, 1996) or microinjection into the prefrontal cortex (Romanides et al., 1999) of NMDAR antagonists impaired working memory. These observations are consistent with our hypothesized importance of NMDARs to working memory. A direct experimental test, however, will need to be done on behaving animals, by combining pharmacological manipulation of NMDARs with neuronal recordings from the prefrontal cortex.

Note that in a model of persistent activity in the gaze control system,Seung (1996) also suggested that slow synaptic transmission is of crucial importance, but for quite different reasons. That network model is only weakly nonlinear, and slow synapses are useful to prolong the lifetime of transient memory storage.

Rate control and robustness of network bistability

I have tested three candidate rate control mechanisms: spike-frequency adaptation, feedback inhibition, and synaptic short-term depression. I argue that a rate control mechanism should be assessed based on its effect on the entire frequency–current curve of the network. A rate control mechanism is judged effective if it reduces the lowest firing rate of persistent activity down to a physiologically plausible range; and at the same time the network bistability should remain robust within a reasonable parameter range. By these criteria, it was found that both spike-frequency adaptation and feedback inhibition are not adequate. Both act in a subtractive way, in the sense that each produces a negative current that counterbalances the recurrent excitatory synaptic current (Eqs. 16,17), and they readily abolish the persistent activity phenomenon. A note of caution is warranted there, because this study used the simple LIF neuron model that does not take into account more complex features of cortical neurons, such as dendritic morphology or other ionic currents that may contribute to single neuron dynamics. In particular, it would be worth re-examining the issue of feedback inhibition in a more realistic situation where, for example, shunting inhibition is located near the soma of a neuron, spatially separated from the excitatory inputs at dendritic sites. Moreover, our conclusion on shunting inhibition follows from the required preservation of network bistability, hence it does not deny the importance of recurrent inhibition as a rate control mechanism in situations without persistent activity, such as sensory processes in the primary visual cortex (Douglas et al., 1995;Borg-Graham et al., 1998). Finally, synaptic inhibition is likely indispensible for the formation of memory fields of the PFC neurons (Goldman-Rakic, 1995; Camperi and Wang, 1998; Rao et al., 1999).

In contrast to spike-frequency adaptation or synaptic inhibition, short-term synaptic depression acts as a divisive mechanism, in the sense that it divides the recurrent synaptic conductance (Eq. 18). Short-term depression reduces the firing rate not by preventing the neuronal saturation, but by saturating the synaptic drive at low firing rates (Fig. 10B). In the divisive but not subtractive case, firing rate of persistent activity is reduced effectively, whereas bistability is preserved in a robust way. Recent in vitro experiments have indicated that short-term depression is a general property of the rat PFC synapses (Hempel, Hartman, Wang, Turrigiano, and Nelson, unpublished observations). It would be interesting to see whether there is evidence for short-term depression in firing patterns of PFC cells of the behaving animal. Similar to our model simulation (Fig. 9B), in a delayed-response task, PFC neurons often display an exponential decrease of the firing rate during the cue presentation, followed by a trough of activity (Chafee and Goldman-Rakic, 1998; Romo et al., 1999; G. Rainer and E. K. Miller, personal communication). Such an effect needs to be measured quantitatively, and its underlying cellular mechanism remains to be elucidated.

Stability and synchronization

To sustain persistent activity, a tonic synaptic drive is required to remain significantly above zero at any moment. This can be achieved by the fast AMPA-type synapses alone, if neuronal firings are asynchronous. However, previous work has shown that the asynchronous state is dynamically unstable if the excitatory synapses are too fast (Abbott and van Vreeswijk, 1993). I found that this problem is much more serious in the presence of a strong negative feedback mechanism for rate control. A pertinent question is to what extent this conclusion holds true in the presence of additional factors that increase the disorder of the network. Previous work has shown that noise has a stabilizing effect on the asynchronous dynamics of a network of excitatory neurons (Abbott and van Vreeswijk, 1993;Gerstner, 1999). In another study, a random network of excitatory and inhibitory neurons, coupled with instantaneous synapses, was found to be less synchronous with sparser connectivity (Brunel, 1999). None of these models contains a slow negative feedback mechanism. Here, in a network where recurrent excitation interacts with slow short-term depression, I found that asynchronous dynamics is not stable if the excitatory synapses are fast, even in the presence of synaptic noise and when the network connectivity is very sparse and the neuronal firing properties are widely heterogeneous (Fig. 11). Further analysis is needed to see if asynchronous dynamics are generally unstable in such fast recurrent excitation/slow negative feedback systems, even in the presence of heterogeneity and noise. The problem of stability of the asynchronous dynamics is of interest in the larger context of balanced excitatory–inhibitory neural networks (Shadlen and Newsome, 1994; van Vreeswijk and Sompolinski, 1996).

Therefore, a general finding here is that when an asynchronous persistent state has a low firing rate, its stability requires that the excitatory synaptic time constant be comparable to the effective time constant of the negative feedback mechanism. For a recurrent network of pyramidal cells and interneurons, the stability of a persistent state critically depends on whether the GABA_AR-mediated inhibition is as fast as the AMPAR-activated excitation. For both AMPARs (Geiger et al., 1995) and GABA_ARs (Macdonald and Olsen, 1994), the deactivation kinetics is regulated by the subunit composition and thus may be specific for each cell type. In hippocampal pyramidal neurons of the rat, the decay time constant of the AMPAR-mediated EPSCs is ≃2 msec (at 35°C) (Hestrin et al., 1990a), whereas that of the fast component of the GABA_AR-mediated IPSCs is ≃6–10 msec (Banks et al., 1998). Hence IPSCs are approximately three to five times slower than EPSCs. The present study showed that such a mismatch of synaptic time constants does not favor the stability of an asynchronous dynamics at low firing rates, and for this reason the slow NMDAR channels could be required for the maintenance of a persistent state.

It is an open question whether completely asynchronous dynamics is indeed the modus operandi of delay-period activity in the PFC circuit. Funahashi (1998) recently reported that simultaneously recorded PFC cells displayed significant temporal correlations in a spatial working memory task. In my model simulations, when both the fast AMPA and slow NMDAR-mediated synaptic components are present, the fast AMPAR-activated recurrent excitation in interplay with slower negative feedback processes often leads to synchronous neural firings and network oscillations. In such a synchronous persistent state, the decay time constant of the slow synaptic component must not be too small compared to the average interspike interval (or oscillation period) of neurons. For typical firing rates of PFC cells of 10–50 Hz, ISI ≃20–100 msec, the NMDAR channels are needed.

From cellular physiology to behavior

The present study raised and highlighted a number of experimental questions, their answers will contribute to bridge the gap between behavior-related neural activity and its underlying biological mechanisms.

Synaptic physiology of the prefrontal cortex

(1) What are the precise time courses of the AMPAR-mediated EPSCs and GABA_AR-mediated IPSCs? Is there a mismatch between the two? (2) In response to a repetitive train of stimuli, is the NMDAR-mediated EPSC a linear function of the stimulus frequency in the steady state? If not, what is the frequency above which the current saturates? (3) What are the relative amplitudes of the AMPAR- and NMDAR-mediated EPSCs? Can they be differentially modulated by neuromodulators such as dopamine (Cepeda et al., 1992)? (4) Across the somatodendritic membrane of a pyramidal neuron, is there a spatial segregation of excitatory and inhibitory synapses? (5) What are the short-term plasticity properties of PFC synapses?

Neural delay-period activity of the behaving animal

(6) Is there evidence for adaptation/depression of neuronal discharges? (7) How variable/random is the neuronal persistent activity? Do spike trains display some regular temporal structure? (8) Do neurons fire asynchronously, or is there synchronization within neural assemblies? (9) Can persistent activity of a neuron be switched off by an intracellularly injected current pulse? (no, if delay-period activity is network-induced; yes, if there is bistability at the single cell level) (10) Would local blockade of NMDA receptors in the PFC impair an animal's working memory performance? What are the correlated changes in the delay-period activity of PFC neurons?

Implications for schizophrenia

In recent years, there is growing evidence that working memory impairments are prominent symptoms in schizophrenia (Goldman-Rakic, 1994; Weinberger and Berman, 1996), and that dysfunction of the NMDAR-mediated neurotransmission in the cortex may be at the origin of these cognitive deficits (Javitt and Zukin, 1991; Coyle, 1996). For example, a noncompetitive NMDA antagonist such as phencyclidine or ketamine produces working memory deficits in healthy human subjects that closely resemble schizophrenia (Javitt and Zukin, 1991; Krystal et al., 1994). Moreover, significant alternations in gene expression of the NMDA receptor subunits were found in PFC of schizophrenics (Akbarian et al., 1996). However, the cellular mechanisms through which working memory relies on the NMDAR channels are largely unknown. The present theoretical work suggests a candidate scenario for the working memory malfunction in PFC, namely, an imbalance between the fast AMPAR- and the slow NMDAR-mediated components of the recurrent synaptic transmission within the PFC circuit can give rise to network dynamical instability and disruption of delay-period persistent activity.

Stability of an asynchronous state

In this Appendix, I show that, in general, an asynchronous persistent state is not stable in a fast recurrent excitation/slow negative feedback system. Using a heuristic approach, I will write a dynamical equation for the population activity, in each of the three cases: spike-frequency adaptation, synaptic shunting inhibition, and short-term synaptic depression. Then I will discuss in detail the stability analysis of such a dynamical system.

General remark

Because the excitatory network has a large number of dynamical variables (at least as many as the number of pyramidal cells), a rigorous stability analysis of the network involves as many degrees of freedom (Abbott and van Vreeswijk, 1993; Treves, 1993; Gerstner, 1999). However, our approach is to focus on the fastest and most stable of all dynamical modes for the system (when decoupled from the negative feedback). This would yield a single dynamical equation for the population firing rate which, combined with another equation describing the negative feedback, forms a two-variable system. The idea is that if a steady state is not stable for the two-variable system, it must be unstable for the original network. On the other hand, if it is stable by this description, it still is not necessarily stable for the full network system.