INTRODUCTION

Studies of appetitive conditioned reinforcement (CRf) have implicated dopamine (DA)-dependent limbic-striatal circuitry in mediating the ability of stimuli previously paired with an appetitive reinforcer (eg, food) to support the acquisition of a new instrumental response (AnR) (Taylor and Robbins, 1984; Kelley and Delfs, 1991). The nucleus accumbens (Nac) is optimally situated to allow influences from both the cortex and limbic regions to modulate the behavioral expression of stimulus–reward associations and has been shown to be particularly sensitive to the enhancement of responding for a CRf using the psychomotor stimulant, d-amphetamine (Taylor and Robbins, 1984, 1986; Cador et al, 1991; Kelley and Delfs, 1991). Subregions within the Nac functionally differ in their enhancement of responding for a CRf. Parkinson et al (1999) lesioned either the Nac core (NacC) or shell (NacS) and infused d-amphetamine in the Nac before AnR, which tests the ability of a conditioned stimulus (CS) to reinforce a new instrumental response. The potentiating effects of d-amphetamine on instrumental behavior were shown to be critically dependent on the NacS, whereas the expression or potentiation of Pavlovian conditioned responses generated by the presentation of the food-related CRf depended on the integrity of the NacC. The involvement of these regions in the enhancement of CRf may be ubiquitous to all positive CRfs including drug-paired stimuli and safety signals (SSs).

The prediction that SSs excite the same appetitive motivational system to that of stimuli predicting reward rests on appetitive–aversive interaction theory (Konorski, 1948, 1967, Dickinson and Dearing, 1979). This theory assumes that there are two motivational systems, an appetitive system and an aversive system that reciprocally inhibit one another when activated. Dickinson and Dearing (1979) advanced this theory predicting that safety signals inhibit the aversive system, through signaling the absence of an aversive US, and thereby produce rebound activation of the appetitive system when released from inhibition by the aversive system. If both safety signals and appetitive stimuli activate a common appetitive system, then it could be hypothesized that the neurochemical basis of their reinforcing properties may also be equivalent. Activation of a common appetitive system by safety signals could explain the persistence of maladaptive avoidance behaviors symptomatic of anxiety disorders that are thought to be reinforced by the relief experienced following their completion, even in the absence of an explicit aversive event (Roper et al, 1973).

SSs reinforce instrumental avoidance behavior through signaling the omission of aversive events, thus inhibiting fearful behavior (Rescorla, 1969; Weisman and Litner, 1969; Morris, 1975; Fernando et al, 2013). The conditioned properties of a SS are predicted to occur as a result of two processes, a Pavlovian process and an instrumental process that occur during avoidance conditioning, formalized in Konorski’s two-factor theory of avoidance (Konorski, 1948, 1967). The first, Pavlovian process, is the inhibition of a conditioned defensive freezing response to the aversive context, with the successful performance of an avoidance response that prevents the occurrence of the negative reinforcer. SSs presented contingent on the avoidance response will thus anticipate the omission of the negative aversive reinforcer leading to their establishment as conditioned fear inhibitors. The second, instrumental process, is the reinforcement of the avoidance response by the SS because of its fear-inhibiting properties. The SS as a conditioned fear inhibitor will therefore not only provide ‘relief’ during its presentation, but will also result in animals responding to produce the SS as a reinforcer of avoidance behavior. The dependence of the reinforcing properties of a SS on its ability to inhibit fearful behavior, that is the aversive motivational system, could result in different neural mediation to that of an appetitive stimulus despite both stimuli reinforcing instrumental behavior.

The few studies of the neural basis of the conditioned properties of a SS have failed to identify regions of the central nervous system that mediate the conditioned properties of a SS (Falls and Davis, 1995; Falls et al, 1997; Gewirtz et al, 1997; Josselyn et al, 2005). Only recently has there been any headway in defining the central nervous system substrates of a SS. Increased neuronal activity has been recorded in the BLA (Sangha et al, 2013) and caudate–putamen (Rogan et al, 2005) to safety cues as well as a reduction in CS-evoked activity in the LA (Rogan et al, 2005), possibly because of changes in LA dendritic spine structure with safety conditioning (Ostroff et al, 2010). Lesion studies of the posterior insular cortex (Christianson et al, 2008) and post-training lesions of the auditory thalamus (Heldt and Falls, 2006) have also been shown to disrupt the inhibitory properties of a SS. These studies assessed the effects of their manipulations on the fear-inhibiting properties of a SS but not on their secondary properties to reinforce instrumental behavior. This study thus assessed the neural and neurochemical basis of the relieving and reinforcing properties of a SS with infusions of d-amphetamine in the NacC or NacS that have been shown previously to enhance the conditioned reinforcing properties of an appetitive stimulus on instrumental behavior. Subjects were first trained on a free-operant lever-press avoidance paradigm and then tested using an experimental procedure sensitive to the measurement of the relieving and reinforcing properties of a safety signal, as described in Fernando et al (2013, Experiment 2) following infusions of d-amphetamine. In separate groups of rats, the selectivity of effects in the NacS was further explored with inactivation of this region using the GABA-A agonist muscimol and GABA-B agonist baclofen before the same test sessions in extinction. Given the catecholamine potentiating effects of d-amphetamine, infusions of the DA receptor (R) antagonist α-flupenthixol (α-flu) and the selective noradrenaline reuptake blocker atomoxetine (ATO) were also tested in the NacS to further characterize the pharmacological effects of d-amphetamine.

MATERIALS AND METHODS

Subjects

We used experimentally naive, male, Lister-hooded rats, weighing ∼300 g at the start of the experiment, obtained from Charles River, UK. Four rats were housed per cage in a reverse light cycle room (12 h light:12 h dark; lights on at 0700 h). Experiment 1 was conducted in two cohorts. In the first cohort, 23 rats were trained in experiment 1, 8 rats were cannulated in the NacC and 10 in the NacS, and 5 did not acquire the task before surgery. Sixteen rats were trained in cohort 2 and cannulated in the shell (n=8) and in the core (n=8). Experiment 2 was conducted in a single cohort of 16 rats. Experiment 3 was conducted in one cohort of 16 rats, 1 rat did not acquire the task and therefore did not undergo surgery, and the other 15 rats were then cannulated in the NacS. Food and water were available ad libitum throughout both experiments. Training and testing occurred during the dark phase and complied with the statutory requirements of the UK Animals (Scientific Procedures) Act 1986.

Apparatus

Eight operant conditioning chambers (Med Associates, Vermont) each measuring 29.5 cm by 32.5 cm by 23.5 cm with a Plexiglas ceiling, front door and back panel, and metal paneling on the sides of the chamber were used for all training and tests conducted in this study. The floor of the chamber was covered with a metal grid with a metal tray beneath. Med Associates shocker generators (ENV-224AMWN, 115 V AC, 60 Hz) were connected to the metal grid and used to produce scrambled 0.5-s, 0.5-mA foot shocks. Each chamber was placed within a sound and light attenuating box and interfaced to a computer through Whisker control software (Cardinal and Aitken, 2010). The SS was a 2900-Hz tone produced by a Med Associate tone generator (ENV-223AM) for half of the rats and a white noise by a Med Associate white noise generator (ENV-2255M) for the remaining rats. Both these generators were attached to the same wall of the chamber, and the stimuli were set to 8 dB above background level. Levers could be extended either side of a central food magazine on the opposite side wall, but no pellets were ever delivered.

Pretraining

Rats were first habituated to the chamber and the levers for four sessions. During the first 2 days, either the left or the right lever was randomly chosen at the start of the session. The designated lever was then extended as the session began and any responses resulted in its retraction for 1 s followed by its immediate extension back into the chamber. For the last 2 days, the opposite lever was extended and the number of responses was limited so that the number of retractions and extensions of the two levers was equated, and houselights remained on until the end of the session. Each daily session lasted for 1 h. No shocks or auditory stimuli were presented during these sessions.

Training

The start of the session was marked with the illumination of the houselight and the extension of a single lever that was randomly chosen as either the right or the left lever at the start of the session. This lever remained permanently extended for the entire session. The session began with an unsignalled avoidance period of 60 s, and in the absence of a lever press was followed by a period of intermittent foot shocks. During this shock period, the shock–shock interval was 10 s. After the presentation of five shocks, the shock period terminated automatically and was immediately followed by the next avoidance period. The maximum number of shocks that could be presented in the session was limited to 30 at which point the session ended. Any lever press during the avoidance or shock periods immediately terminated these periods with a 60-s auditory SS that was then followed by the next avoidance period. No shocks were presented during the 60 s SS that therefore signaled a period of safety. Across sessions, the levers were randomly switched to either right or left on a daily basis, so they were equated for exposure across all stages of training. Lever presses during the signal had no consequence and did not contribute to the assessment of avoidance responding. The durations of the avoidance period and the SS (and contiguous safety period) were gradually reduced across training to a variable interval (VI) of 32 s (range 3–60 s) and 5 s, respectively.

Surgery

Bilateral 22-gauge double guide cannulae (Plastics One, Sevenoaks, UK) were implanted in the NacC (n=16) or NacS (n=18) in experiment 1, in the NacS (n=11) in experiment 2, and in NacS (n=15) in experiment 3. The guide cannulae protruded 4 mm below the pedestal for the NacC and 3 mm below the pedestal for the NacS. Subjects were implanted in the NacC according to the stereotaxic coordinates of AP +0.6, ML+1.9, and DV −2.2, or were implanted in the NacS according to the stereotaxic coordinates of AP +1.7, ML+0.75, and DV −2.0, both measured from bregma and DV measured from skull. After surgery, rats were individually housed and left to recover with both food and water ad libitum. After 1 week of recovery, subjects were retrained on the avoidance task until 3 days of stable baseline was observed.

Drugs

d-Amphetamine sulf, αα-flu, and ATO (all obtained from Sigma, UK) were dissolved in sterile PBS, pH 7.2, for intracerebral infusions. Inactivation of the NacS in experiment 2 was achieved with a mixture of the GABA-B and GABA-A receptor agonists baclofen (bac) and muscimol (mus) dissolved in PBS.

Infusion procedure

Subjects were infused bilaterally through injectors that extended 5 mm beyond the guide cannulae tips for the NacC and 5.25 mm for the NacS. The injectors were attached to an infusion pump (Harvard Apparatus) by polyethylene tubing. Rats were infused with a volume of 0.5 μl per side over 60 s in experiments 1 and 3 and 0.3 μl in experiment 2 (Ghods-Sharifi and Floresco, 2010) with a diffusion time of 120 s for all infusions. Rats in experiment 1 received infusions at two dose levels, veh (PBS) and 5 μg (calculated as free-base), and rats in experiment 2 received infusions of a mixture of baclofen and muscimol (bac-mus) at doses of 0.3 and 0.03 nmol, respectively, that were counterbalanced with infusions of vehicle. Rats in experiment 3 received infusions of α-flu at three dose levels (Veh, 2.5, and 15 μg) and then at two dose levels (Veh and 5 μg). The same subjects then received infusions of the selective noradrenaline reuptake inhibitor, ATO, at three dose levels (Veh, 1.5, and 5 μg). Subjects were placed in the testing chamber and the session started 10 min after infusion (Taylor and Robbins, 1984, 1986) with d-amphetamine in experiment 1, 15 min after infusion with α-flu, and 5 min after infusion with ATO (Economidou et al, 2012) and bac–mus (Jonkman et al, 2012). Rats were infused in continuous 2-day cycles, an infusion day, followed by a drug-free, ‘re-baseline’ day. All rats received a sham infusion before the first drug infusion session in which they were habituated to the infusion procedure and infusion room. Injectors were lowered and 0.9% filtered saline was infused for 1 min and the injectors subsequently left in place for 2 min before the subject was returned to its home cage. This was performed so that any behavioral effects of tissue damage mechanically induced by the first infusion occurred before the test session. Rats were then re-baselined on the avoidance task until stable levels of responding were again seen. Subjects were also given re-baseline training sessions between Latin square sequences.

Test session

Infusions were conducted before extinction tests either with the SS presented or not following responses on a single lever, and no shocks were presented during these test sessions. Test sessions were counterbalanced for order of drug and test session and lasted for 30 min. The single lever was randomly chosen before each test session as either the right or left lever.

Assessment of cannulae placement

Following completion of the behavioral procedures, animals were deeply anesthetized with Dolethal (pentobarbitone sodium, 200 mg/ml, minimum of 1.5 ml intraperitoneal) and perfused transcardially with 0.01 M phosphate-buffered saline followed by 4% paraformaldehyde in PBS. Their brains were removed and postfixed in paraformaldehyde before being dehydrated in 20% sucrose for cryoprotection. The brains were sectioned coronally at 60 μm and stained for ∼5 min with cresyl violet. Following staining, sections were mounted and allowed to dry for examining cannulae positions. Only animals with correct cannulae placements were included in the analyses.

Data analysis

We have previously demonstrated that lever press avoidance behavior can be used to assess the relieving and reinforcing properties of the SS by assessing different rates of responding that occur during an avoidance session (Fernando et al, 2013). Responses made during the SS are taken as a measure of the inhibitory properties of the SS, as when removed, specific increases in fearful, nonconsequential responding were seen during the 5 s unsignaled period of safety (Fernando et al, 2013). Responses that avoid or escape shock and thus produce the SS can be taken as a measure of the reinforcing properties of the safety signal, and removing the SS reduces these responses (Fernando et al, 2013). The test sessions in this study were conducted in extinction (in the absence of shock) in order to measure the effects of the infused drugs on the conditioned properties of the SS and in the absence of the unconditioned effects of shock presentations. The well-documented slow extinction of avoidance behavior (Maier and Klee, 1943; Mowrer, 1947; Solomon et al, 1953) enabled the analysis of the same rates of responding during the extinction test sessions in this study. Responses made during the 5 s following a lever press response were computed as the rate of SS responses; in some sessions, this was in the presence of the SS and in control sessions this was in the absence of the safety signal. Responses on the lever that produced the SS or ‘safety period’ (in control sessions) were taken as the rate of avoidance responding, as during training these responses would normally avoid or escape shock and produce the SS. This analysis of instrumental avoidance behavior has been used in previous studies (Dinsmoor and Sears, 1973; Rescorla and Lolordo, 1965).

Statistical analyses

The rates of lever press responding were square-root transformed for statistical analysis as the variance increased with mean responding. A mixed design ANOVA was conducted on response rates. Within-subject factors were SS (SS vs no safety signal), response (SS responses vs avoidance responses), and dose (Veh vs dose 1, dose 2, and so on, dependent on drug infused). For experiment 1, a between-subject factor of brain region (shell vs core) was initially included in the analysis to investigate interactions between factors, and the effects of d-Amphetamine on the rates of responding were then subsequently analyzed for each region in separate analyses. Violations in the assumptions of sphericity in repeated measures analysis were corrected using Huynh–Feldt epsilon. All post hoc comparisons were subject to the Sidak correction.

RESULTS

Histology

The injector tips for each experiment are shown next to their respective graphical data: Figure 1a, placements in the NacS in rats that had received infusions of d-amphetamine; Figure 2a, placements in the NacC in rats that had received infusions of d-amphetamine; Figure 3a, placements in the NacS of rats that had received inactivations; and Figure 4a, placements in the NacS from rats that had received α-flu followed by ATO. The final numbers after loss of head caps through the course of testing or unstable baseline avoidance behavior were a total of 10 rats cannulated in the NacC and 12 rats in the NacS in experiment 1. Experiment 2 had 5 rats cannulated in the NacS that received infusions with low doses of methylnaloxonium (data not shown, as no effects were detected) and inactivation with a mixture of baclofen and muscimol. Experiment 3 consisted of 10 rats that received infusions of α-flu and 9 rats that received infusions of ATO. There was no gross tissue damage in the local vicinity of the injector tracks in the placements of these rats. Washout periods were given between different drugs infused in the same rat.

Figure 1
figure 1

Infusions of d-amphetamine in the NacS resulted in bidirectional effects, increasing safety signal responding while decreasing the rate of avoidance responding, only when the safety signal was present in the test session. (a) Placements in the NacS. (b) Rate of safety signal responses, responses during safety signals, and (d) rate of avoidance responses, responses that produce the safety signal. (c) Rate of safety signal responses and (e) rate of avoidance responses in test sessions with no safety signal present. Each bar represents the mean of the SQRT of the rate of responding per min±SEM.

PowerPoint slide

Full size image
Figure 2
figure 2

d-Amphetamine infusions in the core had no effect on the rates of responding during the extinction test irrespective of whether the safety signal was present or absent in the session. (a) Placements in the NacC. (b) Rate of safety signal responses, responses during safety signals, and (d) rate of avoidance responses, responses that produce the safety signal. (c) Rate of safety signal responses and (e) rate of avoidance responses in test sessions with no safety signal present. Each bar represents the mean of the SQRT of the rate of responding per min±SEM.

PowerPoint slide

Full size image
Figure 3
figure 3

Inactivations of the NacS selectively decreased the rate of avoidance responding only in sessions where the safety signal was present. (a) Placements in the NacS. (b) Rate of safety signal responses, responses during safety signals, and (d) rate of avoidance responses, responses that produce the safety signal. (c) Rate of safety signal responses and (e) rate of avoidance responses in test sessions with no safety signal present. Each bar represents the mean of the SQRT of the rate of responding per min±SEM.

PowerPoint slide

Full size image
Figure 4
figure 4

Infusions of α-flupenthixol decreased overall rates of responding irrespective of whether the safety signal was present or absent in the test session. (a) Placements in the NacS. (b) Rate of safety signal responses, responses during safety signals, and (d) rate of avoidance responses, responses that produce the safety signal. (c) Rate of safety signal responses and (e) rate of avoidance responses in test sessions with no safety signal present. Each bar represents the mean of the SQRT of the rate of responding per min±SEM.

PowerPoint slide

Full size image

Baseline Analysis

Analysis of the last three sessions of training before infusions of d-amphetamine in the NacC and NacS revealed a significant effect of session (F(1.8, 32.1) =4.4 P<0.05); however, this effect did not interact significantly with cohort or region (session × cohort, F(2, 36) =1.1, P=0.3; session × region, F<1, session × cohort × region, F<1). No significant differences in baseline responding before inactivations (session F<1), the α-flu infusions (session F<1), or ATO infusions were seen (session F(2, 14) =1.2 P=0.3 NS). Rats that received α-flu infusions and then ATO infusions showed no difference in baseline responding before the infusion of the drug (drug; session; drug × session, all F<1), suggesting a stable baseline was maintained through the course of the two drugs tested in experiment 3.

Analysis of responding under vehicle infusions during the extinction tests revealed a higher rate of avoidance responding in sessions where the SS was present following avoidance responses (F(1, 33)=18.8, P<0.001, effect size, P=0.04) that did not differ between experiments (F(3, 33)=2.5, P>0.07 NS, effect size, P=0.2). The SS therefore reinforced avoidance behavior under vehicle conditions across experiments. Analysis of SS responding under vehicle infusions revealed no significant effect of the presence of the SS in test sessions (F(1, 33)=1.1, P=0.3 NS, effect size, P=0.03) and no interaction with experiment (F(3, 33)=1.0, P=0.4 NS, effect size, P=0.08), again suggesting that responding during the extinction test was comparable across experiments. ATO data were not included in these analyses as these rats were also used for the α-flu experiment.

Experiment 1: Effects of central infusions of d-amphetamine in the NacC or NacS on avoidance responding with or without the SS

Infusions of d-amphetamine affected both the rate of SS responding (responses during presentations of the SS) and avoidance responding (responses made outside SS presentations) (response × dose F(1, 20) =7.3, P<0.02). These effects depended on the Nac subregion (response × dose × region, F(1, 11) =9.6, P<0.01). A main effect of response was revealed with the ANOVA (response F(1, 11) =9.1, P<0.05); however, no other factors or interactions reached the conventional levels of significance (SS; SS × dose F’s <1, dose F(1, 11) =1.1, P=0.3 NS, SS × response F(1, 11) =1.8, P=0.2 NS) Subsequent analyses were conducted separately for each region in order to assess the effects of d-amphetamine on rates of responding particular to the region infused.

Nucleus Accumbens Shell

Figure 1 depicts the effects of infusions of d-amphetamine in the NacS. Figure 1b and d demonstrate the bidirectional effects of d-amphetamine (response × dose F(1, 11) =26.6, P<0.001), increasing the rate of SS responses and decreasing the rate of avoidance responses only when the SS was present in the session (SS × response × dose F(1, 11) =7.2, P<0.05). Analysis of responding during the test sessions where the SS was present revealed a significant dose × response interaction (F(1, 11) =33.6, P<0.001). The post hoc comparisons confirmed the observation that infusions of d-amphetamine in the NacS increased the rate of responding during the presentations of the SS (dose P<0.005) but decreased the rate of avoidance responses that produced the SS and avoided or escaped shock (dose P<0.005). Analysis of the test session without the SS shown in Figure 1c and e revealed a trend toward an interaction between dose and response; however, this did not reach conventional levels of significance (F(1, 11) =3.5, P>0.08). No significant effects were seen with post hoc analysis of the effects of d-amphetamine on the rate of SS responding (dose P=0.6; Figure 1c) or on the rate of avoidance responding (dose P=0.1; Figure 1e).

Nucleus Accumbens Core

Figure 2 demonstrates the failure to detect an effect with infusions of d-amphetamine in the NacC, irrespective of whether the SS was present or absent in the session (SS, dose, SS × dose, response × dose, SS × response × dose F<1; SS × response F(1, 9) =2.0, P=0.2 NS).

Experiment 2: Effects of inactivation of the NacS with a mixture of GABA-B and GABA-A receptor agonists bac and mus

The graphical data of Figure 3 suggest that NacS inactivation reduced avoidance response rates when responding was reinforced by the SS. However, statistical analysis did not reveal any significant effects of infusions of bac–mus in the NacS (dose F(1, 4)=4.7, P=0.1 NS; dose × response F(1, 4)=6.5, P=0.06 NS; SS × dose F(1, 4)=2.1 P=0.2 NS). A main effect of the SS was seen, indicating that its presence influenced instrumental responding during the session (SS F(1, 4)=12.6, P<0.05). Separate analyses were therefore conducted on sessions in which the SS was present and sessions where it was absent to assess the selective effects of inactivation of the NacS on responding. Analysis of responding during test sessions where SSs were presented following responding revealed an interaction of the inactivation with the different response rates in the session (dose × response F(1, 4)=10.2, P<0.05). The post hoc comparisons revealed that the inactivation specifically decreased the rate of avoidance responding (dose P<0.05) but not the rate of SS responses (dose P=0.9 NS). A main effect of the inactivation was also observed in test sessions with the SS (dose F(1, 4)=11.1, P<0.05) but no main effect on response rates (F(1, 4)=1.6, P=0.2 NS).

Analysis of responding during test sessions without the SS present revealed no significant effects of inactivation of the NacS on responding (dose F(1, 4)=1.1, P=0.4 NS; dose × response F(1, 4)=1.4, P=0.3 NS). No main effect of response (F<1) was seen with statistical analysis in sessions without SS presentations. Inactivation of the NacS therefore selectively reduced the rate of avoidance responses producing the SS but did not affect responding during the SS itself nor the response rates when the SS was not present in the test session.

Experiment 3a: Effects of central infusions of the DA-R antagonist α-flu in the NacS on avoidance responding with or without the SS

Overall rates of responding did not differ between the two vehicle control infusions of the two Latin squares of α-flu infusions, whether the signal was present or absent in the session (Vehicle F(1, 9) =2.0, P=0.2 NS, Vehicle × SS F<1). A main effect of SS was observed with this ANOVA (SS F(1, 9) =10.9, P <0.01). The two Latin squares were therefore analyzed and presented as a single dose-response curve with responding under the two vehicles averaged.

Figure 4 illustrates the significant decrease in the rates of responding with infusions of α-flu in the shell (dose F(3, 27) =7.9, P<0.005) that did not differ between response types (ie, SS responding and avoidance responding) regardless of whether the signal was present or absent (response × dose, SS × dose, SS × response × dose, Fs <1.4, P>0.2).

Experiment 3b: Effects of central infusions of the SNRI ATO in the NacS on avoidance responding with or without the SS

No effects were observed with administration of the SNRI, ATO, during the test sessions that are supported by statistical analyses (dose; response × dose; SS × dose; SS × response × dose; Fs <1.3, P>0.3, SS F(1, 7)=1.8 P=0.2 NS), suggesting that the selective effects of d-amphetamine in the NacS are not mediated by noradrenergic mechanisms. A main effect of response (F(1, 7) =7.9, P<0.05) and an interaction of SS × response were revealed with the ANOVA (SS × response F(1, 7)=14.7, P<0.01). Means and SEMs are presented in Table 1. Placements of animals are shown in Figure 4a.

Table 1 Effects of Atomoxetine in the NacS in Test Sessions with and without the Safety Signal at Three Dose Levels on the Rate of Safety Signal Responses and Avoidance Responses
Full size table

DISCUSSION

The results of this study indicate a previously unknown role of the NacS in mediating the conditioned properties of a SS on avoidance behavior. The effects of infusions of d-amphetamine, α-flu, and the null effects of ATO in the NacS suggest that this may be achieved via its dopaminergic innervation. This is the first study to assess the role of subregions of the Nac in the conditioned reinforcing properties of a safety signal, demonstrating a striking reduction in its ability to reinforce avoidance behavior with intra-NacS infusions of d-amphetamine. This study not only implicates the NacS in the mediation of conditioned reinforcement by a SS, but also suggests that the effects of psychostimulants and other drugs in the NacS vary depending on the opponent motivational properties of the CS.

Effects of d-Amphetamine on Free-Operant Avoidance Responding

The lack of any effect of d-amphetamine within subregions of the Nac suggestive of enhancement of the reinforcing properties of the SS advocates an alternative neural circuitry to that previously reported for an appetitive CRf in mediating the conditioned properties of a SS. Such SSs also have fear-inhibiting properties as shown by an increase in responding during the period of safety following an avoidance response with their omission (Fernando et al, 2013). The specific increase in responding during the presentation of the SS with infusions of d-amphetamine in the NacS thus suggests a disruption of the conditioned fear-inhibiting properties of the SS (Figure 1b). The reinforcing properties of a SS have in turn been shown to depend on the ability of the SS to act as a conditioned inhibitor of fear (Rescorla, 1969; Weisman and Litner, 1969; Bolles, 1972) as predicted by theories of appetitive–aversive interactions and avoidance (Konorski, 1948, 1967; Soltysik, 1963; Dickinson and Dearing, 1979; Dinsmoor, 1954, 2001). This disruption of the inhibitory properties of the SS with central infusions of d-amphetamine in the NacS thus diminished the ability of SS to act as a CRf. The differential effects of d-amphetamine on responding for a signal of safety versus that of reward may be because of the requirement of a SS first being conditioned as an inhibitor before it can subsequently act as a positive CRf of instrumental avoidance behavior. Intra-NacS d-amphetamine disrupted this capacity of the SS, diminishing its consequential capability to reinforce avoidance behavior.

The effects of infusions of d-amphetamine in the Nac on the inhibitory properties of a SS have been examined using a fear-potentiated startle paradigm (Josselyn et al, 2005). Infusions of d-amphetamine, the AMPA/kainate receptor antagonist CNQX, and lesions of the Nac had no effect on conditioned inhibition of fear-potentiated startle by a SS. Josselyn et al (2005) used an A+/AB− paradigm in order to condition a SS. The inability to detect any effects of manipulations of the Nac in the study of Josselyn et al (2005) could be attributed to a failure to dissociate contributions of the NacC and NacS that have been shown to have dissociable roles in both appetitive (Ito et al, 2000,2004; Murphy et al, 2008; Ghods-Sharifi and Floresco, 2010; Ostlund et al, 2011; Ito and Hayen, 2011; Economidou et al, 2012) and aversive paradigms (Pezze et al, 2001; Badrinarayan et al, 2012; Budygin et al, 2012). This study differs from those previously reported by using a free-operant avoidance paradigm reinforced only with a SS, analyzing different rates of responding during the avoidance session to enable assessment of the Pavlovian inhibitory properties of the SS and its ability to reinforce instrumental avoidance. Infusions in subregions of the Nac using this paradigm thus provides novel data on the selective anatomical effects of d-amphetamine and their specific modulation of the Pavlovian and instrumental conditioned properties of a SS.

Possible Neural and Neurochemical Mechanisms

The contrasting effects of infusions of d-amphetamine when responding for a SS versus that previously reported in appetitive conditioned reinforcement studies could be attributed to transient, phasic changes in DA that signal appetitive and aversive events (Roitman et al, 2004) that are unique to an avoidance trial (Oleson et al, 2012). Oleson et al (2012) observed fluctuations in DA before the performance of an avoidance or escape response and a specific sub-second DA release in the Nac during the safety period following both avoidance and escape responses, as occurs with cues that predict sucrose reward (Roitman et al, 2004) or cocaine (Phillips et al, 2003). Oleson et al (2012) did not distinguish the contributions of the NacS and NacC to this signal; however, this sub-second enhanced release following the termination of an aversive event (tail-pinch) was localized to the NacS (Budygin et al, 2012). This specific sub-second DA release in the NacS may thus be a necessary component of the fear-inhibiting effects of a SS. Intra-NacS infusions of d-amphetamine in an avoidance setting may occlude phasic fluctuations in DA, particularly the sub-second DA release by enhancing tonic levels of DA. This action may thus render the SS ineffective as a conditioned inhibitor of fear, and therefore as a positive reinforcer of avoidance behavior.

The reduction in rate of avoidance responding was further demonstrated with inactivations of the NacS and with α-flu infusions. Greater selectivity was seen with inactivations of the NacS as the reduction in avoidance responding was only seen in the sessions with the SS with no effect on the rate of SS responses. Antagonism of DA D1/D2 receptors with α-flu led to an overall decrease in the rate of SS responses and avoidance responses regardless of whether the stimulus was present or absent in the test session (Figure 4). This nonselective reduction in avoidance behavior following antagonism of DA receptors (R) in a region of the Nac supports the finding of (McCullough et al, 1993) that Nac DA depletions using 6-hydroxydopamine resulted in a substantial decrease in lever pressing to avoid or escape shock. Infusions in the Nac of the D1 antagonist, SCH 23390, and the D2 agonist quinpirole (at presynaptic doses that prevent the reuptake of DA) have also been shown to reduce conditioned two-way active avoidance responding (Boschen et al, 2011; Wietzikoski et al, 2012). The α-flu has also been shown to decrease the efficacy of a conditioned punishing stimulus when administered systemically (Killcross et al, 1997). The effects of d-amphetamine, an indirect DA-R agonist, and α-flu, a D1/D2 R antagonist, were expected to be in opposite directions for responding under the control of SSs. Such contrasting effects were, however, not evident in this study, although in overall terms, d-amphetamine increased, whereas α-flu decreased the rate of SS responding. However, these effects were also accompanied by significant decreases in avoidance responding following both drugs. Reduced avoidance responding was also seen following inactivation of the NacS. However, similar to d-amphetamine, this effect was selective to those test sessions when the SS was present, whereas the effects of α-flu were nonselective, suggesting that the apparently similar effects on rates of avoidance responding with all three drugs may have resulted from different mechanisms. Indeed, the rate-decreasing effect of d-amphetamine may have arisen secondarily as outlined above, as a consequence of the reduction of the conditioned fear-inhibitory effects of the safety signal, whereas this is unlikely to have been the case for either the bac–mus inactivation or for α-flu.

Infusions of ATO in the NacS, a blocker of the noradrenaline transporter like d-amphetamine, had no effect on instrumental avoidance behavior irrespective of whether the stimulus was present or absent. This suggests that the noradrenergic innervation of the NacS is not involved in the inhibitory properties of the SS and that the effects of d-amphetamine in the NacS are more likely to be selectively mediated by activation of the dopaminergic innervation of the NacS. A limitation in interpreting the results of the ATO infusions was that they were performed after infusions of α-flu, leading to a possible reduced sensitivity to further infusions. However, the stable levels of responding during rebaseline sessions between drugs suggests that the inhibitory properties of the safety signal, which were shown to be dependent on the NacS in experiment 1, were still effective in reinforcing avoidance behavior during those rebaselining sessions after infusions of α-flu. Furthermore, a lack of effect of noradrenergic manipulations in the Nac on responding for a positive conditioned reinforcer has also been observed in studies with appetitive stimuli previously paired with water reward (Cador et al, 1991). Comparable doses of ATO in the NacS have been shown to selectively reduce premature responding, a measure of impulsive behavior, during an attentional task (Economidou et al, 2012), confirming the drug is behaviorally active at these doses.

CONCLUSIONS AND SUMMARY

This study reports novel data examining the neural mediation of the conditioned reinforcing properties of an instrumentally trained SS on free-operant avoidance. The study was conducted in two subregions of the Nac both known to facilitate the enhancement of appetitive CRf with d-amphetamine. Unlike previous studies of appetitive conditioned reinforcement, d-amphetamine was shown to decrease the reinforcing properties of the SS when infused in the NacS, demonstrating anatomical selectivity. This effect is attributed to d-amphetamine disrupting the inhibitory properties of the SS through interference with the fluctuating levels of DA occurring during avoidance behavior that signal appetitive and aversive events. This disruption of inhibition by amphetamine is hypothesized to reduce the conditioned reinforcing properties of the SS during avoidance behavior. The subsequent infusion studies in the NacS not only confirm the role of the NacS in mediating the conditioned properties of the SS on avoidance, but also its possible DAergic mediation.

Overall, these findings in conjunction with others argue that SSs do not support avoidance behavior by acting via the same neural mechanisms that mediate effects of appetitive (eg, food or water related) conditioned reinforcers. In other words, subjective ‘relief’ may be construed as a rewarding event by virtue of the omission of an aversive stimulus in a neural system of aversion that is distinct from that of reward.

FUNDING AND DISCLOSURE

TWR is a consultant for Cambridge Cognition, Eli Lilly, GSK, Merck, Sharpe and Dohme, Lundbeck, Teva, and Shire Pharmaceuticals. He is or has been in receipt of research grants from Lundbeck, Eli Lilly, and GSK and is an editor for Springer-Verlag. The other authors declare no conflict of interest.