No full-text available
To read the full-text of this research,
you can request a copy directly from the author.
Delayed reward learning: Disproof of the traditional theory
Abstract
Without the aid of secondary rewards to bridge the temporal gap, each of 15 rats learned to select the rewarded side of a T-maze although the reward was delayed until 1 min after the response was emitted. Similar results were obtained from another group of eight rats for which the length of the delay was 5 min. In a final experiment using the same basic procedure, five groups of rats were trained for 25 days with delays of 0.5, 1.0, 2.0, 4.0, or 8.0 min. The percentage of correct responses did not significantly differ among groups. According to prevailing psychological theory, these results are impossible.
... The elegance of Grice's work convinced most psychologists that the problem of delayed reinforcement had at last been resolved, with the result that little further research was Copyright 1979 by the American Psychological Association, Inc. 0097-7403/79/0503-0224$00.75 carried out in this area over subsequent decades (but see Lawrence & Hommel, 1961). Recently, however, Lett (1973Lett ( , 1974Lett ( , 1975 reported dramatic evidence that even with secondary reinforcement controlled, reinforcement may still be effective after delays of 1 hr. The rationale for Lett's research was based on an analysis of delayed reward proposed by Revusky (1971). ...
... The apparatus and procedure were essentially similar to those used by Lett (1973), but several modifications were introduced in order to facilitate the interpretation of the results. In Lett's experiment, subjects were placed in the central stem of a T-maze, on either side of which were transparent doors leading to black and white side arms. ...
... The apparatus used was similar to the T-maze used by Lett (1973) and had similar dimensions. The major change was the addition of a separate start box (see Figure 1). ...
In 4 experiments, male or female PVG hooded rats were trained on spatial discriminations in which reward was delayed for 1 min. Exp I tested B. T. Lett's hypothesis that responses made in the home cage during the delay interval are less likely to interfere with learning than responses made in the maze. Experimental Ss were transferred to their home cages during the delay interval, and controls were picked up but then immediately replaced in the maze. Contrary to Lett's hypothesis, both groups learned. Further experiments suggested that handling following a choice response was the crucial variable in producing learning: No learning occurred when handling was delayed (Exp II) or omitted (Exp III). One possible explanation for the fact that handling facilitated learning is that it served to mark the preceding choice response in memory so that Ss were then more likely to recall it when subsequently reinforced. In accordance with this interpretation, learning was found to be just as strong when the choice response was followed by an intense light or noise as by handling (Exp IV). The implication of marking for other phenomena such as avoidance, quasi-reinforcement, and the paradoxical effects of punishment is also discussed. (20 ref)
... Lett (1973Lett ( , 1975 showed that rats can learn a spatial discrimination in aT-maze with delays of reward lasting as long as 1 h. The unusual feature of the procedure was that the response and the reward occurred in one place, the T·maze, while the events of the delay were made to occur in another place, the home cage. ...
... Some confirmation of this prediction can be found in Denny's (1974) experiment. In this experiment, two groups of rats were trained to perform a spatial discrimination in aT-maze with a procedure similar to that used by Lett (1973). Briefly, Denny's procedure was as follows. ...
... Apparatus. The training apparatus was a truncated T-maze (Lett, 1973(Lett, , 1975 consisting of a startbox (30.5 x 17.8 x 15.2 cm) flanked on either side by an endbox (33.0 x 12.7 x 15.2 cm). The startbox had gray walls, the left endbox had white walls and the right endbox, black walls. ...
Learning of a spatial discrimination in a T-maze with long delayed reward has been demonstrated by means of a procedure in which the rat is removed after a response to spend the delay in its home cage and later is returned to the T-maze for reward. This procedure was derived from Revusky’s (1971, in press) concurrent interference theory of long delay learning. According to this theory, the usual decrement in learning with a delay of reward is the result of concurrent interference produced when the response and/or reward become involved in extraneous associations with delay events. Removal of the rat to spend the delay outside of the T-maze minimizes concurrent interferences by reducing the associability of the delay events with respect to the events that occur inside the T-maze such as the response and the reward. The present experiments were concerned with a corollary of this theory: If events occurring in the home cage are not readily associated with events occurring in the T-maze, then a reward should be less effective when administered in the home cage than when administered in the T-maze. It was found that learning is retarded when reward is given in the home cage; however, reward given in the home cage as much as 2 min after a response produces learning.
... To test the generality of his concurrent interference theory, Revusky initiated a series of maze studies with delayed reward that were continued by Lett (1973;. The idea behind these studies was that the steep temporal gradient typically found in maze experiments (as discussed in the early part of this review) was due to interfering events during the delay period whose influence could be reduced if the rat was removed from the maze. ...
... In one example of these maze experiments Lett (1973) gave delayed rewards to rats for making a correct turn in a T-maze. After making a response, whether correct or incorrect, rats were removed from the maze and placed in their home cages for a certain period before being returned to the apparatus, where they received food if the preceding response had been correct. ...
... One attempt to replicate Lett's (1973) results included a control group of rats that were picked up and then returned to the maze. Surprisingly, this control group learned the position discrimination despite a 1-min delay of reward as rapidly as those confined to their home cage during the delay period (Lieberman, McIntosh & Thomas, 1979;; Experiment 1). ...
The greater the separation in time between 2 events, A followed by B, the less likely they are to become associated. The dominant explanation of this temporal contiguity effect has been trace decay: During the interval between A and B, the trace left by A becomes too weak by the time B occurs for an association to be formed between them. Pavlov adopted this idea in the context of classical conditioning and Hull used it to account for the deleterious effect of delaying reinforcement on the acquisition of instrumental responses. By 1960 various studies supported the conclusion that animals could not learn to associate 2 events separated by more than around 45 s. Research on human skill acquisition with delayed feedback and later studies using causal or predictive judgment tasks indicated that explicit cognitive processing is generally involved when humans associate events separated by more than a few seconds. The discovery of long-delay taste aversion learning prompted Revusky's (1971) alternative analysis of contiguity effects in terms of interference: The greater the separation between A and B, the more likely that extraneous events compete for association with A and B. Although the analysis of overshadowing provided by associative learning theories provides a context for this account, none of these theories provide a satisfactory account of evidence on temporal contiguity from a wide range of animal studies. Alternative timing theories are arguably also unsatisfactory. (PsycINFO Database Record (c) 2014 APA, all rights reserved).
... There have been few tests of concurrent interference theory, due primarily to the difficulty in controlling potentially interfering events. One such test was reported by Lett (1973), who rewarded rats for making a correct turn in a T-maze. After making a response, whether correct or incorrect, rats were removed from the experimental apparatus and placed in their home cages for a certain period before being returned to the apparatus, where they received food. ...
... After making a response, whether correct or incorrect, rats were removed from the experimental apparatus and placed in their home cages for a certain period before being returned to the apparatus, where they received food. Lett (1973) found that response-reward delays of up to 8 minutes did not impair performance, and in subsequent studies obtained a similar result with delays of up to 20 minutes using the T-maze (Lett, 1975) and up to 1 minute in a black-white discrimination (Lett, 1974). The key factor in all of Lett's experiments was the removal of animals from the apparatus to their home cages. ...
... The general conclusion from the experiments reported in Chapter 6 is that the speed with which human participants learned to suppress an established response following a signal was inversely related to the length of trace interval, and that this effect of contiguity was at least partially mediated by the number of other events occurring during the trace interval. The Martian conditioned suppression preparation thus provides a rare test of concurrent interference theory (Revusky, 1971), demonstrating that the theory has some explanatory power in human learning as well as in non-human animals (e.g., Lett, 1973). Furthermore, transfer of performance to more difficult conditions (i.e., longer trace intervals and/or more distractors) may be explained, at least in part, by Revusky's (1971) overshadowing-plus-blocking account of Harker's (1956) maintenance results in rats; that is, whereas distractors may have overshadowed a signal-shield association with a long trace interval, prior training with a shorter trace interval established a signal-shield relationship that blocked interference from distractors in subsequent long-trace training. ...
... The elegance of Grice's work convinced most psychologists that the problem of delayed reinforcement had at last been resolved, with the result that little further research was Copyright 1979 by the American Psychological Association, Inc. 0097-7403/79/0503-0224$00.75 carried out in this area over subsequent decades (but see Lawrence & Hommel, 1961). Recently, however, Lett (1973Lett ( , 1974Lett ( , 1975 reported dramatic evidence that even with secondary reinforcement controlled, reinforcement may still be effective after delays of 1 hr. The rationale for Lett's research was based on an analysis of delayed reward proposed by Revusky (1971). ...
... The apparatus and procedure were essentially similar to those used by Lett (1973), but several modifications were introduced in order to facilitate the interpretation of the results. In Lett's experiment, subjects were placed in the central stem of a T-maze, on either side of which were transparent doors leading to black and white side arms. ...
... The apparatus used was similar to the T-maze used by Lett (1973) and had similar dimensions. The major change was the addition of a separate start box (see Figure 1). ...
Rats were trained on spatial discriminations in which reward was delayed for 1 min. Experiment 1 tested Lett's hypothesis that responses made in the home cage during the delay interval are less likely to interfere with learning than responses made in the maze. Experimental subjects were transferred to their home cages during the delay interval, and control subjects were picked up but then immediately replaced in the maze. Contrary to Lett's hypothesis, both groups learned. Further experiments suggested that handling following a choice response was the crucial variable in producing learning: No learning occurred when handling was delayed (Experiment 2) or omitted (Experiment 3). One possible explanation for the fact that handling facilitated learning is that it served to mark the preceding choice response in memory so that subjects were then more likely to recall it when subsequently reinforced. In accordance with this interpretation, learning was found to be just as strong when the choice response was followed by an intense light or noise as by handling (Experiment 4). The implication of marking for other phenomena such as avoidance, quasi-reinforcement, and the paradoxical effects of punishment is also discussed.
... They were then returned to their home cages where they remained until the next day, when they were given food only if on the previous day they had chosen the correct arm. Despite the 24-h response-reinforcement delay, the rats appeared to learn which arm to choose (Lett, 1973). In considering Lett's prepublication data, together with other research indicating long-delay learning in paradigms other than CTA, Revusky (1971, pp. ...
Taste aversion learning (aka conditioned taste aversions or CTA) can occur even when there is delay of some hours between experience of the taste and the subsequent onset of illness. This property of CTA is quite distinct from other forms of associative learning, where typically no association between two events is acquired if they are separated by more than a minute. This paper provides an overview of a series of recent experiments based on the assumption that long-delay CTA is possible only when no potentially overshadowing – or ‘concurrently interfering’ (Revusky, 1971) – events occur during the delay. The general method is one in which in a single conditioning session the rats are first given 8% sucrose, providing the sweet target taste, and 65 min later are injected with lithium chloride. What vary across experiments are the potentially interfering events occurring during the 65-min delay period. When the interfering event is a second, and quite different, taste, namely sour-tasting hydrochloric acid solution (HCl), this produces 1-trial overshadowing of the sucrose aversion, to a degree that is greater when HCl is given late in the delay period, greater when HCl is given in the same context as sucrose and greater when HCl has not been pre-exposed. Other intervening events can also overshadow sucrose aversion learning. These include placement into a novel context, as long as this occurs immediately before injection, and even stimuli that evoke memories of food-related experiences. These results can be accounted for by adding to the Rescorla-Wagner model (Rescorla & Wagner, 1972) the assumption that sickness is comprised of a succession of mini-bouts and the assumption that context-event associations (Wagner, 1981) are important in long-delay CTA.
... Revusky (1971Revusky ( , 1977 integrates these findings into what he describes as a more general associative interference theory. This theory inspired Lett (1973Lett ( , 1974Lett ( , 1975Lett ( , 1977 to demonstrate that a rat is capable of bridging a fairly large time interval between a discriminative stimulus and reinforcementand this with more conventional procedures. For this to happen, however, the situation must be designed so that the animal is urged to again "call to mind" the discriminative stimulus during reinforcement. ...
This manuscript is part of a special issue to commemorate professor Paul Eelen, who passed away on August 21, 2016. Paul was a clinically oriented scientist, for whom learning principles (Pavlovian or operant) were more than salivary responses and lever presses. His expertise in learning psychology and his enthusiasm to translate this knowledge to clinical practice inspired many inside and outside academia. Several of his original writings were in the Dutch language. Instead of editing a special issue with contributions of colleagues and friends, we decided to translate a selection of his manuscripts to English to allow wide access to his original insights and opinions. Even though the manuscripts were written more than two decades ago, their content is surprisingly contemporary. The present manuscript was originally published as part of a Liber Amicorum for Paul Eelen’s own supervisor, prof. Joseph Nuttin. In this chapter, Paul Eelen presents a modern view on Pavlovian learning. It appeared in 1980, at the heyday of cognitive psychology which initially dismissed conditioning. Paul Eelen’s perseverance in presenting learning principles as key to study human behaviour has proven correct and ahead of time.
First published as: Eelen, P. (1980). Klassieke conditionering: Klassiek en toch modern. In Liber Amicorum, Prof. J. R. Nuttin, Gedrag, dynamische relatie en betekeniswereld (pp. 321–343). Leuven: Universitaire Pers Leuven.
... Fourteen of the 16 subjects fixated on the arm to which they went when first shocked, which seems consistent with elicitation theory. Lett (1973Lett ( , 1975Lett ( , 1979 has demonstrated what appears to be long-delay learning in the T-maze, and she has used this finding to argue against traditional theories of instrumentallearning (e.g., Grice, 1948). The obvious way to account for Lett's results is to postulate that reinforcement can be effective after much longer delays than had heretofore been thought. ...
If rats in a T-maze made an arbitrarily designated correct response, that response was the terminal trial of the day. If, instead, they made the other incorrect response, they were shocked on the next trial whenever they failed to choose either arm of the maze within 3 sec. All rats quickly learned to choose one arm, but the incorrect arm was chosen as often as the correct. Thus the original hypothesis that escape from shock in conjunction with the tendency to alternate would mediate the learning of the correct response was not supported. However, the animals that learned the incorrect response had a much weaker tendency to alternate on shock trials than had the learners of the correct response. Fourteen of the 16 subjects fixated on the arm to which they went when first shocked, which seems consistent with elicitation theory.
... An outline of the present experimental paradigm corresponds to that of traditional delayed reaction experiments because the delay was between a cue and \ \ '\ ~ the opportunity to respond. In the delay of reward situation, which has traditionally been regarded as basically different from the delayed reaction situation, Lett (1973Lett ( , 1974 and Denny (1974) have used removal of the rat from the apparatus to obtain learning with delays of up to an hour. Although such results are unprecedented in traditional learning situations and are entirely contradictory to traditional empiricistic learning theories, they had been expected on the basis of my theory of long-delay learning (Revusky, 1971). ...
Rats can associate between events separated by intertrial intervals as long as several hours. For instance, they can learn that the type of food reward on one trial in a runway is correlated with whether running will be rewarded on the following trial. They also can learn that they will be rewarded if they alternate responses in a T-maze. In the otherwise similar experiments reported here, more traditional discriminative stimuli were used. Whether running down a runway was to be rewarded with sugar solution depended on whether the goalbox on the preceding trial, over 4 min ago, had been white or black. The rats learned this discrimination and retained it through an extinction phase.
... Further, it seems obvious that memories must be subject to interference outside of the apparatus if the extra-apparatus situation is at all like the experimental apparatus. This notion has recently been explored by Lett (1975) in an appetitive position response task in the T-maze (see also Lett, 1973Lett, , 1974. Lett (1975) trained rats in the T-maze under conditions of long delay of reinforceraent (up to 60 rainutes in Exp. ...
ACKNOWLEDGMENTS I wish,to acknowledge,my gratitude,to Dr. Dennis,Clark,Cogan my major,advisor,and,the,director,of this,dissertation. ,Dr. Cogan has,been,a continuing,source,of inspiration,and,guidance,throughout the,past,four,years,and,I owe most,of my,thinking,about,Psychology and,professional,activities,to him. ,Thanks,are,also,due,to Drs. Robert Bell, Bill Locke, Philip Marshall, and William Portnoy for their,service,as members,of the,committee. 11 TABLE OF CONTENTS ACKNOWLEDGMENTS,ii LIST OF TABLES ,v LIST OF FIGURES ,vi CHAPTER I. INTRODUCTION,1 The Experiment ,8 Experimental Design ,i 9 Extinction Predictions ,9 Aftereffects Position ,10
... La hipótesis de la señalización fue planteada inicialmente por Lieberman y cols. (1979) basándose en investigaciones previas de Lett (1973Lett ( , 1975. Este autor mostró que las ratas son capaces de aprender laberintos bajo condiciones de demora de reforzamiento de varios minutos, siempre y cuando se retire al sujeto del laberinto justo después de emitir la respuesta correcta (o incorrecta). ...
Different studies have showed that delay gradients are less steep when a sig- nal occurs during the delay interval. This paper reviews a number of contem- porary studies on the signaled of delay reinforcement, and includes both cogni- tively and behaviorally oriented studies. The review showed that most of the cognitive studies lack appropriate control groups; on the other hand, most of the behaviorally oriented studies lack appropriate parametric extensions.
... Spatial discrimination training generally followed the procedure employed in Experiment 1, except that (1) the animal remained in the arm of its choice for 10 rather than 60 sec, (2) after correct responses, the animal was rewarded with 20 rather than 15 raisins, which were presented in the center section of the T-maze at the choice point, with both transparent guillotine doors closed, and (3) after incorrect responses, the animal spent a 30~ec period in the center section of the T-maze with both guillotine doors closed without reward or the reward cup presen t (cf. Lett, 1973Lett, , 1975. ...
Used 2 variables to examine whether 14 monkeys were capable of learning a cue-correlated spatial discrimination in a T-maze with a 30-min delay of reinforcement: (1) the degree of preexisting preference for the negative stimulus (S–) arm and (2) the location in which Ss spent the reinforcement delay (home cage or test room). Ss that had only a mild preference for S– and spent the delay interval in their home cages acquired the spatial discrimination efficiently and maintained a high level of performance. Ss that entered training with a strong preference for S– and/or spent the delay interval in the test room either showed no sign of learning the spatial discrimination or, after giving some indication of learning, reverted to almost exclusive choice of S–. (7 ref) (PsycINFO Database Record (c) 2012 APA, all rights reserved)
... La condieion considkrada normalmente como suficiente para el aprendizaje asociativo ha sido la contigüidad entre estimulos o entre estimulos y respuestas. Aunque recientemente se ha resgistrado una fuerte polemica acerca de la necesidad de tal condición, existiendo un importante conjunt0 de datos que confirman la posibilidad del aprendizaje con demoras largas, tanto en el condicionamiento clásico (Revusky, 1971) como en el instrumental (Lett, 1973;D'Amato, Safarjan y Salmon, 1981), es aún mis interesante la cuestión de si la contiguidad garantiza por sí sola el establecimiento de la asociacion o si se precisa alguna condicion adicional. Esta cuestion ha sido dirimida preferentemente en el contexto del condicionamiento pavloviano. ...
Contends that there has been a shift from a behavioristic viewpoint to a more cognitively oriented position that views learning as a matter of new internal structures rather than responses rigidly tied to a stimulus in animal-learning studies. Conditions of learning and determinants of performance are discussed. Comments of recent critics about the assumption of a general learning process (based mainly on the phenomena of taste-aversion learning and the problems of validating laboratory research) are reviewed. The article concludes with speculations about the future development in animal-learning psychology and its role in general psychology. (French abstract) (79 ref) (PsycINFO Database Record (c) 2012 APA, all rights reserved)
... While these factors all indicate shortcomings of the classical approach, and while the data are in line with a growing body of research indicating that rats can learn appetitively reinforced responses over considerable delays, they do not in and of themselves suggest any detailed alternative. The associative interference account of Revusky (1971) and Lett (1973;) provides an accurate model for our results, but it is conceivable that an account -which must wait for further extensive development -based on the cue properties of reinforcement and nonreinforcement derived from their embeddedness in a pattern of reinforcement might also do a proper job. In any event, Hulse's (1973) concept of patterns of reinforcement, sequential patterns in particular, has demonstrated that new and useful information can be generated within the domain of long-delay learning. ...
If rats chose S+ in a brightness discrimination in a T-maze, they experienced, on that run and over four forced runs to S+
which followed, a pattern of reinforcement in which quantity of reward in the goalbox increased from 0 to 14 food pellets,
decreased from 14 to 0 food pellets, or varied randomly. If the rats erred and chose S−, reinforcement was withheld, and they
were forced a second time to 0 reward in the S− goalbox. The results indicate that rats readily learn the brightness discrimination
under these conditions, the animals exposed to the sequentially increasing pattern learning somewhat slower than the others.
This was true in spite of substantial delay of reward. Theoretical accounts based on perseverative inhibitory or facilitatory
tendencies, or upon frustration, fail to describe the data accurately. Instead, analysis of the data shows that the animals
were accurately anticipating the quantity of reward to be obtained on each run, running fast for large quantities and slowly
for small.
... Data that distinguish among these alternatives are not yet available, but if learning is involved, it is difficult to avoid the necessity of a relative contiguity between the associated events, which means that illness probably reactivates the memory of the novel taste, and the association is formed in AM between the reactivated event and the new event (Lett, 1973(Lett, , 1978. ...
Presents a brief history and review of the short-term and long-term memory distinction and concludes that this distinction is no longer adequate for either human or animal memory data. Simple memories for events are apparently formed quickly and are permanent. In such cases, an initial physiologically unstable period is not required. Thus, most forgetting is the result of a retrieval failure rather than a storage failure. A distinction between active memory (AM) and inactive memory (IM) is made. AM is a subset of IM and contains either newly formed memories or established retrieved memories or both. Implications for psychobiology of the AM and IM distinction are discussed. It is suggested, for example, that while in AM, memories are particularly open to disruption either by amnesic agents or through other forms of interference. The forgetting process for new and established memories is time dependent (but independent of memory age) and is based on interference. It is desirable to maintain the distinction between memory storage and memory retrieval even while recognizing that associative storage aids in retrieval. The search for the biological basis of rapidly forming memories, perhaps based on the restructuring of protein fragments, remains important, but the physiological brain processes underlying memory interference and retrieval require greater emphasis. (5½ p ref)
... In a classic study by Grice (1948), for example, delays of .5 sec caused a five-fold increase in the number of trials required to solve a simple visual discrimination, whereas delays of 10 sec prevented learning altogether. In a recent maze-learning experiment by Lieberman, Mclntosh, & Thomas (1979), however, presentation of a brief but salient stimulus immediately following the choice response-a flash of light, a burst of noise, or handling by the experimenter-resulted in substantial learning even when reinforcement was delayed for 2 min (see also Lett, 1973Lett, , 1975. ...
Two-choice spatial discrimination by rats is enhanced if a salient stimulus marker occurs immediately after every choice response and again after a delay interval (Lieberman, McIntosh & Thomas, 1979). Three experiments further explore this effect. Experiment 1 found that the second marker is unnecessary. Experiment 2 found that a marker presented before a response is as effective as one presented after. Both effects could be explained in terms of markers focusing attention on subsequent cues. Experiment 3, however, found that markers after choice enhance learning even when no discriminative cues are present following the marker. Markers thus appear to initiate both a backward search through memory and attention to subsequent events; both processes help to identify events that might be related to the unexpected marking stimulus.
... TPr/TP; the difference between .2 and .7 TPr/TP was only 1% and not significant (p > .2). Again activity changes were not essential for the turn de- creases. InFigure 6bition view says that the TPr reinforce an inhibition process with different properties than responses, and that view predicted the observed bias in favor of the inhibition process. Lett (1973 Lett ( , 1974 Lett ( , 1975 Lett ( , 1977) has reported T-maze conditioning with one min and longer delays of reinforcement. However, Lieberman, McIntosh, and Thomas (1979) have shown that such conditioning across long delays used special stimuli, namely a salient stimulus both after the choice response and again just before the reinfor ...
A differential-reinforcement-of-other-behavior (DRO) schedule with trials and delayed reinforcement was investigated. Periodically a wheel was briefly available to rats, followed six seconds later by brief availability of a bar. Variable-ratio food reinforcement of wheel turns was adjusted to give 95% turns. After variable-ratio-five reinforcement of bar presses produced 100% pressing, then separate ratio schedules were used for presses following turns (turn presses) and presses following nonturns (nonturn presses). Increasing nonturn-press reinforcements decreased turns, even though total reinforcements increased. Reversal by decreasing nonturn-press reinforcements raised turns, though with hysteresis. Thus food reinforcement increased nonturns even though delayed six to ten seconds after nonturns, a delay that greatly reduces response reinforcement. Those and other results indicate that the turn decrease was not due to reinforcement of competing responses. Evidence against other alternatives, and the reduction of responding by increased reinforcement, indicate that the term inhibition is appropriate for the phenomenon reinforced. Response-specific inhibition appears appropriate for this particular kind, since its effects are more specific to particular responses than Pavlovian conditioned-inhibition. Response-specific inhibition seems best considered a behavioral output comparable to responses (e.g., both reinforcible) but with important properties different from responses (e.g., different reinforcement-delay gradients).
... The hypothesis that contiguity between response and reinforcement is fundamental in learning has been criticized by authors of differing theoretical persuasions (Baum, 1973;Bloomfield, 1972;Lett, 1973;Revusky, 1971;Shimp, 1976). We cannot deal with all of these here, but Baum's (1973) position is most relevant to this discussion. ...
The performance of pigeons was studied under conditions in which the completion of a fixed-ratio requirement was not contiguous with the presentation of a reinforcer. Timein and timeout periods alternated throughout the experimental sessions. Responses made by an experimental bird during the timein period were accumulated, and when a fixed-ratio requirement had been met, grain was presented to the experimental bird and a yoked control following their first response in the next timein period. Across most manipulations of the fixed-ratio requirement and of the duration of the timeout period, the response rates of the experimental birds were considerably higher than those of their controls, suggesting that the response-reinforcer dependency controlled the behavior of the experimental bird in the absence of a close temporal association between responding on the ratio schedule and reinforcer presentations.
Reward training may be summarized by the equation
$$S:R \to S^V$$
which designates that a specified response (R) performed in a given stimulus situation (S) results in presentation of a stimulus that is currently valued by the animal (Sv). Thorndike’s (1898) experiment in which hungry cats pulled a loop in a box to gain access to food illustrates one variety of reward training in which Sv is an appetitive stimulus. Non-appetitive forms of Sv include periods of freedom from an aversive stimulus (i.e., “escape training”), certain classes of exteroceptive stimuli (e.g., a change in ambient illumination level for rats), stimulus conditions which allow certain responses to be performed (e.g., presentation of a manipulable puzzle to monkeys), direct stimulation of certain brain areas (e.g., electrical stimulation of the hypothalamus), and classically conditioned stimuli previously paired with Sv (i.e., “conditioned reinforcement” training). When the parameters of training are properly arranged, an occurrence of S:R → Sv results in increased likelihood that R will occur when the animal subsequently encounters S. It is this result which is summarized in the Empirical Law of Effect (Chapter 1).
A constraint on a phenomenon is a limitation or boundary condition for its occurrence. Scientific investigations invariably lead to the identification of such boundary conditions. Studies of learning have documented numerous constraints, including, for example, the interval between conditioned and unconditioned stimuli (CS and US), the intensity and novelty of the CS and US, and the extent to which the unconditioned stimulus is surprising. Limitations on learning brought about by factors such as the CS–US interval and CS familiarity, for example, reliably occur in numerous situations in which classical conditioning is observed. The primary impetus for discussions about biological constraints on classical and instrumental conditioning was that certain learning phenomena were contrary to principles of learning widely held at the time, and being an exception t o the rule constituted a working definition of biological constraints.
General process approaches to learning presuppose that there are learning processes governed by principles general enough to apply to many species in wide varieties of learning situations. A parade example of a general process approach in action is B. F. Skinner’s teaching-machine method. This is a technology of teaching verbal materials to humans largely based on principles derived from conditioning repetitive feeding behaviors in rats and pigeons. Two leaps of faith are implicit: (1) that one can validly extrapolate from pigeons and rats to humans; (2) that one can validly extrapolate from teaching repetitive feeding behaviors to teaching verbal materials. Although Skinner’s approach is an extreme example, most experimental psychologists during the last century have considered learning to be the study of general processes which underlie a great diversity of phenomena.
This chapter reviews the empirical and theoretical analyses of the phenomena that have taken place because of the issue of biological constraints. Recent studies of biological constraints on instrumental and classic conditioning have stimulated the modifications of previously accepted ideas about learning. They have greatly increased the range of factors that are considered to be relevant to predicting learning in various situations. However, such studies have not required abandonment of the pursuit of general theories of learning. Recent experiments discussed in the chapter illustrate that a detailed empirical investigation of a biological constraint on learning often leads to the formulation of new general principles of behavior. Although positive reinforcement is remarkably effective in increasing the probability of a variety of responses, notable exceptions to this general rule have been evident from the inception of research on instrumental conditioning. Some of the most dramatic examples of constraints on instrumental conditioning are found in avoidance learning. Depending on the kind of response that is required to prevent aversive stimulation, avoidance learning may occur very rapidly, at an intermediate rate, or very slowly.
Rats were rewarded for running down a runway if the previous trial, over 16 min earlier, had terminated in a white plastic goalbox, but were not rewarded if the trial had terminated in a larger black wooden goalbox (or vice versa). Overall, the rats learned the discrimination, albeit poorly. There were six groups of two rats each; the factors were 0-, 10-, and 30-sec delays in the startbox and whether the black or white goalbox was S+. Discrimination learning occurred under each of these conditions and, contrary to concurrent interference theory, was not noticeably affected by startbox delay.
Visual discrimination learning with a 1-min delay of reward has been demonstrated in rats with a procedure in which the delay is spent in the home cage (Lett, 1974). Two of the present experiments studied the effect of varying the length of delay on such visual discrimination learning. In Experiment 1, different groups of rats were trained with a delay of reward lasting 30, 60, 120, or 240 sec. All groups learned the discrimination. Although the group delayed for 30 sec appeared to perform better than the other groups, the main statistical analysis yielded no reliable difference in performance among groups. It seemed possible, however, that a delay gradient might be more readily detected when very short delays are studied. In Experiment 2, three groups of rats were trained with delays lasting 0, 10, or 60 sec; the results were that the 0-sec delay group performed better than the 10-sec delay group, which in turn performed better than the 60-sec delay group. Experiment 3 attempted to ascertain whether a parallel gradient would be obtained in spatial discrimination learning under the home-cage delay procedure. Three groups of rats were trained in a spatial discrimination with delays lasting 0, 10, or 60 sec. Superior performance was exhibited by the rats in the 0-sec delay condition, but there was no significant difference in performance between those delayed for 10 and 60 sec.
Evidence concerning the possibility of mental time travel into the future by animals was reviewed. Both experimental laboratory studies and field observations were considered. Paradigms for the study of future anticipation and planning included inhibition of consumption of current food contingent on future receipt of either a larger quantity or more preferred food, choice between quantities of food contingent on future pilfering or replenishment of food, carrying foods to different locations contingent on future access to those locations, and selection of tools for use to obtain food in the future. Studies of non-human primates, rats, black-capped chickadees, scrub-jays, and tayras were considered. It was concluded that current evidence favors future cognition in animals, and some theoretical issues concerning this ability were discussed.
Mongooses (Herpestes auropunctatus) were studied both in captivity and in the field on St. Thomas, the American Virgin Islands. Captive mongooses formed clear conditioned aversions to egg flavor after illness was induced by consuming a dose of carbachol in egg. Suppression of egg consumption among 32 carbachol-treated subjects was reduced to 37% of that of 10 control mongooses over 5 successive daily trials. Mongooses did not, however, avoid toxic eggs at a distance based upon the olfactory cue provided them. Rather, they attacked, tasted and then avoided eggs whether they contained carbachol or not.After consuming carbachol-laced eggs, free-ranging mongooses at 4 treated sites avoided eggs at a distance compared with those at 4 control sites. This avoidance resulted in intact survival of 72% of available eggs and extended to eggs that did not contain carbachol also. Suppression of mongoose predation upon eggs in the field was not long-lived (a total of 14 days). This may have been the result of subtle taste cues that developed in carbachol eggs allowed to remain in the field at warm temperatures. These cues may have allowed mongooses to detect the difference between carbachol eggs and equally available “safe” eggs not containing carbachol. The results of this study strongly suggest that a suitable egg bait appropriately distributed in the field could significantly reduce predation by mongooses upon the eggs of endangered species, including those of marine turtles.
Roberts's (1976) results should not be considered a failure to replicate the experiment by Lett (1974) demonstrating visual discrimination learning with a 1-min delay of reward. Although performance in Roberts's experiment was poor, the rats subjected to Lett's exact procedure showed learning by Roberts's own criterion of a significant-blocks effect. Moreover, by another criterion, the rats subjected by Roberts to three variants of Lett's procedure also exhibited learning. Thus, Roberts's results are consistent with Lett's.
Cue location has been an uncontrolled variable in food-aversion studies. While tastes are always attributes of the ingested object, visual, auditory and olfactory cues are often attributes of the food container or are located elsewhere in the conditioning chamber. A review of experimental studies indicates that cues which are attributes of the ingested object are almost invariably associated with both immediate and delayed illness, regardless of the sense modality of the cue and of the animal species involved. Cues which are attributes of the food container or conditioning chamber are associated with immediate but not delayed illness, again regardless of the sense modality and animal subject. Within the limits of present evidence, the same effects of cue location appear to occur when shock is the reinforcer. It is suggested that the association of attribute cues across delays is mediated by the conditioned behaviour, which is directed at the object of which they are attributes and which is biologically related to the subsequent reinforcement.
Four experiments examined how acquisition of a T-maze discrimination by rats with a 1-min delay of reinforcement was affected by handling (marking) following correct and incorrect choices, and by the location in which the delay-of-reinforcement interval was spent. In Experiments 1A and 1B, rats that were immediately picked up and returned to the arm just chosen showed significant acquisition of the discrimination, but so did rats that were simply left in that arm without handling. Rats that were picked up and placed in their home cages, in the arm opposite to that just chosen, or in the stem of the maze for the delay interval showed little or no acquisition of the discrimination. Experiment 2A showed that rats that were handled, returned to the arm just chosen for a 1-min delay, and then reinforced in that arm if correct learned no faster than a similar group whose choice responses were not marked. Both delay-of-reinforcement groups learned significantly slower, however, than an immediately reinforced group. Experiment 2B replicated these latter results under conditions in which visual cues were reduced or eliminated by ambient red light. Together, our findings suggest that long-delay learning in the T-maze by rats is neither ensured nor facilitated by marking per se, but that acquisition is strongly affected by the location in which the animal spends the delay interval.
Rats were trained to go to one side of a T-maze with delays of reward lasting 1, 20, or 60 min in Expt 1 and 1 or 60 min in Expt 2. Mediation by secondary reward was prevented by administering the same delay treatment regardless of whether the response was correct or incorrect: after a response, the rat was removed from the choice alley and placed in its home cage to spend the delay. Feedback for the response was given in the startbox after the delay interval ended. The rats learned and there were no significant differences in performance among groups trained with different delays. These results had been expected on the basis of Revusky's (1971) hypothesis that removal of the rat from the learning situation to spend the delay elsewhere facilitates long delay learning by reducing associative interference. In Expt 3, this notion was tested explicitly by varying the amount of a 2-min delay to be spent in the experimental situation. Different groups of rats were left in the choice alley after the response for 0, 15, or 60 sec; then the rats were removed to spend the remainder of the 2-min delay in the home cage As predicted, the level of performance decreased as the length of time in the choice alley was increased.
Two experiments were carried out in an attempt to replicate Lett's (1974) demonstration of visual discrimination learning with a 1-min delay of reward. In Experiment 1, controls for olfactory cues were introduced by testing animals in different orders from day to day and by reversing the positions of the discrimination chambers within days; also, secondary reinforcement effects were precluded by the use of opaque rather than transparent doors to the discrimination chambers. No evidence of learning was found either in a group of rats returned to the home cage during the delay or in a group left in the apparatus during the delay. It was hypothesized that the failure to find improvement in Experiment 1 might have resulted from either the controls for odor cues or from a lack of memory retrieval cues provided by the apparatus. In Experiment 2, both of these hypotheses were tested and neither was supported. The Lett group in Experiment 2 was run according to the exact procedures used by Lett and still no learning was demonstrated. It was concluded that further demonstrations of visual discrimination learning with a 1-min delay of reward will be necessary for the acceptance of the validity of this phenomenon.
In two experiments rats were trained in a Morris pool to find a hidden platform in the presence of a single landmark. Circular black curtains surrounded the pool, with the single landmark inside this enclosure, so that no other room cues could provide additional information about the location of the platform. This landmark was hung from a false ceiling and rotated from trial to trial, and the position of the platform also changed on each trial, thus preserving a constant relation between the platform and the landmark. In Experiment 1, the position of the landmark was exactly above the hidden platform in Group Above and was relatively close to the hidden platform in Group Near. At the end of acquisition, test trials without the platform revealed a difference between the groups. Although a preference for searching in the correct quadrant of the pool, where the platform should have been, was found in both groups, this preference was significantly higher for Group Above. In Experiment 2, new rats in Group Near were compared to rats for which the position of the landmark was relatively far from the hidden platform in Group Far. Test trials revealed a preference for searching in the correct quadrant of the pool in both groups, but this preference was significantly higher for Group Near. The implication of these results is that the control acquired by a single landmark is different depending on its relative distance from a hidden platform: Closer landmarks acquire better control than distant ones. These results show a clear parallelism in comparison with the effect of absolute temporal proximity of the CS to the US in classical conditioning.
Hooded rats were given conditioned inhibition training in which the taste of saccharin alone was always followed by induced illness, but the taste of saccharin plus the odor of amyl acetate was not. In a series of three subsequent tests—summation, enhancement of conditioning, and retardation—it was demonstrated that the odor had acquired active inhibitory properties. The results paralleled those obtained with more traditionally studied stimuli and techniques and hence were found to be readily predictable from a recent model of conditioning set forth by Rescorla and Wagner (1972).
Eight rats were successfully trained in a black-white discrimination with a 1-min delay of reward. The procedure was unusual in that the rat spent the delay outside the apparatus in its home cage. Immediately after the rat responded, whether correctly or incorrectly, it was removed from the choice compartment and placed in its home cage. When the delay ended, it was returned to the startbox. If the preceding response had been correct, the rat received a reward of sugar water; otherwise, it was allowed to make another choice response. Mediation by external cues was excluded because there was no difference in the way the rats were treated after a correct or an incorrect response until the delay interval ended. Mediation by proprioceptive stimuli was excluded because position was an irrelevant cue.
Rats were trained on a passive avoidance conditioning in which an US (a brief electric shock of 1.5 sec) was given to them after a step down response, with a delay of 30 sec. A salient stimulus was presented the first day of training according to a response procedure (response /salient stimulus), or a stimulus procedure (apparatus cues / salient stimulus), after which the delay started and then the US. In Experiment 1, with an electric shock of 0.15 mA, results were not stadistically significant. In Experiment 2 we repeated the same procedure as in Experiment 1 but with an electric shock of 0.20 mA. This time the salient stimulus (a white noise of 100 dB/1.5 sec) presented on the first day after the subjects’ response, did not produced any effect either, contrary to what we expected according to Lieberman et al (1979) backward search hypothesis. On the other hand, the salient stimulus presented according to a stimulus procedure produced a retardation on the subjects’ passive avoidance responses on subsequent days, contrary to our predictions based on Thomas et al (1983) arousal and attention hypothesis. These last results are discussed in terms of Wagner’s rehearsal model.
Diversos estudios han demostrado que los efectos decrementales de separar la
respuesta procuradora del reforzador pueden diluirse si una señal exteroceptiva
ocurre durante el intervalo de demora. En este trabajo se presenta una revisión
de experimentos contemporáneos, tanto de orientación conductual como cognitiva,
sobre los efectos de la demora señalada. Las principales conclusiones de la
revisión fueron que los estudios de orientación cognitiva carecen de los grupos
de control apropiados, y, por otro lado, que los estudios de orientación conductual
carecen de las extensiones paramétricas necesarias para evaluar adecuadamente
las diferentes teorías desarrolladas para explicar el fenómeno.
In four experiments using 312 adult albino rats, locomotor avoidance and suppression of drinking were used to monitor learned associations between spatial cues and injections of poison or saline. Rats injected with hypertonic LiCl or hypertonic NaCl immediately upon removal from one side of a two-compartment shuttlebox free of ingestible substances later showed locomotor avoidance of that side. Other animals that were identically trained and later offered novel fluids in either the compartment paired with injection or the compartment paired with noninjection did not differentially suppress drinking. When injections were isotonic, the NaCl-injected animals displayed neither ingestive nor locomotor aversions while the LiCl-injected animals showed a place-specific reluctance to drink before any locomotor avoidance appeared. When injections were administered 4 min after removal from the treatment-paired compartment, LiCl-injected rats developed discriminated suppression of drinking more rapidly than locomotor position preferences whereas NaCl-injected rats displayed no acquisition. Animals injected with LiCl or NaCl 4 min before being placed into the treatment-paired compartment showed no significant acquisition by either ingestive or locomotor measure. These interactions between type of test, nature and concentration of injected agent, and time of injection, not explicable in terms of different response thresholds for a single association, are interpreted as the consequence of hypertonically induced somatic pain associations formed at short CS—US intervals being prepotent to affect locomotor activity, and toxin-induced visceral pain associations formed at short or long CS—US intervals being prepotent to affect ingestive behavior. When both were potentially present with short interstimulus intervals, acute somatic pain apparently overshadowed visceral pain. Some implications of prepotent response systems for the manifestation of information about specific USs are considered in the light of recent failures of the stimulus equivalence hypothesis.
This study examined the effect of lesions of the hippocampus in rats and delay of reinforcement (0 or 10 sec) on the acquisition of a spatial response in a Y-maze. The results indicated that the acquisition of the response is markedly retarded in animals with lesions of the hippocampus when a 10-sec reinforcement delay is used. The results are taken to support the notion of increased distractibility in the lesioned animals.
In the present series of studies 16- and 21-day-old rats acquired an aversion to a black compartment (CS+) paired with footshock, only if given preconditioning exposure to a white chamber (CS-) not paired with footshock. The strength of conditioning was inversely related to the length of the retention interval in both age groups. However, the rate of forgetting was accelerated in the 16-day-old subjects, indicating infantile amnesia for the conditioned aversion. Strength of conditioning was also inversely related to the length of the CS-/CS+ interval in both age groups, indicating that expression of the aversion is critically dependent on an active memory of the CS- at the time of conditioning. Additionally, there was an ontogenetic decrease in the rate of forgetting of the CS-, indicating infantile amnesia for elements of the conditioning episode that did not predict occurrence of US.
Two experiments are reported in which responding for sensory change was studied in a situation in which regular sensory changes also occurred in the absence of responding. In Experiment 1 these regular changes depended on responses being withheld for a specified period but in Experiment 2 they occurred noncontingently. In both experiments responses produced immediate simultaneous light and sound changes. Finding from an earlier experiment in which a quite different procedure was used involving a choice between a single light change or a single sound change were confirmed. However in neither experiment was amount of responding related to the frequency of the regular, noncontingent changes. This finding was interpreted as evidence against the notion that sensory changes as such, are reinforcing. It is suggested instead that responding for sensory change is a form of behavior which in maintained because exerting control over the environment is intrinsically motivating.
In regulating the internal homeostatic environment mammals, by necessity, employ behavioral strategies that differ from the tactics used in coping with contingencies in the external environment. When an animal consumes a meal, the palatability of that meal is automatically adjusted in accordance with the ultimate internal effects of that meal. If the meal causes toxicosis, the animal acquires an aversion for the taste of the meal; conversely, if recuperation follows ingestion of the meal, the taste of that meal is enhanced. Unlike the learning that occurs when externally referred visual and auditory signals are followed by punishment in the form of peripheral pain or reward in the form of food in the mouth, conditioning to the homeostatic effects of food can occur in a single trial and rarely requires more than three to five trials, even though the ultimate effects of the meal are delayed for hours. Paradoxically, the animal need not be aware of the ultimate internal effect in the same sense that it is aware of external contingencies. For example, an aversion can be acquired even if the animal is unconscious when the agent of illness is administered. Thus, the way in which food-effects are stored in memory may be fundamentally different from the way in which memories of specific time-space strategies devised for external contingencies are stored. This separation of function is indicated by limbic lesions which disrupt conditioning to a buzzer that is followed by shock and facilitate conditioning to a taste that is followed by illness.
Operationally speaking, one can describe both aversion conditioning and buzzer-shock conditioning in the space-time associationistic terms of classical conditioning. However, psychologically speaking, one must realize that in aversion conditioning the animal does not act as if it were acquiring an "if-then" strategy. It acts as if a hedonic shift, or a change in the incentive value of the flavor were taking place. Such hedonic shifts are critical in regulation of the internal milieu. When an animal is in need of calories, food tends to be more palatable; as the caloric deficit is restored, food becomes less palatable. If the animal's body temperature is below optimum, a warm stimulus applied to the skin is pleasant. When body temperature is too high, the converse is true. In this way, homeostatic states monitored by internal receptors produce changes in the incentive values of external stimuli sensed by the peripheral receptors, and guide feeding behavior.
In mammals at least, the gustatory system, which provides sensory control of feeding, sends fibers to the nucleus solitarius. This brainstem relay station also receives fibers from the viscera and the internal monitors of the area postrema. Ascending fibers bifurcate at the level of the pons and project toward the feeding areas of the hypothalamus and the cortex. The olfactory system which primarily projects to the limbic system does not play a primary role in adjusting food incentives. Rather, it plays a secondary role in the activation of feeding, as do other external sensory systems.
This specialized conditioning mechanism, which specifically adjusts gustatory hedonic values through delayed visceral feedback, is widespread among animals, including man and rat. These two species are remarkably similar in their thresholds and preferences for gustatory stimuli. The behavioral similarities are based on the animals' having similar gustatory systems, similar convergence of gustatory and internal afferents to the nucleus solitarius, and similar midbrain regulatory mechanisms. Thus, it is not surprising that the feeding of obese rats with internal hypothalamic damage resembles the feeding of obese human beings insensitive to the internal signs of this caloric state. Obviously, man has a highly specialized form of symbolic communication and the rat does not, yet man's cognitive specialization does not prevent him from developing aversions to food consumed before illness even when he knows that his illness was not caused by food.
Rats on an eight-arm radial maze chose between four arms on which a small reward could be obtained after a short delay and four arms on which a larger reward could be obtained after a longer delay. Experiments 1 and 2 showed that rats preferred the long-delay, large-reward arms over the short-delay, small-reward arms. This preference was particularly marked when the arms were made into enclosed alleys. Experiment 3 showed that this effect was not produced by a preference for staying in enclosed alleys. We argue that the rats endured longer delays to obtain larger rewards when fear of predation was minimized.
A review of recent evidence indicates that contingent reinforcement is neither a necessary nor a sufficient condition for operant learning. This dilemma reopens the old question of "what is learned." It is proposed that what laboratory Ss characteristically learn is not a response to a stimulus, but rather 2 kinds of expectancies. 1 kind of expectancy corresponds rather accurately with environmental stimulus-outcome contingencies, while the 2nd is a less faithful representation of response-outcome contingencies. The reinforcement procedure merely permits both expectancies to be learned. (36 ref.) (PsycINFO Database Record (c) 2012 APA, all rights reserved)
Studied discrimination learning in 18 male Wistar albino rats using differential reinforcers as discriminative stimuli in both a runway and a -maze. In the initial experiment, Ss were administered a sequence of 3 runway trial outcomes repeated indefinitely: either milk (M), pellets (P), and nonreinforcement (N) or P, M, and N. A 15-min intertrial interval (ITI) separated each of the 9 daily trials. Results support the prediction that Ss would learn to run more slowly on N trials than on either M or P trials. In the 2nd experiment, Ss were trained to run to the left side of a -maze after an M trial and to the right side after a P trial or vice versa. An ITI of 3 min. was employed. Results indicate that each of the 6 Ss gradually learned to run to the correct side more frequently than the chance level of 50%. Results are interpreted as consistent with E. J. Capaldi's sequential hypothesis. (PsycINFO Database Record (c) 2012 APA, all rights reserved)
"Rats were trained on a one-unit T maze to perform a temporal alternation with 15 min. between trials. The animals alternated significantly above chance even when the delays between trials were increased to several hours . . .. Traditional S-R mechanisms do not appear capable of explaining this behavior. Therefore, it seems reasonable to suppose that rats can remember where they went last, and that this memory serves as the cue for the correct response." (PsycINFO Database Record (c) 2006 APA, all rights reserved).
An audiovisual stimulus was made contingent upon the rat’s licking at the water spout, thus making it analogous with a gustatory stimulus. When the audiovisual stimulus and the gustatory stimulus were paired with electric shock the avoidance reactions transferred to the audiovisual stimulus, but not the gustatory stimulus. Conversely, when both stimuli were paired with toxin or x-ray the avoidance reactions transferred to the gustatory stimulus, but not the audiovisual stimulus. Apparently stimuli are selected as cues dependent upon the nature of the subsequent reinforcer.
Note is made of the change from Hull's original goal gradient hypothesis to the goal gradient which is shown to be a secondary principle derivable from more basic principles. The main portion of this report is concerned with this new formulation and suggestions for certain modifications of it. The author rejects the interpretation that learning under conditions of delay of primary reward involves a backward action of the goal object on the preceding stimulus-response event. As an alternative to this conception it is suggested that "all such learning occurs as the result of the development of secondary reinforcement, the action of which is conceived to take place immediately upon the occurrence of the response. A prominent aspect of this theory is the concept of
differential secondary reinforcement based either on cues from the external environment or internal cues such as proprioceptive stimulation." 19 references. (PsycINFO Database Record (c) 2012 APA, all rights reserved)
This chapter presents the data which support the contention that stimulus selection is a potent effect, even in experimental situations that allow considerable control over the schedules of stimulation and reinforcement to which S is exposed. Yet, it is questioned whether or not such data demand an “attentional” interpretation. A theoretical alternative suggested “modified continuity theory” is discussed, and new data is presented for which this approach appears to offer a relatively unique account. Although at this time the theory must be regarded as especially tentative and incomplete, opportunity will be taken to suggest its potential usefulness in interpreting several phenomena that have been in search of theoretical integration. There are a number of common experimental procedures for evaluating the degree of focusing or stimulus selection. Perhaps the most obvious involves a comparison of the degree of learning exhibited to some stimulus when it is the only relevant cue available, as compared to a condition in which there are additional relevant cues. The degree to which the presence of additional relevant cues reduces the apparent amount learned is taken to indicate the degree of stimulus selection.
This chapter discusses the sequential mode of theorizing along with several reward variables, such as magnitude of reward, delay of reward, and nonreward. Sequential theorizing is found to be very helpful in selective learning. This may also explain many phenomena not deducible from other hypotheses. Moreover, a generalization decrement interpretation sequential theorizing has been shown to be applicable to a wide variety of phenomena, currently explained in diverse terms, such as associational and motivational. Sequential and nonsequential theories normally emphasize different classes of independent variables. Considering partial reward (PR), for example, a nonsequential theory such as the dissonance hypothesis suggests that resistance to extinction (Rn) is regulated by total number of nonrewards regardless of their sequence or order. Where the dissonance hypothesis sees the sequence of nonrewards as irrelevant, the sequential hypothesis sees number of nonrewards as irrelevant.
The literature of specific hungers and bait shyness contains much indirect evidence of learning involving prolonged delay of reinforcement. Specific hunger refers to the selective feeding by animals as they learn to correct a specific dietary deficiency, such as thiamine deficiency, while bait shyness describes the rejection of poisoned baits by animals, which have survived a previous poisoning attempt. It seems unlikely that learning can take place at all with delays of more than a few seconds. Instances of learning with protracted delays of reinforcement are always cases where immediate secondary reinforcement occurs. It is generalized that delayed reinforcement is not effective except under elaborate training conditions. However, when the response is ingestion and the rewards or punishments are changes in the physiological state of the organism, this generalization appears to be incorrect. Most specific hungers can be explained in terms of learned associations involving delayed aftereffects; an exception is the specific hunger for sodium, which appears to be largely innate. If rats are subjected to sodium deficiency and recover from it by drinking salt water, they tend to drink an abnormally large amount of salt water after the deficiency is relieved.
Albino rats were given a series of forced-choice trials in a T maze under a number of specific conditions, and were then tested for alternation behavior. The forced trials were conducted in such a manner as to separately control differential stimulus and response variables. Alternation was significantly above chance when differential response variables were absent, but was at a chance level in the absence of differential stimulus variables. If a series of test trials are given, the alternation tendency rapidly extinguishes. "It was concluded that alternation following forced trials does not depend upon response-associated variables, but is a function of repetitive exposure to particular stimulus conditions."
Considers learning and memory within an adaptive-evolutionary framework, using an analysis of the role of learning in thiamine specific hunger. The demands of the environment on the rat, the contingencies in the natural environment, the importance of the novelty-familiarity dimension, and the realization of 2 new principles of learning, permit a learning explanation of most specific hungers. The 2 new principles, "belongingness" and "long-delay learning," specifically meet the peculiar demands of learning in the feeding system. An attempt is made to develop the laws of taste-aversion learning. It is argued that the laws or mechanism of learning are adapted to deal with particular types of problems and can be fully understood only in a naturalistic context. The "laws" of learning in the feeding system need not be the same as those in other systems. Speculations are presented concerning the evolution and development of learning abilities and cognitive function. It is concluded that full understanding of learning and memory involves explanation of their diversity and the extraction of common general principles. (108 ref.)
Two cues, either size or flavor of food pellet, were conditionally paired with either malaise induced by x-ray or pain induced by shock in four groups of rats. The combination of flavor and illness produced as conditioned decrement in consumption, but that of size and illness did not. Conversely, the combination of size and pain produced an inhibition of eating, but flavor and pain did not. Apparently, effective associative learning depends on central neural convergence of the paired afferent input.
Selective association and conditioning
- Kamin