No full-text available
To read the full-text of this research,
you can request a copy directly from the author.
The relation of secondary reward to gradients of reinforcement
... The shortening of the gradient of reinforcement as a result of greater control over secondary reinforcement led Spence (1947) to question the very existence of a primary gradient, suggesting instead that all learning with delayed reinforcement involved immediate secondary reinforcement. Tentative empirical support for this idea was provided by Perkins (1947) and Grice (1948), who sought to reduce the impact of secondary reinforcement even further than Perin. Perkins (1947) employed a T-maze with delay chambers, similar to that used by Wolfe (1934), and aimed to remove the influence of external cues in two ways. ...
... Second, the maze was rotated 180 degrees each session to prevent the use of cues external to the apparatus. Perkins (1947) found that the rate of learning decreased as delay increased, with the most dramatic difference in performance between rats experiencing delays of 15 seconds or less and those with 45 or 120 seconds. Furthermore, performance of rats in the 45-second condition was inferior to that of a group experiencing a 45-second delay but with distinctive delay boxes, one of which always followed a correct response, thus providing an external cue. ...
... Although Perkins (1947) attempted to control for potential external sources of secondary reinforcement, he noted that rats may utilise proprioceptive feedback in order to bridge delays; that is, a proprioceptive trace of a particular response may remain in the nervous system until food is obtained, and so this trace may acquire secondary reinforcing properties. Grice (1948) sought to remove the influence of such internal secondary reinforcement by employing what he described as a visual, rather than a spatial, discrimination task. ...
... Spence's Treatment of Delay. Perkins (1947), using a T maze, made an attempt to eliminate as many cues as possible by enclosing the maze, interchanging the two delay boxes from one side to another resulting in each delay box being followed by reward half of the time, and rotating the maze 180 degrees at the start of each day. Perkins (1947) found a limit of 120 seconds delay on the ability of the rats to solve the problem. ...
... Perkins (1947), using a T maze, made an attempt to eliminate as many cues as possible by enclosing the maze, interchanging the two delay boxes from one side to another resulting in each delay box being followed by reward half of the time, and rotating the maze 180 degrees at the start of each day. Perkins (1947) found a limit of 120 seconds delay on the ability of the rats to solve the problem. The performance of the experimental subjects who had the delay chambers interchanged was poorer than the performance of a control group for whom the delay chambers were not interchanged, that is, for the control subjects the same delay chamber was always followed by reward. ...
... Presumably, slight differences in the delay chambers were sufficient to provide distinctive cues for differential learning, via secondary reinforcement, of the choice task. It was the data of Perkins (1947), Riesen (1940), andGuide (1941;cited in Grice, 1948) that led Spence (1947) to suggest that: it would not seem unreasonable to hypothesize that there is no primary gradient of reinforcement but that all learning involving delay of primary reward results from the action of immediate secondary reinforcement which develops in the situation .... Such a hypothesis eliminates the necessity of explaining how reward seemingly acts backward over time to influence something which occurred earlier [p. ...
The independent variable of delay of reinforcement is traced from its early empirical history through the systematic treatment provided by Hull and Spence. The empirical findings and theoretical implications of recent experimental studies are noted. The effects of delay of reward and delay of punishment on the learning process are considered, the comparative studies with human Ss are reported, and the relevance of delay of reinforcement to abnormal behavior and personality is summarized. (103 ref.) (PsycINFO Database Record (c) 2012 APA, all rights reserved)
... It has been established that typically response strength declines as a monotonic function of temporal delay of reinforcement following occurrence of a response (Wolie, 1934;Perin, 1943), and that delay of secondary reinforcement produces a similar effect (Perkins, 1947). Kamin (1957aKamin ( , 1957b has shown that delay of CS termination in an instrumental avoidance learning task significantly retards the occurrence of an initial avoidance response and the frequency of avoidance responses, the effect being a monotonic function of delay. ...
... The data satisfactorily demonstrate an empirical gradient effect of delay USC termination on escape learning grossly similar to those obtained underconditions of delay of food reward (e.g. Wolfe, 1934;Perkins, 1947) and delay of CS termination in avoidance learning (Kamin, 1957a(Kamin, , 1957b. However, the present data suggest a much greater inhibitory effect on running speed of very brief days of reinforcement, where BLOCK of TEN TRIALS reinforcement consists of termination of shock, than has characteristically been found with brief delays of primary positive reinforcement or secondary aver-"live stimuli. ...
A 3 (levels of shock) by 4 (delays of shock termination) design investigated effects of delay of reinforcement in escape conditioning in 60 rats in a 2-compartment shuttle box. Different delay gradients were obtained under different levels of shock intensity (p<.01), with all delay groups showing marked inhibition of performance when compared to non-delay groups (p <.01). The significant drive level (controlled by shock intensity) by delay of reinforcement interaction is inconsistent with results of investigations varying degree of food deprivation and delay of food reward (Ramond, 1954; Renner, 1963), and suggest that Spence’s (1956) theory of delay of reinforcement in appetitive learning cannot be generalized to aversive conditioning.
... It has been found in a variety of experimental tasks that performance becomes poorer when the temporal delay in reinforcement is increased (Grice, 1948;Perin, 1943aPerin, , 1943bPerkins, 1947;Ramond, 1954;Warden & Haas, 1927;Wolfe, 1934). ...
... A second finding has been that goal related cues occurring during the delay increase the extent of the temporal gradient of reinforcement (Grice, 1948;Perin, 1943aPerin, , 1943bPerkins, 1947). The acquired secondary reinforcing capacity of such cues frequently has been assumed, but rarely (Peterson, 1956) has an independent demonstration been attempted. ...
A factorial design, using 3 intervals of delay, 2 levels of food deprivation, and presence or absence of goal related stimuli in the delay chamber, was employed to study acquisition and extinction performance of rats in a straight alley runway. The well established temporal gradient of reinforcement was obtained; this gradient was affected by drive level and availability of goal box cues, with drive level and delay combining additively, and with the cues facilitating performance only when reinforcement was delayed. High drive animals extinguished faster than low drive animals. The acquisition and extinction performance of the animals was discussed in terms of delay of reward as a frustration. (PsycINFO Database Record (c) 2012 APA, all rights reserved)
... Subsequently, experimenters tried to eliminate all sources of response-.. contingent cues from their experiments (e.g., Grice, 1948;Perkins, 1947). Grice (1948), for example, designed a unique visual discrimination maze in which were two alleys, one painted black and the other white, whose positions were alternated randomly over trials from left to right. ...
p>It has long been known that the impairment to discrimination learning caused by brief delays to reinforcement can be counteracted by the response-contingent presentation of a conditioned reinforcer during the delay interval following a correct response (Spence, 1947). More recently, it has been shown that reinforcement delay can also be overcome using response-marking procedures, in which the same stimulus contingently follows both correct responses and errors (e.g. Lieberman, McIntosh & Thomas, 1979). This thesis examined the effects of response-marking procedures on human learning of conditional discrimination tasks with delayed reinforcement.
Experiments One to Three employed single case experimental designs (alternating treatments) to evaluate the effect of response marking during matching-to-sample tasks with delayed reinforcement, using children with autism as participants. Experiment One showed that for both marking and conditioned reinforcement supported acquisition of conditional discrimination performance over a 5 s delay, although the latter appeared more efficient. Experiment Two, however, showed that – with more effective techniques – both procedures were equally effective, and that both were more effective than a control in which no response-contingent stimuli occurred during the delay. Experiment Three compared the standard marking procedure with a novel marked-before procedure in which all sample stimuli were marked before a matching response was made. Both procedures produced very similar acquisition rates, and both were more effective in establishing conditional discrimination than a delay only control.
Experiments four to Seven employed group comparison designs to compare marking against conditioned reinforcement, delay and immediate reinforcement using adult humans in a laboratory version of the matching-to-sample task. Marking effects were found only in Experiment Seven, when the confounding effects of verbal behaviour were adequately controlled.
Overall, the findings indicated that response-marking procedures may be effective with human participants but that their effects are more reliable in applied settings with children than in laboratory settings with adults.</p
... Put another way, How far forward in time can an animal learn to anticipate the consequence of its response? Although some early studies have suggested that reward could be delayed for 30 s or more and still lead to learning (Perin, 1943;Perkins, 1947;Watson, 1917;Wolfe, 1934), these studies were plagued with the confounding effects of secondary reinforcement. That is, cues present at the time a reinforcer was consumed were also present immediately after the target response and thus could have acted as a source of immediate reinforcement. ...
People can time travel cognitively because they can remember events having occurred at particular times in the past (episodic memory) and because they can anticipate new events occurring at particular times in the future. The ability to assign points in time to events arises from human development of a sense of time and its accompanying time-keeping technology. The hypothesis is advanced that animals are cognitively stuck in time; that is, they have no sense of time and thus have no episodic memory or ability to anticipate long-range future events. Research on animals' abilities to detect time of day, track short time intervals, remember the order of a sequence of events, and anticipate future events are considered, and it is concluded that the stuck-in-time hypothesis is largely supported by the current evidence.
... As compared with immediate reinforcement, delay of reinforcement leads to inferior performance in the instrumental conditioning situation with animals. For example, the results of studies by Harker (1956), Perin (1943), Perkins (1947), and Ramond (1954) indicated that in general, delay of reinforcement had a detrimental effect on response strength. ...
108 kindergarten children served as Ss in lever-pulling experiment. They were divided into 3 incentive groups (Group High, Medium and Low) for “token” training in which incentive values were given according to the incentive conditions. Each of the 3 groups was subdivided into 2 additional groups of immediate (Group IM) and delayed (10 see) reinforcement (Group D). In each trial, measures of starting and movement time of lever pulling response were taken. For both measures Ss of Groups IM responded faster than those of Groups D. In Groups IM, the higher the incentive, the greater was the movement speed. The data were interpreted in terms of the competing response hypothesis and motivational factor suggested by Spence (1956). © 1975, The Japanese Psychological Association. All rights reserved.
... Much of the literat ure on delay of reinforeement has stressed the importance of the availability of cues for mediating the delay. Studies with animals have shown that delay box cues (Perkins, 1947) and response-produced cues (Grice, 1948) may serve as sources of mediation. When these cues are eliminated, extremely steep delay gradients are obtained. ...
This experiment was designed to study the effects of delay of reinforcement, meaningfulness, and level of retardation (borderline or mild) on performance in a successive discrimination learning task. Ss from both IQ groups were required to learn a left or right motor response to each of six pictures, three meaningful and three nonmeaningful. For a given S, each stimulus from both meaningfulness categories was paired with a different delay interval, 0, 3, or 9 sec. The three main effects proved reliable, but none of the interactions reached significance.
... In 1947 Spence questioned the very existence of a primary gradient, suggesting instead that all learning with delayed reinforcement involved immediate secondary reinforcement. Tentative experimental support for this idea was provided by Perkins (1947) followed by more definitive results from Grice (1948). Grice (1948) employed a black-white discrimination box, in which rats were required to select one of two parallel alleys, one white and the other black, and only one led to food. ...
The greater the separation in time between 2 events, A followed by B, the less likely they are to become associated. The dominant explanation of this temporal contiguity effect has been trace decay: During the interval between A and B, the trace left by A becomes too weak by the time B occurs for an association to be formed between them. Pavlov adopted this idea in the context of classical conditioning and Hull used it to account for the deleterious effect of delaying reinforcement on the acquisition of instrumental responses. By 1960 various studies supported the conclusion that animals could not learn to associate 2 events separated by more than around 45 s. Research on human skill acquisition with delayed feedback and later studies using causal or predictive judgment tasks indicated that explicit cognitive processing is generally involved when humans associate events separated by more than a few seconds. The discovery of long-delay taste aversion learning prompted Revusky's (1971) alternative analysis of contiguity effects in terms of interference: The greater the separation between A and B, the more likely that extraneous events compete for association with A and B. Although the analysis of overshadowing provided by associative learning theories provides a context for this account, none of these theories provide a satisfactory account of evidence on temporal contiguity from a wide range of animal studies. Alternative timing theories are arguably also unsatisfactory. (PsycINFO Database Record (c) 2014 APA, all rights reserved).
... Reinforcement is delayed by interposing an interval between a response and the reinforcement for that response. The delay is found to weaken responding: rates of responding are lower than when reinforcement immediately follows a response, pauses are longer, new responses and discriminations take more time to learn, and a delayed reinforcement is less likely to be chosen than an immediate one (Skinner, 1938, p. 139 ff; Perin, 1943; Perkins, 1947; Grice, 1948; Chung, 1965; Smith, 1967). ...
When interreinforcement intervals were equated, pigeons demonstrated little or no preference between reinforcement after a delay interval and reinforcement presented on a fixed-interval schedule. The small preferences sometimes found for the fixed interval (a) were considerably smaller than when the delay and fixed intervals differed in duration, and (b) were caused by the absence of light during the delay. These results suggest that the effects of delayed reinforcement on prior responding can be reproduced by imposing a temporally equal fixed-interval schedule in place of the delay; and, therefore, that the time between a response and reinforcement controls the probability of that response, whether other responses intervene or not.
... Aside from its function as a delay as such, the use of a delay may accentuate such "preferences". Where the response (or discrimination) was solely positional (right or left) and thus free of such confounding relations, Perkins (1947) found that a correct choice was acquired when the delay following either the reinforced or nonreinforced choice was as long as 2 min. Undoubtedly similar differences hold when a delay is imposed on an established performance. ...
Pigeons were shown to come under discrimination control when the SD and SΔ were temporally separated from reinforcement and non-reinforcement. SD and SΔ consisted of distinctive key illuminations presented separately. Responding on an FR 5 in the presence of SD or SΔ produced a third stimulus containing a schedule requirement. If this third (or interpolated) stimulus was preceded by SD, responding in its presence produced reinforcement followed by a time-out (TO). If, on the other hand, the third stimulus was preceded by SΔ, responding produced TO alone. In this fashion, the same stimulus and the same response requirement were imposed between SD and the reinforcement as between SΔ and the TO. In Experiment I, the schedule employed during the interpolated stimulus was FR; in Experiment II, FI. Discrimination reversal was accomplished in both experiments.
... De acuerdo con este concepto, las diferentes señales exteroceptivas del recorrido en el laberinto se han asociado con alimento (el reforzador primario); así, la conducta del sujeto se mantiene durante el camino debido a que va siendo re-324 forzada de forma inmediata por reforzadores condicionados. En una serie de elegantes experimentos, Perkins (1947) evaluó la hipótesis de Spence midiendo la ejecución de roedores en laberintos rotados. Al rotar la orientación del laberinto de un ensayo a otro, las pistas y señales luminosas que podían funcionar como reforzadores condicionados cambiaban también. ...
Different studies have showed that delay gradients are less steep when a sig- nal occurs during the delay interval. This paper reviews a number of contem- porary studies on the signaled of delay reinforcement, and includes both cogni- tively and behaviorally oriented studies. The review showed that most of the cognitive studies lack appropriate control groups; on the other hand, most of the behaviorally oriented studies lack appropriate parametric extensions.
... However, they might mean that, for some reason, short alleys depress learning for small-reward groups. Thus, the large effects of delay as manipulated by alley length, indicating a somewhat steeper gradient than that found by Wolfe (1934) and Perkins (1947), suggest that the delays caused by reward magnitudes can largely account for the consistent rate of learning differences in spatial tasks. To give us,further pause, if one compares columns and rows in Tables 1 and 3, it is apparent that a given difference in delay, whether caused by reward magnitude or by alley length manipulations, will "produce" roughly the same difference in learning rate. ...
To evaluate B. H. Pubols' (see 35:5) suggestion that magnitude of reward effects in rat discrimination learning might be due to inadvertent differences in delay of reward, delay was manipulated in a 2 * 2 * 3 design, employing 3 alley lengths and 2 reward levels for both brightness and spatial tasks. Ss were 144 male albino rats. Reward magnitude significantly affected speed of learning for both tasks. Alley length affected speed of learning only in the spatial task, demonstrating that delay cannot account for reward-magnitude effects in brightness discriminations. The role of delay in the spatial task was ambiguous. (PsycINFO Database Record (c) 2012 APA, all rights reserved)
... The third issue is the shape of the delay of reinforcement function in the region of 0-1 sec. Previous parametric studies (Grice, 1948;Perin, 1943;Perkins, 1947;Skinner, 1938;Wolfe, 1934) utilized compound primary and secondary reinforcers consisting of food or water plus a click or other stimulus which signaled availability of the S R . Thus the complete reinforcing event was a temporally extended affair lasting from the onset of the magazine stimulus at least until the apprehension of the food or water by the animal-by measurements reported below a period on the order of at least 1.6 sec. ...
Rats were first trained to make a magazine response to a brief discriminative stimulus. This stimulus was then made contingent on a previously untrained bar-press response in the absence of other reinforcement. Delays varying for different groups of Ss in 8 logarithmic steps from 0.1 to 12.8 sec. were interposed between the response and the stimulus. A "primary" delay gradient, i.e., one not attributable to stimulus generalization, was demonstrated by data on latency of the first response following a single reinforcement. A nonmonotonic relation of response rate to delay with an optimum interval of approximately 0.8 sec. was obtained with continuous reinforcement. (PsycINFO Database Record (c) 2012 APA, all rights reserved)
... Since performance is enhanced by the presence of secondary cues, Spence (1947) assumed that all learning under delay of reward stems from the immediate action of secondary reinforcement. For example, studies by Perkins (1947), and especially Grice (1948), showed that when distinctive goal cues are randomized, performance for a delayed reward is poor. Such early experiments (e.g., Grice, 1948) represent an attempt to eliminate the effects of secondary cues. ...
Gave 60 shock-escape trials to naive female albino Sprague-Dawley rats in each of 2 experiments (N = 87 each). In Exp. I, shock offset was delayed either 0, 1.5, 3, or 6 sec. following the lever-press escape response. Presentation of a neutral cue during the entire delay interval greatly facilitated performance relative to groups receiving no cue. In Exp. II, the position of a 1-sec cue, within a 3-sec delay interval, was varied. Facilitation of escape performance was greatest when the cue occurred immediately after the response or just prior to shock offset but significantly less in other positions. It is suggested that maximum facilitation of delayed-reward learning obtains when stimuli are contiguous with the critical events of the delay interval. (PsycINFO Database Record (c) 2012 APA, all rights reserved)
... The empirical relationship between delay of reinforcement and acquisition of instrumental responses in rats has been well established. In a variety of learning situations, the typical finding is that acquisition is a negatively accelerated, decreasing function of the length of delay in reinforcement (5,12,13,25). However, there is another functional relationship involving delay of reinforcement that has until recently been relatively neglected in research. ...
Using male albino rats in a one-unit Y-maze found that the number of days to the discrimination learning criterion was a negatively accelerated, increasing function of delay of reward during acquisition. Response perseveration was an increasing function of delay during acquisition. Discussion in terms of acquisition. Discussion in terms of acquisition as the adapting out of competing responses, and extinction the de-adapting of adapted competing responses. 25 references. (PsycINFO Database Record (c) 2012 APA, all rights reserved)
... In this situation some early experimenters found that animaKshowed undiminished speed of learning and response strength with reward delays up to 5 min, as long as some kind of cue predicting reward occurred immediately (e.g., Warden & Haas, 1927 ). This finding was generally not confirmed by later research (e.g., Perkins, 1947; Wolfe, 1934), although Ferster (1953) found that pigeons which have already learned a response for immediate reward continue to emit it at an undiminished rate if delay of reward is gradually increased to 1 min. There are many different estimates of how rapidly the effect of reward declines. ...
In a choice among assured, familiar outcomes of behavior, impulsiveness is the choice of less rewarding over more rewarding alternatives. Discussions of impulsiveness in the literature of economics, sociology, social psychology, dynamic psychology and psychiatry, behavioral psychology, and "behavior therapy" are reviewed. Impulsiveness seems to be best accounted for by the hyperbolic curves that have been found to describe the decline in effectiveness of rewards as the rewards are delayed from the time of choice. Such curves predict a reliable change of choice between some alternative rewards as a function of time. This change of choice provides a rationale for the known kinds of impulse control and relates them to several hitherto perplexing phenomena: behavioral rigidity, time-out from positive reinforcement, willpower, self-reward, compulsive traits, projection, boredom, and the capacity of punishing stimuli to attract attention. (31/2 p ref)
... Put another way, How far forward in time can an animal learn to anticipate the consequence of its response? Although some early studies have suggested that reward could be delayed for 30 s or more and still lead to learning (Perin, 1943; Perkins, 1947; Watson, 1917; Wolfe, 1934), these studies were plagued with the confounding effects of secondary reinforcement. That is, cues present at the time a reinforcer was consumed were also present immediately after the target response and thus could have acted as a source of immediate reinforcement. ...
People can time travel cognitively because they can remember events having occurred at particular times in the past (episodic memory) and because they can anticipate new events occurring at particular times in the future. The ability to assign points in time to events arises from human development of a sense of time and its accompanying time-keeping technology. The hypothesis is advanced that animals are cognitively stuck in time: that is, they have no sense of time and thus have no episodic memory or ability to anticipate long-range future events. Research on animals' abilities to detect time of day, track short time intervals, remember the order of a sequence of events, and anticipate future events are considered, and it is concluded that the stuck-in-time hypothesis is largely supported by the current evidence.
... Skinner (16) in 1936 demonstrated the strengthening of an S-R connection when it was followed by an established S D . Other investigators have verified and extended Skinner's basic finding (1,2,11,12,13). But there is a surprising lack of studies investigating the effects on learning of a stimulus S A associated with nonreinforcement. ...
To test the hypothesis that a negative discriminatory stimulus has inhibitory functions analogous to the reinforcing properties of a positive discriminative stimulus, two experimental groups of rats were given preliminary training in an alley, associated with a buzzer for either reinforced or non-reinforced trials; control groups did not receive the alley-buzzer training. All groups were trained to criterion on a black-white discrimination problem with 5 sec. delay of reward. During the delay buzzer-sounds were given in association with either correct and reinforced trials or incorrect and non-reinforced trials. Both of the groups that had previous alley experience with the buzzer were superior to their controls, and control groups with buzzer were superior to the non-buzzer control group. It is concluded that "both negative and positive discriminative functions, when established, carry over to the black-white discriminative problem and become immediately effective, thus indicating that the phenomenon of secondary inhibition does exist." 17 references. (PsycINFO Database Record (c) 2012 APA, all rights reserved)
... Another interpretation would be to maintain the initial conclusion that rate of acquisition is independent of incentive magnitude, attributing positive results to artifacts or confounding factors. It has been demonstrated many times (e.g., Perkins, 1947;Pubols, 1958) that rate of selective learning is inversely related to delay of reinforcement. It is suggested that the finding of a positive relationship between incentive magnitude and rate of learning in the present and other studies is actually due to the possibility that those iSs experiencing larger magnitudes were also responding faster and, hence, experiencing shorter delays of reinforcement. ...
Incentive magnitude was varied factorially such that each of 2 values (1 or 4 45-mg. food pellets) during the acquisition of a Y - maze position habit by white rats was paired with the same 2 values during subsequent reversal of the habit. Measures of both level of performance and rate of learning were found to be affected by contemporary, but not prior, incentive magnitude. Although the results imply that incentive magnitude affects rate of acquisition as well as level of performance, the suggestion is made that the obtained differences in rate of learning may be attributable to differences in delay, rather than amount, of reinforcement. (PsycINFO Database Record (c) 2012 APA, all rights reserved)
... This interpretation probably derives in part from early research showing that one of the most important determinants of a primary reinforcer's effectiveness is the delay between the response and its presentation, such that even short delays can have a profound effect (e.g., Grice, 1948). Additionally, early research showed that the impact of delay could be reduced by delivering a responsecontingent cue, such as a tone or a light, during the delay interval (e.g., Perkins, 1947; Wolfe, 1934). Such cues were said to function as conditioned reinforcers because of their association with subsequent primary reinforcement (Hull, 1943). ...
Three children with autism were taught to identify pictures of emotions in response to their spoken names. Their speed of acquisition was compared using a within-child alternating treatments design across three teaching conditions, each involving a 5 second delay to reinforcement. In the marked-before condition, an instruction encouraged the children to visually orient to the cards before they made their choice response; in the marked-after condition, an attention-eliciting verbal cue (e.g., "Look!") was delivered after both correct and incorrect responses; in the delay condition, these marking cues were omitted. Performance in the no-cue control was inferior to both the marked-before and marked-after conditions, but the difference between the latter two conditions was not significant.
... The goal-gradient hypothesis is one famous example. In fact, in those experiments in which delayed and immediate reinforcement generated equivalent control, the control by delayed reinforcement was attributed to secondary reinforcement that counteracted the detrimental behavioral effects of delaying the reinforcer (e.g., Hull, 1943; Kimble, 1961; Lawrence & Hommel, 1961; Perin, 1943a, 1943b; Perkins, 1947; Wolfe, 1934). As a basis for his theoretical analysis of impulsiveness and impulse control, Ainslie (1975) stated that "delaying rewards from the moment of choice causes them to lose effectiveness according to a highly concave function of that delay" (p. ...
In previous studies of delayed reinforcement, response rate has been found to vary inversely with the response-reinforcer interval. However, in all of these studies the independent variable, response-reinforcer time, was confounded with the number of reinforcers presented in a fixed period of time (reinforcer frequency). In the present study, the frequency of available reinforcers was held constant, while temporal separation between response and reinforcer was independently manipulated. A repeating time cycle, T, was divided into two alternating time periods, t(D) and t(Delta). The first response in t(D) was reinforced at the end of the prevailing T cycle and extinction prevailed in t(Delta). Two placements for t(D) were defined, an early t(D) placement in which t(D) precedes t(Delta) and a late t(D) placement in which t(D) follows t(Delta). The duration of the early and late t(D) was systematically decreased from 30 seconds (i.e., t(D) = T) to 0.1 second. Manipulation of t(D) placement and duration controlled the temporal separation between response and reinforcement, but it did not affect the frequency of programmed reinforcers, which was 1/T. The results show that early and late t(D) placements of equal duration have similar overall effects upon response rate, reinforcer frequency, responses per reinforcer, and obtained response-reinforcer temporal separation. A stepwise regression analysis using log response rate as the dependent variable showed that the obtained delay was a significant first-step variable for six of eight subjects, with obtained reinforcer frequency significant for the remaining two subjects.
... 151-153). In some of these studies , the effectiveness of reinforcement has been shown to depend not only on the delay val but also on the presence or absence of an external stimulus intervening between the response and the delayed reinforcer (Grice, 1948; Perkins, 1947). Presumably, as outlined by Spence (1947), these external stimuli, acting as conditioned reinforcers, provide the animal with a means of "bridging" the temporal gap between the response and the primary reinforcer , thus increasing the effectiveness of that reinforcer. ...
Six rats were trained on a two-component multiple schedule with each component consisting of a two-link chain schedule. Differential response suppression in the two initial links, as well as in the two terminal links of the chain schedules, was used as a measure of the relative aversiveness of stimulus events in the two terminal links. When signalled and unsignalled shock-punishment (in addition to equal numbers of food reinforcers) were scheduled in the separate terminal links, subjects responded at lower rates in the initial link preceding unsignalled shock-punishment than in the initial link preceding signalled shock-punishment. Similarly, subjects responded at lower rates in the terminal link containing unsignalled shock-punishment than in the terminal link containing signalled shock-punishment. Reversing the terminal-link positions of signalled and unsignalled shock-punishment led to a reversal of the differential response suppression in the two initial and the two terminal links of the chain schedules. These results indicate that signalled punishment is relatively less aversive than unsignalled punishment and support an "information hypothesis", which assumes that a condition in which information is provided about the onset of environmental events, even negative events such as shock punishment, is more reinforcing than a condition in which such information is absent.
Un propósito del estudio fue determinar si correlacionando cada componente de 4 minutos de un programa múltiple con demoras de reforzamiento de 0, 2, 4 y 8 s se podría obtener un gradiente de demora de reforzamiento dentro de una misma sesión. en cada componente se entregó el reforzador conforme a un programa tándem compuesto de un intervalo al azar que sumado a cada demora nominal resultó en un intervalo entre reforzadores (ieR) constante de 32 o de 128 s. se asignaron tres palomas a cada ieR. las tasa de respuesta en cada componente del programa múltiple disminuyó gradualmente confor-me se alargó la demora de reforzamiento. un segundo propósito del estudio fue determinar el punto en un continuo definido por la probabilidad de señalar el periodo de demora (p(señal)) en el cual los efectos de frecuencia de reforzamiento se convierten en efectos de tiempo relativo en un procedimiento de demora de reforzamiento. La p(señal) fue de 0.00, 0.33, 0.66 o 1.00. Cuando la p(señal) fue de 0.00. las tasas de respuesta ante cualquier demora fueron más altas con el IER de 32 s que con 128 s, mostrando un efecto de frecuencia de reforzamiento. Cuando se incrementó el valor de p(señal) de 0.33 a 1.00 las tasas de respuesta para cualquier demora fueron más altas con el IERR de 128 s que con 32 s, mostrando un efecto de tiempo relativo.
It is the presumption, if not the dictum, of investigators concerned with “basic” psychological research that relatively focused or circumscribed study of a particular problem will often uncover a general principle of behavior. This chapter seeks to illuminate such an outgrowth from the history and current extension of research on “shock-right” facilitation, one of several seemingly paradoxical effects of punishment (see Fowler, 197lb). The phenomenon in question was first detected and theoretically elaborated by Karl Muenzinger (1934). He reported that rats were facilitated in learning a visual discrimination when electric shock was administered for the correct, food-reinforced (“right”) response, and virtually as well as when the same intensity of shock was administered to other animals for the incorrect, non-food-reinforced response (“shock wrong”). The generality of the shock-right facilitation effect did not go uncontested, however, and it is to George Wischner’s credit that the phenomenon was first empirically delimited (Wischner, 1947) and then subsequently elaborated in a systematic program of research that he and the author conducted (e.g., Fowler and Wischner, 1969).
There are different schools of thought on the current uneasy relations between theory and practice in clinical psychology. Theoreticians such as Eysenck (1981) take a procedural point of view and argue that clinicians are not rigorous enough in applying learning theory procedures. Clinicians (e.g., Cullen, 1983) argue that the problems are substantive, that everyday behavior unlike laboratory behavior is a complex business requiring multi- or even megavariate control.
This chapter deals with the effects of the delay of reward or the time interval separating response from reward. Although some of the more important animal studies are reviewed, the emphasis is on research with human subjects (Ss), particularly children. Research dealing with individual differences in the ability to delay gratification of desires is also reviewed. It is believed that an analysis of the characteristics of those who are able to postpone reinforcement in anticipation of long-range rewards may yield useful information about the mechanisms involved in learning under delayed reinforcement conditions. A review of animal studies is also included. Experimental research with children is reviewed and experiments employing adults as Ss are examined. Cross-cultural investigations of children's preferences for immediate and delayed reward are also reviewed. The results of studies at the human level, especially with children, are much more contradictory and difficult to interpret. This is, in part, because of the fact that the experimental conditions prevailing in the experiments with human Ss differ sharply. Finally, the chapter discusses theoretical considerations and conclusions relating to the research reviewed.
Three intra-runway delay durations (0, 15, and 45 sec.) were combined factorially with three training levels (0, 15, and 75 trials) to determine the effect of the level of training at the time of delay introduction on performance. Three models, each with different predictions concerning the effect of training on performance after delay introduction, were evaluated. First, the frustration-drive model was unable to account for either the decremental effects of delay, or the fact that high training groups were more affected by delay and recovered more slowly. Second, delay of reinforcement theory while predicting decremental effects was unable to account for their temporal course and spatial distribution in the runway. Especially detrimental to this theory was complete recovery from the effects of delay in spite of continued delay. Third, the novelty-reactions model, while failing to account for the specific nature of the decremental effects of delay and training level on performance, was able to handle the fact that decrements did take place and recovery did occur and was less rapid for high training groups. Of the three models this latter was considered to be most compatible with the results.
College students (72 male, 72 female) expected that parents would be more likely to follow children's behaving as requested with contact comfort, smiling, and pleasant tone of voice than children's not behaving as requested. If students' expectations reflect reality, then the results, in combination with previous ones, suggest that parents instrumentally condition children to behave appropriately in part by following children's appropriate behaviors with conditioned reinforcers (smiling, pleasant tone of voice) and then a primary reinforcer (contact comfort). This reinforcement situation should be particularly conducive to both the acquisition of appropriate behaviors and the continued maintenance of these behaviors in the face of long delays of contact comfort.
Stable DRL performance of four rats was studied under conditions of delayed reinforcement. A DRL 20 sec requirement remained in effect across all sessions while the delay of reinforcement following a correct response was varied from 8 sec to 28 sec in different blocks of sessions. For delays shorter than the DRL 20 sec requirement, it was found that response rate was not affected by delay of reinforcement, each S responding optimally for the schedule. However, for reinforcement delays longer than the DRL requirement, each S’s response rate decreased. This resulted in performance that was less than optimal for the schedule.
Six rats were trained on a two-component multiple schedule with each component consisting of a two-link chain schedule. Differential response suppression in the two initial links, as well as in the two terminal links of the chain schedules, was used as a measure of the relative aversiveness of stimulus events in the two terminal links. When signalled and unsignalled shock-punishment (in addition to equal numbers of food reinforcers) were scheduled in the separate terminal links, subjects responded at lower rates in the initial link preceding unsignalled shock-punishment than in the initial link preceding signalled shock-punishment. Similarly, subjects responded at lower rates in the terminal link containing unsignalled shock-punishment than in the terminal link containing signalled shock-punishment. Reversing the terminal-link positions of signalled and unsignalled shock-punishment led to a reversal of the differential response suppression in the two initial and the two terminal links of the chain schedules. These results indicate that signalled punishment is relatively less aversive than unsignalled punishment and support an “information hypothesis”, which assumes that a condition in which information is provided about the onset of environmental events, even negative events such as shock punishment, is more reinforcing than a condition in which such information is absent.
Roberts's (1976) results should not be considered a failure to replicate the experiment by Lett (1974) demonstrating visual discrimination learning with a 1-min delay of reward. Although performance in Roberts's experiment was poor, the rats subjected to Lett's exact procedure showed learning by Roberts's own criterion of a significant-blocks effect. Moreover, by another criterion, the rats subjected by Roberts to three variants of Lett's procedure also exhibited learning. Thus, Roberts's results are consistent with Lett's.
Four experiments examined how acquisition of a T-maze discrimination by rats with a 1-min delay of reinforcement was affected by handling (marking) following correct and incorrect choices, and by the location in which the delay-of-reinforcement interval was spent. In Experiments 1A and 1B, rats that were immediately picked up and returned to the arm just chosen showed significant acquisition of the discrimination, but so did rats that were simply left in that arm without handling. Rats that were picked up and placed in their home cages, in the arm opposite to that just chosen, or in the stem of the maze for the delay interval showed little or no acquisition of the discrimination. Experiment 2A showed that rats that were handled, returned to the arm just chosen for a 1-min delay, and then reinforced in that arm if correct learned no faster than a similar group whose choice responses were not marked. Both delay-of-reinforcement groups learned significantly slower, however, than an immediately reinforced group. Experiment 2B replicated these latter results under conditions in which visual cues were reduced or eliminated by ambient red light. Together, our findings suggest that long-delay learning in the T-maze by rats is neither ensured nor facilitated by marking per se, but that acquisition is strongly affected by the location in which the animal spends the delay interval.
Without the aid of secondary rewards to bridge the temporal gap, each of 15 rats learned to select the rewarded side of a T-maze although the reward was delayed until 1 min after the response was emitted. Similar results were obtained from another group of eight rats for which the length of the delay was 5 min. In a final experiment using the same basic procedure, five groups of rats were trained for 25 days with delays of 0.5, 1.0, 2.0, 4.0, or 8.0 min. The percentage of correct responses did not significantly differ among groups. According to prevailing psychological theory, these results are impossible.
The relationship between the duration of stimuli and their conditioned reinforcing effect was investigated using a learning-tests
procedure. In Experiment 1, stimuli were the same duration on training (stimulus → reward) and test (choice response → stimulus).
Ten- and 30-sec stimuli provided effective differential conditioned reinforcement but 3-sec stimuli did not. In Experiment
2, different pigeons had each combination of the 3- and 30-sec stimuli on training and test trials. Evidence of conditioned
reinforcement was obtained only for the birds with 30-sec stimuli on both training and test. The results were interpreted
as indicating that stimuli become effective conditioned reinforcers on test trials only when their duration exceeds the duration
of differential short-term memory cues resulting from a difference in the events that precede them on training and test trials.
If rats chose S+ in a brightness discrimination in a T-maze, they experienced, on that run and over four forced runs to S+
which followed, a pattern of reinforcement in which quantity of reward in the goalbox increased from 0 to 14 food pellets,
decreased from 14 to 0 food pellets, or varied randomly. If the rats erred and chose S−, reinforcement was withheld, and they
were forced a second time to 0 reward in the S− goalbox. The results indicate that rats readily learn the brightness discrimination
under these conditions, the animals exposed to the sequentially increasing pattern learning somewhat slower than the others.
This was true in spite of substantial delay of reward. Theoretical accounts based on perseverative inhibitory or facilitatory
tendencies, or upon frustration, fail to describe the data accurately. Instead, analysis of the data shows that the animals
were accurately anticipating the quantity of reward to be obtained on each run, running fast for large quantities and slowly
for small.
Two experiments were carried out in an attempt to replicate Lett's (1974) demonstration of visual discrimination learning with a 1-min delay of reward. In Experiment 1, controls for olfactory cues were introduced by testing animals in different orders from day to day and by reversing the positions of the discrimination chambers within days; also, secondary reinforcement effects were precluded by the use of opaque rather than transparent doors to the discrimination chambers. No evidence of learning was found either in a group of rats returned to the home cage during the delay or in a group left in the apparatus during the delay. It was hypothesized that the failure to find improvement in Experiment 1 might have resulted from either the controls for odor cues or from a lack of memory retrieval cues provided by the apparatus. In Experiment 2, both of these hypotheses were tested and neither was supported. The Lett group in Experiment 2 was run according to the exact procedures used by Lett and still no learning was demonstrated. It was concluded that further demonstrations of visual discrimination learning with a 1-min delay of reward will be necessary for the acceptance of the validity of this phenomenon.
Eight rats were successfully trained in a black-white discrimination with a 1-min delay of reward. The procedure was unusual in that the rat spent the delay outside the apparatus in its home cage. Immediately after the rat responded, whether correctly or incorrectly, it was removed from the choice compartment and placed in its home cage. When the delay ended, it was returned to the startbox. If the preceding response had been correct, the rat received a reward of sugar water; otherwise, it was allowed to make another choice response. Mediation by external cues was excluded because there was no difference in the way the rats were treated after a correct or an incorrect response until the delay interval ended. Mediation by proprioceptive stimuli was excluded because position was an irrelevant cue.
One purpose of this study was to determine if by correlating each component of a multiple schedule with delays-of-reinforcement of either 0, 2, 4, or 8 s, a within-session delay-of-reinforcement gradient could be obtained. Reinforcement was delivered on each component using a tandem schedule composed by a random interval that added to each nominal delay yielded a constant interreinforcement interval (IRI) of either 32 or 128 s. Three pigeons were assigned to each IRI. Response rates in each component of the multiple schedules decreased gradually as the reinforcement delay was lengthened. A second purpose was to determine the point on a continuum defined by the probability of signaling the delay period p(signal) at which frequency-of-reinforcement effects become relative-time effects in a reinforcement-delay procedure. The p(signal) was either 0.00, 0.33, 0,66 or 1.00. When p(signal) was 0.00, global response rates for any given delay were higher with the 32 s than with the 128 s IRI, showing a frequency-of-reinforcement effect. When p(signal) was increased from 0.33 to 1.00 global response rates for any given delay were higher with the 128 s than the 32 s IRI, showing a relative-time effect.
Thesis (M.A.)--Texas Tech University, 1971. Includes bibliographical references.
Diversos estudios han demostrado que los efectos decrementales de separar la
respuesta procuradora del reforzador pueden diluirse si una señal exteroceptiva
ocurre durante el intervalo de demora. En este trabajo se presenta una revisión
de experimentos contemporáneos, tanto de orientación conductual como cognitiva,
sobre los efectos de la demora señalada. Las principales conclusiones de la
revisión fueron que los estudios de orientación cognitiva carecen de los grupos
de control apropiados, y, por otro lado, que los estudios de orientación conductual
carecen de las extensiones paramétricas necesarias para evaluar adecuadamente
las diferentes teorías desarrolladas para explicar el fenómeno.
This study examined the effectiveness of teaching pelvic floor exercises with use of bladder-sphincter biofeedback compared to training with verbal feedback based on vaginal palpation in 24 women with stress urinary incontinence. Verbal feedback training consisted of instructing the patient to squeeze the vaginal muscles around the examiner's fingers and providing her with verbal performance feedback. Biofeedback patients received visual feedback of bladder pressure, abdominal (rectal) pressure, and external anal sphincter activity. The biofeedback group improved the strength and selective control of pelvic floor muscles; the verbal feedback group did not. Both groups significantly reduced the frequency of incontinence. The biofeedback group averaged 75.9% reduction in incontinence, significantly greater than the 51.0% reduction shown by the verbal feedback group. Twelve of 13 patients in the biofeedback group improved by 60% or better. Six patients in the verbal feedback group improved by 68% or better, and five were less than 30% improved.
To see if the neural representation of the conditioned stimulus (CS) is available to old-age rats beyond the time it is available to young adults, the intensity of the unconditioned stimulus (US) and the length of the CS-US interval were systematically varied in a trace conditioning experiment. Results indicated that increasing US intensity extends the interval over which trace conditioning is evident in old-age rats but not in young adults, suggesting that trace decay occurs more rapidly in young rats. Results were interpreted in terms of age differences in the workings of hypothesized biochemical timing mechanisms that may directly influence the ability to associate stimuli over trace intervals in conditioned taste-aversion procedures.
Tested "implications of the assumption that duration of the goal response affects performance only by producing differential secondary reinforcing properties of sight of the goal object and associated stimuli… . concluded that duration of the goal response determines the reinforcing properties of sight of the goal object and that strength of reinforcement depends on the reinforcing properties of sight of the goal object and is independent of the duration of the goal response on that particular trial." (PsycINFO Database Record (c) 2012 APA, all rights reserved)
A study of experimental extinction in 32 albino rats in a simple T-maze, with black and white food compartments, reveals no differential effects of secondary reinforcement as between the primary drives of hunger and thirst.
ResearchGate has not been able to resolve any references for this publication.