University of Greifswald
Question
Asked 10th Jan, 2022
How to test the significance of the difference between time points?
I have a dataset, assoicated with 5 time points. For each time point, I have three replicates. I understand that for a time-series dataset, repeated-measures ANOVA or Friedman test would be the best solution.
However, the three replicates are totally random. Let's say, I measured the gas flux from three random locations in a small region (20*20 square meters) at each time point. Even though the measurement was done in the same region, I can't relate these random replicates as three sets.
What I did was using independent ANOVA and using time points as the groups. But I know this is inappropriate. Anyone knows if there is an appropriate method for dealing with this situation?
Most recent answer
Sal Mangiafico Thank you so much. This is really helpful! I will try these suggestions.
All Answers (8)
Hasan Kalyoncu University
I could recommend you to Multilevel Regression / Hierarchic linear modeling. It is the best way to deal with repeated measures. Your first level is your measurement regardless of time, and the second level is your location. You could set up a model to test is there any variance. Then you could find out explained variance on each level. So you could understand the relation between measurements and locations. You could use R, HLM, Mlwin, Mplus. I recommend the HLM software because it is the easiest way.
1 Recommendation
Kent State University
Download these notes from Marie Davidian website.. she is a pioneer in this stuff. Best wishes David Booth
1 Recommendation
Rutgers, The State University of New Jersey
Your design is a little unclear to me. Is it the case that for each time point you are sampling one area ? So that the whole experiment is all conducted on one area ?
University of Greifswald
Sal Mangiafico I actually sampled two locations. For each loacation, I took three replicates (randomly). So I would like to see if there is a significant temporal change in each location.
Rutgers, The State University of New Jersey
Friedman's test won't work because you have replicated data in each "cell" of observation. You just wouldn't be able to fit the data into a Friedman test format.
Next question: Are you interested in the results for the specific areas? I mean, do you want to be able to say "Area A responded like this, while Area B responded like this." ? Or are they just two random areas that you could have picked some other random areas and it wouldn't matter to your conclusions ? And it's only the effect of Time that you are interested in.
Hopefully that makes sense, because it matters to how you might analyze the data.
University of Greifswald
Sal Mangiafico Thanks for the following answer. Yes, I'm only intrested in the effect of time, regardless of the location effect. Like you said "they just two random areas that you could have picked some other random areas and it wouldn't matter to your conclusions".
That's why I used Kruskal-Wallis test, using time as the factor. But I guess this is not correct right, because the time points are not independent? I haven't found another way to meet my aim here.
I have also considered Friedman test, by taking into consideration the locations, and use the mean values of the three replicates. But in that case, I only have two sites. I guess this number is not enough for the test, right? I think n=3 is the minimum number for doing that test.
Rutgers, The State University of New Jersey
Haitao Wang , thanks, that is helpful. Normally with this design you would use repeated measures anova or a mixed effects (hierarchical) model. You have one fixed effect independent variable, Time. And you have one random effect independent variable, Area. Area is like your "subject" or "experimental unit" that you are making repeated measures on.
However, because you have only two Areas, it's not real helpful to use Area in your model as a random effect. Usually you need at least, I don't know, 5 or 6 levels for a random effect *. Likewise, you're not going to be able to do anything too useful with the correlation structure in a repeated measures anova.
The upshot is that it probably makes the most sense to treat Area as a block, as a fixed effect, and run the analysis as a regular two-way anova (or a one-way anova with blocks). Or you can include the Time x Area interaction, which is then a two-way factorial anova. You have 2 levels of Area, 5 levels of Time, and 3 replicates for each of these "cells". In R notation, your model is simply:
Y ~ Area + Time
or
Y ~ Area + Time + Area:Time
Calling each Area a "block" may make the analysis acceptable to your reviewers.
If you or your reviewers object to not taking into account the repeated measures nature of your design, then I would recommend the mixed-effects model. Treating Area as a random effect will probably cause no harm**. In R your model is:
Y ~ Time + (1|Area)
or
Y ~ Time, random = ~1|Area
_______
*
- https://stats.stackexchange.com/questions/37647/what-is-the-minimum-recommended-number-of-groups-for-a-random-effects-factor
- https://www.biorxiv.org/content/10.1101/2021.04.11.439357v2.full.pdf , page 6.
**
1 Recommendation
University of Greifswald
Sal Mangiafico Thank you so much. This is really helpful! I will try these suggestions.
Similar questions and discussions
Related Publications
A computer program is given that assists in analyzing data from a replicated randomized complete block experimental design. One worked example is given.
Traditional ANOVA models are often successfully employed in environmental fields. However, there is a need to carefully examine the basic design, the assumptions and the form of the analysis so that valid tests of significance and standard errors are used. Examples will show that wrong conclusions can be drawn if this is not done. Environmental stu...
lem in performing ANOVA tests in the non-randomized designs. The problem is due to the effects of the nuisance parameters, which may be excessive and yet not under our control. To be more specific, as will be shown in the sequel, the conditional distribution of the F-like ratio test statistics, given the fractional replicate chosen, is like that of...