[R-pkg-team] Bug#921938: lme4 breaks r-cran-lmertest autopkgtest

Rune Christensen Rune at ChristensenStatistics.dk
Mon Feb 11 16:21:54 GMT 2019


Dear all,

This is currently being fixed and a new version of lmerTest  is on its way to CRAN, cf. https://cran.r-project.org/package=lmerTest <https://cran.r-project.org/package=lmerTest>

The latest version of lme4 was released to CRAN without our knowledge and that version caused lmerTest to fail its tests on most platforms (mostly because the new default optimiser is much less accurate than the old default optimiser, cf. https://github.com/lme4/lme4/issues/501 <https://github.com/lme4/lme4/issues/501>). After being notified of the problems we have updated lmerTest as soon as possible. 

BR
Rune

> On 11 Feb 2019, at 07.39, Per B. Brockhoff <perbb at dtu.dk> wrote:
> 
> Sendt fra min iPhone
> 
> Start på videresendt besked:
> 
>> Fra: Andreas Tille <tille at debian.org <mailto:tille at debian.org>>
>> Dato: 10. februar 2019 kl. 16.07.12 CET
>> Til: Paul Gevers <elbrus at debian.org <mailto:elbrus at debian.org>>, <921938 at bugs.debian.org <mailto:921938 at bugs.debian.org>>, "Per Bruun Brockhoff" <perbb at dtu.dk <mailto:perbb at dtu.dk>>
>> Emne: Vedr.:⁨ Bug#921938: lme4 breaks r-cran-lmertest autopkgtest⁩
>> 
>> Hi again,
>> 
>> may be you get all the relevant information directly from CRAN:
>> 
>>   https://cloud.r-project.org/web/checks/check_results_lmerTest.html <https://cloud.r-project.org/web/checks/check_results_lmerTest.html>
>> 
>> Kind regards
>> 
>>      Andreas.
>> 
>> On Sun, Feb 10, 2019 at 03:44:55PM +0100, Andreas Tille wrote:
>>> Control: tags -1 help upstream
>>> Control: forwarded -1 Per Bruun Brockhoff <perbb at dtu.dk <mailto:perbb at dtu.dk>>
>>> 
>>> Hi Per,
>>> 
>>> the Debian packaged version of lmerTest seems to show issues with the
>>> latest version of lme4.  From your insight into lmerTest can you give
>>> some hint about possible reasons for this incompatibility?
>>> 
>>> Kind regards
>>> 
>>>        Andreas.
>>> 
>>> PS: Please let me know whether you need further information.
>>> 
>>> On Sun, Feb 10, 2019 at 01:35:53PM +0100, Paul Gevers wrote:
>>>> Source: lme4, r-cran-lmertest
>>>> Control: found -1 lme4/1.1-20-1
>>>> Control: found -1 r-cran-lmertest/3.0-1-2
>>>> Severity: important
>>>> X-Debbugs-CC: debian-ci at lists.debian.org <mailto:debian-ci at lists.debian.org>
>>>> User: debian-ci at lists.debian.org <mailto:debian-ci at lists.debian.org>
>>>> Usertags: breaks needs-update
>>>> 
>>>> Dear maintainers,
>>>> 
>>>> With a recent upload of lme4 the autopkgtest of r-cran-lmertest fails in
>>>> testing when that autopkgtest is run with the binary packages of lme4
>>>> from unstable. It passes when run with only packages from testing. In
>>>> tabular form:
>>>>                       pass            fail
>>>> lme4                   from testing    1.1-20-1
>>>> r-cran-lmertest        from testing    3.0-1-2
>>>> all others             from testing    from testing
>>>> 
>>>> I copied some of the output at the bottom of this report.
>>>> 
>>>> Currently this regression is blocking the migration of lme4 to testing
>>>> [1]. Due to the nature of this issue, I filed this bug report against
>>>> both packages. Can you please investigate the situation and reassign the
>>>> bug to the right package? If needed, please change the bug's severity.
>>>> 
>>>> More information about this bug and the reason for filing it can be found on
>>>> https://wiki.debian.org/ContinuousIntegration/RegressionEmailInformation <https://wiki.debian.org/ContinuousIntegration/RegressionEmailInformation>
>>>> 
>>>> Paul
>>>> 
>>>> [1] https://qa.debian.org/excuses.php?package=lme4 <https://qa.debian.org/excuses.php?package=lme4>
>>>> 
>>>> https://ci.debian.net/data/autopkgtest/testing/amd64/r/r-cran-lmertest/1896879/log.gz <https://ci.debian.net/data/autopkgtest/testing/amd64/r/r-cran-lmertest/1896879/log.gz>
>>>> 
>>>> autopkgtest [04:42:34]: test run-unit-test: [-----------------------
>>>> BEGIN TEST test_a_utils.R
>>>> 
>>>> R version 3.5.2 (2018-12-20) -- "Eggshell Igloo"
>>>> Copyright (C) 2018 The R Foundation for Statistical Computing
>>>> Platform: x86_64-pc-linux-gnu (64-bit)
>>>> 
>>>> R is free software and comes with ABSOLUTELY NO WARRANTY.
>>>> You are welcome to redistribute it under certain conditions.
>>>> Type 'license()' or 'licence()' for distribution details.
>>>> 
>>>> R is a collaborative project with many contributors.
>>>> Type 'contributors()' for more information and
>>>> 'citation()' on how to cite R or R packages in publications.
>>>> 
>>>> Type 'demo()' for some demos, 'help()' for on-line help, or
>>>> 'help.start()' for an HTML browser interface to help.
>>>> Type 'q()' to quit R.
>>>> 
>>>>> # test_a_utils.R
>>>>> 
>>>>> library(lmerTest)
>>>> Loading required package: lme4
>>>> Loading required package: Matrix
>>>> 
>>>> Attaching package: 'lmerTest'
>>>> 
>>>> The following object is masked from 'package:lme4':
>>>> 
>>>>    lmer
>>>> 
>>>> The following object is masked from 'package:stats':
>>>> 
>>>>    step
>>>> 
>>>>> 
>>>>> # test safeDeparse() - equivalence and differences to deparse():
>>>>> deparse_args <- formals(deparse)
>>>>> safeDeparse_args <- formals(lmerTest:::safeDeparse)
>>>>> stopifnot(
>>>> +   all.equal(names(deparse_args), names(safeDeparse_args)),
>>>> +   all.equal(deparse_args[!names(deparse_args) %in% c("control",
>>>> "width.cutoff")],
>>>> +             safeDeparse_args[!names(safeDeparse_args) %in%
>>>> c("control", "width.cutoff")]),
>>>> +   all.equal(deparse_args[["width.cutoff"]], 60L),
>>>> +   all(eval(safeDeparse_args[["control"]]) %in%
>>>> eval(deparse_args[["control"]])),
>>>> +   all.equal(safeDeparse_args[["width.cutoff"]], 500L)
>>>> + )
>>>>> 
>>>>> 
>>>> BEGIN TEST test_anova.R
>>>> 
>>>> R version 3.5.2 (2018-12-20) -- "Eggshell Igloo"
>>>> Copyright (C) 2018 The R Foundation for Statistical Computing
>>>> Platform: x86_64-pc-linux-gnu (64-bit)
>>>> 
>>>> R is free software and comes with ABSOLUTELY NO WARRANTY.
>>>> You are welcome to redistribute it under certain conditions.
>>>> Type 'license()' or 'licence()' for distribution details.
>>>> 
>>>> R is a collaborative project with many contributors.
>>>> Type 'contributors()' for more information and
>>>> 'citation()' on how to cite R or R packages in publications.
>>>> 
>>>> Type 'demo()' for some demos, 'help()' for on-line help, or
>>>> 'help.start()' for an HTML browser interface to help.
>>>> Type 'q()' to quit R.
>>>> 
>>>>> # test_anova.R
>>>>> library(lmerTest)
>>>> Loading required package: lme4
>>>> Loading required package: Matrix
>>>> 
>>>> Attaching package: 'lmerTest'
>>>> 
>>>> The following object is masked from 'package:lme4':
>>>> 
>>>>    lmer
>>>> 
>>>> The following object is masked from 'package:stats':
>>>> 
>>>>    step
>>>> 
>>>>> 
>>>>> # WRE says "using if(requireNamespace("pkgname")) is preferred, if
>>>> possible."
>>>>> # even in tests:
>>>>> assertError <- function(expr, ...)
>>>> +   if(requireNamespace("tools")) tools::assertError(expr, ...) else
>>>> invisible()
>>>>> assertWarning <- function(expr, ...)
>>>> +   if(requireNamespace("tools")) tools::assertWarning(expr, ...) else
>>>> invisible()
>>>>> 
>>>>> # Kenward-Roger only available with pbkrtest and only then validated
>>>> in R >= 3.3.3
>>>>> # (faulty results for R < 3.3.3 may be due to unstated dependencies in
>>>> pbkrtest)
>>>>> has_pbkrtest <- requireNamespace("pbkrtest", quietly = TRUE) &&
>>>> getRversion() >= "3.3.3"
>>>>> 
>>>>> data("sleepstudy", package="lme4")
>>>>> TOL <- 1e-4
>>>>> 
>>>>> ####################################
>>>>> ## Basic anova tests
>>>>> ####################################
>>>>> 
>>>>> m <- lmer(Reaction ~ Days + (Days | Subject), sleepstudy)
>>>>> 
>>>>> ####### ddf argument:
>>>>> (an1 <- anova(m)) # Also testing print method.
>>>> Type III Analysis of Variance Table with Satterthwaite's method
>>>>     Sum Sq Mean Sq NumDF  DenDF F value    Pr(>F)
>>>> Days  30024   30024     1 16.995  45.843 3.273e-06 ***
>>>> ---
>>>> Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
>>>>> (an2 <- anova(m, ddf="Satterthwaite"))
>>>> Type III Analysis of Variance Table with Satterthwaite's method
>>>>     Sum Sq Mean Sq NumDF  DenDF F value    Pr(>F)
>>>> Days  30024   30024     1 16.995  45.843 3.273e-06 ***
>>>> ---
>>>> Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
>>>>> (an2b <- anova(m, ddf="Satterthwaite", type=3))
>>>> Type III Analysis of Variance Table with Satterthwaite's method
>>>>     Sum Sq Mean Sq NumDF  DenDF F value    Pr(>F)
>>>> Days  30024   30024     1 16.995  45.843 3.273e-06 ***
>>>> ---
>>>> Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
>>>>> (an2c <- anova(m, ddf="Satterthwaite", type=2))
>>>> Type II Analysis of Variance Table with Satterthwaite's method
>>>>     Sum Sq Mean Sq NumDF  DenDF F value    Pr(>F)
>>>> Days  30024   30024     1 16.995  45.843 3.273e-06 ***
>>>> ---
>>>> Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
>>>>> stopifnot(isTRUE(
>>>> +   all.equal(an1, an2, tolerance=TOL)
>>>> + ))
>>>>> (an3 <- anova(m, ddf="Sat")) ## Abbreviated argument
>>>> Type III Analysis of Variance Table with Satterthwaite's method
>>>>     Sum Sq Mean Sq NumDF  DenDF F value    Pr(>F)
>>>> Days  30024   30024     1 16.995  45.843 3.273e-06 ***
>>>> ---
>>>> Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
>>>>> stopifnot(isTRUE(
>>>> +   all.equal(an1, an3, tolerance=TOL)
>>>> + ))
>>>>> if(has_pbkrtest) {
>>>> +   (anova(m, ddf="Kenward-Roger"))
>>>> +   (anova(m, ddf="Kenward-Roger", type=3))
>>>> + }
>>>> Type III Analysis of Variance Table with Kenward-Roger's method
>>>>     Sum Sq Mean Sq NumDF DenDF F value    Pr(>F)
>>>> Days  30024   30024     1    17  45.843 3.268e-06 ***
>>>> ---
>>>> Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
>>>>> (an1 <- anova(m, ddf="lme4"))
>>>> Analysis of Variance Table
>>>>     Df Sum Sq Mean Sq F value
>>>> Days  1  30024   30024  45.843
>>>>> (an2 <- anova(m, ddf="lme4", type=3)) # 'type' is ignored with ddf="lme4"
>>>> Analysis of Variance Table
>>>>     Df Sum Sq Mean Sq F value
>>>> Days  1  30024   30024  45.843
>>>>> stopifnot(isTRUE(
>>>> +   all.equal(an1, an2, tolerance=TOL)
>>>> + ))
>>>>> res <- assertError(anova(m, ddf="KR")) ## Error on incorrect arg.
>>>>> stopifnot(
>>>> +   grepl("'arg' should be one of ", unlist(res[[1]])$message)
>>>> + )
>>>>> 
>>>>> ## lme4 method:
>>>>> an1 <- anova(m, ddf="lme4")
>>>>> an2 <- anova(as(m, "lmerMod"))
>>>>> stopifnot(isTRUE(
>>>> +   all.equal(an1, an2, tolerance=TOL)
>>>> + ))
>>>>> 
>>>>> ###### type argument:
>>>>> (an1 <- anova(m, type="1")) # valid type arg.
>>>> Type I Analysis of Variance Table with Satterthwaite's method
>>>>     Sum Sq Mean Sq NumDF  DenDF F value    Pr(>F)
>>>> Days  30024   30024     1 16.995  45.843 3.273e-06 ***
>>>> ---
>>>> Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
>>>>> (an2 <- anova(m, type="I")) # same
>>>> Type I Analysis of Variance Table with Satterthwaite's method
>>>>     Sum Sq Mean Sq NumDF  DenDF F value    Pr(>F)
>>>> Days  30024   30024     1 16.995  45.843 3.273e-06 ***
>>>> ---
>>>> Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
>>>>> stopifnot(isTRUE(
>>>> +   all.equal(an1, an2, tolerance=TOL)
>>>> + ))
>>>>> (an3 <- anova(m, type=1)) # Not strictly valid, but accepted
>>>> Type I Analysis of Variance Table with Satterthwaite's method
>>>>     Sum Sq Mean Sq NumDF  DenDF F value    Pr(>F)
>>>> Days  30024   30024     1 16.995  45.843 3.273e-06 ***
>>>> ---
>>>> Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
>>>>> stopifnot(isTRUE(
>>>> +   all.equal(an1, an3, tolerance=TOL)
>>>> + ))
>>>>> 
>>>>> (an1 <- anova(m, type="2")) # valid type arg.
>>>> Type II Analysis of Variance Table with Satterthwaite's method
>>>>     Sum Sq Mean Sq NumDF  DenDF F value    Pr(>F)
>>>> Days  30024   30024     1 16.995  45.843 3.273e-06 ***
>>>> ---
>>>> Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
>>>>> (an2 <- anova(m, type="II")) # same
>>>> Type II Analysis of Variance Table with Satterthwaite's method
>>>>     Sum Sq Mean Sq NumDF  DenDF F value    Pr(>F)
>>>> Days  30024   30024     1 16.995  45.843 3.273e-06 ***
>>>> ---
>>>> Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
>>>>> stopifnot(isTRUE(
>>>> +   all.equal(an1, an2, tolerance=TOL)
>>>> + ))
>>>>> (an3 <- anova(m, type=3)) # Not strictly valid, but accepted
>>>> Type III Analysis of Variance Table with Satterthwaite's method
>>>>     Sum Sq Mean Sq NumDF  DenDF F value    Pr(>F)
>>>> Days  30024   30024     1 16.995  45.843 3.273e-06 ***
>>>> ---
>>>> Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
>>>>> stopifnot(isTRUE(
>>>> +   all.equal(an1, an3, check.attributes=FALSE, tolerance=TOL)
>>>> + ))
>>>>> 
>>>>> (an1 <- anova(m, type="3")) # valid type arg.
>>>> Type III Analysis of Variance Table with Satterthwaite's method
>>>>     Sum Sq Mean Sq NumDF  DenDF F value    Pr(>F)
>>>> Days  30024   30024     1 16.995  45.843 3.273e-06 ***
>>>> ---
>>>> Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
>>>>> (an2 <- anova(m, type="III")) # same
>>>> Type III Analysis of Variance Table with Satterthwaite's method
>>>>     Sum Sq Mean Sq NumDF  DenDF F value    Pr(>F)
>>>> Days  30024   30024     1 16.995  45.843 3.273e-06 ***
>>>> ---
>>>> Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
>>>>> stopifnot(isTRUE(
>>>> +   all.equal(an1, an2, tolerance=TOL)
>>>> + ))
>>>>> (an3 <- anova(m, type=3)) # Not strictly valid, but accepted
>>>> Type III Analysis of Variance Table with Satterthwaite's method
>>>>     Sum Sq Mean Sq NumDF  DenDF F value    Pr(>F)
>>>> Days  30024   30024     1 16.995  45.843 3.273e-06 ***
>>>> ---
>>>> Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
>>>>> stopifnot(isTRUE(
>>>> +   all.equal(an1, an3, tolerance=TOL)
>>>> + ))
>>>>> assertError(anova(m, type=0)) # Not valid arg.
>>>>> assertError(anova(m, type="i")) # Not valid arg.
>>>>> 
>>>>> ####### Model comparison:
>>>>> fm <- lm(Reaction ~ Days, sleepstudy)
>>>>> (an <- anova(m, fm))
>>>> refitting model(s) with ML (instead of REML)
>>>> Data: sleepstudy
>>>> Models:
>>>> fm: Reaction ~ Days
>>>> m: Reaction ~ Days + (Days | Subject)
>>>>   Df    AIC    BIC  logLik deviance  Chisq Chi Df Pr(>Chisq)
>>>> fm  3 1906.3 1915.9 -950.15   1900.3
>>>> m   6 1763.9 1783.1 -875.97   1751.9 148.35      3  < 2.2e-16 ***
>>>> ---
>>>> Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
>>>>> stopifnot(
>>>> +   nrow(an) == 2L,
>>>> +   rownames(an)[2] == "m"
>>>> + )
>>>>> 
>>>>> m2 <- lmer(Reaction ~ Days + I(Days^2) + (Days | Subject), sleepstudy)
>>>>> (an <- anova(m, m2, refit=FALSE))
>>>> Data: sleepstudy
>>>> Models:
>>>> m: Reaction ~ Days + (Days | Subject)
>>>> m2: Reaction ~ Days + I(Days^2) + (Days | Subject)
>>>>   Df    AIC    BIC  logLik deviance  Chisq Chi Df Pr(>Chisq)
>>>> m   6 1755.6 1774.8 -871.81   1743.6
>>>> m2  7 1756.8 1779.2 -871.41   1742.8 0.8127      1     0.3673
>>>>> stopifnot(
>>>> +   nrow(an) == 2L,
>>>> +   rownames(an)[1] == "m"
>>>> + )
>>>>> 
>>>>> 
>>>>> ####################################
>>>>> ## Example with factor fixef:
>>>>> ####################################
>>>>> 
>>>>> ## 'temp' is continuous, 'temperature' an ordered factor with 6 levels
>>>>> data("cake", package="lme4")
>>>>> m <- lmer(angle ~ recipe * temp + (1|recipe:replicate), cake)
>>>>> (an <- anova(m))
>>>> Type III Analysis of Variance Table with Satterthwaite's method
>>>>             Sum Sq Mean Sq NumDF  DenDF F value Pr(>F)
>>>> recipe         4.00    2.00     2 254.02  0.0957 0.9088
>>>> temp        1966.71 1966.71     1 222.00 94.1632 <2e-16 ***
>>>> recipe:temp    1.74    0.87     2 222.00  0.0417 0.9592
>>>> ---
>>>> Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
>>>>> (an_lme4 <- anova(m, ddf="lme4"))
>>>> Analysis of Variance Table
>>>>            Df  Sum Sq Mean Sq F value
>>>> recipe       2   10.39    5.20  0.2488
>>>> temp         1 1966.71 1966.71 94.1632
>>>> recipe:temp  2    1.74    0.87  0.0417
>>>>> 
>>>>> if(has_pbkrtest) {
>>>> +   (an_KR <- anova(m, ddf="Kenward-Roger"))
>>>> +   # res <- all.equal(an[, c("Sum Sq", "Mean Sq", "F value")],
>>>> +   #                  an_lme4[, c("Sum Sq", "Mean Sq", "F value")])
>>>> +   # stopifnot(isTRUE(res))
>>>> +   res <- all.equal(an[, c("Sum Sq", "Mean Sq", "F value")],
>>>> +                    an_KR[, c("Sum Sq", "Mean Sq", "F value")],
>>>> tolerance=TOL)
>>>> +   stopifnot(isTRUE(res))
>>>> + }
>>>>> stopifnot(all.equal(c(2, 1, 2), an$NumDF, tol=1e-6),
>>>> +           all.equal(c(254.0157612, 222, 222), an$DenDF, tol=TOL))
>>>>> 
>>>>> an3 <- anova(m, type=3)
>>>>> an2 <- anova(m, type=2)
>>>>> an1 <- anova(m, type=1)
>>>>> 
>>>>> ## Data is balanced, so Type II and III should be identical:
>>>>> ## One variable is continuous, so Type I and II/III are different:
>>>>> stopifnot(
>>>> +   isTRUE(all.equal(an3, an2, check.attributes=FALSE, tolerance=TOL)),
>>>> +   !isTRUE(all.equal(an1, an2, check.attributes=FALSE, tolerance=1e-8))
>>>> + )
>>>>> 
>>>>> # Using an ordered factor:
>>>>> m <- lmer(angle ~ recipe * temperature + (1|recipe:replicate), cake)
>>>>> (an1 <- anova(m, type=1))
>>>> Type I Analysis of Variance Table with Satterthwaite's method
>>>>                    Sum Sq Mean Sq NumDF DenDF F value Pr(>F)
>>>> recipe               10.19    5.09     2    42  0.2488 0.7809
>>>> temperature        2100.30  420.06     5   210 20.5199 <2e-16 ***
>>>> recipe:temperature  205.98   20.60    10   210  1.0062 0.4393
>>>> ---
>>>> Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
>>>>> (an2 <- anova(m, type=2))
>>>> Type II Analysis of Variance Table with Satterthwaite's method
>>>>                    Sum Sq Mean Sq NumDF DenDF F value Pr(>F)
>>>> recipe               10.19    5.09     2    42  0.2488 0.7809
>>>> temperature        2100.30  420.06     5   210 20.5199 <2e-16 ***
>>>> recipe:temperature  205.98   20.60    10   210  1.0062 0.4393
>>>> ---
>>>> Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
>>>>> # Type 3 is also available with ordered factors:
>>>>> (an3 <- anova(m, type=3))
>>>> Type III Analysis of Variance Table with Satterthwaite's method
>>>>                    Sum Sq Mean Sq NumDF DenDF F value Pr(>F)
>>>> recipe               10.19    5.09     2    42  0.2488 0.7809
>>>> temperature        2100.30  420.06     5   210 20.5199 <2e-16 ***
>>>> recipe:temperature  205.98   20.60    10   210  1.0062 0.4393
>>>> ---
>>>> Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
>>>>> ## Balanced data and only factors: Type I, II and III should be the same:
>>>>> stopifnot(
>>>> +   isTRUE(all.equal(an1, an2, check.attributes=FALSE, tolerance=TOL)),
>>>> +   isTRUE(all.equal(an1, an3, check.attributes=FALSE, tolerance=TOL))
>>>> + )
>>>>> 
>>>>> (an <- anova(m, type=1))
>>>> Type I Analysis of Variance Table with Satterthwaite's method
>>>>                    Sum Sq Mean Sq NumDF DenDF F value Pr(>F)
>>>> recipe               10.19    5.09     2    42  0.2488 0.7809
>>>> temperature        2100.30  420.06     5   210 20.5199 <2e-16 ***
>>>> recipe:temperature  205.98   20.60    10   210  1.0062 0.4393
>>>> ---
>>>> Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
>>>>> (an_lme4 <- anova(m, type=1, ddf="lme4"))
>>>> Analysis of Variance Table
>>>>                   Df  Sum Sq Mean Sq F value
>>>> recipe              2   10.19    5.09  0.2488
>>>> temperature         5 2100.30  420.06 20.5199
>>>> recipe:temperature 10  205.98   20.60  1.0062
>>>>> res <- all.equal(an[, c("Sum Sq", "Mean Sq", "F value")],
>>>> +                  an_lme4[, c("Sum Sq", "Mean Sq", "F value")],
>>>> tolerance=TOL)
>>>>> stopifnot(isTRUE(res))
>>>>> if(has_pbkrtest) {
>>>> +   (an_KR <- anova(m, type=1, ddf="Kenward-Roger"))
>>>> +   res <- all.equal(an[, c("Sum Sq", "Mean Sq", "F value")],
>>>> +                    an_KR[, c("Sum Sq", "Mean Sq", "F value")],
>>>> tolerance=TOL)
>>>> +   stopifnot(isTRUE(res))
>>>> + }
>>>>> stopifnot(all.equal(c(2, 5, 10), an$NumDF, tolerance=TOL),
>>>> +           all.equal(c(42, 210, 210), an$DenDF, tolerance=TOL))
>>>>> 
>>>>> ########
>>>>> ## Make case with balanced unordered factors:
>>>>> cake2 <- cake
>>>>> cake2$temperature <- factor(cake2$temperature, ordered = FALSE)
>>>>> # str(cake2)
>>>>> stopifnot(
>>>> +   !is.ordered(cake2$temperature)
>>>> + )
>>>>> m <- lmer(angle ~ recipe * temperature + (1|recipe:replicate), cake2)
>>>>> (an1 <- anova(m, type=1))
>>>> Type I Analysis of Variance Table with Satterthwaite's method
>>>>                    Sum Sq Mean Sq NumDF DenDF F value Pr(>F)
>>>> recipe               10.19    5.09     2    42  0.2488 0.7809
>>>> temperature        2100.30  420.06     5   210 20.5199 <2e-16 ***
>>>> recipe:temperature  205.98   20.60    10   210  1.0062 0.4393
>>>> ---
>>>> Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
>>>>> (an2 <- anova(m, type=2))
>>>> Type II Analysis of Variance Table with Satterthwaite's method
>>>>                    Sum Sq Mean Sq NumDF DenDF F value Pr(>F)
>>>> recipe               10.19    5.09     2    42  0.2488 0.7809
>>>> temperature        2100.30  420.06     5   210 20.5199 <2e-16 ***
>>>> recipe:temperature  205.98   20.60    10   210  1.0062 0.4393
>>>> ---
>>>> Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
>>>>> (an3 <- anova(m, type=3))
>>>> Type III Analysis of Variance Table with Satterthwaite's method
>>>>                    Sum Sq Mean Sq NumDF DenDF F value Pr(>F)
>>>> recipe               10.19    5.09     2    42  0.2488 0.7809
>>>> temperature        2100.30  420.06     5   210 20.5199 <2e-16 ***
>>>> recipe:temperature  205.98   20.60    10   210  1.0062 0.4393
>>>> ---
>>>> Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
>>>>> ## Balanced data and only factors: Type I, II, and III should be the same:
>>>>> stopifnot(
>>>> +   isTRUE(all.equal(an1, an2, check.attributes=FALSE, tolerance=TOL)),
>>>> +   isTRUE(all.equal(an3, an2, check.attributes=FALSE, tolerance=TOL))
>>>> + )
>>>>> ########
>>>>> 
>>>>> # No intercept:
>>>>> m <- lmer(angle ~ 0 + recipe * temp + (1|recipe:replicate), cake)
>>>>> (an <- anova(m, type=1))
>>>> Type I Analysis of Variance Table with Satterthwaite's method
>>>>             Sum Sq Mean Sq NumDF DenDF  F value Pr(>F)
>>>> recipe      21442.9  7147.6     3    42 342.2200 <2e-16 ***
>>>> temp         1966.7  1966.7     1   222  94.1632 <2e-16 ***
>>>> recipe:temp     1.7     0.9     2   222   0.0417 0.9592
>>>> ---
>>>> Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
>>>>> (an2 <- anova(m, type=2))
>>>> Type II Analysis of Variance Table with Satterthwaite's method
>>>>             Sum Sq Mean Sq NumDF  DenDF F value Pr(>F)
>>>> recipe         4.48    1.49     3 254.02  0.0714 0.9752
>>>> temp        1966.71 1966.71     1 222.00 94.1632 <2e-16 ***
>>>> recipe:temp    1.74    0.87     2 222.00  0.0417 0.9592
>>>> ---
>>>> Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
>>>>> (an2 <- anova(m, type=3))
>>>> Type III Analysis of Variance Table with Satterthwaite's method
>>>>             Sum Sq Mean Sq NumDF  DenDF F value Pr(>F)
>>>> recipe         4.48    1.49     3 254.02  0.0714 0.9752
>>>> temp        1966.71 1966.71     1 222.00 94.1632 <2e-16 ***
>>>> recipe:temp    1.74    0.87     2 222.00  0.0417 0.9592
>>>> ---
>>>> Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
>>>>> if(has_pbkrtest)
>>>> +   (an_KR <- anova(m, ddf="Kenward-Roger"))
>>>> Type III Analysis of Variance Table with Kenward-Roger's method
>>>>             Sum Sq Mean Sq NumDF  DenDF F value Pr(>F)
>>>> recipe         4.48    1.49     3 254.02  0.0714 0.9752
>>>> temp        1966.71 1966.71     1 222.00 94.1632 <2e-16 ***
>>>> recipe:temp    1.74    0.87     2 222.00  0.0417 0.9592
>>>> ---
>>>> Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
>>>>> (an_lme4 <- anova(m, ddf="lme4"))
>>>> Analysis of Variance Table
>>>>            Df  Sum Sq Mean Sq  F value
>>>> recipe       3 21442.9  7147.6 342.2200
>>>> temp         1  1966.7  1966.7  94.1632
>>>> recipe:temp  2     1.7     0.9   0.0417
>>>>> res <- all.equal(an[, c("Sum Sq", "Mean Sq", "F value")],
>>>> +                  an_lme4[, c("Sum Sq", "Mean Sq", "F value")],
>>>> tolerance=TOL)
>>>>> stopifnot(isTRUE(res))
>>>>> 
>>>>> # ML-fit:
>>>>> m <- lmer(angle ~ recipe * temp + (1|recipe:replicate), cake, REML=FALSE)
>>>>> (an <- anova(m, type=1))
>>>> Type I Analysis of Variance Table with Satterthwaite's method
>>>>             Sum Sq Mean Sq NumDF DenDF F value Pr(>F)
>>>> recipe        10.99    5.49     2    45  0.2666 0.7672
>>>> temp        1966.71 1966.71     1   225 95.4357 <2e-16 ***
>>>> recipe:temp    1.74    0.87     2   225  0.0423 0.9586
>>>> ---
>>>> Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
>>>>> if(has_pbkrtest)
>>>> +   assertError(an <- anova(m, ddf="Kenward-Roger")) # KR fits should be
>>>> REML
>>>>> (an_lme4 <- anova(m, ddf="lme4"))
>>>> Analysis of Variance Table
>>>>            Df  Sum Sq Mean Sq F value
>>>> recipe       2   10.99    5.49  0.2666
>>>> temp         1 1966.71 1966.71 95.4357
>>>> recipe:temp  2    1.74    0.87  0.0423
>>>>> res <- all.equal(an[, c("Sum Sq", "Mean Sq", "F value")],
>>>> +                  an_lme4[, c("Sum Sq", "Mean Sq", "F value")],
>>>> tolerance=TOL)
>>>>> stopifnot(isTRUE(res))
>>>>> 
>>>>> ####################################
>>>>> ## Using contr.sum:
>>>>> ####################################
>>>>> 
>>>>> m <- lmer(angle ~ recipe * temp + (1|recipe:replicate), cake,
>>>> +           contrasts = list('recipe' = "contr.sum"))
>>>>> (an <- anova(m, type=1))
>>>> Type I Analysis of Variance Table with Satterthwaite's method
>>>>             Sum Sq Mean Sq NumDF DenDF F value Pr(>F)
>>>> recipe        10.39    5.20     2    42  0.2488 0.7809
>>>> temp        1966.71 1966.71     1   222 94.1632 <2e-16 ***
>>>> recipe:temp    1.74    0.87     2   222  0.0417 0.9592
>>>> ---
>>>> Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
>>>>> (an2 <- anova(m, type=2))
>>>> Type II Analysis of Variance Table with Satterthwaite's method
>>>>             Sum Sq Mean Sq NumDF  DenDF F value Pr(>F)
>>>> recipe         4.00    2.00     2 254.02  0.0957 0.9088
>>>> temp        1966.71 1966.71     1 222.00 94.1632 <2e-16 ***
>>>> recipe:temp    1.74    0.87     2 222.00  0.0417 0.9592
>>>> ---
>>>> Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
>>>>> (an3 <- anova(m, type=3))
>>>> Type III Analysis of Variance Table with Satterthwaite's method
>>>>             Sum Sq Mean Sq NumDF  DenDF F value Pr(>F)
>>>> recipe         4.00    2.00     2 254.02  0.0957 0.9088
>>>> temp        1966.71 1966.71     1 222.00 94.1632 <2e-16 ***
>>>> recipe:temp    1.74    0.87     2 222.00  0.0417 0.9592
>>>> ---
>>>> Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
>>>>> stopifnot(
>>>> +   isTRUE(all.equal(an2, an3, check.attributes=FALSE, tolerance=TOL))
>>>> + )
>>>>> if(has_pbkrtest)
>>>> +   (an_KR <- anova(m, type=1, ddf="Kenward-Roger"))
>>>> Type I Analysis of Variance Table with Kenward-Roger's method
>>>>             Sum Sq Mean Sq NumDF DenDF F value Pr(>F)
>>>> recipe        10.39    5.20     2    42  0.2488 0.7809
>>>> temp        1966.71 1966.71     1   222 94.1632 <2e-16 ***
>>>> recipe:temp    1.74    0.87     2   222  0.0417 0.9592
>>>> ---
>>>> Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
>>>>> (an_lme4 <- anova(m, ddf="lme4"))
>>>> Analysis of Variance Table
>>>>            Df  Sum Sq Mean Sq F value
>>>> recipe       2   10.39    5.20  0.2488
>>>> temp         1 1966.71 1966.71 94.1632
>>>> recipe:temp  2    1.74    0.87  0.0417
>>>>> res <- all.equal(an[, c("Sum Sq", "Mean Sq", "F value")],
>>>> +                  an_lme4[, c("Sum Sq", "Mean Sq", "F value")],
>>>> tolerance=TOL)
>>>>> stopifnot(isTRUE(res))
>>>>> 
>>>>> 
>>>>> ####################################
>>>>> ## Example with continuous fixef:
>>>>> ####################################
>>>>> 
>>>>> # Example with no fixef:
>>>>> m <- lmer(Reaction ~ -1 + (Days | Subject), sleepstudy)
>>>>> # m <- lmer(Reaction ~ 0 + (Days | Subject), sleepstudy) # alternative
>>>>> stopifnot(length(fixef(m)) == 0L)
>>>>> (an <- anova(m, type=1))
>>>> Type I Analysis of Variance Table with Satterthwaite's method
>>>>     Sum Sq Mean Sq NumDF DenDF F value Pr(>F)
>>>>> (an_2 <- anova(m, type=2))
>>>> Type II Analysis of Variance Table with Satterthwaite's method
>>>>     Sum Sq Mean Sq NumDF DenDF F value Pr(>F)
>>>>> (an_3 <- anova(m, type=3))
>>>> Type III Analysis of Variance Table with Satterthwaite's method
>>>>     Sum Sq Mean Sq NumDF DenDF F value Pr(>F)
>>>>> stopifnot(nrow(an) == 0L,
>>>> +           nrow(an_2) == 0L,
>>>> +           nrow(an_3) == 0L)
>>>>> # anova(m, ddf="lme4") # Bug in lme4 it seems
>>>>> if(has_pbkrtest) {
>>>> +   (an_KR <- anova(m, ddf="Kenward-Roger"))
>>>> +   stopifnot(
>>>> +     nrow(an_KR) == 0L
>>>> +   )
>>>> + }
>>>>> 
>>>>> # Example with intercept only:
>>>>> m <- lmer(Reaction ~ (Days | Subject), sleepstudy)
>>>>> # m <- lmer(Reaction ~ 1 + (Days | Subject), sleepstudy) # alternative
>>>>> stopifnot(length(fixef(m)) == 1L,
>>>> +           names(fixef(m)) == "(Intercept)")
>>>>> (an <- anova(m))
>>>> Type III Analysis of Variance Table with Satterthwaite's method
>>>>     Sum Sq Mean Sq NumDF DenDF F value Pr(>F)
>>>>> (an_2 <- anova(m, type=2))
>>>> Type II Analysis of Variance Table with Satterthwaite's method
>>>>     Sum Sq Mean Sq NumDF DenDF F value Pr(>F)
>>>>> (an_3 <- anova(m, type=3))
>>>> Type III Analysis of Variance Table with Satterthwaite's method
>>>>     Sum Sq Mean Sq NumDF DenDF F value Pr(>F)
>>>>> (an_lme4 <- anova(m, ddf="lme4"))
>>>> Analysis of Variance Table
>>>>     Df Sum Sq Mean Sq F value
>>>>> stopifnot(nrow(an) == 0L,
>>>> +           nrow(an_2) == 0L,
>>>> +           nrow(an_3) == 0L,
>>>> +           nrow(an_lme4) == 0L)
>>>>> if(has_pbkrtest) {
>>>> +   (an_KR <- anova(m, ddf="Kenward-Roger"))
>>>> +   stopifnot(
>>>> +     nrow(an_KR) == 0L
>>>> +   )
>>>> + }
>>>>> 
>>>>> # Example with 1 fixef without intercept:
>>>>> m <- lmer(Reaction ~ Days - 1 + (Days | Subject), sleepstudy)
>>>> Warning message:
>>>> In checkConv(attr(opt, "derivs"), opt$par, ctrl = control$checkConv,  :
>>>>  Model failed to converge with max|grad| = 0.00416642 (tol = 0.002,
>>>> component 1)
>>>>> # m <- lmer(Reaction ~ 0 + Days + (Days | Subject), sleepstudy) #
>>>> alternative
>>>>> stopifnot(length(fixef(m)) == 1L,
>>>> +           names(fixef(m)) == "Days")
>>>>> (an <- anova(m))
>>>> Type III Analysis of Variance Table with Satterthwaite's method
>>>>     Sum Sq Mean Sq NumDF  DenDF F value    Pr(>F)
>>>> Days  93779   93779     1 16.995  143.19 1.054e-09 ***
>>>> ---
>>>> Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
>>>>> (an_2 <- anova(m, type=2))
>>>> Type II Analysis of Variance Table with Satterthwaite's method
>>>>     Sum Sq Mean Sq NumDF  DenDF F value    Pr(>F)
>>>> Days  93779   93779     1 16.995  143.19 1.054e-09 ***
>>>> ---
>>>> Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
>>>>> (an_3 <- anova(m, type=3))
>>>> Type III Analysis of Variance Table with Satterthwaite's method
>>>>     Sum Sq Mean Sq NumDF  DenDF F value    Pr(>F)
>>>> Days  93779   93779     1 16.995  143.19 1.054e-09 ***
>>>> ---
>>>> Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
>>>>> (an_lme4 <- anova(m, ddf="lme4"))
>>>> Analysis of Variance Table
>>>>     Df Sum Sq Mean Sq F value
>>>> Days  1  93779   93779  143.19
>>>>> stopifnot(nrow(an) == 1L,
>>>> +           nrow(an_2) == 1L,
>>>> +           nrow(an_3) == 1L,
>>>> +           nrow(an_lme4) == 1L)
>>>>> if(has_pbkrtest) {
>>>> +   (an_KR <- anova(m, ddf="Kenward-Roger"))
>>>> +   stopifnot(
>>>> +     nrow(an_KR) == 1L
>>>> +   )
>>>> + }
>>>>> 
>>>>> res <- all.equal(an[, c("Sum Sq", "Mean Sq", "F value")],
>>>> +                  an_lme4[, c("Sum Sq", "Mean Sq", "F value")],
>>>> tolerance=TOL)
>>>>> stopifnot(isTRUE(res))
>>>>> stopifnot(isTRUE(all.equal(
>>>> +   c(1, 17), unname(unlist(an[, c("NumDF", "DenDF")])), tolerance=TOL
>>>> + )))
>>>> Error: isTRUE(all.equal(c(1, 17), unname(unlist(an[, c("NumDF",
>>>> "DenDF")])),  .... is not TRUE
>>>> Execution halted
>>>> autopkgtest [04:42:39]: test run-unit-test: -----------------------]
>>>> 
>>> 
>>> 
>>> 
>>> 
>>>> _______________________________________________
>>>> R-pkg-team mailing list
>>>> R-pkg-team at alioth-lists.debian.net <mailto:R-pkg-team at alioth-lists.debian.net>
>>>> https://alioth-lists.debian.net/cgi-bin/mailman/listinfo/r-pkg-team <https://alioth-lists.debian.net/cgi-bin/mailman/listinfo/r-pkg-team>
>>> 
>>> 
>>> -- 
>>> http://fam-tille.de <http://fam-tille.de/>
>> 
>> -- 
>> http://fam-tille.de <http://fam-tille.de/>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://alioth-lists.debian.net/pipermail/r-pkg-team/attachments/20190211/dc2d0cd5/attachment-0001.html>


More information about the R-pkg-team mailing list