[R-pkg-team] Bug#921938: lme4 breaks r-cran-lmertest autopkgtest
Rune Christensen
Rune at ChristensenStatistics.dk
Mon Feb 11 16:21:54 GMT 2019
Dear all,
This is currently being fixed and a new version of lmerTest is on its way to CRAN, cf. https://cran.r-project.org/package=lmerTest <https://cran.r-project.org/package=lmerTest>
The latest version of lme4 was released to CRAN without our knowledge and that version caused lmerTest to fail its tests on most platforms (mostly because the new default optimiser is much less accurate than the old default optimiser, cf. https://github.com/lme4/lme4/issues/501 <https://github.com/lme4/lme4/issues/501>). After being notified of the problems we have updated lmerTest as soon as possible.
BR
Rune
> On 11 Feb 2019, at 07.39, Per B. Brockhoff <perbb at dtu.dk> wrote:
>
> Sendt fra min iPhone
>
> Start på videresendt besked:
>
>> Fra: Andreas Tille <tille at debian.org <mailto:tille at debian.org>>
>> Dato: 10. februar 2019 kl. 16.07.12 CET
>> Til: Paul Gevers <elbrus at debian.org <mailto:elbrus at debian.org>>, <921938 at bugs.debian.org <mailto:921938 at bugs.debian.org>>, "Per Bruun Brockhoff" <perbb at dtu.dk <mailto:perbb at dtu.dk>>
>> Emne: Vedr.: Bug#921938: lme4 breaks r-cran-lmertest autopkgtest
>>
>> Hi again,
>>
>> may be you get all the relevant information directly from CRAN:
>>
>> https://cloud.r-project.org/web/checks/check_results_lmerTest.html <https://cloud.r-project.org/web/checks/check_results_lmerTest.html>
>>
>> Kind regards
>>
>> Andreas.
>>
>> On Sun, Feb 10, 2019 at 03:44:55PM +0100, Andreas Tille wrote:
>>> Control: tags -1 help upstream
>>> Control: forwarded -1 Per Bruun Brockhoff <perbb at dtu.dk <mailto:perbb at dtu.dk>>
>>>
>>> Hi Per,
>>>
>>> the Debian packaged version of lmerTest seems to show issues with the
>>> latest version of lme4. From your insight into lmerTest can you give
>>> some hint about possible reasons for this incompatibility?
>>>
>>> Kind regards
>>>
>>> Andreas.
>>>
>>> PS: Please let me know whether you need further information.
>>>
>>> On Sun, Feb 10, 2019 at 01:35:53PM +0100, Paul Gevers wrote:
>>>> Source: lme4, r-cran-lmertest
>>>> Control: found -1 lme4/1.1-20-1
>>>> Control: found -1 r-cran-lmertest/3.0-1-2
>>>> Severity: important
>>>> X-Debbugs-CC: debian-ci at lists.debian.org <mailto:debian-ci at lists.debian.org>
>>>> User: debian-ci at lists.debian.org <mailto:debian-ci at lists.debian.org>
>>>> Usertags: breaks needs-update
>>>>
>>>> Dear maintainers,
>>>>
>>>> With a recent upload of lme4 the autopkgtest of r-cran-lmertest fails in
>>>> testing when that autopkgtest is run with the binary packages of lme4
>>>> from unstable. It passes when run with only packages from testing. In
>>>> tabular form:
>>>> pass fail
>>>> lme4 from testing 1.1-20-1
>>>> r-cran-lmertest from testing 3.0-1-2
>>>> all others from testing from testing
>>>>
>>>> I copied some of the output at the bottom of this report.
>>>>
>>>> Currently this regression is blocking the migration of lme4 to testing
>>>> [1]. Due to the nature of this issue, I filed this bug report against
>>>> both packages. Can you please investigate the situation and reassign the
>>>> bug to the right package? If needed, please change the bug's severity.
>>>>
>>>> More information about this bug and the reason for filing it can be found on
>>>> https://wiki.debian.org/ContinuousIntegration/RegressionEmailInformation <https://wiki.debian.org/ContinuousIntegration/RegressionEmailInformation>
>>>>
>>>> Paul
>>>>
>>>> [1] https://qa.debian.org/excuses.php?package=lme4 <https://qa.debian.org/excuses.php?package=lme4>
>>>>
>>>> https://ci.debian.net/data/autopkgtest/testing/amd64/r/r-cran-lmertest/1896879/log.gz <https://ci.debian.net/data/autopkgtest/testing/amd64/r/r-cran-lmertest/1896879/log.gz>
>>>>
>>>> autopkgtest [04:42:34]: test run-unit-test: [-----------------------
>>>> BEGIN TEST test_a_utils.R
>>>>
>>>> R version 3.5.2 (2018-12-20) -- "Eggshell Igloo"
>>>> Copyright (C) 2018 The R Foundation for Statistical Computing
>>>> Platform: x86_64-pc-linux-gnu (64-bit)
>>>>
>>>> R is free software and comes with ABSOLUTELY NO WARRANTY.
>>>> You are welcome to redistribute it under certain conditions.
>>>> Type 'license()' or 'licence()' for distribution details.
>>>>
>>>> R is a collaborative project with many contributors.
>>>> Type 'contributors()' for more information and
>>>> 'citation()' on how to cite R or R packages in publications.
>>>>
>>>> Type 'demo()' for some demos, 'help()' for on-line help, or
>>>> 'help.start()' for an HTML browser interface to help.
>>>> Type 'q()' to quit R.
>>>>
>>>>> # test_a_utils.R
>>>>>
>>>>> library(lmerTest)
>>>> Loading required package: lme4
>>>> Loading required package: Matrix
>>>>
>>>> Attaching package: 'lmerTest'
>>>>
>>>> The following object is masked from 'package:lme4':
>>>>
>>>> lmer
>>>>
>>>> The following object is masked from 'package:stats':
>>>>
>>>> step
>>>>
>>>>>
>>>>> # test safeDeparse() - equivalence and differences to deparse():
>>>>> deparse_args <- formals(deparse)
>>>>> safeDeparse_args <- formals(lmerTest:::safeDeparse)
>>>>> stopifnot(
>>>> + all.equal(names(deparse_args), names(safeDeparse_args)),
>>>> + all.equal(deparse_args[!names(deparse_args) %in% c("control",
>>>> "width.cutoff")],
>>>> + safeDeparse_args[!names(safeDeparse_args) %in%
>>>> c("control", "width.cutoff")]),
>>>> + all.equal(deparse_args[["width.cutoff"]], 60L),
>>>> + all(eval(safeDeparse_args[["control"]]) %in%
>>>> eval(deparse_args[["control"]])),
>>>> + all.equal(safeDeparse_args[["width.cutoff"]], 500L)
>>>> + )
>>>>>
>>>>>
>>>> BEGIN TEST test_anova.R
>>>>
>>>> R version 3.5.2 (2018-12-20) -- "Eggshell Igloo"
>>>> Copyright (C) 2018 The R Foundation for Statistical Computing
>>>> Platform: x86_64-pc-linux-gnu (64-bit)
>>>>
>>>> R is free software and comes with ABSOLUTELY NO WARRANTY.
>>>> You are welcome to redistribute it under certain conditions.
>>>> Type 'license()' or 'licence()' for distribution details.
>>>>
>>>> R is a collaborative project with many contributors.
>>>> Type 'contributors()' for more information and
>>>> 'citation()' on how to cite R or R packages in publications.
>>>>
>>>> Type 'demo()' for some demos, 'help()' for on-line help, or
>>>> 'help.start()' for an HTML browser interface to help.
>>>> Type 'q()' to quit R.
>>>>
>>>>> # test_anova.R
>>>>> library(lmerTest)
>>>> Loading required package: lme4
>>>> Loading required package: Matrix
>>>>
>>>> Attaching package: 'lmerTest'
>>>>
>>>> The following object is masked from 'package:lme4':
>>>>
>>>> lmer
>>>>
>>>> The following object is masked from 'package:stats':
>>>>
>>>> step
>>>>
>>>>>
>>>>> # WRE says "using if(requireNamespace("pkgname")) is preferred, if
>>>> possible."
>>>>> # even in tests:
>>>>> assertError <- function(expr, ...)
>>>> + if(requireNamespace("tools")) tools::assertError(expr, ...) else
>>>> invisible()
>>>>> assertWarning <- function(expr, ...)
>>>> + if(requireNamespace("tools")) tools::assertWarning(expr, ...) else
>>>> invisible()
>>>>>
>>>>> # Kenward-Roger only available with pbkrtest and only then validated
>>>> in R >= 3.3.3
>>>>> # (faulty results for R < 3.3.3 may be due to unstated dependencies in
>>>> pbkrtest)
>>>>> has_pbkrtest <- requireNamespace("pbkrtest", quietly = TRUE) &&
>>>> getRversion() >= "3.3.3"
>>>>>
>>>>> data("sleepstudy", package="lme4")
>>>>> TOL <- 1e-4
>>>>>
>>>>> ####################################
>>>>> ## Basic anova tests
>>>>> ####################################
>>>>>
>>>>> m <- lmer(Reaction ~ Days + (Days | Subject), sleepstudy)
>>>>>
>>>>> ####### ddf argument:
>>>>> (an1 <- anova(m)) # Also testing print method.
>>>> Type III Analysis of Variance Table with Satterthwaite's method
>>>> Sum Sq Mean Sq NumDF DenDF F value Pr(>F)
>>>> Days 30024 30024 1 16.995 45.843 3.273e-06 ***
>>>> ---
>>>> Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
>>>>> (an2 <- anova(m, ddf="Satterthwaite"))
>>>> Type III Analysis of Variance Table with Satterthwaite's method
>>>> Sum Sq Mean Sq NumDF DenDF F value Pr(>F)
>>>> Days 30024 30024 1 16.995 45.843 3.273e-06 ***
>>>> ---
>>>> Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
>>>>> (an2b <- anova(m, ddf="Satterthwaite", type=3))
>>>> Type III Analysis of Variance Table with Satterthwaite's method
>>>> Sum Sq Mean Sq NumDF DenDF F value Pr(>F)
>>>> Days 30024 30024 1 16.995 45.843 3.273e-06 ***
>>>> ---
>>>> Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
>>>>> (an2c <- anova(m, ddf="Satterthwaite", type=2))
>>>> Type II Analysis of Variance Table with Satterthwaite's method
>>>> Sum Sq Mean Sq NumDF DenDF F value Pr(>F)
>>>> Days 30024 30024 1 16.995 45.843 3.273e-06 ***
>>>> ---
>>>> Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
>>>>> stopifnot(isTRUE(
>>>> + all.equal(an1, an2, tolerance=TOL)
>>>> + ))
>>>>> (an3 <- anova(m, ddf="Sat")) ## Abbreviated argument
>>>> Type III Analysis of Variance Table with Satterthwaite's method
>>>> Sum Sq Mean Sq NumDF DenDF F value Pr(>F)
>>>> Days 30024 30024 1 16.995 45.843 3.273e-06 ***
>>>> ---
>>>> Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
>>>>> stopifnot(isTRUE(
>>>> + all.equal(an1, an3, tolerance=TOL)
>>>> + ))
>>>>> if(has_pbkrtest) {
>>>> + (anova(m, ddf="Kenward-Roger"))
>>>> + (anova(m, ddf="Kenward-Roger", type=3))
>>>> + }
>>>> Type III Analysis of Variance Table with Kenward-Roger's method
>>>> Sum Sq Mean Sq NumDF DenDF F value Pr(>F)
>>>> Days 30024 30024 1 17 45.843 3.268e-06 ***
>>>> ---
>>>> Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
>>>>> (an1 <- anova(m, ddf="lme4"))
>>>> Analysis of Variance Table
>>>> Df Sum Sq Mean Sq F value
>>>> Days 1 30024 30024 45.843
>>>>> (an2 <- anova(m, ddf="lme4", type=3)) # 'type' is ignored with ddf="lme4"
>>>> Analysis of Variance Table
>>>> Df Sum Sq Mean Sq F value
>>>> Days 1 30024 30024 45.843
>>>>> stopifnot(isTRUE(
>>>> + all.equal(an1, an2, tolerance=TOL)
>>>> + ))
>>>>> res <- assertError(anova(m, ddf="KR")) ## Error on incorrect arg.
>>>>> stopifnot(
>>>> + grepl("'arg' should be one of ", unlist(res[[1]])$message)
>>>> + )
>>>>>
>>>>> ## lme4 method:
>>>>> an1 <- anova(m, ddf="lme4")
>>>>> an2 <- anova(as(m, "lmerMod"))
>>>>> stopifnot(isTRUE(
>>>> + all.equal(an1, an2, tolerance=TOL)
>>>> + ))
>>>>>
>>>>> ###### type argument:
>>>>> (an1 <- anova(m, type="1")) # valid type arg.
>>>> Type I Analysis of Variance Table with Satterthwaite's method
>>>> Sum Sq Mean Sq NumDF DenDF F value Pr(>F)
>>>> Days 30024 30024 1 16.995 45.843 3.273e-06 ***
>>>> ---
>>>> Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
>>>>> (an2 <- anova(m, type="I")) # same
>>>> Type I Analysis of Variance Table with Satterthwaite's method
>>>> Sum Sq Mean Sq NumDF DenDF F value Pr(>F)
>>>> Days 30024 30024 1 16.995 45.843 3.273e-06 ***
>>>> ---
>>>> Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
>>>>> stopifnot(isTRUE(
>>>> + all.equal(an1, an2, tolerance=TOL)
>>>> + ))
>>>>> (an3 <- anova(m, type=1)) # Not strictly valid, but accepted
>>>> Type I Analysis of Variance Table with Satterthwaite's method
>>>> Sum Sq Mean Sq NumDF DenDF F value Pr(>F)
>>>> Days 30024 30024 1 16.995 45.843 3.273e-06 ***
>>>> ---
>>>> Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
>>>>> stopifnot(isTRUE(
>>>> + all.equal(an1, an3, tolerance=TOL)
>>>> + ))
>>>>>
>>>>> (an1 <- anova(m, type="2")) # valid type arg.
>>>> Type II Analysis of Variance Table with Satterthwaite's method
>>>> Sum Sq Mean Sq NumDF DenDF F value Pr(>F)
>>>> Days 30024 30024 1 16.995 45.843 3.273e-06 ***
>>>> ---
>>>> Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
>>>>> (an2 <- anova(m, type="II")) # same
>>>> Type II Analysis of Variance Table with Satterthwaite's method
>>>> Sum Sq Mean Sq NumDF DenDF F value Pr(>F)
>>>> Days 30024 30024 1 16.995 45.843 3.273e-06 ***
>>>> ---
>>>> Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
>>>>> stopifnot(isTRUE(
>>>> + all.equal(an1, an2, tolerance=TOL)
>>>> + ))
>>>>> (an3 <- anova(m, type=3)) # Not strictly valid, but accepted
>>>> Type III Analysis of Variance Table with Satterthwaite's method
>>>> Sum Sq Mean Sq NumDF DenDF F value Pr(>F)
>>>> Days 30024 30024 1 16.995 45.843 3.273e-06 ***
>>>> ---
>>>> Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
>>>>> stopifnot(isTRUE(
>>>> + all.equal(an1, an3, check.attributes=FALSE, tolerance=TOL)
>>>> + ))
>>>>>
>>>>> (an1 <- anova(m, type="3")) # valid type arg.
>>>> Type III Analysis of Variance Table with Satterthwaite's method
>>>> Sum Sq Mean Sq NumDF DenDF F value Pr(>F)
>>>> Days 30024 30024 1 16.995 45.843 3.273e-06 ***
>>>> ---
>>>> Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
>>>>> (an2 <- anova(m, type="III")) # same
>>>> Type III Analysis of Variance Table with Satterthwaite's method
>>>> Sum Sq Mean Sq NumDF DenDF F value Pr(>F)
>>>> Days 30024 30024 1 16.995 45.843 3.273e-06 ***
>>>> ---
>>>> Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
>>>>> stopifnot(isTRUE(
>>>> + all.equal(an1, an2, tolerance=TOL)
>>>> + ))
>>>>> (an3 <- anova(m, type=3)) # Not strictly valid, but accepted
>>>> Type III Analysis of Variance Table with Satterthwaite's method
>>>> Sum Sq Mean Sq NumDF DenDF F value Pr(>F)
>>>> Days 30024 30024 1 16.995 45.843 3.273e-06 ***
>>>> ---
>>>> Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
>>>>> stopifnot(isTRUE(
>>>> + all.equal(an1, an3, tolerance=TOL)
>>>> + ))
>>>>> assertError(anova(m, type=0)) # Not valid arg.
>>>>> assertError(anova(m, type="i")) # Not valid arg.
>>>>>
>>>>> ####### Model comparison:
>>>>> fm <- lm(Reaction ~ Days, sleepstudy)
>>>>> (an <- anova(m, fm))
>>>> refitting model(s) with ML (instead of REML)
>>>> Data: sleepstudy
>>>> Models:
>>>> fm: Reaction ~ Days
>>>> m: Reaction ~ Days + (Days | Subject)
>>>> Df AIC BIC logLik deviance Chisq Chi Df Pr(>Chisq)
>>>> fm 3 1906.3 1915.9 -950.15 1900.3
>>>> m 6 1763.9 1783.1 -875.97 1751.9 148.35 3 < 2.2e-16 ***
>>>> ---
>>>> Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
>>>>> stopifnot(
>>>> + nrow(an) == 2L,
>>>> + rownames(an)[2] == "m"
>>>> + )
>>>>>
>>>>> m2 <- lmer(Reaction ~ Days + I(Days^2) + (Days | Subject), sleepstudy)
>>>>> (an <- anova(m, m2, refit=FALSE))
>>>> Data: sleepstudy
>>>> Models:
>>>> m: Reaction ~ Days + (Days | Subject)
>>>> m2: Reaction ~ Days + I(Days^2) + (Days | Subject)
>>>> Df AIC BIC logLik deviance Chisq Chi Df Pr(>Chisq)
>>>> m 6 1755.6 1774.8 -871.81 1743.6
>>>> m2 7 1756.8 1779.2 -871.41 1742.8 0.8127 1 0.3673
>>>>> stopifnot(
>>>> + nrow(an) == 2L,
>>>> + rownames(an)[1] == "m"
>>>> + )
>>>>>
>>>>>
>>>>> ####################################
>>>>> ## Example with factor fixef:
>>>>> ####################################
>>>>>
>>>>> ## 'temp' is continuous, 'temperature' an ordered factor with 6 levels
>>>>> data("cake", package="lme4")
>>>>> m <- lmer(angle ~ recipe * temp + (1|recipe:replicate), cake)
>>>>> (an <- anova(m))
>>>> Type III Analysis of Variance Table with Satterthwaite's method
>>>> Sum Sq Mean Sq NumDF DenDF F value Pr(>F)
>>>> recipe 4.00 2.00 2 254.02 0.0957 0.9088
>>>> temp 1966.71 1966.71 1 222.00 94.1632 <2e-16 ***
>>>> recipe:temp 1.74 0.87 2 222.00 0.0417 0.9592
>>>> ---
>>>> Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
>>>>> (an_lme4 <- anova(m, ddf="lme4"))
>>>> Analysis of Variance Table
>>>> Df Sum Sq Mean Sq F value
>>>> recipe 2 10.39 5.20 0.2488
>>>> temp 1 1966.71 1966.71 94.1632
>>>> recipe:temp 2 1.74 0.87 0.0417
>>>>>
>>>>> if(has_pbkrtest) {
>>>> + (an_KR <- anova(m, ddf="Kenward-Roger"))
>>>> + # res <- all.equal(an[, c("Sum Sq", "Mean Sq", "F value")],
>>>> + # an_lme4[, c("Sum Sq", "Mean Sq", "F value")])
>>>> + # stopifnot(isTRUE(res))
>>>> + res <- all.equal(an[, c("Sum Sq", "Mean Sq", "F value")],
>>>> + an_KR[, c("Sum Sq", "Mean Sq", "F value")],
>>>> tolerance=TOL)
>>>> + stopifnot(isTRUE(res))
>>>> + }
>>>>> stopifnot(all.equal(c(2, 1, 2), an$NumDF, tol=1e-6),
>>>> + all.equal(c(254.0157612, 222, 222), an$DenDF, tol=TOL))
>>>>>
>>>>> an3 <- anova(m, type=3)
>>>>> an2 <- anova(m, type=2)
>>>>> an1 <- anova(m, type=1)
>>>>>
>>>>> ## Data is balanced, so Type II and III should be identical:
>>>>> ## One variable is continuous, so Type I and II/III are different:
>>>>> stopifnot(
>>>> + isTRUE(all.equal(an3, an2, check.attributes=FALSE, tolerance=TOL)),
>>>> + !isTRUE(all.equal(an1, an2, check.attributes=FALSE, tolerance=1e-8))
>>>> + )
>>>>>
>>>>> # Using an ordered factor:
>>>>> m <- lmer(angle ~ recipe * temperature + (1|recipe:replicate), cake)
>>>>> (an1 <- anova(m, type=1))
>>>> Type I Analysis of Variance Table with Satterthwaite's method
>>>> Sum Sq Mean Sq NumDF DenDF F value Pr(>F)
>>>> recipe 10.19 5.09 2 42 0.2488 0.7809
>>>> temperature 2100.30 420.06 5 210 20.5199 <2e-16 ***
>>>> recipe:temperature 205.98 20.60 10 210 1.0062 0.4393
>>>> ---
>>>> Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
>>>>> (an2 <- anova(m, type=2))
>>>> Type II Analysis of Variance Table with Satterthwaite's method
>>>> Sum Sq Mean Sq NumDF DenDF F value Pr(>F)
>>>> recipe 10.19 5.09 2 42 0.2488 0.7809
>>>> temperature 2100.30 420.06 5 210 20.5199 <2e-16 ***
>>>> recipe:temperature 205.98 20.60 10 210 1.0062 0.4393
>>>> ---
>>>> Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
>>>>> # Type 3 is also available with ordered factors:
>>>>> (an3 <- anova(m, type=3))
>>>> Type III Analysis of Variance Table with Satterthwaite's method
>>>> Sum Sq Mean Sq NumDF DenDF F value Pr(>F)
>>>> recipe 10.19 5.09 2 42 0.2488 0.7809
>>>> temperature 2100.30 420.06 5 210 20.5199 <2e-16 ***
>>>> recipe:temperature 205.98 20.60 10 210 1.0062 0.4393
>>>> ---
>>>> Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
>>>>> ## Balanced data and only factors: Type I, II and III should be the same:
>>>>> stopifnot(
>>>> + isTRUE(all.equal(an1, an2, check.attributes=FALSE, tolerance=TOL)),
>>>> + isTRUE(all.equal(an1, an3, check.attributes=FALSE, tolerance=TOL))
>>>> + )
>>>>>
>>>>> (an <- anova(m, type=1))
>>>> Type I Analysis of Variance Table with Satterthwaite's method
>>>> Sum Sq Mean Sq NumDF DenDF F value Pr(>F)
>>>> recipe 10.19 5.09 2 42 0.2488 0.7809
>>>> temperature 2100.30 420.06 5 210 20.5199 <2e-16 ***
>>>> recipe:temperature 205.98 20.60 10 210 1.0062 0.4393
>>>> ---
>>>> Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
>>>>> (an_lme4 <- anova(m, type=1, ddf="lme4"))
>>>> Analysis of Variance Table
>>>> Df Sum Sq Mean Sq F value
>>>> recipe 2 10.19 5.09 0.2488
>>>> temperature 5 2100.30 420.06 20.5199
>>>> recipe:temperature 10 205.98 20.60 1.0062
>>>>> res <- all.equal(an[, c("Sum Sq", "Mean Sq", "F value")],
>>>> + an_lme4[, c("Sum Sq", "Mean Sq", "F value")],
>>>> tolerance=TOL)
>>>>> stopifnot(isTRUE(res))
>>>>> if(has_pbkrtest) {
>>>> + (an_KR <- anova(m, type=1, ddf="Kenward-Roger"))
>>>> + res <- all.equal(an[, c("Sum Sq", "Mean Sq", "F value")],
>>>> + an_KR[, c("Sum Sq", "Mean Sq", "F value")],
>>>> tolerance=TOL)
>>>> + stopifnot(isTRUE(res))
>>>> + }
>>>>> stopifnot(all.equal(c(2, 5, 10), an$NumDF, tolerance=TOL),
>>>> + all.equal(c(42, 210, 210), an$DenDF, tolerance=TOL))
>>>>>
>>>>> ########
>>>>> ## Make case with balanced unordered factors:
>>>>> cake2 <- cake
>>>>> cake2$temperature <- factor(cake2$temperature, ordered = FALSE)
>>>>> # str(cake2)
>>>>> stopifnot(
>>>> + !is.ordered(cake2$temperature)
>>>> + )
>>>>> m <- lmer(angle ~ recipe * temperature + (1|recipe:replicate), cake2)
>>>>> (an1 <- anova(m, type=1))
>>>> Type I Analysis of Variance Table with Satterthwaite's method
>>>> Sum Sq Mean Sq NumDF DenDF F value Pr(>F)
>>>> recipe 10.19 5.09 2 42 0.2488 0.7809
>>>> temperature 2100.30 420.06 5 210 20.5199 <2e-16 ***
>>>> recipe:temperature 205.98 20.60 10 210 1.0062 0.4393
>>>> ---
>>>> Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
>>>>> (an2 <- anova(m, type=2))
>>>> Type II Analysis of Variance Table with Satterthwaite's method
>>>> Sum Sq Mean Sq NumDF DenDF F value Pr(>F)
>>>> recipe 10.19 5.09 2 42 0.2488 0.7809
>>>> temperature 2100.30 420.06 5 210 20.5199 <2e-16 ***
>>>> recipe:temperature 205.98 20.60 10 210 1.0062 0.4393
>>>> ---
>>>> Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
>>>>> (an3 <- anova(m, type=3))
>>>> Type III Analysis of Variance Table with Satterthwaite's method
>>>> Sum Sq Mean Sq NumDF DenDF F value Pr(>F)
>>>> recipe 10.19 5.09 2 42 0.2488 0.7809
>>>> temperature 2100.30 420.06 5 210 20.5199 <2e-16 ***
>>>> recipe:temperature 205.98 20.60 10 210 1.0062 0.4393
>>>> ---
>>>> Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
>>>>> ## Balanced data and only factors: Type I, II, and III should be the same:
>>>>> stopifnot(
>>>> + isTRUE(all.equal(an1, an2, check.attributes=FALSE, tolerance=TOL)),
>>>> + isTRUE(all.equal(an3, an2, check.attributes=FALSE, tolerance=TOL))
>>>> + )
>>>>> ########
>>>>>
>>>>> # No intercept:
>>>>> m <- lmer(angle ~ 0 + recipe * temp + (1|recipe:replicate), cake)
>>>>> (an <- anova(m, type=1))
>>>> Type I Analysis of Variance Table with Satterthwaite's method
>>>> Sum Sq Mean Sq NumDF DenDF F value Pr(>F)
>>>> recipe 21442.9 7147.6 3 42 342.2200 <2e-16 ***
>>>> temp 1966.7 1966.7 1 222 94.1632 <2e-16 ***
>>>> recipe:temp 1.7 0.9 2 222 0.0417 0.9592
>>>> ---
>>>> Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
>>>>> (an2 <- anova(m, type=2))
>>>> Type II Analysis of Variance Table with Satterthwaite's method
>>>> Sum Sq Mean Sq NumDF DenDF F value Pr(>F)
>>>> recipe 4.48 1.49 3 254.02 0.0714 0.9752
>>>> temp 1966.71 1966.71 1 222.00 94.1632 <2e-16 ***
>>>> recipe:temp 1.74 0.87 2 222.00 0.0417 0.9592
>>>> ---
>>>> Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
>>>>> (an2 <- anova(m, type=3))
>>>> Type III Analysis of Variance Table with Satterthwaite's method
>>>> Sum Sq Mean Sq NumDF DenDF F value Pr(>F)
>>>> recipe 4.48 1.49 3 254.02 0.0714 0.9752
>>>> temp 1966.71 1966.71 1 222.00 94.1632 <2e-16 ***
>>>> recipe:temp 1.74 0.87 2 222.00 0.0417 0.9592
>>>> ---
>>>> Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
>>>>> if(has_pbkrtest)
>>>> + (an_KR <- anova(m, ddf="Kenward-Roger"))
>>>> Type III Analysis of Variance Table with Kenward-Roger's method
>>>> Sum Sq Mean Sq NumDF DenDF F value Pr(>F)
>>>> recipe 4.48 1.49 3 254.02 0.0714 0.9752
>>>> temp 1966.71 1966.71 1 222.00 94.1632 <2e-16 ***
>>>> recipe:temp 1.74 0.87 2 222.00 0.0417 0.9592
>>>> ---
>>>> Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
>>>>> (an_lme4 <- anova(m, ddf="lme4"))
>>>> Analysis of Variance Table
>>>> Df Sum Sq Mean Sq F value
>>>> recipe 3 21442.9 7147.6 342.2200
>>>> temp 1 1966.7 1966.7 94.1632
>>>> recipe:temp 2 1.7 0.9 0.0417
>>>>> res <- all.equal(an[, c("Sum Sq", "Mean Sq", "F value")],
>>>> + an_lme4[, c("Sum Sq", "Mean Sq", "F value")],
>>>> tolerance=TOL)
>>>>> stopifnot(isTRUE(res))
>>>>>
>>>>> # ML-fit:
>>>>> m <- lmer(angle ~ recipe * temp + (1|recipe:replicate), cake, REML=FALSE)
>>>>> (an <- anova(m, type=1))
>>>> Type I Analysis of Variance Table with Satterthwaite's method
>>>> Sum Sq Mean Sq NumDF DenDF F value Pr(>F)
>>>> recipe 10.99 5.49 2 45 0.2666 0.7672
>>>> temp 1966.71 1966.71 1 225 95.4357 <2e-16 ***
>>>> recipe:temp 1.74 0.87 2 225 0.0423 0.9586
>>>> ---
>>>> Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
>>>>> if(has_pbkrtest)
>>>> + assertError(an <- anova(m, ddf="Kenward-Roger")) # KR fits should be
>>>> REML
>>>>> (an_lme4 <- anova(m, ddf="lme4"))
>>>> Analysis of Variance Table
>>>> Df Sum Sq Mean Sq F value
>>>> recipe 2 10.99 5.49 0.2666
>>>> temp 1 1966.71 1966.71 95.4357
>>>> recipe:temp 2 1.74 0.87 0.0423
>>>>> res <- all.equal(an[, c("Sum Sq", "Mean Sq", "F value")],
>>>> + an_lme4[, c("Sum Sq", "Mean Sq", "F value")],
>>>> tolerance=TOL)
>>>>> stopifnot(isTRUE(res))
>>>>>
>>>>> ####################################
>>>>> ## Using contr.sum:
>>>>> ####################################
>>>>>
>>>>> m <- lmer(angle ~ recipe * temp + (1|recipe:replicate), cake,
>>>> + contrasts = list('recipe' = "contr.sum"))
>>>>> (an <- anova(m, type=1))
>>>> Type I Analysis of Variance Table with Satterthwaite's method
>>>> Sum Sq Mean Sq NumDF DenDF F value Pr(>F)
>>>> recipe 10.39 5.20 2 42 0.2488 0.7809
>>>> temp 1966.71 1966.71 1 222 94.1632 <2e-16 ***
>>>> recipe:temp 1.74 0.87 2 222 0.0417 0.9592
>>>> ---
>>>> Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
>>>>> (an2 <- anova(m, type=2))
>>>> Type II Analysis of Variance Table with Satterthwaite's method
>>>> Sum Sq Mean Sq NumDF DenDF F value Pr(>F)
>>>> recipe 4.00 2.00 2 254.02 0.0957 0.9088
>>>> temp 1966.71 1966.71 1 222.00 94.1632 <2e-16 ***
>>>> recipe:temp 1.74 0.87 2 222.00 0.0417 0.9592
>>>> ---
>>>> Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
>>>>> (an3 <- anova(m, type=3))
>>>> Type III Analysis of Variance Table with Satterthwaite's method
>>>> Sum Sq Mean Sq NumDF DenDF F value Pr(>F)
>>>> recipe 4.00 2.00 2 254.02 0.0957 0.9088
>>>> temp 1966.71 1966.71 1 222.00 94.1632 <2e-16 ***
>>>> recipe:temp 1.74 0.87 2 222.00 0.0417 0.9592
>>>> ---
>>>> Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
>>>>> stopifnot(
>>>> + isTRUE(all.equal(an2, an3, check.attributes=FALSE, tolerance=TOL))
>>>> + )
>>>>> if(has_pbkrtest)
>>>> + (an_KR <- anova(m, type=1, ddf="Kenward-Roger"))
>>>> Type I Analysis of Variance Table with Kenward-Roger's method
>>>> Sum Sq Mean Sq NumDF DenDF F value Pr(>F)
>>>> recipe 10.39 5.20 2 42 0.2488 0.7809
>>>> temp 1966.71 1966.71 1 222 94.1632 <2e-16 ***
>>>> recipe:temp 1.74 0.87 2 222 0.0417 0.9592
>>>> ---
>>>> Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
>>>>> (an_lme4 <- anova(m, ddf="lme4"))
>>>> Analysis of Variance Table
>>>> Df Sum Sq Mean Sq F value
>>>> recipe 2 10.39 5.20 0.2488
>>>> temp 1 1966.71 1966.71 94.1632
>>>> recipe:temp 2 1.74 0.87 0.0417
>>>>> res <- all.equal(an[, c("Sum Sq", "Mean Sq", "F value")],
>>>> + an_lme4[, c("Sum Sq", "Mean Sq", "F value")],
>>>> tolerance=TOL)
>>>>> stopifnot(isTRUE(res))
>>>>>
>>>>>
>>>>> ####################################
>>>>> ## Example with continuous fixef:
>>>>> ####################################
>>>>>
>>>>> # Example with no fixef:
>>>>> m <- lmer(Reaction ~ -1 + (Days | Subject), sleepstudy)
>>>>> # m <- lmer(Reaction ~ 0 + (Days | Subject), sleepstudy) # alternative
>>>>> stopifnot(length(fixef(m)) == 0L)
>>>>> (an <- anova(m, type=1))
>>>> Type I Analysis of Variance Table with Satterthwaite's method
>>>> Sum Sq Mean Sq NumDF DenDF F value Pr(>F)
>>>>> (an_2 <- anova(m, type=2))
>>>> Type II Analysis of Variance Table with Satterthwaite's method
>>>> Sum Sq Mean Sq NumDF DenDF F value Pr(>F)
>>>>> (an_3 <- anova(m, type=3))
>>>> Type III Analysis of Variance Table with Satterthwaite's method
>>>> Sum Sq Mean Sq NumDF DenDF F value Pr(>F)
>>>>> stopifnot(nrow(an) == 0L,
>>>> + nrow(an_2) == 0L,
>>>> + nrow(an_3) == 0L)
>>>>> # anova(m, ddf="lme4") # Bug in lme4 it seems
>>>>> if(has_pbkrtest) {
>>>> + (an_KR <- anova(m, ddf="Kenward-Roger"))
>>>> + stopifnot(
>>>> + nrow(an_KR) == 0L
>>>> + )
>>>> + }
>>>>>
>>>>> # Example with intercept only:
>>>>> m <- lmer(Reaction ~ (Days | Subject), sleepstudy)
>>>>> # m <- lmer(Reaction ~ 1 + (Days | Subject), sleepstudy) # alternative
>>>>> stopifnot(length(fixef(m)) == 1L,
>>>> + names(fixef(m)) == "(Intercept)")
>>>>> (an <- anova(m))
>>>> Type III Analysis of Variance Table with Satterthwaite's method
>>>> Sum Sq Mean Sq NumDF DenDF F value Pr(>F)
>>>>> (an_2 <- anova(m, type=2))
>>>> Type II Analysis of Variance Table with Satterthwaite's method
>>>> Sum Sq Mean Sq NumDF DenDF F value Pr(>F)
>>>>> (an_3 <- anova(m, type=3))
>>>> Type III Analysis of Variance Table with Satterthwaite's method
>>>> Sum Sq Mean Sq NumDF DenDF F value Pr(>F)
>>>>> (an_lme4 <- anova(m, ddf="lme4"))
>>>> Analysis of Variance Table
>>>> Df Sum Sq Mean Sq F value
>>>>> stopifnot(nrow(an) == 0L,
>>>> + nrow(an_2) == 0L,
>>>> + nrow(an_3) == 0L,
>>>> + nrow(an_lme4) == 0L)
>>>>> if(has_pbkrtest) {
>>>> + (an_KR <- anova(m, ddf="Kenward-Roger"))
>>>> + stopifnot(
>>>> + nrow(an_KR) == 0L
>>>> + )
>>>> + }
>>>>>
>>>>> # Example with 1 fixef without intercept:
>>>>> m <- lmer(Reaction ~ Days - 1 + (Days | Subject), sleepstudy)
>>>> Warning message:
>>>> In checkConv(attr(opt, "derivs"), opt$par, ctrl = control$checkConv, :
>>>> Model failed to converge with max|grad| = 0.00416642 (tol = 0.002,
>>>> component 1)
>>>>> # m <- lmer(Reaction ~ 0 + Days + (Days | Subject), sleepstudy) #
>>>> alternative
>>>>> stopifnot(length(fixef(m)) == 1L,
>>>> + names(fixef(m)) == "Days")
>>>>> (an <- anova(m))
>>>> Type III Analysis of Variance Table with Satterthwaite's method
>>>> Sum Sq Mean Sq NumDF DenDF F value Pr(>F)
>>>> Days 93779 93779 1 16.995 143.19 1.054e-09 ***
>>>> ---
>>>> Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
>>>>> (an_2 <- anova(m, type=2))
>>>> Type II Analysis of Variance Table with Satterthwaite's method
>>>> Sum Sq Mean Sq NumDF DenDF F value Pr(>F)
>>>> Days 93779 93779 1 16.995 143.19 1.054e-09 ***
>>>> ---
>>>> Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
>>>>> (an_3 <- anova(m, type=3))
>>>> Type III Analysis of Variance Table with Satterthwaite's method
>>>> Sum Sq Mean Sq NumDF DenDF F value Pr(>F)
>>>> Days 93779 93779 1 16.995 143.19 1.054e-09 ***
>>>> ---
>>>> Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
>>>>> (an_lme4 <- anova(m, ddf="lme4"))
>>>> Analysis of Variance Table
>>>> Df Sum Sq Mean Sq F value
>>>> Days 1 93779 93779 143.19
>>>>> stopifnot(nrow(an) == 1L,
>>>> + nrow(an_2) == 1L,
>>>> + nrow(an_3) == 1L,
>>>> + nrow(an_lme4) == 1L)
>>>>> if(has_pbkrtest) {
>>>> + (an_KR <- anova(m, ddf="Kenward-Roger"))
>>>> + stopifnot(
>>>> + nrow(an_KR) == 1L
>>>> + )
>>>> + }
>>>>>
>>>>> res <- all.equal(an[, c("Sum Sq", "Mean Sq", "F value")],
>>>> + an_lme4[, c("Sum Sq", "Mean Sq", "F value")],
>>>> tolerance=TOL)
>>>>> stopifnot(isTRUE(res))
>>>>> stopifnot(isTRUE(all.equal(
>>>> + c(1, 17), unname(unlist(an[, c("NumDF", "DenDF")])), tolerance=TOL
>>>> + )))
>>>> Error: isTRUE(all.equal(c(1, 17), unname(unlist(an[, c("NumDF",
>>>> "DenDF")])), .... is not TRUE
>>>> Execution halted
>>>> autopkgtest [04:42:39]: test run-unit-test: -----------------------]
>>>>
>>>
>>>
>>>
>>>
>>>> _______________________________________________
>>>> R-pkg-team mailing list
>>>> R-pkg-team at alioth-lists.debian.net <mailto:R-pkg-team at alioth-lists.debian.net>
>>>> https://alioth-lists.debian.net/cgi-bin/mailman/listinfo/r-pkg-team <https://alioth-lists.debian.net/cgi-bin/mailman/listinfo/r-pkg-team>
>>>
>>>
>>> --
>>> http://fam-tille.de <http://fam-tille.de/>
>>
>> --
>> http://fam-tille.de <http://fam-tille.de/>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://alioth-lists.debian.net/pipermail/r-pkg-team/attachments/20190211/dc2d0cd5/attachment-0001.html>
More information about the R-pkg-team
mailing list