Understanding The New Statistics: Effect Sizes, Confidence Intervals, and Meta-Analysis by Geoff Cumming

Author: Reviewed by Eoin O'Connell

Understanding The New Statistics: Effect Sizes,

Confidence Intervals, and Meta-Analysis by

Geoff Cumming (part of the Multivariate

Applications Series). Published by Routledge

Academic (July 2011). 535pp.

Paperback: 978-0-415-87968-2.

Hardback: 978-0-415-87967-5.

RRP: £29.95 (Paperback); £62.50 (Hardback).

A sample chapter (Meta-Analysis 1: Introduction

and Forest Plots) is available for free download.

Null hypothesis significance testing (NHST) is rarely a satisfactory method of statistical analysis; we have known this for at least 50 years. The point is worth repeating because for much of this time, journals have published papers with incomplete analyses, reviewers have questioned authors’ use of more precise statistics and – worst of all – undergraduates have been taught that p values are paramount.

Cumming’s book aims to establish the shortcomings of NHST and the superiority of the (not so*) new statistics. It goes further than that however, this is a book intended to teach. There are exercises at the end of every chapter, exercise commentaries at the end of the book, boxed asides to expand on import points or exemplary articles, helpful rules of thumb for interpretation and excellent figures. In addition, Cumming provides a spreadsheet-based tool called ESCI (Exploratory Software for Confidence Intervals) that is downloadable, free for non-commercial use and central to many of the exercises and demonstrations.

Despite its informal tone and end-of-chapter cartoons, this is not a book for the layperson. Cumming assumes a practical knowledge of statistical tests and inference. Three chapters are devoted to meta-analysis and one each to replication, Cohen’s d, power, the non-central t distribution and precision. The last is most interesting because Cumming’s sample size calculations account not only for effect size but also the precision of the estimate.

Cumming succeeds in using graphics, arguments and his ESCI program to clearly show that NHST and omission of confidence intervals can be misleading. The New Statistics is a primer – it is not a comprehensive review of all available methods of analysis and there are many subtle statistical errors to avoid with more advanced analyses. The statistics and ideas laid out in in this book are a necessary grounding in the kind of statistical thinking and methods that every researcher should know.

The main drawback of the book is that it is intended for use in many disciplines but is largely based on just one – Cumming’s own – Psychology. Because of this, researchers in the social sciences will benefit most from this book. In any case, NHST tends to be less widespread in the ‘hard’ sciences such as Chemistry, Physics and Biology, Cumming says as much on the back cover.

If you are an undergraduate, postgraduate or researcher in the social sciences and tend to base your conclusions primarily on p values, this book will enlighten you. If you already dislike NHST and are fed up reading p values in academic articles, parts of this book will be of interest.

Surely it can’t be long before the NHST approach is seen as naïve at best. Undergrads, postgrads, lecturers and journal editors: over to you.

* This is not intended to slight Geoff Cumming. If anything it is aimed at researchers who whole-hearted believe that we have nothing better than NHST.