Applying measures of discriminatory accuracy to revisit traditional risk factors for being small for gestational age in Sweden: a national cross-sectional study
================================================================================================================================================================

* Sol Pía Juárez
* Phillip Wagner
* Juan Merlo

## Abstract

**Objectives** Small for gestational age (SGA) is considered as an indicator of intrauterine growth restriction, and multiple maternal and newborn characteristics have been identified as risk factors for SGA. This knowledge is mainly based on measures of average association (ie, OR) that quantify differences in average risk between exposed and unexposed groups. Nevertheless, average associations do not assess the discriminatory accuracy of the risk factors (ie, its ability to discriminate the babies who will develop SGA from those that will not). Therefore, applying measures of discriminatory accuracy rather than measures of association only, our study revisits known risk factors of SGA and discusses their role from a public health perspective.

**Design** Cross-sectional study. We measured maternal (ie, smoking, hypertension, age, marital status, education) and delivery (ie, sex, gestational age, birth order) characteristics and performed logistic regression models to estimate both ORs and measures of discriminatory accuracy, like the area under the receiver operating characteristic curve (AU-ROC) and the net reclassification improvement.

**Setting** Data were obtained from the Swedish Medical Birth Registry.

**Participants** Our sample included 731 989 babies born during 1987–1993.

**Results** We replicated the expected associations. For instance, smoking (OR=2.57), having had a previous SGA baby (OR=5.48) and hypertension (OR=4.02) were strongly associated with SGA. However, they show a very small discriminatory accuracy (AU-ROC≈0.5). The discriminatory accuracy increased, but remained unsatisfactorily low (AU-ROC=0.6), when including all variables studied in the same model.

**Conclusions** Traditional risk factors for SGA alone or in combination have a low accuracy for discriminating babies with SGA from those without SGA. A proper understanding of these findings is of fundamental relevance to address future research and to design policymaking recommendations in a more informed way.

### Strengths and limitations of this study

*   Our study emphasises the use and interpretation of measures of discriminatory accuracy (ie, capacity to distinguish between small for gestational age (SGA) and non-SGA babies) when evaluating risk factors.

*   We confirm statistical associations between maternal and newborn characteristics and risk for SGA, but we underline that the discriminatory capacity of all the risk factors studied was very low.

*   This low discriminatory capacity suggests that we know very little about the determinants of SGA in the population and that more efforts should be devoted to understand individual heterogeneity of effects.

*   Our finding is of fundamental relevance to address future research and to design policymaking recommendations in a more informed way.

## Introduction

Small for gestational age (SGA) is commonly identified as a proxy for intrauterine growth restriction (IUGR).1 This disorder has been associated with neonatal mortality and morbidities2 as well as with major medical problems across the life course, such as a higher risk of neurodevelopmental impairments,3 ,4 autism,5 schizophrenia,6 impaired cognitive function,7 coeliac disease in boys8 and reduced bone mass during early infancy,9 as well as Barrett's oesophagus and oesophagitis10 ,11 and others.12 ,13 Therefore, the identification of maternal and newborn characteristics (denominated as ‘risk factors’ in the rest of this work) associated with an increased risk for SGA is of obvious relevance in public health and preventive medicine.

Two reviews, one from 198714 and the other from 2009,15 pointed out that SGA is associated with a broad number of genetic, obstetric, demographic and socioeconomic factors as well as maternal morbidities and toxic exposures before and during pregnancy. However, the identification of these risk factors has been exclusively based on measures of average association (eg, OR) but without considering their accuracy for discriminating babies with, from those without, SGA. Indeed, it is common practice to use measures of average association to gauge the ability of a factor to discriminate future cases of disease.16 For example, it is known that maternal hypertension during pregnancy gives a 5.5-fold increased risk of delivering an SGA baby.17 Therefore, this variable is implicitly used as a predictive test to classify who will and who will not deliver an SGA baby. However, in spite of this popular belief, measures of association alone are inappropriate for this discriminatory purpose insofar as there are different scenarios of sensitivity/specificity for a given OR.16–23

Although measures of discriminatory accuracy are extensively applied in other fields of epidemiology like the identification of new biomarkers for cardiovascular diseases,18–21 these measures are still unusual in public health and epidemiology.22 In fact, as far as we know, they have never been explicitly used to formally revisit established maternal and newborn risk factors for SGA.

With this background our study aims to revisit the role of current risk factors for SGA in public health. We do it in two steps. First, using measures of average association, we aim to replicate previous findings and identify maternal and newborn risk factors for SGA. Second, we apply measures of discriminatory accuracy to assess the ability of those risk factors (alone or in combination) to discriminate babies with, from those without, SGA in the whole population and in different subgroups according to gestational age.

## Data and methods

### Study design, setting and participants

This is a cross-sectional study based on a population-based register. We identified all the 811 599 babies born alive and recorded at the Swedish Medical Birth Registry (MBR) between 1 January 1987 and 31 December 1993. The MBR collects detailed and standardised information on nearly all pregnancies in Sweden culminating in delivery.23 ,24 Using a unique personal identification number, the Swedish authorities (National Board of Health and Welfare and Statistics Sweden) linked the MBR to the Register of the Total Population and the Swedish 1990 population census and created a research database. This database was delivered to us without the personal identification numbers to protect the anonymity of the participants.

For the purpose of our study, we selected singletons, because it is known that multiple births (n=19 167) have a different intrauterine growth pattern from gestational weeks 28–30.25 We excluded 13 539 babies born with significant congenital anomalies according to the MBR. Following previously established criteria,26 we also excluded babies with inconsistent information on birth weight according to gestational age (n=9195) and babies weighing less than 500 g (n=51) as well as 15 observations with missing information on maternal age and birth order. The final sample contained 768 059 babies. Thereafter, we stratified the population by gestational age into preterm (<37 gestational weeks), term (≥37 and <42 gestational weeks) and post-term babies (≥42 gestational weeks; figure 1).

![Figure 1](http://bmjopen.bmj.com/https://bmjopen.bmj.com/content/bmjopen/4/7/e005388/F1.medium.gif)

[Figure 1](http://bmjopen.bmj.com/content/4/7/e005388/F1)

Figure 1 
Flow diagram showing the individuals excluded from the study population.

### Variables

The outcome variable combined birth weight and gestational age to dichotomise as being SGA or not, and using the last category as the reference. This variable was available at the MBR, where it is routinely calculated following standard intrauterine growth curves.27 Infants were defined as SGA if they weighed less than 2 SDs below the expected birth weight for gestational age and gender, according to a Swedish intrauterine growth curve.28

In our analyses we included child and maternal characteristics that are known to be associated with low birthweight and SGA.

As child characteristics we used *sex*14 ,29 and *birth order*30 ,31 classified into three categories (ie, firstborn, second, and third or more). Among maternal characteristics we included *birth interval between newborns,*14 ,28 categorised into <1, 1–2, >2 years, ‘only child’ (ie, when we know that the newborns have previous siblings but we do not have their information in our setting) and first child (ie, when we know the newborn has no previous siblings); whether the mother has a *previous child with SGA*32 categorised into yes, no, ‘only child’ and first child; *education*,33 ,34 categorised into low (primary education or less), middle (secondary school) and high education (graduate and PhD); *marital status*,35 ,36 categorised into single, widowed, or divorced, and married or cohabiting; and *maternal age at delivery*,37–39 categorised into four groups (ie, <20, 20–24, 25–34 and >35 years old), as well as information on *smoking habits*,40–43 categorised into non-smoking, light smoking (fewer than 9 cigarettes per day), heavy smoking (more than 9 cigarettes per day), and missing information. Finally, we included information about the presence of *hypertension during pregnancy* (yes vs no),15 ,17 and *maternal origin*, classified as being born in Sweden or not.44

## Statistical methods

To examine the average association between, on the one hand, the categorical variables mentioned above, and on the other, being SGA, we simply calculated ORs and 95% CIs obtained from logistic regression analyses.

The discriminatory accuracy of a risk factor is better appraised by measuring *the true positive fraction* (TPF) and *the false positive fraction* (FPF). For a dichotomous risk factor, the TPF expresses the probability of being exposed to the risk factor when the SGA occurs (ie, cases that are exposed to the risk factor), and the FPF indicates the probability of being exposed to the risk factor when the SGA does not occur (ie, controls exposed to the risk factor). In the ideal scenario the TPF should be 1 and the FPF should be 0, even if a lower TPF or a higher FPF. For instance, if the identification of the risk factors conveys pharmacological treatment, we should try to keep the FPF as low as possible.

For the evaluation of the discriminatory accuracy of the combination of risk factors within a risk score (ie, predicted probability) we obtained *the receiver operating characteristic* (ROC) *curve*. The ROC curve is constructed by plotting the TPF against the FPF for different risk score thresholds.16 ,45 ,46 A traditional measure of discriminatory accuracy is the *area under the ROC curve* (AU-ROC) or C statistic.16 ,45 ,47–49 The AU-ROC extends from 0.5 to 1.0. An AU-ROC=0.5 means that the discriminatory accuracy of the candidate risk factor is similar to that obtained by flipping a coin. That is, a risk factor with an AU-ROC=0.5 is useless. An AU-ROC=1.0 means complete accuracy.

In a series of simple logistic regression models we identified the single variables with the highest discriminatory accuracy. Using this information, thereafter, we performed two models. Model A only with the two variables with the higher discriminatory accuracy (ie, smoking and birth order) and model B which adds the rest of covariates to the initial model A. We ran this second model in order to assess the change of discriminatory accuracy when adding the rest of information to a simpler model. We appraised the incremental value of a model by the difference between AU-ROCs. Owing to a problem of collinearity, Stata automatically deleted the two categories in common (ie, ‘only child’ and first child) shared by the variables of birth interval and previous child with SGA, keeping them only in the former. All models were stratified by gestational age (ie, preterm, term and post-term) because it has been suggested that SGA at term and at preterm may have been driven by a different aetiology.50 We included post-term to complete the classification.

We performed the analyses in the whole population, stratifying by gestational age (ie, preterm, term and post-term). We performed the statistical analyses using STATA V.12.0 (College Station, StataCorp LP, Texas, USA) and SPSS V.21.0 (Armonk, IBM Corp, New York, USA).

## Results

Table 1 shows the maternal and individual characteristics of the population of newborns by the SGA status. We see that SGA is much more prevalent among preterm babies (10.14%) than among term (1.87%) and post-term (3.03%) babies. Females show higher prevalence of SGA than males among preterm, and slightly lower prevalence among those born post-term. Regardless of gestational age, firstborns had a higher risk of SGA than their siblings. SGA is more prevalent among children who had a previous sibling during the same year, except among those babies born at preterm, but this may be due to the larger amount of missing information about the previous siblings (11%). Mothers who had a previous child with SGA are more likely to have a current SGA baby regardless of gestational age.

View this table:
[Table 1](http://bmjopen.bmj.com/content/4/7/e005388/T1)

Table 1 
Prevalence of SGA in the whole population of babies and in strata of gestational age in Sweden 1987–1993

SGA was more frequent in mothers younger than 20 years of age, among divorced, widowed and single women, and among those who were born outside Sweden and those with low educational achievement. In babies born with SGA, hypertension was more frequent among preterm than among post-term babies.

Table 2 indicates that the risk for being SGA was similar in boys and girls. However, as expected, not being a firstborn reduced the risk of being SGA. With respect to maternal characteristics, mothers younger than 20 years and those 35 years and older had a higher risk of delivering an SGA baby than 20–24-year-old mothers. Mothers who had a previous child during the same year have a higher risk of having a SGA baby as well as those who had a previous child with SGA. Mothers who experienced hypertension during pregnancy had a higher risk of delivering SGA babies. Compared with non-smoker mothers, light and heavy smoker mothers had a higher risk of delivering an SGA baby. Divorced and widowed mothers as well as single mothers were more likely to deliver an SGA baby than married and cohabiting mothers. Mothers with primary and secondary education had a higher risk of delivering SGA babies than mothers with a university degree. Similarly, mothers who were not born in Sweden were at higher risk of delivering an SGA baby.

View this table:
[Table 2](http://bmjopen.bmj.com/content/4/7/e005388/T2)

Table 2 
Measures of association between offspring and maternal characteristics, and being small for gestational age (SGA), in the whole population of babies and in strata of gestational age in Sweden 1987–1993

Figure 2 shows the values for the AU-ROC of the variables included in table 2. Overall, their discriminatory accuracy was rather low. Newborn babies had the lower discriminatory accuracy. Having a SGA child and hypertension, despite the risk factors being most strongly associated with SGA (OR 5.48 and 4.02, respectively), led to a very low discriminatory accuracy (AU-ROC 0.54 and 0.51). Birth order and smoking were the variables with the highest accuracy (AU-ROC 0.59).

![Figure 2](http://bmjopen.bmj.com/https://bmjopen.bmj.com/content/bmjopen/4/7/e005388/F2.medium.gif)

[Figure 2](http://bmjopen.bmj.com/content/4/7/e005388/F2)

Figure 2 
Area under the receiver operating characteristic curve to compare the discriminatory accuracy of different models to distinguish between small for gestational age (SGA) and non-SGA babies.

Figure 3 shows the AU-ROC for SGA of different risk factors after stratification by preterm, term and post-term. As in the non-stratified analysis, the discriminatory accuracy of the variables was low. Smoking at term showed the highest discrimination (AU-ROC 0.60).

![Figure 3](http://bmjopen.bmj.com/https://bmjopen.bmj.com/content/bmjopen/4/7/e005388/F3.medium.gif)

[Figure 3](http://bmjopen.bmj.com/content/4/7/e005388/F3)

Figure 3 
Area under the receiver operating characteristic curve for specific maternal and newborn characteristics.

Figure 4 shows that the discriminatory accuracy of the general model A, including only birth order and smoking, was slightly improved (just 0.05 proportion units), when all variables were included in the full model B. Among preterm babies model B improved the discriminatory accuracy of the model by 0.1 proportion units, while this improvement was much lower among SGA term and SGA post-term babies.

![Figure 4](http://bmjopen.bmj.com/https://bmjopen.bmj.com/content/bmjopen/4/7/e005388/F4.medium.gif)

[Figure 4](http://bmjopen.bmj.com/content/4/7/e005388/F4)

Figure 4 
Area under the receiver operating characteristic curve for specific maternal and newborn characteristics after stratifying by gestational age (preterm, term and post-term).

## Discussions

We were able to verify a number of recognised maternal and newborn risk factors for SGA. For instance, we found that smoking (OR 2.56) and especially having had a previous SGA baby (OR 5.48) and maternal hypertension (OR 4.02) were ‘strongly’ associated with being SGA. However, even if the magnitude of the ORs was of a size normally considered as undoubtedly relevant in epidemiology, none of those traditional risk factors for SGA provided enough accuracy to discriminate babies with SGA from other babies. In fact, the AU-ROC for having had a previous SGA child and maternal hypertension was slightly higher than 0.5, which means that the accuracy of this variable for discriminating babies with SGA from those without SGA was rather similar to that obtained by flipping a coin. That is, we need to recognise that, although on average, mothers with hypertension were four times more likely to have an SGA baby, many mothers with hypertension delivered babies without SGA, and many SGA babies were born to mothers without hypertension. Our findings, therefore, seriously question the utility of maternal hypertension during pregnancy for planning strategies of prevention against SGA. This statement, however, does not mean that hypertension during pregnancy is irrelevant to understanding the origin of SGA, but rather that we need to determine who among hypertensive mothers is actually prone to deliver an SGA baby.

There is a tacit but fallacious belief that the discriminatory accuracy of a risk factor is high when it is supported by a ‘strong’ association (eg, an OR of 4, as in the case of maternal hypertension). However, for an association to be an accurate instrument for discrimination, it must be of a magnitude rarely identified in epidemiological studies.16 ,51–53 Following our example, a low discriminatory accuracy only indicates that any attempt of intervention based on the existence of the risk factor will be inefficient and even inappropriate, because health professionals will unnecessarily treat many mothers. The decision to start an intervention should seriously take into account the existence of important (physical or emotional) side effects in the false-positive women. That is, it is always important to consider the principle of *primum non nocere*.22

Compared with the other variables studied, birth order and smoking presented a higher discriminatory capacity. However, their discriminatory accuracy was still very low in absolute terms (AU-ROC≈0.59). Also, combining all the variables in the same model did not substantially increase the discriminatory accuracy (AU-ROC=0.69). In other words, our results indicate that we actually do not know so much about what determines being SGA.

The existence of a low discriminatory accuracy suggests that around the population average risk there is considerable individual heterogeneity. Therefore, a logical consequence should be to identify which women are most susceptible to the risk factors. Hence, we explored the discriminatory accuracy of the chosen risk factors in different strata of gestational age at birth. We found that the combination of all variables in the same model had a minor improvement for discriminatory accuracy among those born at term or preterm as well as post-term.

Our finding suggests the existence of individual heterogeneity of responses to some specific variables, so the discriminatory accuracy depended, for instance, on whether the baby was preterm, term or post-term. In fact, smoking, birth order, maternal origin and marital status had a lower discriminatory capacity among preterm than among babies at term. On the contrary, newborn babies and hypertension had a higher discriminatory accuracy among preterm babies than among term babies. In the same way, newborn babies and maternal origin had a lower discriminatory capacity among term than among post-term babies, but for smoking and having had a previous SGA baby we found the opposite relationship. The variation of the magnitude of the discriminatory accuracy by gestational age at birth expresses the existence of individual heterogeneity.

In addition, the definition of SGA may also actively contribute to reducing the discriminatory accuracy of the traditional risk factors since discrimination depends on the outcome and exposure. Thus, low discrimination can result from the fact that SGA fails to distinguish between pathological and constitutionally small babies, that is, to properly capture the health dimension that it is supposed to be a proxy for IUGR.1 In order to address this shortcoming, we stratified SGA by gestational age as this has been identified as a good strategy to distinguish between these two.50 However, we do not find support for this approach since we found a lower discriminatory accuracy among preterm SGA babies (presumably pathologically small) than among term SGA babies. In this regard, our findings show awareness of the caveats pointed out by previous studies on the use of SGA as a proxy for IUGR,1 and encourage further research aiming to better capture IUGR.

Our findings have important research and policymaking implications. A possible reason for the low discriminatory accuracy of many average associations is that average effects are a mixture of individual level effects and therefore mix inter-individual heterogeneity (ie, some individuals respond intensively to the exposure, while others are resilient or might even respond in the opposite direction). The approach based on discriminatory accuracy understands average effects as an idealised mean value that does not necessarily represent the heterogeneity of individual effects.22 Some scholars prefer to conceive individual outcome as the expression of a stochastic phenomenon that is best estimated by the average risk using a probabilistic approach.54 Our understanding instead is that individual outcome reflects the interindividual heterogeneity of responses that can be potentially determined; lack of knowledge could be amended by a better understanding of individual responses.55 See elsewhere for a better explanation of these ideas.22 ,56 ,57 From this perspective, reducing exposure to a risk factor would only be effective when acting on the susceptible, but not on the resilient, individuals. For instance, we need to better capture babies who suffer from IUGR, since, so far, we have been incapable of distinguishing between babies who are constitutionally small from those who are pathologically growth restricted.1 By stratifying between preterm, term, and post-term, we might be able to better approach the underlying heterogeneity.

From the policymaking perspective, our findings suggest that hitherto there has not been enough knowledge to identify any specific risk factor or combination of them that could discriminate with accuracy children with and without the SGA status. Our findings support policymaking oriented to lifestyle modification, as according to the principle of *primum non nocere*;22 they have mostly positive consequences, even for ‘false-positive’ mothers. For instance, persuading women to quit smoking reduces the risk of SGA in some babies, but it improves general well-being in everyone. However, other risk factors with low discriminatory accuracy that lead to pharmacological treatment or screening might result in unnecessary side effects and cost. In the long run, an uncritical use of variables with low discriminatory accuracy may hinder the identification of pertinent risk factors and susceptible individuals and damage the scientific credibility of modern epidemiology.22 ,56 ,57

Our conclusions are based on classical measures of discriminatory accuracy such as the AU-ROC curve. These measures have been criticised as insensitive to small changes in predicted individual risk.58 Some authors propose more specific measures of reclassification, like the net reclassification improvement (NRI), and the integrated discrimination improvement (IDI).59–62 We applied NRI and IDI in a sensitivity analysis (results not shown in tables). For example, using NRI, we observed a reclassification of 30%. However, this figure does not add substantial information to our results, since NRI (as well as IDI) refers to the misclassification occurring all along the risk scale, instead of capturing the misclassification which takes place around the fixed threshold. Furthermore, the new NRI and IDI measures have also been criticised,63 and some authors64 have explicitly advised against their use in common epidemiological practice because, unlike IDI and NRI, traditional measures of discrimination like the AU-ROC curve have the advantage that prognostic performance cannot be manipulated.64 Therefore, we preferred to quantify discriminatory accuracy by analysing ROC curves and AU-ROCs.

Our analyses are based on a national medical registry covering almost the entire population of residents in Sweden. Nearly all births are registered in the MBR, because giving birth at home is very unusual in Sweden. In addition, estimation of SGA is routinely calculated at the MBR following standard intrauterine growth curves.27 However, our study also has a number of limitations. Because of lack of data, we could not assess many other variables identified in the literature as ‘risk factors’, such as genetic or nutritional factors.15 ,65 In spite of the quality of the MBR, the information regarding smoking is based on a self-reported questionnaire (anamnesis) administered by the mid-wife at the first antenatal visit (ie, between 10 and 12 gestational weeks), which to some extent might bias the result by including misclassification of exposure.66 However, a study conducted in Sweden comparing self-reported nicotine exposure and plasma levels of cotinine in early and late pregnancy concluded that self-reported smoking information had acceptable validity.67

Unfortunately, we could not identify those mothers who suffered from preeclampsia, for which the discriminatory accuracy concerning SGA may be higher than for hypertension in our model. Further analysis on this aspect is required. Another limitation of our study is that we calculated the discriminatory accuracy in the same sample used for constructing the predicted model. This procedure, however, might overestimate the discriminatory accuracy of the models, so the low discriminatory accuracy found may be an underestimation.

Since our study has been carried out with data from 1987 to 1993, we performed a sensitivity analysis to check possible differences in current years (2000–2010) but the results remain the same. Given the consistence of the results, we preferred to maintain the results for years 1987–1993 to use a more accurate outcome since the definition of SGA is based on standard curves estimated for Sweden with data from 1985 to 1989.29 Moreover, the period we cover is of relevance to our study since most of the risk factors which are discussed in our paper were mainly identified in that period and, in the case of Sweden, with the data we used.

In conclusion, applying measures of discriminatory accuracy rather than measures of association only, our study revisits known risk factors of SGA and discusses their role from a public health perspective. We found that neither models including simple variables nor models including several variables at the same time have a good discriminatory accuracy to discriminate babies with SGA from those without SGA. This finding is of fundamental relevance to address future research and to design policymaking recommendations in a more informed way.

As noted elsewhere,56 ,57 there is need of a new epidemiological approach that systematically provides information on the discriminatory accuracy and interindividual heterogeneity of effects and does not rely only on average measures of association.68 In this line, new statistical methods like logic regression seem promising.69 ,70 A fundamental change is needed in the way traditional risk factors are currently interpreted in public health epidemiology. If the *discriminatory accuracy* of most classical risk factors is very low, what happens with the vast majority of recommendations given so far in epidemiology and public health? Are health professionals misleading the community by raising the alarm about risks that may be harmless for most individuals? What are the ethical repercussions of using risk factors with low discriminatory accuracy? Are there problems of inefficiency, medicalisation and stigmatisation? We believe that these questions have a high significance for both the community and the future of public health research.

## Acknowledgments

The authors thank Dr Karin Kallén for her very helping comments on a previous version of this paper.

## Footnotes

*   Contributors JM had the original idea of applying measures of discriminatory accuracy for the interpretation of risk factors and discussed it with PW and SPJ. JM and SPJ initiated the study. JM, SPJ and PW contributed to the design of the study; SPJ performed the analyses under the supervision of JM and PW. SPJ wrote the first draft of the manuscript, and JM contributed to the writing of the final version. All authors made substantial contributions to the interpretation of the results and manuscript revision and approved the final version of the manuscript.

*   Funding This work was supported by the Swedish Research Council (VR) (Dnr #2013-2484, PI Juan Merlo), the Centre for Economic Demography and the SIMSAM early life Lund (Dnr #2013-5474).

*   Competing interests None.

*   Ethics approval The database was approved by the regional ethical review board in southern Sweden. Being a register-based study, the board did not require explicit informed consent from the women.

*   Provenance and peer review Not commissioned; externally peer reviewed.

*   Data sharing statement No additional data are available.

This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: [http://creativecommons.org/licenses/by-nc/4.0/](http://creativecommons.org/licenses/by-nc/4.0/)

## References

1.  Urquia ML, Ray JG. Seven caveats on the use of low birthweight and related indicators in health research. J Epidemiol Community Health 2012;66:971–5.
    
    [Abstract/FREE Full Text](http://bmjopen.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NDoiamVjaCI7czo1OiJyZXNpZCI7czo5OiI2Ni8xMS85NzEiO3M6NDoiYXRvbSI7czoyNToiL2Jtam9wZW4vNC83L2UwMDUzODguYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

2.  Grisaru-Granovsky S, Reichman B, Lerner-Geva L, et al. Mortality and morbidity in preterm small-for-gestational-age infants: a population-based study. Am J Obstet Gynecol 2012;206:150e151–6.
    
    

3.  Streimish IG, Ehrenkranz RA, Allred EN, et al. Birth weight- and fetal weight-growth restriction: impact on neurodevelopment. Early Hum Dev 2012;88:765–71.
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1016/j.earlhumdev.2012.04.004&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=22732241&link_type=MED&atom=%2Fbmjopen%2F4%2F7%2Fe005388.atom) 
    
    [Web of Science](http://bmjopen.bmj.com/lookup/external-ref?access_num=000307610800011&link_type=ISI) 

4.  Eikenes L, Martinussen MP, Lund LK, et al. Being born small for gestational age reduces white matter integrity in adulthood: a prospective cohort study. Pediatr Res 2012;72:649–54.
    
    

5.  Moore GS, Kneitel AW, Walker CK, et al. Autism risk in small- and large-for-gestational-age infants. Am J Obstet Gynecol 2012;206:314.
    
    

6.  Nielsen PR, Mortensen PB, Dalman C, et al. Fetal growth and schizophrenia: a nested case-control and case-sibling study. Schizophr Bull 2013;39:1337–42.
    
    [Abstract/FREE Full Text](http://bmjopen.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6Njoic2NoYnVsIjtzOjU6InJlc2lkIjtzOjk6IjM5LzYvMTMzNyI7czo0OiJhdG9tIjtzOjI1OiIvYm1qb3Blbi80LzcvZTAwNTM4OC5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

7.  Løhaugen GC, Ostgård HF, Andreassen S, et al. Small for gestational age and intrauterine growth restriction decreases cognitive function in young adults. J Pediatr 2013;163:447–53.
    
    

8.  Wingren CJ, Agardh D, Merlo J. Revisiting the risk of celiac disease in children born small for gestational age: a sibling design perspective. Scand J Gastroenterol 2012;47:632–9.
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=22428795&link_type=MED&atom=%2Fbmjopen%2F4%2F7%2Fe005388.atom) 

9.  van de Lagemaat M, Rotteveel J, van Weissenbruch MM, et al. Small-for-gestational-age preterm-born infants already have lower bone mass during early infancy. Bone 2012;51:441–6.
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1016/j.bone.2012.06.017&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=22750451&link_type=MED&atom=%2Fbmjopen%2F4%2F7%2Fe005388.atom) 

10. Forssell L, Cnattingius S, Bottai M, et al. Increased risk of Barrett's esophagus among individuals born preterm or small for gestational age. Clin Gastroenterol Hepatol 2013;11:790–4.
    
    

11. Forssell L, Cnattingius S, Bottai M, et al. Risk of esophagitis among individuals born preterm or small for gestational age. Clin Gastroenterol Hepatol 2012;10:1369–75.
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=22989864&link_type=MED&atom=%2Fbmjopen%2F4%2F7%2Fe005388.atom) 

12. Dalla Pozza RD, Bechtold S, Putzker S, et al. Young adults born small for gestational age: is reduced baroreceptor sensitivity a risk factor for hypertension? Clin Cardiol 2006;29:215–18.
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=16739394&link_type=MED&atom=%2Fbmjopen%2F4%2F7%2Fe005388.atom) 

13. Crispi F, Figueras F, Cruz-Lemini M, et al. Cardiovascular programming in children born small for gestational age and relationship with prenatal signs of severity. Am J Obstet Gynecol 2012;207:121.e121–9.
    
    

14. Kramer MS. Determinants of low birth weight: methodological assessment and meta-analysis. Bull World Health Organ 1987;65:663–737.
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=3322602&link_type=MED&atom=%2Fbmjopen%2F4%2F7%2Fe005388.atom) 
    
    [Web of Science](http://bmjopen.bmj.com/lookup/external-ref?access_num=A1987L477100010&link_type=ISI) 

15. McCowan L, Horgan RP. Risk factors for small for gestational age infants. Best Pract Res Clin Obstet Gynaecol 2009;23:779–93.
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1016/j.bpobgyn.2009.06.003&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=19604726&link_type=MED&atom=%2Fbmjopen%2F4%2F7%2Fe005388.atom) 

16. Pepe MS, Janes H, Longton G, et al. Limitations of the odds ratio in gauging the performance of a diagnostic, prognostic, or screening marker. Am J Epidemiol 2004;159:882–90.
    
    [Abstract/FREE Full Text](http://bmjopen.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NzoiYW1qZXBpZCI7czo1OiJyZXNpZCI7czo5OiIxNTkvOS84ODIiO3M6NDoiYXRvbSI7czoyNToiL2Jtam9wZW4vNC83L2UwMDUzODguYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

17. Catov JM, Nohr EA, Olsen J, et al. Chronic hypertension related to risk for preterm and term small for gestational age births. Obstet Gynecol 2008;112:290–6.
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1097/AOG.0b013e31817f589b&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=18669725&link_type=MED&atom=%2Fbmjopen%2F4%2F7%2Fe005388.atom) 
    
    [Web of Science](http://bmjopen.bmj.com/lookup/external-ref?access_num=000258237100014&link_type=ISI) 

18. Zethelius B, Berglund L, Sundstrom J, et al. Use of multiple biomarkers to improve the prediction of death from cardiovascular causes. N Engl J Med 2008;358:2107–16.
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1056/NEJMoa0707064&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=18480203&link_type=MED&atom=%2Fbmjopen%2F4%2F7%2Fe005388.atom) 
    
    [Web of Science](http://bmjopen.bmj.com/lookup/external-ref?access_num=000255849300004&link_type=ISI) 

19. Cooney MT, Dudina AL, Graham IM. Value and limitations of existing scores for the assessment of cardiovascular risk: a review for clinicians. J Am Coll Cardiol 2009;54:1209–27.
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1016/j.jacc.2009.07.020&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=19778661&link_type=MED&atom=%2Fbmjopen%2F4%2F7%2Fe005388.atom) 
    
    [Web of Science](http://bmjopen.bmj.com/lookup/external-ref?access_num=000270082700001&link_type=ISI) 

20. Law MR, Wald NJ, Morris JK. The performance of blood pressure and other cardiovascular risk factors as screening tests for ischaemic heart disease and stroke. J Med Screen 2004;11:3–7.
    
    [Abstract/FREE Full Text](http://bmjopen.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NToic3Btc2MiO3M6NToicmVzaWQiO3M6NjoiMTEvMS8zIjtzOjQ6ImF0b20iO3M6MjU6Ii9ibWpvcGVuLzQvNy9lMDA1Mzg4LmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

21. Melander O, Newton-Cheh C, Almgren P, et al. Novel and conventional biomarkers for prediction of incident cardiovascular events in the community. JAMA 2009;302:49–57.
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1001/jama.2009.943&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=19567439&link_type=MED&atom=%2Fbmjopen%2F4%2F7%2Fe005388.atom) 
    
    [Web of Science](http://bmjopen.bmj.com/lookup/external-ref?access_num=000267492800021&link_type=ISI) 

22. Merlo J. Multilevel analysis of individual heterogeneity: a fundamental critique of the current probabilistic risk factor epidemiology (invited commentary). Am J Epidemiol 2014;180:208–12.
    
    [Abstract/FREE Full Text](http://bmjopen.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NzoiYW1qZXBpZCI7czo1OiJyZXNpZCI7czo5OiIxODAvMi8yMDgiO3M6NDoiYXRvbSI7czoyNToiL2Jtam9wZW4vNC83L2UwMDUzODguYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

23. National Board of Health and Welfare, Centre for Epidemiology. Fakta om mammor, förlossningar och nyfödda barn (Facts about Mothers, Births and Newborn Children): Medicinska Födelseregistret 1973–2000. Medical Birth Register 1973–2000 (in Swedish) (No. 2002-125-12). National Board of Health and Welfare, Stockholm, 2002.
    
    

24. Cnattingius S, Ericson A, Gunnarskog J, et al. A quality study of a medical birth registry. Scand J Soc Med 1990;18:143–8.
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=2367825&link_type=MED&atom=%2Fbmjopen%2F4%2F7%2Fe005388.atom) 
    
    [Web of Science](http://bmjopen.bmj.com/lookup/external-ref?access_num=A1990DL06300009&link_type=ISI) 

25. Cunningham FG. Williams obstetrics. 3rd edn. McGraw-Hill, 2005.
    
    

26. Källén B. A birth weight for gestational age standard based on data in the Swedish Medical Birth Registry, 1985–1989. Eur J Epidemiol 1995;11:601–6.
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1007/BF01719316&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=8549738&link_type=MED&atom=%2Fbmjopen%2F4%2F7%2Fe005388.atom) 
    
    [Web of Science](http://bmjopen.bmj.com/lookup/external-ref?access_num=A1995TE53500019&link_type=ISI) 

27. Marsál K, Persson PH, Larsen T, et al. Intrauterine growth curves based on ultrasonically estimated foetal weights. Acta Paediatr 1996;85:843–8.
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1111/j.1651-2227.1996.tb14164.x&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=8819552&link_type=MED&atom=%2Fbmjopen%2F4%2F7%2Fe005388.atom) 
    
    [Web of Science](http://bmjopen.bmj.com/lookup/external-ref?access_num=A1996UX51600014&link_type=ISI) 

28. Grisaru Granovsky S, Gordon ES, Haklai Z, et al. Effect of interpregnancy interval on adverse perinatal outcomes—a national study. Contraception 2009;80:512–18.
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1016/j.contraception.2009.06.006&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=19913144&link_type=MED&atom=%2Fbmjopen%2F4%2F7%2Fe005388.atom) 
    
    [Web of Science](http://bmjopen.bmj.com/lookup/external-ref?access_num=000272479200005&link_type=ISI) 

29. Alexander GR, Kogan MD, Himes JH. 1994–1996 U.S. singleton birth weight percentiles for gestational age by race, Hispanic origin, and gender. Matern Child Health J 1999;3:225–32.
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1023/A:1022381506823&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=10791363&link_type=MED&atom=%2Fbmjopen%2F4%2F7%2Fe005388.atom) 

30. Swamy GK, Edwards MJ, Gelfand A, et al. Maternal age, birth order, and race: differential effects on birthweight. J Epidemiol Community Health 2012;66:136–42.
    
    [Abstract/FREE Full Text](http://bmjopen.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NDoiamVjaCI7czo1OiJyZXNpZCI7czo4OiI2Ni8yLzEzNiI7czo0OiJhdG9tIjtzOjI1OiIvYm1qb3Blbi80LzcvZTAwNTM4OC5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

31. Elshibly EM, Schmalisch G. The effect of maternal anthropometric characteristics and social factors on gestational age and birth weight in Sudanese newborn infants. BMC Public Health 2008;8:244.
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1186/1471-2458-8-244&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=18638377&link_type=MED&atom=%2Fbmjopen%2F4%2F7%2Fe005388.atom) 

32. Voskamp B, Kazemier BM, Ravelli AC, et al. Recurrence of small-for-gestational-age pregnancy: analysis of first and subsequent singleton pregnancies in The Netherlands. Am J Obstet Gynecol 2013;208:374.e371–6.
    
    

33. Kramer MS, Séguin L, Lydos J, et al. Socio-economic disparities in pregnancy outcome: why do the poor fare so poorly? Paediatr Perinat Epidemiol 2000;14:194–210.
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1046/j.1365-3016.2000.00266.x&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=10949211&link_type=MED&atom=%2Fbmjopen%2F4%2F7%2Fe005388.atom) 
    
    [Web of Science](http://bmjopen.bmj.com/lookup/external-ref?access_num=000088859700003&link_type=ISI) 

34. Auger N, Park AL, Harper S, et al. Educational inequalities in preterm and term small-for-gestational-age birth over time. Ann Epidemiol 2012;22:160–7.
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.2188/jea.JE20110068&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=22285866&link_type=MED&atom=%2Fbmjopen%2F4%2F7%2Fe005388.atom) 

35. Shah PS, Zao J, Ali S, Knowledge Synthesis Group of Determinants of preterm/LBW births. Maternal marital status and birth outcomes: a systematic review and meta-analyses. Matern Child Health J 2011;15:1097–109.
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1007/s10995-010-0654-z&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=20690038&link_type=MED&atom=%2Fbmjopen%2F4%2F7%2Fe005388.atom) 
    
    [Web of Science](http://bmjopen.bmj.com/lookup/external-ref?access_num=000294825900033&link_type=ISI) 

36. Castro T. Single motherhood and low birthweight in Spain: narrowing social inequalities in health? Demogr Res 2010;22:863–90.
    
    

37. Fraser AM, Brockert JE, Ward RH. Association of young maternal age with adverse repreductive outcomes. N Engl J Med 1995;332:1113–18.
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1056/NEJM199504273321701&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=7700283&link_type=MED&atom=%2Fbmjopen%2F4%2F7%2Fe005388.atom) 
    
    [Web of Science](http://bmjopen.bmj.com/lookup/external-ref?access_num=A1995QU30200001&link_type=ISI) 

38. Odibo A, Nelson D, Stamilo DM, et al. Advanced maternal age is an independent risk factor for intrauterine growth restriction. Am J Perinatol 2006;23:325–8.
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1055/s-2006-947164&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=16799913&link_type=MED&atom=%2Fbmjopen%2F4%2F7%2Fe005388.atom) 

39. Campbell MK, Cartier S, Xie B, et al. Determinants of small for gestational age birth at term. Paediatr Perinat Epidemiol 2012;26:525–33.
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=23061688&link_type=MED&atom=%2Fbmjopen%2F4%2F7%2Fe005388.atom) 

40. Rogers JM. Tobacco and pregnancy: overview of exposures and effects. Birth Defects Res 2008;84:1–15.
    
    

41. Kabir Z, Daly S, Clarke V, et al. Smoking ban and small-for-gestational age births in Ireland. PLoS ONE 2013;8:e57441.
    
    

42. Mitchell EA, Thompson JM, Robinson E, et al. Smoking, nicotine and tar and risk of small for gestational age babies. Acta Paediatr 2002;91:323–8.
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1080/08035250252834003&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=12022307&link_type=MED&atom=%2Fbmjopen%2F4%2F7%2Fe005388.atom) 
    
    [Web of Science](http://bmjopen.bmj.com/lookup/external-ref?access_num=000174840100017&link_type=ISI) 

43. Juárez SP, Merlo J. Revisiting the effect of maternal smoking during pregnancy on offspring birthweight: a quasi-experimental sibling analysis in Sweden. PLoS ONE 2013;8:e61734.
    
    

44. Li X, Sundquist K, Sundquist J. Risks of small-for-gestational-age births in immigrants: a nationwide epidemiological study in Sweden. Scand J Public Health 2012;40:634–40.
    
    [Abstract/FREE Full Text](http://bmjopen.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NToic3BzanAiO3M6NToicmVzaWQiO3M6ODoiNDAvNy82MzQiO3M6NDoiYXRvbSI7czoyNToiL2Jtam9wZW4vNC83L2UwMDUzODguYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

45. Pepe MS. The statistical evaluation of medical tests for classification and prediction. Oxford, New York: Oxford University Press, 2003.
    
    

46. Zweig MH, Broste SK, Reinhart RA. ROC curve analysis: an example showing the relationships among serum lipid and apolipoprotein concentrations in identifying patients with coronary artery disease. Clin Chem Aug 1992;38(8 Pt 1):1425–8.
    
    

47. Pepe MS, Janes H, Gu JW. Letter by Pepe *et al* regarding article, “Use and misuse of the receiver operating characteristic curve in risk prediction”. Circulation 2007;116:e132; author reply e134.
    
    [FREE Full Text](http://bmjopen.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiRlVMTCI7czoxMToiam91cm5hbENvZGUiO3M6MTQ6ImNpcmN1bGF0aW9uYWhhIjtzOjU6InJlc2lkIjtzOjEwOiIxMTYvNi9lMTMyIjtzOjQ6ImF0b20iO3M6MjU6Ii9ibWpvcGVuLzQvNy9lMDA1Mzg4LmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

48. Royston P, Altman DG. Visualizing and assessing discrimination in the logistic regression model. Stat Med 2010;29:2508–20.
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1002/sim.3994&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=20641144&link_type=MED&atom=%2Fbmjopen%2F4%2F7%2Fe005388.atom) 

49. Gerds TA, Cai T, Schumacher M. The performance of risk prediction models. Biom J 2008;50:457–79.
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1002/bimj.200810443&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=18663757&link_type=MED&atom=%2Fbmjopen%2F4%2F7%2Fe005388.atom) 
    
    [Web of Science](http://bmjopen.bmj.com/lookup/external-ref?access_num=000258634300001&link_type=ISI) 

50. Ananth CV, Vintzileos AM. Distinguishing pathological from constitutional small for gestational age births in population-based studies. Early Hum Dev 2009;85:653–8.
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1016/j.earlhumdev.2009.09.004&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=19786331&link_type=MED&atom=%2Fbmjopen%2F4%2F7%2Fe005388.atom) 
    
    [Web of Science](http://bmjopen.bmj.com/lookup/external-ref?access_num=000272011400011&link_type=ISI) 

51. Boyko EJ, Alderman BW. The use of risk factors in medical diagnosis: opportunities and cautions. J Clin Epidemiol 1990;43:851–8.
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1016/0895-4356(90)90068-Z&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=2213074&link_type=MED&atom=%2Fbmjopen%2F4%2F7%2Fe005388.atom) 
    
    [Web of Science](http://bmjopen.bmj.com/lookup/external-ref?access_num=A1990EC89200002&link_type=ISI) 

52. Wald NJ, Hackshaw AK, Frost CD. When can a risk factor be used as a worthwhile screening test? BMJ 1999;319:1562–5.
    
    [FREE Full Text](http://bmjopen.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiRlVMTCI7czoxMToiam91cm5hbENvZGUiO3M6MzoiYm1qIjtzOjU6InJlc2lkIjtzOjEzOiIzMTkvNzIyNC8xNTYyIjtzOjQ6ImF0b20iO3M6MjU6Ii9ibWpvcGVuLzQvNy9lMDA1Mzg4LmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

53. Khoury MJ, Newill CA, Chase GA. Epidemiologic evaluation of screening for risk factors: application to genetic screening. Am J Public Health 1985;75:1204–8.
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.2105/AJPH.75.10.1204&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=3862352&link_type=MED&atom=%2Fbmjopen%2F4%2F7%2Fe005388.atom) 
    
    [Web of Science](http://bmjopen.bmj.com/lookup/external-ref?access_num=A1985AUE6300013&link_type=ISI) 

54. Cook NR. Use and misuse of the receiver operating characteristic curve in risk prediction. Circulation 2007;115:928–35.
    
    [Abstract/FREE Full Text](http://bmjopen.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MTQ6ImNpcmN1bGF0aW9uYWhhIjtzOjU6InJlc2lkIjtzOjk6IjExNS83LzkyOCI7czo0OiJhdG9tIjtzOjI1OiIvYm1qb3Blbi80LzcvZTAwNTM4OC5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

55. Zernicka-Goetz M, Huang S. Stochasticity versus determinism in development: a false dichotomy? Nat Rev Genet 2010;11:743–4.
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1038/ni.1897&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=20877326&link_type=MED&atom=%2Fbmjopen%2F4%2F7%2Fe005388.atom) 
    
    [Web of Science](http://bmjopen.bmj.com/lookup/external-ref?access_num=000283052800001&link_type=ISI) 

56. Merlo J, Wagner P. The tyranny of the averages and the indiscriminate use of risk factors in public health: a call for revolution. Eur J Epidemiol 2013;28(Suppl 1):148.
    
    

57. Merlo J, Wagner P, Juarez S, et al. The tyranny of the averages and the indiscriminate use of risk factors and population attributable fractions in Public Health: the case of coronary heart disease. Working paper version 2013–09–26. *Unit for Social Epidemiology, Department of Clinical Sciences, Faculty of Medicine, Lund University*. 2013; [http://www.med.lu.se/english/klinvetmalmo/unit\_for\_social\_epidemiology/abstracts\_c\_working\_papers\_c](http://www.med.lu.se/english/klinvetmalmo/unit\_for\_social\_epidemiology/abstracts\_c_working_papers_c).
    
    

58. Cook NR. Use and misuse of the receiver operating characteristic curve in risk prediction. Circulation 2007;115:928–35.
    
    [Abstract/FREE Full Text](http://bmjopen.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MTQ6ImNpcmN1bGF0aW9uYWhhIjtzOjU6InJlc2lkIjtzOjk6IjExNS83LzkyOCI7czo0OiJhdG9tIjtzOjI1OiIvYm1qb3Blbi80LzcvZTAwNTM4OC5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

59. Pencina MJ, D'Agostino RB Sr., D'Agostino RB Jr., et al. Evaluating the added predictive ability of a new marker: from area under the ROC curve to reclassification and beyond. Stat Med 2008;27:157–72.
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1002/sim.2929&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=17569110&link_type=MED&atom=%2Fbmjopen%2F4%2F7%2Fe005388.atom) 
    
    [Web of Science](http://bmjopen.bmj.com/lookup/external-ref?access_num=000253098900001&link_type=ISI) 

60. Pencina MJ, D'Agostino RB, Pencina KM, et al. Interpreting incremental value of markers added to risk prediction models. Am J Epidemiol 2012;176:473–81.
    
    [Abstract/FREE Full Text](http://bmjopen.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NzoiYW1qZXBpZCI7czo1OiJyZXNpZCI7czo5OiIxNzYvNi80NzMiO3M6NDoiYXRvbSI7czoyNToiL2Jtam9wZW4vNC83L2UwMDUzODguYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

61. Pencina MJ, D'Agostino RB Sr., Steyerberg EW. Extensions of net reclassification improvement calculations to measure usefulness of new biomarkers. Stat Med 2011;30:11–21.
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1002/sim.4085&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=21204120&link_type=MED&atom=%2Fbmjopen%2F4%2F7%2Fe005388.atom) 
    
    [Web of Science](http://bmjopen.bmj.com/lookup/external-ref?access_num=000285846700002&link_type=ISI) 

62. Hilden J. On NRI, IDI, and “good-looking” statistics with nothing underneath. Epidemiology 2014;25:265–7.
    
    

63. Pepe MS. Problems with risk reclassification methods for evaluating prediction models. Am J Epidemiol 2011;173:1327–35.
    
    [Abstract/FREE Full Text](http://bmjopen.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NzoiYW1qZXBpZCI7czo1OiJyZXNpZCI7czoxMToiMTczLzExLzEzMjciO3M6NDoiYXRvbSI7czoyNToiL2Jtam9wZW4vNC83L2UwMDUzODguYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

64. Hidden J, Gerds T. Evaluating the impact of novel biomarkers: do not rely on IDI and NRI. Denmark: Department of Biostatistics, University of Copenhagen, 2012; Research Report 12/08.
    
    

65. Kramer MS. Determinants of low birth weight: methodological assessment and meta-analysis. Bull World Health Organ 1984;65:663–737.
    
    

66. Frisell T, Öberg S, Kuja-Halkola R, et al. Sibling comparison designs: bias from non-shared confounders and measurement error. Epidemiology 2012;23:713–20.
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1097/EDE.0b013e31825fa230&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=22781362&link_type=MED&atom=%2Fbmjopen%2F4%2F7%2Fe005388.atom) 
    
    [Web of Science](http://bmjopen.bmj.com/lookup/external-ref?access_num=000307560700010&link_type=ISI) 

67. Cnattingius S, Ericson A, Gunnarskog J, et al. A quality study of a medical birth registry. Scand J Soc Med 1990;18:143–8.
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=2367825&link_type=MED&atom=%2Fbmjopen%2F4%2F7%2Fe005388.atom) 
    
    [Web of Science](http://bmjopen.bmj.com/lookup/external-ref?access_num=A1990DL06300009&link_type=ISI) 

68. Grove A. Rethinking clinical trials. Science 2011;333:1679.
    
    [Abstract/FREE Full Text](http://bmjopen.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6Mzoic2NpIjtzOjU6InJlc2lkIjtzOjEzOiIzMzMvNjA1MC8xNjc5IjtzOjQ6ImF0b20iO3M6MjU6Ii9ibWpvcGVuLzQvNy9lMDA1Mzg4LmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

69. Kooperberg C, Ruczinski I, LeBlanc ML, et al. Sequence analysis using logic regression. Genet Epidemiol 2001;21:s626–31.
    
    

70. Janes H, Pepe M, Kooperberg C, et al. Identifying target populations for screening or not screening using logic regression. Stat Med 2005;15:1321–38.