NBC News and PEOPLE Magazine commissioned Princeton Survey Research Associates International (PSRAI) to conduct a nationally representative telephone survey of young teenagers and their parents. The goal of this study was to ask teens aged 13-16 key questions about their sexual health, sexual activity, and the sexual pressures facing teens today. Parents were asked a similar but shorter set of questions. We asked parents how they thought teens today view sex, the sexual pressures they face and about the flow of communication between parents and teens. Princeton Data Source, LLC (a subsidiary of Princeton Survey Research Associates located in Fredericksburg, Virginia) conducted 1000 interviews with parents and teenagers aged 13-16 during the period of September 4, 2004 and November 7, 2004.
The margin of error for results based on the full sample of teens is ±3.4%. For teens aged 13-14 the margin of error ±5.0% and for 15 and 16 year-olds the margin of error is ±4.6%. The margin of error for results based on the full sample of parents is ±3.4%. Details on the design, execution and analysis of the survey are discussed below.
Design and Data Collection Procedures
Sample Design
The goal of this study was to conduct a nationwide survey of young teenagers (aged 13 to 16) on issues related to sexual health and activity. A companion survey was conducted among parents, in part to acquaint parents with the survey topic so they could give informed consent for their teen to participate and in part to provide a point of comparison against which to view teen responses.
The sample was designed to be generalizable to the population of young teens in the continental U.S. and to allow separate analyses, where possible, of teenagers 13-14 and 15-16. PSRAI accomplished this by using prescreened sample of households identified in PSRAI Demographic Tracking Surveys to be likely to have a child between the ages of 13 to 16. The PSRAI Demographic Tracking Survey is a short 10 minute nationally representative survey that asks about household composition including number of children, the age and sex of adult household members, the race and ethnicity of the respondent, the religion of the respondent and the total household income. The RDD sample for the survey was provided by Survey Sampling, Inc. (SSI) according to PSRAI specifications. To draw the sample every active block of telephone numbers (area code + exchange + two-digit block number) that contained one or more residential directory listing is equally likely to be selected; after selection two more digits are added randomly to complete the phone number. This method guarantees coverage of every assigned phone number regardless of whether that number is directory listed, purposely unlisted, or too new to be listed. After selection, the numbers are compared against business directories and matching numbers are purged.
Questionnaire Development and Testing
The questionnaire was developed by NBC and PEOPLE Magazine in collaboration with PSRAI. The questionnaire was pretested with a small number of parent and young teen respondents. Pretest interviews were monitored by PSRAI project staff and representatives from NBC and PEOPLE Magazine. Interviews were conducted using experienced female interviewers who had previous experience doing similar types of studies. These interviewers are particularly well suited to judge the quality of the answers given and the degree to which respondents understood the questions. After the pretest, a couple of questions were added to the instrument and a few were deleted. Additionally, some final changes were made to question wording and order based on the monitored pretest interviews. All interviews were conducted using a fully-programmed CATI instrument.
Contact Procedures
Interviews were conducted during the period September 4, 2004 through November 7, 2004. Only experienced female interviewers were used for this study. Interviews were first conducted with a parent at the sampled household. After completion of the parent interview, interviewers asked for permission to speak with either the oldest or the youngest child in the household age 13 to 16 to conduct a survey about the same issues. After obtaining consent from the parent, interviews were conducted with the teen respondent. If the teen respondent was not available, interviewers arranged a time to call back when the teen was likely to be at home.
As many as 10 attempts were made to contact a parent at every sampled telephone number. Sample was released for interviewing in replicates, which are representative subsamples of the larger sample. Using replicates to control the release of sample ensures that complete call procedures are followed for the entire sample and ensures that the geographic distribution of numbers called is appropriate.
Calls were staggered over different times of the day and days of the week to maximize the chance of making contact with potential respondents. Each household received at least one daytime call in an attempt to find someone at home.
Questionnaire Monitoring
During the first few weeks of the project, both the management of PDS and project staff at PSRAI listened to tapes of the interviews to make sure that the instrument was working as designed and to hear how teens were responding to the questions. In addition to the regular daily monitoring of interviewers by PDS supervisors, the tapes provided another layer of interviewer quality control.
Project staff at PSRAI continued to listen to tapes of interviews of both sexually active and inactive teens throughout the field period in an effort to gauge how comfortable teens were talking about their experiences and viewpoints. After having listened to interviews with approximately 25 sexually active teens and 25 teens who are not sexually active, PSRAI can say with confidence that teens were open and engaged in telling us about their experiences and viewpoints. While understandably some teens were a bit shy, these teens did not seem to be evasive or deceptive in their responses. When teens hesitated to respond to a question, the interviewer reminded the teen that they could skip over any question that made them feel uncomfortable or that they did not want to answer. These young teenagers appeared to have no difficulty telling interviews if they did not want to answer a question. At the same time, most teens completed the entire questionnaire, usually in an open, frank and matter of fact manner.
Because confidentiality and respect for respondents’ privacy is of paramount concern at PSRAI, tapes of interviews did not include any information that could identify respondents and tapes were promptly destroyed.
Weighting and Analysis
Weighting is generally used in survey analysis to compensate for patterns of nonresponse that might bias results. The interviewed sample of teens was weighted to match national parameters for sex, age, race, Hispanic origin, parent’s education, parent’s marital status and region (U.S. Census definitions). These parameters came from a special analysis of the Census Bureau’s 2003 Annual Social and Economic Supplement (ASEC) that included all households in the continental United States that had a telephone.
Weighting was accomplished using Sample Balancing, a special iterative sample weighting program that simultaneously balances the distributions of all variables using a statistical technique called the Deming Algorithm. Weights were trimmed to prevent individual interviews from having too much influence on the final results. The use of these weights in statistical analysis ensures that the demographic characteristics of the sample closely approximate the demographic characteristics of the national population. Table 1 compares weighted and unweighted sample distributions to population parameters.
Effects of Sample Design on Statistical Inference
Post-data collection statistical adjustments require analysis procedures that reflect departures from simple random sampling. PSRAI calculates the effects of these design features so that an appropriate adjustment can be incorporated into tests of statistical significance when using these data. The so-called "design effect" or deff represents the loss in statistical efficiency that results from systematic non-response. The total sample design effect for this survey is 1.21.
PSRAI calculates the composite design effect for a sample of size n, with each case having a weight, wi as:
In a wide range of situations, the adjusted standard error of a statistic should be calculated by multiplying the usual formula by the square root of the design effect (vdeff ). Thus, the formula for computing the 95% confidence interval around a percentage is:
where is the sample estimate and n is the unweighted number of sample cases in the group being considered.
The survey’s margin of error is the largest 95% confidence interval for any estimated proportion based on the total sample— the one around 50%. For example, the margin of error for the entire sample is ±3.4%. This means that in 95 out every 100 samples drawn using the same methodology, estimated proportions based on the entire sample will be no more than 3.4 percentage points away from their true values in the population. It is important to remember that sampling fluctuations are only one possible source of error in a survey estimate. Other sources, such as respondent selection bias, questionnaire wording and reporting inaccuracy, may contribute additional error of greater or lesser magnitude.
Response Rate
Table 2 reports the disposition of all sampled telephone numbers ever dialed from the original telephone number sample. The response rate estimates the fraction of all eligible respondents in the sample that were ultimately interviewed. At PSRAI it is calculated by taking the product of three component rates:
Contact rate – the proportion of working numbers where a request for interview was made – of 91 percentCooperation rate – the proportion of contacted numbers where a consent for interview was at least initially obtained, versus those refused – of 70 percentCompletion rate – the proportion of initially cooperating and eligible interviews that were completed – of 87 percent
Thus the response rate for this survey was 55 percent.
Analysis of respondent refusals
Surveys conducted on potentially sensitive topics such as sexual health and activity always raise the concern that respondents differ in some systematic way from those that refuse to participate. Mindful of this concern, we compared parents who completed the survey but refused their teen’s participation in the survey to respondents for which we have both a completed parent and teen interview.
We found no large systematic demographic differences between parents that granted permission for their teen to participate versus those that refused their young teen’s participation. Those that refused their teen’s participation are equally likely as those who permitted their teen to complete the survey to have a son or a daughter or be Catholic or Protestant. The two groups are also indistinguishable by their educational and ethnic/racial background. There is a small difference in terms of household income with lower income parents being a bit more likely to give consent for their teen’s participation. And although parents of the youngest teens are a little more likely than parents of 15 to 16 year-olds to refuse their teens participation, the difference is not large. Likewise, parents who refuse their teen’s participation are a bit more likely to live in the Northeast and less likely to live in the West, but there is no difference based on whether the household is in an urban, rural or suburban location that would suggest a systematic bias in the data.
An analysis of answers to the parent portion of the questionnaire comparing the responses of parents who granted permission for their teen to participate to those that refused their teen’s participation also show many similarities. There are, however, are a few small differences. Namely, parents who refused their teen’s participation are a bit less likely than those who’s teen completed the questionnaire to very often talk to their teen about sex and sexual relationships (31% vs. 39%), to strongly agree that waiting to have sex is a nice idea but not many teens do (38% vs. 45%) or believe their teen is sexually active beyond kissing (8% vs. 14%).
The final dataset contains paired teen and parent responses and does not include the cases in which we have a completed parent interview without an accompanying teen interview—272 parents who refused their teen’s participation and 133 parents granted permission but we were unable to reach the teen before the end of the field period. There were 59 teens that declined to participate after parental permission was granted.
[1] PSRAI’s disposition codes and reporting are consistent with the American Association for Public Opinion Research standards.
[2] PSRAI assumes that 75 percent of cases that result in a constant disposition of “No answer” or “Busy” over 10 or more attempts are actually not working numbers.
[3] Note: these percentages are based on unweighted data.