Methodology

The local television news project was begun in 1998 in order to study the relationship between content and viewership. Using criteria established by a team of industry professionals, we created measurements for analyzing these newscasts according to basic variables of quality and comparing those to accepted measurements of commercial success. Newscasts are studied in a market's highest-rated time slot for news, and the sample has remained the same throughout – two weeks of half-hour newscasts, one week during sweeps and one week during the regular season. The numbers have added up. In the past five years, we have studied more than 33,000 stories, gathering information on at least thirty separate variables for each – more than 1,000,000 pieces of data in all.

The study this year included a content analysis of broadcast news programs in seventeen cities, comprising fifty-three English-language stations and eight Spanish-language stations. Markets were selected to ensure both regional and market-size diversity. Newscasts were selected on the basis of the most popular time slot for news in each market and, in the case of markets where Spanish-language news programs were analyzed, on the basis of the most popular time slot for news in each language individually. Taping occurred during a March non-sweeps week and an April-May sweeps week. The final dataset comprised 9,173 stories, including 7,423 English-language stories, 1,094 Spanish-language stories, and a control sample of 656 stories in both English and Spanish. Content analysis was performed using standardized codebooks containing the criteria established by the project design team, and the results were used to assign quality scores and grades.

For Spanish-language stations, the Community Relevance variable was adjusted to reflect the needs and interests of the target audience. Additional codes were added to give appropriate weight to stories presenting information of particular importance to the local Spanish-speaking community, i.e., stories that provided information about either local Hispanic/Latino communities, or Spanish-language homelands.

Testing performed to measure uniform coding showed that daily scores were reliable within +/-0.49 points per day for all broadcasts. Results for Spanish-language broadcasts were within the intercoder performance parameters established over the five years of this study.

Ratings, share, audience retention and demographic data were generated by an independent researcher using Nielsen Station Index data for the newscasts studied in the twelve sweeps-period ratings books from February 1999 to November 2001 (February 2002 ratings information was not used to avoid viewership skewing effects of the Winter Olympics in Salt Lake City). Prior PEJ studies referred to in these articles used similar data.

This year's study also included a national mail survey of news directors, conducted between June and August 2002. A random sample of 233 news directors was selected from an enumerated list of broadcast stations that produce local news. One hundred three news directors completed the surveys, for a response rate of 44 percent. The sample of 103 news directors represents more than 15 percent of all stations producing local news. Since the survey respondents come from more than half of the local news markets throughout the country, the results are unweighted by market size. The news directors' survey has a margin of error of plus or minus 5 percent, which means that if we were to draw 100 different random samples the same way that we did in this instance, the results in 95 of those samples would not differ more than 5 percent either way from the results reported here.

Taping, Screening and Inclusion

Video Monitoring Services, Inc. taped newscasts in the designated markets during the following 2002 Monday-Friday time periods: March 4 –March 8 (non-sweeps primary) and March 11 – March 15 (non-sweeps secondary); April 29 – May 3 (sweeps primary) and May 6 – May 10 (sweeps
secondary).

For both monitoring periods, primary days were used, unless unavailable due to preemption or taping error. In those cases, broadcasts from the secondary taping period were substituted, making every effort to match the appropriate day of the week.

Note: the following substitutions were necessary due to taping error:

WTVF Nashville May 17 Sweeps
WSVN Miami May 30 Non-Sweeps
WTVJ Miami July 9 Non-Sweeps
WIS Columbia July 12 Sweeps