With a 70-year history, it’s at the forefront of today’s educational challenges

By Wendy Greenberg | Photos by Shutterstock

In 1947, a small nonprofit organization with a mission of advancing equity in education began its work in a brick building at 20 Nassau Street in Princeton. After more than seven decades, Educational Testing Service (ETS), located since 1964 on a scenic campus off Rosedale Road just outside of Princeton in Lawrence Township, still adheres to its original mission to “advance quality and equity in education” and “measure knowledge and skills, promote learning and performance, and support education and professional development for all people worldwide.”

But since the early years of ETS, the testing and assessment landscape has evolved. The topic of standardized testing has been in the news, both heralded and under scrutiny, with debates focusing on the importance to college and graduate school admissions, and whether the tests are indeed equitable to all. Like ETS in the US, most nations tend to have their assessment practices. For example, in Australia, any registered training organization often follows the four Principles of Assessment. They can be termed as fairness, flexibility, validity and reliability, which can provide the necessary parameters for checking an assessment whether in college or any official training program.

Even with its long history, ETS is facing forward, and has evolved as the needs of learners have changed. While it is considered the epicenter of research in and development of tests like the Graduate Record Exam (GRE) and TOEFL (Test of English as a Foreign Language) and development of test questions used on the College’s Board’s Scholastic Aptitude Tests (SATs), ETS is also immersed in conducting research to improve quality and equity in education.

Among topics for ETS researchers are how to best ensure that testing is fair; identifying promising teaching practices; why intercultural competence is important; what language skills are needed for effective communication; what information do colleges and grad schools need to make sure their students are successful; and how achievement gaps are reduced.

It becomes important to accurately set down these standards and gauge them to bring out the best material and plans for the future. For this, determining OKRs (Objectives and Key Results), which is often a corporate term, can prove to be beneficial. It can help streamline the requirements and make goals clear, which can help researchers come up with actionable results.

At the same time, ETS develops, administers, and scores more than 50 million tests annually in more than 180 countries at more than 9,000 locations worldwide, with some 150,000 new “items” or questions each year.

A philanthropic arm supports education locally and across the country. Led by Lenora Green, the Center for Advocacy and Philanthropy (CAAP), established 2013, provides grants to community nonprofits that raise awareness of educational issues, especially among underserved groups. CAAP collaborates with the Research and Development division to go out into the community and provide workshops conducted by ETS research scientists and assessment specialists. “This is one way in which we can ensure the community has a greater understanding of the impactful work that we do at ETS,” said Green.

A cadre of volunteers, said Green, supports education and social services in the local community. An individual employee giving campaign last fall raised close to $860,000, she noted.

During the annual giving campaign, ETS employees may donate to the ETS Employees’ Community Action Fund scholarship program. Since the inception of this program, employees have funded more than $1 million to nearly 250 students in New Jersey. This year 25 students were awarded scholarships ranging from $2,500 to $10,000. All 2019 awards totaled $100,000 and winners were from 12 New Jersey counties, seven from Mercer County.

Writing the Test Questions

In 1947, three organizations – the American Council on Education, the Carnegie Foundation for the Advancement of Teaching, and the College Entrance Examination Board – contributed testing programs, assets, and key employees to form ETS, believing that an independent, nonprofit organization devoted to educational research and assessment could best enable opportunity for all learners, regardless of income or social status.

Through this arrangement, ETS designs and delivers its own assessments such as the GRE, TOEFL, and others. The College Board owns the Scholastic Aptitude Test (SAT) and Advanced Placement (AP) tests. The College Board is the largest client for ETS, where ETS staff experts design and write test questions, among other tasks undertaken at the request of the College Board.

Writing the test questions posed to millions of students is a meticulous process.

“We need to make sure the questions are relevant to the intended purpose of the assessment. We also need to make sure they are fair,” explained Ida Lawrence, vice president of research and development. She is seen in the 2010 ETS film
The Life of an Item, which describes the test writing process, and, in the film, she says that “it’s vitally important that these items are fair because, taken together, they are the basis for a decision that will be made about the person who takes the assessment.”

The process begins with deciding what skills or knowledge the item is intended to measure, and whether the skills or knowledge are relevant for the test’s purpose. The process includes a sensitivity review to see if the question is fair and free from cultural bias. The rigorous process includes 10-15 sets of process points, as the item goes through multiple levels of review. The items are often pre-tested or tried out before they count toward a score.

A “red flag analysis” determines if the item is performing differently for different genders and populations.

In addition to development and scoring, the Research and Development division, which includes some 1,200 staff, also carries out research on learning and assessment.

Changing Priorities

To meet the changing needs of the educational community, ETS reframed its research agenda two years ago around nine educational challenges. Its priorities are to increase sustainable access to a diverse and high-quality teaching pool; to improve science, technology, engineering, arts, and mathematics (STEAM) pathways; to raise U.S. literacy levels; and to support U.S. English learners’ development of language proficiency.

Priorities also include raising global English-language proficiency; increasing the diversity and quality of admissions to higher education; promoting assessment and development of critical competencies in a global context; increasing workplace preparedness and success; and supporting underserved learners and those who teach them.

Much of ETS’s research is documented in a recent ETS-authored book, Advancing Human Assessment (Springer 2017, editors Randy E. Bennett and Matthias von Davier). The book describes continuous programs of research involving, for example, topics such as computerized adaptive testing, measuring aptitudes and abilities, and procedures to help ensure test fairness.

As it seeks solutions to today’s challenges, such as achievement gaps among racial and ethnic groups that could lead to inequities in global workplace opportunities, ETS is mindful that “employers want candidates who are technologically skilled, communicatively proficient, culturally aware and agile, and in possession of a range of other competencies” such as collaborative problem solving, said Lawrence.

New Methods of Learning and Other Roles

Education and technology landscape changes have resulted in new methods of learning and measurement. “Many years ago, ETS changed the assessment industry by introducing computer adaptive testing,” said Scott Nelson, senior vice president, strategy marketing and growth and CMO. “Today, that groundbreaking spirit continues to drive us forward into new ways to deliver effective solutions.”

In computer adaptive testing, the difficulty level of a test can change, based on how the test taker answers each question. “Today we are looking at assessment in an immersive environment,” said Nelson. “In an assessment where one can do a lab experiment in an immersive environment, we would be able to understand not only if the test taker got the correct answer, but we could learn from how they got the answer. Being able to have this type of information excites educators because they can better understand how students learn.”

The line between assessment and learning is blurring. “What if measurement became part of learning in the classroom?” said Nelson. “This is a critical question for us that will change assessment as we know it and we believe will be a welcome development for teachers.”

Recently ETS has offered Massive Open Online Courses (MOOCs) to help students prepare for tests, and is partnering with “education accelerators,” such as LearnLaunch, to identify start-ups in education that provide “access to innovation outside ETS, and allow us to share our capabilities with them,” added Nelson, as a few examples of the new solutions ETS is building and exploring “to ensure we continue to meet our mission using today’s technologies and innovations.”

Also, test-taking for the disabled has opened an entire research program into accessibility and the process for evaluating.

Several ETS centers and programs have more specific roles. The Institute of Student Achievement (ISA) has the mission of improving schools, providing coaching to teachers. The Policy Evaluation and Research Center (PERC) conducts research based on public policy issues such as closing achievement gaps for underrepresented populations. PERC’s clients include schools and colleges, foundations, professional associations, education organizations, state and federal research and education agencies, and private corporations.

The ETS Center for Research on Human Capital and Education publishes books and studies on opportunity, and gaps between degrees and skills. For example, one study looks at why someone with a college degree may not have the skills to succeed in the workplace.

ETS offers Massive Open Online Courses (MOOCs) to help students prepare for tests.

College Coaching Courses?

Despite the broad ETS portfolio, parents look to ETS for practical matters, like how students can improve test scores. ETS, according to its website guidance, does not recommend coaching courses. “Although the courses vary according to objectives, duration, and method, we believe that most of the claims made by commercial coaching companies are overblown. However, we do recommend that students prepare for any important test. The best way to do that is to familiarize yourself with the format of the test and to review the content areas the test will cover, and you can do this on your own. . . . Note, however, that last-minute cramming will never replace years of study and effort.”

Despite the focus of some on scores, the focus of ETS has always been more concerned with creating the best educational opportunities possible for the most people.

A plaque still found at 20 Nassau Street quotes Nancy S. Cole, fourth president, ETS, “What we teach, what we test, even what we aspire to is but a mere glimpse of the possibilities that lie within our students, our schools, and those great human questions that will outlast all our technologies.”