Preparing and Submitting a TESS Proposal
Any faculty member, postdoctoral fellow, or graduate student of any social science or social science-related department anywhere in the world. We regret that we cannot provide opportunities to any other individuals and groups.
Proposals are being accepted now and on a continuous basis. Excepting special competitions, TESS has no submission deadlines.
All proposals must be submitted through our proposal handling system.
Each proposal must also designate a contact author in the submission system. The contact author must be listed first on all proposal documents and is the person to whom all official TESS corresponds will be held.
Proposals are limited to five pages of text (including footnotes/endnotes), plus references, up to two pages of tables, and the actual survey items to be included.
Power analyses, which are encouraged, may also be referenced in the main text and placed in a short appendix that does not count against the five page limit. When revisions are invited to proposals, these may be accompanied by a memo that details changes; concision here is encouraged. The entire proposal with all appendices or supplements of any sort may not exceed twenty pages under any circumstances.
Proposals must be double-spaced and 12 point font. A smaller font is allowed in footnotes and endnotes, and there are not specific formatting requirements for tables.
Proposals that exceed these limits will be returned by TESS staff.
To be successful, a proposal must include: A title, provided at the top of the first page of the proposal. A thorough description of the study design. An explanation of how the study will make a valuable contribution to science and society. An explanation of how people in other scientific disciplines will benefit from this study. A request for a particular number of respondent-items with justification. An appendix with actual questions and description of stimuli. In sum, the proposed experiments must evaluate important and clearly-stated hypotheses and be likely to generate new and broadly-applicable knowledge. To preserve the anonymity of the review process, we ask that the main text of proposals be stripped of content that identifies the proposer. Since proposals are linked to their authors by their user profiles, there is no need to include any of this information in the proposal itself. Proposer names should not be listed on the front page or any page of the proposal, although references to previous research that are in stated in the third person are acceptable. If a proposal includes self-identifying content, it will be returned to the contact author along with a request that it be resubmitted without this information.
We seek proposals that break new ground in the hypotheses they investigate, the procedures they employ, or both.
The key to TESS success is to win over reviewers in your chosen field. Ideally, your proposal should offer the potential for a clear scientific advance whose relevance expands beyond any one discipline.
Proposals that report trial runs of novel and focal ideas will be viewed as more credible.
While not required, it is desirable if the proposal is conducted in coordination with non-TESS data collection endeavors, such as traditional laboratory experiments or field work.
Starting with proposals accepted in mid-2015, TESS proposals are made available a year after the data are delivered. They should be included along with study materials in the OSF pages for each project. (If you notice one is missing, please let us know.)
No. See the introduction for an elaboration of what we mean by an experimental design.
The "size" of a TESS experiment is a function of both the length of the experiment and the number of respondents (N). The shorter the experiment, the more respondents on which it can be conducted. This page provides the maximum N for studies of different length, as well as guidelines for how study length is calculated. Note that experiments that involve subsampling will involve some % reduction in the maximum N.
This allotment does not include the demographic and socioeconomic data that TESS provides for all studies. The standard delivery includes: Gender Race/ethnicity Age Education Household income Employment status Marital status # of members in household Housing type Home ownership Home internet access State of residence Census region Metropolitan Statistical Area (MSA) status Telephone service Device type (used to take survey) Party affiliation Political ideology Religious affiliation Religious service attendance
More information on the precise measurements provided for family income is available here.
If the population subgroup can be identified by data collected by NORC, it might serve as the basis for an experiment. One issue is whether there are enough members of the subgroup in the AmeriSpeak Panel, after taking usual patterns in fielding and recruitment into account. For an experiment of 500 individuals, our experience suggests that being able to field an experiment on a subgroup representing less than 10% of the general US adult population is very unlikely. For experiments that need 1000 respondents to be adequately powered, these figures should be doubled.
TESS provides a free service to investigators whose proposals are endorsed by the external reviewers, relevant Associate PI's and can meet standard human subjects requirements without placing an extraordinary burden on TESS resources. That TESS is a collective endeavor implies that there are strict limits on what services we can provide to any one investigator.
The resource limits stated herein are rTESS provides a free service to investigators whose proposals are endorsed by the external reviewers, relevant Associate PI's and can meet standard human subjects requirements without placing an extraordinary burden on TESS resources. That TESS is a collective endeavor implies that there are strict limits on what services we can provide to any one investigator. The resource limits stated herein are real. Proposals are more likely to succeed, both in the review process and then once out in the field, if these limits are strictly interpreted.eal. Proposals are more likely to succeed, both in the review process and then once out in the field, if these limits are strictly interpreted.
A list of the items included for free in the TESS data delivery from NORC is provided here.
TESS is conducted in the course of NORC AmeriSpeak Panel surveys, which also include sets of "profile" variables. Investigators in TESS studies may add profile variables to their studies, which are counted as just a fraction of an original survey item. The profile variables expected to be of most interest to TESS investigators are those on the Public Affairs Profile, the Health Profile, and the Financial Profile.
In a limited number of cases, TESS can provide additional respondent-questions. Such requests, however, are required to pass higher review standards than regular proposals. If the request entails substantial additional costs on TESS, we will have to reject the proposal or ask the proposer to pay the additional cost. More systematically, opportunities for larger studies have been provided through TESS special competitions.
Researchers who intend to employ open-ended questions in their surveys should be aware that the maximum character limit for responses to such questions is 4000 characters. Researchers who anticipate needing over 1000 characters for these questions, or who will use prompts longer than a few sentences, should specify this in their proposal. Such questions may count as multiple units.
TESS can provide samples of subpopulations, depending upon the type of subpopulation sought and the expense involved in reaching an adequate number of people within such a group. A few things of note on subpopulations: First, not all subpopulations can be reached with an adequate sample size, given the extant panel. Second, if the request is feasible but entails substantial additional costs for TESS, we will have to reject the proposal or ask the proposer to pay the additional costs. Finally, if a subsample is taken, we cannot guarantee it will be perfectly representative of that subpopulation since the probability sample is based on US population, in full, and not particular subsamples.
An endowment experiment is an experiment in which a real-stakes reward is offered to participants. For example, participants may be offered a choice between some payoff for sure and a larger payoff that is subject to a gamble, where a payment in real money is made to the participant in accordance with their choice. TESS can be used to perform such experiments, but the investigator will have to provide funds (1) to cover the actual payments to respondents and (2) a 15% surcharge to NORC for handling the distribution of payments to respondents. In other words, if an endowment effect distributes $5 on average to 1000 respondents, the investigator will need to provide $5750 ($5000 for the payments and $750 for the surcharge).
All TESS studies require IRB approval from the investigator's home institution and so would any deception would likewise need to be approved. In addition, any deception would need to be approved for fielding by NORC, which has conducted studies that provide a short 1-2 paragraph textual debriefing afterward.
A common experimental technique is to have subjects write for a few minutes in response to a prompt, for example in order to induce a particular emotional state. We think such manipulations are very valuable when they work, but have become worried that they are not a good fit for TESS. Many respondents on web panel surveys often provide extremely cursory responses, and stipulations of our survey vendor do not allow investigators to press them to spend a particular amount of time on the item or write responses of a particular length. Of course, if the writing task is not taken seriously, there is no chance for the experiment to work, even if the hypothesis is correct. While we do not prohibit proposals for such experiments, we have become wary of fielding them without reason to believe they will surmount the problem of engaging participants sufficiently for the manipulation to work.
Yes. For an extra charge, TESS can provide information on response times in milliseconds and can also present stimuli to experiments for a length of time specified in milliseconds.
TESS has run experiments that include a component that links to software implemented outside the AmeriSpeak platform. An example would be a survey that includes a component linking to an Implicit Attitudes Task. For purposes of calculating length with respect to TESS's size limits, each minute of task time is counted as 4 units.
Maybe! The platform can be used to do this for information at various geographic levels, down even to the census tract. However, this must be done in a way that the information can still be included as data that TESS can release publicly following the embargo period. For any geographic information below the state level, the information given to a respondent cannot be so specific that it might identify their area uniquely. For example, if an experimental stimulus was to include information about the racial composition of a census tract, the information that the tract was "56.45% White" might identify it uniquely, as opposed to "55-60% White."
Yes. In a randomized block design, randomization is done within categories of a variable and can improve the efficiency (power) of a study. Blocking can be done on profile variables that have been collected by NORC prior to the administration of your study.
There are no limits on the number of times investigators may use TESS. In fact, we encourage investigators to build on their previous TESS findings for subsequent proposals.
To submit a proposal to TESS's Short Studies Program please follow the instructions provided here.
In some cases, yes. NORC’s AmeriSpeak Panel is a multi-mode panel, in which respondents may take surveys by web or phone. Multi-mode surveys can be advantageous in reaching a representative sample of U.S. adults, including non-internet households. However, not all projects are suitable for phone mode, such as experiments that use visual stimuli and TESS Short Studies, which are often bundled with other short studies. Projects will incur a 20% surcharge for including phone mode. Those additional costs may require you to reduce the sample size or provide supplemental funding to cover the difference, if the overall project reaches our limits. When you submit your project description, be sure to include that you would like your study conducted by web and phone. Otherwise, we will assume applicants prefer web mode only.
Follow-up waves with TESS are possible in theory. In practice, these additions tend to be expensive and researchers can expect imperfect retention rates to decrease the overall sample size.