Qualtrics

Qualtrics


What is Qualtrics?

Qualtrics is a survey software package. After evaluating several survey software options, in the fall of 2011 Bloomsburg University purchased a site license for Qualtrics, which a committee BU faculty and administrator reviewers selected. They found it powerful and flexible, both for designing survey instruments and for analyzing results.

The Qualtrics site license makes it possible for units across campus to use shared survey software. There is no cost associated with using Qualtrics on campus. The availability of Qualtrics does not mean other tools (such as Survey Monkey and Google Docs) can no longer be used on campus.

Why use Qualtrics?

  • Feedback surveys: Qualtrics can be used for participant response surveys of various sorts, such as after programs or events. It can also be used for exit interviews and alumni surveys.
  • Surveys for scholarly and for student research: Qualtrics might be used for faculty research, research on BU, or research done by students as part of a course. As for all research involving human subjects, including research by students as part of a course, IRB approval will be required. Faculty must review and release Qualtrics surveys created by their students.
  • Testing: Qualtrics can be used to administer pre- and post-tests in courses, majors, or programs. It has been used to administer placement tests (so far, foreign language placement tests).
  • Rubrics — collecting and analyzing data: Those using rubrics may want to use Qualtrics, for ease of data collection and for later analysis. Qualtrics can be used for some of the functions offered by WayPoint, rubric software that was available through Blackboard and not continued with BOLT. While Qualtrics will not integrate with BOLT, it can be used to grade assignments. In Qualtrics this is an easy-to-use, built-in feature, and students can be shown whether they got questions right or wrong. It is quite flexible.

How do I get access to Qualtrics?

Qualtrics software is available for use by anyone on campus. To get a Qualtrics account, contact Louis Kramarz, Office of Planning and Assessment (lkramarz@bloomu.edu, 389-4225). You will be asked what you will be using Qualtrics for, notified about IRB (Institutional Review Board) requirements for research involving human participants (see IRB web page and associated policy 3990), and given some guidance about minimizing “survey fatigue.” Surveys developed through Qualtrics will be identifiable by respondents as originating at Bloomsburg University. The data will be retrievable only by the person who designed and administered the survey.

How can I learn how to use Qualtrics?

Qualtrics Qualtrics offers detailed guidance on how to set up surveys and how to use Qualtrics to analyze the results. Qualtrics training materials are located at: www.qualtrics.com/university/researchsuite.

Locally, Louis Kramarz (lkramarz@bloomu.edu, 389-4225) can also assist.

How do I get my data if I have used Qualtrics?

It is up to the creator and administrator of Qualtrics survey tools to retrieve and analyze data from Qualtrics.

One note about getting the data you want from Qualtrics: Survey responses are anonymous in Qualtrics unless setup is modified to collect identifying information. Therefore, if you want to connect responses to a particular respondent, you will need to include a request for identifying information in the survey you design, design the survey to track IP addresses of respondents, or set up users with passwords and accounts.

What is the relationship between Qualtrics and General Education Outcomes Assessment?

Qualtrics

Figure 1: From the Qualtrics General Education reporting form for English 152: Literature and Society, Fall 2012.

In establishing mechanisms for reporting rubric-based General Education assessment data for the 2012-2013 pilot year of General Education assessment, the Office of Planning and Assessment has used Qualtrics. The rubrics associated with courses approved for General Education Points have been entered into Qualtrics, and data about the number of students achieving at each level of the rubric is entered.

Who enters what kind of General Education assessment data into Qualtrics?

The way the data is entered varies by departmental preferences and procedures. In some departments, a single person enters data aggregated from several sections. In other departments, individual instructors enter data for their own sections. Some departments or courses are using sampling, while some are reporting data from every student. The Qualtrics survey associated with assessment of a given course reflects discussion between the Office of Planning and Assessment and departmental assessment committees or representatives.

How does the General Education data gathered through Qualtrics get reported, and to whom?

The Office of Planning and Assessment gives tallies of General Education assessment data collected through Qualtrics to two places: to the General Education Council about all courses, and to each department about its own courses. The raw assessment data entered into Qualtrics cannot be seen by anyone. Rather, the Office of Planning and Assessment aggregates the data and provides the GEC and departments with tallies based on these aggregated data. The data reports, which include tallies and no analysis, are provided to departments and the GEC via TracDat (more on TracDat, the database being used to house assessment information, below).

The format of Qualtrics data reports — currently in Excel form — is, like everything else about General Education and its assessment, new and in pilot form: the format can be changed to meet departmental needs. Please contact Sheila Dove Jones (sjones@bloomu.edu, 389-4027) or Louis Kramarz (lkramarz@bloomu.edu, 389-4225) with questions, requests, or suggestions, or to ask about data you think you should have received but cannot locate in TracDat.

How does Qualtrics data relate to the whole outcomes assessment process?

Data from Qualtrics is only data: while it is, ideally, a useful part of the assessment process, it is not equal to it. General Education Council and departmental analysis and discussion of the data, and action taken in response to it, will make it meaningful. Departments and the GEC will be in charge of communicating with each other and with other entities, such as colleges, about what they see in their data, and how they want to respond to it.

Must Qualtrics be used as part of outcomes assessment for majors, for other units, and for courses not approved for General Education Points?

Departments and units may choose to use Qualtrics for other kinds of assessment, but it is not required. Qualtrics can be helpful for keeping track of direct measures of student learning: it may work well for tallying student achievement as rated on rubrics, for instance. It also can be used as a standard survey tool for indirect measures of student learning, like exit surveys and alumni surveys. Some departments have begun using it to administer pre- and post-tests, and others for academic placement. Many approaches to outcomes assessment data collection are useful and workable, and departments should choose methods that make the most sense for them.

What is the relationship between Qualtrics and TracDat?

As explained above, Qualtrics is not a required component of outcomes assessment plans, except for courses approved for General Education Points. However, the use of TracDat, a different software tool from a different company, is a necessary part of outcomes assessment at Bloomsburg University: TracDat is the tool used to record and communicate outcomes assessment.

TracDat

What is TracDat and where do I go to sign in?

TracDat TracDat is essentially a sophisticated database system, designed specifically for documenting assessment. TracDat is being used at BU as a place to store outcomes assessment plans, data, examples, analyses, and action plans.

If you have been given access to TracDat, you can sign in at bloomtracdat.passhe.edu/tracdat/ Participation was high: data were submitted for 92% of courses giving general education points.

Access to TracDat is password-protected, limited based on departmental decisions and levels of authority at the University, and tailored to particular users. For instance, units may opt to give an assessment leader the ability to read, modify, and add to information in TracDat, and to give read-only access to the rest of the unit.