Analyses

Under Results > Analyses, it is possible to analyse the results of an assessment matrix, assessment session or paper assessment. To view information about an assessment matrix, assessment session or paper assessment, first select the desired tab. A selection must then be made from the dropdown list(s). The overview will look different for each tab; this article explains the data for each tab.

Tab Blueprint/ Test moment / Paper-based exams

Once you have selected one of the tabs, information about the relevant key will be displayed. A field may appear at the top showing response model changes for specific candidates, blueprints or test moments baed (only when adjustments have been made). As an example, see the below image.

Filters are available to display more specific results for this assessment matrix. The filters that can be used are: period, user groups, candidate, status and evaluations. Multiple filters can be used at the same time.

Further down, an overview is shown of how candidates scored on this assessment matrix. This can be seen in the image below. On the left is a graph. The horizontal axis shows the percentage of the total number of points that can be achieved (maximum score). The vertical axis shows the percentage of candidates who fall within this score range. 

The pie charts show the proportion of candidates who scored a pass or a fail. One pie chart is always displayed for the test result (the one on the far right). If feedback categories have also been set up (and are permitted to be displayed), these are shown first. In this case, there are three feedback categories. In this example, just over half of the candidates achieved a pass.

Please note!

Whether a pie chart for a feedback category is displayed, when it is displayed, and whether colours are shown in the pie chart depends on the settings for the relevant feedback category (see Feedback categories). To display a pie chart: tick the box for ‘Score range name, as defined below’ in step 3 of the settings.

The pie chart displays only shades of grey if you have chosen not to apply a cut-off score (see image on the right). If a cut-off mark is applied, green or red colours are displayed.

The reliability of the test (blueprint) is calculated using Cronbach’s alpha:

See the image below. The section on the left shows the reliability when no corrections have been made to the answer key (uncorrected). The view on the right shows the reliability after applying active corrections to the results. The adjustments cause the reliability and the standard error of measurement of the assessment to change. An icon with a heart is displayed between the two options. This can be used to switch between the two possibilities. Click the icon, after which it will move to the right-hand side of the unselected option.

Overviews

At the bottom of the overview, there is a table containing various details for each candidate. This table will again contain a number of tabs. Which tabs are available depends on the settings. By default, the 'Overview' tab is open. The 'Analysis details', 'Group analysis' and 'Analyse questions' tabs will also be available.

Overview

This tab displays the results for each candidate. For each candidate, the start time of the test (blueprint/ test moment) is shown, along with the time taken to complete the test. If feedback components have been added to the test, there will be a column showing the results for each added feedback component. Hovering over the coloured bar reveals how the candidates answered the questions for each section. The column with the heart icon above it indicates the reliability score, but only if this can be calculated for this test. This is not always the case; composite cut-off scores will not have a reliability score shown here. Finally, there are columns showing the candidate’s score, the status of the test and, on the far right, the details of the result (for the candidate in question).

Analysis details

This tab contains an overview of the results of all candidates for this assessment matrix. If there are different feedback components, these can be found using a dropdown list under the 'Feedback' filter.

The data presented here can be exported as raw data. At the bottom of the page, select the 'Export raw data to Excel' action.

Group analysis

Group analysis can be used to compare candidates who have been divided into user groups at group level. This makes it possible to analyse how the groups scored in relation to one another. A candidate can also be linked to different groups, allowing comparisons to be made in different ways. For example, groups can be compared based on level, location or year of study. To compare user groups, they must be linked to the same category. For more information about creating categories for user groups, see Delivery environment: User groups.

The group analysis displays a summary of the results of the assessment session. The summary includes, among other things, the average score and score range for all candidates. If an assessment contains feedback components, the same information is also shown for each feedback component.

To compare groups, the category for the user group must be selected from the drop-down list at the top of the page. A category could, for example, be ‘Location’, with three user groups: ‘Amsterdam’, ‘Rotterdam’ and ‘The Hague’. When the ‘Location’ category is selected for the comparison, the results for the three groups are displayed side by side. Green and red icons indicate how the value in the box compares to the average value of all results (green indicates a higher score, red indicates a lower score). Please note that the colour does not imply any judgement as to whether the result is better or worse!

See the image below for an example of a group analysis by location.

Analyse questions

The Analyse questions tab contains various data and statistical values calculated by Remindo for each assessment version. For an explanation of the statistical values, see the corresponding article: Statistical values.

The final column also contains three options. 

• The details of this question can be viewed here; 
• This displays the interaction analysis for the question – here you can find further information about the question, including the number of times each answer option has been selected and various statistical values. It is also possible to make an response model change from here. An example view is shown below
• This allows you to send a comment about the question to the author of the Manager environment.

Comments

If candidates have made any comments regarding questions during the examination, an additional column will be added to this overview. The "Comments" tab contains all comments made by candidates during the test take or review moment.

A candidate can post one comment per question during the test take. A tutor can respond to this comment once. After that, no further communication can take place via Remindo.

Comments can be filtered by question, user group, specific review moment and status. A comment can have different statuses, which are explained below:

• Open This comment is still open and has not yet been dealt with.
• Read The candidate has read the response to the comment.
• Answered The tutor has answered the comment, but the candidate has not yet marked it as read.
• Closed The comment has been closed without a response being given.

In the overview, there are buttons on the right to respond to, edit or close a comment. It is also possible to go directly to the result details.

Actions per overview

Various actions are available at the bottom of the page of each tab.

Actions for the 'Overview' tab

In the Overview tab, the following actions are available for exporting:

• Export to PDF: this allows specific parts of the result to be omitted. The PDF file can therefore be customised more closely.
• Export to Excel: nothing can be adjusted with this option, and after a short wait a download will automatically appear in the browser.
• Export to Osiris (worksheet or text): this export can be used in combination with Osiris.
• Download uploads as ZIP file: this function includes files uploaded by candidates in a structured ZIP file.

Actions for the 'Analysis details' tab

The following actions are available:

• Export question scores to Excel: this exports only the question scores.
• Export raw data to Excel: this action places 26 types of data in the Excel file. Use this to download as much data as possible about a result in a single document.
• Export to JSON file: a .JSON file is created in which the data is stored. This file format can be used in other applications.

Actions in the 'Comments' tab

Actions are also available in the 'Comments' tab:

• Answer the comments: this answers all selected comments at once. It is not possible to respond to a comment that has already been answered.
• Close comment(s): this action closes the selected comments without posting a comment.
• Reopen comment(s): this action can be used to reopen a closed comment so that it can still be answered.
• Export to Excel: this exports all information relating to the comments to an Excel file.

Please note!

When a test take is interrupted without a grade being calculated, this result is also not included in the various analyses. This can be recognised by the comment on the 'Analyse questions' tab stating that not all answers are used in the analysis.