​​​​​​Secondary Scoring

​NOTE: Only Site Owners, selected staff, and KU staff can see the Secondary Scoring section. Go to Other Permission Roles​ to see how to grant permission to staff who need to do scoring but do not need full control of the site.

The Secondary Scoring feature of the Community Check Box (CCB) is designed to serve as a user-friendly platform for assuring the quality and accuracy of the data in Accomplishments entries. The scoring includes entering secondary scores, calculating rates of inter-observer reliability, viewing coding discrepancies, providing comments and feedback on documented events, reflecting on next steps for improving levels of inter-observer agreement, generating recommendations for improvement, and sharing overall feedback and ideas for improvement with those entering data.

Customized Secondary Scoring Sheets can be created for scoring and providing feedback on data (for example, on a monthly or quarterly basis). You set the parameters for which data you would like to review, and once Scoring Sheets are created, you can save and resume your work at any time. Data entered in Secondary Scoring is synchronized with CCB ​​Accomplishments, so you can select to display a green or red flag next to that entry in the Accomplishments listing. See Secondary Scoring Flags section below for details.

We will use the terms Observer 1, Observer 2, and Observer 3 to refer to the following:

• Observer 1: Primary Scorer
• Observer 2: Secondary Scorer
• Observer 3: Tertiary Scorer

CCB Administrators assign visibility and access to Secondary Scoring for users based on their specific needs and responsibilities. Secondary Scoring can be accessed from the left navigation bar in the CCB.​ From this Secondary Scoring Platform, secondary and tertiary scorers can easily access the data they need to review.

Secondary Scoring flags:

​	No Flag – Indicates the record has not yet been scored by Observer 2 or Observer 3
​	​Gr​een Flag – Indicates agreement on scoring by all Observers who have coded the entry​​​
​	​Yellow Flag - Indicates lack of agreement on scoring between Observer 2 and Observer 3
​	​Red Flag – Indicates lack of agreement on scoring between Observer 1 and Observer 2

​ 

Administration - for Site Owners

Secondary Scoring is an optional section available to all projects using the CCB.  By default, the section is not displayed until a determination has been made if it will be used by a project. Administrators need to determine who within the project will serve as Observer 2 and Observer 3 and provide access to the Secondary Scoring section. Documenters do not need access to this tool.

Administrators also need to determine if flags should be displayed within Accomplishments to identify if there is Observer 1 and Observer 2 agreement on entries.  If selected, display of the flags in the Accomplishments is a feature that needs to be enabled.  As Administrator, you can select whether your site users will be able to view flags which link to the related Secondary Scoring feedback.

Before you can use Secondary Scoring, you need to turn it on for the CCB form you will score.

1. Go to Administration.
   
2. Select Accomplishment Forms.
3. Click on the Form Name for the CCB you want and click Edit.
4. To enable Secondary Scoring slide the Secondary Scoring Enabled toggle to the right.
5. If you also want the scoring flags to appear on the Accomplishments listing page, slide the Accomplishment Flag Enabled toggle to the right.

Using a Secondary Scoring Scoresheet 

​NOTE: When you first click on Secondary Scoring, the only action link that will be available is New. After you select a previously created scoresheet, the other options of Edit, Delete, Scoring and Download will be active.

Create a new scoresheet

1. ​Click New.
   
2. Enter the Title.
   
3. Select the Date Range.
   
4. Enter your name as Observer.
   
5. Select your Observer Role.
   
6. Select the Form Name. When you do that two more fields will appear.
   
7. Check the Scoring Fields you want to use. Normally this is Code, or Activity Type.
   
8. Check all the Fields to View ​you want to use​. These are the fields that will help you decide if the scoring is correct. The description field is one of the best fields, but select any that will help you.
   

​ 

​Perform Scoring

To complete a score sheet, follow protocols outlined for your project to score an entry.

If a flag is present in the score sheet, this indicates the entry has previously been scored by another observer. You can change the score selected, or leave as-is.​

1. ​Click once on the scoresheet you just created and then click Scoring.
2. ​You will see the following table with the columns you selected for scoring and viewing.
3. Review the viewing fields you selected, then select the correct option under the scoring column(s), which all have a 2 after the question.​​
        NOTE: There is a separate scroll bar for the scoresheet.
4. If you want, add a comment in the Inner Observer Comment field for any entry. These comments can provide encouragement, or instructions for improving coding, ​for observer 1.
   
5. Continue until you have all the entries scored.
   
6. If you have an overall comment, add it in the ​Overall Comments field at the bottom of the screen.
   
7. Click Save when you are finished.
   
   

 ​​​