Scoring Model/Scripting

The Scoring Model is an Evidence Centered Design process by which to assess work products through the application of scoring rules and textual feedback. The Scoring Model is used to create complex rules for the grading of activities. When using the new Scoring Model, it is possible to display customized feedback for every work product in the assessment tree (both positive and negative feedback) as well as overall and partial feedback. It is not even necessary to link the feedback text to a work product if the purpose is to provide a simple message to the user.

The Scripting engine is a component of Packet Tracer that was designed to allow greater flexibility in creating scoring rules for Packet Tracer activities that use the new Scoring Model. The Scripting engine is what allows this complex scoring model to exist. Each item created in the Scoring Model exists as an object in the Scripting engine so that it can be accessed by any expression/scoring rule that the user creates.

An example of where the Scoring Model and Scripting engine would be beneficial is when an instructor wants to display a message to the user indicating that the IP addressing is mostly correct, but not completely correct and not totally incorrect. This may happen when there are several devices on the stage and one or two of the three are correctly addressed, but the third is not. Using the new Scoring Model, it is possible to provide this sort of feedback based on the combination of multiple work products. The major benefit here is the ease of customization in complex scoring scenarios, as well as a model that more closely aligns with the Evidence Centered Design process.

Scoring Model Interface

The Scoring Model tab allows you to define individual Work Product Features along with corresponding Scoring Rules. For each section, there are buttons to Add, Remove, Edit, and Move the work product features and scoring rules. Refer to the "Example" & "Terms" sections below for further explanation and terminology.

Scripting Interface

Packet Tracer comes bundled with four important scripts written in JavaScript. EventManager.js stores custom event handlers for Core Events. ProficiencyModel.js stores and evaluates the Scoring Model. ScoringModel.js holds the data classes for the Scoring Model. Main.js is the main function called from the core, where subsequent communication from the core is processed via Events. By default, Main.js supports the Scoring Model by adding events for saving, loading, evaluating, and reporting for the Proficiency Model.

In the Scripting tab, you can add, remove, edit, rename, import, and export scripts including the four core scripts described above. Optionally, you can click on the debug button to reveal a debug window, which can assist you in scripting.

In most use cases, you will not need to edit the four core scripts. However, should you have any reason to edit these scripts, each script is fully editable and removable per activity file. If you edited or removed any of the four core scripts and you want to restore them, go to Info tab and select PT 5.2 Activity from the template list to bring back the scripts to their original configurations.

In the screenshot below, an XML file called PROFICIENCY_MODEL is also in the list. Refer to the "Example" & "Terms" sections below for further explanation and terminology.

Example

An instructor wants to test that students know how to properly power on three devices in a network. However, the instructor wants to provide positive feedback if all devices are turned on, and a single negative feedback if at least one of the devices is not powered. This particular style of feedback doesn’t fit the standard Packet Tracer model of scoring and feedback so it is necessary to use the Scoring Model.

The instructor starts by placing three devices on the stage, Router0, Router1, and Router2. The Activity Wizard is then started and the existing network is to be used as the answer network.

In the Answer Network page, the instructor will select the Scoring Model tab instead of using the Assessment Tree tab. Here, the instructor will need to make a work product feature for each item that is to be retrieved from the user’s network. Since the activity is to test the power status of three devices, three work product features will be needed. For simplicity, the items created will be named r0power, r1power, and r2power.

To create one of these work product features, the instructor clicks the + button under the Work Product Features section. In the dialog that pops up, the name "r0power" is entered into the Name field. The Description is optional and left blank. For the expression, the instructor must use the help in the nodes box at the bottom to find the proper path to the network setting that is to be checked. By default, the search starts in the "Network" and moves into devices from there. Clicking after the string "Network:", the instructor then types any character to see the updated available nodes. Noticing that Router0 is available, the phrase is completed to form "Network:Router0:". Once the final colon is entered, the list of options inside Router0 is shown. Since, the purpose is to test the power, the final phrase is "Network:Router0:Power". The instructor would then create a work product feature for each of Router1 and Router2 as well.

Now that the work product features are created, the instructor needs to compose the tests for each of the work product features. This is done using the Scoring Rules section beneath that. According to Evidenced Centered Designed process, each work product feature must be assessed by applying a scoring rule to it, which creates a Primary Observable. Previously, the work product features just defined the items we were interested in. Primary observables take those interesting pieces of information and test them against some rule or expression. In this case, the instructor needs to determine if the devices are powered, which is to say that our work product features have the value '1'. To make naming simple, the primary observables are to be called r0poweron, r1poweron, and r2poweron.

To do this, the instructor clicks the + button under the Scoring Rules section, and selects Primary Observable from the Type drop-down list. In the Name field, "r0poweron" is entered. The Description again is optional so is not entered. Next is the expression. This is a JavaScript string that will be used to test the value of the work product feature. The template contains a portion for the condition, a true value, and a false value. The evaluation of this expression is directly assigned to the value of the primary observable created. For this instance, the instructor wants to assign the value 1 if the test is true and the value 0 if the test is false (it could also be assigned text strings, other numbers, scores, or other valid script commands as well). The condition part of the expression is to test to see that our work product feature r0power has the value '1'. Erasing the condition part of the template, it is replaced with "r0power == '1'". This directly compares the work product feature r0power against the string value of '1'. Since the instructor wants to assign the number 1 when true, and 0 when false, each of those portions is replaced with 1 and 0 respectively. This produces the final expression of "( r0power == ‘1’ ) ? 1 : 0;". This JavaScript command will assign the value 1 to r0poweron when the work product feature r0power is equal to '1', and the value 0 for everything else. The instructor then repeats the process for the other two primary observables.

Next, the instructor needs to create an indicator for all three devices. This particular type of observable is called a Compound Observable since it will be combining multiple primary observables in some manner into a single value. This value could be a score, or just another indicator of correctness. To keep things simple, the instructor has decided to create it as an indicator which will take on the value 1 if all three devices are turned on and 0 if at least one of the devices is off. The observable will be named "allpower".

To create this observable, the instructor again clicks the + button underneath Scoring Rules, but this time selects Compound Observable in the Type drop-down list. In the Name field, "allpower" is entered and the Description is left blank. Like the primary observable, the same template applies to compound observables. Here, the instructor needs to test that all three primary observables have the value 1 and assign the compound the value 1 if they do, or 0 if they do not. One way the instructor can do this is to add up the values of all three primary observables and determine if the value is 3. If it is, then all devices are powered on. If it is less than 3, at least one device is unpowered. The expression the instructor enters is the following "( r0poweron + r1poweron + r2poweron == 3 ) ? 1 : 0;".

Lastly, the instructor wants to display some feedback. Since Packet Tracer has only one text to display feedback in it is necessary to place the entire string into the unique Reporting Variable called OVERALL_REPORT. If all devices are powered on, the feedback should be "You have powered all devices." If at least one device is unpowered the feedback should read "X devices are unpowered. Please fix the error and try again." The value X should be the number of devices that are incorrectly powered.

Again, the instructor clicks the + under the Scoring Rules section and creates a Reporting Variable this time. In the Name field, the special name "OVERALL_REPORT" is used. The expression this time is a little more complicated since the observable allpower does not contain the number of devices, but a correctness indicator. It is possible to create another compound observable to show this, but the instructor decides to do the calculations right in the expression to save time. The resulting expression is:

This particular expression checks the value allpower. If it is true then it display the first string, otherwise it composes an alternate string that contains the value 3 – the number of correct devices (the number of incorrect devices).

When the instructor checks the Scripting tab, the instructor will notice that a new XML file called DATA: PROFICIENCY_MODEL has been automatically generated which includes all of the work product features and scoring rules that were created as seen in the screenshot in the "Scripting" section above.

Before the instructor can test the activity, the initial network is configured by copying it from the answer network and turning off the power to all three devices. Instructions are entered to tell the student what to do, and then the activity is tested.

When the instructor tests the activity without turning any devices on it displayed "3 devices are unpowered. Please fix the error and try again." However, if two devices are powered on, the feedback is "1 devices are unpowered. Please fix the error and try again." And lastly, when all three devices are powered on, the feedback is "You have powered all devices."

Terms

Expression - A sequence of JavaScript commands that results in a value that can be assigned to an observable.

Work Product Feature - A particular element of interest in the Packet Tracer network. For example, the IP address assigned to Router0’s FastEthernet0/0 port.

Primary Observable - The result of the application of a scoring rule or expression against a work product feature. Usually there is one primary observable for each work product feature, and typically the value is true or false. As an example, if the IP address on Router0’s FastEthernet0/0 port is equal to 10.0.0.1, then award 2 points.

Compound Observable - An application of aggregation rules to multiple Primary Observables. The values stored in a compound observable can be any legal JavaScript value but typically are numerical scores or correctness indicators. As an example, add the points from IP addressing and routing together.

Reporting Observable - Typically a string value named that represents some portion of feedback. For example: If the score is greater the 70, display "Passed".