In this article, we'll discuss how InnerOrbit produces dimensional data in the Dimension Report (also applies to admin's District Report).
We'll unpack this by answering:
How does InnerOrbit tag questions?
InnerOrbit roots assessment design in the Next Generation Science Standards (NGSS). We first start with a Performance Expectation (PE) and pay special attention to the "elements" within each dimension (the bulleted list under the dimension):
These elements guide the creation of a question! You will see both the overall dimension (ex: The Crosscutting Concept "Structure and Function") AND the elements tagged in every InnerOrbit question:
Keep in mind that InnerOrbit has scaffolded questions, meaning that questions build in rigor as the dimensions build.
Tip 1: The purple "EVS" bar are the NGSS Evidence Statements.
Tip 2: Click on the dimension bar to see the elements underneath! The number next to the bar hints at how many elements are present for that dimension.
How do those tags turn into dimensional data?
Dimensional data comes from tags assigned to each question. The percentage represents the number of correctly answered questions tagged with an element in that dimension.
Here’s how it works:
Each tag adds weight to its dimension.
If a question is answered correctly, it contributes "1/1" to each tagged element.
If answered incorrectly, it contributes "0/1".
The element percentage is calculated as total correctly answered questions divided by total assessed questions for that element.
Element percentages are summed to calculate the overall dimension percentage.
This means that dimension percentages are weighted based on the total possible units for each element, not a simple average of each element's percentage.
For example, this question is a 2D Content + Concepts question, meaning one "unit" will apply to both:
DCI: PS4.B Electromagnetic Radiation element "When light shines on an object, it is reflected, absorbed, or transmitted through the object, depending on the object’s material and the frequency (color) of the light."
In the Dimensions Report, 58% of questions tagged with that element were answered correctly. For the overall Dimension (PS4: Waves), the proficiency of 57% takes into account this element and OTHER elements assessed in this dimension.
How do scaffolded questions impact dimensional data?
Scaffolded questions from InnerOrbit vary in their impact on dimensional data. Some questions focus on just one dimension (e.g., Core Content 1D questions), while others cover all three dimensions (e.g., Sensemaking 3D questions).
This means that all "units" resulting from scaffolded questions are grouped under the same dimension. For instance, a DCI "unit" from a 1D question is aggregated with a DCI "unit" from a 3D question.
This approach helps in evaluating overall performance across dimensions in all questions and assessments. However, when tracking dimension proficiency growth between assessments, the design of the assessment itself becomes crucial.
For more details on how this affects assessments, refer to the next section focusing on Diagnostic assessments.
Tips for Diagnostic (1D) assessments
Assessments with complex 2D/3D questions will yield DCI student data that reflects their difficulty.
Conversely, assessments with only 1D questions (like a PreMade diagnostic) will show less rigorous data.
This difference might make it seem like your DCI data has declined when comparing a diagnostic assessment to a later one.
When comparing growth between a DCI-only diagnostic and another assessment with 2D/3D questions, consider the question rigor.
If you use diagnostics solely for instructional insight, no change is needed.
If you aim to measure dimensional proficiency growth, ensure the pre/post test rigor is similar for direct dimensional data comparison.
Need more help?
For additional assistance, click on "Talk to a person" in the site chat or email us at support@innerorbit.com.