Instrument Overview:
Cycle 1 of the Domain-Informed Instruction Collaborative utilizes a fixed-form diagnostic assessment designed to measure student performance across four defined language domains.
The instrument consists of:
24 multiple-choice items
6 items per domain
Parallel pre- and post-instruction forms
A1–A2 proficiency band
The assessment is designed for completion within a 25–30 minute classroom period.
Domain Structure (Cycle 1: Sentence Clarity + Flow)
1. Sentence Types and Structures
Measures the ability to:
Identify simple, compound, and complex sentences
Combine sentences for clarity
Recognize sentence fragments
Items are structured to isolate sentence-level clarity without introducing advanced vocabulary or mechanical complexity.
2. Grammar Mechanics
Measures ability to identify:
Capitalization
End punctuation
Comma use in compound sentences
Fragments and run-ons
Mandarin support is provided in this section header to reduce misunderstandings about terminology that are construct-irrelevant.
3. Sequencing and Transitions
Measures:
Logical ordering of events
Recognition of transition words
Paragraph flow in short narrative passages
Passages are limited to 60–90 words to maintain a manageable cognitive load.
4. Main Idea and Details
Measures:
Identification of central topic
Recognition of supporting details
Detection of irrelevant information
Passages range from 80–120 words and use concrete, accessible content (e.g., animals or simple science topics).
Design Principles
Construct Validity Considerations
The instrument was constructed according to the following principles:
Equal item distribution across domains
Single construct per item
Controlled vocabulary (A1–A2)
No adaptive routing in pilot phase
Parallel structure for pre/post reliability
The goal is not comprehensive proficiency measurement, but domain-sensitive growth detection within a short instructional window.
Parallel Form Construction
Form A and Form B:
Maintain identical domain structure
Use parallel item types
Maintain comparable difficulty bands
Use different content to reduce memorization effects
This structure supports repeated administration while preserving measurement integrity.
Limitations of the Instrument
Fixed-form design (non-adaptive)
Limited item count per domain
Focused on discrete skill measurement
Not designed as a placement or summative proficiency tool
The instrument is exploratory and domain-specific in purpose.
Pre/Post Parallel Design
To support repeated measurement:
Form A and Form B maintain identical domain distribution.
Item types remain consistent across forms.
Content is altered while preserving the difficulty range.
No items are repeated between forms.
This design seeks to balance test familiarity with construct stability.
Measurement Sensitivity
With six items per domain, the instrument prioritizes:
Detecting directional change
Comparing domain-level responsiveness
Supporting descriptive growth analysis
The instrument is not designed for high-stakes evaluation or fine-grained proficiency scaling.
The Cycle 1 instrument is designed to measure discrete, domain-specific language skills rather than broad proficiency.
Each domain represents a defined construct:
Sentence Types and Structures → sentence-level syntactic clarity
Grammar Mechanics → surface-level mechanical accuracy
Sequencing and Transitions → logical cohesion within narrative flow
Main Idea and Details → paragraph-level comprehension structure
The intent is to isolate skill domains sufficiently to detect domain-specific instructional responsiveness.
Construct Isolation
To reduce construct contamination:
Vocabulary difficulty is limited to A1–A2 level.
Only one primary construct is assessed per item.
Grammar items avoid reading-level complexity.
Reading items avoids advanced grammatical traps.
Mechanical errors are not embedded within structure-based items.
This separation supports a clearer interpretation of domain growth.
Linguistic Load Control
Because the pilot population operates within an A1–A2 proficiency range:
Sentences are limited in length.
Passage length is controlled (60–120 words).
Content topics are concrete and familiar.
Grammar-domain headers include brief Mandarin clarification to reduce terminology confusion.
Mandarin support is intentionally restricted to grammar sections to avoid compromising reading-domain construct measurement.
Threats to Validity (Acknowledged)
Several limitations are recognized:
Small sample size (single classroom in pilot phase)
Fixed-form design (non-adaptive)
Limited item count per domain
Researcher-teacher role overlap
Short instructional duration (once weekly)
eDash Tutoring Services