(1) Pilot applicants who pass a written examination and whose scores are among the top 20 (or such other number as may be set by the board) of those taking the written examination (plus any pilot applicants who tie a qualifying score) shall be eligible to take the simulator evaluation set forth in this section.

(2) The simulator evaluation shall take place at a marine simulator facility designated by the board and shall be recorded. In this evaluation pilot applicants shall be observed by available board members but shall be evaluated only by those available board members who hold, or have held a minimum U.S. Coast Guard license as master of steam or motor vessels of not more than 1600 gross tons. The board, with guidance from its contracted psychometrician, shall appoint a minimum of two additional evaluators who hold, or have held within 10 years of the examination date, a state pilot license issued by Washington or another state. Applicants for either Puget Sound or Grays Harbor shall be evaluated as one applicant pool.

(3) The simulator evaluation shall be designed to assess an applicant's competency in:

(a) Docking and undocking;

(b) Safe navigation;

(c) Shiphandling;

(d) Restricted waterway transit; and

(e) Other tasks deemed by the board, in consultation with its contracted psychometrician and a subject matter expert (SME) panel, to be necessary to the performance of safe, efficient, and competent pilotage service.

(4) The design of the exercise shall be conducted by a SME panel, consisting of at least two pilots. The board may appoint additional SMEs to the panel. The panel will work closely with the psychometrician to minimize bias as well as to maximize measurement efficiency. Subsequently, the simulator facility development team shall work with the panel and the psychometrician to develop, beta test, and finalize the exercise with the accompanying scoring rubric. All SMEs will sign confidentiality agreements and will not disclose any content of the simulator evaluation.

(5) All pilot applicants will be graded on the simulator exercise based on a validated rubric created by a SME simulation development panel, with assistance from the psychometrician. The simulator rubric will provide objective measurement points that map to the shiphandling tasks described in subsection (3)(e) of this section.

(6) A preliminary cut score will be derived via psychometric practices (e.g., a modified-Angoff methodology) prior to the administration of the simulator evaluation. Following an item analysis (conducted by the contracted psychometrician) and subsequent item revisions or removal recommendations, a secondary cut score will be recommended to the board by the psychometrician. The board will then approve a final cut score, taking into account the preliminary cut score, the item analysis results, and the recommendations of the psychometrician.

(7) The board may require that the cost of the simulator evaluation be at the expense of the pilot applicant.