The challenged ballots previously placed in separate envelopes shall be placed in a sealed envelope marked "challenged ballots" and sent along with the tally sheet to the chief election officer. The challenged ballots shall not be opened or counted unless the counting of such ballots might affect the results of the election. If the challenged ballots might affect the results of the election, the chief election officer shall conduct an investigation into, or if necessary a formal hearing on, the validity of the challenges made. If he concludes that the challenge was properly made, that ballot shall be excluded from the count. Otherwise, such ballot shall be counted as cast.