Electrode holder with rinsing device