The answer is: keep both maxed.
Swords overall do more damage per hit, but daggers attack faster, plus you get the same amount of damage from your en spells. At level 16, I'm finding that daggers are more useful, since I 1) Don't hit very well with swords, 2) Don't hit for much when I do hit anyway, and 3) think en-spells rock.
But at high levels, there are some realyl cool looking swords. Sign me up for a flueret (sp), heh. Looks pretty spiffy.

"Hard Work Often Pays Off After Time, but Laziness Always Pays Off Now."
Minari @ Midgardsormr - Currently: 57RDM/28WHM
370+ Job Levels, Nothing over 57. Permanewb Extraordinaire!