First of all, that’s not right. The fight ends when Letho has 10-15% vitality left (unless you are playing in easy mode).
My own approach to Letho worked like a charm, this is how I did it:
I tried to have Quen up as often as possible, but managed well without it when I didn’t have the time to put it up.
When Letho uses Quen, just run around him (outside his reach), or from side to side in a flanking method. That way you can steer clear when he uses Aard and Igni. Letho’s Quen will not last for long, even if you don’t hit him, so just wait until it wears of. When that happens, you are ready to go at him.
Try to notice how Letho moves – you can predict his path of mayhem when he swings his sword in a whirlwind style. Dodge away from him, and either run or dodge to his back (dodging is faster if timed right). Hit him hard 1 time only, wait until Letho is ready to swing at you again, and the second he turns at you, dodge past him so you end up behind him and hit him hard again – repeat.
Do never use block against Letho, he will blast you with Aard and use Quen.
If Letho should be so lucky as to hit you with Aard or Igni while doging around him, get up and use Quen. Stay clear of him until his Quen is used up and if needed be, you can “semi-slow” him by using grapeshots or the like when his Quen is up.

Letho’s style: When you block or stand still, he will blast you with Aard or Igni. If you are too far away, he will throw bombs or poison at you.
Letho blocks most of the time, so repeatedly hitting him will not work.
When you are midrange to him, he will launch a whirlwind attack against you, if you are up and close, he will throw some heavy hits against your general direction. He uses Quen when you are blasted away by his Aard or his sword.

If done right (and with a little luck) you can take out Letho quite easily – but be prepared to take some hits once in a while (that’s why you use Quen as to not get blown back when he hits you).

Yrden is somewhat useful against him – I recommend not using it at all as it uses up potential Quen instead.