Iceland is full of geothermal heated pools or "lagoons." In fact, you can find them anywhere... not just in these 2 main attractions. As it is out of this world in either case, I would go for both. It is a once in a lifetime experience for sure. If you're fluctuating back and forth, here is my impression of the 2 lagoons.
Let me know in the comments if you've had any experiences at these lagoons, or any other lagoons!