I was the same 6 months ago.
Its worth looking at your technique. If you can only do 4 lengths, and your otherwise relatively fit, your probably wasting a lot of energy. Check out the swim technique videos on here or youtube.
The best place to start is slowing your stroke down, and making sure your legs arn't sinking. check out this youtube channel
Love Tri Swim Werkx
stick with it...
Get some lessons / training and some stamina.
If youre struggling with swimming more than 4 lengths, I'd say you werent a good swimmer and need some practice!