Throw in the towel

A discarded towel
The phrase throw in the towel means to give up. It means that you are stopping because something is too difficult.
This saying originates from boxing, where a boxer would throw down his towel to say he is surrendering.

He almost threw in the towel, but kept going.