We’re not throwing in the towel over a minor mishap

Throw in the towel 举白旗
It means to quit.This phrase comes from boxing. When a boxer is too beat up to continue, his coach throws a towel into the ring to signal that the fight is over.

Note:

Throw in the towel的意思是认输，来源于拳击比赛。当拳击手打不过对方，他的教练会抛一条毛巾入场，表示认输。

例：The company threw in the towel after losing all of its major customers.
公司丢了所有的主要客户之后只好举白旗认输了。

好像应该是“我们不应该忽视细节”