If you aim to avoid idle talk, then you ought also aim to avoid idle thought. A maxim to mind:
Avoid the near occasion of useless conversation.
This applies both to conversation with others and with oneself. The latter is avoided by internal situational awareness which is classically enjoined by:
Guard the mind.
Not easy. It is easy to avoid others, but not easy to avoid one's garrulous self.