Idle Talk and Idle Thought

If you aim to avoid idle talk, then you ought also aim to avoid idle thought. A maxim to mind:

Avoid the near occasion of useless conversation.

This applies both to conversation with others and with oneself. The latter is avoided by internal situational awareness which is classically enjoined by:

Guard the mind.

Not easy. It is easy to avoid others, but not easy to avoid one's garrulous self.