But we also sometimes move in the opposite direction. We “transition from patent nonsense to something which is disguised nonsense” (see PI §524).
That is, that transition in the opposite direction has a point.
Addendum:
Even if all philosophical confusions involve some idling of language, it doesn’t follow that all idling of language involves philosophical confusion.