Currently, creating a Page Filter on the Admin > Fields page like "Entity Type" "contains" "Connection" does not work.
We've heard feedback that it would be great to easily filter by Connection Entities on the Fields page so that it's easier to manage these Connection Entity fields.
As a little workaround, you can add a filter widget for Entity Type, and then enter "Connection" in the search box, and this will highlight all the Connection Entity Types (in addition to "Show more..."), but you'll have to select them all individually, and this isn't ideal.