The plug's outer plastic housing is a pretty close fit to the plastic socket surround. Some are closer than others. Some plugs also have a matt finish which sticks more than the smooth ones. I have used paraffin /wax bicycle chain lube on the plastic of the handle on our J1772 plug from time to time..
The other thing to notice is that the handle is well below the plug. When you pull on the handle you will be pulling up on the plug as much as out. This causes it to bind. I make a habit of grasping the plug up by the release button. This allows me to pull the plug straight out.
I have a Duosida "portable" L2 16 amp charger I used to charge my 2017 Bolt and now my 2020 Bolt. The charge head was always tight to engage and difficult to take out. I could see that it was not tight around the perimeter housing so I thought the 3 larger "female" connector holes in the charger might be a little tight on the matching male connectors in the car plug. I tapped a center punch into those three larger connector holes on the plug head - to spread them out just a touch. That made a world of difference. It's no longer tight but just snug - easy to plug in and take out.
Wow, thanks for those hints. I have a Duosida also and have always struggled with the tight fit on the Bolt, C-Max and 500e. The center punch worked to make it a little looser but when I sprayed the dry lube, it made a much bigger difference.