Charles Barkley declined Travis Kelce and Taylor Swift’s wedding invitation for a simple reason