How difficult something is has no bearing on whether or not it is a sport. Brain surgery, very complicated and difficult to master. Also, not a sport.
Golf is a hobby like gardening, knitting or basket weaving.
Are there gardening tournaments, The US Basket Weaving Open, where you play outside in let's say, 100 degree heat, for 4 days and compete against others for a championship and use your arms, back, legs, etc to walk and hit a golf ball??
I better look for the World Series of Knitting then...what a revelation!!
So just my merely USING my brain...ZERO physical skill at all, I can be a great golfer???