It is shameful if some garages are charging over the odds for petrol on the Isle of Man during the TT. However as an Island resident (Yorkshire-born) for 5 years, I have learned to live with paying around 5p a litre more for petrol than on the mainland.
So I would love to say that it is greedy garage owners profiteering - but it is over-priced for the other 50 weeks of the year, too. I personally have not seen the price of £1.04 or £1.08, but paid £1.0199 on the Sunday after the first practice.
Try not to dwell on this negative aspect of your visit and take the locals for what most of them are: Warm, welcoming and genuine people who are enthusiastic about the TT and the visitors it attracts. Come, enjoy & ride safe!