Anyone who says don't go here, haven't given it much of ago. I have been there quite a few times and plan on taking my kids aswell.
You can get either a ferry over which takes longer or be quicker and take the Hovercraft from Southsea which takes you to Cowes.
There are plenty of places to see, too name a few there is Ryde which is a nice place to go to,the needles and the Black gang Chine theme park.
There are plenty of hotels to stay in from B&B's to five star hotels catering for everyone. It isn't a expensive place but you need a weekend too have a good look around the island.
They hold the Boat races here as the water is good here. so give it a try don't be put of, it may be a small island but people go back year after year and it gets busy in the summer.