The following is a list of hidden references that can be found around the island.
The body of Tobus Turnicus can be found near the Wyrmstooth Crypt exit southeast of Wyrmstooth Barrow. The body will be wedged in a tree surrounded by junk, a note can be looted from him describing his final thoughts.
Once the mining settlement rebuilds you can visit the home of Bazrob while he's out working the mines during the day. In his home you'll find his journal that speaks of his destiny to become the king of nipples among other indelible thoughts. Another note can be pickpocketed off Bazrob himself.
The Young Turks
Travel to Dunyick's Camp then search the muddy area for a skeleton. A note can be found beside it referencing the surnames of TYT hosts in a short spiel about politics in The Elder Scrolls.
The skeleton of Francis surrounded by the corpses of many draugr can be found on the east side of the island near the coast north-east of Bloodfrost Burrow. It should be north of the shrine containing a Windhelm Guard then down the glacial cliff. A note can be found on the skeleton mentioning a futile search for the 'dew of the mountain'.
Once the mining settlement has rebuilt you can enter the Handsome Hermit Inn. On the counter is a note about a soldier heading into war, referencing the lyrics of the Manowar song 'Warriors Of The World'.
A sliced-up corpse can be found in the troll cave of Coldwater Crescent near the Stormcloak Camp on the north-west side of the island. Search the crates nearby for a poetic letter to Olga.
The Innkeeper of the Handsome Hermit Inn is named Sigur, his wife is named Rose.
In the Luminatory down in Dimfrost you can find the body of Vork on a table in the room before the thresher corridor. Beside his body is his journal which explains his demise. A Necklace of Peerless Haggling can be looted off his body.
A location on the east side of the island 'Haetar's Cave' is inspired by an episode of The Guild (season 6) where an irate game designer creates a cave filled with carnage dubbed the Hater's Cave. True to this the Haetar's Cave in Wyrmstooth is splattered with blood, guts and corpses.
A note can be found at Wanderer's Camp, southwest of Fort Valus. The note writes about an unnamed individual and his misfortuntes at the hands of Zaragoza and DeFranco and their mischief-making.
Baldur's Gate II
In the side-quest "Someone With Backbone" you are tasked with finding and returning the remains of a talking skull that turns out to be a powerful lich; an ode to Kangaxx from Baldur's Gate II.
There are a few more references that I may have forgotten about, search around the island and add them to the list if you find any.