There are many family stories about how Joseph Tyson Stevenson arrived in Australia, and many possibilities outside of these stories. The information on the Stevenson page relating to a crew member named Joseph Stevenson jumping ship from the Hougoumont fits all of the remembered details of the most common story, but is only one of many possibilities.

That particular family story relates that by the age of sixteen Joseph was among the crew of a ship bound for Australia, though the ships name and destined port are uncertain. At the age of sixteen Joseph jumped ship in port somewhere in Australia, along with another young man and certain other crew members. They supposedly deserted at night, with Joseph left to loose the rope of the small boat used for their escape. They were captured after some short time, and presented with the option of returning to their vessel or being put in prison. Joseph apparently opted for prison, though the other young man is remembered as having returned to the ship. The gaoler was apparently a good man and soon took a liking to young Joseph, and an arrangement was soon made where by Joseph worked for the gaoler, tending his garden. It was understood by his descendants that Joseph had jumped ship in Portland, lured by the discovery of gold. This is the story handed down of the beginning of his new life in Australia.

This is the most common tale, but each family line remembers slightly different details. The main contender for the port at which Joseph jumped ship remains Portland, Victoria.

One version of Joseph's arrival in Australia relates that upon arrival in the Colony of Victoria he went by the name of Steele. Perhaps he took this name because he was a deserter, to avoid being captured for the crime, or perhaps because he had served time in a gaol for deserting and was seeking a fresh start with a new name. However, this version of the story does not remember him as a deserter at all. It is remembered that he was set to disembark from a ship at a certain port (possibly Geelong or Port Phillip), but that there was a certain young lady aboard the ship who was stricken with grief. She would sit on the deck and cry every day. Joseph felt deeply for her in her sadness, and when she disembarked at Portland he could not bear to be apart from her so he left the ship there also.

Official immigration records do little to clear up the mystery. The most likely candidate to be Joseph Tyson Stevenson was a Jospeh Stevenson who sailed from England on board the Epsom on 21 January 1855. The Epsom arrived Geelong, Port Phillip, on 27 April 1855. (April 1855 - 82/12) He was an Agricultural Laborer from Leicestershire, religon C. of E., who could read and write, aged 16 years. He was engaged by George Stevart of Bellerine Hills for a period of three months, for £65. Joseph Tyson Stevenson would have only been aged 13 at the time, so if it was him he lied about his age to make the passage. However, he did always work as a farm labourer, so this is possibility.