He was born in Semley, Wiltshire, England. Morley attended Elizabeth College (an independent school in Guernsey), followed by RADA, and made his West End stage debut in 1929 in Treasure Island at the Strand Theatre and his Broadway debut in 1938 in the title role of Oscar Wilde at the Fulton Theatre. Although soon won over to the big screen, Morley remained both a busy West End star and successful author, as well as tirelessly touring.

A versatile actor, especially in his younger years, he played roles as divergent as those of Louis XVI, for which he received an Academy Award Nomination as Best Supporting Actor (Marie Antoinette 1938). He gave powerful performances in Oscar Wilde (1960) and as a missionary in The African Queen (1951), but did not receive Oscar nominations for either.

As a playwright he co-wrote and adapted several plays for the stage, having outstanding success in London and New York with Edward, My Son, a gripping family drama written in 1947 (with Noel Langley) in which he played the central role of Arnold Holt. But the disappointing film version, directed by George Cukor at MGM Elstree in 1949, instead starred the miscast Spencer Tracy, who turned Holt, an unscrupulous English businessman, into a blustering Canadian expatriate.

Morley also personified the conservative Englishman in many comedy and caper films. He was also the face of BOAC (British Airways) in television commercials of the 1970s. British Airways: 'We'll take more care of you'. Later in his career, he received critical acclaim and numerous accolades for his performance in Who Is Killing the Great Chefs of Europe?. Renowned for repartee and for being an eloquent conversationalist, Morley gained the epitheton of being a "wit".

Morley was honoured by being the first King of Moomba appointed by the Melbourne Moomba festival committee and, in typical humility, he accepted the crown in bare feet. Morley was in Australia touring his one-man show, The Sound of Morley.

He married Joan Buckmaster (1910–2005), a daughter of Dame Gladys Cooper. Their elder son, Sheridan Morley was a well-known writer and critic. They also had a daughter Annabel and another son Wilton. He was appointed a Commander of the Order of the British Empire (CBE) in 1957. He was also offered a Knighthood during the Wilson government but declined it. He died in Reading, Berkshire from a stroke, aged 84. He was cremated and his ashes scattered across the graveyard of the parish church at Wargrave, Berkshire.