Murphy entered politics in 1952 by joining the leadership of the California Republican Party, having also directed the entertainment for the Eisenhower-Nixon inauguration that same year.

In 1964, he was elected as a Republican to the Senate, having defeated Pierre Salinger, the former presidential press secretary in the Kennedy White House, who had been appointed several months earlier to serve the remainder of the late Clair Engle's unexpired term. Murphy served from January 1, 1965, to January 3, 1971. Murphy assumed his seat two days early, when Salinger resigned from the seat to allow Murphy to gain an edge in seniority. Murphy was then appointed by Democratic Governor Pat Brown to serve the remaining two days of Salinger's term.

Murphy was in demand for a time to assist other Republican candidates seeking office. In 1966, he hosted a fundraising dinner in Atlanta, for U.S. Representative Howard "Bo" Callaway, the first Republican candidate for Governor of Georgia since Reconstruction. In the election, Callaway outpolled Democrat Lester Maddox, but did not get a majority, and the state legislature elected Maddox.

In 1967 and 1968, Murphy was the chairman of the National Republican Senatorial Committee. During his Senate term, Murphy developed throat cancer, and part of his larynx had to be removed. For the rest of his life, he was unable to speak above a whisper.

Murphy voted in favor of the Voting Rights Act of 1965 as well as the confirmation of Thurgood Marshall to the U.S. Supreme Court, and the Civil Rights Act of 1968.

In 1970, Murphy ran for re-election; he was challenged by Democratic US Representative John V. Tunney, the son of famed heavyweight boxing champion Gene Tunney. Murphy's surgery and staunch support for the lingering Vietnam War worked against him, as did reports that he had continued to receive a salary from Technicolor after taking office. Tunney's successful Senate race in 1970 was reportedly the inspiration for the 1972 Robert Redford film