Brown had played piano for Elvis Presley. He toured for much of Elvis's final two years of touring and was a part of the legendary "Jungle Room" sessions held at Graceland. In 1979 he joined Emmylou Harris's legendary Hot Band taking over for former Elvis sideman Glen D. Hardin, he stayed with the band until 1981. Later, he became a session musician in Nashville and toured with acts such as Rosanne Cash.
After leaving that position, Brown became a producer, producing albums for several artists, predominantly on the MCA Nashville Records label. Among these acts were: Tracy Byrd, Steve Earle, Vince Gill, The Mavericks, McBride & the Ride, Reba McEntire, Brooks & Dunn, Heidi Newfield and George Strait. In the 1980s, he was also the keyboardist in The Cherry Bombs, Rodney Crowell's backing band.