Like any struggling or aspiring comedian in the nineteen seventies, David Letterman was taking any television gig he could get while appearing in the evenings as an unknown at The Comedy Store and other clubs out and around Hollywood.
It wasn't until one of the most influential comedy agents in the business, Jack Rollins, took him on as a client that things started happening. Letterman appeared on episodes of Mork and Mindy, The Gong Show, The Jacksons and several game shows. His appearance on Mary Tyler Moore's variety program Mary has been mocked on his talk show several times over the course of the last twenty years. He even competed on an edition of The Battle of The Network Stars! I haven't seen that footage so I'm not sure what event he participated in.
The most embarrassing program David appeared on was probably The Starland Vocal Band Show, a short lived variety program that revolved around the group who had the one-hit wonder, Afternoon Delight. Apparently Letterman was even a staff writer for that relic.
As a judge on a 1976 episode of The Gong Show:
David Letterman's hero was Steve Allen. Many of Letterman's most famous bits, not to mention the whole stylistic aspect of his show, are completely derivative of Steve Allen's work. So it must have been a huge thrill for Letterman when he appeared on this episode of Mork and Mindy alongside former Tonight Show with Steve Allen regular, Tom Poston.
Letterman on an episode of The Jackson Five comedy-variety program, The Jacksons. One of the few times you'll hear David playing to canned laughter: