After the sheer brilliance of Earthshock, it is utterly gobsmacking to find such a duff and dire story following on its coattails. I have a hard time believing that Earthshock and Time-Flight are both Doctor Who stories, let alone two stories of the same season. On it's original broadcast, Time-Flight would have almost completely wiped the memory of Earthshock from existence, despite having finished airing just two weeks after Earthshock ended.
Time-Flight also has one of my least favourite music scores ever. Roger Limb's stuff was decent in The Keeper of Traken, and quite effective in Four To Doomsday. But his score in this story is tinny and full of what sounds like synthesized slide whistles. It's the first of a trilogy of weak scores from Limb, and one that, like the story itself, follows something that was much, much better.
The sad thing, though, is that Time-Flight marks the end of an era for Doctor Who. Episode One achieved a viewing figures total of 10.1 million, the last time Doctor Who would ever break the double digit barrier until 2005's Rose. The rest of the episodes in Time-Flight fared almost as well, hovering around the nine million mark. On the whole, 10 million viewers per episode was a common number for Season 19. When Season 20 started in January 1983, that average number would be closer to 7 million, where it would hover around until the end of Season 22. Where did those three million viewers go? Why did they never come back? Was it The Doctor? The stories? The productions values? The twice weekly broadcast schedule?
There may be too many factors to determine one specific reason, but one thing was for sure - Doctor Who was becoming a niche show almost overnight.