Of all the medieval instruments, to me , the lute is the nicest sounding-I have always enjoyedlute music.
However, with its many (and closely-spaced) strings, learning and playing the lute must be very difficult. The guitar is a much easier instrument to play-at what point did it replace the lute?
Incidentally, is any modern music being written for the lute? 
I don’t know when the transition happened, but I believe the guitar eclipsed the lute because it’s louder. The lute does have a lot of strings, but (unless I’m mistaken) the lute’s strings are paired, like those on a mandolin, so you really play half as many frettings as the number of strings. The lute’s hand-carved round body doesn’t speak as loudly as the guitar’s big 8-shaped body, so it limited the number of people who could hear the lutinist.
For the luthier (the builder) the guitar is simpler to build. The sides are sawn rather than carved. The back, and often the front, is flat. The guitar’s neck is simpler than the lute’s, because the angle to the peghead is straighter. Then there’s the tuning pegs; only six, cheaper.
I’d like to have a lot of scholarly cites for you, but I have none.