Given $v = kv_e$, where $k < 1$. Thus $v < v_e$. From conservation of mechanical energy, $\frac{1}{2}mv^2 - \frac{GMm}{R} = -\frac{GMm}{(R+h)}$. $\Rightarrow \frac{v^2}{2} = \frac{GM}{R} - \frac{GM}{(R+h)} = \frac{GMh}{R(R+h)}$. Since $v_e^2 = \frac{2GM}{R}$, we have $\frac{1}{2}k^2v_e^2 = \frac{GMh}{R(R+h)} \Rightarrow \frac{1}{2}k^2(\frac{2GM}{R}) = \frac{GMh}{R(R+h)} \Rightarrow k^2 = \frac{h}{R+h} \Rightarrow k^2(R+h) = h \Rightarrow k^2R = h(1-k^2) \Rightarrow h = \frac{Rk^2}{1-k^2}$.