Asking stakeholders is good. Often then will have no idea, or say something generic like "make it fast".

Try to pin them to something more concrete.

"So how about adding this new feature makes X happen no more than 5% slower than it currently does"