This error occurs whenever a user doesn’t have Organization Admin rights in Azure DevOps. Either the users license level is stakeholder or the user is not part of Organizational Project Admin collection group.

So as soon as the user is added to Project Collection Admin group, the user will have access to delete the members from “Stakeholder”.