The user in TFS/VSTS has all the rights enabled to delete work items. The problem is that when you open the work items the "Delete" button is missing, and you do not have the option to delete in query lists.
When you look at the inherited rights you see something like this:
All indication is that the permission is allowed, but the end result is that it is denied.
This may be that the user is in the "Stakeholder" access level. If you pay close attention to the "unavailable features" you will notice that deleting work items is one of the things a stakeholder can't do.
You can now either acquire a license for the user and move him/her into the basic access level or higher, or the user will need to ask someone that does hove those rights to perform the deletions.