One of my major bugbears in Home Assistant (HASS) has always been that there's no proper way to completely delete a device or entity.
It seems so odd to me that there is just no functionality beyond "disable".

Note: It's been pointed out to me that this is only true for entities which are available. For those which are marked "unavailable", there is an option to delete. I will keep this up though because I'm sure it will still be useful for someone!

However, it's not quite true that there's no way to delete them... It is possible, but it can also be dangerous.

Entities and devices are stored in /config/.storage/core.device_registry and core.entity_registry respectively. Depending on the type, they have also have an entry in core.config_entries.

Below is the process to entirely remove a device and/or entity in Home Assistant.
Please note that I cannot take any responsibility if you break your HASS installation.

1. Backup.

Make sure you have a backup or snapshot of HASS from before you make these changes.
You can backup the entire machine it's running on, snapshot the VM (if it's on a VM), or take a snapshot in HASS's Supervisor tab.

2. Shutdown Home Assistant.

If you're running HassOS this may not be possible. I'm not familiar with the ins and outs of HassOS, having never used it myself.
You could try editing these files and restarting HASS, but I can't guarantee it will work.

In general it's not a good idea to modify these core files while HASS is running. Since they aren't supposed to be user-modifiable, HASS doesn't expect them to be changed by anything other than itself and therefore may not handle it nicely if they are changed while it's running.

3. Modify the files.

There are many ways to access HASS's config while it's not running, but probably the easiest in most cases will be SSH.
SSH to HASS's host system and locate your config directory. If you're running HASS in Docker, it will probably be: /usr/share/hassio/homeassistant/
Move to the .storage directory in your config directory and, with your choice of text editor, edit core.device_registry, core.entity_registry, and core.config_entries.
In each one, search for references to the name of the device or entity you want to remove and delete the entire section for that object.

Make sure you keep the syntax correct!
This is of the utmost importance. If you get this wrong, it will break.
If you're unsure, paste the entire file into JSON Lint which will validate JSON syntax for you.

4. Start Home Assistant.

With all luck, HASS will start back up without a care in the world, just the way it shut down but without those pesky entities you didn't want!

I always welcome feedback on my posts, please contact me if you have any. I'm also happy to answer any related questions if I know the answer.