Scouring the internet for a good way to unpin these two dreaded items from the taskbar in Windows 10 was not an easy task. There were many old forum posts and solutions that didn't work because they tried to use just Powershell commands.

Alas, I come across a TwistedTek forum post by the site's Admin, syntax, with a godsend script. I condensed it a bit so there's no error reporting.

$appname = "Microsoft Edge"
((New-Object -Com Shell.Application).NameSpace('shell:::{4234d49b-0245-4df3-b780-3893943456e1}').Items() | ?{$_.Name -eq $appname}).Verbs() | ?{$_.Name.replace('&','') -match 'Unpin from taskbar'} | %{$_.DoIt(); $exec = $true}
$appname = "Store"
((New-Object -Com Shell.Application).NameSpace('shell:::{4234d49b-0245-4df3-b780-3893943456e1}').Items() | ?{$_.Name -eq $appname}).Verbs() | ?{$_.Name.replace('&','') -match 'Unpin from taskbar'} | %{$_.DoIt(); $exec = $true}

In detail:
It enumerates the Shell/taskbar COM OBJECT namespace
Matches the item with the name $appname (in this case, Edge and Store)
Gets the verbs of that com object
Executes the verb "Unpin from Taskbar"

Source: twistedtek