This works fine for me, even on Windows 10. Does the YouTube URL even open and start to play at all? If not, make sure to use the latest version of the youtube.lua script. If it does play but you get no recording, please share the full verbose logs from Tools > Messages, so we can see what is going wrong exactly. Also, what language settings do you use on your system?
Even though it worked fine when I tried earlier once, I could later reproduce the issue. I believe this is a transient issue caused by #24957 (closed), which affects some videos more than others, although it's unrelated to emojis. However a quirk in a fallback mechanism caused the file name picked for the recording to be based on the video stream URL rather than the video title: this was fixed by https://git.videolan.org/?p=vlc.git;a=commitdiff;h=b42890f04ce0bd8e6cd383236ead8c82cb220323
There may be a second bug though: even with the wrong name, recording should still work. Here apparently that name fails with a file system error:
file error: cannot create C:\Users\Username\Videos\vlc-record-2020-09-18-23h55m19s-videoplayback_expire=1600491316&ei=1DplX5CzIP-1xN8P3ceCWA&ip=122.12.211.11&id=o-AOheogXeNVh9CVR0JDeV2Fjs5mjqLuLTsOgDiBRykMD3&itag=22&source=youtube&requiressl=yes&mh=Uw&mm=31,29&mn=sn-cu-abfe,sn-cu-ajtes&ms=au,rdu&mv=m&mvi=4&pcm2cms=yes&pl=25&initcwndbps=1251250&vprv=1&mime=video_mp4&ratebypass=yes&dur=294.521&lmt=1583593217222644&mt=1600469551&fvip=4&c=WEB&txp=5432432&sparams=expire,ei,ip,id,itag,source,requiressl,vprv,mime,ratebypass,dur,lmt&sig=AOq0QJ8wRAIgAWg-pP3wnYRw4j-SWRaU8VsM70KZG7sNPpUcaVo8xAsCIB4JJ2iGW2kmHS4kxlM5WR9HXvaBSSfiQHOLQ0L_-oUJ&lsparams=mh,mm,mn,ms,mv,mvi,pcm2cms,pl,initcwndbps&lsig=AG3C_xAwRQIhAIqFwhrSQplQqrEJAn9naqaG_DihpDnN3OkDa6ATO7FpAiAITt8z9M3Bm8TdMzwA3lE-LvUeDQcPL-FlVjNBC5f--Q==-.mp4: No such file or directory
I can reproduce on Linux, getting a File name too long error instead, so that's probably what's going on. The file name is already sanitized to remove troublesome characters, I guess it should also check that it's not too long, and anyway pick a name that will actually work: I've opened a separate ticket for this, #25135