Some valid characters in URL caused problem in URL parsing

AFAICT, the server is sending semi-garbage value for Location.

closed

@Courmisch What exactly do you mean by "semi-garbage"?

It's definitly a valid Locaton redirect since Chrome and Safari can follow it. MPV and PotPlayer can follow it and play just fine. And so does curl:

$ curl -I -X GET -L https://nc.raws.dev/0:/%5BNC-Raws%5D%20%E6%88%91%EF%BC%8C%E5%B0%8F%E6%80%BC%20-%2011%20%5BB-Global%5D%5BWEB-DL%5D%5B1080p%5D%5BAVC%20AAC%5D%5BCHS_CHT_ENG_TH_SRT%5D%5BMKV%5D.mkv
HTTP/2 301
server: nginx
date: Fri, 10 Sep 2021 18:28:50 GMT
content-type: text/html
content-length: 162
location: https://twilight-voice-fc85.fqnu0wav.workers.dev/0:/[NC-Raws]%20%E6%88%91%EF%BC%8C%E5%B0%8F%E6%80%BC%20-%2011%20[B-Global][WEB-DL][1080p][AVC%20AAC][CHS_CHT_ENG_TH_SRT][MKV].mkv
cache-control: no-cache

HTTP/2 200
date: Fri, 10 Sep 2021 18:28:53 GMT
content-type: video/x-matroska
content-length: 68738517
cf-ray: 68cab7eb2ba86420-SJC
cache-control: private, max-age=0, must-revalidate
content-disposition: inline
expires: Fri, 10 Sep 2021 18:28:53 GMT
vary: Origin, X-Origin
cf-cache-status: DYNAMIC
alt-svc: h3=":443"; ma=86400, h3-29=":443"; ma=86400, h3-28=":443"; ma=86400, h3-27=":443"; ma=86400
expect-ct: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
x-goog-hash: crc32c=S1HmtA==
x-guploader-uploadid: ADPycdvQe2Z-6ibCWo8KVkjL6e0vofAY_A_MZo7cJO8Sd93TocYlGl-JHFX-AOj5p3uTzXhIxqSQkWZ6pWXiaKInKArhGkCTuQ
report-to: {"endpoints":[{"url":"https:\/\/a.nel.cloudflare.com\/report\/v3?s=QQZftl6H8YUuT0Tl3ylMXzy5v%2FhkumoXPq97r9ZasjAQxL5aBmcPpqVBbK2RRUGFaPFB7jFLILI%2FfyLqX2LPUIijodObvrmebnDhz7ddzQaXvuEoacxkdHyjLNr77GqxG4OjPxmPcL59SzyLyAUD4XfG0DZUeduzudRW"}],"group":"cf-nel","max_age":604800}
nel: {"success_fraction":0,"report_to":"cf-nel","max_age":604800}
server: cloudflare

That is a plain ridiculous argument. Chromium and Safari can recognise plenty of invalid URLs and fix them up. It's obvious if you write an URL with a white space and it gets magically encoded into %20. Even VLC can (and needs to) do that. That does not make those URLs valid.

And we even checked in an earlier discussion that wget and curl don't even agree with one another on that topic.

Yes some softwares fix invalid URLs, but if the same URL got recognized in many different softwares but one, it definitely says something.

Which is why my following argument is to look at URL standards. There can be implementation differences and undefined behaviors, but when we have a standard, it's best to start from there.

reopened

It contains separators (:, [ and ]) out of place, presumably missing encoding.

VLC 2.x would probably have not cared, because it did not resolve URLs correctly, but that caused a lot of problems with valid relative URLs.

The HTTP Standard doesn't specify any special level of encoding on the URL. The URI Standard only requires percent-encoding when it creates ambiguity. Those :, [ and ] characters in the Location URI doesn't create ambiguity for parsing it as an URI-reference component.

If you can't cite from some standard to prove the invalidity of the URL, then it seems to be a valid bug in VLC's URL parsing logic.

There is no such thing as level of encoding for an URL. The HTTP specification cannot require something that does not exist. Only individual URI components can be encoded or not, but Location is carrying a whole URL, not a single component.

A percent-encoding mechanism is used to represent a data octet in a component when that octet's corresponding character is outside the allowed set or is being used as a delimiter of, or within, the component.

You can either escape ONLY when the character creates parsing ambiguity, or you can escape WHENEVER you encounter an Reserved Character. That's what I meant by level of encoding. Both practice are valid according to the above standard.

What's important is, other popular networking software like Chrome, Safari, and curl, don't treat this URL as invalid, and works properly. Which is enough for us to ask for understanding why VLC behaves differently, and how can we make it support properly.

I mocked an HTTP response that applies percent-encoding on [ and ] characters: https://mockbin.org/bin/41cba4ad-cb82-435b-8b3d-0f2e01fed870/view

And this time VLC resognized the URL and is able to follow. This confirmed that it's a valid bug in VLC's URL parsing code, unless you can provide standard citations to prove the original URL was invalid.

It only confirms that it's caused by the brackets. Whether it's a bug in VLC or in the server is a matter of interpretation. (People like to think by bug-compatibility rather than specification, but that goes nowhere: you could blame nginx for breaking VLC or VLC for breaking nginx.)

Insofar as it does not really break anything, I don't really care. VLC already accepts : in the path part anyway.