Hacker News

points by numpad0 2 days ago | 0 comments

fyi: filesystems have max path length/byte count limits, and yt-dlp can exceed that(e.g. "/var/home/username/Documents/Videos/YouTube/Five-reasons-you-want-powdered-detergents-for-your-dishwasher-fourth-one-surprises-you.mp4.part.012" > $THE_MAX_LEN). If this happens, it throws a Python `[Errno 36] File name too long` error and yt-dlp gives up.

This can be mitigated by adding `-o "%(title).150B [%(id)s].%(ext)s" ` to command line, which trims filenames to 150 bytes max. There are other options like --trim-filenames, but yt-dlp don't apply those to the interim part files and that triggers the failure anyway. The relevant issue has been reported on GH years ago and closed by the maintaining author with no PR from me(hey).

[–] EvanAnderson 2 days ago

Using Youtube titles for filenames is an exercise in wild and crazy non-ASCII characters. Make sure your filesystem, OS, and client tooling can handle it.

[–] fushihara 2 days ago

When building a system to download content locally, the first thing everyone tries to do is use the content's title as the filename. But this causes a surprising number of problems.

You should use a whitelist of allowed characters. In particular, make sure to test every ASCII character. There aren't even 100 non-alphanumeric ASCII characters. I keep a list of ASCII symbols in my notes, and whenever I build a system like this, that's one of the first things I test.

Make sure you explicitly remove all control characters.

Unicode has some great features, but it's perfectly fine to strip out anything that isn't considered a "normal character" in your region. Some titles might end up looking a little strange, but fewer than 1% of videos use unusual characters anyway. Unicode simply has too many possible characters to deal with.

Explicitly normalize filenames to either NFC or NFD. Otherwise, you can run into problems when viewing content uploaded by Mac users on Windows.

Sometimes, removing invalid characters can leave you with an empty video title. Having a filename that consists of nothing but an extension can cause all sorts of problems, so make sure to include a fixed prefix or suffix in the filename. Personally, I always append the resolution, duration, and video ID to the end of the filename.

On Windows, you also need to be careful about filename length. For historical reasons, the limits are measured in UTF-16 code units, and there are restrictions on both individual filenames and full paths.

Don't get greedy and try to use the full 250 characters allowed for a filename. I'd recommend limiting filenames to around 100 characters at most. Full paths should also stay within 256 characters.

You may have heard that Windows supports full paths of up to around 30,000 characters with long path support enabled, but don't rely on that unless you've verified that every tool in your workflow supports it.

Before downloading any actual content, generate a list of titles from the latest 1,000 or 10,000 items in your system. Then try creating zero-byte files locally using the filenames your system generates, and verify that everything works correctly.