Hacker News
points by nxobject 2 days ago | 0 comments
Above all, I’m a little sad that the musl developers decided to make a lot of “well it’s convenient” changes to core crypto functions — at least without the level of auditing that would have caught this! It’s tarnished their reputation in my eyes a little.
[–] rustyhancock 2 days ago
I think everyone is over fixated on AI finding complex bugs beyond a typical person's understanding either to locate or exploit.

I think for any one of those bugs there might simply be half a dozen footguns or bugs that once it's spelled out you wonder how anyone let it slide.

I think that's the case where LLMs might indentify most.

It's challenging though as the current AI audits produce so many false positives it's hard to know when it's being pedantic or when it's genuinely caught a human being a bone head

[–] jamal-kumar 2 days ago
Finding more and more that these complex bugs people gloss over like you are now are being used in exploit chains and the end result is still my clients getting hacked
[–] svnt 2 days ago
This one might be hard to find using the author’s approach of throwing everything at various builds of ssh.

It is not really that hard to find in the wild if you had a device built this way, when overflowing buffers on embedded devices is a standard attack vector.

It might be hard to root cause, but it would not be that hard to find.

[–] 0xbadcafebee 2 days ago
Musl has a history of introducing (and not fixing) major bugs in their library that aren't in glibc; I would not depend on it for production
[–] rpcope1 2 days ago
Honestly the ideas around musl are interesting, especially being able to statically link, but given the sort of nonsense around how they handled pthreads, and generally other trade offs, it's hard to imagine wanting to use it over glibc (which everyone shits on) if you don't absolutely need static linking.
[–] pelagicAustral 2 days ago
Everything in this industry is about convenience.
[–] trebligdivad 2 days ago
In many ways each individual decision makes some sense; wanting to stop loads of programs being affected by a (remotely triggerable?) crash wanting to avoid time based attacks wanting to be able to set accounts to not have a password

Each one relies on a perfectly valid argument - that * can't match because we all know it'll be rejected by the other thing.

[–] yangm97 2 days ago
I would be more concerned about the lack of explicit return types, one of the arguments for strongly typed languages is that you should not have ambiguous data and yet…

Granted I mostly work with high level programming languages so I may be missing something.

[–] preg_match 2 days ago
Yes this is C, C is not strongly typed at all and the type system is very primitive. NULL ptr is the return value you should probably use if something goes wrong. But you can’t, say, return a sum type with the result and an error. It’s not uncommon at all to use magic sentinel values in C to indicate error conditions.
[–] Retr0id 2 days ago
> Each one relies on a perfectly valid argument - that * can't match because we all know it'll be rejected by the other thing.

But that's not a perfectly valid argument, and it never was.

[–] aidenn0 2 days ago
I guarantee that every single developer of musl in 2012 had at least account on their machine with "*" as the password hash. Also crypt() is a standardized function and the standard states that a null pointer is returned on error.

That being said, OpenSSH did mismanage their attempts at handling a garbage return-value from crypt(), so there's definitely some blame to go around.

[–] woah 2 days ago
If they had just all used a different garbage symbol would this have been avoided?
[–] zarzavat 2 days ago
The real problem here is the C language. Our industry should have dumped it a long time ago and we’re now paying the price of prevarication.
[–] oneeyedpigeon 2 days ago
Which of the several flaws in this bug chain would have been materially different with a language other than C?
[–] nh2 2 days ago
Any use of sum types would have prevented this immediately.

The whole thing exists because APIs that return a NULL value that "nobody checks", plus a string that you shouldn't use if you actually did check the null value.

In languages with sum types, this cannot happen.

Either you get your hash, or no result at all. You couldn't get eg NULL and *.

[–] jppittma 2 days ago
C has unions. C has sum types. People have been doing tagged unions in C for longer than rust has been a language
[–] creata 2 days ago
It is extremely unidiomatic to use tagged unions in C for error handling. It's definitely possible, but awkward, and the compiler doesn't check that you checked the tag.
[–] K0nserv 2 days ago
I don't think I've ever seen a C function that uses unions for this purpose though?

I can think of two common patterns that I've seen frequently:

* Return NULL for errors (sometimes with a global errorno() function).

* Return an integer error with any return values as out parameters.

So if it is possible to improve, why doesn't anyone do it?

[–] Woodi 2 days ago
Because usually NULL is good enough and you need to have domain use case to make library for it.

> * Return an integer error with any return values as out parameters.

You must first standarize all that (usually negative) integers, make docs, check for it, do not make mistakes in your if's or case's...

And errno is just that - integers with a meaning. errno is even thread local now and quite safe to use but people on internet scream loudly when someone try to use that not app global varible :) I guess their llms statistically have less new texts feeded in ;)

At some point it is just better to return pointer to struct with all info you can imagine.

And how you think other languages implemented their Error type returns ? Just a struct or union with tag. Because current CPU's operate on simple type registers there is no other sane way to do it.

[–] aliceryhl 2 days ago
Yet the authors of crypt did not use them. The C language is not designed in a way to encourage people to write code that correctly uses tagged unions (or nullable pointers for that matter).
[–] knome 2 days ago
because POSIX dictates the shape of crypt from a function written in 1973 and unions/structs were only introduced to the C language in 1976.

the fault for sshhole isn't just "C bad", it's a series of poor choices in workarounds falling together to create a vulnerability.

being condescending about decisions made 50 years ago, before any of the better languages you're mad the developer didn't use even existed, is silly.

if you want to lay blame, lay it at the feet of more recent devs that started returning values instead of NULL because people were using crypt incorrectly. likely sha512 returned a fake value instead of a real one because prior hashes couldn't fail. the interface should have been updated then, but instead they worked around it.

[–] jppittma 2 days ago
Where are you going with those goal posts?
[–] zarzavat 1 day ago
That's a universe apart from real sum types with exhaustiveness checking and pattern matching.
[–] xorcist 2 days ago
This argument is weak. C, and specifically libc, has errno. The error code is there. Not in band.

But there has to be something in the payload field when it is not applicable. A null is perfectly fine.

The argument for changing that to "*", if I understand the above linked discussion correctly, was that some users of this api threw away errors and a dummy value would be harder to ignore. Maybe one could argue that these behaviours are less common in Rust or Go, but not because of a technical difference.

The world is not perfect and sometimes you really need to work around bad users. Even in Rust or Go. I am not convinced this was such a case. If you are convinced it was it would still have been better to select a less trivial dummy value (such as "*:ERROR:*").

[–] forresthopkinsa 2 days ago
> But there has to be something in the payload field when it is not applicable

Not in languages with ergonomic sum types. A `Result` or `Either` will have no payload field on the error type. The compiler won't let you dereference it.

[–] lemonwaterlime 1 day ago
In Haskell, one would return a Maybe or an Either here.

    Maybe
    ——————
    Just hash
    Nothing

    Either
    ——————————
    Left err
    Right hash
The program would be designed around not allowing this failure mode to even be possible.
[–] aidenn0 2 days ago
This is wrong. errno is not to be used to detect errors, only to get the cause of the error once an error has been detected.
[–] dolmen 2 days ago
Nope.

Because in fact all the changes were about changing the function signatures. The function signature is enforced by the linker. The semantic of strings that go through isn't.

Returning 2 values is possible in C (via pointers input parameters). The main issue here is that libc is a strong API contract based on C function signatures, that nobody can change, only augment with new symbols. Such strong API compatibility constraints happen in any programming language, and workaround to fit new behaviours in old APIs happens everywhere.

[–] K0nserv 2 days ago
I don't think nh2 is saying that this is how the "issue" should've been solved here, but that it's a fatal flaw of C itself (yes I know, unions, but no one uses them).

But also, its not like there were many alternative languages when the POSIX standards were designed so it's fair that it works like this.

[–] aidenn0 2 days ago
Even C has a better type system than C: All modern C compilers support an extension that lets you flag a function's return-value as requiring a null check.
[–] muvlon 2 days ago
C's lack of all of

- sum/union types

- multiple return values/tuple types

- exceptions

means that most ways of returning an error from a function are not available in C. Because of this, many C APIs, including `crypt` in this case, resort to in-band error signalling by using a sentinel value (like NULL, a negative integer, "*" etc.).

Most other languages lack one or two of the above, but very few lack all three, and so most have developed a different culture around error reporting (Result in Rust, multiple return values in Go, exceptions in Java, Python etc.).

[–] tialaramex 2 days ago
One of the small but important significances of Rust is that it very often gets to have the same machine code as a correct "in-band error signalling" strategy from C but with source code that the maintainers can actually understand.

And really often when your "in-band error signalling" in Rust is more expensive you realise (perhaps to your horror) that your C wouldn't be correct, so you just dodged a bullet. For example Option<OwnedFd> is, like OwnedFd, a 32-bit value, Unix has always reserved -1 to mean invalid, so Rust can treat that as a niche. But the Windows equivalent Option<OwnedHandle> can't do this because, well, cue Raymond Chen: https://devblogs.microsoft.com/oldnewthing/20040302-00/?p=40...

[–] creata 2 days ago
> One of the small but important significances of Rust is that it very often gets to have the same machine code as a correct "in-band error signalling" strategy from C but with source code that the maintainers can actually understand.

But aside from NonZero- and NonNull-based layout optimizations, I don't think Rust is doing anything special to obtain that machine code - it's just relying on LLVM to optimize things.

[–] tialaramex 2 days ago
Well, all of the niche optimisations, so that includes every normal C-style user-defined enumeration because (unlike C or C++) Rust says those aren't just the integers wearing a funny hat.

But sure, for the most part LLVM will do The Right Thing™ here without much help and Rust is just giving better ergonomics to the same backend.

[–] Gibbon1 1 day ago
I remember mucking with returning multiple values as anonymous structs. It was tantalizing close but no banana.

I think with C23 you got half a banana now. Still can't do it because C23 didn't adopt proposals to allow returns of untagged/anonymous structs.

If you could then you could write

  auto rtn = Foo();
  if(rtn.err) 
    printf("err=%i", rtn.err);
  else
    printf("Foo=%i\n", rtn.foo);
[–] sscaryterry 2 days ago
The first 2 it has, and the 3rd, you really don't want.
[–] muvlon 2 days ago
The first 2 C has in the same way that I don't need Dropbox if I just have an FTP account and curlftpfs (https://news.ycombinator.com/item?id=9224).
[–] sscaryterry 2 days ago
Your analogy is way off...
[–] jppittma 2 days ago
C has sum types/unions? What are you talking about? You don't need multiple return. You can use a struct and it's the same thing.
[–] adgjlsfhk1 2 days ago
not in a way that forces the user to actually handle them properly. C is a weakly typed language and as such is pretty badly suited to systems programming.
[–] uecker 2 days ago
C is not a weakly typed language, this is just nonsense. There is also [[nodiscard]] which you can use to make sure a return value is handled.
[–] muvlon 2 days ago
C unions are sum types if you just "draw the rest of the fucking owl". C has sum types the way x86 assembly has function calls. You see, there are "call" and "ret" instructions. Just figure out calling convention and the stack yourself, it's easy. And yes I can stick the tag next to the union myself in C and write a bunch of functions that hopefully deal with it correctly. I can even write some macros that look like pattern matching and maybe convince myself that it's a language feature, but it just isn't.

And it's very evident why the difference matters. People gravitate to what's readily available and ergonomic. That's why every C API under the sun (most definitely including libc itself) does janky stuff like returning -1 or "*" on error instead of building their own bespoke Result-style abstraction. A programming language is not just the things that are in some way possible within it, but more importantly the things that are easy in it.

[–] uecker 2 days ago
While I agree that this should come out of the box, the "rest of the fucking owl" is a tiny macro or function, so not exactly something difficult to do complex as "drawing an owl" would imply.
[–] K0nserv 2 days ago
For the other part of the issue (the truncation in SHA512), in Rust you could've had:

    struct Input<'a>(&'a [u8]);
    

    impl<'a> Input<'a> {
      fn new(input: &'a [u8]) -> Result<Self, Error> {
        if input.len() >= 256 {
           return Err(Errro...);
        }
        
        Ok(Self(input))
      }
    }
Now the truncation is a terrible idea in the first place, but at least with this it would have been entirely obvious to the caller that it was there.
[–] uecker 2 days ago
In C you could do just the same: https://uecker.codeberg.page/2026-10-24.html Whether you should is a different question.

In any case, you can not change the API of an existing function. But returning NULL for error has required by POSIX would have been just fine.

[–] dmantis 2 days ago
Wouldn't Rust's Result or Go's error checking prevent that?

While not mandatory, a person would probably use it instead of the * hack.

Simply because the calling side wouldn't have an option to forget the check like with the NULL case

[–] ninkendo 2 days ago
Rust and go still make it pretty easy to not check results if you don’t care about the success value (lots of code just does `let _ = foo()`), but if you do want the value (like with a hashing function), Rust makes it really hard to not check the error too. There’s no syntax to say “assume this worked and give me the success value, including whatever garbage is there if it’s not set”:

- result.unwrap() will panic if it failed

- result.ok() throws away the error but gives you a None value instead (still can’t use it as if it’s a success)

- matching on the Ok variant and doing nothing on the Err variant, you only get to use the success value inside the Ok arm

Etc etc.

The problem is when the compiler says “NULL is a valid instance of T”, which go lets you do, and rust doesn’t (without using the unsafe keyword and intentionally casting a null pointer, which you don’t do by accident.) Go lets you write code like `var hash string; hash, _ = hashfn()` and it compiles and now you have null instead of a valid hash.

[–] tialaramex 2 days ago
> NULL is a valid instance of T

This by the way is Tony Hoare's famous "Billion Dollar Mistake"

[–] 65a 1 day ago
Minor nit, strings are never nil in Go, so that would be "" instead. Usually nil is only a valid instance of pointer-to-T. The point stands though, a common pattern is "defer foo.Close()", the Close() signature almost always returns an error and almost no one checks it...which is probably fine, but not automatically fine.
[–] uecker 2 days ago
A NULL pointer for error would be a safe interface where you get a trap when there is no value.
[–] f1shy 2 days ago
I understand that other languages could help avoiding the semi-predicate problem. That being said, in C there are better ways around semi-predicates, as inelegant as they may be, is possble, is not an inherent problem of the language, but may a bad habit/idiom of it.
[–] 01HNNWZ0MV43FF 2 days ago
There is no industry, it's just people
[–] randomNumber7 2 days ago
Do you work in industry or academia?
[–] WesolyKubeczek 2 days ago
You must be new here. While I applauded the idea of musl when it first came out (coincidentally, glibc wasn't having the best time in its development then), the developer's arguments why <some behavior drastically different from every libc under the sun> was akchyually correct, which was why you had to fix a program that hadn't been broken for 30 years and now suddenly was, made me put him under the "clowns" category and never look at musl seriously again.

When later it turned out that musl's procrustian stack size made Python not really runnable under it, it cemented my resolve to not go near musl.

And now we have this.

[–] acdha 2 days ago
I’m also reminded of the clowns pushing “it’s smaller!” container images based on social media vibes rather than analysis where, yes, not using Debian+glibc saved tens of megabytes on the base OS but did so at the expense of measurable performance reductions & compatibility issues like the one you mentioned, and literally every single one of those dudes I worked with was shipping containers pushing a gigabyte so the size reduction had no measurable impact on deployments.
[–] NewJazz 2 days ago
Uh, isn't this the case of the opposite? The musl devs trying to accomodate broken programs?
[–] throwawayffffas 2 days ago
I am sorry, a little? I am deeply disappointed that libc decided to do the same just with another sequence.

And why because some dude was complaining sshd or whatever was crashing because it didn't handle nulls? Just handle the null my guy you already know why your bullshit is crashing.

[–] IshKebab 2 days ago
Let's be real here, the main culprit is C. People love to say "but you can still get logic errors with Rust", and no shit of course you can. But it's much less likely. You wouldn't have gotten this bug.
[–] semiquaver 2 days ago
A little?!? I’m sorry, but musl is a joke, this is yet another proof of that fact, and I have no idea why anyone uses it in production, they have only themselves to blame after the many warning signs.