Hacker News
points by slopinthebag 7 hours ago | 0 comments
you completely missed the point that the "ankle biters" were making - smaller models can still be useful despite being less intelligent. nobody makes the argument that luna is just as smart as opus, but they do make the argument that someone sufficiently knowledgeable and skilled can use smaller models to great effect. just because you and others require the maximally powerful models to make sense of vague, underspecified prompts, to hold your hand throughout the entire development lifecycle and make decisions for you, does not mean we all do.
[–] anonymous908213 7 hours ago
> underspecified prompts

This is the same stupid argument trotted out repeatedly in the thread, and it doesn't hold up to reality. Would you care to explain how adding specifications helps a model that demonstrably ignores specifications? A model that cannot even handle the trivial specification of "the filename must be lowercase" is not going to get better when you add more specifications it ignores. I would guess that you're actually significantly underspecifying relative to me. Perhaps you're happy with whatever slop you get and don't notice Luna's overwhelming incapability to actually provide what one requires, because you don't have requirements.

[–] slopinthebag 7 hours ago
yes, your prompts are underspecified relative to the size of changes you are making. you want to write a handful of paragraphs and have a sota model vibe out a +1000loc diff, where it makes all sorts of decisions and architectural/functional choices for you (that you didn't even realise needed to be made). if you use a smaller model, it's just not going to be sufficient. you need to work more iteratively with the model, with more steering and more human judgement. astra/opus can do this relatively well in some domains, especially the relatively simple ones like web, agent harnesses, etc. but someone with intention and skill can make use of the less powerful models, it's just a different workflow (and one that I prefer, when I'm not being lazy).
[–] anonymous908213 7 hours ago
> you want to write a handful of paragraphs and have a sota model vibe out a +1000loc diff

Wrong. What I want Luna to be able to do is ~deterministically make small edits or generations of 1~10 LoC according to a specification, and it literally cannot even do that. It violates specifications no matter how small you aim.

[–] slopinthebag 7 hours ago
that doesn't even make any sense. deterministically generating 10loc based on a spec? you should be generating that code based on specific instructions of what you want those 10 lines of code to do, not based on some spec. and there is nothing deterministic about using llm's for codegen. i feel like you're just lost here mate.
[–] anonymous908213 6 hours ago
The specification constrains the implementation details. Repeating the demonstrated example, again: filename must be lowercase is a perfect example of such a specification. You can have a hundred different tasks that require different things but all require that common specification.

> there is nothing deterministic about using llm's for codegen

And if you're okay with that, you're more like a farmer eating the slop meant for his pigs than an engineer. Using Sol Max, you can get near-deterministic output for a much wider range of tasks and specifications. Not all specifications; at a certain degree of sufficiently detailed specification it will begin failing again. But some, which is better than none.

[–] slopinthebag 6 hours ago
i just asked luna to write a function that transformed a file path while upholding an invariant about lowercase file names. completed that task just fine. i see no reason it wouldn't work if i asked it for 99 other functions with that same constraint.

again, you're expecting to use it without steering. that's not how you get the most out of the smaller models. i don't want to say you're using them wrong, but you clearly are. it could be deficiencies in your harness as well which the stronger models mask. try pi.

[–] anonymous908213 6 hours ago
You are sorely missing the point. You made the constraint the specific and only focus of the task. The example from the OP pertains to non-filename-related-tasks that have an additional specification added on to them. If your filenames need to be lowercase for all tasks, not just specifically filename-related tasks, you start running into problems.

> you're expecting to use it without steering

If I have to manually "steer" every mistake it makes on every trivial task, instead of being able to deterministically eliminate classes of failures so that fixing the task one time ensures that failure never happens again on similar future tasks, it is in fact faster to just type the fucking code myself.

[–] easterncalculus 2 hours ago
The leap to a conclusion about points not even mentioned in the original comment could win a gold medal
[–] slopinthebag 1 hour ago
what are you even talking about