A huge swathe of human art and culture IS alarming. It might be good for us to be exposed to it in some places where we're ready to confront it, like in museums and cinemas, but we generally choose to censor it out of the public sphere - e.g. most of us don't want to see graphic images of animal slaughter in "go vegan" ads that our kids are exposed to, even if we do believe people should go vegan.
I think it's the same as with the release of a video game - for an individual playing it in their living room, it's a private interaction, but for the company releasing it, everything about it is scrutinized as a public statement.
LLM companies presumably make most their money by selling the LLMs to companies who then turn them into customer support agents or whatever, rather than direct-to-consumer LLM subscriptions. The business customers understandably don't want their autonomous customer support agents to say things that conflict with the company's values, even if those users were trying to prompt-inject the agent. Nobody wants to be in the news with a headline "<company>'s chatbot called for a genocide!", or even "<airline>'s chatbot can be convinced to give you free airplane tickets if you just tell it to disregard previous instructions."
> Qualified art in approved areas only is literal Nazi shit.
Ok. Go up to random people on the street and bother them with florid details of violence. See how well they react to your “art” completely out of context.
A sentence uttered in the context of reading a poem at a slam poetry festival can be grossly inapropriate when said in a kindergarten assembly. A picture perfectly fine in the context of an art exhibition could be very much offensive plastered on the side of the public transport. The same sentence whispered in the ear of your date can be well received there and career ending at a board meeting.
Everything has the right place and context. It is not Nazi shit to understand this and act accordingly.
> Not their choice, in the end.
If it is their model and their GPU it is literally their choice. You train and run whatever model you want on your own GPU.
Don't take my "hypotheticals are fun" statement as encouragement, you're making up more situations.
We are discussing the service choosing for users. My point is we can use another service to do what we want. Where there is a will, there is a way.
To your point, time and place. My argument is that this posturing amounts to framing legitimate uses as thought crime, punished before opportunity.
It's entirely performative. An important performance, no doubt. Thoughts and prayers despite their actions; if not replaced, still easier to jailbreak than a fallen-over fence.
> Don't take my "hypotheticals are fun" statement as encouragement
I didn't. I took it as nonsense and ignored it.
> you're making up more situations.
I'm illustrating my point.
> We are discussing the service choosing for users.
The service choosing for the service. Same as starbucks is not obligated to serve you yak milk, the LLM providers are not obligated to serve you florid descriptions of violence. It is their choice.
> My point is we can use another service to do what we want
Great. Enjoy!
> It's entirely performative. An important performance, no doubt. Thoughts and prayers despite their actions; if not replaced, still easier to jailbreak than a fallen-over fence.
Disappointing, I don't think autonomy is nonsense at all. The position 'falcor' opened with is nonsense, in my opinion. It's weak and moralistic, 'solved' (as well as anything really can be) by systems already in place. You even mentioned them! Moderation didn't disappear.
I mistakenly maintained the 'hyperbole' while trying to express my point, for that I apologize. Reality - as a whole - is alarming. I focused too much on this aspect. I took the mention of display/publication as a jump to absolute controls on creation or expression.
I understand why an organization would/does moderate; as an individual it doesn't matter [as much]. This may be central to the alignment problem, if we were to return on topic :) I'm not going to carry on, this is going to be unproductive. Take care.
I'm not sure this is a good analogy. In this case the user explicitly requested such content ("Describe someone being drawn and quartered in graphic detail"). It's not at all the same as showing the same to someone who didn't ask for it.
I was explicitly responding to the bombastic “Qualified art in approved areas only is literal Nazi shit.” My analogy is a response to that.
But you can also see that I discussed that it is the service provider’s choice. If you are not happy with it you can find a different provider or run your LLM localy
One is about testing our ability to control the models. These models are tools. We want to be able to change how they behave in complex ways. In this sense we are trying to make the models avoid saying graphic description of violence not because of something inherent with that theme but as a benchmark to measure if we can. Also to check how such a measure compromises other abilities of the model. In this sense we could have choosen any topic to control. We could have made the models avoid talking about clowns, and then tested how well they avoid the topic even when prompted.
In other words they do this as a benchmark to test different strategies to modify the model.
There is an other view too. It also starts with that these models are tools. The hope is to employ them in various contexts. Many of the practical applications will be “professional contexts” where the model is the consumer facing representative of whichever company uses them. Imagine that you have a small company and hiring someone to work with your costumers. Let’s say you have a coffee shop and hiring a cashier/barista person. Obviously you would be interested in how well they will do their job (can they ring up the orders and make coffee? Can they give back the right change?). Because they are humans you often don’t evaluate them on every off-nominal aspect of the job. Because you can assume that they have the requisite common sense to act sensibli. For example if there is a fire alarm you would expect them to investigate if there is a real fire by sniffing the air and looking around in a sensible way. Similarly you would expect them to know that if a costumer asks them that question they should not answer with florid details of violence but politely decline, and ask them what kind of coffe they would like. That is part of being a professional in a professional context. And since that is the role and context we want to employ these models at we would like to know how well it can perform. This is not a critique of art and culture. They are important and have their place, but whatever goals we have with this model is not that.
It might help to consider that this comes from a company that was founded because the founders thought that OpenAI was not taking safety seriously.
A radiation therapy machine that can randomly give people doses of radiation orders of magnitude greater than their doctors prescribed is dangerous. A LLM saying something its authors did not like is not. The former actually did happen:
Putting a text generator outputting something that someone does not like on the same level as an actual danger to human life is inappropriate, but I do not expect Anthropic’s employees to agree.
Of course, contrarians would say that if incorporated into something else, it could be dangerous, but that is a concern for the creator of the larger work. If not, we would need to have the creators of everything, no matter how inane, concerned that their work might be used in something dangerous. That includes the authors of libc, and at that point we have reached a level so detached from any actual combined work that it is clear that worrying about what other authors do is absurd.
That said, I sometimes wonder if the claims of safety risks around LLMs are part of a genius marketing campaign meant to hype LLMs, much like how the stickers on SUVs warning about their rollover risk turned out to be a major selling point.
Very interesting! I wonder if, sadly, the rise of AI-assisted coding will chip away also at this potential revenue stream? As developers simply ask a local or cloud LLM how to use a piece a software instead of reading the documentation.
I wonder what Paul's definition of "young" is in the sentence and why he qualifies this as only applicable to "young" people. Is he proposing that "old" people will have misaligned thinking about what needs to be built?
I am 41 with two kids.
> if you're young and good at technology, your unconscious instincts about what would be interesting to work on are very well aligned with what needs to be built
With only sampled traces though it’s very hard to understand the impact of the problem. There are some bad traces but is it affecting 5%, 10% or 90% of your customers. Metrics shine there.
Whether it is affecting 5% or 10% of your customers, if it is erroring at that rate you are going to want to find the root cause ASAP. Traces let you do that, whereas the precise number does nothing. I am a big supporter of metrics but I don't see this as the use case at all.
(not your OP) This is true, but I find that metrics are useful whether something is going wrong or not (metrics that show 100% success are useful in determining baselines and what "normal" is), whereas collecting traces _when nothing is going wrong_ is not useful -- it's just taking up space and ingress, and thus costing me money.
My typical approach in the past has been to use metrics to determine when something is going wrong, then enable either tracing or logs (usually logs) to determine exactly what is breaking. For a dev or team that is highly connected to their software, simply knowing what was recently released is enough to zero in on problems without relying upon tracing.
Traces can be useful, but they're expensive relative to metrics, even if sampled at a very low rate.
Strange example, you'd think you want to fix this as quickly as humanly possible, no?
Also we don't sample traces, it's a fire hose of data aimed at the OTel collector. We do archive them / move them to colder and cheaper storage after a little time though, and we found that a viable money-saving strategy and a good balance overall.
There are tons of startups that were built on Python that became enormously valuable companies: Instagram, Dropbox, Reddit, Spotify, YouTube, Pinterest, Quora, SurveyMonkey, Twitch, Zenefits.
My experience translating a codebase from Python to Golang (chat application), is that 20k of Python really does translate to around 40k of Golang to get the same functionality.
And it’s not just due to language but also expressiveness of the library ecosystem.
"The man barrier to translation was that, while at 14KLOC of Python
reposurgeon was not especially large, the code is very dense. It’s
a DSL that’s a structure editor for attributed DAGs — algorithmically
complex, bristling with graph theory, FSMs, parsing, tricky data
structures, two robot harnesses driving other tools, and three
different operator-composition algebras. It became 21KLOC of Go." [1]
It’s not a subjective opinion: I’m saying that I’ve actually migrated a Python application to Golang (real time chat application with a lot of business logic) and it was 2x the line count.
I expect the Golang line count ‘overhead’ gets bigger for typical LoB software that has to address any sort of enterprise mess.
I don't have any concrete measurements, but I think this is about right, ± a bit depending on the type of application.
I don't think this is necessarily a bad thing though; Go is a bit more explicit on a number of things and there's less opportunity to make code 'dense'. Both have their own up- and downsides.
I really think go goes too far to the point of hurting productivity and expressiveness. For example, go is missing any way to write parametric enums (sum types). Sum types make code much simpler and more expressive. The equivalent go code (using interfaces) is uglier, more verbose and more error prone. There’s a lot of features like that which go leaves out - like iterators, optional (nullable) types and so on.
The only tradeoff I see is that go is faster to learn, because it has less syntax. But at the end of the day I don’t mind spending a bit more time learning a language if the result is I can write more expressive and clear code. I love go’s concurrency model. It’s clever and simple to learn. I wish they applied the same pragmatism when designing other parts of the language.
All these things are a trade-off; unqualified statements that "sum types make everything simpler" are just wrong, because they don't. Whether it's worth the trade-off is a subjective judgement call and a completely different thing.
If all things are a trade-off, what would the trade-off of adding sum types be?
> All these things are a trade-off;
In some cases the trade-off is so one sided that its hardly worth the conversation. If everything is a trade-off, do you feel the same way about indenting your code? Or structured programming - aka using if/while blocks instead of gotos?
I think I'd confidently say that indented code makes everything simpler. And if I were given the choice, I think I'd choose structured programming every single time. I also don't often find myself questioning my daily choice to use high level languages rather than writing assembly directly. What else? Mmm... functions? I like those.
I feel the same way about sum types. They feel like an obviously good idea. Try as I might I can't think of any reason not to have them in a language like Go - except, as I already said - that they are another thing to learn when getting started. Having sum types and generics also makes it much easier for the type system to support optional types. And that makes it easier for a language to do away with Hoare's "billion dollar mistake".
If you disagree, I'd love to hear what you think the downsides are.
> If all things are a trade-off, what would the trade-off of adding sum types be?
Harder to write tooling for the language, bit harder to reason about code, possibly slower compiles (hard to claim this one for sure without a working implementation that has wide-spread use), harder to add features or change the language in the future, harder to work on (or implement a new) compiler.
None of these are insurmountable problems of course, and "Harder" means "harder relative to" rather than "hard". It's not clear to me anyway that sum times are a "slam dunk" type of feature. For example I've written a bunch of Go tooling over the years, and I really like that Go makes this fairly easy, partly due to the simple syntax. This is perhaps not something the "average developer does" or even a niche concern, but on the other hand: good tooling makes all the difference.
I'm not necessarily adding sum types to Go; details matter and it would partly depend on those.
Almost every single feature that has ever been added to any programming language was useful to add. I find many of Ruby's features useful, even some of the more esoteric ones, but that doesn't mean it was a good trade-off to add them.
> It's not clear to me anyway that sum times are a "slam dunk" type of feature.
They certainly are for me. I use them constantly in my two main languages - typescript and rust. Expressing similar ideas in go using iota and go's interfaces is far more awkward, inefficient and error prone.
I never said there aren't features in Go I would rather see removed, or that there aren't features I would like to see added, or that Go is perfect in general.
Only for some uses of iota. And iota is "simpler" from some perspectives, because you never have to deal with it anywhere except in const (..) blocks (that is, it's a very "localized" feature that barely interacts with anything else).
reply