I think dart made the wrong choice here, but I've never used the language personally or professionally.
I give the author the benefit of the doubt but Optionals are so much more powerful than what this article covers. You can map, filter, reduce, chain, compose functions that all work with optionals, and lift functions that work on numbers (or any other type) to be functions that work on Optional<number> but none of that is even mentioned in this piece (likely cause then its harder to justify picking nullable types instead)
I also found the Some(Some(3)) example just plain wrong. In those scenarios, typically you just use chain (aka flatMap) instead of map... Seems odd this was even included as a motivating reason when this is very much a solved problem.
I'm certainly biased, as I think FP is an incredible mental model for coding and it seems to be catching on in lots of places too.
I understand language designers have to make choices that benefit their users now and be as simple as possible to opt into without any overhead to account for changes, but man, what a missed opportunity to give developers better tools and mechanisms in addition to opening up the doorway for many people to functional programming. But even Java has support for optionals.
Overall, this appears to be like a missed opportunity and the justification doesn't seem to really reflect an understanding of the benefits optionals provide, and misconstrues their use and capability.
> You can map, filter, reduce, chain, compose functions that all work with optionals, and lift functions that work on numbers (or any other type) to be functions that work on Optional<number> but none of that is even mentioned in this piece (likely cause then its harder to justify picking nullable types instead)
Maybe I'm missing something, but I don't see how that's special to option types. Here's a set of extension methods in Dart that provide the operations you describe:
extension NullableExtensions<T> on T? {
R? map<R>(R Function(T) transform) => this == null ? null : transform(this);
T? filter(bool Function(T) predicate) {
if (this == null) return null;
if (predicate(this)) return this;
return null;
}
}
extension FunctionExtensions<T, R> on R Function(T) {
R? Function(T?) lift() => (T? param) => param == null ? null : this(param);
}
extension NullableIterableExtensions<T> on Iterable<T?> {
Iterable<R?> map<R>(R Function(T) transform) =>
this.map((e) => e == null ? null : transform(e));
Iterable<T?> filter(bool Function(T) predicate) =>
where((e) => e == null || predicate(e));
}
Of course, this is not entirely as expressive as option types because of the inability to nest, but the article is pretty clear that nesting is the major advantage of option types.
The simplest illustration is the ability to have multiple levels of semantically meaningful optionality:
Option<Option<int>> is a meaningful type, and None, Some(None), and Some(Some(1)) are all meaningfully-distinct values. int? doesn't compose to int?? and has no good way to differentiate what Option<Option<int>> represents as None and Some(None).
The reason Option<Option<int>> isn't a problem is because it's a monad, so people will naturally transform it into Option<int> while writing their programs.
There is no irony there. If it wasn't a monad, the GGP would be incorrect, and ad-hock solutions would be very valuable.
In a language with Haskell's guarantees, you can mechanically get from the standard "bind" function to prove that "m (m a)" can be converted to "m a" [1]. That function is called "join", and is a lesser-well known way to write monad implementations which is equivalent to the more famous bind function.
(IMHO, join is a better way to understand the typeclass intuitively. The standard bind is better to program with in general use, but much harder to grok.)
So, in Haskell, it's more than just "the APIs would tend to encourage not nesting the types"... having a valid monad implementation means they really are equivalent.
By compose I quite literally mean the compose function.
const add3 = val => val + 3;
const multiplyBy10 = val => val * 10;
const subtract5 = val => val - 5;
const doABunchOfMath = compose(add3, subtract5, multiplyBy10);
const ninetyEight = doABunchOfMath(10);
const optionallyNinetyEight = Some(10).map(doABunchOfMath)
That Dart already provides these extensions feels like they are providing an optional type but through a more complicated mechanism than a simple class
Nowhere in any of the functions I wrote above do I as a user have to worry about handling the case where no number exists, that's handled by the option type and signified by the type (The value in the option may or may not exist) and I can write my functions without ever having to worry about handling that case
extension NullableExtensions<T> on T? {
R? map<R>(R Function(T) transform) => this == null ? null : transform(this!);
}
V Function(V) compose<V>(Iterable<V Function(V)> functions) =>
functions.reduce((composedFunction, function) {
return (V value) => composedFunction(function(value));
});
num add3(num val) => val + 3;
num multiplyBy10(num val) => val * 10;
num subtract5(num val) => val - 5;
final doABunchOfMath = compose([add3, subtract5, multiplyBy10]);
final optionallyDoABunchOfMath = (num? value) => value.map(doABunchOfMath);
doABunchOfMath(10); // 98
optionallyDoABunchOfMath(10); // 98
optionallyDoABunchOfMath(null); // null
The nullable syntax (${Type}?) also makes it clear to readers that this is a type which may or may not contain a value. If you don't supply the trailing '?', Dart will enforce that the value must exist at compile-time and you can write your functions without worrying about nulls sneaking in where they're unwanted.
In effect, int? is almost nearly Option<int>, except you cannot represent Option<Option<int>> with Dart's nullable syntax.
I do on a regular basis (front end dev). For an example of how, I found this article[1] to be a nice demonstration of how it can be useful at a basic level for ReactJS. Than you just begin applying the same concepts one level higher for your use case and keep going.
Even if you don't use compose directly, the mindset of working to build systems in a composable manner is invaluable.
Its a mental shift that leads you to thinking how a system can be represented as a series of inputs being piped from one function to the next between boundries of my code (server response data --> client side caller --> transformation functions --> local data store --> UI).
Achieving these pipelines requires expressing intent with functions and with a self imposed constraint of writing pure functions, you begin needing functors and applicatives and monads to help store stateful information.
If at any point you need a new capability or need to add a new code path, you just modify the relevant pipeline(s), write any new (pure) functions along with any adapters you might need to inject it in your pipeline, and you're good to go. If everything is a pure function, testing and debugging instantly become easier. All this from wanting to build composable functions.
Some languages also support piping (runs the functions in the reverse order that compose does) which can help visually since functions are invoked in left to right order which is how we read too.
Threading syntax is a giant compose. OK, but that's an operator.
In the TXR internals, I have a C function called chain, which composes N functions together (left to right, not right to left like typical compose functions). It is variadic: the end of the arguments is signaled by nao: a not-an-object constant:
The _f variables are pre-computed function objects, stored in globals, to avoid consing them repeatedly. That func_n1(cdr) seen in match.c could be replaced by cdr_f , not to mention by the func_f1(rest); it conses a new function object referencing the C function cdr each time it is called.
Language culture matters. Scala has both `Option` and `null`, but there's a strong culture of "`null` is only for Java interop", so you very rarely see a `null` in the wild from libraries, or wonder whether it'd be valid to pass it in.
I think that the expectation is that idiomatic Scala code will mostly still use `Option`, while `null` will be used for interoperability with Java, JavaScript or Scala libraries that happen to use `null`.
In any case, I am very much looking forward to these additions.
I generally agree with you but the one point they make in this article that really matters for Dart is it does not have pattern matching. Dart’s nominative type system makes much of standard FP much harder than you’d imagine. I know Rust is nominal too but Rust has a lot of complexity to support these things.
It is very simple to add a Maybe and Either type to Dart (my code base has variants of these to deal with null issues) but imperative destructing is a pain. You can get most of the benefits of those techniques in user land without the language knowing about it, but I agree it still misses out on some stuff.
If Dart already had pattern matching or was willing to add it first then the ADT version could have been chosen, but without that prerequisite nullable types was the only ergonomic choice.
I don't buy the pattern matching argument. I've been using Vavr's Option type in Java a lot lately (having banned the use of null from my code), and I don't have any issues with it. I avoid Vavr's pattern matching because I find the syntax to be clunky (not a criticism of the Vavr developers; I think what they've done is impressive within Java's syntax limitations). I do miss pattern matching in general, but I don't find using Option to be any more difficult without it. And no, I never use Option.get(); I have an ArchUnit test set up that will fail the build if anyone tries.
On the flip side, I do find Option a pain to use in Rust sometimes, which does have pattern matching. I think that's a function of the borrow checker making some things harder to express. So perhaps it's not Dart's missing pattern matching that would make Option hard to use there, but some other feature (or missing feature) of the language or type system?
That's an interesting counter point, though another comment in a different post for this same article[1] pointed out that Dart is considering adding pattern matching too.
Though I have trouble understanding how a nominal type system affects this? Does dart not have generics? That's all you need to support these patterns? I'm having trouble understanding how the type of type system plays a role here.
Also, what's 'imperative destructing'? Haven't heard that term before, is that just pulling out a property from an object like this from JS?
Yes Dart has generics which is how you would implement optional types yourself. The type of type system doesn't play a role here you are right. I was thinking of the really good pattern matching I have experienced usually being present with structural subtyping.
Dart is considering adding pattern matching (and real tuples!) and I think they will eventually get there. I just think that would have had to come first to make the case of optionals over nullable types more compelling.
I would call that example destructing in the normal sense and it is basically a form of pattern matching. To do that in an imperative language without pattern matching (i.e. what I am calling imperative destructuring), you end up with code like
if (maybeFoo.isSome) {
final foo = maybeFoo.asValue;
...
}
If you are starting from scratch and designing a brand new language then ADT's are a clearly better choice than allowing null. However Dart is not a brand new language. It's a preexisting language that already has null. Given that context nullable types are the only option that doesn't break compatibility with previous languages and allows code to be made null safe. If you already have to have null values because of prior decisions then ADT's don't add null safety. There may be other reasons to want them that aren't about handling null safety. But you'll still have the problem that values can be null and you won't have given developers a tool to handle those potentially null values safely.
> I also found the Some(Some(3)) example just plain wrong. In those scenarios, typically you just use chain (aka flatMap) instead of map
I think you misunderstood. The point here was that you can represent different things using nested Options:
1. Some(Some(3)) = I checked it and the value is 3
2. Some(None) = I checked it and there is no value
3. None = I haven't checked it
It's very useful when building a cache, and also for things like JSON parsing (property was not present, property present but null, property present and has non-null value).
You just can't represent that using Dart's nullable types system, because "int?" and "int??" are the same, indistinguishable type.
Anyway...
One thing not mentioned in the article is that the way they've implemented it in dart is via "magic syntax", which I consider to be a big negative. Option<T> is something anyone could implement without language support (assuming the type system already supports it). Anyone could build their own Option type and use it. (Sure, it's best to have this sort of thing in the stdlib so the stdlib will use it itself.) Nullable types with the "?" syntax makes the language itself bigger, and is something a regular user of the language couldn't implement if they wanted to.
(Yes, I'm aware that the "?" suffix is just a shortcut for the more generalized union type syntax, but I think my argument holds.)
The use of these types in regular code also requires what I consider a sort of special unintuitive requirement on how you structure your code. I have to do explicit null checks and then the compiler/interpreter just "knows" that code in certain places is null-safe through flow analysis. That kind of magic really turns me off.
I also don't buy the argument that Option types are a pain to use if your language doesn't have pattern matching. I've been doing a bit of Java lately and have banned null from my code, instead using Option from the excellent Vavr library (I find java.util.Optional to be lacking, ditto for Java's built-in collections library). I tend not to use Vavr's pattern matching, as I find it clunky. But I also have no trouble using Vavr's Option.
I guess I just prefer to use languages that have strong type systems where I can express constraints and safety through the types themselves, and not have to rely on language/compiler features to implement those constraints. I think the latter also gives you less flexibility as well.
The difference I see is, when you work with a nullable type, each function has to do the check
so
const add3 = (val) => val ? val + 3 : null;
const multiplyBy10 = (val) => val ? val * 10 : null
So the argument type goes from being
val
to
val?
where the calling code doesn't necessarily have type safety since this argument is now optional (it can be a number or null and either is acceptable) instead of the following (with no conditionals or ternaries)
const add3 = val => val + 3;
const multiplyBy10 = val => val * 10;
This is possible cause
optional.map
implicitly handles the case where the data is missing by default, its built into the data type and the code you write doesn't need to worry about it. Its the responsibility of the Optional object.
You could define operations on nullable types to allow this in Dart, like:
extension NullableNumExtensions on num? {
num? operator +(num? other) {
if (this != null && other != null) return this + other;
return null;
}
}
main() {
num? i, j;
print(i + j);
}
For method calls, we have a null-safe method call syntax that automatically lifts the operation to give you a nullable result:
int? i;
var b = i?.isEven; // b has type null?
We could have made all operations implicitly do this lifting, but our experience is that this isn't what users want. They want to know as early in their code when an operation on a potentially-absent value is attempted and reconcile right there what behavior they want.
That is where you use optional chaining + extension method. And that is basically identical to Optional.map or whatever method in those language with Optional type
in kotlin
func add3 (val: Int?) = val ?.let { it + 3 }
The language gives you a way to say: `this value plus 3 if it exist or just return null`
Unfortunately, there is no such method in js currently, so it looks half baked.
Ah, I gotcha, and I had already read another of your comments below. Yep, agree that is nicer in these cases. I still agree with the author that they made the right choice for Dart.
>There are two main solutions: Use an option or maybe type or Use a nullable type
I don't get it. These are literally all exactly the same thing, all slightly varying in ergonomics and compiler support. They may differ slightly but to call them two different categories of solution is just creating a false dichotomy for yourself.
> However, the type system is a little more flexible than with option types. The type system understands that a union type is a supertype of its branches. In other words, int is a subtype of int?. That means we can pass a definitely-present-integer to something that expects a maybe-present-integer since that’s safe to do.
You can do that in Swift, despite being placed as an example of "Solution 1".
> The short answer is that, yes, it is entirely possible to live without null, and languages like Rust do.
If you can call say that Swift has "null", then you can equally say that Rust has "null"--it's called "None". The only way "None" and "null" from other languages differ is in built-in compiler support.
They're not the same. The difference is that what the article calls "nullable types" are based on commutative unions, while option types are based on non-commutative unions.
In haskell, Either a (Either b c) is distinct from Either (Either a b) c. In typescript, a | (b | c) is identical to (a | b) | c. And further, x | x is identical to x in typescript.
The upshot is that Haskell (using Either instead of Maybe for clarity) lets you have Either (Either a ()) () as distinct from Either a (), but the equivalent typescript is (a | null) | null, which is equivalent to a | (null | null) which is further equivalent to a | null.
So nullable types are less powerful than option types. I'm not certain that they're worse, because dealing with nested option types can get confusing, and maybe making them impossible to express would encourage API design that requires less thinking, but unfortunately I think it's more likely that it would lead to API design with more footguns.
They touch on why they're different a bit, but it's kinda subtle.
foo(int? i) {
if (i != null) {
print(i + 1);
}
}
In that block, you can see that the compiler can understand that the null check on `i` means that it's safe to use `i` as an int within. Likewise, I can call `foo(int? i)` with `var something = 1`. `something` is an `int` and all `int`s are also `int?`s (but `int?`s aren't `int`s without guarding against nulls). By contrast, if a method takes `Optional<Integer>` then you have to wrap every `Integer` you have to call that method (with something like `Optional.of(something)`).
The author might be wrong about Swift being an example of Solution 1 (looking at Swift's nullability syntax).
But the point is that nullable types provide a bit more power because they're not just a data-structure, but a language feature. If you look at the source of `Option` in Rust, it's an enum like `Result` or (kinda) anything you could write rather than a language feature. Rust has built-in some things like the `?` "try" operator for special enums like `Option` and `Result` to unwrap them (or error), but it isn't quite the same as the null-guards in Dart/Kotlin and you still need to wrap them for functions like `Some(myVar)` rather than being able to pass `myVar` directly for an `Option<T>` parameter.
Again, they are almost the same, but they do talk about reasons for choosing Solution 2 over Solution 1. In Rust, you pattern-match like `match myVar { Some => x(); None => y() }` which is very functional, but they wanted something that felt like traditional null checks with conditionals. They note that nullable types are basically erased at runtime - an `int?` at runtime is either something like `7` or `null`. At compile time, you've checked that you're never assigning a `null` to an `int`, but the runtime doesn't need to know anything about it because the compiler has checked everything. Rust's enums aren't just a single special case, but something that you could make more complicated. Maybe you want a `SchrodingersResult` which could have `Success`, `Err`, or `Huh`. Ultimately, something like `Option` is an algebraic data type that you can compute off of - ex. pattern-match. The runtime needs to know that it's an `Option` because you can write code for that.
It is almost the same. The question is whether you create a one-off language feature for nullability and get advantages like knowing that any `int` can be assigned to an `int?` without needing to wrap it or whether you decide to create a type that works like any other type in your system like Java's `Optional`. Both are reasonable ways to go, but they have subtle differences and the article outlines a lot of those differences.
No, this is valid in languages with nullable types:
String s = null;
But this is not legal in languages without them:
String s = None;
In those languages, in order to have a value of None, it must be of type Option<String> (or whatever the syntax is), and in order to get a value of type String, you must assert its presence.
This is a fundamental difference with many ramifications. It really isn't just "the same but with better compile support".
This is a compile error in Dart now. We added nullable types in order to make all other types not nullable. So unless you explicitly opt in to nullability by putting "?" on the type, you can a type that does not permit null.
> No, this is valid in languages with nullable types:
> String s = null;
Only in languages with default-nullable types, (where the non-nullable form, if it exists at all, would be something like “String s!”); in languages with explicity billable types, the above is not allowed, because you would need to explictly opt-in to nullability:
This comment is misleading. They may appear to be the same but they are not because in certain languages (like Rust) you are forced to handle the `Option` case when a value is `None` which guides a programmer's thinking in the direction of what to do in that situation.
Without these higher level types runtime null pointer exceptions are very common, using `Option` or `Maybe` creates a situation where these sorts of errors are a lot less likely.
> They may appear to be the same but they are not because in certain languages (like Rust) you are forced to handle the `Option` case when a value is `None` which guides a programmer's thinking in the direction of what to do in that situation.
They're not the same, and the biggest difference is explained in the Map<int, Option<String>> example: union types merge different uses of null that you usually don't want merged.
> I don't get it. These are literally all exactly the same thing, all slightly varying in ergonomics and compiler support. They may differ slightly but to call them two different categories of solution is just creating a false dichotomy for yourself.
Option types can be nested, nullable types cannot.
That's why I laugh when I see that a language "doesn't have null."
It's more like newer languages make it hard to accidentally have a null reference error.
Nullable versus Option just seems like a semantics argument. The discussion makes sense when designing a language, but when choosing a language, it's more important to just look for "compiler makes it possible to enforce that a value isn't null."
BTW:
In Rust, calling unwrap() on an Option can panic. It's just that the compiler will prevent you from passing None to a function that expects a value.
C# has nullable scalars: (int? char? float?, ect.) They compile to a Nullable<T> struct, so you can do a lot of generalizing like you can with Option<T>.
Back when I was really into FP, but didn't use functional languages, I decided that the humble list was the answer to all my problems. Operations over lists behave the same regardless of their length, and a 0 element list is not a special case. Say you want to send mail to someone's mailing addresses:
addresses = user.addresses()
for address in addresses {
send_mail(address)
}
If they have no addresses, the program doesn't blow up. If they have one address, they get mail!
Of course, it's possible that having two addresses is just as annoying as having 0 addresses, in which case lists don't help you. Someone will add a second address, and then you won't know where to walk to to go see them.
I'll also point out that it's the same number of lines of code as just handling null:
address = user.address()
if address != null {
send_mail(address)
}
So it's kind of a wash, and I haven't really thought about it since.
In many cases you'd want your program to blow up in some form when trying to send mail to someone without an email address, because that would be better than silently assuming they received a message you never ended up even trying to send.
While this is true, this kind of flexibility makes the other layers harder.
On the database side, if you use a relational store, it forces you to normal form 5 which few people are used to.
On the front end, allowing everywhere an arbitrary number elements instead of mandatory one, means you need custom widgets everywhere. You need a way of marking the default one. Validation also gets exponentially more complicated.
On the UX side, people are simply not used to this level of flexibility. It will get labeled as impenetrable.
As much as I like the idea, and believe it is elegant, most probably YAGNI.
I know this is just a throwaway example, but feel just looking at the code layer this kind of elegance is comes at the cost of clarity and conveying intent, and I see that happen a lot.
The second method explicitly checking for null is great because it's telling me so many things very plainly:
1. There can be no address for a user
2. There is otherwise one address per one user
3. If there is no address we explicitly do nothing (if clause with no else)
If we do the same analysis for the first method it actually feels like it's conveying false conclusions. It's implying there's multiple addresses per user, for example.
And the fact that if a user has no address we don't do anything is still true, but the intent feels buried in the behavior of the loop.
By default I try to value writing code in a way that conveys the most intent possible over how it feels to write
For example, another language with nullable types (Kotlin) has
user.address()?.let { it.sendMail() }
I've seen people write code like this, and it just feels so unnecessary, how much longer does it take to parse? And it still doesn't have the clarity of intent that a simple null check has.
Me and my team prefer Option to nullable types. Option is much more clear and idiomatic FP than nullable types.
Granted, Option.map and flatMap and nullable type ?.let{} are functionally equivalent but the former just reads so much better, and we want not just functor and monad but applicative as well, no?
Eg given a List<Option>, we can use applicative to declaratively and with referential transparency etc etc (you know the usual FP sales pitch) turn it into an empty list if it contains a single non-populated option, or a list of Foo if all options in the list are populated.
With nullable types I don't think you could do it as declaratively and elegantly.
Actually, even if you could, it's kind of beside the point...
To me, the point is, Option has been designed from scratch to be an FP style Maybe type with all that comes with that in terms of it being a functor, applicative functor, monad etc etc whereas while nullable types in some ways are functionally equivalent, a nullable type doesn't implement a Functor interface, or a Monad interface, or an Applicative interface, and when it behaves like it does, it's mostly kind of by accident driven by a pragmatic need, not any real understanding of reusable, lawful FP abstractions.
> it is entirely possible to live without null, and languages like Rust do.
It seems as though there is this common misconception that Rust does not have a concept of null. Rust does have null pointers [1]! The reason many people do not see null often is because working with and dereferencing raw pointers is an unsafe operation
> It seems as though there is this common misconception that Rust does not have a concept of null.
I don't think that's quite relevant, though. Rust doesn't have a concept of nulls like like Java, Javascript, Python, etc do in which just about any variable might contain a value or null. Rust certainly doesn't have that.
I would not say that it is not relevant. Raw pointers are a fundamental part of the language. It is just that Rust abstracts over them so that many users do not have to worry about them.
I don't think people mean it doesn't have a raw null pointer. Normally it's about a null object reference that you can assign to (almost) anything like Java's / c's null.
I agree that union-typed null was the best solution for Dart, but I think many of the article's criticisms of option types aren't good.
> We can’t perform arithmetic on an Option<int> any more than we could on a List<int>.
Why not? One sensible definition is
Some(x) + Some(y) = Some(x + y)
_ + _ = None()
> In fact, with Dart, we’ve found that most existing code is already statically null safe
This is a very good reason to use union types — lots of programs and lots of people are used to thinking this way, and pulling that out from under their feet wouldn't be very nice.
> Nullable types, since they have no explicit runtime representation, are implicitly flattened.
Yes, flattening things that shouldn't be flattened is the biggest problem with union types. Different people will assign different meanings to null, and it's a real headache when trying to write generic code. Discriminated unions make this a no-brainer.
> With nullable types, since there is no representation difference, you can pass a value of the underlying type directly: takesMaybeInt(3);
That's a very minor difference, but if it's that important, Rust lets you write
It really depends on what the meaning of None is. OP's definition makes perfect sense if it means "unknown". Yours make sense if it means "zero", but that's far less useful. If it means "missing" - which is what the article says they wanted - then any operation involving None should be an error (but Some can still be handled automatically).
It can be interesting to see what other languages do in this situation. For example, C# got nullable value types in version 2, and had to decide what to do with operators in a similar vein (including pre-existing overloaded operators in user code) - they called it "lifting":
You might notice that it almost, but not quite, has consistent semantics: null means "unknown", which is particularly obvious from Booleans: (true | null) is true, and (false & null) is false, but (true & null) and (false | null) are both null. However, comparison operators aren't consistent: you'd expect x == null to be null if it meant "unknown", but the language will always give you either true or false; and ditto for relative comparisons. This last one means that it's possible for (x == y), (x < y), and (x > y) to all be false.
What's interesting is that, in practice, it's rare to see reliance on any of this behavior in idiomatic C# code - "null" is usually used as "missing", not as "unknown", so all this magic is, at best, irrelevant, and at worst, actively harmful (because it defers a logic error).
> It really depends on what the meaning of None is.
Exactly. This will generally be application specific so we can't choose The One Right Definition for adding `Option<i32>`s.
> Yours make sense if it means "zero", but that's far less useful.
Not in my experience but I agree there are different situations. Personally, I'd default to `Option<i32>::None` meaning "no number there, so just skip it"[1] and prefer `Result<i32, E>::Err` to denote "there should be a number here but we don't have one", which would behave like a SQL NULL (propagate in all calculations).
1. What's the length of data in a header-only packet? I wouldn't say it's zero, especially when trying to fit a read() like API on top of a framed protocol (read() returning Ok(0) marks the end of file).
> Different people will assign different meanings to null, and it's a real headache when trying to write generic code.
This is a legitimate concern, but my experience has been that it is a surprisingly rare problem in practice. We software engineers are trained to believe that any system that supports 1 of something is obligated to generalize and support N of them. But you'd be surprised how far just one sentinel value gets you.
I mean, one of the key observations underpinning having nullable (and thus non-nullable) types is that most places in your code need zero of these sentinel values. Most type, around 90% the last time I tried to count, don't permit null at all, so needing two distinct absent values is quite rare.
The bigger problem in generic code is that flattening means that you can't distinguish between incoming nullable values (that you don't know are nullable, because it's a type parameter!) that are null, and nulls in your own code.
Yeah, it is a rare problem, but it's not that rare when writing anything generic. For example, look at all the special-casing of null (ctrl-f "null") in the Java HashMap docs.[0] All of that would immediately go away if they didn't have null.
I sometimes wonder if the problem is that we don't have enough nulls.
null is typically used as a flag value, but the meaning can be ambiguous: maybe it's the absence of a value, maybe an error occured, etc. Sometimes it has more than one meaning for the same type.
Maybe types should be allowed to declare multiple nulls (effectively like an Enum in java) for different flag values. Operations on the different nulls would throw specialized exceptions indicating why that particular null value exists. Equality... hmmm.
Completely reasonable request, easy to implement along the lines of:
public class Name {
private Name(String name, int flag){...}
public static Name asMissing(){new Name(null, -1);}
public static Name asNotApplicable(){new Name(null, -2);}
public static Name asNotAvailableYet(){new Name(null, -3);}
public static Name asForbidden(){new Name(null, -4);}
public static Name create(String name){new Name(name, 0)}
public boolean isMissing(){flag==-1}
public boolean isNotApplicable(){flag==-2}
public boolean isNotAvailableYet(){flag==-3}
public boolean isForbidden(){flag==-4}
public String get() {if (flag!=0) throw new Exception(flag) else name}
}
Most of those meanings don't make sense for the type itself to define, because it's the context where the type is used that determines whether something can be missing, or there can be an error etc. ADTs solve that.
Nah, I was in early on both Swift and nullable dart - at the end of the day, Dart is preferable because it's using flow analysis to know when operations are safe, avoiding the noise of if...let and guard... (Yes, those are nice too, Dart is better! It feels much more ObjC-y)
Working with Dart now and it looks like it’s straight from the 90’s in a lot of aspects. It is not s bad as vanilla JS but when given a clean slate to design a new platform so many better choices could have been made...
I use Dart every now and then and for me "straight from the '90s" is a benefit. I learned the language in about a day. IDE works great, no issues with autocomplete whatsoever (which is often an issue in "modern" languages).
Most mobile developers come from Kotlin or Swift, that are pretty similar to each other and Kotlin is the official language for Android anyway. I had no trouble doing Kotlin coming from a Swift background, the differences are related to the fact it's a JVM language running on Android but the ideas are largely the same.
It would make bridging for native plug-ins a lot easier as well. I don't know how Google Flutter interfacing Google Android could look so medieval when they're basically the same company. They managed to make Xamarin look great in comparison.
I think saying that Option types are different from nullable types is not true. They differ at the semantic level, around how you can interact with them (one requires you to case on whether or not the value is there, the other requires you to do that with an if statement), but at the type level the describe the same construct.
I think saying something is not the same as something else, when evidently the difference can to reduced down to syntactic sugar is a bad way categorizing and differentiating types.
The article mentions unions vs discriminated unions and then mentions nesting nullable being different than nesting optional, but it doesn't really tie the knot.
Imagine (sorry for pseudocode):
type Option T = Some T | None
type Nullable T = T | null
The difference here is that Option "tags" each part of its union, which guarantees that the parts are disjoint. In Nullable, the parts of the union are disjoint only if T is not itself nullable. If T is S | null, Nullable T is S | null | null, which is just S | null (since null and null are not disjoint.
I think I'd put it even more on the syntax. Option requires you to explicitly deal with the fact that it is an Option. For nullable types, frequently the syntax doesn't require you to acknowledge that it is nullable at all, and will allow you to continue forward on the assumption that the object doesn't contain null.
This is a very readable writeup. From a distance, Dart seems to focus on usability, while other languages focus more on simplicity (Go) or expressive power (most modern languages). I appreciate the focus on usability.
It's not unique to Dart (Python, ...), but many languages that are popular around here tend to favor expressiveness over usability.
> Dart seems to focus on usability, while other languages focus more on simplicity (Go) or expressive power (most modern languages).
Author here. I think we try to focus on all three, but the latter are largely constrained by the language as it was designed in Dart 1.0 (which was mostly done by a different set of people than the ones who work on the language now).
I do wish Dart was a simpler language and worry a lot about the added cognitive load of the usability features we add. But it's really hard to take away features. In order to make non-nullable types a non-breaking change, we added a general versioning system to the language [1]. We have to use it carefully, of course, because users don't like migrating code. But I have some hope that we can use that to gradually simplify the language over time. Complexity has a direct usability cost.
We also care a lot about expressiveness, but I think other languages tend to define that in terms of "how many different things can you express in a minimal amount of code"? With Dart, we weight more by how often you want to certain things, and by how you want to express that. We're willing to make some things more verbose if it makes common things shorter or more familiar.
You don't necessarily have to take away features to reduce cognitive load. You can actually reduce it by adding features in some cases.
For example, I believe that adding Sum Types would actually simplify most class-based languages, because it would mean that you can use a straightforward tool with a 1:1 mapping from real-world concept to language concept for representing "or" (a very fundamentally logical construct!). As-is you have to use an awkward inheritance pattern with classes. Or use a union and keep track of the tag manually. There's an extra feature, but the usages of the feature become straightforward. It's the same principle as writing longer but more straightforward code rather than a gnarly one-liner.
Wow tough crowd. I appreciate the pragmatic approach taken here, and as long as there is type safety round nulls im happy to use the language.
Nitpick: As of C# 8 they’ve mostly solved nullable types for classes too with nullable reference types (though retrofitting it made it awkward and imperfect).
If you talk about type systems, the static FP folks naturally show up and according to many of them, anything that SML/Haskell doesn't do must be wrong. :)
I guess a third option would be to use an "Option" but with much more syntactic sugar. So, rather than calling func(some(3)), you'd just call func(3) and the compiler would automatically wrap it up. Func would be declared this way: def func(int?) to indicate that it's an optional type. IMO you get the best of both world. And then, in many places in the code, you'd have to do "r?.foo()" if r is an Option type. It's different than javascript where the "?" isn't enforced by the compiler.
> So, rather than calling func(some(3)), you'd just call func(3) and the compiler would automatically wrap it up.
You can do implicit conversions and that helps somewhat, but it's not quite the same as actual subtyping. In many cases, there's no natural or efficient way to insert that conversion so you still run into restrictions. For example, in a language with nullable types you can write:
int sumPresentValues(Iterable<int?> values) {
var result = 0;
for (var value in values) if (value != null) result += value;
return result;
}
main() {
Iterable<int> values = [1, 2, 3, 4];
print(sumPresentValues(values)); // <--
}
The marked line is passing an `Iterable<int>` to a function expecting an `Iterable<int?>`. That works because `int` is an actual subtype of `int?` with no conversion required. With a boxing step, you'd need to somehow wrap the entire collection in one that does the conversion.
TypeScript also uses union types for null & undefined. But then on top of that, it has optionals. So you can have:
function foo(x: number | undefined);
or you can have:
function foo(x?: number);
The type of x inside foo is the same in both cases, but the type of foo differs. The first version can only be called as foo(x) - where x is possibly undefined - but the second one can also be invoked as foo(), which then has the same meaning as foo(undefined).
But this is mostly because they were trying to come up with a type system to capture existing JavaScript patterns. I doubt it'd look like that if it could be designed from scratch.
In terms of compiler comfort I'd much rather the compiler be able to look at func(3) and deduce that "func accepts an int" rather than "maybe func accepts an int and maybe func accepts an Option<Int>"
Good article, but there is no "choice". You need both and the article even explains why!
You need union types if you have subtyping to avoid wrapping/unwrapping all the time.
And you also need option/maybe types for the cases where you must not lose information at runtime. These types can implemented by union types even, by simply "tagging" (= wrapping) the runtime values with a class).
> You need union types if you have subtyping to avoid wrapping/unwrapping all the time.
In my experience, the option types are always more concise. That's mostly because having option types encourages people to build out nice helper functions like these[0] and these[1]. Haskell's do notation and Rust's Try notation also save a lot of space.
I was actually not referring to specifically option types but generally sum types. E.g. inside a function I have a result of (union) type "Integer | ErrorA" in one branch and "Integer | ErrorB" in another branch. ErrorA and ErrorB are completely unrelated types (coming from thirdparty libraries, can't change them).
With union types I don't have to do anything and the return type for my function will be inferred as "Integer | ErrorA | ErrorB". Or if you prefer to work with a result type / either type, then it will become "Either Integer (ErrorA | ErrorB)". In most languages I'd have to make it "Either Integer (Either ErrorA ErrorB)" and also change the function code by lifting accordingly. That's verbose and many times I don't want to do that.
In Rust, you would implement From<ErrorA> and From<ErrorB> for a custom Error enum defined in your application/library. This will allow you to convert any ErrorA or ErrorB into Error.
Then, you could simply return Result<Integer, Error> from your function, and the conversion would happen for you behind the scenes.
Of course, unless you track the original error types, you will lose some information. But it’s a very clean way to handle multiple error types within your library imo.
> This will allow you to convert any ErrorA or ErrorB into Error.
I chose my example for cases where I want to keep track of the different error cases. But even if I don't, I still have a _lot_ of overhead for converting them.
I have this funny question all the time: why dont we just use a bit boolean for representing option/nullable/unknown instead? (one bit of hasValue/isEmpty like C# HasValue, but like struct uncertain_value<T> { T value; bool dont_care: 1; } and uninitialized value f<T>() => uncertain_value<T> { dont_care = 1 })
Given a 64-bit variable, you can fill it with 64 bits of value, or 64 bits of pointer to some value elsewhere.
With your scheme that would be 63 (which I'm not objecting to).
Pointers are already special-cased to only contain 0 already. There's no use-case for having 63 bits of pointer and an extra bit saying "ignore the other 63 bits".
As for the value case, I think the argument is pretty similar. Why store 63 bits of value if the 64th bit says to ignore them?
If you're not talking memory layouts and are looking at adding an extra field to a struct/class. Take you value field - if it's not initialised yet, and null doesn't exist, then what is its state?
if we extend the value to (value, is_null: bit<1>) we can have operated on the value field directly while also knowing whether the value is loaded or not. but you are right, maybe the compiler will outright "optimize" this to (value: align_pow2(sizeof T), is_null: byte) since the compiler will have to align value anyway, so that it will always have to expand the "sentinel".
My original idea exactly contrary to your thinking: rather than making 0 NULL and give it the sentinel we always had, why don't we just avoid using it like using some data structure magic? You know, the null is infamous for causing the special case in integral constant expression that C++ inherited from C, and is confusing too, because then 0 is can be considered as (void ) and thus can cause an overload fiasco, that consider f(void) and f(int), what do you think f(0) would be? So the gist is I always think using x=0 to be able to represent NULL/false and !(x=0) to represent true is really really confusing (at least in C/C++). This also caused other languages to have some probable cases of negative number being cast to true so like `if (-1) console.log('lol')` it will print, wat?
Also, with my configuration, not only we can always assume the value was intended to have a place to stay without indirection load, but the optional/null/don't-care/noninit will always have a stable state between 0 and 1 -- so we can clamp down without having to deal with extra states like I shown above, which is good from theoretical (because we can express its definite intent rather than guessing the type of the wrapped value) & logical standpoint (that we should not think in 3VL for null, rather, judge it by its concrete properties), not quite so in practice however (again, optimization kicks in nullifying all the good works), this might not entirely escape the pointer-not-null-then-consider-valid problem (what if we loaded a garbage value that might have a random 0/1? nowhere to tell also).
In fact, this kind of configuration is not rare to be found: an open addressing hash table can use this kind of setting to represent "tombstone"/deleted field, but in a more optimized manner that we "bit set" because we are going to allocate a contiguous block of memory anyway so like the current universe can have 32 entries then we use lg(32) bytes bit-set at the beginning/end, if insert or delete value at entry x then toggle bit-tombstone of x, if search value y then find first i, x in entries if compare(x, y) = exact and !bit-tombstone[i] then (yes, i) else no...so on.
But at the end of the day even that I don't like using 3VL to represent NULL, and my configuration is still like mapping 3VL into double binary anyway, iirc its like if value = U then don't care (either true and false; neither true nor false) else value.
An analogy for NULL: if addr = NULL then value is no where to be found/random garbage value else load memory from addr.
Had wondered this for a bit myself - I'm a bimobile developer, ObjC, Swift, to Java, to Kotlin over the last decade, now on Dart.
Dart is by far the most fun I've had in a language because it has an extremely simple surface area, and they really optimize for that, ex. here. But, Flutter is the revolution - utterly paradigm-shifting tool. Much like ObjC, you probably use Dart because it's what Flutter does, but not like ObjC, the simplicity, ease, and clarity of thought it allows is a benefit, as well as Swiss he army knife way the language can be deployed (Android app? Web app? Linux CLI tool? Windows 10 app?)
You seem to have a very strong opinion about the issue, but I cannot tell whether you prefer a missing value to be represented by null or by Option.empty.
If we look at set theory the NULL set is a valid set. The problem with NULL is often it's just treated as zero or if passed to a language that does not have NULL it's pretty often to just treat it as Zero. This bit equivalence between zero and NULL zero is often what you see lead to problems.
It's why for instance with GPS coordinates NULL island exists.
However, errors aside NULL is a perfectly logical value to encounter or need.
The main change with adding nullable and non-nullable types to the language is that we no longer treat Null as a subtype of all types. Instead, you have to opt in using "?", which is basically syntactic sugar for "| Null".
NULL can be its own type, but the union between the set of integers and NULL is equally valid. However, that new set created by this union is not just integers.
To represent a type that can be NULL or an integer it should be a different type.
Like I said the problem again is mostly your data is not always contained to your language. If you export GPS coordinates some systems may treat this composite type incorrectly. So even if you have a strong type system people will just be lazy.
I give the author the benefit of the doubt but Optionals are so much more powerful than what this article covers. You can map, filter, reduce, chain, compose functions that all work with optionals, and lift functions that work on numbers (or any other type) to be functions that work on Optional<number> but none of that is even mentioned in this piece (likely cause then its harder to justify picking nullable types instead)
I also found the Some(Some(3)) example just plain wrong. In those scenarios, typically you just use chain (aka flatMap) instead of map... Seems odd this was even included as a motivating reason when this is very much a solved problem.
I'm certainly biased, as I think FP is an incredible mental model for coding and it seems to be catching on in lots of places too.
I understand language designers have to make choices that benefit their users now and be as simple as possible to opt into without any overhead to account for changes, but man, what a missed opportunity to give developers better tools and mechanisms in addition to opening up the doorway for many people to functional programming. But even Java has support for optionals.
Overall, this appears to be like a missed opportunity and the justification doesn't seem to really reflect an understanding of the benefits optionals provide, and misconstrues their use and capability.