This problem could be solved in a RESTful way by returning a resource that includes all the needed information. I understand he describes that as not being robust, but that doesn't mean it isn't RESTful. In fact fielding acknowledges this very trade-off:
"Like most architectural choices, the stateless constraint reflects a design trade-off. The disadvantage is that it may decrease network performance by increasing the repetitive data (per-interaction overhead) sent in a series of requests, since that data cannot be left on the server in a shared context."
Also, because REVAT URLs are randomly generated the server needs keep track of the resources being pointed to instead of being able to understand it from the semantics of the request. Which works fine right up until you have to scale your application to more than one server. Now you have to deal with the complexity of keeping a distributed system consistent - otherwise those REVAT URLs might sometimes return a 404 - or you're limited to a single centralized link server. These are both scenarios Fielding was trying to avoid by including statelessness as a constraint.
> Also, because REVAT URLs are randomly generated the server needs keep track of the resources being pointed to instead of being able to understand it from the semantics of the request.
This is where he lost me. If you need to generate unique URL's, why not go for a hash approach ala git?
The problem described isn't a technology problem. It's a business ___domain problem, and it should be solved by fixing the ___domain modeling. The example doesn't uncover any problems with REST architecture.
If prices are changing and you need to capture them, the "Product" concept isn't enough. You need to model the concept of price, and perhaps price change events too. So a customer doesn't buy a product, but a product together with a "ProductPrice".
I generally don't understand the example. An "Invoice" object should capture everything relevant to the purchase, including prices, taxes, shipping, etc. Identifying a "Product" via ID is fine, and that Product may even have a "current price" attached to it. But in no way should that referenced Product object impact the captured Invoice object.
I don't understand. That's exactly what ___domain modeling is. Putting names to the data you need to store and, in this case, capture at a particular moment in time.
Let's put it this way. Do you think it is reasonable to store the taxes as a reference, or the shipping cost? We would never consider it appropriate to say, "Go look up the shipping cost for a 5lb package from here to here." Oh, and by the way, this is using a live reference to a customer object, for which the shipping address might very well have changed.
Of course we would not do this. So why would it appropriate for a product price? An invoice is the capture of all this information at the moment the purchase is made. That is its purpose. If we did not need this moment-in-time capture, then the concept of an invoice would not even exist.
Coming up with names for groups of data that we want to consider as atomic is what object orientation does, it seems to me.
I also think that that is its Achilles heel - by giving names to groups, only those privy to the internal representation of the atom can combine their functionality with its data. So you need to either merge your functionality with the atom's data (aka define an instance method) thereby coupling "things" with "actions", or you need to leak the atom's internals to outsiders (aka write a method somewhere else that "knows too much" about the atom's internal representation).
Jonahx your response summarizes my feelings pretty well. To add to this, I found the post perplexing considering that REST also has cache constraints as described at http://www.ics.uci.edu/~fielding/pubs/dissertation/rest_arch.... This makes some statements in the article strange. For example: "Very rarely will a RESTful URL specify an Expires header . . ." Leaving aside that Expires is not the only cache header http offers (see http://odino.org/rest-better-http-cache/ for others), the main problem with this statement is that it suddenly makes the system he is describing not-quite-RESTful. It raises the question are you really in a RESTful style if you throw your hands up and ignore the cache part of the style. I would say that ignoring cache is an assumption that Dziuba is making which partly results in the problem he describes, along with having incorrect ___domain modeling. Restore the cache piece to the system and it would be able to deal with the stale-price-problem just fine by noticing that the price is stale and requesting a fresh one.
FTA: "To patch it up, we attach the product price to the order..."
The author already discussed attaching the price to the product into a ProductPrice. His argument continues that price is only one thing, there may be more.
FTA: "If our analytics department needs to know the product category for their reporting, or if our accounting department needs to know cost-of-goods-sold for revenue calculations, we must attach these as well, as they may change over time. As these requirements grow, you end up sending more and more of the object graph attached to the order, just to maintain consistency for everything that happens after-the-fact."
He's essentially extracted all this information out into its own class caleed "State" which can be referenced by a single ID and passes that along with his product, a ProductState if you will.
All of his examples should be solved at the ___domain level. They just don't have anything to do with REST architecture.
The information doesn't need to be sent over the wire. There are many ways to solve it, but at the end of the day all that needs to be sent over the wire is the product and the time of purchase. Everything else can be deduced on the backend, and how hard that is to do will depend on how well you've modeled the concepts your interested in.
> His argument continues that price is only one thing, there may be more.
Indeed. A new model is needed, and that new model would have all the relevant attributes. I can think of two ways to do this (using ActiveRecord-ish naming conventions).
One way would be to have an InvoiceRow to represent the presentational details of each row in an Invoice, like name and price, perhaps along with a reference to the actual model it came from, like product_id. The Product model could change, but the Invoice would still have all the InvoiceRows it needs to display and reason about that invoice without fear of mutation.
Another way would be to have a robust versioning system. So a Product might have only an id, and there could be many ProductVersions that hold all the real info (name, price, etc.) and reference the same Product via product_id. ProductVersions would of course be immutable. Then the Order model in the example would return product_version_id, and could therefore look up the actual name and price at the time the order was made.
For the narrow example from this article, I think the first example fits better and is much simpler.
The "robust versioning system" system you mentioned is a specific application of the general solution he's suggesting in the article. Essentially every data model can "freeze" its current value at a point in time, be assigned a UUID, and always have the main product_id redirected to the most up-to-date value.
You're taking his example too literally and solving for it alone. He's talking about adding a generic feature onto your entire REST API to deal with these issues in a consistent way.
I agree here, the problem is with the author's contrived example. This ___domain problem can easily exist within a relational database, rest, soap or even snail mail.
The problem is a poor ___domain model and should be fixed there and not by adding an overly complex cache or state object storage solution that seems to only apply in a REST model. If the ___domain model is fixed then REST, soap, RDBMS or anything else becomes trivial.
This does not seem like a reasonable idea in a business-rules sense. the 12345 in http://api.example.com/product/12345 is presumably a SKU, which is the natural key for a product you're selling. You don't get to convert it into a GUID and declare it immutable.
If the price isn't effectively fixed, the API client is going to need to generate a quote ('shopping cart'). The API rates the product when the quote is made, and then there is some business rule about how to handle quotes from before the price change.
The value-based URL refers to an entity ('SKU-price pair from some time in the past') which has no reasonable use.
So generate immutable ProductState objects that contain all parts of the ___domain model that can change over time, and maintain a mapping from SKU to the correct object.
So GET /product/12345 returns UUID_1 at time A, when details of the offered item change at time B a UUID_2 gets generated and a new GET /product/12345 returns UUID_2.
If you want to be really anal retentive, generate the product state objects with timestamps to compare to the Date header in requests and have a list of product states that have been referenced by a given SKU to traverse through to get correct product details even when the catalog changes between sending and receipt of the request.
>So generate immutable ProductState objects that contain all parts of the ___domain model that can change over time, and maintain a mapping from SKU to the correct object.
But that just recreates the very problem the author set out to solve, which is that you have to send increasingly huge sub graphs of the database with each order.
I assumed the direction he was going to go was to let the client send the at-the-time data back with each subsequent request for time consistency, but require them to pass a server-provided signature to keep them from making up prices, and saving the server from having to keep a time history of every case.
If the identifier is an SKU, it seems to follow that to distinguish between different prices one could have an additional "version" identifier? Or perhaps even a more generic on-this-date parameter, in this case representing date of quotation (which then dovetails quite nicely with the expiry mechanisms already built into HTTP).
These problems are similar to the normalization problems that occur when designing a database.
If you have tables such as Orders, OrderItems, and Items, you need to account for the attributes of Items changing over time and how that will affect past Orders. The problems are potentially bigger in REST, but they're not necessarily new problems.
Return customer and product in same HTTP response?
Also, that's not how sales orders work. A sales order is a commitment to provide a product at a price at a time to a party. The price is tied to the order. If the price of the product changes, that does not affect the order.
Memento the "HTTP Framework for Time-Based Access to Resource States" [0,1] tries to approach the "time" problem of REST with an additional set of headers and intermediate gate-keepers.
Arguably, you could the fix OP's issue by improving/changing the ___domain model. In this example, issue of state arises because the Order relies on the inherent possibility of changes to the attributes of a Product. Adding line items(quantity,price,product_id,etc) instead of direct product associations would provide a more stable representation that's easy to reason about without the additional complexity of REVAT.
As an example alternative, Mongoose has versioning in its schemas. If you GET /product/12345, you'll receive the product info, along with its revision number, let's say this is revision 3. The user decides to buy the product, so they POST /orders with {productId:12345, __v: 3}. In the meanwhile, the store has updated the product, and Mongoose updates the revision to 4. Versions mismatch, send a 409.
http://aaronheckmann.tumblr.com/post/48943525537/mongoose-v3...
So, separating your REST API into indexes and values effectively creates a database over HTTP. This transfers complexity to the client though, which I don't think is the point of having an API in the first place (you could just give clients the login to your database).
The problem that I see with REST is that it leads in general to APIs that tend to transfer logic to the client. This is basically a loss of encapsulation and makes it harder to change the server implementation later without breaking clients. It took a long time for the industry to move towards behavioural interfaces for good reasons. Are those reasons now void? I know that there is some encapsulation because only a representation is transferred but still, REST interfaces tend to represent state and not behaviour. Am I wrong?
> The problem that I see with REST is that it leads in general to APIs that tend to transfer logic to the client.
That's just bad API design -- specifically poorly chosen resources.
> I know that there is some encapsulation because only a representation is transferred but still, REST interfaces tend to represent state and not behaviour. Am I wrong?
They by definition transfer the state of a resource. So, in a trivial sense, you are correct. Substantively, though, there is no reason an individual REST resource couldn't be an instance of a behavior, or that state modifications to a resource couldn't trigger modifications to many back-end entities (and other front-end exposed resources.)
To use RDBMS languages, REST resources are often treated like base tables when they should be more like application-specific updateable views. This has nothing to do with REST architecture, and everything to do with the fact that ___domain modelling is harder than implementing an API, and yet (regardless of the architecture chosen for the API) people often spend more effort on implementing the API than designing the ___domain model.
Yes, getting a ___domain model right up front can be very hard. That's why behavioural interfaces that hide as much as possible were found to be more forgiving when we realise that we failed to predict the future correctly when dreaming up V1 of the product. They make it easier to evolve.
First, let me caveat: I'm not a REST expert, but I've been known to pretend to be one.
It seems to me you're under the impression that REST tends to lead to systems that:
1. Transfer logic to the client
2. Put the client in charge with modifying the state
3. Lack Behavioral APIs.
Too many people assume RESTful services are these simplistic services that return JSON objects, which are then modified by the client and PUT back on the server. This isn't the case. When you request a resource (think object) from a RESTful service, it doesn't just send you its state. It sends you its state and metadata describing the interface in which you can interact with this resource. If you we were talking about an HR database, requesting an employee object would not only return the employee's data, it would also include relevant metadata on how to request vacation days. That operation is not a PUT operation, it's likely you'd POST a vacation-request message to a URL that would start some sort of workflow.
A RESTful architecture is very good at behaving like a network file system by default, but it can also support behavioral interfaces in a RESTful way when you need them using POST. You may be familiar with the CQRS architecture, which explicitly separates commands from queries when designing APIs. The idea is anything queries your system should *never* change its state, and anything that changes the state of a system should be sent as a one-way message. If you've done CQRS correctly, it should be easy to log all messages that change your system's state similar to a database transaction log. A RESTful architecture makes the same demands and then some more.
1. If you're implementing GET, don't modify the state.
2. If you're implementing PUT, it should modify the resource addressed at the URL and it should be idempotent.
3. If a resource implements POST, you get none of the above guarantees.
So, I don't think REST prevents behavioral API's so much as it demands that anything behavioral is POSTed as command-based message.
Thanks for the response but I Don't think 2. I think that rest encourages thinking about the nouns rather than the verbs and ends up exposing too much state to the client. His first example shows the client dereferencing the product id in other to figure out how much to charge. The client should not be doing that. If he was thinking in verbs it would naturally come out as make_sale(product, customer). But REST has only a few verbs which kind of hobbles your ability for expression.
> I think that rest encourages thinking about the nouns rather than the verbs
Sure -- but actions and events that need to be represented are nouns, even if they are instance of behavior that can be described with verbs.
In fact, they are usually the most important nouns. There's nothing about REST that encourages missing this point, although there may be something about the common tutorial-type material which tends to focus on simple examples where, unlike most non-toy applications, this is not the case and the important entities are not actions.
But that's because they are examples of implementing REST, not examples of ___domain modelling for complex application domains.
> But REST has only a few verbs which kind of hobbles your ability for expression.
HTTP (if not extending it, which without very good cause you shouldn't in REST if HTTP is used as the communication channel) has only a few methods. But the methods of the communications channel are not the verbs of the ___domain, nor ought they map to the kinds of actions in the ___domain.
There's a particularly common, but flawed, naive approach to ___domain modelling which maps the obvious nouns of the ___domain to entities/resource in the model and tries to map actions in the ___domain to methods in the medium in which the model is instantiated [1]. But this almost always ends up biting you -- instance of actions are nouns and need to be entities/resources too. If you do this, the more-obvious nouns of the ___domain tend to end up being read-only or read-mostly entities over the exposed interface, with most or all of the mutations happening through the entities/resources representing actions.
[1] While this happens with REST, it happens with other architectural patterns as well, and with different levels of application implementation -- not just web interfaces.
Agreed, actions should be represented as nouns. It's just that rest tends not to encourage this and this seems to be what happened to the author. He should have a sale resource and put to it but because he doesn't he transfers the price to the client which then gets out of date. The client is being coupled to the server and the server lacks cohesion; all for the lack of thinking in terms of actions/behaviours at the interface level. Instead he is putting the behaviour in the client.
I saw a comment from Fielding that touted this as a benefit of rest. Expose the data and the clients have freedom to use it in ways the server never imagined. Good for mashups maybe. Not so good for accounting systems.
> Agreed, actions should be represented as nouns. It's just that rest tends not to encourage this
How, specifically, is this the case -- what is it about REST that is concretely bad for this compared to alternatives?
As I see it, REST is about what you do with your ___domain model, not how you make the ___domain model. They are orthogonal concerns; people very often suck at ___domain modelling, but that's not any architectural pattern's fault.
> He should have a sale resource and put to it but because he doesn't he transfers the price to the client which then gets out of date. The client is being coupled to the server and the server lacks cohesion; all for the lack of thinking in terms of actions/behaviours at the interface level.
And how is choosing the wrong resources REST's fault?
> If he was thinking in verbs it would naturally come out as make_sale(product, customer).
If you think about verbs as messages (data), then you would simply POST a make_sale(product, customer) message to some URL, and your RESTful interface will redirect you to the URL of the sale you just posted.
> If you think about verbs as messages (data), then you would simply POST a make_sale(product, customer) message to some URL
I would say instead that if you were thinking in actions-as-___domain-resources, then you would post a "sale(product, customer)" entity to a "sales" collection resource (via POST to the appropriate URL, in an HTTP API) -- which is pretty much the same thing, though not calling it "make_sale" is more REST-appropriate; the combination of the operation and the resource should make the "make" part superfluous.
The good thing about REST is that the limited vocabulary enforces good design practices, but it doesn't save you from fundamental problems.
On the article, that would be the conceptual mistake of considering the price is an attribute of the product, which in turn would require a second request to retrieve, then breaking the atomicity of having all the necessary state in a single request.
To solve that, he proposed a "database over REST", where the main resources act like an index - thus increasing the abstraction by one notch and moving some complexity to the client. This doesn't solve a problem inherent to REST, but rather a conceptual problem that applies to all data systems, which is how to maintain and recall states in an atomic way.
I'm delighted to see a Ted Dziuba post! I'm very sad his posts criticizing Node disappeared. I never decided whether I bought his argument, but I thought they were valuable contributions to the discussion about the event-driven model. They did not seem trollish to me but earnest, and I would have liked to read a debate about them.
This REVAT post is also something I'd love to see get picked up and discussed more. I certainly have felt the need for immutable values indicating a snapshot of a thing's state. Personally I'd like to know how people solve this at the RDBMS level more than the REST API level, but there's a need in both places.
To be fair, his argument against Node.js is applicable to any system with co-operative scheduling (twisted, tornado, eventmachine, Go etc), and the only way around it is pre-emptive scheduling (Erlang/Elixir, Haskell etc). Node has a lot of problems, but co-operative scheduling is not one of the more important ones.
This is just another design decision. In git terms, linking to a particular revision is sometimes useful. So is linking to HEAD. There's no ___domain-independent rule that says you have to support both or prefer one or the other, so long as you know what your customers need and don't get them confused.
If you really want to link to a snapshot that can be downloaded later, using a content hash like git or camlistore might be a good idea.
Why does HTTP specify such odd stuff like "never expires" being 1 year in the future, and "should not" send higher values. What does that tell a program to do? Cause instead of defining some sort of sensical behaviour ("send a thousand years as never"), you've got this odd scenario where never might be a year. Or maybe you send 2 years, and clients act improperly.
It seems that whoever wrote this spec wasn't writing much software, and didn't intent others to write software from the spec.
It doesn't, at least anymore, though I agree it would've been good if they'd specified some reasoning for that magical value. Granted it only changed in June, but he's quoting an obsolete RFC. (And if you look up the RFC on the IETF's website, the header color will indicate this, along with the "Obsoleted by…") Currently that section¹ actually reads:
> Historically, HTTP required the Expires field-value to be no more than a year in the future. While longer freshness lifetimes are no longer prohibited, extremely large values have been demonstrated to cause problems (e.g., clock overflows due to use of 32-bit integers for time values), and many caches will evict a response far sooner than that.
Honestly, they could have just specified a "never" value, like the word "never". That said, I'm not sure a forever resource would be a good idea: the other side's cache would never empty, and honestly, I don't think I've ever seen a piece of data that never changed, either because the spec did eventually change, or the data was incorrect due to human error, etc.
Further, you can't really guarantee a response will be cached forever: caches get cleared all the time for a variety of reasons. Cache to reduce load, not to put some sort of new behavior in the system.
I don't think this has strictly something to do with the state in REST. This is about state of things changing in general.
What you can do about it (other than creating something along the lines of your idea) is to create a history/snapshot on updates. I'm pretty sure most languages/frameworks has something for that, in Java/JPA you can do this by using Hibernate Envers' revisions. Exposing these revisions via REST API pretty much is what you suggested.
Many people here doesn't seem to get what the author is talking about. If I link to http://www.google.com, the content might have changed when you open it. The URL always references the state of the resource at the time of retrieval, which is not always the intention. REVAT is just a proposed way to reference a specific state of a resource.
There’s a consistency problem: the price may have
changed between the time that we received the order
and the time we looked up the product. To patch it up,
we attach the product price to the order:
What you are describing is the lost update problem. This is solved in several ways. For example, an If-Unmodified-Since or If-Match header. In the first POST method, you had to get the product ID from somewhere, so you might have obtained a date or etag representing that product at a specific point in time.
When you POST the order, you could put the date of the last time you saw the price change in a header (or in the JSON itself, if you post multiple products). If the price changed, the server could respond with a 412 - Precondition failed. If you want to post a collection of items, maybe you could use If-Match. If you want to enforce this, you could return a 428 if the client didn't include some sort of preconditions.
Every product has a price, every product listed on an order has a price, these aren't necessarily the same thing. What if the order includes something that the manager threw in for free to sweeten the deal? The price of that product on that order is zero, but the product's list price at any given point in time was not zero.
I didn't say there is a consistency problem, I quoted FTA.
My suggestion actually has nothing to do with the price of the object, it only has to do with the server state of the object (product) and whether or not that state was updated (price, quantity available, etc...), which is what the article was worrying about (the lost update problem).
i.e. if the underlying objects in a transaction changed between the time a client retrieved information about the object and the time where a user was interested in creating an order with the object, in a way which might invalidate the transaction, I'm highlighting a potential solution to notifying the client.
What if the order includes something that the manager
threw in for free to sweeten the deal? The price of
that product on that order is zero, but the product's
list price at any given point in time was not zero.
Well, in that case, I hope there is a system the "manager" can create a discount object to and attach it to the order.
PUT /orders/12345?discount=FREE_IPAD_YO
PUT /orders/12345;discounts [{"discount_id":"FREE_IPAD_YO"}]
whatever flavor you want, etc...
Note: I don't work in the ecommerce ___domain, but I do develop systems which are often transactional in nature.
As the developer of a REST centric ecommerce platform[1], I agree. The price of an order item is specific to that order and does not necessarily match the product's base price. It may be calculated using special account pricing rules, tiered by quantity, or discounted by a coupon. In such a system I have found many occurrences where data has to be de-normalized for this reason. Although the the new state is derived, it may otherwise differ from the source. Not just a function of value and time.
The proposed protocol allows for PUTs, which means the server needs to atomically check whether or not that UUID is already in the database. Using a cryptographic hash as an ID removes the need for atomicity in the case of PUTs.
My only concern is if the app wants the value immediately. I don't want to have to make a separate GET after receiving a VALUE url to get the info. I understand why this is useful if I'm passing it along, but what if I'm not?
It could return a 200 response with the actual value and the value id in an additional header. Not as elegant as 302 but more practical (from the efficiency point of view).
I think you could solve this using SPDY, which lets to send an expected follow-on request in the same stream of bytes before it has been requested - originally designed for things like CSS but it should apply equally well here (especially given the UUID URL caching semantics)
Whether or not that's more efficient boils down to how much the server can predict the client's behavior though, right?
I'm just thinking that if the server was configured to respond with the /value/ reference and then immediately follow that up with the resource, your client doesn't really have much say in how noisy the conversation is over the wire.
Isn't the solution to your problem widely adopted since the early 90s? Store a checkout, push and pop products to it and then total up the real time values when the transaction is completed. Passing the value to be charged through in the POST is incredibly insecure.
I don't think you're getting the point of the article. It is talking about dereferencing the STATE and the VALUE portions of a RESTful API. Not what's the most secure way of sending a credit card number over TCP/IP.
That's a very good point and I understand that. But webhat's comment makes it seem like the whole article is invalid or the example is not correct. I definitely would not consider this as an 'obvious error' with the example. Maybe bad practice. Would you say the copy/paste issue is solved if the author just does a mass replace from 'http' to 'https' on his articel?
I would say yes, I've also advised the author of this mistake and he declined to modify it. This is the less that friendly reply I got:
> You missed the absence of Content-Length in some of my examples? If you're going to be on this internet, friend, you've got to step up your pedantry game.
But there are no REST semantics for indicating state. REST is stateless: http://www.ics.uci.edu/~fielding/pubs/dissertation/rest_arch.... I think maybe he's conflating HTTP with REST.
This problem could be solved in a RESTful way by returning a resource that includes all the needed information. I understand he describes that as not being robust, but that doesn't mean it isn't RESTful. In fact fielding acknowledges this very trade-off:
"Like most architectural choices, the stateless constraint reflects a design trade-off. The disadvantage is that it may decrease network performance by increasing the repetitive data (per-interaction overhead) sent in a series of requests, since that data cannot be left on the server in a shared context."
Also, because REVAT URLs are randomly generated the server needs keep track of the resources being pointed to instead of being able to understand it from the semantics of the request. Which works fine right up until you have to scale your application to more than one server. Now you have to deal with the complexity of keeping a distributed system consistent - otherwise those REVAT URLs might sometimes return a 404 - or you're limited to a single centralized link server. These are both scenarios Fielding was trying to avoid by including statelessness as a constraint.