Is using OData\IQueryable in your Web API an inherently bad thing?
I recently came across this article from a year or so ago, along with some comments and analysis on the matter. The writer and many commentators have a very strong opinion that IQueryable is a leaky abstraction, that this is bad, and that basing APIs around it is also bad.
Now I know these posts are old, but as I’ve recently built an API which exposes an IQueryable, I thought I’d weigh in. Partly I’m curious… I’m interested to know if people still think this way given the recent shift towards REST and HTTP centric APIs in general. But it’s also connected to other debates that are more pertinent, such as this one regarding the Repository Pattern, and also something that goes hand in hand with IQueryable… OData.
The OData Issue
I’m going to start by setting out my position from the outset. I think that OData is misunderstood and has some very useful components… but at the same time it is bulky and tries to do too much. The concept of metadata reeks of WSDL, the current implementation is tightly coupled to Entity Framework, many of the mechanisms\outputs are noisy and a there’s an overall, very particular approach reminiscent of learning to work with WebForms.
Where OData has got it very, very right however is with the URI Query syntax. This provides a relatively clean, pragmatic syntax allowing the client to request filters, paging\ordering and projections on an endpoint that the server will then honour when providing data… all via the querystring. And to top it all off, this is standardised and pretty well documented.
I’m a great believer in standards as a way of encouraging useful, interoperable frameworks\libraries (promises anyone?) but Microsoft currently dominates both the standard and the implementation and this seems to have a negative network effect that prevents the true usefulness of this syntax from gaining ground. And that is something that I would like to, and am currently trying to address.
I feel I should also add a disclaimer here before we get in too deep. I do not advocate OData query syntax for use with complex domains or DDD. Where this standardised syntax is useful is to augment API methods that provide data that may be displayed in a form of list, tabulated or used in charts and graphs.
OData – the good parts
Now that I’ve hopefully made my position clear, the rest of this post is going to focus entirely on the query aspect of the OData standard that I have mentioned. This feature has ended up tarred with the same brush as the rest of OData, and has had the same vehement criticism directed at it.
To dismiss such a powerful, useful tool for developing APIs for this reason is absolute madness. I’m actually pretty shocked at the number of people in the community that I respect highly for their skill and pragmatism that won’t even give OData query syntax the time of day. Yet when I have conversations about it, it turns out that a lot of the major problems don’t really exist at all.
I’ve made it very clear that I’m not currently interested in the rest of what OData has to offer… this isn’t to say that it’s inherently bad though. For experienced OData scholars designing CRUD based systems, or for use in Sharepoint style services I’m sure it has a lot of merit. I just tend to design my APIs so that they draw more from RESTful principals, and wider OData features aren’t generally flexible enough or as widely adopted.
So what’s the beef?
Common criticisms of OData that people have include:
It’s not RESTfulIt encourages anaemic models and so doesn’t sit well with DDD- IQueryable is a leaky abstraction that is rarely (or never) fully implemented
- It forces you to expose elements of your DAL through your API
- It’s intertwined with MS tech
- It doesn’t work with loosely typed data
Now I’ve stuck through two of these criticisms already, as they don’t really apply to the query syntax. If we’re being real sticklers, it is kinda hard to take an endpoint that accepts OData query syntax and make it discoverable as would be required for REST. But it’s by no means impossible. The second is the weakest and even daftest criticism of the lot. OData just isn’t designed for complex domain models, but exactly the opposite – anaemic CRUD stuff.
I’ll come back to the last two in just a moment, but as promised I said I’d weigh in on the IQueryable debate. It’s pretty well understood that leaky abstractions should be avoided. And that inversion of control is something that makes code easier to write, and maintain. We also all know that when it comes down to it, it’s really about how much an abstraction (particularly a header interface) brings to the table vs. how much pain it causes us so I won’t be drawn on a debate about that.
IQueryable in my book is a complete no-brainer. It gives us a common syntax – Linq – that we can access as an expression tree and compile to whatever we want. You can use it to control exactly what comes back from the provider, reducing query times, bandwidth and throughput. These things in itself should make it worth a fair bit of pain. Sure it’s pretty extensive and hard to implement a provider yourself, but it’s nowhere near as hard as implementing the whole thing from scratch.
An interface is not a runtime contract
At this point I’d like to make an observation on leaky abstractions. If you take a bunch of classes providing slightly disparate functionality, and then hide them behind the same layer of abstraction then this is leaky. But flip it around and consider designing the interface first. Now if someone implementing this interface chooses to throw a not implemented exception, this is not the same as being leaky.
No-one said that in your implementation all the methods have to be useful. An interface is not a runtime\library level concern, it merely specifies that a class will provide a set of members. And yes, IQueryable is is a header interface… but it’s a damn useful one and if you are really above that then you can have fun getting reimplementing it yourself in a coffee shop on your Macbook Air while the other hipsters watch.
Of course having said that, the onus is on you not to make a flawed implementation, or no-one will use it and that would be your own fault. But if you make an IQueryable provider that is useful and gets used (as many people have), then that speaks for itself more than any argument I can present.
But I digress… lets go back to the criticisms that we have left:
- It forces you to expose elements of your DAL through your API
- It’s intertwined with MS tech\Entity Framework
- It doesn’t work with loosely typed data
Notice anything? These are all criticisms with one particular implementation rather than the standard itself. And what do we expect when a swathe of the community has overlooked it? There’s only one way to rectify the situation and iron out the remaining niggles… contribute to the standards process or implement a better version.
How you should view OData Query Syntax
Rather than focusing entirely on the negatives, lets end by taking a look at some of the awesome things that standardised query syntax does give us:
- Provide rich data features quickly and easily
- Apply filtering to any endpoint you like quickly and easily, and store filters as raw OData queries to be retrieved later
- Benefit from third party components that also work against the standard, such as Breeze.js or KendoUI
- Project data inline so you don’t consume any more bandwidth than needed
- Queries against DTO projections can filter down to your DAO without directly exposing it, ensuring your database doesn’t do any more work than needed
In Summary
You can argue that OData itself rightly receives a certain level of criticism, however the query syntax standard definitely does not deserve this treatment. It is elegant and powerful with great performance boons, yet it knows it’s place and doesn’t try to be something it’s not.
If you have a rich domain model then it may not be for you, but regardless I urge you to take another look at this highly underrated aspect of OData! If you are curious to find out more, take a look around this blog or search on google for alternatives to the Microsoft offering.
Pete
P.S – Comments and discussion are particularly welcome after this post. The opinions expressed here are my own and I’d love to hear any counter arguments and thoughts!
References
http://blog.ploeh.dk/2012/03/26/IQueryableTisTightCoupling/
http://www.infoq.com/news/2012/03/IQueryable-api
http://tech.pro/blog/1191/say-no-to-the-repository-pattern-in-your-dal
http://www.odata.org/documentation/odata-v2-documentation/uri-conventions/#4_Query_String_Options
http://en.wikipedia.org/wiki/Open_Data_Protocol
http://www.nuget.org/packages/microsoft.aspnet.webapi.odata
http://stackoverflow.com/questions/9577938/odata-with-servicestack?answertab=active#tab-top
https://www.oasis-open.org/committees/tc_home.php?wg_abbrev=odata
http://github.com/roysvork/LinqToQuerystring
http://www.breezejs.com/
http://www.kendoui.com/
roysvork 
About IQueryable and leaky abstractions, those are valid concerns when defining a repository contract. Odata is a protocol of querying remote data and it sits below the repository level. Using IQueryable to work with OData is like using EF with Linq2Sql, it is a way of accessing data and it’s nothing wrong with it.
As long as the repository shields the rest of the app from the data access details, it doesn’t matter you’re using IQueryable for OData or doing black magic. And you can use OData with DDD too. See how the repository solves ANY data access problem? 🙂
If the question is about whether OData query language is good or bad (as query languages go) then it’s a yes. Having a query language on your API allows clients to tailor their requests
If the question is “is exposing IQueryable a bad thing?” then I say, “it depends on who the consumer is”. Throw it open to the public and you will run into problems, whether it’s from ad-hoc querying or people complaining that they can’t get the server to do a join between the PhoneNumber field and UserId. For developers with good communication lines to the API developers, it’s probably fine.
I think a way to work with IQueryables on a public API would be to expose well defined aggregates that have been loaded from the database using a named/parameterized query. This will prevent both ad-hoc querying from choking the db, and NotSupportedExceptions. This does in some ways reduce OData to a query language though, so maybe that’s just what you were saying!
Sorry I couldn’t quite understand your first paragraph… are you saying OData query syntax is good as they go, or are you saying that query languages as a whole are generally a bad idea?
I think we are on the same page otherwise though… you definitely have to put various safeguards in place to prevent misuse such as throttling\max page sizes and I would never condone exposing your DB model directly, always work through a DTO. What some frameworks allow you to do is to project to the DTO and *then* query it, all while still in the context of the database which is what I am advocating (although is not supported via MS’ implementation).
I’m not saying they are a generally bad idea, but I have a notion that a sufficiently rich RESTful api might not need a query language, as the necessary state transitions for transforming and filtering the resources could be in the hypermedia content.
Of course that’s just the restafarian in me talking, the pragmatist says you’ll always want a query language defined as part of the known protocol so you can make efficient requests.
I don’t understand how OData is tightly coupled to Entity Framework. OData is a specification. It can be implemented with different persistence mechanisms and even in client layers such as JayData.
I think the argument that the authors of the references are proposing is that the tight coupling comes through the use of the IQueryable interface\contract, as this exposes various methods which may not be implemented in providers other than EF.
By this logic, as OData in WebAPI depends on IQueryable this is incorrectly claimed to be a failing of OData also. In fact, almost all of OData’s purported failings originate from the implementation rather than the standard.
This is quite a shame, because OData query syntax as a standard is very powerful, as I will demonstrate in upcoming posts later this month!
Odata 4 is an OASIS approved standard. It is true that MSFT technologies easily integrate with it (they created it) but nothing stops you from abstracting the interface and controlling the queries before they go down to the DAL (although it contradicts most of the examples on internet). Nobody said that you should use EF at all. In fact, you can use anything since you can parse the queries or convert them into Linq expressions if you use .NET (you have plenty of client libraries for different languages). Additionally, you can manipulate those expressions to avoid injections as well as unauthorized access to certain properties. You can prevent network throttling as well as any other API. Caching can be more troublesome since the API explose a dynamic interface but you could Cache at repository level then 🙂
One of the main benefits with OData, as partially mentioned in the article, are the query capabilities. When designing an interface that has to provide data for Mobile, Tablet, Desktop, or even Office or dashboards; OData becomes very handy. With traditional Web APIs you will (most likely) end up with hundreds of DTOs (or explosing unnecessarily big DTOs) that you have to maintain.
With Odata 4 you can expose non-typed objects (e.g. dictionaries) so you can include few extensibility points. Additionally you got Enum exposure (so you can avoid magic numbers / strings travelling through the network) and finally they support actions as well. Actions look very much like any standard WebAPI but the different is that the model EDMX (which has also been documented by OASIS) will expose the returned type 🙂
OData works great with IoC since you can define the interface using what in .NET we call (code first). You can have your own DTO modeling of your domain and then inject those entities when the different modules are being loaded Extending the interface.
Finally, OData, as WebApi, uses Controllers. If using Owin, you can use the pipelines to easily introduce power mechanism of logging, User access controls, Versioning, etc.
Best regards,
Alex
I think you can use DTOs and you don’t need to return IQueryable. By looking at the implementation of EnableQuery it internally uses ODataQueryOptions that you can specify directly as a parameter of your action in your MVC controller. Then you can pass that ODataQueryOptions down to your service layer or somewhere and call ApplyTo on it to apply it to the specified IQueryable.
The problem is EnableQuery attribute handles errors and creates error response as well so you would have to move that logic to some other custom action filter attribute like ValidateQuery. It’s pretty tricky and undocumented though. The only thing they documented is EnableQuery attribute.
This still forces your other layers to depend on ASP.NET/ASP.NET Core. But why would you want to use anything else in C#?