I presented my session at the 2014 SharePoint Conference today (I’m typing this while sitting on the plane on my way home) and as promised, here’s the write up. This will include all of the material I presented at SPC as well as some additional material which I did not have time to include in my session (sessions were 75 minutes long, and I watched the timer tick down the last few seconds as I was wrapping up my final slides. Talk about perfect timing!). In addition to this write up, I’ll also provide links to my source code at the end of this post. I’m not providing the slide deck here as it would simply be extraneous – all of the material is covered in this post. Sit back and get comfy, this is going to be a long one.
Here’s what the session was all about:
The main focus was to help developers pick an API with which to access SharePoint from off-server. There were quite a number of announcements in the conference and developer keynotes related to new capabilities which, when combined with existing confusion, inaccurate information and FUD, often led developers to make bad choices with regard to their API. Actually, scratch that. As I tried to get across in the session, it’s usually not a bad choice, it just may not be the best choice.
|There are very few bad choices between the APIs – one API may just be a better choice in a particular situation.
Before going any further, I should at least mention the other popular option for SharePoint development. If you have the choice to run code on-server, then you’d be using the SSOM – Server Side Object Model. This is the one Microsoft is shunting to the side with their focus on “THE CLOUD.” This is the one with which most of us SharePoint developers are most comfortable. I won’t say that the SSOM is never going away because that is obviously not my call to make. However, so long as there is an on-premises version of SharePoint, there will be an SSOM. Truth be told, the SSOM is currently the only one Microsoft actually writes and maintains – the other two are generated from it. So you could think of the SSOM as Amber, and CSOM and REST mere Shadows. (An obscure reference for the geeks reading this, but essentially what I’m saying is that the SSOM is the source of both CSOM and REST).
Anyway, that’s not what I came to talk about. I came to talk about
the draft REST vs CSOM (anybody catch that reference?). REST and CSOM are each perfectly valid choices for interacting with SharePoint from a remote location – essentially anywhere except the SharePoint server itself. With two “perfectly valid choices” how is a developer supposed to know which is the right one to use? Is there a right one? Why are there two to begin with?
That’s why this session exists. Let’s dig in and get started answering those questions and more…
First of all, there are two APIs because they have different approaches, different capabilities and different situations in which they provide value, and that’s the reason this session was created – help devs to understand the differences and know when to pick one over the other.
- .NET CSOM = just the .NET implementation
And no, there really isn’t a Silverlight-specific acronym as no one really talks about Silverlight anymore.
There are five basic elements that should be considered when you begin a project involving remotely accessing SharePoint with regard to selecting an API set. If I were to rank them in order of typical performance for the projects I work on, they would currently look like this:
- Technology Concerns
- Intellectual Property Protection
- Developer Concerns
For any given situation in which you are likely to find yourself, you may rank these differently, and that’s perfectly OK. I’d rank them differently in some situations, too, but most of the time I think they go in that order. We’ll touch upon each of these in the rest of this article. But first I want to deal with some that didn’t make the list:
· Browser Choice – this one didn’t come up in the session but I want to call it out here. Some people might think that the browser you choose is going to impact your CSOM or REST code and that is actually not true. For any browser which SharePoint supports (see here), both REST and CSOM will work perfectly well.
· State Management – I find this one interesting because in my experience few, if any, devs take advantage of this. Essentially, what it comes down to is that CSOM can maintain the state of your objects across calls back to the server. I cover this in my Pluralsight course and it is definitely a nice feature when you need it. As I said, however, most devs don’t know about it and it’s not really a very strong reason to pick one API over the other
OK, so let’s start looking at the element we do need to consider.
This concern breaks down into several sub-areas that are going to come into play in most situations. The one commonality across them is that they are typically not things which can be changed. They are environmental and typically much larger than simply accessing SharePoint for one project.
These are the easy ones because it’s completely cut and dry – things either work or they don’t. These fall completely on the pro-CSOM side as they are things that simply do not (yet) work here for REST. If you need to access either Managed Metadata or Workflow, you cannot do so with REST and so you must use CSOM.
This element concerns your basic computer (server, workstation, tablet, etc) platform and which tools or libraries you may be using. For our purposes here, computer platform breaks down into:
We can drill into this a little further to encompass more specific gradations such as iOS, Android, *nix, etc.
While you can generally make either CSOM or REST work across any of these scenarios, in general, REST is going to work better for the non-Microsoft platforms and third party libraries. It is unlikely worth the effort to run Mono just to get access to the .NET CSOM and while downloading the JSOM wouldn’t be that difficult, it’s only going to work in browser applications. If you’re making significant use of any third party libraries, the effort required to make the JSOM work (things like collections and objects, etc.) is unlikely worth the benefit provided by the JSOM. Especially when you have a perfectly valid REST implementation available.
If you’re on a Microsoft platform and/or only using SharePoint-specific libraries, then either CSOM or REST are equally useful.
Role of SharePoint
This technology concern covers how you are using SharePoint:
· As simply another data-store – the vast majority of what you’re code is doing is simply reading and writing data that just happens to be stored in SharePoint.
· Application pieces of SharePoint – your code spends more of its time working with SharePoint-specific constructs – sites/webs, lists/libraries, security, users, etc.
This element is a little harder to call a winner for because both REST and CSOM can operate equally well here. However, if SharePoint is just another data-store for your application, it is likely to be a little easier to use REST as it works with more traditional data structures – XML and JSON. If you’re interacting more with the application elements then CSOM has a slight advantage due to its similarities with the SSOM; it will be marginally easier for developers to make the client-side transition from the SSOM since the objects are going to be so similar.
How and where you are going to deploy your code is another technology factor to consider and it’s another that’s pretty cut and dry:
· If you’re deploying full-trust solutions, then most of your code is probably SSOM, but for the client side elements, you could do CSOM or REST equally well
· If you’re deploying apps:
o Cloud-hosted apps: This also falls back on the “role of SharePoint” but also gets impacted by the technology stack in use. If you’re on a Microsoft server, I think CSOM is a better choice. If you’re on a non-Microsoft stack, then REST in whatever language is supported by your server is probably best.
This last element here is pretty straightforward. There are four scenarios we’ll look at:
1. SharePoint, either on-premises or in the cloud
2. Non-SharePoint elements of Office 365 (Exchange, Lync), including the newly announced “Office 365 platform” API
3. Other Microsoft Cloud Applications – Dynamics, Project, etc
4. Non-Microsoft Cloud Applications – Salesforce, etc.
This one is pretty easy as CSOM supports only SharePoint, either on-premises or in the cloud. While the others may support some flavor of client side API, it is not technically the “CSOM” in the sense we’re talking about. This includes the “Office 365 platform” unveiled at SPC14 – that is available only via REST.
So – CSOM or REST are an option for SharePoint, for anything else, you’re using REST.
|This is a good time to stress an important point. Microsoft is committed to CSOM for SharePoint. I confirmed this with a few folks at SPC. CSOM isn’t going anywhere anytime soon, so we’re safe using it now and for the foreseeable future.
Performance is important for any application and comes in second place here only because the elements of the first place Technology Concerns cannot often be changed. We must get the best performance we can within the confines of the technology concerns with which we are stuck.
For our purposes here, performance is also going to encompass user experience as it is nearly impossible to have a good user experience with poor performance.
Performance when you’re conducting round-trip calls from client to server and back again is mostly concerned with how often you hit the wire, how much data flows across the wire per call and the total time required to complete the calls. Those are the three factors we’re concerned about here. In one of them, CSOM wins hands down, in the other two REST wins.
CSOM wins its portion because it has to hit the wire far less often than REST. Compared to CSOM, REST is exceedingly chatty. In the tests I performed, REST made about 70x more calls back to the server than did CSOM. This is because currently, REST has to call back to the server for each and every operation whereas CSOM allows you to batch calls (within some limitations) and send a series of operations to the server in a single call. This makes CSOM much more efficient.
I tested a series of pretty common operations in both REST and CSOM:
- Creating Lists (10)
- Creating List Items (100 per List, for a total of 1000)
- Reading List Items by ID
- Querying for List Items (retrieving 50 per List)
- Deleting List Items
- Deleting Lists
To complete these operations, CSOM made 42 calls back to the server with a total of about 2.5MB traversing the wire and an average payload size for each call of a little over 40K. REST, on the other hand, required over 3000 calls with total network traffic of a little more than 5 MB, but had an average payload size of a little less than 2K. Overall, each REST call was far smaller than each CSOM call and about 85% of each REST call was comprised of the HTTP Headers. If we could batch our REST calls (which IS supported by the underlying architecture but not likely something we’ll see soon) then REST would be much more of a contender here. By batching our REST calls, we’d have less calls and therefore less header traffic.
The other interesting thing here is that despite requiring more calls and more traffic (again, mostly headers) REST outperformed CSOM from a total time perspective by about 30%.
In a nutshell, there are plusses and minuses for both REST and CSOM from a performance perspective. While in general, we would award the winner to whichever took the least time (REST), the sheer number of calls made by REST and the 2x amount of data flowing across the wire were too much to overcome and we have to award this round to CSOM. For your situation, the operations you perform may not require as many calls and so REST will be a more logical choice. Remember, on a per-call basis, REST outperformed CSOM for both time and payload size, so if you don’t require as many calls, REST can outperform CSOM.
Testability is important. As I had one my slides in the SPC talk, there’s a little developer haiku I wrote which reflects this:
Unit Tests protect and calm
This ship has sailed
Unfortunately, as SharePoint developers, we tend to be woefully behind the curve in unit testing our code. For that reason alone, this particular element is bumped up in importance to being a major factor to contend with when picking an API.
Historically, SSOM has been exceedingly difficult to unit test for many reasons (sealed classes, internal constructors, etc). That has led a lot (most?) SharePoint developers to not bother unit testing their SharePoint code.
With the move to the client side, we have the opportunity to remedy this. Hopefully, the SharePoint community will take this opportunity to start unit testing more of our code.
With that goal in mind, let’s examine the two APIs from a testability point of view. While I’ve tried to stick with facts as much as possible for this article so far, testability is an area that is really a matter of opinion. Largely this is so because either API is equally testable so it really comes down to which you find easier/better. At SPC, the afternoon before my session, Steve Walker (from the SharePoint product team) had a session in which he stated that unit testing CSOM code was easier than REST code. I find the exact opposite to be true. Steve didn’t go into details on his statement, but for me, I felt that I had less test code to write to work with REST and what I did have to write was easier and more “web development” standard than what I had to write for CSOM.
In either case, a good quality mocking product is required. In my session, I showed Telerik’s JustMock, my current favorite for mocking. In the past, I’ve used TypeMock’s Isolator product for SSOM work, and there are numerous other mocking products that ought to work well, too. The mocking tool is required because we need to be able to have our test code function without the dependency of a SharePoint server. We do this by creating fake objects (mocks) as stand-ins for the real objects. For these fake objects, we control the horizontal and the vertical; we control every aspect of their existence – from what their methods do to what their properties return.
For example, when mocking .NET CSOM, I had to set up fake objects for ClientContext, Web, ListCollection and List for my demo, like so:
var mockCtx = Mock.Create<ClientContext>(Constructor.Mocked);
var mockWeb = Mock.Create<Web>();
var mockLists = Mock.Create<ListCollection>();
var mockList = Mock.Create<List>();
Figure 1: Mocking .NET CSOM with Telerik’s JustMock
Then when instantiating or accessing things, I needed to make sure that my mock objects were returned instead of the real objects, or in some cases (the Load and ExecuteQuery calls) that the mocked objects did nothing:
Mock.Arrange( () => mockLists.GetEnumerator()).IgnoreInstance().Returns(getEnum());
Mock.Arrange( () => mockCtx.Web).IgnoreInstance().Returns(mockWeb);
Mock.Arrange( () => mockWeb.Lists).IgnoreInstance().Returns(mockLists);
Mock.Arrange( () => mockCtx.Load<ListCollection>(mockLists)).IgnoreInstance().DoNothing();
Mock.Arrange( () => mockCtx.ExecuteQuery()).IgnoreInstance().DoNothing();
Figure 2: Configuring operations and properties hen testing .NET CSOM with JustMock
With the help of one more utility function, my CSOM test was complete:
private IEnumerator<List> getEnum()
for (int i = 0; i < 20; i++)
var mockList = Mock.Create<List>();
Mock.Arrange( () => mockList.Title).Returns(expected.ElementAt(i));
yield return mockList;
Figure 3: One final important function to round out my .NET CSOM unit test
(Full source code for the tests, as well as all of my session demos, is available at the bottom of this article)
With all of this in place, I could test my .NET CSOM code as part of my Unit Tests without requiring that the SharePoint server be involved, or even running. In my SPC demo, I shut down the W3C service before running the tests to prove that the mocked objects were working properly.
Contrast the above code with what it required in REST (again using JustMock) and you’ll get an idea of why I feel testing REST to be a little easier:
var mockRequest = Mock.Create<HttpWebRequest>();
var mockResponse = Mock.Create<HttpWebResponse>();
Mock.Arrange(() => WebRequest.Create(Arg.AnyString)).Returns(mockRequest);
Mock.Arrange(() => mockRequest.GetResponse()).Returns(mockResponse);
Mock.Arrange(() => mockResponse.GetResponseStream()).Returns(GetFakeResponseStream());
Figure 4: Testing REST with Telerik’s JustMock
There is actually one supporting method for the REST test (GetFakeResponseStream) that is not shown here. It is pretty simple but I didn’t reproduce it here because it contains one very long string (the content of the response) that wouldn’t show very well here. You can see it in the full source code listing.
In general, I found testing REST to be easier because it required less lines of code and because I was really dealing with testing standard web objects that are going to be far more familiar to most devs. The SharePoint-specific objects in CSOM were just a little bit harder to work with. You may feel differently, and that’s OK.
The good news is that neither API is particularly difficult to test, so there is really no reason not to test your code. Please, please, please begin writing unit tests if you’re not already doing so, or write more if you’re already writing some. The overall quality of your code WILL go up, and that’s not a bad thing.
Intellectual Property Protection
If all you do is write code for use by your company, then this element is likely not very important for you. With the introduction of apps and the SharePoint app store, however, more and more developers are beginning to produce products for use in SharePoint by others outside their company. Suddenly protecting the fruits of your labors – your intellectual property – is now important to you. For this reason, combined with the fact that it is something many of us have not had to deal with much before, it is called out here.
Protecting your IP is all about keeping someone who may have a legitimate license to have your application (either because they paid you for it or because they’ve downloaded a trial version) from reverse engineering it and reusing the code to produce their own application which may or may not compete directly with yours.
In the SharePoint world, we’re primarily talking here about Apps. Perhaps these are apps they’re purchased or acquired through the SharePoint store, perhaps ones they’ve purchased from your website and installed by themselves. In a smaller number of cases, these may be full-trust solutions they’ve purchased from you and installed on their on-premises farm, but mostly we’re talking about apps here.
As you should already know, there are primarily two types of apps – SharePoint-hosted and Cloud-hosted (the latter encompassing both provider- and auto-hosted).
It’s not much, but really, it’s about the best you can do if you’re stuck on the client because of your SharePoint-hosted app. If this is the case, you pretty much need to face the fact that protecting your IP is not going to happen if someone is determined to get it. The best you can hope for is to make things a little more difficult and that maybe, just maybe, the thief will decide it’s not worth it and move on to someone else’s app.
If the application you’re building is not an app, but instead either a full trust SharePoint solution or else a rich client application, then things look a little different. In each of these cases, the decision between REST and CSOM is pretty minor because most of your IP protection is going to come by investing in a good DLL obfuscator and running your assemblies through it before distributing them. A good .NET obfuscator will make it impossible to open your assembly in something like Reflector and get at the raw source code. The only thing left now is the traffic over the wire. As with the SharePoint-hosted apps this is a concern because the thief can see the endpoints and the data for the request and the response. The situation here is the same – CSOM is going to provide a little (and I do mean a very little) extra protection. It may not be enough to make someone move on to another app, but it’s about all we’ve got.
As I said in the SPC session, I intentionally ranked this one last. It’s not because this isn’t important; it’s not because developers aren’t important (I’m a developer, so this impacts me too). I ranked this last because, as much as we may all hate to admit it, that’s where it really needs to be. The arguments we developers put forward for why we can or can’t do one API or the other don’t, in my opinion, hold much water. They tend to boil down into one of these:
- Lack of documentation (REST)
- Learn new things (REST & CSOM)
- Lines of Code
- Professional Development Discipline
The first one is really the biggie. I hear this one pretty frequently – I don’t/can’t do REST because the documentation is bad. Unfortunately, it’s not true. As someone who has largely switched from CSOM to REST as my “default” API, I can tell you that between the official documentation and the hundreds of blog posts on the REST API in SharePoint or REST in general, there is plenty of information out there. Is 100% of the REST API documented? No, but neither is CSOM. There’s enough REST documented, however, for you to figure out how to do anything you need to do. The other thing about this one is that it is not a permanent problem. The documentation is getting better, between official sources on MSDN and unofficial blog posts where someone figured something out and wrote about it.
The second class of arguments is a little odd. If you don’t want to learn new things, you really ought to consider a new field. Technology changes too rapidly to be stuck in one place. I’m not saying that you need to jump at every new shiny object that comes along, but you ought not to shy away from something just because it is new. Interestingly, I hear this as an argument against both APIs.
Lines of code is an interesting one because it is so easy to prove or disprove, depending on your point of view. I’ve found, though, that even when presented with facts, people still don’t change their argument. Quite often I hear people complain that REST requires too many lines of code to accomplish any given task, and that they won’t use it for this reason. I wonder about how they are determining lines of code when I hear this. I wonder why the number of lines of code required for something matters. Now, certainly, if we’re talking really significant differences here then certainly that matters. But if one API takes 5 lines and the other even 15, 20, or 30, who really cares?
I do generally hear this as an argument not to do REST as it requires more code to accomplish any given task. However, in my testing, and I showed this in the session, the exact opposite is true. For the six common tasks I used for my performance testing (listed above) REST took less than half the number of lines as doing the same thing in CSOM, provided that you are smart about writing reusable libraries (for each API, to be fair). A well written utility library can encapsulate the common, repetitive tasks from each API and not require you to retype them each time. Once this code is in a utility library, it should no longer count in your “lines of code” tally. My argument for excluding it from your count boils down to – do you count jQuery lines when calculating lines of Code? You probably don’t. And that’s correct. You shouldn’t. Once code is a part of a well-crafted, reusable library, it is no longer directly maintained for the project at hand and so no longer counts.
Last but not least in this area is “professional development discipline”. I’m a huge fan of architecting and coding in a good, professional manner. I’ve been preaching that message for years. But neither REST nor CSOM leads to bad code or bad coding habits by their very nature. Can you write bad code in either one? Sure. Is it easier or more prevalent in one API over the other? No. This is another one that I chalk up to just us developers being cranky.
So before you think I’m a masochistic developer who likes to make things hard for myself, let me assure you that I am not. I don’t like to work any harder than I have to. I do, however, like to use the right tool for the job, even if it means abandoning something that has worked for me in the past and taking on something new if it is the right thing for the task at hand.
[OK, I’ll jump down off my high horse now. Sorry about that…]
You’ll see a fair amount of stuff on the web saying essentially that “REST is the future.” While I won’t argue this fact, I think that it is, it is also true that for SharePoint, at least, Microsoft is committed to CSOM. I mentioned this before but it is worth repeating – CSOM isn’t going away any time soon. If you’re doing CSOM, or thinking about doing CSOM, don’t worry. It’s not a dead end.
Just realize that it’s a SharePoint-only technology. If you need to reach outside of the SharePoint world, you’re almost certainly doing REST for that. This means that there’s a pretty good chance that REST will be part of your repertoire.
The Final Verdict
In the end, I took a cop-out in my session and declared both APIs the winner – it’s like Kindergarten, everybody wins. Some people, I think, were a little disappointed that I didn’t crown one API the winner and declare that you should use it for all remote SharePoint development from now on. I don’t think it’s that cut-and-dry, though. They’re both perfectly valid in certain situations and you need to make up your own mind. I’ve given you the things to think about here when you need to pick an API, now you need to make up your own mind.
One final point here: there is no reason you actually have to pick one API over the other! They can coexist very nicely. I know people who use REST for reads and CSOM for Writes, Updates or Deletes. I know other people who use REST for interacting with SharePoint data, but CSOM for interacting with SharePoint constructs – Lists/Libraries, security, Sites/Webs, users, etc.
|There is no reason you actually have to pick one API over the other! They can coexist very nicely
Based on everything presented here, and my experience with each API, I generally prefer REST over CSOM. My default position is to use REST unless I can’t. That doesn’t mean I don’t use CSOM. I do, but it’s maybe 10% of the code I’ll be writing moving forward. As I said, I used to do a lot of CSOM, but I’m transitioning. This should have absolutely no bearing on your decision. I hesitated before putting it in here or including it in my session, but in the end decided it was useful to include.
Typically, I use CSOM in these situations:
- Managed Metadata
- High volume/Low bandwidth situations (where every byte on the wire counts)
- When a client is already invested in CSOM and doesn’t want to introduce REST
That’s it. That is pretty much everything I covered in my SPC session (minus the juggling). I hope you found this valuable. If you want to see the session and watch the demos (but you still can’t see the juggling), the recording is available here: http://channel9.msdn.com/Events/SharePoint-Conference/2014/SPC423.
If you want the source code from the demos I did, it’s available here: http://1drv.ms/1Smdumi
What do you think? What factors do you consider when choosing an API? Sound off below, but keep it civil…