Partial Classes to the Rescue!

I’m working with a fairly large codebase – a few of my base classes have several thousand lines of code.  Not huge, but certainly unwieldy when you need to be hopping around the codebase.  There’s just no great way to organize everything.  While some folks eschew regions, I find them useful when used sparingly and what I’ll call “correctly”.  In other words, regions never go inside a method, that can quickly lead to a mess; but they’re useful to group a bunch of related members and be able to quickly hide them.  In some cases, those members can and should be moved to a separate class, and in those cases that’s what I do.  But in some cases, a separate class doesn’t make sense.  Regions do.

But even that isn’t enough sometimes.  In my current codebase, I have a bunch of interfaces which my various classes implement.  Each interface defines 5-10 members, so they’re pretty tiny, but even that can get unwieldy.  My core base class implements 5 of those interfaces, which means that it has something around 40 methods just to implement its interfaces – that’s not getting into some of the core logic for that abstract base class, just its interface implementations.  It also has a bunch of private methods and properties which it needs to do its job.  Wading through all of that code can be a headache sometimes.  Worse, I’m still a little old fashioned and sometimes I want to (gasp) *print* some of my code to be able to analyze it and draw circles and arrows while working on it.  Regions don’t help when printing – everything collapsed inside the region gets printed.

That’s where partial classes come in. 

I’ve started breaking my larger classes into partial classes – one file for each interface.  Now everything related to ISecurable or IStorable, for example, is on one place.  Each code file is now closer to 7-800 lines of code instead of 3-4000.  Much easier to work with. 

The best part is, neither Visual Studio nor the compiler care.  The compiler happily combines the separate files together into the exact same MSIL as if they had been all in one big file to begin with.  Visual Studio doesn’t miss a beat, either.  Not only does it have no trouble with the separate files, it still shows me proper Intellisense across files:

SNAGHTML441cfcd5

Furthermore, Visual Studio is smart enough to show me all of the members of my class, even those defined in another file from the one I’m currently working in, and it even shows them dimmed so I know they’re in another file:

image

Other Visual Studio functions, such as Peek Definition, Go to Definition, Find All References, etc., continue to work as well.

Typically partial classes are used to separate generated code from developer-written code.  Other folks use them to allow multiple developers to work on the same class at the same time.  This is just one more case where partial classes provide real value.  I know some folks will disagree with this, and that’s fine.  To each their own.  To my mind, partial classes, like regions, provide significant value when used appropriately and in moderation.

One final point, I organize my partial classes a little differently than I think other folks might, and I think this helps keep things straight.  Every partial class in my code base is in a folder named for the class.  Each file of the partial class has the same base part of its name – the name of the class, but then it also includes the interface implemented in that file.  Here’s a view of Solution Explorer showing one of my partial classes (RepositoryItem):

image

Each interface gets its own file, and then there’s RepositoryItem-Core, which contains everything else for the class.

New Role: Technology Advisor, Walnut Street Labs

I’ve been co-working at Walnut Street Labs since December of 2013 and recently stepped up my efforts to support WSL and the start-up community in the Philly ‘burbs.  I’m now a volunteer Technology Advisor for the Labs.  In this position, I’ll advise start-ups on their technology for free.

I’ve been “working with computers” (as my friends and family call it) for just shy of 20 years.  In that time, I’ve done everything from Novell and Windows NT networking to Lotus Notes development and administration, cc:Mail and Exchange and now SharePoint.  Recently, I’ve been involved with Windows Azure, Amazon Web Services (AWS), WordPress, PHP, ASP.NET MVC, Orchard CMS and mobile technologies – Apple, Android and Windows Phone.

If you’re a start-up in the southeastern, PA area and need some help with your technology, want to discuss and white-board some issues, or just want a second set of eyes to look things over, then please feel free to ping me via my scheduling link: http://ohours.org/DMann.  I’m available for four fifteen-minute sessions every Thursday morning at WSL:

23 N. Walnut Street

West Chester, PA 19380

During these sessions, we can chat and work through whatever technology topics you would like help with – guidance, planning, etc.  When you make your appointment, please describe what you’d like to chat about.  If it happens to be a technology with which I’m not familiar, that will allow me to track down someone at WSL that is comfortable with it and able to help. 

While these sessions aren’t intended to be debugging or direct coding help, we can delve into those areas if they’re a part of a larger technology or capabilities problem.  The sessions will be most valuable for start ups trying to figure out their technology direction, determine which technology will be most advantageous to their product, figure out how to support new technologies, design an architecture, etc.

These sessions are free.  While there are no legal NDAs involved, I can assure you that your information will not be shared.  I prefer to do business on a handshake and am generally too busy with my own start up ideas to “steal” someone else’s ideas.  You’ll find, once you’ve been doing this for a while, that the idea and the technology pieces tend to be the easy parts.  My list of product ideas is pushing 50 applications these days; I’m not looking to add any more products to my repertoire.  If your start up is too close to something I already have in the works or am planning to move forward with in the near future, I’ll disclose this potential conflict of interest to you up front and we can figure out how to move forward.

If you’d like to schedule a meeting, please visit my scheduling page: http://ohours.org/DMann

Presenting at Walnut Street Labs Startup Meetup

 

I’m thrilled to announce that I’ll be presenting at the Walnut Street Labs Startup Meet up next Tuesday morning at 9.  I’ll be presenting a two-part talk:

Part 1: Opportunities for Startups in the Microsoft World

For various reasons the startups getting the lion’s share of the attention are focused primarily on the non-Microsoft world of Apple, Google, Ruby, PHP, WordPress, etc.  Basically ABM – Anything But Microsoft.   In this discussion, we’ll look at reasons why the Microsoft startup ecosystem is a great place to be and one that is simmering beneath the headlines.  We’ll discuss everything from tools and platforms to market share and opportunity.  You might be surprised at how amenable the evil empire is to startups and realize that you ignore this potential market at your startup’s peril.

Part 2: Moving from Consulting to Product

Consulting is easy money, but it’s not sustainable.  I’ve been a consultant for 19 years, but I can’t do it for another 19.  Over the past year, I’ve been positioning my company to make the jump from the grind of consulting to the sustainable, recurring revenue model of products.  In this session I’ll talk about things that have worked for me and things that haven’t, lessons I’ve learned, and ideas I have for the future.  If you’re in a similar place, or want to be in a similar place, then maybe you can learn from my experience and avoid some of the mistakes I’ve made.

The presentation will be live-streamed so pay attention to Twitter for the URL.  If you’re in the West Chester, PA area feel free to stop by – 23 N. Walnut St, West Chester, PA next Tuesday at 9AM.  There will be lots of time for questions and discussion and I’d love to hear your experiences.

SPC423 Recap: Deep Dive – REST and CSOM Comparison

I presented my session at the 2014 SharePoint Conference today (I’m typing this while sitting on the plane on my way home) and as promised, here’s the write up. This will include all of the material I presented at SPC as well as some additional material which I did not have time to include in my session (sessions were 75 minutes long, and I watched the timer tick down the last few seconds as I was wrapping up my final slides. Talk about perfect timing!). In addition to this write up, I’ll also provide links to my source code at the end of this post. I’m not providing the slide deck here as it would simply be extraneous – all of the material is covered in this post. Sit back and get comfy, this is going to be a long one.

Here’s what the session was all about:

SharePoint 2013 is fully embracing the world of client-side technologies, both on-premises and in the cloud. Regardless of which programming language you use (C#, VB.NET or JavaScript) a fundamental decision you must make is which API you will use to reach back into SharePoint – REST or CSOM? This session will briefly introduce each API, and then quickly move into a demo-heavy comparison of both approaches, citing the pros and cons of each. By the end of the session, you will have a solid understanding of when and why you would use one API over the other (regardless of programming language) as well as plenty of examples of each in action.

The main focus was to help developers pick an API with which to access SharePoint from off-server. There were quite a number of announcements in the conference and developer keynotes related to new capabilities which, when combined with existing confusion, inaccurate information and FUD, often led developers to make bad choices with regard to their API. Actually, scratch that. As I tried to get across in the session, it’s usually not a bad choice, it just may not be the best choice.

There are very few bad choices between the APIs – one API may just be a better choice in a particular situation.

Before going any further, I should at least mention the other popular option for SharePoint development. If you have the choice to run code on-server, then you’d be using the SSOM – Server Side Object Model. This is the one Microsoft is shunting to the side with their focus on “THE CLOUD.” This is the one with which most of us SharePoint developers are most comfortable. I won’t say that the SSOM is never going away because that is obviously not my call to make. However, so long as there is an on-premises version of SharePoint, there will be an SSOM. Truth be told, the SSOM is currently the only one Microsoft actually writes and maintains – the other two are generated from it. So you could think of the SSOM as Amber, and CSOM and REST mere Shadows. (An obscure reference for the geeks reading this, but essentially what I’m saying is that the SSOM is the source of both CSOM and REST).

Anyway, that’s not what I came to talk about. I came to talk about the draft REST vs CSOM (anybody catch that reference?). REST and CSOM are each perfectly valid choices for interacting with SharePoint from a remote location – essentially anywhere except the SharePoint server itself. With two “perfectly valid choices” how is a developer supposed to know which is the right one to use? Is there a right one? Why are there two to begin with?

That’s why this session exists. Let’s dig in and get started answering those questions and more…

First of all, there are two APIs because they have different approaches, different capabilities and different situations in which they provide value, and that’s the reason this session was created – help devs to understand the differences and know when to pick one over the other.

People will often ask why I use “CSOM” when referring to a JavaScript example or situation. My answer is that the acronym CSOM encompasses all three of the client-side object models – .NET, Silverlight and JavaScript. So when I’m talking about something that applies to all three, I use CSOM, even if my example is in JavaScript. Technically, the acronym JSOM, which many people use to refer just to the JavaScript implementation, is inaccurate as there really is no such thing as a JavaScript Object Model; there is just the client-side object model implemented in JavaScript. That’s really splitting hairs and I do use JSOM when I’m talking only about the JavaScript implementation of CSOM, or about things which only apply to the JavaScript implementation. Along the same lines, I’ll use .NET CSOM if I’m talking exclusively about the .NET implementation.

So, to summarize:

  • CSOM = all three implementations, .NET, Silverlight and JavaScript.
  • JSOM = just the JavaScript implementation
  • .NET CSOM = just the .NET implementation

And no, there really isn’t a Silverlight-specific acronym as no one really talks about Silverlight anymore.

There are five basic elements that should be considered when you begin a project involving remotely accessing SharePoint with regard to selecting an API set. If I were to rank them in order of typical performance for the projects I work on, they would currently look like this:

  1. Technology Concerns
  2. Performance
  3. Testability
  4. Intellectual Property Protection
  5. Developer Concerns

For any given situation in which you are likely to find yourself, you may rank these differently, and that’s perfectly OK. I’d rank them differently in some situations, too, but most of the time I think they go in that order. We’ll touch upon each of these in the rest of this article. But first I want to deal with some that didn’t make the list:

· Security – someone shouted this out during the “game show” portion of the session (you had to be there) and honestly I was a bit flummoxed. My first reaction was “how did I miss something as important as security?” But when I had a moment to think about it, I realized that security didn’t make the list because there really is no difference between the API sets. SharePoint is always the final arbiter of security for content it holds and the API with which you access SharePoint doesn’t come into play here – each can work with SharePoint in a secure manner. A related concern can come into play but that’s a language choice, not an API choice. If you’re writing your REST or CSOM code in JavaScript, then your security options ARE limited. Essentially you can’t do impersonation in JavaScript as it always runs in the browser context. This, however, as I said, is a result of your language choice – JavaScript – not your API choice – REST or CSOM. In .NET, REST and CSOM each allows for impersonation.

· Browser Choice – this one didn’t come up in the session but I want to call it out here. Some people might think that the browser you choose is going to impact your CSOM or REST code and that is actually not true. For any browser which SharePoint supports (see here), both REST and CSOM will work perfectly well.

· State Management – I find this one interesting because in my experience few, if any, devs take advantage of this. Essentially, what it comes down to is that CSOM can maintain the state of your objects across calls back to the server. I cover this in my Pluralsight course and it is definitely a nice feature when you need it. As I said, however, most devs don’t know about it and it’s not really a very strong reason to pick one API over the other

OK, so let’s start looking at the element we do need to consider.

Technology Concerns

This concern breaks down into several sub-areas that are going to come into play in most situations. The one commonality across them is that they are typically not things which can be changed. They are environmental and typically much larger than simply accessing SharePoint for one project.

Low-Hanging Fruit

These are the easy ones because it’s completely cut and dry – things either work or they don’t. These fall completely on the pro-CSOM side as they are things that simply do not (yet) work here for REST. If you need to access either Managed Metadata or Workflow, you cannot do so with REST and so you must use CSOM.

Technology Stack

This element concerns your basic computer (server, workstation, tablet, etc) platform and which tools or libraries you may be using. For our purposes here, computer platform breaks down into:

1. Microsoft

2. Non-Microsoft

We can drill into this a little further to encompass more specific gradations such as iOS, Android, *nix, etc.

Tools and libraries generally break down to “those that were made for SharePoint” (specifically an implementation of the CSOM) and everything else. From the JavaScript side, this includes everything from jQuery to a utility library you write by yourself. For .NET, this ranges from something like Entity Framework to again, a simple library you write on your own.

While you can generally make either CSOM or REST work across any of these scenarios, in general, REST is going to work better for the non-Microsoft platforms and third party libraries. It is unlikely worth the effort to run Mono just to get access to the .NET CSOM and while downloading the JSOM wouldn’t be that difficult, it’s only going to work in browser applications. If you’re making significant use of any third party libraries, the effort required to make the JSOM work (things like collections and objects, etc.) is unlikely worth the benefit provided by the JSOM. Especially when you have a perfectly valid REST implementation available.

If you’re on a Microsoft platform and/or only using SharePoint-specific libraries, then either CSOM or REST are equally useful.

Role of SharePoint

This technology concern covers how you are using SharePoint:

· As simply another data-store – the vast majority of what you’re code is doing is simply reading and writing data that just happens to be stored in SharePoint.

· Application pieces of SharePoint – your code spends more of its time working with SharePoint-specific constructs – sites/webs, lists/libraries, security, users, etc.

This element is a little harder to call a winner for because both REST and CSOM can operate equally well here. However, if SharePoint is just another data-store for your application, it is likely to be a little easier to use REST as it works with more traditional data structures – XML and JSON. If you’re interacting more with the application elements then CSOM has a slight advantage due to its similarities with the SSOM; it will be marginally easier for developers to make the client-side transition from the SSOM since the objects are going to be so similar.

Deployment Mechanism

How and where you are going to deploy your code is another technology factor to consider and it’s another that’s pretty cut and dry:

· If you’re deploying full-trust solutions, then most of your code is probably SSOM, but for the client side elements, you could do CSOM or REST equally well

· If you’re deploying apps:

o SharePoint-hosted apps: these tend to be more data-centric (though certainly don’t have to be) so to a certain extent, we fall back on the “Role of SharePoint” element and so choose REST. CSOM would also be a perfectly valid choice, especially if your app interacts more with SharePoint as an application. You’re using the JavaScript implementation of one API or the other here.

o Cloud-hosted apps: This also falls back on the “role of SharePoint” but also gets impacted by the technology stack in use. If you’re on a Microsoft server, I think CSOM is a better choice. If you’re on a non-Microsoft stack, then REST in whatever language is supported by your server is probably best.

Technology Target

This last element here is pretty straightforward. There are four scenarios we’ll look at:

1. SharePoint, either on-premises or in the cloud

2. Non-SharePoint elements of Office 365 (Exchange, Lync), including the newly announced “Office 365 platform” API

3. Other Microsoft Cloud Applications – Dynamics, Project, etc

4. Non-Microsoft Cloud Applications – Salesforce, etc.

This one is pretty easy as CSOM supports only SharePoint, either on-premises or in the cloud. While the others may support some flavor of client side API, it is not technically the “CSOM” in the sense we’re talking about. This includes the “Office 365 platform” unveiled at SPC14 – that is available only via REST.

So – CSOM or REST are an option for SharePoint, for anything else, you’re using REST.

This is a good time to stress an important point. Microsoft is committed to CSOM for SharePoint. I confirmed this with a few folks at SPC. CSOM isn’t going anywhere anytime soon, so we’re safe using it now and for the foreseeable future.

Performance

Performance is important for any application and comes in second place here only because the elements of the first place Technology Concerns cannot often be changed. We must get the best performance we can within the confines of the technology concerns with which we are stuck.

For our purposes here, performance is also going to encompass user experience as it is nearly impossible to have a good user experience with poor performance.

Performance when you’re conducting round-trip calls from client to server and back again is mostly concerned with how often you hit the wire, how much data flows across the wire per call and the total time required to complete the calls. Those are the three factors we’re concerned about here. In one of them, CSOM wins hands down, in the other two REST wins.

CSOM wins its portion because it has to hit the wire far less often than REST. Compared to CSOM, REST is exceedingly chatty. In the tests I performed, REST made about 70x more calls back to the server than did CSOM. This is because currently, REST has to call back to the server for each and every operation whereas CSOM allows you to batch calls (within some limitations) and send a series of operations to the server in a single call. This makes CSOM much more efficient.

I tested a series of pretty common operations in both REST and CSOM:

  • Creating Lists (10)
  • Creating List Items (100 per List, for a total of 1000)
  • Reading List Items by ID
  • Querying for List Items (retrieving 50 per List)
  • Deleting List Items
  • Deleting Lists

To complete these operations, CSOM made 42 calls back to the server with a total of about 2.5MB traversing the wire and an average payload size for each call of a little over 40K. REST, on the other hand, required over 3000 calls with total network traffic of a little more than 5 MB, but had an average payload size of a little less than 2K. Overall, each REST call was far smaller than each CSOM call and about 85% of each REST call was comprised of the HTTP Headers. If we could batch our REST calls (which IS supported by the underlying architecture but not likely something we’ll see soon) then REST would be much more of a contender here. By batching our REST calls, we’d have less calls and therefore less header traffic.

The other interesting thing here is that despite requiring more calls and more traffic (again, mostly headers) REST outperformed CSOM from a total time perspective by about 30%.

In a nutshell, there are plusses and minuses for both REST and CSOM from a performance perspective. While in general, we would award the winner to whichever took the least time (REST), the sheer number of calls made by REST and the 2x amount of data flowing across the wire were too much to overcome and we have to award this round to CSOM. For your situation, the operations you perform may not require as many calls and so REST will be a more logical choice. Remember, on a per-call basis, REST outperformed CSOM for both time and payload size, so if you don’t require as many calls, REST can outperform CSOM.

Testability

Testability is important. As I had one my slides in the SPC talk, there’s a little developer haiku I wrote which reflects this:

Codebase fluctuates
Unit Tests protect and calm
This ship has sailed

Unfortunately, as SharePoint developers, we tend to be woefully behind the curve in unit testing our code. For that reason alone, this particular element is bumped up in importance to being a major factor to contend with when picking an API.

Historically, SSOM has been exceedingly difficult to unit test for many reasons (sealed classes, internal constructors, etc). That has led a lot (most?) SharePoint developers to not bother unit testing their SharePoint code.

With the move to the client side, we have the opportunity to remedy this. Hopefully, the SharePoint community will take this opportunity to start unit testing more of our code.

With that goal in mind, let’s examine the two APIs from a testability point of view. While I’ve tried to stick with facts as much as possible for this article so far, testability is an area that is really a matter of opinion. Largely this is so because either API is equally testable so it really comes down to which you find easier/better. At SPC, the afternoon before my session, Steve Walker (from the SharePoint product team) had a session in which he stated that unit testing CSOM code was easier than REST code. I find the exact opposite to be true. Steve didn’t go into details on his statement, but for me, I felt that I had less test code to write to work with REST and what I did have to write was easier and more “web development” standard than what I had to write for CSOM.

In either case, a good quality mocking product is required. In my session, I showed Telerik’s JustMock, my current favorite for mocking. In the past, I’ve used TypeMock’s Isolator product for SSOM work, and there are numerous other mocking products that ought to work well, too. The mocking tool is required because we need to be able to have our test code function without the dependency of a SharePoint server. We do this by creating fake objects (mocks) as stand-ins for the real objects. For these fake objects, we control the horizontal and the vertical; we control every aspect of their existence – from what their methods do to what their properties return.

For example, when mocking .NET CSOM, I had to set up fake objects for ClientContext, Web, ListCollection and List for my demo, like so:

var mockCtx = Mock.Create<ClientContext>(Constructor.Mocked);

var mockWeb = Mock.Create<Web>();

var mockLists = Mock.Create<ListCollection>();

var mockList = Mock.Create<List>();

Figure 1: Mocking .NET CSOM with Telerik’s JustMock

Then when instantiating or accessing things, I needed to make sure that my mock objects were returned instead of the real objects, or in some cases (the Load and ExecuteQuery calls) that the mocked objects did nothing:

Mock.Arrange( () => mockLists.GetEnumerator()).IgnoreInstance().Returns(getEnum());

Mock.Arrange( () => mockCtx.Web).IgnoreInstance().Returns(mockWeb);

Mock.Arrange( () => mockWeb.Lists).IgnoreInstance().Returns(mockLists);

Mock.Arrange( () => mockCtx.Load<ListCollection>(mockLists)).IgnoreInstance().DoNothing();

Mock.Arrange( () => mockCtx.ExecuteQuery()).IgnoreInstance().DoNothing();

Figure 2: Configuring operations and properties hen testing .NET CSOM with JustMock

With the help of one more utility function, my CSOM test was complete:

private IEnumerator<List> getEnum()

{

    for (int i = 0; i < 20; i++)

    {

        var mockList = Mock.Create<List>();

        Mock.Arrange( () => mockList.Title).Returns(expected.ElementAt(i));

        yield return mockList;

    }

}

Figure 3: One final important function to round out my .NET CSOM unit test

(Full source code for the tests, as well as all of my session demos, is available at the bottom of this article)

With all of this in place, I could test my .NET CSOM code as part of my Unit Tests without requiring that the SharePoint server be involved, or even running. In my SPC demo, I shut down the W3C service before running the tests to prove that the mocked objects were working properly.

Contrast the above code with what it required in REST (again using JustMock) and you’ll get an idea of why I feel testing REST to be a little easier:

var mockRequest = Mock.Create<HttpWebRequest>();

var mockResponse = Mock.Create<HttpWebResponse>();

Mock.Arrange(() => WebRequest.Create(Arg.AnyString)).Returns(mockRequest);

Mock.Arrange(() => mockRequest.GetResponse()).Returns(mockResponse);

Mock.Arrange(() => mockResponse.GetResponseStream()).Returns(GetFakeResponseStream());

Figure 4: Testing REST with Telerik’s JustMock

There is actually one supporting method for the REST test (GetFakeResponseStream) that is not shown here. It is pretty simple but I didn’t reproduce it here because it contains one very long string (the content of the response) that wouldn’t show very well here. You can see it in the full source code listing.

In general, I found testing REST to be easier because it required less lines of code and because I was really dealing with testing standard web objects that are going to be far more familiar to most devs. The SharePoint-specific objects in CSOM were just a little bit harder to work with. You may feel differently, and that’s OK.

The good news is that neither API is particularly difficult to test, so there is really no reason not to test your code. Please, please, please begin writing unit tests if you’re not already doing so, or write more if you’re already writing some. The overall quality of your code WILL go up, and that’s not a bad thing.

Intellectual Property Protection

If all you do is write code for use by your company, then this element is likely not very important for you. With the introduction of apps and the SharePoint app store, however, more and more developers are beginning to produce products for use in SharePoint by others outside their company. Suddenly protecting the fruits of your labors – your intellectual property – is now important to you. For this reason, combined with the fact that it is something many of us have not had to deal with much before, it is called out here.

Protecting your IP is all about keeping someone who may have a legitimate license to have your application (either because they paid you for it or because they’ve downloaded a trial version) from reverse engineering it and reusing the code to produce their own application which may or may not compete directly with yours.

In the SharePoint world, we’re primarily talking here about Apps. Perhaps these are apps they’re purchased or acquired through the SharePoint store, perhaps ones they’ve purchased from your website and installed by themselves. In a smaller number of cases, these may be full-trust solutions they’ve purchased from you and installed on their on-premises farm, but mostly we’re talking about apps here.

As you should already know, there are primarily two types of apps – SharePoint-hosted and Cloud-hosted (the latter encompassing both provider- and auto-hosted).

For SharePoint hosted apps, there is not any way to really protect your IP. Everything runs on the client (SharePoint-hosted apps cannot run server code) and so all of the HTML, CSS, and JavaScript is downloaded to the client. This makes it all available to someone to inspect and do with what they choose. True, there are ways to obfuscate some of this and make it more difficult, but for every obfuscator, there is a corresponding de-obfuscator. The best you can do here is make things a little more difficult and maybe not worth the effort for someone trying to reverse-engineer your application. If you’re lucky, maybe they’ll look at yours, decide it’s going to be too hard to steal, and just move on to another app.

In my session I talked about one somewhat interesting use of something like TypeScript would be to complicate your JavaScript just enough to make it too much work for someone trying to steal your IP. Typescript, if you’re not familiar with it, generates 100% valid JavaScript based on the more C#-like code typed in by the developer. The JavaScript generated is more complicated than what most JavaScript developers write and so may be a little harder for a thief to decipher after its been obfuscated and then de-obfuscated.

One last point to be made, and again, it’s not much, is that CSOM is going to be a smidge harder for a thief to reverse engineer than REST. This isn’t because of anything actually in the APIs themselves, but really just because of the nature of the traffic they put on the wire. REST is going to expose the explicit endpoints your code calls as well as the payload for the request and the response, typically in pretty easy to understand JavaScript Object Notation (JSON). CSOM on the other hand, is going to submit all calls to a ProcessQuery endpoint and the payload, while still either JSON or XML, is going to include additional information as well as internal object path references that can make things a little harder to decipher.

It’s not much, but really, it’s about the best you can do if you’re stuck on the client because of your SharePoint-hosted app. If this is the case, you pretty much need to face the fact that protecting your IP is not going to happen if someone is determined to get it. The best you can hope for is to make things a little more difficult and that maybe, just maybe, the thief will decide it’s not worth it and move on to someone else’s app.

Things are much better if you move to cloud-hosted apps because these give you the option of moving your important code to a server somewhere, which means that it is not accessible to a thief. In this case, either CSOM or REST is valid because the thief cannot access the code or the traffic over the wire so everything is pretty well protected. All they can get to is the end HTML/CSS/JavaScript which hopefully doesn’t have too much of your secret sauce baked into it.

If the application you’re building is not an app, but instead either a full trust SharePoint solution or else a rich client application, then things look a little different. In each of these cases, the decision between REST and CSOM is pretty minor because most of your IP protection is going to come by investing in a good DLL obfuscator and running your assemblies through it before distributing them. A good .NET obfuscator will make it impossible to open your assembly in something like Reflector and get at the raw source code. The only thing left now is the traffic over the wire. As with the SharePoint-hosted apps this is a concern because the thief can see the endpoints and the data for the request and the response. The situation here is the same – CSOM is going to provide a little (and I do mean a very little) extra protection. It may not be enough to make someone move on to another app, but it’s about all we’ve got.

Developer Concerns

As I said in the SPC session, I intentionally ranked this one last. It’s not because this isn’t important; it’s not because developers aren’t important (I’m a developer, so this impacts me too). I ranked this last because, as much as we may all hate to admit it, that’s where it really needs to be. The arguments we developers put forward for why we can or can’t do one API or the other don’t, in my opinion, hold much water. They tend to boil down into one of these:

  1. Lack of documentation (REST)
  2. Learn new things (REST & CSOM)
  3. JavaScript (in general)
  4. Lines of Code
  5. Professional Development Discipline

The first one is really the biggie. I hear this one pretty frequently – I don’t/can’t do REST because the documentation is bad. Unfortunately, it’s not true. As someone who has largely switched from CSOM to REST as my “default” API, I can tell you that between the official documentation and the hundreds of blog posts on the REST API in SharePoint or REST in general, there is plenty of information out there. Is 100% of the REST API documented? No, but neither is CSOM. There’s enough REST documented, however, for you to figure out how to do anything you need to do. The other thing about this one is that it is not a permanent problem. The documentation is getting better, between official sources on MSDN and unofficial blog posts where someone figured something out and wrote about it.

The second class of arguments is a little odd. If you don’t want to learn new things, you really ought to consider a new field. Technology changes too rapidly to be stuck in one place. I’m not saying that you need to jump at every new shiny object that comes along, but you ought not to shy away from something just because it is new. Interestingly, I hear this as an argument against both APIs.

People like to complain about JavaScript. While this isn’t strictly an argument for or against a particular API, it does get thrown out as a reason not to do client side development – JavaScript isn’t “real” code (whatever that means), JavaScript is too fragmented/loose/hard/insecure/whatever. Fine, I get it, JavaScript is new, JavaScript might be a totally different paradigm from what you’re used to. But that’s no reason not to learn it if that’s what you need to do. I railed against JavaScript for years a decade or more ago, but finally realized that this was a tide there was no way I was turning back and I had to either embrace it or pick a different career.

Lines of code is an interesting one because it is so easy to prove or disprove, depending on your point of view. I’ve found, though, that even when presented with facts, people still don’t change their argument. Quite often I hear people complain that REST requires too many lines of code to accomplish any given task, and that they won’t use it for this reason. I wonder about how they are determining lines of code when I hear this. I wonder why the number of lines of code required for something matters. Now, certainly, if we’re talking really significant differences here then certainly that matters. But if one API takes 5 lines and the other even 15, 20, or 30, who really cares?

I do generally hear this as an argument not to do REST as it requires more code to accomplish any given task. However, in my testing, and I showed this in the session, the exact opposite is true. For the six common tasks I used for my performance testing (listed above) REST took less than half the number of lines as doing the same thing in CSOM, provided that you are smart about writing reusable libraries (for each API, to be fair). A well written utility library can encapsulate the common, repetitive tasks from each API and not require you to retype them each time. Once this code is in a utility library, it should no longer count in your “lines of code” tally. My argument for excluding it from your count boils down to – do you count jQuery lines when calculating lines of Code? You probably don’t. And that’s correct. You shouldn’t. Once code is a part of a well-crafted, reusable library, it is no longer directly maintained for the project at hand and so no longer counts.

Last but not least in this area is “professional development discipline”. I’m a huge fan of architecting and coding in a good, professional manner. I’ve been preaching that message for years. But neither REST nor CSOM leads to bad code or bad coding habits by their very nature. Can you write bad code in either one? Sure. Is it easier or more prevalent in one API over the other? No. This is another one that I chalk up to just us developers being cranky.

So before you think I’m a masochistic developer who likes to make things hard for myself, let me assure you that I am not. I don’t like to work any harder than I have to. I do, however, like to use the right tool for the job, even if it means abandoning something that has worked for me in the past and taking on something new if it is the right thing for the task at hand.

[OK, I’ll jump down off my high horse now. Sorry about that…]

The Future

You’ll see a fair amount of stuff on the web saying essentially that “REST is the future.” While I won’t argue this fact, I think that it is, it is also true that for SharePoint, at least, Microsoft is committed to CSOM. I mentioned this before but it is worth repeating – CSOM isn’t going away any time soon. If you’re doing CSOM, or thinking about doing CSOM, don’t worry. It’s not a dead end.

Just realize that it’s a SharePoint-only technology. If you need to reach outside of the SharePoint world, you’re almost certainly doing REST for that. This means that there’s a pretty good chance that REST will be part of your repertoire.

The Final Verdict

In the end, I took a cop-out in my session and declared both APIs the winner – it’s like Kindergarten, everybody wins. Some people, I think, were a little disappointed that I didn’t crown one API the winner and declare that you should use it for all remote SharePoint development from now on. I don’t think it’s that cut-and-dry, though. They’re both perfectly valid in certain situations and you need to make up your own mind. I’ve given you the things to think about here when you need to pick an API, now you need to make up your own mind.

One final point here: there is no reason you actually have to pick one API over the other! They can coexist very nicely. I know people who use REST for reads and CSOM for Writes, Updates or Deletes. I know other people who use REST for interacting with SharePoint data, but CSOM for interacting with SharePoint constructs – Lists/Libraries, security, Sites/Webs, users, etc.

There is no reason you actually have to pick one API over the other! They can coexist very nicely

My Decision

Based on everything presented here, and my experience with each API, I generally prefer REST over CSOM. My default position is to use REST unless I can’t. That doesn’t mean I don’t use CSOM. I do, but it’s maybe 10% of the code I’ll be writing moving forward. As I said, I used to do a lot of CSOM, but I’m transitioning. This should have absolutely no bearing on your decision. I hesitated before putting it in here or including it in my session, but in the end decided it was useful to include.

Typically, I use CSOM in these situations:

  • Managed Metadata
  • Workflow
  • High volume/Low bandwidth situations (where every byte on the wire counts)
  • When a client is already invested in CSOM and doesn’t want to introduce REST

Closing

That’s it. That is pretty much everything I covered in my SPC session (minus the juggling). I hope you found this valuable. If you want to see the session and watch the demos (but you still can’t see the juggling), the recording is available here: http://channel9.msdn.com/Events/SharePoint-Conference/2014/SPC423.

If you want the source code from the demos I did, it’s available here: http://1drv.ms/1lC0Xvl 

What do you think? What factors do you consider when choosing an API? Sound off below, but keep it civil…

Speaking at SPC14

I’m happy to announce that I’ll be speaking at the SharePoint Conference in March. My session is Deep Dive: REST and CSOM Comparison, which fits in nicely with the Pluralsight course I made that just went live (and it’s successive parts coming in Q1).

Here’s the synopsis for my session: SharePoint 2013 is fully embracing the world of client-side technologies, both on-premises and in the cloud. This means JavaScript; but which API should you use – REST or CSOM? This session will briefly introduce each, and then quickly move into a demo-heavy comparison of both approaches, citing the pros and cons of each API. By the end of the session, you’ll have a solid understanding of when and why you would use one over the other (and some situations where you combine them into a hybrid approach) as well as plenty of examples of each in action.

 

Details available here: http://www.sharepointconference.com/content/sessions/SPC423

 

I’ll post more information here as the conference gets closer.  See you in Vegas!

My Pluralsight Course is Live!

It’s been a long time coming, but my first “real” course for Pluralsight is now live*. The course is “Developing SharePoint 2013 Solutions with JavaScript” and is available here: http://pluralsight.com/training/Courses/TableOfContents/developing-sharepoint-2013-javascript

This is actually the first of three parts to the course, with the next two parts coming in Q1 2014. Here’s the table of contents for part 1:

  1. JavaScript Fundamentals for SharePoint Developers
  2. SharePoint and JavaScript: Getting Started
  3. Core JavaScript Object Model Programming
  4. Reading/Writing SharePoint Data with the JavaScript Object Model
  5. Programming Security in SharePoint with the JavaScript Object Model
  6. Fiddler and an Introduction to REST
  7. Reading SharePoint Data with REST APIs
  8. Creating, Updating and Deleting SharePoint Data with REST APIs
  9. Deploying JavaScript Libraries
  10. Downloading JavaScript Libraries
  11. Common Third-Party JavaScript Libraries
  12. Final Thoughts for Part 1

Altogether, part 1 here is just over 6 hours long. Parts 2 and 3 add on an additional 9-10 hours for a grand total of over 15 hours of SharePoint JavaScript goodness by the time I’m done. I’ll post the outlines for the next 2 parts soon.

I’m obviously a bit biased, but I think there’s a LOT of good material in here – including some that I don’t think you can find anywhere else, or that other published content is just plain wrong or incomplete. It intentionally starts out very basic but then quickly moves into the deep end of the pool, covering both REST and JSOM. There are also modules on the best/proper ways to work with libraries (both deploying to SharePoint and downloading to the user’s browser) and finishes with a discussion on the REST vs. CSOM debate and which one you should use.

Stay tuned for more highlights on the content. If you don’t have a Pluralsight subscription, you can get one for $29/month, or wait for the companion e-Book (more on that later!)

 

*: I say “real” because there is a course I recorded for Critical Path Training originally that was converted to a PS course but I don’t really count that one

A New Project and New Tools

A few months ago my primary client asked me to take over a project that had been developed in-house. I’ve spent some time since then getting acclimated to the project, fixing some high-priority bugs and am now beginning to look at truly taking ownership of the codebase. As part of that I’m looking at trying out some new tools to help. Thanks to the folks at Telerik for giving me a copy of their tools (JustCode, JustTrace, JustDecompile, and JustMock) to kick the tires on. I’m also using Visual Studio 2013. To a certain extent, using a whole new toolset (I have previously used DevExpress’ CodeRush, which I still love but I wanted to try something new to see if there’s something better) is going to slow things down, but the client is OK with that and I’m looking forward to it. Things may get a little confusing as I try to figure out whether something new is VS2013 or Telerik, but I’ll do my best to keep them straight.

To make things even more interesting, the code is ALL VB.NET, which I haven’t coded in for…well…I’ve never coded VB.NET. I’ve done VB6 and its predecessors but never VB.NET. I think that’s the part that’s giving me more grief at the moment. Reading VB.NET is easy, but the muscle memory in my fingers is way too used to typing C#. Someone should just create a VS plugin that automatically deletes the semicolons from the end of your lines if you’re in VB.

One last point to make…this is also NOT a SharePoint project. It’s a straight .NET/SQL Server web application which is kinda cool and I’m looking forward to really flexing my (somewhat rusty) “regular” programming muscles a bit.

To give you an idea of what I’m up against, I snapped this little screenshot before I started:

Yikes! Nearly 1500 problems to take care of. Some of it is stupid like not following naming conventions, but some of it is serious:

  • No separation of layers – data, business and UI code all freely intermingled
  • No/minimal object use
  • No stored procs – SQL statements directly in the code
  • Hundreds (if not thousands) of lines of repeated code (copy/paste programming at its best)
  • No unit tests
  • Some of the code I can’t even run if I’m not on the client’s network to get access to resources
  • Easily 15% of the total lines of code is commented out old code that is no longer used

 

I’ll be posting articles over the next few months as I whip this codebase into shape. Some of it will be tool-related as I test out Telerik and VS2013, some of it will be rants and some of it will be about the code itself.

Security, Permissions, AllowEveryoneViewItems and Script Libraries

I came across something interesting today that I’m still mulling over; I haven’t gotten comfortable with this yet and so don’t really know how I feel about it and its implications. What I’m talking about is a property on SPList called AllowEveryoneViewItems. MSDN has the following to say about this property:

Gets or sets a Boolean value specifying whether everyone can view documents in the document library or attachments to items in the list. The AllowEveryoneViewItems property does not apply to all list items, but only to documents in document libraries or to attachments in list items. This property only works when users browse directly to a file through the browser, and it has no effect on the rest of the user interface or the object model.

The net effect of setting this property (either declaratively in your ListTemplate Elements file, via PowerShell or programmatically) is that you can make files in a document library or attachments on list items available (read-only) to all users, almost completely bypassing SharePoint security. I say almost because it does not, in my testing, enable anonymous access to the files. But if the users have any access to your site, they have access to all files in any list with this property set to true. If you turn on anonymous access for the site, then everyone has read access to these files

On the one hand, I see this as a huge boon for something like a script library – where we’re storing JavaScript files that should be available, read-only, to all users and we don’t want anyone to be able to turn that off or block access because then bad things will happen. On the other hand, I don’t like anything that bypasses SharePoint security. Security has always been a strong selling point for SharePoint in my mind so this concerns me a bit. At the very least, there should be some indication on the permissions page for the list or library (and the item if item-level permissions are in use) that this property is set to true and therefore everyone has read access.

What do you think? Is this good or bad?

Dave

YASSC – Yet Another SharePoint SKU Comparison

So, there’s already a couple of these on the web (see here, here, and here, for example). These are all great resources to browse, but it seems to me that I’m always trying to answer one question for clients as I review these resources – what do I lose by dropping down a license level? So, with that in mind, here’s my unofficial list – note that this is strictly for on-premises licensing. I’ll get around to an online version the next time I need to address that for a client. This is based on the “official” feature matrix available here: http://technet.microsoft.com/en-us/library/jj819267.aspx#bkmk_FeaturesOnPremise as of 9/20/2013. Click on any item to see more information about it.

These things are in Enterprise Editionthis is everything SharePoint on-premises does

 


Developer features

IT Professional features

Content features

Insights features

Search features

Sites features

Social features

Dropping from Enterprise Edition to Standard Edition, you lose:

 

Developer Features:

IT Pro Features:

Content Features:

Insights Features (Business Intelligence):

Search Features:

Sites Features

  • (None)

Social Feature

  • (None)

Dropping from Standard Edition to Foundation, you lose everything listed to the left, plus:

 

Developer Features

IT Pro Features

Content Features

Insights Features (Business Intelligence):

  • (None)

Search Features

Sites Features

Social Features