Category Archives: JavaScript

SPC423 Recap: Deep Dive – REST and CSOM Comparison

I presented my session at the 2014 SharePoint Conference today (I’m typing this while sitting on the plane on my way home) and as promised, here’s the write up. This will include all of the material I presented at SPC as well as some additional material which I did not have time to include in my session (sessions were 75 minutes long, and I watched the timer tick down the last few seconds as I was wrapping up my final slides. Talk about perfect timing!). In addition to this write up, I’ll also provide links to my source code at the end of this post. I’m not providing the slide deck here as it would simply be extraneous – all of the material is covered in this post. Sit back and get comfy, this is going to be a long one.

Here’s what the session was all about:

SharePoint 2013 is fully embracing the world of client-side technologies, both on-premises and in the cloud. Regardless of which programming language you use (C#, VB.NET or JavaScript) a fundamental decision you must make is which API you will use to reach back into SharePoint – REST or CSOM? This session will briefly introduce each API, and then quickly move into a demo-heavy comparison of both approaches, citing the pros and cons of each. By the end of the session, you will have a solid understanding of when and why you would use one API over the other (regardless of programming language) as well as plenty of examples of each in action.

The main focus was to help developers pick an API with which to access SharePoint from off-server. There were quite a number of announcements in the conference and developer keynotes related to new capabilities which, when combined with existing confusion, inaccurate information and FUD, often led developers to make bad choices with regard to their API. Actually, scratch that. As I tried to get across in the session, it’s usually not a bad choice, it just may not be the best choice.

There are very few bad choices between the APIs – one API may just be a better choice in a particular situation.

Before going any further, I should at least mention the other popular option for SharePoint development. If you have the choice to run code on-server, then you’d be using the SSOM – Server Side Object Model. This is the one Microsoft is shunting to the side with their focus on “THE CLOUD.” This is the one with which most of us SharePoint developers are most comfortable. I won’t say that the SSOM is never going away because that is obviously not my call to make. However, so long as there is an on-premises version of SharePoint, there will be an SSOM. Truth be told, the SSOM is currently the only one Microsoft actually writes and maintains – the other two are generated from it. So you could think of the SSOM as Amber, and CSOM and REST mere Shadows. (An obscure reference for the geeks reading this, but essentially what I’m saying is that the SSOM is the source of both CSOM and REST).

Anyway, that’s not what I came to talk about. I came to talk about the draft REST vs CSOM (anybody catch that reference?). REST and CSOM are each perfectly valid choices for interacting with SharePoint from a remote location – essentially anywhere except the SharePoint server itself. With two “perfectly valid choices” how is a developer supposed to know which is the right one to use? Is there a right one? Why are there two to begin with?

That’s why this session exists. Let’s dig in and get started answering those questions and more…

First of all, there are two APIs because they have different approaches, different capabilities and different situations in which they provide value, and that’s the reason this session was created – help devs to understand the differences and know when to pick one over the other.

People will often ask why I use “CSOM” when referring to a JavaScript example or situation. My answer is that the acronym CSOM encompasses all three of the client-side object models – .NET, Silverlight and JavaScript. So when I’m talking about something that applies to all three, I use CSOM, even if my example is in JavaScript. Technically, the acronym JSOM, which many people use to refer just to the JavaScript implementation, is inaccurate as there really is no such thing as a JavaScript Object Model; there is just the client-side object model implemented in JavaScript. That’s really splitting hairs and I do use JSOM when I’m talking only about the JavaScript implementation of CSOM, or about things which only apply to the JavaScript implementation. Along the same lines, I’ll use .NET CSOM if I’m talking exclusively about the .NET implementation.So, to summarize:

  • CSOM = all three implementations, .NET, Silverlight and JavaScript.
  • JSOM = just the JavaScript implementation
  • .NET CSOM = just the .NET implementation

And no, there really isn’t a Silverlight-specific acronym as no one really talks about Silverlight anymore.

There are five basic elements that should be considered when you begin a project involving remotely accessing SharePoint with regard to selecting an API set. If I were to rank them in order of typical performance for the projects I work on, they would currently look like this:

  1. Technology Concerns
  2. Performance
  3. Testability
  4. Intellectual Property Protection
  5. Developer Concerns

For any given situation in which you are likely to find yourself, you may rank these differently, and that’s perfectly OK. I’d rank them differently in some situations, too, but most of the time I think they go in that order. We’ll touch upon each of these in the rest of this article. But first I want to deal with some that didn’t make the list:

· Security – someone shouted this out during the “game show” portion of the session (you had to be there) and honestly I was a bit flummoxed. My first reaction was “how did I miss something as important as security?” But when I had a moment to think about it, I realized that security didn’t make the list because there really is no difference between the API sets. SharePoint is always the final arbiter of security for content it holds and the API with which you access SharePoint doesn’t come into play here – each can work with SharePoint in a secure manner. A related concern can come into play but that’s a language choice, not an API choice. If you’re writing your REST or CSOM code in JavaScript, then your security options ARE limited. Essentially you can’t do impersonation in JavaScript as it always runs in the browser context. This, however, as I said, is a result of your language choice – JavaScript – not your API choice – REST or CSOM. In .NET, REST and CSOM each allows for impersonation.

· Browser Choice – this one didn’t come up in the session but I want to call it out here. Some people might think that the browser you choose is going to impact your CSOM or REST code and that is actually not true. For any browser which SharePoint supports (see here), both REST and CSOM will work perfectly well.

· State Management – I find this one interesting because in my experience few, if any, devs take advantage of this. Essentially, what it comes down to is that CSOM can maintain the state of your objects across calls back to the server. I cover this in my Pluralsight course and it is definitely a nice feature when you need it. As I said, however, most devs don’t know about it and it’s not really a very strong reason to pick one API over the other

OK, so let’s start looking at the element we do need to consider.

Technology Concerns

This concern breaks down into several sub-areas that are going to come into play in most situations. The one commonality across them is that they are typically not things which can be changed. They are environmental and typically much larger than simply accessing SharePoint for one project.

Low-Hanging Fruit

These are the easy ones because it’s completely cut and dry – things either work or they don’t. These fall completely on the pro-CSOM side as they are things that simply do not (yet) work here for REST. If you need to access either Managed Metadata or Workflow, you cannot do so with REST and so you must use CSOM.

Technology Stack

This element concerns your basic computer (server, workstation, tablet, etc) platform and which tools or libraries you may be using. For our purposes here, computer platform breaks down into:

1. Microsoft

2. Non-Microsoft

We can drill into this a little further to encompass more specific gradations such as iOS, Android, *nix, etc.

Tools and libraries generally break down to “those that were made for SharePoint” (specifically an implementation of the CSOM) and everything else. From the JavaScript side, this includes everything from jQuery to a utility library you write by yourself. For .NET, this ranges from something like Entity Framework to again, a simple library you write on your own.

While you can generally make either CSOM or REST work across any of these scenarios, in general, REST is going to work better for the non-Microsoft platforms and third party libraries. It is unlikely worth the effort to run Mono just to get access to the .NET CSOM and while downloading the JSOM wouldn’t be that difficult, it’s only going to work in browser applications. If you’re making significant use of any third party libraries, the effort required to make the JSOM work (things like collections and objects, etc.) is unlikely worth the benefit provided by the JSOM. Especially when you have a perfectly valid REST implementation available.

If you’re on a Microsoft platform and/or only using SharePoint-specific libraries, then either CSOM or REST are equally useful.

Role of SharePoint

This technology concern covers how you are using SharePoint:

· As simply another data-store – the vast majority of what you’re code is doing is simply reading and writing data that just happens to be stored in SharePoint.

· Application pieces of SharePoint – your code spends more of its time working with SharePoint-specific constructs – sites/webs, lists/libraries, security, users, etc.

This element is a little harder to call a winner for because both REST and CSOM can operate equally well here. However, if SharePoint is just another data-store for your application, it is likely to be a little easier to use REST as it works with more traditional data structures – XML and JSON. If you’re interacting more with the application elements then CSOM has a slight advantage due to its similarities with the SSOM; it will be marginally easier for developers to make the client-side transition from the SSOM since the objects are going to be so similar.

Deployment Mechanism

How and where you are going to deploy your code is another technology factor to consider and it’s another that’s pretty cut and dry:

· If you’re deploying full-trust solutions, then most of your code is probably SSOM, but for the client side elements, you could do CSOM or REST equally well

· If you’re deploying apps:

o SharePoint-hosted apps: these tend to be more data-centric (though certainly don’t have to be) so to a certain extent, we fall back on the “Role of SharePoint” element and so choose REST. CSOM would also be a perfectly valid choice, especially if your app interacts more with SharePoint as an application. You’re using the JavaScript implementation of one API or the other here.

o Cloud-hosted apps: This also falls back on the “role of SharePoint” but also gets impacted by the technology stack in use. If you’re on a Microsoft server, I think CSOM is a better choice. If you’re on a non-Microsoft stack, then REST in whatever language is supported by your server is probably best.

Technology Target

This last element here is pretty straightforward. There are four scenarios we’ll look at:

1. SharePoint, either on-premises or in the cloud

2. Non-SharePoint elements of Office 365 (Exchange, Lync), including the newly announced “Office 365 platform” API

3. Other Microsoft Cloud Applications – Dynamics, Project, etc

4. Non-Microsoft Cloud Applications – Salesforce, etc.

This one is pretty easy as CSOM supports only SharePoint, either on-premises or in the cloud. While the others may support some flavor of client side API, it is not technically the “CSOM” in the sense we’re talking about. This includes the “Office 365 platform” unveiled at SPC14 – that is available only via REST.

So – CSOM or REST are an option for SharePoint, for anything else, you’re using REST.

This is a good time to stress an important point. Microsoft is committed to CSOM for SharePoint. I confirmed this with a few folks at SPC. CSOM isn’t going anywhere anytime soon, so we’re safe using it now and for the foreseeable future.

Performance

Performance is important for any application and comes in second place here only because the elements of the first place Technology Concerns cannot often be changed. We must get the best performance we can within the confines of the technology concerns with which we are stuck.

For our purposes here, performance is also going to encompass user experience as it is nearly impossible to have a good user experience with poor performance.

Performance when you’re conducting round-trip calls from client to server and back again is mostly concerned with how often you hit the wire, how much data flows across the wire per call and the total time required to complete the calls. Those are the three factors we’re concerned about here. In one of them, CSOM wins hands down, in the other two REST wins.

CSOM wins its portion because it has to hit the wire far less often than REST. Compared to CSOM, REST is exceedingly chatty. In the tests I performed, REST made about 70x more calls back to the server than did CSOM. This is because currently, REST has to call back to the server for each and every operation whereas CSOM allows you to batch calls (within some limitations) and send a series of operations to the server in a single call. This makes CSOM much more efficient.

I tested a series of pretty common operations in both REST and CSOM:

  • Creating Lists (10)
  • Creating List Items (100 per List, for a total of 1000)
  • Reading List Items by ID
  • Querying for List Items (retrieving 50 per List)
  • Deleting List Items
  • Deleting Lists

To complete these operations, CSOM made 42 calls back to the server with a total of about 2.5MB traversing the wire and an average payload size for each call of a little over 40K. REST, on the other hand, required over 3000 calls with total network traffic of a little more than 5 MB, but had an average payload size of a little less than 2K. Overall, each REST call was far smaller than each CSOM call and about 85% of each REST call was comprised of the HTTP Headers. If we could batch our REST calls (which IS supported by the underlying architecture but not likely something we’ll see soon) then REST would be much more of a contender here. By batching our REST calls, we’d have less calls and therefore less header traffic.

The other interesting thing here is that despite requiring more calls and more traffic (again, mostly headers) REST outperformed CSOM from a total time perspective by about 30%.

In a nutshell, there are plusses and minuses for both REST and CSOM from a performance perspective. While in general, we would award the winner to whichever took the least time (REST), the sheer number of calls made by REST and the 2x amount of data flowing across the wire were too much to overcome and we have to award this round to CSOM. For your situation, the operations you perform may not require as many calls and so REST will be a more logical choice. Remember, on a per-call basis, REST outperformed CSOM for both time and payload size, so if you don’t require as many calls, REST can outperform CSOM.

Testability

Testability is important. As I had one my slides in the SPC talk, there’s a little developer haiku I wrote which reflects this:

Codebase fluctuates
Unit Tests protect and calm
This ship has sailed

Unfortunately, as SharePoint developers, we tend to be woefully behind the curve in unit testing our code. For that reason alone, this particular element is bumped up in importance to being a major factor to contend with when picking an API.

Historically, SSOM has been exceedingly difficult to unit test for many reasons (sealed classes, internal constructors, etc). That has led a lot (most?) SharePoint developers to not bother unit testing their SharePoint code.

With the move to the client side, we have the opportunity to remedy this. Hopefully, the SharePoint community will take this opportunity to start unit testing more of our code.

With that goal in mind, let’s examine the two APIs from a testability point of view. While I’ve tried to stick with facts as much as possible for this article so far, testability is an area that is really a matter of opinion. Largely this is so because either API is equally testable so it really comes down to which you find easier/better. At SPC, the afternoon before my session, Steve Walker (from the SharePoint product team) had a session in which he stated that unit testing CSOM code was easier than REST code. I find the exact opposite to be true. Steve didn’t go into details on his statement, but for me, I felt that I had less test code to write to work with REST and what I did have to write was easier and more “web development” standard than what I had to write for CSOM.

In either case, a good quality mocking product is required. In my session, I showed Telerik’s JustMock, my current favorite for mocking. In the past, I’ve used TypeMock’s Isolator product for SSOM work, and there are numerous other mocking products that ought to work well, too. The mocking tool is required because we need to be able to have our test code function without the dependency of a SharePoint server. We do this by creating fake objects (mocks) as stand-ins for the real objects. For these fake objects, we control the horizontal and the vertical; we control every aspect of their existence – from what their methods do to what their properties return.

For example, when mocking .NET CSOM, I had to set up fake objects for ClientContext, Web, ListCollection and List for my demo, like so:

var mockCtx = Mock.Create<ClientContext>(Constructor.Mocked);

var mockWeb = Mock.Create<Web>();

var mockLists = Mock.Create<ListCollection>();

var mockList = Mock.Create<List>();

Figure 1: Mocking .NET CSOM with Telerik’s JustMock

Then when instantiating or accessing things, I needed to make sure that my mock objects were returned instead of the real objects, or in some cases (the Load and ExecuteQuery calls) that the mocked objects did nothing:

Mock.Arrange( () => mockLists.GetEnumerator()).IgnoreInstance().Returns(getEnum());

Mock.Arrange( () => mockCtx.Web).IgnoreInstance().Returns(mockWeb);

Mock.Arrange( () => mockWeb.Lists).IgnoreInstance().Returns(mockLists);

Mock.Arrange( () => mockCtx.Load<ListCollection>(mockLists)).IgnoreInstance().DoNothing();

Mock.Arrange( () => mockCtx.ExecuteQuery()).IgnoreInstance().DoNothing();

Figure 2: Configuring operations and properties hen testing .NET CSOM with JustMock

With the help of one more utility function, my CSOM test was complete:

private IEnumerator<List> getEnum()

{

    for (int i = 0; i < 20; i++)

    {

        var mockList = Mock.Create<List>();

        Mock.Arrange( () => mockList.Title).Returns(expected.ElementAt(i));

        yield return mockList;

    }

}

Figure 3: One final important function to round out my .NET CSOM unit test

(Full source code for the tests, as well as all of my session demos, is available at the bottom of this article)

With all of this in place, I could test my .NET CSOM code as part of my Unit Tests without requiring that the SharePoint server be involved, or even running. In my SPC demo, I shut down the W3C service before running the tests to prove that the mocked objects were working properly.

Contrast the above code with what it required in REST (again using JustMock) and you’ll get an idea of why I feel testing REST to be a little easier:

var mockRequest = Mock.Create<HttpWebRequest>();

var mockResponse = Mock.Create<HttpWebResponse>();

Mock.Arrange(() => WebRequest.Create(Arg.AnyString)).Returns(mockRequest);

Mock.Arrange(() => mockRequest.GetResponse()).Returns(mockResponse);

Mock.Arrange(() => mockResponse.GetResponseStream()).Returns(GetFakeResponseStream());

Figure 4: Testing REST with Telerik’s JustMock

There is actually one supporting method for the REST test (GetFakeResponseStream) that is not shown here. It is pretty simple but I didn’t reproduce it here because it contains one very long string (the content of the response) that wouldn’t show very well here. You can see it in the full source code listing.

In general, I found testing REST to be easier because it required less lines of code and because I was really dealing with testing standard web objects that are going to be far more familiar to most devs. The SharePoint-specific objects in CSOM were just a little bit harder to work with. You may feel differently, and that’s OK.

The good news is that neither API is particularly difficult to test, so there is really no reason not to test your code. Please, please, please begin writing unit tests if you’re not already doing so, or write more if you’re already writing some. The overall quality of your code WILL go up, and that’s not a bad thing.

Intellectual Property Protection

If all you do is write code for use by your company, then this element is likely not very important for you. With the introduction of apps and the SharePoint app store, however, more and more developers are beginning to produce products for use in SharePoint by others outside their company. Suddenly protecting the fruits of your labors – your intellectual property – is now important to you. For this reason, combined with the fact that it is something many of us have not had to deal with much before, it is called out here.

Protecting your IP is all about keeping someone who may have a legitimate license to have your application (either because they paid you for it or because they’ve downloaded a trial version) from reverse engineering it and reusing the code to produce their own application which may or may not compete directly with yours.

In the SharePoint world, we’re primarily talking here about Apps. Perhaps these are apps they’re purchased or acquired through the SharePoint store, perhaps ones they’ve purchased from your website and installed by themselves. In a smaller number of cases, these may be full-trust solutions they’ve purchased from you and installed on their on-premises farm, but mostly we’re talking about apps here.

As you should already know, there are primarily two types of apps – SharePoint-hosted and Cloud-hosted (the latter encompassing both provider- and auto-hosted).

For SharePoint hosted apps, there is not any way to really protect your IP. Everything runs on the client (SharePoint-hosted apps cannot run server code) and so all of the HTML, CSS, and JavaScript is downloaded to the client. This makes it all available to someone to inspect and do with what they choose. True, there are ways to obfuscate some of this and make it more difficult, but for every obfuscator, there is a corresponding de-obfuscator. The best you can do here is make things a little more difficult and maybe not worth the effort for someone trying to reverse-engineer your application. If you’re lucky, maybe they’ll look at yours, decide it’s going to be too hard to steal, and just move on to another app.

In my session I talked about one somewhat interesting use of something like TypeScript would be to complicate your JavaScript just enough to make it too much work for someone trying to steal your IP. Typescript, if you’re not familiar with it, generates 100% valid JavaScript based on the more C#-like code typed in by the developer. The JavaScript generated is more complicated than what most JavaScript developers write and so may be a little harder for a thief to decipher after its been obfuscated and then de-obfuscated.

One last point to be made, and again, it’s not much, is that CSOM is going to be a smidge harder for a thief to reverse engineer than REST. This isn’t because of anything actually in the APIs themselves, but really just because of the nature of the traffic they put on the wire. REST is going to expose the explicit endpoints your code calls as well as the payload for the request and the response, typically in pretty easy to understand JavaScript Object Notation (JSON). CSOM on the other hand, is going to submit all calls to a ProcessQuery endpoint and the payload, while still either JSON or XML, is going to include additional information as well as internal object path references that can make things a little harder to decipher.

It’s not much, but really, it’s about the best you can do if you’re stuck on the client because of your SharePoint-hosted app. If this is the case, you pretty much need to face the fact that protecting your IP is not going to happen if someone is determined to get it. The best you can hope for is to make things a little more difficult and that maybe, just maybe, the thief will decide it’s not worth it and move on to someone else’s app.

Things are much better if you move to cloud-hosted apps because these give you the option of moving your important code to a server somewhere, which means that it is not accessible to a thief. In this case, either CSOM or REST is valid because the thief cannot access the code or the traffic over the wire so everything is pretty well protected. All they can get to is the end HTML/CSS/JavaScript which hopefully doesn’t have too much of your secret sauce baked into it.

If the application you’re building is not an app, but instead either a full trust SharePoint solution or else a rich client application, then things look a little different. In each of these cases, the decision between REST and CSOM is pretty minor because most of your IP protection is going to come by investing in a good DLL obfuscator and running your assemblies through it before distributing them. A good .NET obfuscator will make it impossible to open your assembly in something like Reflector and get at the raw source code. The only thing left now is the traffic over the wire. As with the SharePoint-hosted apps this is a concern because the thief can see the endpoints and the data for the request and the response. The situation here is the same – CSOM is going to provide a little (and I do mean a very little) extra protection. It may not be enough to make someone move on to another app, but it’s about all we’ve got.

Developer Concerns

As I said in the SPC session, I intentionally ranked this one last. It’s not because this isn’t important; it’s not because developers aren’t important (I’m a developer, so this impacts me too). I ranked this last because, as much as we may all hate to admit it, that’s where it really needs to be. The arguments we developers put forward for why we can or can’t do one API or the other don’t, in my opinion, hold much water. They tend to boil down into one of these:

  1. Lack of documentation (REST)
  2. Learn new things (REST & CSOM)
  3. JavaScript (in general)
  4. Lines of Code
  5. Professional Development Discipline

The first one is really the biggie. I hear this one pretty frequently – I don’t/can’t do REST because the documentation is bad. Unfortunately, it’s not true. As someone who has largely switched from CSOM to REST as my “default” API, I can tell you that between the official documentation and the hundreds of blog posts on the REST API in SharePoint or REST in general, there is plenty of information out there. Is 100% of the REST API documented? No, but neither is CSOM. There’s enough REST documented, however, for you to figure out how to do anything you need to do. The other thing about this one is that it is not a permanent problem. The documentation is getting better, between official sources on MSDN and unofficial blog posts where someone figured something out and wrote about it.

The second class of arguments is a little odd. If you don’t want to learn new things, you really ought to consider a new field. Technology changes too rapidly to be stuck in one place. I’m not saying that you need to jump at every new shiny object that comes along, but you ought not to shy away from something just because it is new. Interestingly, I hear this as an argument against both APIs.

People like to complain about JavaScript. While this isn’t strictly an argument for or against a particular API, it does get thrown out as a reason not to do client side development – JavaScript isn’t “real” code (whatever that means), JavaScript is too fragmented/loose/hard/insecure/whatever. Fine, I get it, JavaScript is new, JavaScript might be a totally different paradigm from what you’re used to. But that’s no reason not to learn it if that’s what you need to do. I railed against JavaScript for years a decade or more ago, but finally realized that this was a tide there was no way I was turning back and I had to either embrace it or pick a different career.

Lines of code is an interesting one because it is so easy to prove or disprove, depending on your point of view. I’ve found, though, that even when presented with facts, people still don’t change their argument. Quite often I hear people complain that REST requires too many lines of code to accomplish any given task, and that they won’t use it for this reason. I wonder about how they are determining lines of code when I hear this. I wonder why the number of lines of code required for something matters. Now, certainly, if we’re talking really significant differences here then certainly that matters. But if one API takes 5 lines and the other even 15, 20, or 30, who really cares?

I do generally hear this as an argument not to do REST as it requires more code to accomplish any given task. However, in my testing, and I showed this in the session, the exact opposite is true. For the six common tasks I used for my performance testing (listed above) REST took less than half the number of lines as doing the same thing in CSOM, provided that you are smart about writing reusable libraries (for each API, to be fair). A well written utility library can encapsulate the common, repetitive tasks from each API and not require you to retype them each time. Once this code is in a utility library, it should no longer count in your “lines of code” tally. My argument for excluding it from your count boils down to – do you count jQuery lines when calculating lines of Code? You probably don’t. And that’s correct. You shouldn’t. Once code is a part of a well-crafted, reusable library, it is no longer directly maintained for the project at hand and so no longer counts.

Last but not least in this area is “professional development discipline”. I’m a huge fan of architecting and coding in a good, professional manner. I’ve been preaching that message for years. But neither REST nor CSOM leads to bad code or bad coding habits by their very nature. Can you write bad code in either one? Sure. Is it easier or more prevalent in one API over the other? No. This is another one that I chalk up to just us developers being cranky.

So before you think I’m a masochistic developer who likes to make things hard for myself, let me assure you that I am not. I don’t like to work any harder than I have to. I do, however, like to use the right tool for the job, even if it means abandoning something that has worked for me in the past and taking on something new if it is the right thing for the task at hand.

[OK, I’ll jump down off my high horse now. Sorry about that…]

The Future

You’ll see a fair amount of stuff on the web saying essentially that “REST is the future.” While I won’t argue this fact, I think that it is, it is also true that for SharePoint, at least, Microsoft is committed to CSOM. I mentioned this before but it is worth repeating – CSOM isn’t going away any time soon. If you’re doing CSOM, or thinking about doing CSOM, don’t worry. It’s not a dead end.

Just realize that it’s a SharePoint-only technology. If you need to reach outside of the SharePoint world, you’re almost certainly doing REST for that. This means that there’s a pretty good chance that REST will be part of your repertoire.

The Final Verdict

In the end, I took a cop-out in my session and declared both APIs the winner – it’s like Kindergarten, everybody wins. Some people, I think, were a little disappointed that I didn’t crown one API the winner and declare that you should use it for all remote SharePoint development from now on. I don’t think it’s that cut-and-dry, though. They’re both perfectly valid in certain situations and you need to make up your own mind. I’ve given you the things to think about here when you need to pick an API, now you need to make up your own mind.

One final point here: there is no reason you actually have to pick one API over the other! They can coexist very nicely. I know people who use REST for reads and CSOM for Writes, Updates or Deletes. I know other people who use REST for interacting with SharePoint data, but CSOM for interacting with SharePoint constructs – Lists/Libraries, security, Sites/Webs, users, etc.

There is no reason you actually have to pick one API over the other! They can coexist very nicely

My Decision

Based on everything presented here, and my experience with each API, I generally prefer REST over CSOM. My default position is to use REST unless I can’t. That doesn’t mean I don’t use CSOM. I do, but it’s maybe 10% of the code I’ll be writing moving forward. As I said, I used to do a lot of CSOM, but I’m transitioning. This should have absolutely no bearing on your decision. I hesitated before putting it in here or including it in my session, but in the end decided it was useful to include.

Typically, I use CSOM in these situations:

  • Managed Metadata
  • Workflow
  • High volume/Low bandwidth situations (where every byte on the wire counts)
  • When a client is already invested in CSOM and doesn’t want to introduce REST

Closing

That’s it. That is pretty much everything I covered in my SPC session (minus the juggling). I hope you found this valuable. If you want to see the session and watch the demos (but you still can’t see the juggling), the recording is available here: http://channel9.msdn.com/Events/SharePoint-Conference/2014/SPC423.

If you want the source code from the demos I did, it’s available here: http://1drv.ms/1Smdumi 

What do you think? What factors do you consider when choosing an API? Sound off below, but keep it civil…

Speaking at SPC14

I’m happy to announce that I’ll be speaking at the SharePoint Conference in March. My session is Deep Dive: REST and CSOM Comparison, which fits in nicely with the Pluralsight course I made that just went live (and it’s successive parts coming in Q1).

Here’s the synopsis for my session: SharePoint 2013 is fully embracing the world of client-side technologies, both on-premises and in the cloud. This means JavaScript; but which API should you use – REST or CSOM? This session will briefly introduce each, and then quickly move into a demo-heavy comparison of both approaches, citing the pros and cons of each API. By the end of the session, you’ll have a solid understanding of when and why you would use one over the other (and some situations where you combine them into a hybrid approach) as well as plenty of examples of each in action.

 

Details available here: http://www.sharepointconference.com/content/sessions/SPC423

 

I’ll post more information here as the conference gets closer.  See you in Vegas!

My Pluralsight Course is Live!

It’s been a long time coming, but my first “real” course for Pluralsight is now live*. The course is “Developing SharePoint 2013 Solutions with JavaScript” and is available here: http://pluralsight.com/training/Courses/TableOfContents/developing-sharepoint-2013-javascript

This is actually the first of three parts to the course, with the next two parts coming in Q1 2014. Here’s the table of contents for part 1:

  1. JavaScript Fundamentals for SharePoint Developers
  2. SharePoint and JavaScript: Getting Started
  3. Core JavaScript Object Model Programming
  4. Reading/Writing SharePoint Data with the JavaScript Object Model
  5. Programming Security in SharePoint with the JavaScript Object Model
  6. Fiddler and an Introduction to REST
  7. Reading SharePoint Data with REST APIs
  8. Creating, Updating and Deleting SharePoint Data with REST APIs
  9. Deploying JavaScript Libraries
  10. Downloading JavaScript Libraries
  11. Common Third-Party JavaScript Libraries
  12. Final Thoughts for Part 1

Altogether, part 1 here is just over 6 hours long. Parts 2 and 3 add on an additional 9-10 hours for a grand total of over 15 hours of SharePoint JavaScript goodness by the time I’m done. I’ll post the outlines for the next 2 parts soon.

I’m obviously a bit biased, but I think there’s a LOT of good material in here – including some that I don’t think you can find anywhere else, or that other published content is just plain wrong or incomplete. It intentionally starts out very basic but then quickly moves into the deep end of the pool, covering both REST and JSOM. There are also modules on the best/proper ways to work with libraries (both deploying to SharePoint and downloading to the user’s browser) and finishes with a discussion on the REST vs. CSOM debate and which one you should use.

Stay tuned for more highlights on the content. If you don’t have a Pluralsight subscription, you can get one for $29/month, or wait for the companion e-Book (more on that later!)

 

*: I say “real” because there is a course I recorded for Critical Path Training originally that was converted to a PS course but I don’t really count that one

SharePoint JavaScript: removeByLoginName

The SharePoint 2013 JavaScript object model includes a few new methods on the UserCollection object that on the surface seems like nice shortcuts, but I’m not convinced of their usefulness (yet – I could change my mind). These methods are:

  • getByLoginName
  • removeByLoginName

I’ve only tested the remove operation, but I’m assuming the get operation operates the same way.

In my testing, the loginName parameter must be the full claim name, for example (i:0#.w|wingtipmdebakey) instead of the friendlier WingtipMDebakey. This severely limits the usefulness of these functions because, unless you are going to hard code the claim token and provider information (the i:0#.w| part) and then tack the user-friendly login name on the end, which is probably a bad idea as it restricts your code from working across different claim types or providers, you’re going to have to get the user object first and get the login name from there (via user.get_loginName), at which point, you might as well just use the remove method on the UserCollection object.

In other words, instead of this:

var user = web.ensureUser(“Wingtip\MDebakey”);

userCollection.removeByLoginName(user.get_loginName());

 

Just do this:

var user = web.ensureUser("Wingtip\MDebakey");

userCollection.remove(user);

SharePoint JavaScript: Use the right reference material!

This is quite obvious when you think about it, but it’s easy to miss nonetheless when you’re searching for something in a hurry and could lead to some problems:

When looking up a method or property for the SharePoint 2013 JavaScript Object Model, make sure you’re looking at the correct version on MSDN!

It’s easy to tell which version you’re in, it’s right at the top:

 

With the server-side OM, a lot of times it didn’t matter which version you referenced because, unless you were looking for something about a new capability, the server-side OM was pretty well established and the core bits didn’t change very much.

This is not so with the JS OM. It is very much still evolving and even core bits are seeing massive changes. One case in point, the Web class had 14 methods in SharePoint 2010 but that count more than doubles in 2013 to 30 methods.

If you’re looking at the wrong page in MSDN, you’re going to miss a lot of interesting capabilities.

For reference: here’s the starting point of the 2013 JavaScript documentation: http://go.dmann.me/SP2013JS

“this” and Function.createDelegate in JSOM

There are two prevailing methods of registering a callback for the executeQueryAsync function:

context.executeQueryAsync(

    Function.createDelegate(this, _onSucceed), 

    Function.createDelegate(this, _onFail)

);

and:

context.executeQueryAsync(_onSucceed, _onFail);

As you can see, the difference is simply the inclusion or exclusion of the Function.createDelegate call.  If you’ve always wondered what the difference between the two was, as I have, the answer is pretty simple: the former allows you to override what the value of the this variable will be inside the function.

In the latter case, not using Function.createDelegate, when you are inside either the _onSucceed or _onFail  callbacks, this will reference the callback function itself – onSucceed or onFail  respectively.

In the former case, using Function.createDelegate, when you are inside either the _onSucceed or _onFail callbacks, this will reference whatever you pass in as the first parameter to the createDelegate function.  In this example, we’re passing in the current value of this, so inside the callback, this will be the same as in the function that made the call to createDelegate.  It’s important to note that you can pass any object in as that first parameter:

Function.createDelegate(window, _onSucceed) 

Function.createDelegate(new myWidget(), _onSucceed)

Function.createDelegate({}, _onSucceed)

Function.createDelegate(someOtherObject, _onSucceed)

 

will all work fine, making this equal to window, a new instance of myWidget, an empty object, and whatever object is represented by someOtherObject, respectively.

Typescript Webinar–Part 1

I just wrapped up my latest webinar on the Critical Path Training free webinar series – this time on Typescript.   It was pretty well attended – about 60 people if I saw the attendee counter properly.  Smile  I think the session went OK – it started out a little slow but then picked up and as usual, I had way too much content to fit into a 45 minute session (allowing time for Q&A).

If you missed the live session, the recording is available here: http://t.co/bSZvjTYE.

Anyway, as promised, this post is a summary of the material I covered and the first of what I hope to be several posts on the subject of Typescript and SharePoint.  I do need to revisit my JavaScript series, which has been sorely neglected over the past few months, and see how it needs to be refocused to fit into the brave new world of Typescript.

What Is Typescript?

So, first of all…what is Typescript?  From the www.typescriptlang.org website, we learn that:

Typescript is a language for application-scale JavaScript development.
Typescript is a typed superset of JavaScript that compiles to plain JavaScript.
Any browser. Any host. Any OS. Open Source.

Great.  That and $5 will buy me a cup of coffee.  Honestly, that description barely scratches the surface of Typescript.  Personally, I prefer this as a definition:

Typescript is a language and accompanying tools that support robust, professional development using JavaScript for cross-browser, cross-host applications of any size.  It facilitates the creation of standards-compliant JavaScript (ECMAScript) in a type-safe, easy-to-use manner which can be quickly grokked by most managed code or other professional programmers.

I think that is a far more encompassing definition. 

Now on to the good stuff.

Getting Started with Typescript

To get started with Typescript, you first need to install a couple of tools.  First, you’ll need the Visual Studio plug-in and I highly recommend Web Essentials (for more than just Typescript):

Web Essentials: http://visualstudiogallery.msdn.microsoft.com/07d54d12-7133-4e15-becb-6f451ea3bea6

VS Plugin: http://go.microsoft.com/fwlink/?LinkID=266563 (If you don’t use Visual Studio, Typescript is also available for Sublime Text, Emacs and Vim.  My guess is that if you’re doing SharePoint development you use VS…)

With those installed, you’re ready to go.  You can now create a SharePoint project (Farm, Sandbox or App – doesn’t matter) and add a Typescript file to it:

image

 

After you first add the Typescript file to your project, you’ll see it in Solution Explorer:

image

(Notice that there is no accompanying .js file yet)

Once you compile (which could be as simple as just saving the .ts file, depending on how you’re environment is configured), the appropriate .js files are created and nested under the .ts file in Solution Explorer:

image

The configuration settings to pay attention to are in Tools|Options|Web Essentials|Typescript:

image

Most are pretty straightforward and can be left at their defaults for now.  I’ll explore these options in more detail in a later article.  The only ones I like to make sure are set a certain way are “Keep Comments,” “Compile on Save”,” and “Minify generated JavaScript” – I make sure they’re all set to TRUE.  Believe it or not, we want to leave “Compile all Typescript files on build” set to FALSE.  We’ll address that next.

To make sure that our Typescript code is compiled into JavaScript at the proper time, we need to go through a little hack for any SharePoint projects.  Rob Bogue spells it out here: http://www.thorprojects.com/blog/archive/2012/10/13/including-typescript-in-a-sharepoint-project.aspx and the end result looks like this in your .csproj file:

image

The highlighted part is what we added.  Note that you’ll need to perform that little hack for every project in your SharePoint VS Solution that needs to compile Typescript.

The last thing you’re going to want to do is remove the .ts file from your deployment package – there’s no sense including it in the package – neither SharePoint nor the web server would have any idea what to do with it.  Doing this is simply a matter of deleting it from, or commenting it out of, the Elements.xml file:

image

Now we’re finally ready to begin… 

The Typescript Editor

The Typescript editor is a two-paned affair.  You type your Typescript on the left, the right is a preview of your generated JavaScript:

image

Each and every Typescript file you add will have some sample code included with it.  This is great while you’re learning but just an annoyance when you’re actually trying to be productive.  I hope this changes in the RTM of the tooling.  If, for some reason the tooling doesn’t change at RTM, or if deleting the sample code every time gets on your nerves, you could just create a Text file and change the extension to .ts.

After pondering over the sample code for a few minutes, go ahead and delete everything in the left hand pane – Ctrl-A | Delete will do the trick.  Now save your .ts file and, assuming you set your options as specified above, the JavaScript code in the preview on the right hand side will disappear as well.

This is probably a good place to end part 1 of this article.  We’ll pick up with Part 2 shortly covering our first baby-steps into writing Typescript code.  I’ll leave you with some resources to help you learn basic Typescript.  Part 2 will review some of that quickly and then move on to SharePoint-specific Typescript code.

Resources

Typescript Language Specification: http://go.microsoft.com/fwlink/?LinkId=267238

Pluralsight Typescript course: http://pluralsight.com/training/courses/TableOfContents?courseName=typescript

My webinar slides: blog.mannsoftware.com/wp-content/uploads/2013/02/typescriptandsharepoint.pdf

 

That’s it for part 1. 

 

Dave

Custom Field Types in SharePoint 2013 Apps

Synopsis and Key Take-Aways

Custom field types in SharePoint provide the ability to customize all aspects of SharePoint form fields– from how they are displayed, to how values are validated to how values are stored within SharePoint and a couple of other things along the way. They provide significant capability for enforcing business rules and providing a much more user friendly experience than what is available via the out-of-the-box field types.  Unfortunately, this power has never been available in a cloud-hosting scenario such as Office 365.  Things are changing for the better, however, with new capabilities available in SharePoint 2013 Preview.  We now have the ability to provide the same functionality to our end users without running custom code on the server.  With SharePoint 2013 Preview, we can modify the presentation and validation of a custom field (or technically any field) on any form in SharePoint as well as in Views simply via JavaScript.  This is an incredibly powerful capability.  It provides a nice, easy, standard (and supported) way of customizing the end user experience to be more efficient and friendly.

This article introduces the scenario, walks through the end user experience using a sample custom field and then dives into a code-level review of how to implement this new functionality.  The commented source code listing for the required JavaScript is available for download.

Note that as of the initial publication of this article (October 2012), this is based on the SharePoint 2013 Public Preview.  Once I can download RTM I’ll look to revisit this and update the article as necessary.

Introduction

Custom field types in SharePoint provide the ability to customize all aspects of SharePoint form fields– from how they are displayed, to how values are validated to how values are stored within SharePoint and a couple of other things along the way.  They provide significant capability for enforcing business rules and providing a much more user friendly experience than what is available via the out-of-the-box field types (single line of text, yes/no, date/time, user, url, etc.)

Unfortunately, this power and flexibility is simply not available in a cloud scenario such as Office 365 because they require that files be deployed to the server file system which you can’t do in a cloud solution.  For the 2010 release of SharePoint, there have been various attempts at simulating custom field types in a cloud scenario, but all have suffered from varying degrees of fragility or other problems.

Fast-forward to SharePoint 2013 and Microsoft has made things much easier for developers and much less fragile all around with regard to custom field types.  For the first time, it is possible to truly get most of the value of custom field types in a cloud scenario such as Office 365 or an on-premises scenario (without all of the work of doing it the old way).  Notice that I said most of the value.  There are some things that are still not possible and two major caveats of which you must be aware.  First the caveats:

  1. Even though we are building things which look and act like custom field types, we are not technically building custom field types.  We are technically just customizing the rendering and validation of the out of the box field types.  More on this later; just remember it for now
  2. All of our customizations are happening on the client side via JavaScript.  There are implications to this for your data and business rules.  What happens if a user has JavaScript turned off and therefore your customizations do not run?  What are the implications to the security and validity of your data?  We will touch briefly on custom validation later, but just keep it in mind – it’s not guaranteed to run)

Acknowledgements

Before going any further, I need to give credit where credit is due.  Andrew Connell and I hacked up the first portion of this together.  I took the rough POC AC and I had pulled together that almost did everything we wanted, finished it off and cleaned it up to produce the samples in this article.  Keep an eye on AC’s blog as he’ll be posting a follow-up article taking some of this in new directions.

Overview of the Solution

With all of that said, let’s see what it is that we’re going to get at the end of this.  Here are some screenshots and descriptions of what we’re going to build out in the rest of this article:

The New Item Form

In this example, we’ve customized the rendering of our custom field:

image_thumb40


Site Column 1 is our custom field.  Technically, it is just a simple text field.  Without our customizations, it would render as a textbox.  Our customizations change it’s rendering to a dropdown list.

The choices available in the dropdown (One through Five) are added dynamically.  In a real world scenario, these could come from a web service call.  To keep things simple and focused here, I’m simply hardcoding an array.  You’ll see when we get to the code how simple it would be to make this truly dynamic.

We’ve also added some rudimentary validation.  For the sake of this demo, FOUR is not a valid choice:

image_thumb20


(Michael Palin would be so proud)

Notice, too, that our field is marked as required in SharePoint and regular field validation continues to function:

image_thumb38


One of the things I’d like to play with in a later version of this is the possibilities for additional validation via CSOM and perhaps even some event receiver work to provide additional server-side validation.

Display in Views

Once we’ve entered a legal value, our item is shown in a SharePoint View.  We also have the opportunity to customize the rendering here as well:

image_thumb36


For the sake of this demo, I’m not actually showing the field value in the View. Instead, I show a JavaScript link that pops up a simple alert when clicked to show the field value:

image_thumb28

The Display Form

Again, for the sake of this demo, I’m customizing the presentation of our field on the Display form as well.  Instead of showing the actual value, I show it’s numeric equivalent:

image_thumb32

The Edit Form

The Edit form looks largely identical to the New form (internally, it’s actually rendered via the same code).  The only difference is that we have to make sure that the current value of the field shows as selected in the dropdown:

image_thumb34

Technical Walkthrough

Now that we’ve seen what we’re building, let’s dive into how to make it all work.  Once you’ve figured it all out, it’s actually pretty easy.  (Figuring it all out, though, that was a royal PITA, especially during a beta cycle).

The heart of everything we’re doing here is a new property on the out-of-the-box  SPField class (and all of it’s children) within the SharePoint server object model: JSLink (don’t  bother clicking on the link – there is NO documentation worth anything there right now – just a placeholder).  Similar to the JSLink property on SPView, SPField.JSLink gives us the opportunity to specify a custom JavaScript file that will be used to render our field.  Cool.

From our SharePoint 2013 App (or, really, any SharePoint 2013 solution), we add an attribute to the Elements file that defines our custom Field to specify the JSLink value:

<Elements xmlns="http://schemas.microsoft.com/sharepoint/">

  <Field

       ID="{23d7e8ae-29c3-4f05-ac4a-08bbf2bc2b1d}"

       Name="SiteColumn1"

       DisplayName="Site Column 1"

       Type="Text"

       Required="TRUE"

       JSLink="~site/Scripts/SampleJSField.js"

       Group="Custom Site Columns">

  </Field>

</Elements>

 

The SampleJSField.js file will now be used to render our field wherever it is used.  The full SampleField.js file is available at the end of this article for your reading pleasure..I’ve added some comments to explain what is going on, but the following sections spell out the highlights

IIFE

Because we are only specifying a JavaScript filename in our JSLink attribute, and not a particular method to be called, we need some way to actually execute our code.  We do this via an IIFE (Immediately Invoked Function Expression) declared right at the top of the file:

(function () {

    initFieldTypes();

    var r = new JSFieldText("SiteColumn1", viewFunc, editFunc, editFunc, displayFunc);

    r.initField()

})();

 

You could do this different ways, but this is the most concise and easiest.  Basically, in one shot, I’m declaring an anonymous JavaScript function and invoking it.  No magic inside the IIFE itself, it simply calls a function to set up our custom field objects (initFieldTypes), creates an instance of my custom JSFieldText object (passing in the field name and a couple of function objects – more on these later) and then initializes the field.

I’m not going to go into details on the whole initFieldTypes function here.  It’s in the full source listing and it’s a bunch of pretty funky JavaScript at this point.  I plan on cleaning this all up but there’s no sense in doing that until I’m working with RTM bits.  Eventually, I’d like to centralize all of this in a common JavaScript library that can be used by multiple fields, but that will come later.  For now, it was enough just to build out a POC to show that this all works.  A good chunk of the  code in the “Global Code” section of the full source listing is based on some code from http://www.ruzee.com/blog/2008/12/javascript-inheritance-via-prototypes-and-closures but I’m not sure whether I’ll stick with that.  It introduces some complexity and I’m not sure it gets me a lot.  In the short term, however, it filled a void and let me focus on more pressing things.

Registering Our Custom Rendering

I will, however, focus on a few key areas within the initFieldTypes function.  The first is where we actually override the default rendering and tell SharePoint to use our custom rendering.  This is done via the initField function on the JSFieldText object:

if (hasValue(this.get_fieldName())) {

      var newCtx = {};

 

      newCtx["Templates"] = {};

      newCtx.Templates["Fields"] = {};

      newCtx.Templates.Fields[this.get_fieldName()] = {};

      //Views are handled slightly differently because of the way they are called by SP

      newCtx.Templates.Fields[this.get_fieldName()]["View"] = hasValue(this.viewCallback) ? this.viewCallback : null;

      newCtx.Templates.Fields[this.get_fieldName()]["NewForm"] = this.renderForNew;

      newCtx.Templates.Fields[this.get_fieldName()]["EditForm"] = this.renderForEdit;

      newCtx.Templates.Fields[this.get_fieldName()]["DisplayForm"] = this.renderForDisplay;

 

      SPClientTemplates.TemplateManager.RegisterTemplateOverrides(newCtx);

}

 

This code sets up an object hierarchy in a manner prescribed by SharePoint 2013 and registers the new functions to render each of our possible views of this field:

  • Views
  • Display Form
  • Edit Form
  • New Form

The last line of the snippet is what actually registers our custom handlers with  SharePoint.

Views are handled slightly differently because of how they are called internally by SharePoint.  For Views, we simply pass the callback that was supplied in the constructor of our field. Note that we do check for null and undefined via some helper functions.  If we set the View or any of the Forms to a null or undefined value, SharePoint will use it’s default rendering.

The Forms all use a function on our JSFieldText object because we need to do a little processing before invoking the callback.

Once the callbacks are registered, they’ll be called by SharePoint instead of the default rendering mechanisms.

One of the things that you’ll notice about all of these callbacks is that we’re not dealing with HTML objects.  In each case, we need to return the actual HTML text to render our desired UI, so we’re doing a lot of string manipulation.  That’s just the way SharePoint expects things to be, so we just run with it.

Technical Details: Views

The View callback is a simple method that modifies the value written into the standard SharePoint View as we saw in the screenshots above.  Here’s the code for that:

/// Function called to render the field in a View

function viewFunc(renderCtx, field, listItem, listSchema) {

    var currentVal = '(null)';

    if (renderCtx != null && renderCtx.CurrentItem != null)

        currentVal = eval('renderCtx.CurrentItem.' + field.Name);

    return "<a href="javascript:alert('This field has a value of: " + currentVal + "')">Show Field Value</a>";

}

 

Notice that the name of our function (viewFunc in this case) was the value passed into our JSFieldText constructor earlier.  The other callbacks operate the same way.  Note also the use of the eval statement in there.  I’m not thrilled with that and would like to find a way around it but haven’t come up with anything yet.

Technical Details: Edit Form

The Edit Form callback is used to render the field on an Edit Form.  Here’s the code for the sample I built:

/// Function called to render the NewForm & EditForm view of the field

function editFunc(renderCtx) {

    var fieldHtml = '<select ';

    fieldHtml += 'id="myNiftySelect" onchange="moveValue('' + this.get_fieldId() + '')">';

    fieldHtml += getSelectOptions(this.get_fieldValue());

    fieldHtml += '"</select>';

    fieldHtml += getStandardInputHtml(this, "hidden");

    return fieldHtml;

}

 

This one is a little more complex than the View, but it is still just building up an HTML string to be returned to SharePoint and ultimately stuffed down the pipe to the client’s browser.  There are a couple of helper functions that you can review in the full source code listing.  In the listing above, all I’m rendering is the select box to present the choices.  The actual field used by SharePoint is handled in the getStandardInputHtml function.  Remember, it renders as a hidden field and I use a simple JavaScript function (moveValue, shown below) to copy the selected value from the dropdown list to the hidden field so it is available to SharePoint.

The moveValue function also handles my custom validation:

function moveValue(targetId) {

    var mySelect = document.getElementById('myNiftySelect')

    var hiddenField = document.getElementById(targetId);

 

    var selectedVal = mySelect.options[mySelect.selectedIndex].text

    if (customValidationPasses(selectedVal)) {

        hiddenField.value = selectedVal;

    }

    else {

        mySelect.selectedIndex = mySelect.selectedIndex - 1;

        selectedVal = mySelect.options[mySelect.selectedIndex].text

        hiddenField.value = selectedVal;

    }

}

 

function customValidationPasses(newValue) {

    if (newValue === "Four") {

        alert("Four is right out!  Counteth thou to THREE and then lobbeth thou the holy handgrenade of Antioch at thy foe, who, being naughty in my sight, shall snuff it.");

        return false;

    }

    return true;

}

There’s a fair amount going on in the editFunc function, but none of it is very complex.  This is another area I’d like to work on tweaking a bit.

As I said before, for the sake of this demo, I have hardcoded in an array to provide the choices for the dropdown:

//keeping it simple for the sake of this demo code - not really calling a service to get values

function getOptionsFromSimulatedWebServiceCall() {

    return ["One", "Two", "Three", "Four", "Five"];

}

 

If you needed to make this dynamic, simply change the code in that method (and probably its name, too) to do whatever you need.

Technical Details: New Form

The New Form rendering is actually handled by the editFunc function.  Notice back in the IIFE when we create the JSFieldText object that we pass editFunc in twice:

var r = new JSFieldText("SiteColumn1", viewFunc, editFunc, editFunc, displayFunc);

The first one is actually the New Form callback.  If you needed to render your field differently on the New Form, simply pass in a unique callback here.

Technical Details: Display Form

Last but not least is the Display Form.  Remember this shows the numeric representation of our field’s value (kind of lame, I know, but I wanted to show something different for each type of rendering and this was the best I could come up with).  The displayFunc function handles this for us:

/// Function called to render the DisplayForm view of the field

function displayFunc(renderCtx) {

    if (renderCtx != null && renderCtx.CurrentFieldValue != null) {

        var fieldVal = renderCtx.CurrentFieldValue.toString();

        switch(fieldVal)

        {

            case "One":

                return "1";

                break;

            case "Two":

                return "2";

                break;

            case "Three":

                return "3";

                break;

            case "Four":

                return "4";

                break;

            default:

                return "5";

        }

    }

    return '';

}

 

The Future’s So Bright…

I’m quite excited about the possibilities being opened up for developers with changes like the JSLink property in SharePoint 2013.  While a number of folks are bemoaning the move to the cloud and the changes that brings, I’m quite bullish on the whole thing.  Sure, there are some things I don’t like, but all in all, as a developer, I’m happy.  At least in the 2013 release, I don’t really lose anything very important for my on-premises deployments and I pick up significant ground in my cloud implementations.  A win all around.

Specifically for custom field types, I see this capability as incredibly powerful.  A nice, easy, standard (and supported) way of customizing the end user experience to be more efficient and friendly opens up a lot of possibilities.  I predict a small segment of the app store will be based around custom fields utilizing this capability.

Conclusion

That’s all there is to this.  As I said at the beginning, with it all laid out and explained, there’s not a lot to this and it is pretty easy.  I think that bodes well for this type of customization in the future.  I plan on spending a lot more time on this once the RTM bits are available.  Once I get it all working and cleaned up, I’ll integrate it into my JavaScript series and probably into a SharePoint 2013 version of the CKS:ScriptLibrary project as well.

Comments and ideas for additional functionality are welcome.  I do realize that a lot of the code in this needs to be cleaned up and like I said, I’ll get to that with the RTM bits.  If you have specific comments about something that doesn’t work, or could be refactored to make this better, I’d specifically like to hear those.

Source File: SampleJSField.js.zip

The SharePoint JavaScript Object Model–Resources and Real World Examples

Series Contents

  • Part 1: Introduction and Basics
  • Part 2: Deploy & Download
  • Part 3: The Client Side Object Model  (this post)
  • Part 4: Functions, Objects and Classes
  • Part 5: Avoid the Global Namespace *
  • Part 5: Download & Caching *
  • Part 6: Efficiency & Performance*
  • Part 7: Testing *
  • Part 8: Professional JavaScript Development *
  • Part 9: Error Handling *
  • Part 10: Advanced Topics *
  • Part 11: Security *
  • Part 12: Implementing Business Logic *
  • Part 13: Tools and Resources *
  • Part 14: Wrap-Up *
  • Bonus: JavaScript in SharePoint 2013 *

*: Coming Soon…

Synopsis and Key Take-Aways

This article introduces the JavaScript client-side object model (CSOM) added to SharePoint 2010, covering a discussion of when and why you would use the JavaScript CSOM and some important aspects that impact the performance and scalability of your application.  It finishes with a handful of examples of the CSOM in action performing typical (and perhaps not so typical) actions as well as links to other examples and reference material.

Introduction

If you’re new to the JavaScript CSOM, the best place to start learning about it is a decent overview.  You need to get an

Why Use the JavaScript CSOM?

There are a few key pieces of information about the JavaScript CSOM that will help to answer this question:

  • The code runs off of the SharePoint server in the users browser
  • The code can still interact with SharePoint at a pretty low level – retrieving content from SharePoint, creating and modifying content stored in SharePoint, modifying aspects of SharePoint itself (managing lists and libraries, pages web parts, etc.)
  • The code can be combined with other JavaScript libraries – most importantly JQuery

With that initial understanding, you can start to see benefits offered by the JavaScript CSOM: we can provide the rich, engaging user experience that our user’s want while still interacting with SharePoint in a way that maintains the integrity of our SharePoint server environment.

To give you an idea of what is possible before going into detail, here are a few things that can be done from the CSOM:

  • Full CRUD (Create, Read, Update & Delete) capabilities for List Items, Lists, Content Types, Webs and other key objects.
  • Add web parts to a page
  • Remove web parts form a page
  • Create new workflows associations for existing workflow templates
  • Work with user permissions for securable objects
  • Manage security groups
  • Activate/deactivate Features from a given scope
  • Manage fields on lists, libraries and content types
  • Work with files in document libraries
  • …and a whole lot more…

Why Would You NOT Use the JavaScript CSOM?

Not everything is available in the CSOM – some areas of SharePoint simply cannot be accessed by code utilizing the CSOM.  More on this in a moment.  In addition, some requirements are best met with server side code – see below for details.

What Can We Access?

In general, CSOM code can interact with SharePoint at the site collection level and below.  This means that the following are available:

  • SPSite (although we can’t reach outside of our current site collection)
  • SPWeb
  • SPList
  • SPDocumentLibrary
  • SPListItem
  • SPFile
  • SPField
  • SPUser
  • web parts
  • security
  • etc.

The The Official JavaScript Object Model Reference lists out all of the available objects.  It is important to note that not all aspects of some of these objects are available – some properties and methods available in the server side object model are blocked.  For example, you can access the Site object to get to a site collection, but not the SPSite constructor, which would allow you to connect to a different site collection; it is blocked.  One final note, the SPSecurity object is not available as well.  This means that you cannot do any type of elevation of privilege from the JavaScript CSOM.  All code runs as the current user.

When analyzing requirements to determine whether CSOM is an option, consider the level of access required.  In general, if it is within a single collection, you can probably do it with CSOM.

What Scenarios Don’t Work?

Even if you are only using allowed objects in your JavaScript code, there are some scenarios that just don’t lend themselves to client-side processing.  For example:

  • Processing large amounts of data – pulling all of that data across the wire down to the client may not be a good way to process it
  • Security concerns – pulling data across the wire to process it without some type of encryption (e.g. SSL) may present security problems
  • Code that needs to run whether users are connected to and browsing your site or not, for example workflows and timer jobs. 
  • Code that must respond to server-side events, such as after an item has been added to the database
  • Code that must run, and must run exactly as written.  This is a hard one.  User’s can turn off JavaScript.  Malicious users could also modify the cached version of the JavaScript file on their machine and run their modified code instead of the code you intended for them to run.  What happens to your data integrity and environment if your JavaScript code doesn’t run, or doesn’t run the way you expect it to run?  Understand that large portions of SharePoint will stop functioning or at least have a severely reduced level of functionality, but it is not impossible to use SharePoint with JavaScript turned off.  What happens to your environment then?  It seems to me that this is a largely ignored area, but that doesn’t mean it isn’t important.  Honestly, I don’t have a really good answer for this yet. It’s just an area you need to think about – what are the implications?

Requirements

To make use of the JavaScript CSOM in your SharePoint site, you need to make sure that the CSOM library is loaded by your page.  This will be done for you from the Master Page if you use one of the out-of-the-box  Master Pages.  If you use a custom Master Page, you must make sure that the sp.js file is loaded.  Part 2: Deploy & Download covers some aspects of this.  Additional material will be covered in later modules.

Advanced Topics

With a basic understanding of the JavaScript CSOM and what is and is not possible, we can move on to some more advanced topics.

Batch processing

All commands in the JavaScript CSOM are not executed immediately when the JavaScript engine in your browser evaluates the line of code.  Instead, the command is added to a batch and executed along with the other commands in the batch at a point in the future.  This is one of the fundamental paradigms of JavaScript CSOM programming that must be understood.  The hardest part of this whole operation is understanding when you must commit a batch and how to do so. 

At the most basic level, you must commit a batch of commands prior to interacting with the properties of any objects referenced in the commands.  An example will help to clarify this:

   1: function runCode() {

   2:   var ctx = new SP.ClientContext(); 

   3:   //Store the Announcements list in a variable:

   4:   var list = ctx.get_web().get_lists().getByTitle('Announcements');

   5:   //Store a list item in a variable

   6:   listItem = list.getItemById(4);

   7:   //Load the list item so we have access to its 

   8:   //properties

   9:   ctx.load(listItem);  

  10:   //To this point, we have not left the client and 

  11:   //none of our code has actually "done" anything.  

  12:   //All of the commands have simply been added to a 

  13:   //batch.  The next line sends the batch to the 

  14:   //server so all of our commands will execute.  

  15:   //After the following line has returned from its 

  16:   //call to the server, the list and list item we 

  17:   //"retrieved" are available for us to access and 

  18:   //work with its properties

  19:   ctx.executeQueryAsync(onQuerySucceeded, onQueryFailed);

  20: }

  21:  

  22: function onQuerySucceeded() {

  23:     alert('Item Title is ' + listItem.get_item('Title'));

  24: }

  25:  

  26: function onQueryFailed(sender, args) {

  27:   alert('Request failed.');

  28: }

Don’t worry about the details of what’s going on in that code sample for now.  We’ll look into those details later.  Read the comments to get an understanding of the code, but here are the things you need to know to understand the batch process:

  • All of the code from line 1 to line 18 runs on the client and has no interaction with the server.  For example, on line 4, the call to getItemById does not actually connect to the server and grab a reference to the SPListItem.  It simply adds the command(s) to the list of commands queued up to be committed (this is the “batch” referred to earlier)
  • Line 19 commits the batch to the server.  When this line executes, the batch is packaged up and sent across the wire to the server.  The server executes the code we’ve told it to run and returns a result to our page.
  • Line 19 also registers 2 callback methods – one for if the batch runs successfully (on lines 22-24), one if the batch fails (on lines 26-28).  After the batch has returned form the server, the appropriate callback method is run and the items collected in lines 1-18 (list & listItem) are available for use.
  • Line 23 references one of the properties from the listitem (specifically the Title property) that is now available to us because we have committed the batch to the server and gotten a populated ListItem back.

Why Batches?

The JavaScript CSOM code runs on the client machine, inside their browser.  As we saw above, it commits batches of commands to the server to be executed.  It then receives data from the server.  By operating in batches like this, the number of round trips across the wire is minimized.  Sending data to the server and getting a response is the slowest part of the CSOM; traversing the network is inherently slow.  The batch operation keeps our CSOM applications from being excessively chatty which would slow things down considerably.

Things to Know About Batching and Objects

The batch operations have a couple of quirks that are important to know.

Scalar and Collection Properties

Besides the fact that CSOM operates in batches like this, the only really important thing to know about the batched nature of CSOM operations is how we go about accessing objects and their properties in a CSOM application. 

For example, consider the following code:

var ctx = new SP.ClientContext(); 

var web = ctx.get_web();

ctx.load(web);

This code is responsible for filling up the (in this case) web object so that we can access its properties.  It is important to note, however, that not all properties of the object are filled up (populated) by this command.  It is only the scalar (i.e. non-collection) properties that will have data after a load command and its corresponding executeQueryAsyc command.  In other words, simple properties – ones with data types like string, int, bool, etc. are populated.  Properties that contain collections are not populated; they must be explicitly filled up.  In the case of the Web object, the following are examples of properties that would be populated after a call to load like we have in the code above:

  • id
  • rootFolder
  • serverRelativeUrl
  • title
  • etc.

These are all simple scalar properties.  (Note that this is not an exhaustive list of the scalar properties, and each object has its own collection of scalar properties.)

On the other hand, collection properties such as the following are not populated by the call to load shown above:

  • lists
  • folders
  • features
  • fields
  • availableContentTypes
  • etc.

These properties all represent a collection of items.  In order to populate these properties, we need to explicitly load the collection property:

var ctx = new SP.ClientContext(); 

var web = ctx.get_web();

var lists = web.get_lists();

ctx.load(lists);

ctx.executeQueryAsync(onQuerySucceeded, onQueryFailed); 

We can now work with the collection of lists from the Web.  Notice that we don’t need to load up the web object because we don’t access any of its properties, except the lists collection property, which we are explicitly loading.

If you don’t fill up a collection property and commit the batch before trying to access the collection property, you will get an error message that says The collection has not been initialized. It has not been requested or the request has not been executed. It may need to be explicitly requested.

Performance: Property/Column Values and RowLimit

A key tenet of querying SharePoint data server-side is to only pull from the database the minimum you need to accomplish your requirement.  In other words, don’t pull back all of the columns on a list item if you’re only going to work with, for example, the Title and Body column values.  A similar approach is important for retrieving content from SharePoint with the JavaScript CSOM, with one additional reason – as stated previously, sending data across the wire is expensive in terms of performance.  The more data that has to traverse the wire, the greater the performance impact.  Therefore, it is important to only extract and pull down the values we need to work with.  In the JavaScript CSOM, this is done in one of two ways:

  1. Listing the property names we need in the load statement
  2. Trimming our queries to retrieve only the columns we need

The latter is the same concept as what we’ve done using SPQuery for years, although the syntax is slightly different, as you’ll see in a moment.  The former is new to CSOM. Unlike the load statements shown above which do not specify the properties we need, and therefore pull down all properties, a more efficient load statement would look like this:

ctx.load(lists, 'Include(Title,DataSource)');  

This would limit the properties that are populated on our lists object to just the Title of each list and the DataSource. If you do limit the properties that are populated in this way, you need to make sure that you specify every property that you need.  Similar to the error above if a collection property is not populated, if you attempt to access a scalar property that has not been populated, you will get an error The property or field has not been initialized. It has not been requested or the request has not been executed. It may need to be explicitly requested.

The syntax for trimming the columns sent down to the client is similar to what we use on the server, but just slightly different syntactically:

var ctx = new SP.ClientContext();

targetList = ctx.get_web().get_lists().getByTitle('Announcements');

query = new SP.CamlQuery();

query.set_viewXml("<View><ViewFields><FieldRef Name='Title'/></ViewFields><RowLimit>2</RowLimit></View>");

listItems = targetList.getItems(query);

ctx.load(listItems); 

ctx.executeQueryAsync(onQuerySucceeded, onQueryFailed);

You can see the way the ViewFields elements are set into the query, which is stored in the viewXml property of the CamlQuery object.  This example will only retrieve the Title column value from the database and send it down the wire.

This example also shows how to limit the number of records that are sent down to the client.  This is another important aspect of maintaining performance. In this example, the result set is limited to the first two records by use of the RowLimit element..

Examples of Real World Tasks

There are only a certain number of core operations that are typically going to be done via the JavaScript CSOM.  There are dozens or hundreds of other operations to perform in CSOM, but the majority of the time, your operations will involve the following in some form and then add the other operations on top of these.

NOTE: I wrote some of these samples for MSDN back during the beta of 2010 and some of them are available in the SDK, although I tweaked some of them a little and tested them all for this blog post.

Get ListItem By ID

If you know the ID of a ListItem, this sample demonstrates how to retrieve it.  It also demonstrates how to retrieve a List by name.

 

var itemId = 4;   

var targetListItem;

 

function runCode() {

  var ctx = new SP.ClientContext(); 

  var targetList = ctx.get_web().get_lists().getByTitle('Announcements');

  targetListItem = targetList.getItemById(itemId);

  ctx.load(targetListItem, 'Title');  

  ctx.executeQueryAsync(onQuerySucceeded, onQueryFailed);

}

 

function onQuerySucceeded() {

    alert('Request Succeeded. nRetrieved Item Title is ' + targetListItem.get_item('Title'));

}

 

function onQueryFailed(sender, args) {

  alert('Request failed. nError: ' + args.get_message() + 'nStackTrace: ' + args.get_stackTrace());

}

Submit a CAML Query

This sample shows you how to query a list using CAML and iterate through all items in a collection.

 

var listItems;

 

function runCode() {

  var ctx = new SP.ClientContext();

  var targetList = ctx.get_web().get_lists().getByTitle('Announcements');

  var query = new SP.CamlQuery();

  query.set_viewXml("<View><Query><Where><Contains><FieldRef Name='Title'/><Value Type='Text'>MyValue</Value></Contains></Where></Query></View>");

  listItems = targetList.getItems(query);

  ctx.load(listItems);

  ctx.executeQueryAsync(onQuerySucceeded, onQueryFailed);

}

 

function onQuerySucceeded() {

  var listEnumerator = listItems.getEnumerator();

  while (listEnumerator.moveNext()) {

    alert("Found matching Item! nTitle=" + listEnumerator.get_current().get_item("Title"));

  }  

}

 

function onQueryFailed(sender, args) {

  alert('Request failed. nError: ' + args.get_message() + 'nStackTrace: ' + args.get_stackTrace());

}

Paging Through a ListItemCollection

This sample pages through the items in a list two at a time, unlike the previous example which iterates through all items in the returned collection in one operation.

var listItems;                   //holds retrieved list items

var query;                       //must continue to reuse the same query object for paging to work

var targetList;                  //list from which to retrieve items

var ctx;

               

 

function runCode() {

  ctx = new SP.ClientContext();

  targetList = ctx.get_web().get_lists().getByTitle('Announcements');

  query = new SP.CamlQuery();

  query.set_viewXml("<View><ViewFields><FieldRef Name='Title'/></ViewFields><RowLimit>2</RowLimit></View>");

  listItems = targetList.getItems(query);

  ctx.load(listItems); 

  ctx.executeQueryAsync(onQuerySucceeded, onQueryFailed);

}

 

function onQuerySucceeded() {

  var message = "Found Matching Items! ";

  var listEnumerator = listItems.getEnumerator();

  while (listEnumerator.moveNext())  {

    message += "nTitle=" + listEnumerator.get_current().get_item("Title")

  }

  alert(message);

  var position = listItems.get_listItemCollectionPosition();

  if (position != null) {

    query.set_listItemCollectionPosition(position);

    listItems = targetList.getItems(query);

    ctx.load(listItems);  

    ctx.executeQueryAsync(onQuerySucceeded, this.onQueryFailed);

  }

  

}

 

function onQueryFailed(sender, args) {

  alert('Request failed. nError: ' + args.get_message() + 'nStackTrace: ' + args.get_stackTrace());

}

 

Retrieve a Web – #1

This sample just retrieves the current web (whichever Web hosts the page containing the JavaScript.)

var targetWeb;

 

function runCode() {

    var ctx = new SP.ClientContext();

    targetWeb = ctx.get_web();

    ctx.load(targetWeb);

    ctx.executeQueryAsync(onQuerySucceeded, onQueryFailed);

}

 

function onQuerySucceeded() {

    var message = "Web retrieved:";

    message += "n Title: " + targetWeb.get_title();

    message += "n ID: " + targetWeb.get_id();

    message += "n Language: " + targetWeb.get_language();

    message += "n UI Version: " + targetWeb.get_uiVersion();

    message += "n Description: " + targetWeb.get_description();

    message += "n Created: " + targetWeb.get_created();

    alert(message);

}

 

function onQueryFailed(sender, args) {

    alert('Request failed. nError: ' + args.get_message() + 'nStackTrace: ' + args.get_stackTrace());

}

Retrieve a Web – #2

This sample shows how to retrieve a Web based on its server-relative URL.  Note: the Web being retrieved must be in the same site collection.

var targetWeb;

 

function runCode() {

    var ctx = new SP.ClientContext('/relative_url');

    targetWeb = ctx.get_web();

    ctx.load(targetWeb);

    ctx.executeQueryAsync(onQuerySucceeded, onQueryFailed);

}

 

function onQuerySucceeded() {

    var message = "Web retrieved:";

    message += "n Title: " + targetWeb.get_title();

    message += "n ID: " + targetWeb.get_id();

    message += "n Language: " + targetWeb.get_language();

    message += "n UI Version: " + targetWeb.get_uiVersion();

    message += "n Description: " + targetWeb.get_description();

    message += "n Created: " + targetWeb.get_created();

    alert(message);

}

 

function onQueryFailed(sender, args) {

    alert('Request failed. nError: ' + args.get_message() + 'nStackTrace: ' + args.get_stackTrace());

}

Retrieve a Web – #3

This sample shows how to retrieve the site at the root of a site collection from a child web somewhere else in the site collection.

var targetWeb;

 

function runCode() {

    var ctx = new SP.ClientContext('/');

    targetWeb = ctx.get_web();

    ctx.load(targetWeb);

    ctx.executeQueryAsync(onQuerySucceeded, onQueryFailed);

}

 

function onQuerySucceeded() {

    var message = "Web retrieved:";

    message += "n Title: " + targetWeb.get_title();

    message += "n ID: " + targetWeb.get_id();

    message += "n Language: " + targetWeb.get_language();

    message += "n UI Version: " + targetWeb.get_uiVersion();

    message += "n Description: " + targetWeb.get_description();

    message += "n Created: " + targetWeb.get_created();

    alert(message);

}

 

function onQueryFailed(sender, args) {

    alert('Request failed. nError: ' + args.get_message() + 'nStackTrace: ' + args.get_stackTrace());

}

 

Additional Examples From the SDK

There’s a section in the SDK with some good examples of common tasks.  Rather than completely reinvent the wheel, I’ll just link to them:

Reference Material

SDK: The Official JavaScript Object Model Reference (object/class reference)

SDK: CSOM Introduction (covers all three client object models and gets pretty deep pretty fast, but not a bad read)

JavaScript: Deployment and Download

Series Contents

  • Part 1: Introduction and Basics
  • Part 2: Deploy & Download (this post)
  • Part 3: The Client Side Object Model *
  • Part 4: Functions, Objects and Classes
  • Part 5: Avoid the Global Namespace *
  • Part 5: Download & Caching *
  • Part 6: Efficiency & Performance*
  • Part 7: Testing *
  • Part 8: Professional JavaScript Development *
  • Part 9: Error Handling *
  • Part 10: Advanced Topics *
  • Part 11: Security *
  • Part 12: Implementing Business Logic *
  • Part 13: Tools and Resources *
  • Part 14: Wrap-Up *
  • Bonus: JavaScript in SharePoint 2013 *

*: Coming Soon…

Synopsis and Key Take-Aways

This article covers important considerations for the mechanics of using JavaScript in your custom solutions and concludes with the recommendations for using a SharePoint Document Library as the repository for your script files, and the Custom Action approach to getting the files downloaded to the user’s browser.  Other approaches are reviewed briefly and discarded

 

Deploy and Download

In the first part of this series, we did a very basic overview of JavaScript, not really touching on SharePoint at all.  In this posting, we’re going start touching upon some SharePoint-specific aspects of working with JavaScript.  So far, all we’ve done is type some JavaScript directly in to the Console of our browser, we haven’t needed a server.  Obviously, that is not realistic for a production scenario.

In a real-world situation, we will have one or more JavaScript files, called libraries, which contain our code and need to be deployed to the server and then downloaded to the user’s browser when we need to access the functionality provided by the library.  Typically these libraries will have a .js extension and will be included along with other files in our Visual Studio project.  That project will generate a WSP (solution) file which will be deployed to our server environment at either the sandbox or the farm level.  When the WSP is deployed, it will make our JavaScript file available for download.  We then need to make sure that the file is downloaded.

Getting Started

For the sake of this post, I’ll assume that you have already created a VS project using one of the SharePoint project templates.  It doesn’t matter whether you create a farm solution or a sandbox solution.  I’ve used the Empty SharePoint Project template.  Create a new JavaScript library by right-clicking on your project node in Solution Explorer and select Add | New Item on the context menu.  In the dialog box that pops up, click Web in the template category listing on the left and then select JScript File:

image

This produces a blank JavaScript file in the root of your project, which is fine for now; we’ll move it in a little bit.  The name of my file is the default JScript1.js and I’ll refer to this name throughout this post.  If you use a different filename, you’ll need to adjust references later in the post as appropriate.  Add the following very basic JavaScript code to the new file and save it:

function sayHello(){

    alert("Hello World");

}

This is about the simplest JavaScript function you could imagine, but it is sufficient for our needs – we’re focusing on getting our JavaScript deployed and downloaded in this post, not on actually writing JavaScript.

Now that we have a file with some JavaScript in it, we can move ahead with deployment and then getting it downloaded to the user’s browser.  First we need to explore some options for deployment to understand the best option for the majority of the scenarios we will encounter.

Deployment Options

In a production scenario, we need to place the file which contains our JavaScript functions on the server and have it downloaded to the client to be executed.  The first thing this means is that we need to identify a location on the server to which our JavaScript file can be deployed.  The only real requirement for this location is that it be URL-addressable so that the user’s browser can request it and download it.  In SharePoint, there are several locations that could fit this bill:

  1. The _layouts virtual directory
  2. Ghosted site pages
  3. A Document Library

We’ll touch upon each of those in the following sections, and rule out all but the last.  A few notes about the criteria used to rate each option:

  • Performance is a consideration, though not necessarily a huge one thanks to browser caching
  • I’d prefer to keep my approach to JavaScript downloading as simple as possible so I’m looking for the broadest applicability of each approach. Approaches that work across a wide range of scenarios will rank higher.
  • Ease of maintenance is important
  • Supporting multiple versions of a library is important
  • Supporting metadata about the libraries is important
  • Flexibility is important – I don’t want my choice for deployment to dictate anything about the approach to usage of the libraries if at all possible

_Layouts

Deploying our .js files to the _layouts virtual directory is the most like a “regular” ASP.Net site.  When the user’s browser requests the file, it is delivered to them straight from the file system of the server.  In SharePoint, deploying to the _layouts virtual directory is easy – especially if we are using the Visual Studio Tools for SharePoint.  While it is arguably the easiest location to which we can deploy our .js files, as well as the location which delivers the best download performance, it is unfortunately not the best location for our script files.  For one thing, it is not sandbox compatible.  It is not possible to deploy files to the _layouts folder in a sandbox solution.  Another problem is that it offers the least capability from a metadata and management point of view.  For these reasons, it is not a recommended location to which to deploy JavaScript files.

 

Site Pages

The next option is to deploy our script files as ghosted site pages using a SharePoint Feature.  This overcomes some of the limitations of _layouts pages but not all of them.  Unlike _layouts, site pages are fully sandbox compatible.  That’s a good thing.  From a performance point of view, in non-sandbox solutions (i.e. farm solutions), performance will be comparable to _layouts pages as the site pages will be delivered from the file system, even though we deploy them as site pages.  In sandbox solutions, performance will not be quite as good as the content will be served from the content database instead of the file system.  That’s not necessarily a bad thing, but it is worth mentioning.  The one thing that site pages does not address is the manageability of the script libraries.  Managing our files requires SharePoint Designer and there is still no effective means for managing metadata about our script files when deployed as site pages, so we will not be exploring this option any further.

Document Library  **Recommended**

The final choice for deploying script files is to push them to a document library via a Feature.  This is the recommended location for our script files.  Primarily this is because it offers us the most capabilities for managing those script libraries, as we’ll see in just a moment.  It is important to note, however, that deploying to a document library will have a slight performance impact when our files are requested and downloaded to the user’s browser.  Normally a known performance impact is something we would try to avoid.  However, in this case it is OK for one main reason –   JavaScript files are cached on the user’s machine and so they do not get downloaded with every page.  This means that we typically only incur that slight performance hit once per user.  The rest of the time, when the browser needs our JavaScript file, it pulls it from the local cache instead of re-requesting it from the server.  Thus the performance impact is avoided on all subsequent page loads.  If the user clears their browser cache, the file will be automatically redownloaded.

This works so long as you have repeat visitors (so the JavaScript files will be cached on the client), such as in an intranet or perhaps an extranet.  In Internet-facing sites, the performance implications may be more severe and the benefits of the document library somewhat lessened.  In that case, the _layouts folder may be a better option.

Deploying to a Document Library

Deploying JavaScript files to a document library is trivially easy, as you’re about to see.  We’ll begin from our very basic SharePoint project discussed above, containing just the single JavaScript library.  In order to deploy this file to a document library, we first need to make sure that the document library exists in our site. 

Note: If you know that this document library already exists in your site, you can skip this step.

 

Creating the Scripts library

In SharePoint 2010, we can very easily add an instance of the Scripts document library to our project:

  1. Right click on your project name in Solution Explorer
  2. Select Add | New Item
  3. Select the SharePoint | 2010 category from the list of installed template categories
  4. Select the List Instance template
  5. Change its name to Scripts
  6. Click Add

image

In the next dialog box, configure the options as shown and click Finish:

image

Add the JavaScript file to the Scripts document library

Whether we just created the document library or it existed already, we add our JavaScript file to it in the same manner:

  1. Right click on your project name in Solution Explorer
  2. Select Add | New Item
  3. Select the SharePoint | 2010 category from the list of installed template categories
  4. Select the Module template
  5. Change its name to ScriptFiles
  6. Click Add
  7. In Solution Explorer, delete the Sample.txt file included in the ScriptFiles module
  8. In Solution Explorer, click on the JavaScript file we created earlier, drag it and drop it on the ScriptFiles node.
  9. If the Elements.xml file for the ScriptFiles module is not open, double click it in Solution Explorer to open it
  10. Notice that the Elements.xml file has been updated to include a reference to the file we just added to the module
  11. Change the XML in the Elements.xml file to look like this:
<?xml version="1.0" encoding="utf-8"?>

<Elements xmlns="http://schemas.microsoft.com/sharepoint/">

  <Module Name="ScriptFiles" Url="Scripts" List="101">

      <File Path="ScriptFilesJScript1.js" Url="JScript1.js" Type="GhostableInLibrary" />

</Module>

</Elements>

     

That’s all you need to do to make sure that your JavaScript file is properly deployed to your SharePoint environment.  If you’d like to test it out, hit F5 and when the browser comes up, navigate to <site URL>/Scripts.  You will see your JavaScript file listed:

image

Getting the file deployed is only half the battle.  Now we need to get it downloaded to the users browser so it can be used.

Downloading Options

JavaScript libraries are useless if they never leave the server; they must be downloaded to the user’s browser in order for the code they contain to be executed.  Similar to getting our library deployed, there are several options for getting it downloaded.  We’ll explore each option, discussing its pros and cons before declaring a winner.

A couple notes about the criteria I’m using to evaluate these options:

  1. Performance is a consideration, though not necessarily a huge one thanks to browser caching
  2. I’d prefer to keep my approach to JavaScript downloading as simple as possible so I’m looking for the broadest applicability of each approach.  Approaches that work across a wide range of scenarios will rank higher.
  3. I’m looking for the broadest reach when deploying.  I want one place to manage all of my JavaScript downloads.  I am not interested in deploying specific libraries only to specific pages in my application or conditionally downloading libraries based upon some criteria.  The vast majority of your JavaScript is likely to be used in multiple places so combining it together and downloading it in a small number of requests and taking advantage of browser caching makes sense.  While it is true that some JavaScript will only be needed on certain pages, it is far better to use a delayed loading or script-on-demand approach (discussed in part 11 in this series) to control those downloads.
  4. Compatibility with a delayed-loading scheme such as SharePoint’s Script On Demand is important

ScriptLink Control

The ScriptLink control was introduced in SharePoint 2007 and is still available in 2010.  Its main responsibility is to cause a JavaScript file to be downloaded to the user’s browser.  It does this by ensuring that the proper <script> tag is written into the HTML stream.  While the ScriptLink control is often recommended, I am going to shoot it down for two reasons:

  • it is not sandbox compatible.  Say what you will about the sandbox, and even toss in the fact that it is not a recommended approach for SharePoint 2013, it is an option in 2010, is the only option for server-side code in Office 365 and therefore important, at least for now.  As SharePoint and SharePoint Online evolve, this will need to be revisited.
  • it is hard to maintain as needs change.  In this way, it is similar to adding the <script> tags directly to the master page – changing it requires editing the master page in SharePoint Designer, which is something that should be avoided.

It is unfortunate that the ScriptLink suffers from these drawbacks, because it does provide value that we’re going to be hard-pressed to replicate.

Master Page Script Tags

Adding <script> tags to your master page is just not a good idea.  They’re hard to maintain, hard to work with initially, don’t travel well if you switch master pages, require SharePoint Designer to edit and various other problems.  While they may seem initially like a good idea, they’re just not in the long run.  Don’t do it.

Web Part

This option entails using the dreaded Content Editor Web Part to add a <script> tag directly to a page.  This is troublesome for a few reasons:

  • It is page-specific.  It only impacts the page to which you add it.  While this may seem like a good thing, it is not.  There are many pages in SharePoint that you can not add a web part to – how will you get your JavaScript onto those pages?  It also means that you need to add this web part on to every page upon which you want the JavaScript available.  If your needs change, there are a lot of places that you will now need to change.
  • It does not version well, cannot be included in source control and makes debugging more difficult.  These all fall under an umbrella of professional development disciplines and the CEWP fails miserably here.

Custom Action   **Recommended**

Adding the HTML to our page via the SharePoint Custom Action element offers many advantages and no disadvantages.  It is easily the best option:

  • It is fully sandbox compatible and fully farm compatible.  This means that I can use the same approach no matter how my solution is being deployed.
  • It is global – all pages in the scope at which we activate its Feature (Site or Site Collection) will have the appropriate HTML added to it
  • It is flexible and easy to maintain.  We take advantage of the SharePoint Feature framework to be able to turn the functionality on and off at whatever scope we need it
  • There are no performance penalties to using the Custom Action
  • It is fully compatible with SharePoint’s Script on Demand functionality and various other delayed loading schemes we’ll talk about in part 11 of this series. 

Therefore it is easy to declare the Custom Action as the best option for getting our JavaScript files downloaded.  The next section walks through how to accomplish this.

Downloading with a Custom Action

Adding the necessary HTML to our pages to cause our JavaScript files to be downloaded with a Custom Action element is easy:

  1. Right click on your project name in Solution Explorer
  2. Select Add | New Item
  3. Select the SharePoint | 2010 category from the list of installed template categories
  4. Select the Empty Element template
  5. Change its name to MyJSFile
  6. Click Add

Add the following to the Elements.xml file that Visual Studio creates and opens for you (placing it between the opening and closing Elements tags):

<CustomAction

    Location="ScriptLink"

    ScriptSrc="~Site/Scripts/JScript1.js"

    Sequence="10"

    Id="MyScriptLibrary"

/>

That’s it. SharePoint will now add the following to every HTML page in our site:

document.write('<script type="text/javascript" src="/scripts/jscript1.js"></' + 'script>');

This will cause our custom JavaScript library to be download (if it is not already cached on the client) and made available for use.  The functions in our library are now available to be called and executed.  Notice in the Custom Action element the use of the token ~Site.  When activated within SharePoint, this will be replaced with the URL of the site at which the Feature is activated.  Another useful token is ~SiteCollection, which gets replaced with the URL of the site collection.

If you’re curious why SharePoint adds the tag using a document.write instead of simply adding the <script> tag, it’s a mechanism for overcoming some quirks in how browsers download and parse JavaScript files. Without going into too much detail (which I’ll do in another post), it prevents one script from blocking the download of other page elements (scripts, images, CSS, etc) while the script is downloaded and parsed. More on that in a separate post.

 

To test this out, simply hit F5 to deploy and run your code.  After the browser opens to the homepage of your site, type the following into the address bar of the browser: javascript:sayHello(); and hit Enter.  The results should look something like this:

image

Our JavaScript code in the library (the sayHello function defined earlier) ran and popped the alert box up on the screen.  A very simple example, but sufficient to test out our process.  We’ve successfully deployed a JavaScript library to SharePoint and had it downloaded to the client.  Later in this series we’ll be developing more complex JavaScript libraries to begin offering real world functionality, but we’ll take advantage of the same mechanisms we’ve covered in this post for deployment and download.

Other Considerations

There are a number of tangential topics that need to be addressed in order to wrap up any discussion about deploying and downloading JavaScript libraries – improving download and caching, overall performance, etc.  These topics will be addressed in parts 5 (Download & Caching), 6 (Efficiency & Performance) and 10 (Advanced Topics). 

Wrap Up

This post has answered the question of “What is the best way to deploy my custom JavaScript to SharePoint and then have it get downloaded to the user’s browser?”  We’ve discussed a number of options and whittled down to the best approach – document libraries and Custom Action elements.  No matter what functionality we’re looking to introduce – form the simplest of examples (such as the project used here) to the most complex of scenarios, this approach will work equally well.