JQuery PeriodicalUpdater (AJAX long polling/server polling)

I’m working on an app that is using the JQuery JavaScript framework. Time came for a bit of AJAX long-polling (which I can no longer say without snickering thanks to WebDevGeekly), and so I went looking for a way to do that in jQuery: specifically, I wanted something like Prototype’s Ajax.PeriodicalUpdater, which has a nice decay to pull load off the server if not a lot is changing.

Unfortunately, such a beast doesn’t exist within the core JQuery code. I bitched about the lack of one on Twitter (cite), and ddelponte resisted routing me to http://letmegooglethatforyou.com/?q=jquery+periodicalupdater and instead pointed out the #1 hit on Google: 360innovate’s port.

That port didn’t do quite what I wanted, and I saw a few places to eke efficiencies out of the code, so I did. The new version of the code is hosted at http://github.com/RobertFischer/JQuery-PeriodicalUpdater/. Specific advantages over the 360innovate version are:

• Any option in jQuery’s $.ajax can be used, including any callbacks. The only exception is the flag that treats modifications as errors. That’s always going to be true (see the next bullet).
• 304 Not Modified pages are now treated like they weren’t modified (that is, timeout increases). Their treatment before was as errors, which caused the timeout to reset to the base value.
• The settings passed into the function are now deep-copied, which means the setting object can be mangled after the call without hosing up the entire works.
• As much work as possible is done up front, so the actual polling AJAX call is fairly fast and lightweight. This is important so that it doesn’t clog up the limited resource that is JavaScript user processing threads.
• The first poll begins once the document has finished loading, which should speed initial page load and avoid issues caused by the AJAX response returning before the page is totally rendered.

The code for the PeriodicalUpdater is pretty cool. One stunt which people should definitely pay attention to is using executable code blocks for factoring out loop-invariant checks. In this case, it’s demonstrated in the logic to boost the decay:

 // Function to boost the timer (nop unless multiplier > 1) var boostPeriod = function() { return; }; if(settings.multiplier > 1) { boostPeriod = function() { timerInterval = timerInterval * settings.multiplier;
  if(timerInterval > settings.maxTimeout) { timerInterval = settings.maxTimeout; } }; }

In this case, this behavior is either a nop (for multipliers <= 1), or it's got some involved logic. The 360innovate version did the multiplier check each time the interval was going to be boosted, but that multiplier isn't going to change. Since the multiplier isn't going to change, the code can be factored out and the check can be saved.

The same functionality could be done by the function null unless there is logic attached, but then call points look like this:

if(boostPeriod) boostPeriod();

And I’ll take the overhead of a call to the nop method to get easier-on-the-eyes, more maintainable code.

Comments

• June 25, 2009, John McCollum wrote: Hi Robert, (I'm the original plug in author :) ) Nice work on refactoring the plugin. It was originally developed to fill a need for us, and there are definitely bits that could be re-thought. To be honest, this is probably something that should be added to the jQuery core, and I was surprised to learn that it wasn't. Anyway, nice work, and I'm glad you were able to build on my original plugin.
• June 25, 2009, Robert Fischer wrote: @John McCollum Despite my article in JSMag #1, my practical JavaScript Fu is pretty weak. Without the work you did, I couldn't have done what I've done: I had no idea how to start writing such a beast, and your impl was nicely commented and very readable. So thanks for putting your stuff out there, and with such a great license!
• June 26, 2009, Hsiu-Fan Wang wrote: Just an FYI: a function call is pretty expensive in Javascript. Whether "expensive" is "more expensive than a conditional" is a different question though.
• June 26, 2009, Robert Fischer wrote: @Hsiu-Fan Wang I ignore any critique based on what is or is not "expensive" unless I see strong numbers. It's nothing personal: it's just that developers have bitched about everything being "expensive" at one point or another -- all the way back to C. In fact, I imagine the first firmware developer was being told by their circuit-building coworkers that there was no way this "firmware" stuff could ever catch on, because it's just too expensive to translate from electrical impulses to logic. The burden of proof required to get me to change it is this: 1. Demonstrate it's expensive enough that the user will actually notice. 2. Demonstrate that the conditional is cheap enough that the user stops noticing. 3. Demonstrate that the difference between the two is great enough that it is worth littering my code with repetitious logic structures and adding a potential maintenance trick.
• June 26, 2009, Robert Fischer wrote: BTW, the PeriodicalUpdater is under active development in GitHub, so you probably want to track it there if you're interested. http://github.com/RobertFischer/JQuery-PeriodicalUpdater/
• June 26, 2009, Robert Fischer wrote: Oh, I see where there's a bit of confusion. To clarify: the time interval logic was nontrivial and repeated, so it needed to be its own function in any case. Whereas the old implementation had that check inside (actually, two checks), the new implementation breaks out the two cases. I am *not* arguing that you're better off creating functions and pre-calculating the implementation (basically hand window-optimizing). You might be, you might not be -- don't have numbers either way. But if you've got a function kicking around anyway, you might as well factor as much as possible out of the function, and since the function is just a variable, that variable can (and should) be redefined freely to provide the most specific implementation known. Note that the limitations to what can or can't be known about the implementation are all mutable data questions. Clojure is increasingly convincing me that when it comes to sane, sturdy code, it's not about static vs. dynamic typing stunts, it's really about mutable vs. immutable data.
• June 28, 2009, Kenneth Stein wrote: Greetings Robert, A bit confused when I look at your example code in comparison to your README file. In the README file you describe the usage as "$.PeriodicalUpdater('/path/to/service', {..." In the example you have "$.PeriodicalUpdater({url:"queryMe.html"},". The first mangles any "sendData' I include, and the second throws a fatal error. Trying to find out why that is, but have come to the decision that you're in a much better position to discover it much more quickly than me. Appreciate your work, as well as John McCollumn's!! Kenneth Stein
• June 29, 2009, Robert Fischer wrote: @Kenneth Stein The library API changed between this post and now. I got tired of having to use curly braces and the url key name, even if I didn't have any other configuration options, so I pulled the URL out. This API will be consistent from now on. Not sure what you mean by "mangles any 'sendData' I include". The PeriodicalUpdater wraps $.ajax directly, so you can use the configuration available to $.ajax. It doesn't look like there is a sendData property: did you mean data?
• June 29, 2009, Kenneth Stein wrote: Thanks for your quick response Robert. SendData is one of the Settings for periodicalUpdater, namely an array of values to be passed - e.g. {name: "John", greeting: "hello"}. SEE the README file. The example you provide at Github doesn't use the sendData setting, and it doesn't seem to be passing the information into the ajax call. John McCollum's implementation passes the sendData setting infor without any problem. Something broke in the refactoring it seems. It's awesome that you really got after this code and tightened it up. Just hoping that in doing so you haven't uncovered something more problematic. Will be looking for your response.
• June 29, 2009, Robert Fischer wrote: @Kenneth Stein I changed from using $.post/$.get to using $.ajax, apparently which changed the API due to inconsistencies on jQuery's side¹. Change "sendData" to "data" in your call and you'll be fine. I've added a hack to route sendData into data in the GitHub version, too. But data should be preferred to sendData to retain consistency with $.ajax, which PeriodicalUpdater explicitly mimics. I've updated the README with the improved usage. ¹ Inconsistencies in jQuery's API are its one deep flaw.
• June 29, 2009, Kenneth Stein wrote: Bingo! Thanks again Robert. Awesome!!
• July 2, 2009, Alex wrote: I don't know if it should be like that, but i can't use two updaters on one page. $.PeriodicalUpdater('/path/to/service', { method: 'get', }, function(data) { // update div1 }); $.PeriodicalUpdater('/path/to/another/service', { method: 'get', }, function(data) { // update div2 }); On this page only div1 will be updated with content from '/path/to/another/service'
• July 2, 2009, Robert Fischer wrote: I'm using it precisely that way on my client's page: should work just fine. My guess is that you're assigning and then re-assigning a variable, and the re-assigned version is getting used.
• July 9, 2009, Z wrote: is there an easy way to stop and start the service?
• July 9, 2009, Robert Fischer wrote: @Z Not at this point. Patches are welcome.
• July 24, 2009, Kris wrote: Hi I just tried example for the periodicUpdater but it didn't load properly. Here are the details: Tested on IE 8.0.6001 Webpage error details -------------------------------- User Agent: Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0; GTB6) Timestamp: Fri, 24 Jul 2009 09:28:44 UTC Message: Object doesn't support this property or method Line: 3451 Char: 4 Code: 0 URI: http://localhost/tmp/ajax/jquery/periodicUpdater/jquery-1.3.2.js I tried loading the jquery file from google and locally and got the same error both times.

    $.PeriodicalUpdater({url:"queryMe.html"}, function(newData) { $('body').append(newData); $('body').append(""); $('body').append(new Date().toString()); } );
  

• July 24, 2009, Kris wrote: pls ignore the code at the end.. some of it got filtered by this noticeboad :)
• July 24, 2009, Robert Fischer wrote: Are you using a version that's got implicit variables in a for loop? IE8 balks on that sometimes.
• July 25, 2009, Kris wrote: I guess you are not refering to a version of jquery since I got the same error loading the jquery from ajax.googleapis.com. It can't be the brower since I tried it on Firefox 3.5.1 and chrome 2.0 but I just get blank pages there too. I am using your code from below http://github.com/RobertFischer/JQuery-PeriodicalUpdater/tree/master# I am using periodical updater out of the box too. Using firebug I get this message when i step through s.url.replace is not a function var ret = s.url.replace(/(\?|&)_=.*?(&|$)/, "$1_=" + ts + "$2");\n This is the same line IE8 breaks on also. Others on this noticeboard don't seem to have this problem report this so I am a little stumped. I hardcoded the url in periodicUpdater and I got another error regarding the console so I commented out the console logging and it goes into the update loop but the page is always blank. The url doesn't seem to be picked up. It might just be my environment but I am really not sure what. I'll try it elsewhere. Any ideas what it could be? Do you have a public test page with this code? As you might be able to tell I am new to jquery and haven't touched js in years. Thanks in advance, Cheers Kris
• August 13, 2009, Cry for Help | Enfranchised Mind wrote: [...] jQuery PeriodicalUpdater should be updated to use jQuery.ajax’s dataFilter to track if changes have occurred in the [...]
• September 26, 2009, Updated GORM Labs and JQuery PeriodicalUpdater | Enfranchised Mind wrote: [...] just released an update to GORM Labs and JQuery PeriodicalUpdater (which was introduced in this post). While the modifications to GORM Labs were improvements, the JQuery PeriodicalUpdater update ended [...]
• October 1, 2009, John Richards wrote: You might want to check out StreamHub Comet Server: http://www.stream-hub.com/ It will do all the long-polling/server-push heavy-lifting for you.
• October 1, 2009, Robert Fischer wrote: @John Richards Yeah — unfortunately it's non-costless version is hamstrung to be only useful for development. But if someone wants to spend some money on a solution, there you go.
• October 21, 2009, David wrote: Hi Robert, Trying your poller out to send post data to a php script which in turn returns a json encoded string. I keep getting 304 not modified all the time though, even at the first call. Tried settings headers on the php script (cache control, expires) to force updates to no avail. Any ideas?
• October 21, 2009, Robert Fischer wrote: @David Can you put up a demo page showing the issue?
• October 22, 2009, David wrote: @Robert Let me get back to you as soon as I've got some more time. Went with setInterval for now.
• October 26, 2009, Chris Mattmann wrote: Hi Robert, Great contribution! It didn't work out of the box for me though. I'll put up a patch here: http://sunset.usc.edu/~mattmann/patches/jquery.periodicupdater.ajax.102609.patch.txt What it boiled down to is that my Web Service kept returning status == "notmodified" rather than status == "success", and thus remoteData was never being set. From reading the jquery documentation, it seems that status == "notmodified" is a viable status return (when it's a 302 or 3xx HTTP msg). So, my patch does an OR to check if status == "notmodified", and it also checks ot make sure remoteData != null, and if it is, it uses the value of rawData as the current value (.success never gets called if status != "success", so in that case, remoteData is never set). Hope this helps someone else like me! Cheers, Chris
• October 26, 2009, Robert Fischer wrote: If the status isn't modified, shouldn't we take that to mean that it's not modified and therefore the success route shouldn't fire? Or are people having webservers returning 302 even when it's new data? I'm applying the patch: we'll see if that fixes the issues.
• October 26, 2009, Chris Mattmann wrote: Thanks Bob. I think normally the answer to taking 302 to mean "not modified" would be yes, but I'm seeing in Apache Tomcat 6.0.16 a return status of 302 from a web service even when new content is being generated. Might be a web server issue, but still if we catch this here, it makes the code more robust. Thanks a lot for applying the patch! Cheers, Chris
• November 4, 2009, Andrew wrote: This is excellent! Just what I've been looking for. It worked pretty much perfectly out of the box. The one question/problem I'm having is that I want to use a variable in the data that is updated every time the updater runs. Unfortunately, it keeps sending the original value as the data. The function and every other element on the page recognizes the updated data, but updater doesn't use it. Am I out of luck? Any thoughts or suggestions? Thanks, Andrew
• November 4, 2009, Robert Fischer wrote: How are you attaching the PeriodicalUpdater and the variable?
• November 4, 2009, Andrew wrote:

  $.PeriodicalUpdater('/build/'+la+','+lo, { method: 'get', data: 'ltime='+varLtime, minTimeout: 1000, maxTimeout: 8000, multiplier: 2, type: 'html' }, function(data) { $('#loc+'-'+locid).html(data); varLtime=$('#loc+'-'+locid+' .time:first').text();
  });
  

  So I want to pass varLtime in the data value. varLtime gets its value from an element on the page. I tried changing the varLtime variable in the data section to the actual jQuery selector. Then I also tried having the varLtime variable updated in the function after the data is retrieved. Either way, it doesn't pass the data in. The locid is static, but the varLtime variable changes after each request. Thanks so much for your help! This is a great feature! Andrew
• November 4, 2009, Robert Fischer wrote: @Andrew You've got an extra quote on the first line in your function (the syntax highlighting draws out the problem). The "data" value is evaluated only once, so of course you won't see an updated value. There's no functionality to dynamically calculate the data value, and that's on jQuery: I don't know a way to tell jQuery ot dynamically calculate the "data" value of the $.ajax call.
• November 4, 2009, Robert Fischer wrote: @Andrew Actually, what to do just popped into my head. You should not be able to do this:

  $.PeriodicalUpdater('/build/'+la+','+lo, { method: 'get', data: function() { return 'ltime='+varLtime; }, minTimeout: 1000, maxTimeout: 8000, multiplier: 2, type: 'html' }, function(data) { $('#loc'+'-'+locid).html(data); varLtime=$('#loc+'-'+locid+' .time:first').text();
  });
  

  Let me know if that works for you.
• November 4, 2009, Andrew wrote: Thanks so much for your help. Looks like adding a function for data does some pretty messed up things and breaks the script. (A whole host of errors and didn't seem to return the data at all - even if it was just a text string.) I can get around some of this by changing the supporting services, but I was hoping to be able to do it in javascript. Thanks again for your help! Andrew
• November 5, 2009, Robert Fischer wrote: @Andrew It works for me: http://demo.smokejumperit.com/demo.html Did you update the script to the newest version in GitHub?
• November 5, 2009, Andrew wrote: Just nearly amazing. Thanks!! Looks like it is working pretty well. The only problem is that everytime the updater runs it appends the data to the data. I'm doing this: data: function() { return 'ltime='+varLtime; }, so, the request is: http://blahblah.com/blah.html?ltime=1234567 then the next request is: http://blahblah.com/blah.html?ltime=1234567&ltime=1234569 then the next request is: http://blahblah.com/blah.html?ltime=1234567&ltime=1234569&ltime=1234570 and son on. Do you know of anyway to clear the data? Thanks again for all your assistance. Andrew
• November 5, 2009, Robert Fischer wrote: I'm confused why that is true, and I'm not seeing that behavior on my example. Fire up a debugger (FireBug on FireFox is my favorite) and step through it. If the issue is in my code, let me know and I'll fix it.
• November 5, 2009, Andrew wrote: Hi Robert, Thanks for your response. Yeah, it's pretty strange. I use Firebug - which is how I saw the request going out with the appended and appended and appended data element. When I look at the demo I don't see the data being included in the GET request while mine appends the data to the URL. When I set mine up like your demo (with the return ctr++) it also doesn't include the data in the request and thus doesn't append it to the URL. When I change it to return a string and the value then it gets appended. So, I think if the demo returned a string then it would append it over and over on the URL. Dose that make sense? Again, your help is much appreciated. Andrew
• November 5, 2009, Robert Fischer wrote: @Andrew Oh, this is awesome. The problem is jQuery: it mangles the "url" parameter of the settings you pass in.
• November 5, 2009, Robert Fischer wrote: @Andrew Issue is fixed in the code currently up on GitHub. Actually, a few issues are fixed. But it should be solid now. Check out the demo again: http://demo.smokejumperit.com/demo.html
• November 5, 2009, Andrew wrote: Oh excellent! Works like a charm! Brilliance! : ) Thanks so much! Andrew
• November 6, 2009, New PeriodicalUpdater Feature, and, Mutable Data Sucks: A Case in Point | Enfranchised Mind wrote: [...] jQuery PeriodicalUpdater, which has been getting lots of love because of feedback and support over here, has a new feature. The “data” configuration argument now will work properly with [...]
• November 28, 2009, boncey.org - Machine tags on Flickr wrote: [...] could use to show me the status of my re-tagging action (I had about 1700 photos to tag) and found JQuery PeriodicalUpdater so I wired that up to give me a countdown. The last thing to mention is the photodb:id machine tag [...]
• November 30, 2009, MattA wrote: Hey. I am having some issues with the json calls... When I use .ajax, or the 360innovate version data.length returns the number of objects, but when I use your code it returns the char count and treats it as a string... Any idea what is going wrong? I have been at this for hours!!! Many thanks :)
• November 30, 2009, Robert Fischer wrote: Use FireFox + FireBug and see what's up with the return values. Is it what you expect? If not, what's going on.
• November 30, 2009, MattA wrote: the issue is that its returning as html content, and yes I did clear my cache and such. Also I simply replaced your file for the one from 360innovate, (only changing where I state the url) and it worked... but obviously yours is better so I would rather use that. *I also tried just using getjson() to test the data being returned by my php script and that worked without an issue...
• November 30, 2009, Robert Fischer wrote: Are you specifying type:'json'?
• November 30, 2009, MattA wrote: I sure am...
• November 30, 2009, Robert Fischer wrote: Hrm. Dunno. Don't have time to debug this right now. Source is available (obviously)—let me know if you figure out what's up.
• December 1, 2009, MattA wrote: Ok so after a few hours, I cant figure out why the returned data is not being treated as json... so I created this (very hackish mod to the file)... it works, but thats about all I can say for it... I hope to go back and truly fix this soon... btw for the record, im using safari on mac. http://mattapperson.com/stuff/jquery.periodicalupdater.js
• December 1, 2009, Robert Fischer wrote: There's a standing working demo of using JSON in the vanilla version here: JQuery PeriodicalUpdater demo. Dunno why you're having trouble. What did you hack to get it to work in your code?
• December 12, 2009, J wrote: I use the following code. I wanted something that you could run without a lot of parameters, yet still retain the ability to override them when I need to. I too switched from Prototype to jQuery and needed to I also wanted a way to start and stop the timeout. This code has not been throughly in any sort of production environment. Hope this helps someone looking for a way to control the timeout. :)

  jQuery.fn.extend({ ajaxPeriodicUpdate: function(url, options, callback) { var prevResp = null; var events = {}; var defaults = { url: url, data: {}, type: 'post', cache: false, async: true, dataType: 'html', minFrequency: 2, maxFrequency: 10, multiplier: 2, timerID: null } // Determine if options exist if (options == null) { options = {}; } else { if (jQuery.isFunction(options)) { // Assume that options is the callback callback = options; options = {}; } } var self = this; var settings = jQuery.extend({}, defaults, options); var frequency = settings.minFrequency; var incrementFrequency = function() {return;}; if(settings.multiplier > 1) { incrementFrequency = function() { frequency *= settings.multiplier; if(frequency > settings.maxFrequency) { frequency = settings.maxFrequency; } }; } if (settings.complete) { events.complete = settings.complete; } if (settings.error) { events.error = settings.error; } settings.complete = function(xhr, status) { if (status =='success' || status == 'notmodified') { var respText = xhr.responseText; if (respText == prevResp) { incrementFrequency(); } else { prevResp = respText; frequency = settings.minFrequency; self.html(respText); } } if (events.complete) { events.complete(xhr, status); } if (callback) { self.each(callback, [jQuery.trim(xhr.responseText), status, xhr]); } settings.timerID = setTimeout(update, frequency * 1000); }; settings.error = function(xhr, status) { if(status == "notmodified") { incrementFrequency(); } else { prevResp = null; frequency = settings.minFrequency; } if (options.error) { options.error(xhr, statustatus); } }; function update() { console.log('Updating at frequency of ' + frequency); jQuery.ajax(settings); } jQuery.fn.ajaxPeriodicUpdate.stop = function() { clearTimeout(settings.timerID); settings.timerID = null; prevResp = null; }; jQuery.fn.ajaxPeriodicUpdate.start = function() { if (settings.timerID == null) { jQuery(function() {update();}); } }; jQuery(function() {update();}); return this; }
  });
  

• December 16, 2009, Enfranchised Mind » jQuery PeriodicalUpdater Updates wrote: [...] couple updates to the jQuery PeriodicalUpdater (original post): the $400 update and a yet-more-succinct Updater API (not a huge fan of that name, but it’s [...]

(Digital Fingerprint: bcecb67d74ab248f06f068724220e340 (69.147.112.168) )