Thursday, September 27, 2007

DOM.ensure a DOMReady approach


DOM.ensure( function(){
$("node").observe("click",method1);
});


If we decide that we want to cache that markup and reinsert it back into the DOM later we're still safe. If we had used the DOMReady approach our code would have needed to be much more complex to track when it's ok to call the method and when it needs to be called in the event. This method is really away of letting us say we want to ensure that this block of code is not executed until the DOM is really ready.



Here's a solution I created for implementing DOM.ensure.



DOM = Class.create();
Object.extend(DOM,{
callQueue: $A([]), // store all calls in this queue
isReady: false,
// make sure the DOM may be effected before calling this method
// if the DOM is ready then the method is called right away, else a timeout is set
// until the DOM is ready
ensureDOMIsReady: function(callback)
{
if( DOM.isReady ){
callback(); // call the method now
}
else{ // delay the call until the DOM is ready
DOM.callQueue.push( callback );
}
},
initialize: function() // call any methods that were queued up to be run when the DOM is ready
{
DOM.isReady = true;
var queue = DOM.callQueue;
while( queue.length > 0 ){
queue.pop()();
}
}
});

// detect DOM ready once for the whole page for each specific browser type
if( Prototype.Browser.IE ){
document.onreadystatechange = function(){
if( document.readyState == "complete" ){
DOM.initialize();
}
}
}
else if( Prototype.Browser.WebKit ){// safari 2.0
var _timer = setInterval(function() {
if (/loaded|complete/.test(document.readyState)) {
clearInterval(_timer);
DOM.initialize();
}
}, 10);
}
else if( document.addEventListener ){
document.addEventListener("DOMContentLoaded", DOM.initialize.bind(this), false);
}



Really it's a change in the calling semantics for a DOMReady to think of it as more of a question of whether or not the DOM is ready for me to manipulate it.

Tuesday, September 25, 2007

Scalable File Uploads with Merb


For a new application I'm working on we need to upload some large files to our web application. Also because the files are both large and in processing them there are many sql insert and updates that follow the whole process is pretty intensive taking roughly 1 to 2 minutes a file. I've been reading about merb for a few months and off and on created small projects with it hoping to one day put it to use. Tonight I was pleased that it really does make file uploading easy and appears fast. Here's a quick overview of what I did to get it setup.



sudo gem install merb -y

Create the basic merb project



merb -g uploader

Setup the upload controller and view



cd uploader/dist
mkdir app/views/uploader
touch app/controllers/uploader.rb
touch app/views/uploader/index.rhtml

We'll have two actions defined in the Uploader controller:


  • index, to display the form

  • upload, to handle the file upload post


class Uploader < Application
# http://www.cs.tut.fi/~jkorpela/forms/file.html
def index
render
end

def upload
puts params[:file].inspect
FileUtils.mv params[:file][:tempfile].path, MERB_ROOT+"/uploads/#{params[:file][:filename]}"
redirect "/uploader"
end
end

Define the form view



<p>Upload a new file</p>
<form action="/uploader/upload" method="post" enctype="multipart/form-data">
<fieldset>
<input type="file" name="file" size="80"/>
<input type="submit" value="Upload"/>
</fieldset>
</form>

Finally, start your merb application up (from the uploader folder not instead the dist directory).



merb


You might need to disable sql_session in your dist/conf/merb.yml


change:

:sql_session: true

to:

:sql_session: false

The server is running on port 4000 and the request path is /uploader e.g. http://localhost:4000/uploader


Next I'd like to work more tightly integrating this with my rails application. A few things I want to focus on:



  • the traffic is routed in my nginx setup.

  • whether or not the merb process should do the long processing needed after the file is uploaded.

  • how to get the results of the processed xml files back into my database for my rails app to read from.

  • how to indicate to the users that the file has been uploaded but may still be being processed.


Here's a pretty simple Architecture I drew up using Graffle



Merb will handle reading files from the client and writing them to disk. Naming them with an identifiable name and sending the redirect back to our rails application or possibly a status page.


Next I'm thinking another process will monitor the folder merb copies files into. When new files appear pick them up, process them, and parse them back into our sql database. To determine when the filesystem has changed, I'm thinking inotify looks like the best approach and conveniently it already has a ruby binding so I can do my CRUD operations with ActiveRecord. At least this is my initial thinking as to how I can handle file uploads. Any suggestions or feedback would be great!

Sunday, September 16, 2007

Improving test performance by rendering less


I have for sometime had a feeling that functional and integration tests could be speed up even more by disabling the rendering of the layout within each test.



Here's some sample code I used to remove layouts from all my integration and functional tests.


class ApplicationController

# remove layout to improve test time

def render_with_layout_disabled(*

args
)

args << {:layout => false} # force layout to always be false

render_without_layout_disabled(*args)

end

alias_method_chain :render, :layout_disabled

end


Tonight I've found that I get about 188% increase in integration test time and about 248% increase in functional test time by removing the layout.



Disabling the layout was pretty easy using alias_method_chain. My next question is does doing this break anything or make the test less meaningful? My thinking is no. Tests should be small in what they test. We can have a test to specifically test the layout of our application. Repeatedly testing the layout is redundant. So, making this change to my application for my tests seems like a good move.

Saturday, September 15, 2007

Ruby win32 and the system call

require 'rubygems'
require 'win32/process'

# win32 system call that returns correct status
def system(cmd)

pid = fork
if !pid
args = cmd.split(' ')
args.first << ".exe" unless args.first.match(/\..+$/)
cmd = args.join(' ')
exec cmd
end

ret = Process.waitpid2(pid)

(ret.last == 0)
end



Update:


If you try using this in a rake task you'll notice an error similar to this:

c:/ruby/bin/rake.bat:1: syntax error, unexpected tIDENTIFIER, expecting $end

To fix this I patched win32/process.rb fork to gsub out rake.bat with rake, since this is a pretty specific case within win32 ruby.

Here's the line.

cmd.gsub!(/rake.bat/,'rake')

rv = CreateProcess(0, cmd, 0, 0, 1, 0, 0, 0, startinfo, procinfo)

Just before the call to CreateProcess line 565 for win32-process 0.5.3

Update:


It looks like the above is the result of how the one click installer uses .bat files. Upgrading Rake to 0.7.3 and the patch is no longer needed in win32-process.

Reading list