Autotest w/ ScriptingContainers
November 18, 2010 @ 05:37 PMIn my last blog post, I demonstrated a couple ways to improve JRuby startup performance by remove various startup steps (Bundler). In this post, I'll show you how to reduce your waiting time when running Autotest/RSpec in JRuby by instantiating ScriptingContainers and loading library code so it's ready to go when needed.
There are a couple other libraries that do similar things (Spork, for example), but I have been unable to get any of them to work [well].
A warning: whereas my last post showed The Way It Should Be (according to me), this post is a quick-n-dirty hack that just happens to work for me. Benchmarks are far from scientific, but they shouldn't surprise anyone either.
The Project
Same little application as the last post. There's only one actual spec, the rest are Pending. We're focusing more on the time it takes to start executing the specs, not finish them.
For both runs I'm including the initial execution of all specs, plus three individual executions triggered by modified spec files.
Autotest / rspec_rails
First, running an almost-standard autotest setup. The only thing different here is I hacked in the "time " prefix.
Initial run of all teststime bundle exec /Users/brasten/.rvm/rubies/jruby-1.5.5/bin/jruby -rrubygems -S /Users/brasten/.rvm/gems/jruby-1.5.5/gems/rspec-core-2.1.0/bin/rspec --autotest '/Users/brasten/Development/Projects/Scratch/warehouse/spec/controllers/customers_controller_spec.rb' '/Users/brasten/Development/Projects/Scratch/warehouse/spec/controllers/inventory_items_controller_spec.rb' '/Users/brasten/Development/Projects/Scratch/warehouse/spec/helpers/customers_helper_spec.rb' '/Users/brasten/Development/Projects/Scratch/warehouse/spec/helpers/inventory_items_helper_spec.rb' '/Users/brasten/Development/Projects/Scratch/warehouse/spec/models/inventory_item_spec.rb' '/Users/brasten/Development/Projects/Scratch/warehouse/spec/requests/customers_spec.rb' *****. Pending: CustomersController add some examples to (or delete) /Users/brasten/Development/Projects/Scratch/warehouse/spec/controllers/customers_controller_spec.rb # Not Yet Implemented # ./spec/controllers/customers_controller_spec.rb:4 InventoryItemsController add some examples to (or delete) /Users/brasten/Development/Projects/Scratch/warehouse/spec/controllers/inventory_items_controller_spec.rb # Not Yet Implemented # ./spec/controllers/inventory_items_controller_spec.rb:4 CustomersHelper add some examples to (or delete) /Users/brasten/Development/Projects/Scratch/warehouse/spec/helpers/customers_helper_spec.rb # Not Yet Implemented # ./spec/helpers/customers_helper_spec.rb:14 InventoryItemsHelper add some examples to (or delete) /Users/brasten/Development/Projects/Scratch/warehouse/spec/helpers/inventory_items_helper_spec.rb # Not Yet Implemented # ./spec/helpers/inventory_items_helper_spec.rb:14 InventoryItem add some examples to (or delete) /Users/brasten/Development/Projects/Scratch/warehouse/spec/models/inventory_item_spec.rb # Not Yet Implemented # ./spec/models/inventory_item_spec.rb:4 Finished in 0.208 seconds 6 examples, 0 failures, 5 pending real 0m15.324s user 0m21.998s sys 0m1.223sRunning modified specstime bundle exec /Users/brasten/.rvm/rubies/jruby-1.5.5/bin/jruby -rrubygems -S /Users/brasten/.rvm/gems/jruby-1.5.5/gems/rspec-core-2.1.0/bin/rspec --autotest '/Users/brasten/Development/Projects/Scratch/warehouse/spec/requests/customers_spec.rb' . Finished in 0.164 seconds 1 example, 0 failures real 0m15.428s user 0m22.083s sys 0m1.258s time bundle exec /Users/brasten/.rvm/rubies/jruby-1.5.5/bin/jruby -rrubygems -S /Users/brasten/.rvm/gems/jruby-1.5.5/gems/rspec-core-2.1.0/bin/rspec --autotest '/Users/brasten/Development/Projects/Scratch/warehouse/spec/models/inventory_item_spec.rb' * Pending: InventoryItem add some examples to (or delete) /Users/brasten/Development/Projects/Scratch/warehouse/spec/models/inventory_item_spec.rb # Not Yet Implemented # ./spec/models/inventory_item_spec.rb:4 Finished in 0.002 seconds 1 example, 0 failures, 1 pending real 0m15.415s user 0m22.214s sys 0m1.231s time bundle exec /Users/brasten/.rvm/rubies/jruby-1.5.5/bin/jruby -rrubygems -S /Users/brasten/.rvm/gems/jruby-1.5.5/gems/rspec-core-2.1.0/bin/rspec --autotest '/Users/brasten/Development/Projects/Scratch/warehouse/spec/controllers/customers_controller_spec.rb' * Pending: CustomersController add some examples to (or delete) /Users/brasten/Development/Projects/Scratch/warehouse/spec/controllers/customers_controller_spec.rb # Not Yet Implemented # ./spec/controllers/customers_controller_spec.rb:4 Finished in 0.001 seconds 1 example, 0 failures, 1 pending real 0m15.908s user 0m22.447s sys 0m1.233s
Autotest w/ ScriptingContainers
To get this running, make the following changes:
-
Modify (or create) the autotest/discover.rb file in your project so that it looks like the following:
Autotest.add_discovery { "jruby" } Autotest.add_discovery { "rails" } Autotest.add_discovery { "rspec2" } -
Next, create autotest/jruby_rails_rspec2.rb with the following contents:
require 'java' require 'benchmark' require 'autotest/rails_rspec2' require 'pathname' # Sends RSpec to stdout and provided buffer # class ResultsCollector class << self # ... inexplicable syntax preference def into(buffer) self.new(buffer) end end def initialize(resultsBuffer) @buffer = resultsBuffer end def puts(line="") Kernel.puts(line) @buffer << "#{line}\n" end def print(line="") Kernel.print(line) @buffer << line end end class Autotest::JrubyRailsRspec2 < Autotest::RailsRspec2 java_import 'org.jruby.embed.ScriptingContainer' java_import 'org.jruby.embed.LocalContextScope' java_import 'org.jruby.embed.LocalVariableBehavior' def initialize super setup_scripting_container end # Setup a ScriptingContainer so it's ready to go # the next time we need it # def setup_scripting_container trap('INT') do exit!(1) end tms = Benchmark.measure { puts "setting up scripting container..." @runtime = ScriptingContainer.new(LocalContextScope::SINGLETHREAD, LocalVariableBehavior::PERSISTENT) # ... requiring spec_helper here offloads even more startup time from our testing cycle, but for # some projects (maybe most?) this may not work. If spec_helper does anything that loads your # application files, then you should comment out that line. # # This should be a config option somewhere probably. # @runtime.runScriptlet(<<-SCRIPT) $: << 'spec' require 'config/application' RSpec::Core::Runner.disable_autorun! SCRIPT } puts "scripting container set up in #{format('%.3f', tms.real)} seconds" puts end def run_tests hook :run_command new_mtime = self.find_files_to_test return unless new_mtime self.last_mtime = new_mtime files_to_test = self.files_to_test file_list = normalize(files_to_test).keys.flatten.join(' ') return if file_list.empty? file_display = normalize(files_to_test).keys.flatten.map {|f| " -> " + Pathname.new(f).relative_path_from(Pathname.new(File.expand_path('../..', __FILE__))) }.join("\n") puts "Running:" puts file_display self.results = "" @runtime.put("outputStream", ResultsCollector.into(self.results)) tms = Benchmark.measure { # ... we should probably respect config settings here... eventually. # @runtime.runScriptlet(<<-SCRIPT) specs = %w(--color --autotest #{file_list}) RSpec::Core::Runner.run(specs, outputStream, outputStream) SCRIPT } # ... ping any success/failure hooks first hook :ran_command handle_results(self.results) puts "tests completed in #{format('%.3f', tms.real)} seconds" puts puts # ... then clean up @runtime.terminate setup_scripting_container end end
And you're done. To the specs!
Initial run of all testssetting up scripting container... scripting container set up in 7.047 seconds Running: -> spec/controllers/customers_controller_spec.rb -> spec/controllers/inventory_items_controller_spec.rb -> spec/helpers/customers_helper_spec.rb -> spec/helpers/inventory_items_helper_spec.rb -> spec/models/inventory_item_spec.rb -> spec/requests/customers_spec.rb *****. Pending: CustomersController add some examples to (or delete) /Users/brasten/Development/Projects/Scratch/warehouse/spec/controllers/customers_controller_spec.rb # Not Yet Implemented # ./spec/controllers/customers_controller_spec.rb:4 InventoryItemsController add some examples to (or delete) /Users/brasten/Development/Projects/Scratch/warehouse/spec/controllers/inventory_items_controller_spec.rb # Not Yet Implemented # ./spec/controllers/inventory_items_controller_spec.rb:4 CustomersHelper add some examples to (or delete) /Users/brasten/Development/Projects/Scratch/warehouse/spec/helpers/customers_helper_spec.rb # Not Yet Implemented # ./spec/helpers/customers_helper_spec.rb:14 InventoryItemsHelper add some examples to (or delete) /Users/brasten/Development/Projects/Scratch/warehouse/spec/helpers/inventory_items_helper_spec.rb # Not Yet Implemented # ./spec/helpers/inventory_items_helper_spec.rb:14 InventoryItem add some examples to (or delete) /Users/brasten/Development/Projects/Scratch/warehouse/spec/models/inventory_item_spec.rb # Not Yet Implemented # ./spec/models/inventory_item_spec.rb:4 Finished in 0.29 seconds 6 examples, 0 failures, 5 pending tests completed in 2.624 seconds setting up scripting container... scripting container set up in 5.168 secondsRunning modified specsRunning: -> spec/requests/customers_spec.rb . Finished in 0.156 seconds 1 example, 0 failures tests completed in 1.952 seconds setting up scripting container... scripting container set up in 4.635 seconds Running: -> spec/models/inventory_item_spec.rb * Pending: InventoryItem add some examples to (or delete) /Users/brasten/Development/Projects/Scratch/warehouse/spec/models/inventory_item_spec.rb # Not Yet Implemented # ./spec/models/inventory_item_spec.rb:4 Finished in 0.001 seconds 1 example, 0 failures, 1 pending tests completed in 1.785 seconds setting up scripting container... scripting container set up in 4.441 seconds Running: -> spec/controllers/customers_controller_spec.rb * Pending: CustomersController add some examples to (or delete) /Users/brasten/Development/Projects/Scratch/warehouse/spec/controllers/customers_controller_spec.rb # Not Yet Implemented # ./spec/controllers/customers_controller_spec.rb:4 Finished in 0.001 seconds 1 example, 0 failures, 1 pending tests completed in 1.825 seconds setting up scripting container... scripting container set up in 4.405 seconds
The Results
It's a little tough to see exactly what's happening here just from the text. Autotest is ensuring that a fresh ScriptingContainer is set up and ready to go shortly after a test run is completed. The initial run isn't going to see any significant improvements, of course, but executions triggered by detecting file modifications already have an environment set up to run against.
Anyway, it's been a long day so I'm sure I did not do a fantastic job explaining this, but give it a shot and see for yourself.
Update: This seems to leak memory. I'll be filing a JRuby bug momentarily. In the meantime you may need to restart Autotest every once in a while. Like I said at the beginning, hack.