• Page
• Discussion
• View source
• History

From Odwiki

DrQueue is a free open source distributed render queue. It supports a wide range of standard render engines - including mantra - but can also be used for general computation tasks. Please see more info on RenderQueues.

Contents 1 Houdini/Mantra Integration 2 The Pipeline 3 Accessing DrQueue from Houdini 4 Archive of Snippets

Houdini/Mantra Integration

Mantra rendering is supported out-of-the box. drqman - a python GUI - allows you to submit IFD sequences to your 'farm'. Drqueue also offers a python binding which is a great if you want to integrate Drqueue into your pipeline.

The Pipeline

While everybody talks about the pipeline it's hard to understand what the pipeline really is. Like in real life the problem often arises at the welding seams between pipe sections. That's why this page currently rather just archives snippets and snatches than a how-to.

Dr Queue for Houdini's Mantra is a page provided by Martian Labs. It covers many aspects of installing, maintaining and controlling DrQueue.

DrQueue default configuration allows *just* 100 jobs. This can be changed inside of constants.h. http://drqueue.org/cwebsite/drqueue_community/comments.php?DiscussionID=469&page=1#Item_1

Accessing DrQueue from Houdini

You need to install the DrQueue Python bindings to be able to talk to DrQueue from inside Houdini. For Houdini to 'see' python modules some environment variables need to be set. Example:

export HOUDINI_PYTHON_LIB_PATH=/usr/lib

export HOUDINI_PYTHON_BIN_PATH=/usr/bin/python2.5

Archive of Snippets

• Toggle Localhost Active State

This command line snippet (de-)activates the drqueue client running on localhost:

#!/usr/bin/env python
import sys
#sys.path.insert(0,'..')
import drqueue.base.libdrqueue as drqueue
import os

def toggle(me):
	if me.limits.enabled:
		me.request_disable(drqueue.CLIENT)
		print "\tComputer: %s\tDisabled !"%(me.hwinfo.name)
	else:
		me.request_enable(drqueue.CLIENT)
	        print "\tComputer: %s\tEnabled !"%(me.hwinfo.name)
	
def main(argv):
	
	
	print "Quering Master: %s\n"%(os.environ["DRQUEUE_MASTER"])
	
	if len(sys.argv) > 1:
		target_computer = sys.argv[1]
	else:
		target_computer = os.getenv("HOSTNAME")

	computer_list = drqueue.request_computer_list (drqueue.CLIENT)	
	if target_computer == "all":
		for computer in computer_list:
			toggle(computer)
	else:
		for computer in computer_list:
			if computer.hwinfo.name == target_computer:
				toggle(computer)
				break
	return 0

if __name__ == "__main__":
	main(sys.argv)

[1] Shelf version of this script for Houdini

• Dependend Job

Sometimes you need to submit dependend jobs - like running a mencoder to create a preview after rendering a range of frames.

#!/usr/bin/env python
import sys
import drqueue.base.libdrqueue as drqueue

# In this example job 1 waits for 10 seconds and then creates a file. Job 2 copies the file.
# Job 2 can only be executed if job 1 has been finished.
 
def main(argv):
	print "test dependend jobs"
	job1 = drqueue.job()
	job2 = drqueue.job()
	job1.name = "first"
	job2.name = "second"
	job1.cmd = "sleep 10\ntouch /home/rdg/testout/done_job1"
	job2.cmd = "cp /home/rdg/testout/done_job1 /home/rdg/testout/done_job2"
	job1.frame_start = 1
	job1.frame_end = 1
	job2.frame_start = 1
	job2.frame_end = 1
	job1.send_to_queue()
	job2.flags = job2.flags | drqueue.JF_JOBDEPEND
	job2.dependid = job1.id
	job2.send_to_queue()

if __name__ == "__main__":
	main(sys.argv)

• Manage Pools

DrqQueue allows you to assign computers to pools. This snippet adds or removes the local computer from the pool specified:

#!/usr/bin/env python
import sys
import drqueue.base.libdrqueue as drqueue
import os
	
def getThisComputer(computer_list):
	this_hostname = os.getenv("HOSTNAME")
	this_id = -1
	computer_list = drqueue.request_computer_list (drqueue.CLIENT)
	for computer in computer_list:
		if computer.hwinfo.name == this_hostname:
			this_id = computer.hwinfo.id
			break
	return computer
		
def main(argv):
	
	print "Quering Master: %s\n"%(os.environ["DRQUEUE_MASTER"])
	
	if len(sys.argv) != 3 or sys.argv[1] not in ("-a", "-r"):
		print "Usage:\n\tdrq_pool -a pool to add to pool\n\tdrq_pool -r pool to remove from pool"
	else:
		computer_list = drqueue.request_computer_list (drqueue.CLIENT)	
		me = getThisComputer(computer_list)
		error = 1
		if sys.argv[1] == "-a":
			if drqueue.request_slave_limits_pool_add(me.hwinfo.name, sys.argv[2], drqueue.CLIENT):
				print "Successfully added '%s' to pool '%s'." % (me.hwinfo.name, sys.argv[2])
				error = -1
		else:
			if drqueue.request_slave_limits_pool_remove(me.hwinfo.name, sys.argv[2], drqueue.CLIENT):
				print "Successfully removed '%s' from pool '%s'." % (me.hwinfo.name, sys.argv[2])
				error = -1
		if error == 1:
			print "Something went wrong. Sorry."
	return 0
 	
if __name__ == "__main__":
	main(sys.argv)

• List existing pools.

Drqueue doesn't keep a global list of existing pools (?). This snippet iterates through all connected slaves and collects the assigned pools.

#!/usr/bin/env python
import sys
#sys.path.insert(0,'..')
import drqueue.base.libdrqueue as drqueue
import os
	
def main(argv):
	
	print "Quering Master: %s\n"%(os.environ["DRQUEUE_MASTER"])
	
	pool_list = []
	
	computer_list = drqueue.request_computer_list (drqueue.CLIENT)
	for computer in computer_list:
		local_list = computer.list_pools()
		for i in local_list:
			if i.name not in pool_list:
				pool_list.append(i.name)
	
	print "Existing Pools:"
	for i in pool_list:
		print "\t%s" % i 

	return 0
	 

if __name__ == "__main__":
	main(sys.argv)