Previously job management was handled by a simple python based queue module. To add a frame for rendering a command would but ‘put’ into the queue and get() would be run when a new command is required for rendering. However, by running get() the command is removed from the queue and is no longer as an object to PyFarm. For certain items such as a the current state of a frame information must be retained for the duration of the program’s execution, without that information PyFarm would simply pass commands along without informing the user of their progress. By replacing the queue with a nested dictionary large amounts of data can be stored, modified, and queried quickly and easily with very little code required. With this new system tasks such as saving out the current que to formats such as XML (which also observes the parent -> child relationship) will allow the user to backup the current job information for later use. See the example job dictionary below for an example.
Example Job Dictionary:
jobs = { "Job1" : {
"status" : "Rendering",
"frames" : {
"complete" : 1,
"total" : 3,
"failed" : 1
},
"subjobs" : {
"sub1" : {
"status" : "Rendering",
"complete" : 1,
"total" : 3,
"failed" : 1,
"frames" : {
1 : {"status" : "Failed", "host" : "render01", "pid" : 9978, "command" : "render -r mr -v 5 -s 1 -e 1 scene.mb",
"stdout" : "This is the first line in the log\nThis is the second"},
2 : {"status" : "Rendering", "host" : "render02", "pid" : 8243, "command" : "render -r mr -v 5 -s 2 -e 2 scene.mb",
"stdout" : "This is the first line in the log\nThis is the second"},
3 : {"status" : "Complete", "host" : "render03", "pid" : 10124, "command" : "render -r mr -v 5 -s 3 -e 3 scene.mb",
"stdout" : "This is the first line in the log\nThis is the second"},
}
}
}
}
}
frames = jobs["Job1"]["subjobs"]["sub1"]["frames"]
for frame in frames.keys():
thisFrame = frames[frame]
print "Frame %i:\n\tStatus: %s\n\tHost: %s\n\tPID: %i\n\tCommand: %s\n\tLog: %s" % \
(frame, thisFrame["status"], thisFrame["host"], thisFrame["pid"], thisFrame["command"], thisFrame["stdout"].split('\n'))
Output:
Frame 1:
Status: Failed
Host: render01
PID: 9978
Command: render -r mr -v 5 -s 1 -e 1 scene.mb
Log: ['This is the first line in the log', 'This is the second']
Frame 2:
Status: Rendering
Host: render02
PID: 8243
Command: render -r mr -v 5 -s 2 -e 2 scene.mb
Log: ['This is the first line in the log', 'This is the second']
Frame 3:
Status: Complete
Host: render03
PID: 10124
Command: render -r mr -v 5 -s 3 -e 3 scene.mb
Log: ['This is the first line in the log', 'This is the second']
