Removing temp files in working directory

Post Reply
jhowe
Posts: 6
Joined: Tue May 23, 2017 1:02 pm

Removing temp files in working directory

Post by jhowe » Tue May 23, 2017 1:14 pm

I have CGRU 2.2.1 set up and running on Windows 10 and things are working well overall. The issue I just ran into is that the temporary .exr files that are being created on all of the nodes running render are not being deleted, as one of the nodes had its hard drive almost completely full. (Thank you for including HDD space in the Watch display.)

Is there a way to configure afrender to not write temp files on the render nodes? I am submitting jobs with files located on a shared network folder and checked "Use scene folder", assuming temp files would be written there, but they are still being created locally on the render nodes at C:/temp.

Would it be possible to skip saving those temp files entirely? Or deleting them when a specific task is complete?

User avatar
timurhai
Site Admin
Posts: 337
Joined: Sun Jan 15, 2017 8:40 pm
Location: Russia, Korolev
Contact:

Re: Removing temp files in working directory

Post by timurhai » Tue May 23, 2017 4:28 pm

Hi.
Temp folder is also used by afrender to store temp task data, thumbnail for example.
It should not be a network folder, c:\temp is a good default place (configurable).
After task finish (with any result) afrender deletes this folder.
Try to see afrender log may it can`t delete it and print some error.

If the final image appears in a temp folder, lets talk about submission script.
What soft and what submission script do you use?
Timur Hairulin
CGRU 2.2.3 Ubuntu 14.04LTS (gcc4.8.4)

jhowe
Posts: 6
Joined: Tue May 23, 2017 1:02 pm

Re: Removing temp files in working directory

Post by jhowe » Tue May 23, 2017 6:18 pm

I submit jobs from the "Submit Job..." tool in Keeper, which gives me this

Code: Select all

"python" "C:\cgru-windows\cgru\afanasy\python\afjob.py" "\\SERVER8\Maps\CgruWorkingDirectory\Aff-Ext-1080-FQ.blend" 5000 6500 -by 1 -fpt 1 -seq 1 -os any -pwd "C:\temp" -priority 90 -name "Aff-Ext-1080-FQ.blend"
I changed the folder permissions for C:/temp to allow full control. The (very large) EXR files still don't seem to be deleted once a given task is complete. I also didn't see anything in the task logs that would hint at failure to delete a file.

Looking at the files in more detail, I think I see what is happening. I am using a File Output node in Blender to write out my composite Multi-Layer EXR back to a networked folder, and it looks like CGRU's Blender task is taking the default render image format, also an EXR, and saving it in Temp. I had the output path in Blender's render panel left blank, as I just care about the File Output node file. Maybe the default image is being generated regardless and being stored? At a minimum, I should be able to cut down on the C:\Temp folder file sizes just by setting the default format to JPEG.

jhowe
Posts: 6
Joined: Tue May 23, 2017 1:02 pm

Re: Removing temp files in working directory

Post by jhowe » Wed May 24, 2017 12:51 am

Confirmed that the empty render destination in Blender was creating the files in C:/temp. By setting the format to JPEG it added JPEG images to the same location. I set the default render location in Blender to a common network folder allows me to keep this from filling up the render nodes' disks. I wonder if there is a way to use these JPEGs as the thumbnails in Watch...

User avatar
timurhai
Site Admin
Posts: 337
Joined: Sun Jan 15, 2017 8:40 pm
Location: Russia, Korolev
Contact:

Re: Removing temp files in working directory

Post by timurhai » Wed May 24, 2017 8:43 am

Hi.
Afrender should delete temporary tasks files after task finish.
An output image is not an afrender temporary file.
If command place an output image in a bad location it is a command problem.
Afanasy (afrender) just launch commands.
You can run task command manually by copy-paste it in a console.
If you can fix command to make Blender to place images in a correct place, you can fix submission to generate a correct command.
I am not familiar with Blender, you can more Blender-experienced people to find out what is wrong with the command (how to make Blender to render with a command line more correctly).
Timur Hairulin
CGRU 2.2.3 Ubuntu 14.04LTS (gcc4.8.4)

jhowe
Posts: 6
Joined: Tue May 23, 2017 1:02 pm

Re: Removing temp files in working directory

Post by jhowe » Wed May 24, 2017 12:06 pm

Thank you for the reply. With the changes I made, things are working great. It was a Blender setup issue, not a CGRU problem. CGRU is deleting files properly.

With the changes I made to Blender, using separate network locations for both the default render settings and the File Output node in the compositor, I am saving JPEG preview images and final OpenEXR files out to different folders on the server with no files being left behind on the render nodes. With this setup, CGRU is working much better than my old Backburner setup. Thanks Timur!

Post Reply

Who is online

Users browsing this forum: No registered users and 1 guest