Windows to Linux paths errors

Mikas
Posts: 38
Joined: Thu Oct 10, 2019 8:52 am

Windows to Linux paths errors

Post by Mikas »

Hi,

I'm setting up a linux renderfarm. Jobs are being sent from Windows workstations. I setup path conversions, it shows properly in logs:

{
...
"working_directory": "/mnt/LUA_BlockDevice1/PRJ/RND/18008_AzureTests",
"files_block": [
"/mnt/LUA_BlockDevice1/PRJ/RND/18008_AzureTests/SHOTS/test/IMG/RENDERS/LIGHT_CHAR_PIG_V003/LIGHT_CHAR_PIG_acescg_V003_@####@.exr"
],
"parsed_files": [
"/mnt/LUA_BlockDevice1/PRJ/RND/18008_AzureTests/SHOTS/test/IMG/RENDERS/LIGHT_CHAR_PIG_V003/LIGHT_CHAR_PIG_acescg_V003_0001.exr"
]
}

But instead of rendering to the correct location on the Linux server it renders into correct project folder where it creates windows drive/folder structure, in my case N: drive and so on, so ifd's are not available to mantra obviously.
Any suggestions how to debug this behavior?
Mikas
Renderfarm - Ubuntu 20.04 LTS
Clients - mixed Windows and Linux
Afanasy 3.1.1
User avatar
timurhai
Site Admin
Posts: 911
Joined: Sun Jan 15, 2017 8:40 pm
Location: Russia, Korolev
Contact:

Re: Windows to Linux paths errors

Post by timurhai »

Hi.

Who and when "creates windows drive/folder structure"?
Timur Hairulin
CGRU 3.3.1, Ubuntu 20.04, 22.04, MS Windows 10 (clients only).
Mikas
Posts: 38
Joined: Thu Oct 10, 2019 8:52 am

Re: Windows to Linux paths errors

Post by Mikas »

hbatch on linux machine, though I cannot see that in Afanasy log.
Mikas
Renderfarm - Ubuntu 20.04 LTS
Clients - mixed Windows and Linux
Afanasy 3.1.1
Mikas
Posts: 38
Joined: Thu Oct 10, 2019 8:52 am

Re: Windows to Linux paths errors

Post by Mikas »

Basically I'm using $JOB for all my paths inside of HIP files. In windows it's N: drive, in Linux it's /mnt/LUA_BLockDevice1

I added windows to linux path conversion in config files, I even hardcoded $JOB variable on linux render node just to be sure. In Houdini and Mantra logs in Afanasy it shows correct Linux paths, but when renders it looks like all windows paths are added as absolute paths instead of reletive to $JOB variable.
Mikas
Renderfarm - Ubuntu 20.04 LTS
Clients - mixed Windows and Linux
Afanasy 3.1.1
Mikas
Posts: 38
Joined: Thu Oct 10, 2019 8:52 am

Re: Windows to Linux paths errors

Post by Mikas »

Even if ifds are being generated on Linux machine, not on windows machine.
Mikas
Renderfarm - Ubuntu 20.04 LTS
Clients - mixed Windows and Linux
Afanasy 3.1.1
Mikas
Posts: 38
Joined: Thu Oct 10, 2019 8:52 am

Re: Windows to Linux paths errors

Post by Mikas »

When I manually moved ifds to correct location, I checked the mantra log and it looks for windows paths as well, meaning IFDs are already generated with absolute windows paths somehow, unless I'm interpreting it wrong:

Generating Image: N:/PRJ/RND/18008_AzureTests/SHOTS/test/IMG/RENDERS/LIGHT_CHAR_PIG_V003/LIGHT_CHAR_PIG_acescg_V003_0001.exr

instead of /mnt/LUA_BlockDevice1/...
Mikas
Renderfarm - Ubuntu 20.04 LTS
Clients - mixed Windows and Linux
Afanasy 3.1.1
User avatar
timurhai
Site Admin
Posts: 911
Joined: Sun Jan 15, 2017 8:40 pm
Location: Russia, Korolev
Contact:

Re: Windows to Linux paths errors

Post by timurhai »

Better to use $HIP variable, or (and even better) your custom, that you control manually.

ps
We are not using $JOB variable at all, it seems that $JOB behavior is differ on Linux&Windows.
Timur Hairulin
CGRU 3.3.1, Ubuntu 20.04, 22.04, MS Windows 10 (clients only).
Mikas
Posts: 38
Joined: Thu Oct 10, 2019 8:52 am

Re: Windows to Linux paths errors

Post by Mikas »

This is raw object from afanasy ifd generation job:

Raw Object:
{
"name": "frame 1",
"service": "hbatch",
"capacity": 1000,
"time_start": 1570802256,
"job_id": 2,
"block_num": 1,
"task_num": 0,
"command_block": "hrender_af -s @#@ -e @#@ --by 1 -t \"_current_\" \"/mnt/LUA_BlockDevice1/PRJ/RND/18008_AzureTests/SHOTS/test/LIGHT/pig_oracle_test_open_shader_v008-afanasy1-_current_-r1280.1011-162713-276.hip\" \"/out/PIG\"",
"command_task": "",
"frame_start": 1,
"frame_finish": 1,
"frames_inc": 1,
"frames_num": 1,
"parser": "hbatch",
"working_directory": "/mnt/LUA_BlockDevice1/PRJ/RND/18008_AzureTests",
"files_block": [
"/mnt/LUA_BlockDevice1/PRJ/RND/18008_AzureTests/SHOTS/test/IMG/RENDERS/LIGHT_CHAR_PIG_V003/LIGHT_CHAR_PIG_acescg_V003_@####@.exr"
],
"parsed_files": [
"/mnt/LUA_BlockDevice1/PRJ/RND/18008_AzureTests/SHOTS/test/IMG/RENDERS/LIGHT_CHAR_PIG_V003/LIGHT_CHAR_PIG_acescg_V003_0001.exr"
]
}

So parsed_files points to the correct location
Mikas
Renderfarm - Ubuntu 20.04 LTS
Clients - mixed Windows and Linux
Afanasy 3.1.1
User avatar
timurhai
Site Admin
Posts: 911
Joined: Sun Jan 15, 2017 8:40 pm
Location: Russia, Korolev
Contact:

Re: Windows to Linux paths errors

Post by timurhai »

So may be it is just Houdini scene question, not CGRU Paths Mapping.
Try not to use $JOB (as i wrote before).
Timur Hairulin
CGRU 3.3.1, Ubuntu 20.04, 22.04, MS Windows 10 (clients only).
Mikas
Posts: 38
Joined: Thu Oct 10, 2019 8:52 am

Re: Windows to Linux paths errors

Post by Mikas »

OK, thanks for the suggestion. I will try to use custom one, I do have it anyway for different purposes. Thanks for your help.
Mikas
Renderfarm - Ubuntu 20.04 LTS
Clients - mixed Windows and Linux
Afanasy 3.1.1
Post Reply