rendering with different operating systems

General discussions and questions.
Post Reply
dasrow
Posts: 10
Joined: Tue Jun 29, 2021 12:26 pm

rendering with different operating systems

Post by dasrow »

Hello, is there a chance to render the same project with different operating systems?
I have a farm using samba or home network. I want to render when the pc is idle. However, windows cannot render because the server path of linux mac devices is different. For example, if I send renders on linux, only linux machines can receive it. After setting the first find part of this folder it's all the same. Can't we fix this? There was a unix part in the backburner that provides the setting for it. No matter how much I looked in CGRU, I couldn't find it.
User avatar
timurhai
Site Admin
Posts: 911
Joined: Sun Jan 15, 2017 8:40 pm
Location: Russia, Korolev
Contact:

Re: rendering with different operating systems

Post by timurhai »

Timur Hairulin
CGRU 3.3.1, Ubuntu 20.04, 22.04, MS Windows 10 (clients only).
dasrow
Posts: 10
Joined: Tue Jun 29, 2021 12:26 pm

Re: rendering with different operating systems

Post by dasrow »

I couldn't solve this problem. CGRU root location gives an error due to a different address on windows.
Image
User avatar
timurhai
Site Admin
Posts: 911
Joined: Sun Jan 15, 2017 8:40 pm
Location: Russia, Korolev
Contact:

Re: rendering with different operating systems

Post by timurhai »

Hi!
I see a warning in logs, that a task working directory does not exist.
What was a working directory on a machine where job was send?
What is you paths map?

It seems that you are using cgru root folder as a working directory.
May be you are not using it at all?
Or you should have a rules for it on paths map.
Timur Hairulin
CGRU 3.3.1, Ubuntu 20.04, 22.04, MS Windows 10 (clients only).
dasrow
Posts: 10
Joined: Tue Jun 29, 2021 12:26 pm

Re: rendering with different operating systems

Post by dasrow »

Let me show you like this. I use it for rendering through Blender.I have PCs with windows and linux operating systems.I am running cgru using the path in the image below.I am running on server.
Image

Setting 2 or 3 configs sounded really confusing.


config_default in afanasy

Code: Select all

{"cgru_config":{

	"af_servername":"127.0.0.1",
		"":"Afanasy server name or direct IP address literals",

	"af_serverport":51000,

		"":"Afanasy server will use IPv6 if it is configured on server interface",
	"af_ipv6_disable":1,
		"":"IPv6 is disabled by default, to enable it, set this parameter value to 0",

		"":"Afanasy server HTTP serve folder, if not set CGRU root folder will be used",
	"-af_http_serve_dir":"/work/cgru",
		"":"Afanasy server HTTP response on an empty GET request",
	"af_http_site_index":"/afanasy/browser/index.html",
		"":"Afanasy server HTTP response on a directory GET request",
	"af_http_directory_index":"index.html",

		"":"Trusted IP addresses masks. From these IPs no authentication needed.",
	"af_ip_trust":["*.*","127.0.0.1","192.168.*","*:*","0:0:0:0:0:0:0:1","FC00:*","FD00:*"],
		"":"Mask should have at least one '.' for IPv4 or one ':' for IPv6.",

		"":"Digest authentication file path relative to CGRU_LOCATION folder",
	"af_digest_file":".htdigest",

		"":"Commands arguments:",
		"":"@ARG@ - will be replaced with render name:",
		"":"@ASK@ - raise dialog to ask a string:",
		"":"@IP@  - will be replaced with render ip address:",

	"OS_unix":{
		"af_store_folder":"/var/tmp/afanasy",
		"af_rendercmds":["vncviewer @ARG@","xterm -e \"ping @ARG@; read -s -n 1\"","xterm -e \"ping @IP@; read -s -n 1\""],
		"af_rendercmds_admin":["xterm -e \"ssh @ARG@; read -s -n 1\"","xterm -e \"ssh @ARG@ \\\"@ASK@\\\"; read -s -n 1\""],
		"":"Render:",
		"af_render_cmd_reboot":"reboot",
			"":"Command for render to reboot a machine.",
			"":"For MS Windows you can use 'shutdown /r'.",

		"af_render_cmd_shutdown":"shutdown -P 1 afrequest",
			"":"Command for render to shutdown a machine.",
			"":"For MS Windows you can use 'shutdown /s'.",

		"af_render_cmd_wolsleep":"wolsleep",
			"":"On UNIX sleep command usually needs root privileges.",
			"":"Try to use 'sudo wolsleep' and add a line in /etc/sudoers:",
			"":"render ALL = (ALL) NOPASSWD: /opt/cgru/bin/wolsleep",
			"":"Or configure your own way to sleep.",
		"af_render_launch_cmds":["Free Mem|sudo free_mem.sh"],
			"":"From GUI you can ask render to launch a command",
		"af_render_launch_cmds_exit":["Update|sudo update.sh"],
			"":"Ask render to launch a command and exit no waiting for finish",
		"":""
	},
	"OS_windows":{
	
		"af_rendercmds":["vncviewer @ARG@","start ping /t @ARG@","start ping /t @IP@"],
		"af_store_folder":"c:/temp/afanasy",
		"af_render_cmd_reboot":"shutdown /r",
		"af_render_cmd_shutdown":"shutdown /s",
		"af_render_cmd_wolsleep":"cmd.exe /c wolsleep",


        "pathsmap":[
            ["Z:/",             "/work/"],
            ["C:/Users/Can/Documents/cgru",  "/work/cgru/"]

        ],	
		
		"af_render_windowsmustdie":[
			"ImageMagick Studio library and utility programs",
			"Microsoft Visual C++ Runtime Library",
			"QuickTimeHelper-32.exe - Application Error",
			"Visual Studio Just-In-Time Debugger"

		]
	},

	"af_perm_user_mod_his_priority":true,
	"af_perm_user_mod_job_priority":true,

	"af_cmdprefix":"",
		"":"If 'AF_CMD_PREFIX' environment variable exists, it will be used. All commands with be prefixed with it.",

	"af_node_log_lines_max":100,
		"":"Log history for any afnode (job, render, user)",

	"af_priority":99,
	"af_max_running_tasks":-1,

"":"Job:",
	"af_task_default_capacity":1000,
	"af_task_progress_change_timeout":-1,
		"":"This is a default value for a job block.",
		"":"If task progress did not change within this time, consider that it is erroneous.",
		"":"A value of -1 means not to check timeout.",

	"af_task_default_service":"generic",
	"af_task_log_linesmax":100,

"":"Render:",
	"-af_render_networkif":"eth0",
		"":"Network interface to measure traffic",
		"":"If not specified all used except loopback",

	"af_render_hddspace_path":"/",
		"":"Path to measure free disk space.",
		"":"'/' - for MS Windows means the root of the system disk.",

	"af_render_iostat_device":"*",
		"":"Device to measure disk IO statistics.",
		"":"For MS Windows - statistics only for first disk available.",

	"-af_render_resclasses":["example","iostat","nvidia_smi"],
		"":"Custom resources meters:",
			"":"example - simple resource meter for an example only",
			"":"iostat - parse output of linux 'iostat' command",

	"-af_render_customiostat_devices":"sda",
		"":"Device(s) for custom python resource meter 'iostat'.",
		"":"To watch several devices use regular expression, for example 'sda[2-4]'.",
		"":"If several devices matching pattern are founded,",
		"":"it calculates the sum of traffic parameters and maximum of utilization parameters.",

	"af_render_nice":10,
		"":"Renice task command process on render.",
		"":"MS Windows priority:",
		"":"DWORD priority = NORMAL_PRIORITY_CLASS;",
		"":"if( nice >   0 ) priority = BELOW_NORMAL_PRIORITY_CLASS;",
		"":"if( nice >  10 ) priority = IDLE_PRIORITY_CLASS;",
		"":"if( nice <   0 ) priority = ABOVE_NORMAL_PRIORITY_CLASS;",
		"":"if( nice < -10 ) priority = HIGH_PRIORITY_CLASS;",

	"af_task_reconnect_timeout":30,
		"":"After server restart, running tasks will wait renders to reconnect.",

	"af_render_cmd_wolwake":"wolwake",

	"af_render_overflow_mem" :-1,
	"af_render_overflow_swap":-1,
	"af_render_overflow_hdd" :-1,
		"":"Percentage of a free memomry, swap and hdd, to consider thar render overflowed",
		"":"-1 disables checking.",

	"":"",

"":"Thumbnail:",
	"af_thumbnail_extensions":["exr","dpx","jpg","jpeg","png","tif","tiff","tga"],
	"af_thumbnail_cmd":"convert -identify \"%(image)s\" %(pre_args)s -alpha Opaque -thumbnail x100 -gravity center -colorspace sRGB \"%(thumbnail)s\"",

"":"User:",
	"af_errors_avoid_host":3,
	"af_task_error_retries":3,
	"af_task_errors_same_host":3,
	"af_errors_forgivetime":18000,
	"af_max_run_tasks_per_host":-1,
	"af_jobs_life_time":0,

"":"Monitor: (server side - any gui)",
	"af_monitor_zombietime":16,
    "af_monitor_render_idle_bar_max":3600,

"":"Watch: (qt gui - client side)",
	"af_watch_get_events_sec":1,
	"af_watch_refresh_gui_sec":5,
	"af_watch_connection_lost_time":20,
		"":"If watch can`t update server for this time, it considers that server connection lost and tries to register again.",
	"af_watch_render_idle_bar_max":3600,
	"af_watch_work_user_visible":true,

"":"Database:",
		"":"sting to pass to PQconnectdb http://www.postgresql.org/docs/current/static/libpq-connect.html",
	"af_db_conninfo":"host=192.168.1.55 dbname=afanasy user=afadmin password=AfPassword",
	"af_db_stringquotes":"$$",
	"af_db_stringnamelen":512,
	"af_db_stringexprlen":4096,

"":"System job:",
	"af_sysjob_tasklife":1800,
	"af_sysjob_tasksmax":1000,
	"af_sysjob_wol_service":"wakeonlan",
	"af_sysjob_postcmd_service":"postcmd",
	"af_sysjob_events_service":"events",

"":"Network:",
	"":"Number of threads to process clients connections.",
	"af_server_sockets_processing_threads_num":10,
		"":"Ten threads to process should be enough for any situation.",

	"":"Number of threads to read/write clients connections.",
	"af_server_sockets_readwrite_threads_num":10,
		"":"Ten threads does not mean that server can handle only about 10 clients.",
		"":"This only means that it can read/write to 10 clients at the same time.",
		"":"Other clients will just wait in IO queue.",
		"":"For a fast networks designed for rendering 10 IO threads should be enough for any farm size.",

	"":"Server sockets processing stack size.",
	"-af_server_sockets_processing_threads_stack":32768,
	"-af_server_sockets_readwrite_threads_stack":32768,
		"":"Zero, negative value or no parameter means to use system default",

	"":"You can use non-blocking IO on Linux server, based on Linux epoll facility",
	"af_server_linux_epoll":0,
		"":"If it is disabled (by default), Linux server will use blocking IO based on threads, like other platforms",

	"":"Server waits client close socket first. Web browsers do it, only if we ask it in HTTP header by special header",
	"af_server_http_wait_close":1,
		"":"If you browser ignores 'Connection: close' header, you can make server not to wait it.",

	"":"Socket options that can be set to play with:",
	"af_so_server_RCVTIMEO_sec":12,
	"af_so_server_SNDTIMEO_sec":12,
	"af_so_server_TCP_NODELAY":-1,
	"af_so_server_REUSEADDR":-1,
	"af_so_server_TCP_CORK":-1,
	"af_so_server_LINGER":-1,

	"af_so_client_RCVTIMEO_sec":12,
	"af_so_client_SNDTIMEO_sec":12,
	"af_so_client_TCP_NODELAY":-1,
	"af_so_client_REUSEADDR":-1,
	"af_so_client_TCP_CORK":-1,
	"af_so_client_LINGER":-1,
		"":"Use -1 value not to set socket option at all",
		"":"See man socket for details.",

	"af_server_profiling_sec":1024,
		"":"Server will output some network statistics by this period",

	"af_wolwake_interval":10,
		"":"Number of cycles (seconds) between waking each render",

	"":""
}}

config_cgru in the cgru folder

Code: Select all

{"cgru_config":{
	"":"This is a CGRU default config file",
	"":"Describe your own configuration in 'config.json', it will be included last",
	"":"To disable property you can just prefix it with '-'",

	"include":["afanasy/config_default.json","config.json"],

	"":"'OS_unix': Any UNIX (Linux, MacOSX) specific settings",
	"":"'OS_windows': MS Windows specific settings",
	"":"'OS_linux': Linix specific settings",
	"":"'OS_macosx': MacOSX specific settings",

	"":"Can be used globally for windows titles",
	"-company":"CGRU-DEVEL",

	"":"Keeper system tray icon",
	"-tray_icon":"cgru-devel",

	"":"Path where custom icons will be searched",
	"-icons_path":"/cg/icons;/local/icons",
		"":"Can be ; or : separated",

	"":"Keeper menu custom path",
	"-menu_path":"/cg/apps",


		"":"Digest authentication realm",
	"realm":"CGRU",

	"":"To open location/shot in RULES:",
	"-rules_url":"http://192.168.1.55/cgru/#/",
	"":"By default it is disabled",

	"projects_root":["Z:/work/teset/","/work/teset/","/cgru/rules_root/"],

	"-nuke_filenamefilter":true,

	"open_folder_cmd":"openfolder \"@PATH@\"",
	"open_terminal_cmd":"openterminal -e '@CMD@'",

	"previewcmds":[
		"Nuke|nuke -v \"@ARG@\"",
		"Mplay|mplay \"@ARG@\"",
		"Mplay *|mplay `dirname \"@ARG@\"`/*",
		"DJV|djv_view \"@ARG@\"",
		"IMDisplay|display \"@ARG@\"",
		"XTerm Folder|cd `dirname @ARG@` && xterm"
	],
	"annotations":[
		"APPROVED",
		"FAILED"
	],

	"docs_url":"https://cgru.readthedocs.io",
	"forum_url":"https://forum.cgru.info",

	"":"Configure SMTP server to send emails from",
	"email_smtp_server":"localhost",
	"":"You can keep 'localhost', but configure machines with RULES server",
	"":" and Afanasy 'events' service to have such service running",

	"":"This command provided with CGRU and uses utilities/emailsend.py Python script",
	"email_send_cmd":"emailsend",

	"email_sender_address_host":"cgru.info",

	"keeper_port":50999,
	"keeper_port_https":50443,

	"timeformat":"%a %d %b %H:%M.%S",
		"":"'strftime' function format http://www.cplusplus.com/reference/clibrary/ctime/strftime",

	"file_name_size_max":250,

	"platforms":["linux","mac","windows"],

	"OS_unix":{
		"cmd_shell":"/bin/sh -c"
	},
	"OS_windows":{
		"open_folder_cmd":"explorer \"@PATH@\"",
		"cmd_shell":"cmd.exe /c",

			"":"This is a paths map example (disabled)",
			"":"Client:Server",
		"-pathsmap":[
			["Z:/work/teset/","/work/teset/"]
		],
			"":"Client slashes direction plays no role, as it tries both slashes when searchs a path to replace.",
			"":"But will replace with specified slashes, if unix slashes not forced (for Nuke, for example)."
	},
config in the cgru folder

{"cgru_config":{
"af_servername":"afanasy",
"":""

}}

What I want to do is to be able to render with 2 operating systems in my local network. Good luck, it's a very successful application. But it got too complicated. Especially to be able to install on arch-based systems.I think I was able to make the necessary adjustments for rendering. But I could not set the cgru root folder. It felt really weird asking him to find this file.
User avatar
timurhai
Site Admin
Posts: 911
Joined: Sun Jan 15, 2017 8:40 pm
Location: Russia, Korolev
Contact:

Re: rendering with different operating systems

Post by timurhai »

https://cgru.readthedocs.io/en/latest/c ... nfig-files

It is a bad idea to modify config_default.json files directly. As native files can be changed in a new version. And you should apply your changes each time you update cgru.

To prevent this, you should create a new file cgru/config.json, where you just override only needed parameters, such as server address and paths map.If you have linux and windows, better to use linux paths as server paths, and windows as client.So each windows client should have pathsmap in config, linux clients should not have any.

Then you can say me, that i have such config.json on windows workstation, where i created a job: ...
On linux machine task have such command: ... (afwatch client on linux will show linux paths).
On windows machine task looks like: ...
Afrender on windows machine have such a warning in output: ...
Are there some warnings on linux?

And so on. But you even have not told what machine produced that output. I just can guess that it is a cmd.exe window by fonts looks like.
Timur Hairulin
CGRU 3.3.1, Ubuntu 20.04, 22.04, MS Windows 10 (clients only).
dasrow
Posts: 10
Joined: Tue Jun 29, 2021 12:26 pm

Re: rendering with different operating systems

Post by dasrow »

Thank you so much. I'm saying right now. Don't worry. I phrased it very wrong. I am writing with translation. That's why these problems happen.I'll try and get back to you in a better way as you said.
dasrow
Posts: 10
Joined: Tue Jun 29, 2021 12:26 pm

Re: rendering with different operating systems

Post by dasrow »

https://youtu.be/B2IP2g3yK0I

I do not know English. That's why I use translate. So I may have misdirected. I wanted to shoot a video too. I reset the configs while shooting the video.

Even if my purpose is to render on linux or windows. Clients can render. But if I use pathmap, the path does not change. even if i add it to config. When I change the default_config in afanasy, I can do it as c:/cgru=/work/cgru. But even if I add the config, it does not change to the client or server in the same way. In no way could I show where to render.
I struggled for 1-2 days. I tried about it as much as possible, I looked at similar articles before. I feel stupid. I'm really sorry for bothering you.
If so, can you give an example of how I can set the config?
User avatar
timurhai
Site Admin
Posts: 911
Joined: Sun Jan 15, 2017 8:40 pm
Location: Russia, Korolev
Contact:

Re: rendering with different operating systems

Post by timurhai »

Hi!

This is not valid json syntax:
config.jpg
config.jpg (15.15 KiB) Viewed 4743 times
You should delete 3-5 lines.

I do not understand this error:
error.jpg
error.jpg (11.17 KiB) Viewed 4743 times
May be it is not paths map theme at all.
Timur Hairulin
CGRU 3.3.1, Ubuntu 20.04, 22.04, MS Windows 10 (clients only).
User avatar
timurhai
Site Admin
Posts: 911
Joined: Sun Jan 15, 2017 8:40 pm
Location: Russia, Korolev
Contact:

Re: rendering with different operating systems

Post by timurhai »

It is very hard (needs more my time) to want movies.
Please do not send me movies, i have no time to watch it.
Do not edit config_default files.
Create your own cgru/config.json file.
On any client it should contain server address, on Windows it can contain server address and paths map (or you can have one shared config file for linux and windows with OS_windows or OS_unix sections).
You can just copy&paste it here to show me, without any screenshots or movies.
Timur Hairulin
CGRU 3.3.1, Ubuntu 20.04, 22.04, MS Windows 10 (clients only).
Post Reply