Parser error string

General discussions and questions.
Post Reply
g-LuL
Posts: 10
Joined: Mon Mar 27, 2017 7:37 am

Parser error string

Post by g-LuL »

Hello !

I've created a parser for a new software.This software is a command line python script...
This parser inherits of the standard parser (parser.parser). Its str_error is equal to a brand new list

Code: Select all

class my_parser(parser.parser):
	def __init__(self):
		parser.parser.__init__(self)
		self.str_error = ["BakeError"]
But, when the job is launched, one of my subprocesses prints "Error: etc etc" and afanasy stops the job nevertheless my str_error attribute of my parser is "BakeError". Am I missing something ? Or is it the normal behaviour ?
Thanks !
Manu
User avatar
timurhai
Site Admin
Posts: 911
Joined: Sun Jan 15, 2017 8:40 pm
Location: Russia, Korolev
Contact:

Re: Parser error string

Post by timurhai »

Hi.
Can you explain (paste) more exactly what happened?
What process, where and what printed?
What is afrender log? Task log and output?
What does it mean "afanasy stops the job"? ( afserver puts job to "Offline" state or something happen with a task?)
Timur Hairulin
CGRU 3.3.1, Ubuntu 20.04, 22.04, MS Windows 10 (clients only).
g-LuL
Posts: 10
Joined: Mon Mar 27, 2017 7:37 am

Re: Parser error string

Post by g-LuL »

Hi Timur,
Shame on me, sorry for the lack of informations in my post. I've sent it in a hurry before meeting...
So i've created a new "software". This consists of a simple command line calling python executing a python script.
In this script, several blender subprocesses are called successively.
Unfortunately, Blender prints some "Error :" during its execution but I don't care of these. So I want Afanasy to ignore those errors message.
The python script prints warning and error with a particular string prefix : "Bake" (so "BakeError" and "BakeWarning").
I create a python submitter to send job to AF. The job sent is like this :

Code: Select all

{
    "blocks": [
        {
            "capacity": 1000, 
            "command": "python /s/cgru/linux/cgru/software_setup/bin/nonnon_animtorender_app/start_nonnon_animtorender.py 00-220-01", 
            "flags": 1, 
            "frame_first": 1, 
            "frame_last": 1, 
            "frames_inc": 1, 
            "frames_per_task": 1, 
            "name": "BAKE NONNON", 
            "parser": "nonnon_animtorender", 
            "service": "nonnon_animtorender", 
            "working_directory": "/z/___SERIE/NON-NON"
        }
    ], 
    "host_name": "hackintosh3", 
    "name": "00-220-01 bake", 
    "priority": 99, 
    "time_creation": 1490684825, 
    "user_name": "graph"
}
Here is my new parser :

Code: Select all

from parsers import parser
class nonnon_animtorender(parser.parser):
	"""Non-Non Batch
	"""
	def __init__(self):
		parser.parser.__init__(self)
		self.str_error = ['BakeError']
		self.str_warning = ['BakeWarning']
		self.finishedsuccess = ['Bake Done :']

	def do(self, data, mode):
		pass
When i send the job, all is fine, and AF assign it to an afrender. AF prints the output of the python script normally.
But when the blender "Error:" appears in the log, AF raise an error host and assign the job to another afrender.
This error is raised before the second call of the blender subprocess.

Here's the log :

Code: Select all

Read new prefs: /home/graph/.config/blender/2.78/config/userpref.blend
found bundled python: /usr/local/blender-2.78c-linux-glibc219-x86_64/2.78/python
read blend: /z/___SERIE/NON-NON/episodes/00/plans/00-220-01/projets/anim/00-220-01_anim_v017.blend
Info: Read library: '/z/___SERIE/NON-NON/banque/decors/arche_interieur/arche_interieur_v22.blend', '//../../../../../../banque/decors/arche_interieur/arche_interieur_v22.blend', parent ''
Info: Read library: '/z/___SERIE/NON-NON/banque/props/interieur_arche/velo_triporteur_v05.blend', '//../../../../../../banque/props/interieur_arche/velo_triporteur_v05.blend', parent '/z/___SERIE/NON-NON/banque/decors/arche_interieur/arche_interieur_v22.blend'
Info: Read library: '/z/___SERIE/NON-NON/banque/props/sacGrouillette/sacGrouillette_v07.blend', '//../../../../../../banque/props/sacGrouillette/sacGrouillette_v07.blend', parent '/z/___SERIE/NON-NON/banque/decors/arche_interieur/arche_interieur_v22.blend'
Info: Read library: '/z/___SERIE/NON-NON/banque/props/caisseOutils/caisseOutils_v01.blend', '//../../../../../../banque/props/caisseOutils/caisseOutils_v01.blend', parent '/z/___SERIE/NON-NON/banque/decors/arche_interieur/arche_interieur_v22.blend'
Info: Read library: '/z/___SERIE/NON-NON/banque/props/interieur_grocroc/int_grocroc_boites_v02.blend', '//../../../../../../banque/props/interieur_grocroc/int_grocroc_boites_v02.blend', parent '/z/___SERIE/NON-NON/banque/decors/arche_interieur/arche_interieur_v22.blend'
Info: Read library: '/z/___SERIE/NON-NON/banque/props/interieur_arche/valises_v01.blend', '//../../../../../../banque/props/interieur_arche/valises_v01.blend', parent '/z/___SERIE/NON-NON/banque/decors/arche_interieur/arche_interieur_v22.blend'
Info: Read library: '/z/___SERIE/NON-NON/banque/props/interieur_arche/paniers_v01.blend', '//../../../../../../banque/props/interieur_arche/paniers_v01.blend', parent '/z/___SERIE/NON-NON/banque/decors/arche_interieur/arche_interieur_v22.blend'
Info: Read library: '/z/___SERIE/NON-NON/banque/props/interieur_arche/couverture_v01.blend', '//../../../../../../banque/props/interieur_arche/couverture_v01.blend', parent '/z/___SERIE/NON-NON/banque/decors/arche_interieur/arche_interieur_v22.blend'
Info: Read library: '/z/___SERIE/NON-NON/banque/props/interieur_arche/sacs_v02.blend', '//../../../../../../banque/props/interieur_arche/sacs_v02.blend', parent '/z/___SERIE/NON-NON/banque/decors/arche_interieur/arche_interieur_v22.blend'
Info: Read library: '/z/___SERIE/NON-NON/banque/persos/bio/bio_setanim_v62.blend', '//../../../../../../banque/persos/bio/bio_setanim_v62.blend', parent ''
Info: Read library: '/z/___SERIE/NON-NON/banque/persos/canard/canard_setanim_v15.blend', '//../../../../../../banque/persos/canard/canard_setanim_v15.blend', parent ''
Info: Read library: '/z/___SERIE/NON-NON/banque/persos/cochon/cochon_setanim_v15.blend', '//../../../../../../banque/persos/cochon/cochon_setanim_v15.blend', parent ''
Info: Read library: '/z/___SERIE/NON-NON/banque/persos/denis/denis_setanim_v16.blend', '//../../../../../../banque/persos/denis/denis_setanim_v16.blend', parent ''
Info: Read library: '/z/___SERIE/NON-NON/banque/persos/grouillette/grouillette_setanim_v23.blend', '//../../../../../../banque/persos/grouillette/grouillette_setanim_v23.blend', parent ''
Info: Read library: '/z/___SERIE/NON-NON/banque/persos/marmotte/marmotte_setanim_v14.blend', '//../../../../../../banque/persos/marmotte/marmotte_setanim_v14.blend', parent ''
Info: Read library: '/z/___SERIE/NON-NON/banque/persos/oie/oie_setanim_v10.blend', '//../../../../../../banque/persos/oie/oie_setanim_v10.blend', parent ''
Info: Read library: '/z/___SERIE/NON-NON/banque/persos/taupe/taupe_setanim_v08.blend', '//../../../../../../banque/persos/taupe/taupe_setanim_v08.blend', parent ''
Info: Read library: '/z/___SERIE/NON-NON/banque/persos/taupeAdulte/taupeAdulte_setanim_v09.blend', '//../../../../../../banque/persos/taupeAdulte/taupeAdulte_setanim_v09.blend', parent ''
Info: Read library: '/z/___SERIE/NON-NON/banque/persos/zoubi/zoubi_setanim_v25.blend', '//../../../../../../banque/persos/zoubi/zoubi_setanim_v25.blend', parent ''
Info: Read library: '/z/___SERIE/NON-NON/banque/persos/magaiveur/magaiveur_setanim_v25.blend', '//../../../../../../banque/persos/magaiveur/magaiveur_setanim_v25.blend', parent ''
Info: Read library: '/z/___SERIE/NON-NON/banque/persos/grocroc/grocroc_setanim_v29.blend', '//../../../../../../banque/persos/grocroc/grocroc_setanim_v29.blend', parent ''
Info: Read library: '/z/___SERIE/NON-NON/banque/fx/ocean_normals-waves/ocean_normals-waves_v28.blend', '//../../../../../../banque/fx/ocean_normals-waves/ocean_normals-waves_v28.blend', parent ''
Info: Read library: '/z/___SERIE/NON-NON/banque/props/interieur_grocroc/int_grocroc_outils_v03.blend', '//../../props/interieur_grocroc/int_grocroc_outils_v03.blend', parent '/z/___SERIE/NON-NON/banque/decors/arche_interieur/arche_interieur_v22.blend'
Info: Read library: '/z/___SERIE/NON-NON/banque/props/atelier_grocroc/atelier_grocroc_v02.blend', '//../../props/atelier_grocroc/atelier_grocroc_v02.blend', parent '/z/___SERIE/NON-NON/banque/decors/arche_interieur/arche_interieur_v22.blend'
Info: Read library: '/z/___SERIE/NON-NON/banque/nodes/shaders/clay_bump_generator.blend', '//../../nodes/shaders/clay_bump_generator.blend', parent '/z/___SERIE/NON-NON/banque/decors/arche_interieur/arche_interieur_v22.blend'
Info: Read library: '/z/___SERIE/NON-NON/banque/nodes/shaders/clay.blend', '//../../nodes/shaders/clay.blend', parent '/z/___SERIE/NON-NON/banque/persos/zoubi/zoubi_setanim_v25.blend'
Info: Read library: '/z/___SERIE/NON-NON/banque/props/interieur_grocroc/int_grocroc_crayons_pinceaux_craie_v02.blend', '//../interieur_grocroc/int_grocroc_crayons_pinceaux_craie_v02.blend', parent '/z/___SERIE/NON-NON/banque/props/sacGrouillette/sacGrouillette_v07.blend'
Info: Read library: '/z/___SERIE/NON-NON/banque/props/interieur_grouillette/int_grouillette_skateboard_v03.blend', '//../interieur_grouillette/int_grouillette_skateboard_v03.blend', parent '/z/___SERIE/NON-NON/banque/props/sacGrouillette/sacGrouillette_v07.blend'
Info: Read library: '/z/___SERIE/NON-NON/banque/props/interieur_grouillette/int_grouillette_reveils_v02.blend', '//../interieur_grouillette/int_grouillette_reveils_v02.blend', parent '/z/___SERIE/NON-NON/banque/props/sacGrouillette/sacGrouillette_v07.blend'
Info: Read library: '/z/___SERIE/NON-NON/banque/nodes/shaders/pixiSwitch.blend', '//../../nodes/shaders/pixiSwitch.blend', parent '/z/___SERIE/NON-NON/banque/persos/bio/bio_setanim_v62.blend'
Registering activateCuda
Registering jpgFromExrComp
Registering log
Registering refreshBeforeRender
Exception in module register(): '/usr/local/blender-2.78c-linux-glibc219-x86_64/2.78/scripts/addons/object_boolean_tools.py'
Bake transforms : 100
Bake transforms : 101
Bake transforms : 102
Bake transforms : 103
Bake transforms : 104
Bake transforms : 105
Bake transforms : 106
Bake transforms : 107
Bake transforms : 108
Bake transforms : 109
Bake transforms : 110
Bake transforms : 111
Bake transforms : 112
Bake transforms : 113
Bake transforms : 114
Bake transforms : 115
Bake transforms : 116
Bake transforms : 117
Bake transforms : 118
Bake transforms : 119
Bake transforms : 120
Bake transforms : 121
Bake transforms : 122
Bake transforms : 123
Bake transforms : 124
Bake transforms : 125
Bake transforms : 126
Bake transforms : 127
Bake transforms : 128
Bake transforms : 129
Bake transforms : 130
Bake transforms : 131
Bake transforms : 132
Bake transforms : 133
Bake transforms : 134
Bake transforms : 135
Bake transforms : 136
Bake transforms : 137
Bake transforms : 138
Bake transforms : 139
Bake transforms : 140
Bake transforms : 141
Bake transforms : 142
Bake transforms : 143
Bake transforms : 144
Bake transforms : 145
Bake transforms : 146
Bake transforms : 147
Bake transforms : 148
Bake transforms : 149
Bake transforms : 150
Bake transforms : 151
Bake transforms : 152
Bake transforms : 153
Bake transforms : 154
Bake transforms : 155
Bake transforms : 156
Bake transforms : 157
Bake transforms : 158
Bake transforms : 159
Bake transforms : 160
Bake transforms : 161
Bake transforms : 162
Bake transforms : 163
Bake transforms : 164
Error: Not freed memory blocks: 26, total unfreed memory 0.833603 MB

Blender quit
When all the afrender are on error hosts, afwatch prints "Error:Bad Output"

Here's the log (not the output) printed in afwatch:

Code: Select all

Mon 27 Mar 14:38.03 : SESSION #1: Starting on "r01"
Mon 27 Mar 14:41.26 : Parser error.
Mon 27 Mar 14:41.26 : Automatically retrying error task1 of 3.
Mon 27 Mar 14:41.26 : SESSION #2: Starting on "r02"
Mon 27 Mar 14:44.57 : Parser error.
Mon 27 Mar 14:44.57 : Automatically retrying error task2 of 3.
Mon 27 Mar 14:44.57 : SESSION #3: Starting on "r03"
Mon 27 Mar 14:49.11 : Parser error.
Mon 27 Mar 14:49.11 : Automatically retrying error task3 of 3.
Mon 27 Mar 14:49.11 : SESSION #4: Starting on "r04"
Mon 27 Mar 14:53.56 : Parser error.
Here's an excerpt of the afrender log:

Code: Select all

Started PID=15932 SID=15932(-1) GID=15932(1813): [nonnon_animtorender:1000] graph: 00-220-01 bake[BAKE NONNON][frame 1]
AFERROR: msgsendtoaddress: connect failure for msgType 'TRenderUpdate':
192.168.75.225:51000: Connection refused

Finished PID=15932: Exit Code=15 Status=0 (stopped)
Task terminated/killed by signal: 'Terminated'.
Mon 27 Mar 14:24.02: INFO    Connection lost count = 1 of 3
Mon 27 Mar 14:24.03: INFO    Reconnected to the server
Mon 27 Mar 14:24.03: ERROR   Zero ID recieved, no such online render, re-connecting...
Mon 27 Mar 14:24.03: WARNING Render connection lost, trying to reconnect...
Mon 27 Mar 14:24.04: INFO    Reconnected to the server
Mon 27 Mar 14:24.04: INFO    Render registered.
I'm on Linux Mint 18.1 with CGRU 2.2.0.
I don't understand the full process of the parser so I'm missing something for sure.
I hope there's more clues in the logs...
Thanks for your time
Manu
User avatar
timurhai
Site Admin
Posts: 911
Joined: Sun Jan 15, 2017 8:40 pm
Location: Russia, Korolev
Contact:

Re: Parser error string

Post by timurhai »

Hi.
Have you updated blender parser on all renders and restarted afrenders?
Since 2.2.1 base parser class does not look for an 'error' and 'warning' strings.
https://github.com/CGRU/cgru/commit/2d5 ... 1e31cbddc4
You can try to do the same in a base parser class too.

btw
self.finishedsuccess should be used in a very special cases.
You process does not finish itself with a good exit status?
If it does ( it should work so ), you do need to ask parser to kill it.
It can be used with some very buggy programs (windows+afterfx, for example),
when we sure that process is finished success, but it does not.
And it is a very bad practice to not to let process to finish itself.
Timur Hairulin
CGRU 3.3.1, Ubuntu 20.04, 22.04, MS Windows 10 (clients only).
g-LuL
Posts: 10
Joined: Mon Mar 27, 2017 7:37 am

Re: Parser error string

Post by g-LuL »

Hi Timur,

All my afrender have the same cgru location on the network. So they have all the same parsers.
I've retarted afserver every time I modified the parser BUT I've not restarted my afrender :( My mistake...
Restarting my afrenders worked for now ! I will try on multiple jobs just to be sure.

My processes generate errors but they finish themselves without another process killing them.
Ok for the 2.2.1 parser. I'll have a look on that and maybe I'll update my cgru installation.
And ok for the buggy programs (that's not 100% my case but I'll remember it)

Thanks for your time. I'll keep you posted if it works for the next jobs !
Manu
g-LuL
Posts: 10
Joined: Mon Mar 27, 2017 7:37 am

Re: Parser error string

Post by g-LuL »

Hi Timur,

Just to keep you posted, it works like a charm now. Just a afrender restart and it's ok.
Thanks again !
Manu
User avatar
timurhai
Site Admin
Posts: 911
Joined: Sun Jan 15, 2017 8:40 pm
Location: Russia, Korolev
Contact:

Re: Parser error string

Post by timurhai »

Hi.
BTW, read this: viewtopic.php?f=34&t=106
Timur Hairulin
CGRU 3.3.1, Ubuntu 20.04, 22.04, MS Windows 10 (clients only).
Post Reply