As @Chris Davidson mentioned, you shouldn't put lookupex or lookup in an IF statement.
You are better off creating separate columns:
LookupEx Helper=if(Column 1="xxyy",Column 1,"No Lookup")
Column 2=lookupex(LookupEx Helper,TableName,Column Name,Return Column Name)
We use "No Lookup" to prevent the lookupex from working.
Secondly if you want to do multiple levels of Lookup, I would say do it in transforms. So you do the first lookupex in Dataset A. Then take a transform (Dataset A Stage 2) and do the second lookupex. That way you control the order of operations.
BTW - remember that lookup and lookupex is case sensitive. so I tend to use the: optional parameters in the following way:
=lookupex(LookupEx Helper,TableName,Column Name,Return Column Name,false,false,true)