Overview

There are cases where you would see a message on the top of the script designer mentioning that the script includes an extremely large variables number. Such a number is typically way to large with respect to common projects and may result in issues with exporting the data, analyzing it etc.

The most common cause to this is the fact that nested loops have been configured during scripting. Nested loops may generate a very large data set as the variables number is a multiplication of the internal question variables.

Each variable of each question inside the loop chapter will start with the iteration index - "I_NumOfIteration" and then the variable name as it is defined to be exported.

Example: If the loop is iterating a maximum of two iteration and inside the loop chapter I have a single choice question with a variable name: Q_1 then the exported iteration data for this question will have these two names: I_1_Q_1 and I_2_Q_1

For Nested Loops you will have an additional nesting of variables for example   I_1_I_1_Q_1, I_1_I_2_Q_1.... and I_2_I_1_Q_1, I_2_I_2_Q_1...

Basically each variable is created multiplied by the max number of iterations of the loops that can of course quickly become a very large number.

Assume one multiple selection question with 25 answers (events drink in) residing inside a nested loop of 2 levels iterating on 30 beverage categories and 150 possible corresponding beverage brands overall will result in 30x150x25 variables meaning 112,500 variables just for that question.  

We know in advance that most of the variables will always be empty (for example a combination of Still water as Category and brand as Coca Cola will never be answered as the logic will not display that brand for that category).

On the other hand all that we are interested in is the N beverages that the inner question asks for (for example events that you often drink the few beverages you mentioned you drink often).

That may result eventually in 10-20 drinks that you often drink for which you would mention 5-10 max events you often drink those in...that is a much smaller amount of variables than the 112,500 that are mentioned above.   

That very large data set may take extremely long to export when exported, result in a huge data set that will be extremely difficult to analyze and may result in certain cases with inability to export the data.

There are various ways to approach such situations.

Scripting Method to build an efficient variables structure

In most cases there would be ways to script the survey differently in a way that would eventually allow you to have a data structure that is reasonable to analyze.

The most common path is to build the part that does the nesting only for display (but is not set to get exported) then have another structure for the exporting purpose.

The following 2 samples explain that concept:
1) Simple nesting without thinking of the implication on variables
2) One that looks the same when run and collected the same data partially but has been structured to allow a reasonable data structure.

They also include a usage of an excel file that represents the relations of the categories and the brands as this would normally be the case in such nesting scenarios.
To view and be able to compare the 2 scripts: 

• Import the 2 surveys to your organization: https://support.dooblo.net/hc/en-us/articles/208293905-How-To-Import-a-Survey
  
  
• Create the Subject Store that is used in both using the Excel (explained here: https://support.dooblo.net/hc/en-us/articles/208294845-How-To-Use-an-Excel-File-as-a-Data-List-in-a-Survey). The code to use the subject store is already implemented in the scripts. Call the generated subject store the same as the excel i.e. BrandsCategories

Once imported you could review how both are structured. Mainly notice the one that exports without the  'exceeding Variables number' issue:
1) The 'exportable' script has the nested chapter set to not export (Advanced tab). 
2) The "Export variables" chapter is not displayed but is the one that is set to export
3) The exportable questions are set by the expression question (located after the nested loops and before the "Export variables" chapter).

The basic idea of this sample is that, while collecting data on a product with nested loops (Click here to learn more on Nested Loops), as each product is combined with several attributes, its final structure would be one iteration (that is each iteration in the "Export variables" chapter represents one product), with the selected attributes. 

Overcoming the challenge during export

There are case where you do not have an option to adjust your script to be better structured with regards to the exported variables  (for various reason such as scripting expertise, time or resource constraints). In such cases you can follow this guide to see what options are available to reduce that number during exportContact our support at support@dooblo.com for any question