This is a non-exhaustive list of common questions and error messages:
“All reliabilities must be greater than 0 and less than or equal to 1”
“All correlations must be within [-1,1]”
“All correlations must be within [-1,1], this could be due to skew being too high.” or “No valid settings”
adjust.correlations = TRUE, and a variable is binary or skew is not 0, the algorithm will increase the correlations so that the output correlations match the effects input by the user. This is necessary because transforming normal variables to be binary or skewed reduces their correlations with the other variables. However, it can happen that a variable is so skewed that this is impossible, as it would require correlations greater than |1|. If you encounter this error, reduce your skew, or your correlations between variables.
power_interaction()automatically removes these settings from the input.
“Settings produce a negative y-variance” or “No valid settings”
“Correlation matrix is impossible - is not positive semi-definite.”
The predicted value where power is reached is outside the range of values tested by the simulation. Simply increase the range of values tested.
The predicted value is within the range of values tested, but only barely. For example, say
power_target = .8, and one simulation achieved a
pwr = 0.804. Even though 0.804 was achieved in one round of simulations, the regression analysis may predict, based on the other values tested in the simulation, that an even larger value is needed to achieve 80% power. The solution is again to increase the range of values tested.
Power-curves taper-off as power increases, eventually flattening out.
power_estimate() fits a threshold regression model using the chngpt package, to identify whether the power-curve has flattened, and to remove the flattened portion if so. This is so that the main
power_estimate() regression model will be more accurate. However, occasionally the threshold regression model is greedy, and it removes too much. This can happen particularly if the simulation only includes a few values. The solution is again to increase the range of values tested.
Results differ each time I run a simulation
n.iter = 1000simulations, as this is enough to give a general sense of the pattern of results. However, we recommend
n.iter = 10000for more stable results.
power_estimate()fits a regression model, and as with any regression, the number of observations, as well as their variance, impacts results. So, increasing the range and density of parameters in the simulation will also improve the stability of results. The
seedparameter can also be set, so that identical results are given each time. However, this is not recommended until results have stabilized (by increasing
n.iter = 50will yield the same results every time, but that doesn’t mean those results are informative.