extendedmosaicperm.experiments.base module

class BaseExperiment

Bases: object

Base utilities and containers for Monte Carlo experiments.

Subclasses should implement run() to populate self.results and extract_method() to map a label string to a method identifier.

results

Nested mapping of the form {label: {violation_strength: np.ndarray of p-values}}, populated by run().

Initialize an empty experiment container.

extract_method(label)

Extract the method name from a label string.

Parameters:

label (str) – Label produced by a concrete experiment, e.g. "random_small_sym_sign".

Returns:

Method identifier used by the experiment, e.g. "sign", "perm", "ridge", "ols".

Return type:

str

Raises:

NotImplementedError – If a subclass does not override this method.

flatten()

Flatten nested results into a long-form DataFrame.

Returns:

A DataFrame with columns:

• "label",

• "violation_strength",

• "method",

• "p_value".

Return type:

pandas.DataFrame

Raises:

AssertionError – If self.results is None (i.e. run() has not been executed).

load(path)

Load results from a pickle file.

Parameters:

path (str) – Input file path containing serialized results created by save().

Return type:

None

plot_power(df_flat, generator='random', symmetry='sym')

Plot empirical power curves vs violation strength.

Creates three panels for size settings "small", "medium", and "large".

Parameters:

• df_flat (DataFrame) – Long-form DataFrame returned by flatten().

• generator (str) – Generator key used in labels, e.g. "random", "block", "common".

• symmetry (str) – Symmetry flag used in labels, either "sym" or "asym".

Returns:

Figure with three subplots showing power by method.

Return type:

matplotlib.figure.Figure

plot_pval_histogram(df_flat, label, v=0.0, bins=20)

Plot histograms of p-values at a given violation strength.

Parameters:

• df_flat (DataFrame) – Long-form DataFrame returned by flatten().

• label (str) – Label prefix to filter rows, e.g. "random_small_sym".

• v (float) – Violation strength value to slice on.

• bins (int) – Number of histogram bins.

Returns:

Figure with two panels, one histogram per method.

Return type:

matplotlib.figure.Figure

plot_qq(df_flat, label_bases, v)

Draw QQ-plots of p-values for selected label bases at a fixed violation level.

Parameters:

• df_flat (DataFrame) – Long-form DataFrame returned by flatten().

• label_bases (list[str]) – Base labels (without method suffix) to compare, e.g. ["random_small_sym", "random_medium_sym"].

• v (float) – Violation strength value to slice on.

Returns:

Figure with two QQ plots (empirical vs uniform).

Return type:

matplotlib.figure.Figure

save(path)

Serialize results to a pickle file.

Parameters:

path (str) – Output file path for the serialized results.

Return type:

None

summarize()

Summarize empirical power and p-value moments.

For each label and violation strength, compute empirical rejection probability and basic p-value summaries.

Returns:

A summary DataFrame with columns:

• "label",

• "method",

• "violation_strength",

• "power",

• "mean_pval",

• "median_pval",

• "std_pval",

• "n".

Return type:

pandas.DataFrame

Raises:

AssertionError – If self.results is None.