_spurious#

Metric functions relating to spurious rate.

Classes#

SpuriousRate

Computes the fraction of groups in the DP output but not in the baseline output.
class SpuriousRate(output, join_columns, *, name=None, description=None, baselines=None)#

Bases: tmlt.analytics.metrics._base.SingleBaselineMetric

Computes the fraction of groups in the DP output but not in the baseline output.

Note

This is only available on a paid version of Tumult Analytics. If you would like to hear more, please contact us at info@tmlt.io.

Note

Below, released means that the group is in the DP output, and spurious means that the group is not in the baseline output.

How it works:

  1. The algorithm takes two dictionaries as input:

  • dp_outputs: A dictionary containing the differentially private (DP) outputs, where keys represent output identifiers and values represent corresponding DP output. The DP output data is generated by a differentially private mechanism.

  • baseline_outputs: A dictionary containing the baseline outputs, where keys represent output identifiers and values represent corresponding baseline table (baseline).

Before performing computations, the algorithm checks if the released count of DP output (released count) is zero. If so, it returns NaN, indicating that no computation can be performed due to the absence of released data. If not, the algorithm performs a left anti-join between the DP and baseline tables based on join_columns. This returns all rows from the DP output (left dataframe) where there is no match in the baseline output (right dataframe). The count of these rows is the spurious released count.

  1. After performing the join, the algorithm computes the spurious rate by dividing the spurious released count by the total count of released data points (released_count), using the formula \(\text{spurious released count} / \text{released count}\). The result represents the proportion of released data points in the DP output that have no corresponding data points in the baseline output.

Example

>>> dp_df = spark.createDataFrame(
...     pd.DataFrame(
...         {
...             "A": ["a1", "a1", "a2", "c"],
...             "X": [50, 110, 100, 50]
...         }
...     )
... )
>>> dp_outputs = {"O": dp_df}
>>> baseline_df = spark.createDataFrame(
...     pd.DataFrame(
...         {
...             "A": ["a1", "a1", "a2", "b"],
...             "X": [100, 100, 100, 50]
...         }
...     )
... )
>>> baseline_outputs = {"O": baseline_df}

>>> metric = SpuriousRate(
...     output="O",
...     join_columns=["A"]
... )
>>> metric.join_columns
['A']
>>> metric.compute_for_baseline(dp_outputs, baseline_outputs)
0.25
Parameters

  • output (str) –

  • join_columns (List[str]) –

  • name (Optional[str]) –

  • description (Optional[str]) –

  • baselines (Optional[Union[str, List[str]]]) –

__init__(output, join_columns, *, name=None, description=None, baselines=None)#

Constructor.

Parameters

  • output (strstr) – The output to compute the spurious rate for.

  • join_columns (List[str]List[str]) – The columns to join on.

  • name (str | NoneOptional[str] (default: None)) – A name for the metric.

  • description (str | NoneOptional[str] (default: None)) – A description of the metric.

  • baselines (str | List[str] | NoneUnion[str, List[str], None] (default: None)) – The name of the baseline program(s) used for the error report. If None, use all baselines specified as custom baseline and baseline options on tuner class. If no baselines are specified on tuner class, use default baseline. If a string, use only that baseline. If a list, use only those baselines.

property output#

Returns the name of the run output or view name.

Return type

str

property join_columns#

Returns the name of the join columns.

Return type

List[str]

format(value)#

Returns a string representation of this object.

check_compatibility_with_program(program, output_views)#

Checks if the metric is compatible with the program.

This is a dynamic check and is performed by verifying whether the output attribute of the metric object is present in the annotations of the Outputs attribute of the program. If the output attribute is not found in the annotations, a ValueError is raised.

Parameters
compute_for_baseline(dp_outputs, baseline_outputs, unprotected_inputs=None, program_parameters=None)#

Computes spurious rate given DP and baseline outputs.

Parameters
property name#

Returns the name of the metric.

Return type

str

property description#

Returns the description of the metric.

Return type

str

property baselines#

Returns the baselines used for the metric.

Return type

Optional[Union[str, List[str]]]

__call__(dp_outputs, baseline_outputs, unprotected_inputs=None, program_parameters=None)#

Computes the given metric on the given DP and baseline outputs.

Parameters

  • dp_outputs (Dict[str, pyspark.sql.DataFrame]) – The differentially private outputs of the program.

  • baseline_outputs (Dict[str, Dict[str, pyspark.sql.DataFrame]]) – The outputs of the baseline programs.

  • unprotected_inputs (Optional[Dict[str, pyspark.sql.DataFrame]]) – Optional public dataframes used in error computation.

  • program_parameters (Optional[Dict[str, Any]]) – Optional program specific parameters used in error computation.

Return type

List[tmlt.analytics.metrics.MetricOutput]