Datafly algorithm

Datafly algorithm is an algorithm for providing anonymity in medical data. The algorithm was developed by Latanya Arvette Sweeney in 1997−98.^[1][2] Anonymization is achieved by automatically generalizing, substituting, inserting, and removing information as appropriate without losing many of the details found within the data. The method can be used on-the-fly in role-based security within an institution, and in batch mode for exporting data from an institution. Organizations release and receive medical data with all explicit identifiers -- such as name -- removed, in the erroneous belief that patient confidentiality is maintained because the resulting data look anonymous. However the remaining data can be used to re-identify individuals by linking or matching the data to other databases or by looking at unique characteristics found in the fields and records of the database itself.

The Datafly algorithm has been criticized for trying to achieve anonymization by over-generalization. The algorithm selects the attribute with the greatest number of distinct values as the one to generalize first.^[3]

Core algorithm

An outline of the Datafly algorithm is presented below.^[4]

Input: Private Table PT; quasi-identifier QI = ( A₁, ..., A_n ), k-anonymity constraint k; domain generalization hierarchies DGH_Ai, where i = 1,...,n with accompanying functions f_Ai, and loss, which is a limit on the percentage of tuples that can be suppressed. PT[id] is the set of unique identifiers or keys for each tuple.

Output: MGT a generalization of PT[QI] that enforces k-anonymity

Assumes: | PT | ≤ k, and loss * | PT | = k

algorithm Datafly:

// Construct a frequency list containing unique sequences of values across the quasi-identifier in PT,

// along with the number of occurrences of each sequence.

1. let freq be an expandable and collapsible vector with no elements initially. Each element is of the form ( QI, frequency, SID ), where SID = { id_i : ∃ t[id] ∈ [id] ⇒ t[id] = id_i }; and, frequency = |SID|. Therefore, freq is also accessible as a table over (QI, frequency, SID).

2. let pos $\gets$ 0, total $\gets$ 0

3. while total ≠ |PT| do

3.1 freq[pos] $\gets$ ( t[QI], occurs, SID ) where t[QI] ∈ [QI], ( t[ QI ],__, ___ ) $\not\in$ freq; occurs = |PT| - |PT[QI] – {t[QI]}|; and, SID = { id_i : ∃ t[id] $\gets$ PT[id] ⇒ t[id] = id_i }

3.2 pos $\gets$ pos + 1, total $\gets$ total + occurs

// Make a solution by generalizing the attribute with the most number of distinct values

// and suppressing no more than the allowed number of tuples.

4. let belowk $\gets$ 0

5. for pos $\gets$ 1 to |freq| do

5.1 ( __, count ) $\gets$ freq[pos]

5.2 if count < k then do

5.2.1 belowk $\gets$ belowk + count

6. if belowk > k then do: // Note. loss * |PT| = k.

6.1 freq $\gets$ generalize(freq)

6.2 go to step 4

7. else do

// assert: the number of tuples to suppress in freq is ≤ loss * |PT|

7.1 freq $\gets$ suppress(freq, belowk )

7.2 MGT $\gets$ reconstruct(freq)

8. return MGT.

External links

• Details of the Datafly algorithm

References

1. Latanya Sweeney. "Datafly: a system for providing anonymity in medical data". Retrieved 19 January 2014.
2. L. Sweeney, Datafly: a system for providing anonymity in medical data. Database Security, XI: Status and Prospects, T. Lin and S. Qian (eds), Elsevier Science, Amsterdam, 1998.
3. Li Xiong. "Data Anonymization - Generalization Algorithms" (PDF). Retrieved 19 January 2014.
4. Latanya Sweeney. Computational Disclosure Control A Primer on Data Privacy Protection. MIT. p. 113.