Rust implementation of the CVM algorithm for counting distinct elements in a stream
More on ownership
author
Stephan Hügel
date
(May 24, 2024, 1:42 PM +0100)
commit
3c43e77d
3c43e77d4644a38cd8555f2c81801a6d331e4724
parent
69997f64
69997f6440e0d3a952eb2376a039c618d9f9e0e2
+1
-1
+1
-1
README.md
+1
-1
README.md
···
14
14
# What is an Element
15
15
In this implementation, an element is anything implementing the [`PartialOrd`](https://doc.rust-lang.org/std/cmp/trait.PartialOrd.html) and [`PartialEQ`](https://doc.rust-lang.org/std/cmp/trait.PartialEq.html) traits: various integer flavours, strings, any Struct on which you have implemented the traits. Not `f32` / `f64`, however.
16
16
17
-
You will also note that I didn't mention `&str`: that's because the buffer has to keep ownership of its elements. In practice, this is not a problem: relative to its input stream size, the buffer is very small.
17
+
You will also note that I didn't mention `&str`: that's because the buffer has to keep ownership of its elements. In practice, this is not a problem: relative to its input stream size, the buffer is very small. This is also the entire point of the algorithm: your data set is very large; you **don't** want to keep the original data around in order to store references to it!
18
18
19
19
## Further Details
20
20
Don Knuth has written about the algorithm (he refers to it as **Algorithm D**) at https://cs.stanford.edu/~knuth/papers/cvm-note.pdf, and does a far better job than I do at explaining it. You will note that on p1 he describes the buffer he uses as a data structure called a [treap](https://en.wikipedia.org/wiki/Treap#:~:text=7%20External%20links-,Description,(randomly%20chosen)%20numeric%20priority.)