Orange: Similarity Hashing

From OnnoWiki
Revision as of 10:27, 29 January 2020 by Onnowpurbo (talk | contribs)
Jump to navigation Jump to search

Sumber: https://orange3-text.readthedocs.io/en/latest/widgets/similarityhashing.html


Computes documents hashes.

Input

Corpus: A collection of documents.

Output

Corpus: Corpus with simhash value as attributes.

Similarity Hashing is a widget that transforms documents into similarity vectors. The widget uses SimHash method from from Moses Charikar.

Similarity-Hashing-stamped.png
  • Set Simhash size (how many attributes will be on the output, corresponds to bits of information) and shingle length (how many tokens are used in a shingle).
  • Commit Automatically output the data automatically. Alternatively, press Commit.

Contoh

We will use deerwester.tab to find similar documents in this small corpus. Load the data with Corpus and pass it to Similarity Hashing. We will keep the default hash size and shingle length. We can observe what the widget outputs in a Data Table. There are 64 new attributes available, corresponding to the Simhash size parameter.

Similarity-Hashing-Example.png


Referensi

Charikar, M. (2002) Similarity estimation techniques from rounding algorithms. STOC ‘02 Proceedings of the thirty-fourth annual ACM symposium on Theory of computing, p. 380-388.



Referensi

Pranala Menarik