Difference between revisions of "Orange: Similarity Hashing"
Onnowpurbo (talk | contribs) (Created page with "Sumber: https://orange3-text.readthedocs.io/en/latest/widgets/similarityhashing.html Computes documents hashes. Inputs Corpus: A collection of documents. Outputs...") |
Onnowpurbo (talk | contribs) |
||
| Line 14: | Line 14: | ||
Similarity Hashing is a widget that transforms documents into similarity vectors. The widget uses SimHash method from from Moses Charikar. | Similarity Hashing is a widget that transforms documents into similarity vectors. The widget uses SimHash method from from Moses Charikar. | ||
| − | + | [[File:Similarity-Hashing-stamped.png|center|200px|thumb]] | |
Set Simhash size (how many attributes will be on the output, corresponds to bits of information) and shingle length (how many tokens are used in a shingle). | Set Simhash size (how many attributes will be on the output, corresponds to bits of information) and shingle length (how many tokens are used in a shingle). | ||
Commit Automatically output the data automatically. Alternatively, press Commit. | Commit Automatically output the data automatically. Alternatively, press Commit. | ||
| − | + | ==Contoh== | |
We will use deerwester.tab to find similar documents in this small corpus. Load the data with Corpus and pass it to Similarity Hashing. We will keep the default hash size and shingle length. We can observe what the widget outputs in a Data Table. There are 64 new attributes available, corresponding to the Simhash size parameter. | We will use deerwester.tab to find similar documents in this small corpus. Load the data with Corpus and pass it to Similarity Hashing. We will keep the default hash size and shingle length. We can observe what the widget outputs in a Data Table. There are 64 new attributes available, corresponding to the Simhash size parameter. | ||
| − | + | [[File:Similarity-Hashing-Example.png|center|200px|thumb]] | |
| − | + | ||
| + | |||
| + | ==Referensi== | ||
Charikar, M. (2002) Similarity estimation techniques from rounding algorithms. STOC ‘02 Proceedings of the thirty-fourth annual ACM symposium on Theory of computing, p. 380-388. | Charikar, M. (2002) Similarity estimation techniques from rounding algorithms. STOC ‘02 Proceedings of the thirty-fourth annual ACM symposium on Theory of computing, p. 380-388. | ||
Revision as of 15:56, 24 January 2020
Sumber: https://orange3-text.readthedocs.io/en/latest/widgets/similarityhashing.html
Computes documents hashes.
Inputs
Corpus: A collection of documents.
Outputs
Corpus: Corpus with simhash value as attributes.
Similarity Hashing is a widget that transforms documents into similarity vectors. The widget uses SimHash method from from Moses Charikar.
Set Simhash size (how many attributes will be on the output, corresponds to bits of information) and shingle length (how many tokens are used in a shingle). Commit Automatically output the data automatically. Alternatively, press Commit.
Contoh
We will use deerwester.tab to find similar documents in this small corpus. Load the data with Corpus and pass it to Similarity Hashing. We will keep the default hash size and shingle length. We can observe what the widget outputs in a Data Table. There are 64 new attributes available, corresponding to the Simhash size parameter.
Referensi
Charikar, M. (2002) Similarity estimation techniques from rounding algorithms. STOC ‘02 Proceedings of the thirty-fourth annual ACM symposium on Theory of computing, p. 380-388.