Orange: Topic Modelling - Revision history

Onnowpurbo: /* Output */

2020-04-12T12:08:55Z

Output

← Older revision		Revision as of 12:08, 12 April 2020
Line 22:		Line 22:
	Latent Dirichlet Allocation (LDA) lebih mudah di interpretasikan, akan tetapi lebih lambat di bandingkan Latent Semantic Indexing (LSI). Hierarchical Dirichlet Process (HDP) mempunyai banyak parameter - parameter yang sesuai dengan jumlah topik adalah Top level truncation level (T). Jumlah topik terkecil yang dapat diambil seseorang adalah 10.		Latent Dirichlet Allocation (LDA) lebih mudah di interpretasikan, akan tetapi lebih lambat di bandingkan Latent Semantic Indexing (LSI). Hierarchical Dirichlet Process (HDP) mempunyai banyak parameter - parameter yang sesuai dengan jumlah topik adalah Top level truncation level (T). Jumlah topik terkecil yang dapat diambil seseorang adalah 10.

	[[File:Topic-Modelling-stamped.png\|center\|~~200px~~\|thumb]]		[[File:Topic-Modelling-stamped.png\|center\|600px\|thumb]]

	* Topic modelling algorithm:		* Topic modelling algorithm:

Onnowpurbo: /* Visualisasi Topik */

2020-04-12T12:08:04Z

Visualisasi Topik

← Older revision		Revision as of 12:08, 12 April 2020
Line 55:		Line 55:
	===Visualisasi Topik===		===Visualisasi Topik===

	Dalam contoh ~~kedua~~, kita akan melihat korelasi antara topik dan kata / dokumen. Kita menggunakan corpus grimm-tales-selected.tab. Dalam widget Preprocess Text kita menggunakan preprocessing default, dengan filter tambahan berdasarkan frekuensi dokumen (0,1 - 0,9). Dalam Topic Modeling kita menggunakan model Latent Dirichlet Allocation (LDA) dengan 5 topik.		Dalam contoh berikut, kita akan melihat korelasi antara topik dan kata / dokumen. Kita menggunakan corpus grimm-tales-selected.tab. Dalam widget Preprocess Text kita menggunakan preprocessing default, dengan filter tambahan berdasarkan frekuensi dokumen (0,1 - 0,9). Dalam Topic Modeling kita menggunakan model Latent Dirichlet Allocation (LDA) dengan 5 topik.

	Hubungkan widget Topic Modelling ke widget MDS. Pastikan link diatur ke All Topics - Data. Widget Topic Modelling akan menampilkan matriks bobot kata berdasarkan topik.		Hubungkan widget Topic Modelling ke widget MDS. Pastikan link diatur ke All Topics - Data. Widget Topic Modelling akan menampilkan matriks bobot kata berdasarkan topik.

Onnowpurbo: /* Visualisasi Topik */

2020-04-11T12:10:04Z

Visualisasi Topik

← Older revision		Revision as of 12:10, 11 April 2020
Line 63:		Line 63:
	[[File:Topic-Modelling-Example2-MDS.png\|center\|400px\|thumb]]		[[File:Topic-Modelling-Example2-MDS.png\|center\|400px\|thumb]]

	~~We can now explore which words are representative for the topic~~. ~~Select~~, ~~say~~, ~~Topic~~ 5 ~~from the~~ plot ~~and connect~~ widget MDS to widget ~~Box Plot~~. ~~Make sure the~~ output ~~is set to~~ Data - Data (~~not~~ Selected Data - Data).		Kita sekarang dapat mengeksplorasi kata-kata mana yang mewakili topik tersebut. Pilih, katakanlah, Topik 5 dari plot dan hubungkan widget MDS ke Plot Box widget. Pastikan output diatur ke Data - Data (bukan Selected Data - Data).

	In widget Box Plot, ~~set the~~ subgroup to Selected ~~and check the~~ Order by relevance to subgroups box. ~~This option will sort the variables by how well they separate between the selected subgroup values~~. ~~In our case~~, ~~this means which words are the most representative for the topic we have selected in the~~ plot (subgroup Yes ~~means~~ selected).		Di widget Box Plot, atur subgroup ke Selected dan centang Order by relevance ke subgroups box. Opsi ini akan mengurutkan variabel berdasarkan seberapa baik mereka memisahkan antara nilai-nilai subkelompok yang dipilih. Dalam kasus kita, ini berarti kata-kata mana yang paling representatif untuk topik yang telah kami pilih dalam plot (subgroup Yes berarti selected).

	~~We can see that~~ little, children and kings ~~are the most representative words for Topic~~ 5, ~~with good separation between the word frequency for this topic and all the others~~. ~~Select other topics in~~ widget MDS ~~and see how the~~ widget Box Plot ~~changes~~.		Kita dapat melihat bahwa little, children and kings adalah kata yang paling representatif untuk Topik 5, dengan pemisahan yang baik antara frekuensi kata untuk topik ini dan yang lainnya. Pilih topik lain dalam widget MDS dan lihat bagaimana perubahan widget Box Plot.

	[[File:Topic-Modelling-Example2-BoxPlot.png\|center\|400px\|thumb]]		[[File:Topic-Modelling-Example2-BoxPlot.png\|center\|400px\|thumb]]

Onnowpurbo: /* Visualisasi Topik */

2020-04-11T11:49:02Z

Visualisasi Topik

← Older revision		Revision as of 11:49, 11 April 2020
Line 57:		Line 57:
	Dalam contoh kedua, kita akan melihat korelasi antara topik dan kata / dokumen. Kita menggunakan corpus grimm-tales-selected.tab. Dalam widget Preprocess Text kita menggunakan preprocessing default, dengan filter tambahan berdasarkan frekuensi dokumen (0,1 - 0,9). Dalam Topic Modeling kita menggunakan model Latent Dirichlet Allocation (LDA) dengan 5 topik.		Dalam contoh kedua, kita akan melihat korelasi antara topik dan kata / dokumen. Kita menggunakan corpus grimm-tales-selected.tab. Dalam widget Preprocess Text kita menggunakan preprocessing default, dengan filter tambahan berdasarkan frekuensi dokumen (0,1 - 0,9). Dalam Topic Modeling kita menggunakan model Latent Dirichlet Allocation (LDA) dengan 5 topik.

	~~Connect~~ widget Topic Modelling to widget MDS. ~~Ensure the~~ link ~~is set to~~ All Topics - Data. Widget Topic Modelling ~~will output a matrix of word weights by topic~~.		Hubungkan widget Topic Modelling ke widget MDS. Pastikan link diatur ke All Topics - Data. Widget Topic Modelling akan menampilkan matriks bobot kata berdasarkan topik.

	In widget MDS, ~~the points are now topics~~. ~~We have set the size of the points to~~ Marginal ~~topic probability~~, ~~which is an additional columns of~~ All Topics - ~~it reports on the~~ marginal ~~probability of the topic in the~~ corpus (~~how strongly represented is the topic in the~~ corpus).		Dalam widget MDS, poinnya sekarang menjadi topik. Kita telah mengatur ukuran poin ke probabilitas topik Marginal, yang merupakan kolom tambahan dari All Topics - ini melaporkan probabilitas marginal dari topik dalam corpus (seberapa kuat representasi topik dalam corpus).

	[[File:Topic-Modelling-Example2-MDS.png\|center\|400px\|thumb]]		[[File:Topic-Modelling-Example2-MDS.png\|center\|400px\|thumb]]

Onnowpurbo at 11:45, 11 April 2020

2020-04-11T11:45:13Z

← Older revision		Revision as of 11:45, 11 April 2020
Line 55:		Line 55:
	===Visualisasi Topik===		===Visualisasi Topik===

	Dalam contoh kedua, kita akan melihat korelasi antara topik dan kata / dokumen. Kita menggunakan corpus grimm-tales-selected.tab. Dalam widget Preprocess Text kita menggunakan preprocessing default, dengan filter tambahan berdasarkan frekuensi dokumen (0,1 - 0,9). Dalam Topic Modeling kita menggunakan model LDA dengan 5 topik.		Dalam contoh kedua, kita akan melihat korelasi antara topik dan kata / dokumen. Kita menggunakan corpus grimm-tales-selected.tab. Dalam widget Preprocess Text kita menggunakan preprocessing default, dengan filter tambahan berdasarkan frekuensi dokumen (0,1 - 0,9). Dalam Topic Modeling kita menggunakan model Latent Dirichlet Allocation (LDA) dengan 5 topik.

	Connect Topic Modelling to MDS. Ensure the link is set to All Topics - Data. Topic Modelling will output a matrix of word weights by topic.		Connect widget Topic Modelling to widget MDS. Ensure the link is set to All Topics - Data. Widget Topic Modelling will output a matrix of word weights by topic.

	In MDS, the points are now topics. We have set the size of the points to Marginal topic probability, which is an additional columns of All Topics - it reports on the marginal probability of the topic in the corpus (how strongly represented is the topic in the corpus).		In widget MDS, the points are now topics. We have set the size of the points to Marginal topic probability, which is an additional columns of All Topics - it reports on the marginal probability of the topic in the corpus (how strongly represented is the topic in the corpus).

	[[File:Topic-Modelling-Example2-MDS.png\|center\|400px\|thumb]]		[[File:Topic-Modelling-Example2-MDS.png\|center\|400px\|thumb]]

	We can now explore which words are representative for the topic. Select, say, Topic 5 from the plot and connect MDS to Box Plot. Make sure the output is set to Data - Data (not Selected Data - Data).		We can now explore which words are representative for the topic. Select, say, Topic 5 from the plot and connect widget MDS to widget Box Plot. Make sure the output is set to Data - Data (not Selected Data - Data).

	In Box Plot, set the subgroup to Selected and check the Order by relevance to subgroups box. This option will sort the variables by how well they separate between the selected subgroup values. In our case, this means which words are the most representative for the topic we have selected in the plot (subgroup Yes means selected).		In widget Box Plot, set the subgroup to Selected and check the Order by relevance to subgroups box. This option will sort the variables by how well they separate between the selected subgroup values. In our case, this means which words are the most representative for the topic we have selected in the plot (subgroup Yes means selected).

	We can see that little, children and kings are the most representative words for Topic 5, with good separation between the word frequency for this topic and all the others. Select other topics in MDS and see how the Box Plot changes.		We can see that little, children and kings are the most representative words for Topic 5, with good separation between the word frequency for this topic and all the others. Select other topics in widget MDS and see how the widget Box Plot changes.

	[[File:Topic-Modelling-Example2-BoxPlot.png\|center\|400px\|thumb]]		[[File:Topic-Modelling-Example2-BoxPlot.png\|center\|400px\|thumb]]

Onnowpurbo: /* Visualisasi Topik */

2020-04-11T11:43:25Z

Visualisasi Topik

← Older revision		Revision as of 11:43, 11 April 2020
Line 55:		Line 55:
	===Visualisasi Topik===		===Visualisasi Topik===

	~~In the second example~~, ~~we will look at the correlation between topics and words~~/~~documents~~. ~~We are still using the~~ grimm-tales-selected.tab ~~corpus~~. In Preprocess Text ~~we are using the~~ default ~~preprocessing~~, ~~with an additional~~ filter ~~by document frequency~~ (0.1 - 0.9). In Topic ~~Modelling we are using~~ LDA ~~model with~~ 5 ~~topics~~.		Dalam contoh kedua, kita akan melihat korelasi antara topik dan kata / dokumen. Kita menggunakan corpus grimm-tales-selected.tab. Dalam widget Preprocess Text kita menggunakan preprocessing default, dengan filter tambahan berdasarkan frekuensi dokumen (0,1 - 0,9). Dalam Topic Modeling kita menggunakan model LDA dengan 5 topik.

	Connect Topic Modelling to MDS. Ensure the link is set to All Topics - Data. Topic Modelling will output a matrix of word weights by topic.		Connect Topic Modelling to MDS. Ensure the link is set to All Topics - Data. Topic Modelling will output a matrix of word weights by topic.

Onnowpurbo: /* Visualisasi Topik */

2020-04-11T11:38:43Z

Visualisasi Topik

← Older revision		Revision as of 11:38, 11 April 2020
Line 61:		Line 61:
	In MDS, the points are now topics. We have set the size of the points to Marginal topic probability, which is an additional columns of All Topics - it reports on the marginal probability of the topic in the corpus (how strongly represented is the topic in the corpus).		In MDS, the points are now topics. We have set the size of the points to Marginal topic probability, which is an additional columns of All Topics - it reports on the marginal probability of the topic in the corpus (how strongly represented is the topic in the corpus).

	[[File:Topic-Modelling-Example2-MDS.png\|center\|~~200px~~\|thumb]]		[[File:Topic-Modelling-Example2-MDS.png\|center\|400px\|thumb]]

	We can now explore which words are representative for the topic. Select, say, Topic 5 from the plot and connect MDS to Box Plot. Make sure the output is set to Data - Data (not Selected Data - Data).		We can now explore which words are representative for the topic. Select, say, Topic 5 from the plot and connect MDS to Box Plot. Make sure the output is set to Data - Data (not Selected Data - Data).
Line 69:		Line 69:
	We can see that little, children and kings are the most representative words for Topic 5, with good separation between the word frequency for this topic and all the others. Select other topics in MDS and see how the Box Plot changes.		We can see that little, children and kings are the most representative words for Topic 5, with good separation between the word frequency for this topic and all the others. Select other topics in MDS and see how the Box Plot changes.

	[[File:Topic-Modelling-Example2-BoxPlot.png\|center\|~~200px~~\|thumb]]		[[File:Topic-Modelling-Example2-BoxPlot.png\|center\|400px\|thumb]]

	==Referensi==		==Referensi==

Onnowpurbo: /* Topic Visualization */

2020-04-11T11:38:03Z

Topic Visualization

← Older revision		Revision as of 11:38, 11 April 2020
Line 53:		Line 53:
	Kita kemudian memilih topik pertama dan menampilkan kata-kata paling sering dalam topik di widget Word Cloud. Kita juga menghubungkan widget Preprocess Text ke widget Word Cloud agar dapat menampilkan dokumen yang dipilih. Sekarang kita dapat memilih kata tertentu di widget Word Cloud, katakan 'little'. Ini akan berwarna merah dan juga di highlight dalam daftar kata di sebelah kiri. Selanjutnya, kita dapat mengamati dokumen berisi kata 'little' di widget Corpus Viewer.		Kita kemudian memilih topik pertama dan menampilkan kata-kata paling sering dalam topik di widget Word Cloud. Kita juga menghubungkan widget Preprocess Text ke widget Word Cloud agar dapat menampilkan dokumen yang dipilih. Sekarang kita dapat memilih kata tertentu di widget Word Cloud, katakan 'little'. Ini akan berwarna merah dan juga di highlight dalam daftar kata di sebelah kiri. Selanjutnya, kita dapat mengamati dokumen berisi kata 'little' di widget Corpus Viewer.

	===~~Topic Visualization~~===		===Visualisasi Topik===

	In the second example, we will look at the correlation between topics and words/documents. We are still using the grimm-tales-selected.tab corpus. In Preprocess Text we are using the default preprocessing, with an additional filter by document frequency (0.1 - 0.9). In Topic Modelling we are using LDA model with 5 topics.		In the second example, we will look at the correlation between topics and words/documents. We are still using the grimm-tales-selected.tab corpus. In Preprocess Text we are using the default preprocessing, with an additional filter by document frequency (0.1 - 0.9). In Topic Modelling we are using LDA model with 5 topics.
Line 70:		Line 70:

	[[File:Topic-Modelling-Example2-BoxPlot.png\|center\|200px\|thumb]]		[[File:Topic-Modelling-Example2-BoxPlot.png\|center\|200px\|thumb]]



	==Referensi==		==Referensi==

Onnowpurbo: /* Exploring Individual Topics */

2020-04-11T11:37:45Z

Exploring Individual Topics

← Older revision		Revision as of 11:37, 11 April 2020
Line 43:		Line 43:
	==Contoh==		==Contoh==

	===~~Exploring~~ Individual ~~Topics~~===		===Explorasi Topik Individual===

	Dalam contoh ini, diperlihatkan penggunaan sederhana widget Topic Modelling. Pertama-tama kita load kumpulan data grimm-tales-selected.tab dan menggunakan widget Preprocess Text untuk tokenize hanya dengan kata-kata dan menghapus stopword. Kemudian kami menghubungkan widget Preprocess Text ke widget Topic Modelling, di mana kita menggunakan Latent Semantic Indexing (LSI) sederhana untuk menemukan 10 topik dalam text.		Dalam contoh ini, diperlihatkan penggunaan sederhana widget Topic Modelling. Pertama-tama kita load kumpulan data grimm-tales-selected.tab dan menggunakan widget Preprocess Text untuk tokenize hanya dengan kata-kata dan menghapus stopword. Kemudian kami menghubungkan widget Preprocess Text ke widget Topic Modelling, di mana kita menggunakan Latent Semantic Indexing (LSI) sederhana untuk menemukan 10 topik dalam text.
Line 51:		Line 51:
	Latent Semantic Indexing (LSI) memberikan bobot positif dan negatif per topik. Bobot positif berarti kata yang digunakan sangat mewakili suatu topik, sedangkan bobot negatif berarti kata tersebut sangat tidak mewakili topik (semakin sedikit hal itu terjadi dalam text, semakin besar kemungkinan topik itu). Kata-kata positif berwarna hijau dan kata-kata negatif berwarna merah.		Latent Semantic Indexing (LSI) memberikan bobot positif dan negatif per topik. Bobot positif berarti kata yang digunakan sangat mewakili suatu topik, sedangkan bobot negatif berarti kata tersebut sangat tidak mewakili topik (semakin sedikit hal itu terjadi dalam text, semakin besar kemungkinan topik itu). Kata-kata positif berwarna hijau dan kata-kata negatif berwarna merah.

	~~We then select the first topic and display the most frequent words in the topic in~~ Word Cloud. ~~We also connected~~ Preprocess Text to Word Cloud ~~in order to be able to output selected documents~~. ~~Now we can select a specific word in the word cloud~~, ~~say~~ little. ~~It will be colored red and also highlighted in the word list on the left~~.		Kita kemudian memilih topik pertama dan menampilkan kata-kata paling sering dalam topik di widget Word Cloud. Kita juga menghubungkan widget Preprocess Text ke widget Word Cloud agar dapat menampilkan dokumen yang dipilih. Sekarang kita dapat memilih kata tertentu di widget Word Cloud, katakan 'little'. Ini akan berwarna merah dan juga di highlight dalam daftar kata di sebelah kiri. Selanjutnya, kita dapat mengamati dokumen berisi kata 'little' di widget Corpus Viewer.

	~~Now we can observe all the documents containing the word~~ little in Corpus Viewer.

	===Topic Visualization===		===Topic Visualization===

Onnowpurbo: /* Exploring Individual Topics */

2020-04-11T11:33:29Z

Exploring Individual Topics

← Older revision		Revision as of 11:33, 11 April 2020
Line 45:		Line 45:
	===Exploring Individual Topics===		===Exploring Individual Topics===

	~~In the first example~~, ~~we present a simple use of the~~ Topic Modelling ~~widget~~. ~~First we~~ load grimm-tales-selected.tab ~~data set and use~~ Preprocess Text to tokenize ~~by words only and remove stopwords~~. ~~Then we connect~~ Preprocess Text to Topic Modelling, ~~where we use a simple~~ Latent Semantic Indexing ~~to find~~ 10 ~~topics in the~~ text.		Dalam contoh ini, diperlihatkan penggunaan sederhana widget Topic Modelling. Pertama-tama kita load kumpulan data grimm-tales-selected.tab dan menggunakan widget Preprocess Text untuk tokenize hanya dengan kata-kata dan menghapus stopword. Kemudian kami menghubungkan widget Preprocess Text ke widget Topic Modelling, di mana kita menggunakan Latent Semantic Indexing (LSI) sederhana untuk menemukan 10 topik dalam text.

	[[File:Topic-Modelling-Example1.png\|center\|~~200px~~\|thumb]]		[[File:Topic-Modelling-Example1.png\|center\|600px\|thumb]]

	LSI ~~provides both positive and negative weights~~ per ~~topic~~. ~~A positive weight means the word is highly representative of a topic~~, ~~while a negative weight means the word is highly unrepresentative of a topic~~ (~~the less it occurs in a~~ text, ~~the more likely the topic~~). ~~Positive words are colored green and negative words are colored red~~.		Latent Semantic Indexing (LSI) memberikan bobot positif dan negatif per topik. Bobot positif berarti kata yang digunakan sangat mewakili suatu topik, sedangkan bobot negatif berarti kata tersebut sangat tidak mewakili topik (semakin sedikit hal itu terjadi dalam text, semakin besar kemungkinan topik itu). Kata-kata positif berwarna hijau dan kata-kata negatif berwarna merah.

	We then select the first topic and display the most frequent words in the topic in Word Cloud. We also connected Preprocess Text to Word Cloud in order to be able to output selected documents. Now we can select a specific word in the word cloud, say little. It will be colored red and also highlighted in the word list on the left.		We then select the first topic and display the most frequent words in the topic in Word Cloud. We also connected Preprocess Text to Word Cloud in order to be able to output selected documents. Now we can select a specific word in the word cloud, say little. It will be colored red and also highlighted in the word list on the left.