Python: Generating Network Graph of Twitter Follower - Revision history

Onnowpurbo: /* Step 2. Process twitter data to generate an output file of relationships between twitter accounts */

2017-01-29T05:48:13Z

Step 2. Process twitter data to generate an output file of relationships between twitter accounts

Onnowpurbo: /* Step 2. Process twitter data to generate an output file of relationships between twitter accounts */

2017-01-29T05:47:50Z

Step 2. Process twitter data to generate an output file of relationships between twitter accounts

Onnowpurbo: /* Python file: get_followers.py */

2017-01-29T05:44:33Z

Python file: get_followers.py

Onnowpurbo at 05:43, 29 January 2017

2017-01-29T05:43:20Z

Onnowpurbo at 05:42, 29 January 2017

2017-01-29T05:42:37Z

Onnowpurbo at 04:18, 29 January 2017

2017-01-29T04:18:22Z

Onnowpurbo at 04:06, 29 January 2017

2017-01-29T04:06:03Z

Onnowpurbo at 03:47, 29 January 2017

2017-01-29T03:47:16Z

Onnowpurbo at 07:02, 22 January 2017

2017-01-22T07:02:27Z

Onnowpurbo at 06:57, 22 January 2017

2017-01-22T06:57:45Z

@@ Line 331: / Line 331: @@
               edge_exists[key] = True
   view raw
 ==Python file: twitter_network.py==

@@ Line 280: / Line 280: @@
 ==Step 2. Process twitter data to generate an output file of relationships between twitter accounts==
-The script below will process the data collected from the twitter API and generate an edge list. That is a list of relationships between twitter accounts. A weight value is included, this value is the total number of followers for the first twitter account, this value is retrieved from the API. The weight value can be used later to prune the network graph.
+Script [https://gist.github.com/mjcreativeventures/58a037a03b63355e02a3 twitter_network.py] di bawah ini akan memproses data yang dikumpulkan oleh twitter API dan membuat edge list. List ini berisi hubungan antar twitter account. Weight value dimasukan, nilai ini merupakan jumlah total follower untuk twitter account pertama, nilai ini di ambil dari API twitter. Weight value akan digunakan nanti saat kita menggambar network graph.
   import glob

← Older revision		Revision as of 05:44, 29 January 2017
Line 261:		Line 261:


−	~~==Python file: get_followers.py==~~
−
	I ran this script twice first without a filter on the screen name but limiting the maximum number of following accounts to 20 then again but this time filtering for accounts starting with ‘TED’ (line 102) and allowing up to 200 following accounts to be queried. This will give a mix of TED and non-TED twitter accounts. Running the script:		I ran this script twice first without a filter on the screen name but limiting the maximum number of following accounts to 20 then again but this time filtering for accounts starting with ‘TED’ (line 102) and allowing up to 200 following accounts to be queried. This will give a mix of TED and non-TED twitter accounts. Running the script:

@@ Line 7: / Line 7: @@
   sudo pip install tweepy
   mkdir following
-  mkdir twitter_user
+  mkdir twitter_users
 Login ke https://dev.twitter.com/apps, dapatkan:

@@ Line 1: / Line 1: @@
-Generating a network graph of Twitter followers using Python and NetworkX
+==Persiapan==
+Instalasi
   sudo apt install python-pip
-  pip install --upgrade pip
+  sudo pip install --upgrade pip
-  pip install tweepy
+  sudo pip install tweepy
-Before you can access the twitter streaming api you need to register your client application (the python script) with Twitter. There is a tweepy authentication tutorial that explains step by step how to set up your app for oauth authentication.
+Login ke https://dev.twitter.com/apps, dapatkan:
-Once you have registered with Twitter go to My Applications page on Twitter and you should see your newly created application. Click on the link and you should see the application details page:
+* CONSUMER_KEY
+==Langkah Secara umum==
 * From initial seed account collect followers using the Snowball Sampling technique.
 * Process the collected twitter data to generate an output file of relationships between twitter accounts.
 * Visualize network data in a network graph using the NetworkX library.
 ==Cara Cepat==
@@ Line 35: / Line 31: @@
 ==Step 1. Collect follower data from the Twitter API==
-You will need to have API keys to be able to query the Twitter API. I have written in previous articles how to do this, e.g. Collecting tweets using Python.
+follower data akan di simpan di folder following, sesudah proses di jalankan akan data di simpan dalam CSV format.
   $ ls following/
@@ Line 58: / Line 50: @@
   XXXXXXXX        maroo_king      Marou
-The second directory is called ‘twitter-users’, it is a cache of twitter user details, each file contains cached data for a twitter user including friend and follower counts and a list of follower IDs (up to a maximum of 5000 follower IDs can be queried from the API).
+Directory yang kedua adalah ‘twitter-users’, menyimpan twitter user detail dalam format JSON.
   $ ls twitter-users/
@@ Line 85: / Line 78: @@
+   }
-Here is the script to collect this data:
+Script [https://gist.github.com/mjcreativeventures/41de04c6bbe47ee14411 get_followers.py] untuk mengumpulkan data adalah sebagai berikut,
   import tweepy
@@ Line 264: / Line 257: @@
           print 'Sorry, could not find twitter user with screen name: %s' % twitter_screenname
   view raw
- get_followers.py hosted with ❤ by GitHub
 ==Python file: get_followers.py==

← Older revision		Revision as of 04:18, 29 January 2017
Line 28:		Line 28:
	==Cara Cepat==		==Cara Cepat==

		+	python get_followers.py -s TEDxSingapore -d 3
		+	python twitter_network.py
		+	python visualize.py

@@ Line 25: / Line 25: @@
 * Visualize network data in a network graph using the NetworkX library.
-Step 1. Collect follower data from the Twitter API
+==Step 1. Collect follower data from the Twitter API==
 You will need to have API keys to be able to query the Twitter API. I have written in previous articles how to do this, e.g. Collecting tweets using Python.
@@ Line 56: / Line 56: @@
   -rw-r--r-- 1 mark mark  57K Jul 24 16:46 XXXXXXXX.json
   -rw-r--r-- 1 mark mark 6.3K Jul 24 17:01 XXXXXXXXXX.json
-−
-... Lots more ...
+ ... Lots more ...
-−
   $ cat twitter-users/XXXXXXXX
-−
-  "name": "TEDxSingapore",
-  "friends_count": 147,
+   "name": "TEDxSingapore",
-  "followers_count": 12814,
+   "friends_count": 147,
-  "followers_ids": [
+   "followers_count": 12814,
-   XXXXXXXXXX,
+   "followers_ids": [
-   XXXXXXXXXX,
+    XXXXXXXXXX,
-   XXXXXXXXX,
+    XXXXXXXXXX,
-   ...
+    XXXXXXXXX,
-   XXXXXXXXXX,
+    ...
-   XXXXXXXXXX
+    XXXXXXXXXX,
-  ],
+    XXXXXXXXXX
-  "id": XXXXXXXX,
+   ],
-  "screen_name": "TEDxSingapore"
+   "id": XXXXXXXX,
+   "screen_name": "TEDxSingapore"
 Here is the script to collect this data:
@@ Line 258: / Line 258: @@
   get_followers.py hosted with ❤ by GitHub
-Python file: get_followers.py
+==Python file: get_followers.py==
 I ran this script twice first without a filter on the screen name but limiting the maximum number of following accounts to 20 then again but this time filtering for accounts starting with ‘TED’ (line 102) and allowing up to 200 following accounts to be queried. This will give a mix of TED and non-TED twitter accounts. Running the script:
@@ Line 277: / Line 277: @@
   ... Lots more output ...
-Step 2. Process twitter data to generate an output file of relationships between twitter accounts
+==Step 2. Process twitter data to generate an output file of relationships between twitter accounts==
 The script below will process the data collected from the twitter API and generate an edge list. That is a list of relationships between twitter accounts. A weight value is included, this value is the total number of followers for the first twitter account, this value is retrieved from the API. The weight value can be used later to prune the network graph.
@@ Line 333: / Line 333: @@
 twitter_network.py hosted with ❤ by GitHub
-Python file: twitter_network.py
+==Python file: twitter_network.py==
 The output generated from this script:
@@ Line 351: / Line 351: @@
   ...
-Step 3. Visualizing the Network using the NetworkX library
+==Step 3. Visualizing the Network using the NetworkX library==
 We now have all the data we need to generate a network graph. Here are the steps used to visualize the network graph:
@@ Line 452: / Line 452: @@
 visualize.py hosted with ❤ by GitHub
-Python file: visualize.py
+==Python file: visualize.py==
 * Line 7 Load edge data from disk

← Older revision		Revision as of 07:02, 22 January 2017
Line 261:		Line 261:

	I ran this script twice first without a filter on the screen name but limiting the maximum number of following accounts to 20 then again but this time filtering for accounts starting with ‘TED’ (line 102) and allowing up to 200 following accounts to be queried. This will give a mix of TED and non-TED twitter accounts. Running the script:		I ran this script twice first without a filter on the screen name but limiting the maximum number of following accounts to 20 then again but this time filtering for accounts starting with ‘TED’ (line 102) and allowing up to 200 following accounts to be queried. This will give a mix of TED and non-TED twitter accounts. Running the script:
		+
		+	$ mkdir following
		+	$ mkdir twitter_user

	$ python get_followers.py -s TEDxSingapore -d 3		$ python get_followers.py -s TEDxSingapore -d 3

← Older revision		Revision as of 06:57, 22 January 2017
Line 8:		Line 8:

	sudo apt install python-pip		sudo apt install python-pip
		+	pip install --upgrade pip
	pip install tweepy		pip install tweepy