Towards Identification of Political Posts in Social Media Text Using Deep Learning Models

arXiv:2411.04542 · cs.HC · Submitted 2024-11-07 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: Computer Science. The day in computer science: software, systems and theory on arXiv.

Tom: We're looking at the paper: "Towards Identification of Political Posts in Social Media Text Using Deep Learning Models".

Jane: Here's the gist of the paper. Automatic Identification of Political Hate Articles from Social Media using Recurrent Neural Networks is presented to analyze and predict political ideology from social media text in Bangla, addressing a gap in research for this language The word embedding feature with LSTM outperforms all other models with 88.28% accuracy Feature Extraction Techniques The work extracts three types of features to identify political ideology from the text, such as the stylometric feature, the word-embedding feature, and the TFIDF feature The authors plan to create a dataset from Facebook containing political posts and neutral posts The pre-processing steps involve filtering out all characters except Bangla characters, tokenizing texts, and removing stop words using Bangla stop words from GitHub repository

17: Model Architectures Implemented The study implements two different approaches to identify political ideology from the text, namely a deep learning model and a traditional machine learning model classifier The deep learning model architecture involves LSTM and GRU models, while the traditional machine learning model uses SVM and NB models The specific architectures compared include Model with Stylometric Features, Model with TF-IDF feature, and Model with Word Embedding feature Experimental Results Analysis The performance evaluation shows that the LSTM model with the word embedding approach outperforms all other models Specifically, LSTM and GRU models outperform SVM and NB models when using the word embedding feature The highest accuracy obtained by our proposed methods is 88.28% Comparison with Existing Research The comparison of our methods with other works on determining political ideology from the text in Table V shows that among those done for Bangla language, our work is showing comparatively better performance The authors state that there is a lack of research in the Bangla language for this problem The dataset created contains 1980 texts where the number of political class labels is 814 and the number of neutral class labels is 1166 Future work will involve increasing the size of the dataset and implementing more models to obtain better accuracy In conclusion, various types of machine learning classifiers and deep learning models are used to identify the ideology in the text The ideology will be used to deduce which party is better for the country, and we can predict the political party affiliation from text using these methods The authors have made their dataset public so that future work on this domain can be done using this dataset TABLE I: Data Distributions Class Label Number of posts Political Hate Post 814 Neutral Post 1166 TABLE II: Performance measure for Stylometric Feature: Model Feature Vector Accuracy F1-score LSTM Styometric Feature 73.18% 61.96% TABLE III: Performance measure for Word Embedding Feature: Model Feature Vector Accuracy F1-score LSTM Word Embedding Feature 88.28% 85.41% TABLE IV: Performance measure for TF-IDF feature: Model Feature Vector Accuracy F1-score GRU TF-IDF Feature 69.94% 46.15% TABLE V: Comparison of Proposed Method with Previous Works Language Proposed by Methodology Feature Extraction Accuracy Bangla

tasnim2021political: None CBOW, Skip-Gram 76.22% Bangla

rahman2018datasets: KNN, SVM, RF TF-IDF 71.21% English

baly2020we: LSTM, BERT Word Embedding 72.37% Bangla Proposed Method LSTM, GRU, SVM, NB Stylometric Feature, TF-IDF Feature, Word Embedding Feature 88.28% REFERENCES

1: A.

Tom: Abbasi and H.

Jane: Chen.

Tom: “Applying Authorship Analysis to Arabic Web Content”.

Paper summary: Tom: So, we're looking at this paper called "Towards Identification of Political Posts in Social Media Text Using Deep Learning Models". It seems the whole point is using deep learning models to figure out political ideology from social media text in Bangla.

Jane: Right, and it addresses a gap because analyzing political ideas on social media is really important now, especially for languages like Bangla where research has been underdeveloped. The authors claim they built a way to do this by creating their own dataset from Facebook posts, both political and neutral ones.

Lu: And they didn't just stop at gathering data; they extracted three different types of features from the text to feed into their machine learning models: a bag-of-words technique, stylometric features capturing the writing style, and word embedding approaches to turn the text into feature vectors.

Meng: Three features is a lot of work for data preparation. I'm curious about how feasible this is when dealing with social media text that's usually really messy. What kind of cleaning were they doing before feeding it into those models?

Lalam: They applied various pre-processing steps to the data, filtering out everything except Bangla characters and then tokenizing the text and removing stop words using Bangla stop words from a GitHub repository.

Tom: Okay, so the next thing is what kind of machine learning architectures they tested. The paper mentions they looked at both deep learning models, specifically LSTM and GRU models, as well as traditional classifiers like Support Vector Machines and Naive Bayes.

Jane: That's interesting because the paper suggests that deep learning-based recurrent neural models can automate political ideology extraction because they don't necessarily require those handpicked features you mentioned earlier. It’s about letting the model figure out what matters from the text itself.

Lu: Exactly, and they compared models with stylometric features, TF-IDF features, and word embedding features to see which one performed best in identifying that political ideology.

Meng: So they are running a comparison test across three different feature sets using both deep learning and traditional machine learning classifiers? That sounds like a pretty thorough setup for testing the approach.

Lalam: The results show that the LSTM model combined with the word embedding approach outperformed all other models, achieving an accuracy of eighty-eight point two eight percent.

Tom: eighty-eight point two eight percent is a solid number when you’re dealing with text classification. That suggests that using those word embedding features with an LSTM architecture was the most effective combination they tested for this task in Bangla.

Jane: It's significant because the paper highlights that among existing research done on Bangla, their proposed method shows comparatively better performance. They also note there was a lack of research in this specific area for the Bangla language.

Lu: The dataset they built is substantial, containing one thousand nine hundred eighty texts where eight hundred fourteen are political hate posts and one thousand one hundred sixty-six are neutral posts, which gives them a good balance for training those models <ref:2411.04542#pg3>.

Meng: So you have this dataset of nearly two thousand texts to work with. But what’s the catch? They mention they plan future work involves increasing the size of the dataset and implementing more models to get even better accuracy.

Tom: That makes sense, scaling up data always helps, but I wonder how much computational power that would actually cost for their deep learning models to run on large volumes of text.

Jane: The paper does mention that the ideology identified by these methods could be used to deduce which political party might be better for the country and predict political party affiliation from text.

Lu: That’s a big implication—moving beyond just identifying hate speech to understanding broader political sentiment and affiliations in this language. It suggests that analyzing social media can give us real insight into public opinion dynamics.

Meng: From a deployment standpoint, if we were to take this system and put it on a live platform, the biggest question would be how robust it is when faced with the kind of noise you described in their pre-processing steps.

Lalam: The authors have made their dataset public so that other researchers can use it for future work on this domain.

Tom: So, to wrap up this overview of "Towards Identification of Political Posts in Social Media Text Using Deep Learning Models", we’ve seen they used a combination of deep learning and traditional methods with word embedding features on a Bangla dataset to tackle the problem.

Jane: And the main point is that this specific combination hit an accuracy of eighty-eight point two eight percent, showing promise for analyzing political ideology in this language.

Tom: It shows that for language-specific tasks like this, feature engineering matters, and in this case, word embeddings provided the best performance compared to stylometric or TF-IDF features.

Lu: The core contribution seems to be validating that deep learning models can indeed perform well here without needing perfectly engineered handcrafted features upfront.

Meng: It’s a good validation if it holds up when we move away from controlled datasets and onto real traffic where the noise level is much higher than one thousand nine hundred eighty texts <ref:2411.04542#pg3>.

Lalam: The paper lays out the process clearly, from data creation to feature extraction, and their method is available for anyone interested in testing this approach further.

Conclusion: Tom: So, we're talking about this paper titled "Towards Identification of Political Posts in Social Media Text Using Deep Learning Models". It’s about using deep learning to figure out political ideas hidden in Bangla social media posts.

Jane: Right, so the authors built a system to see if a piece of text leans toward political hate or something neutral. They used data from Facebook posts, both the angry ones and the regular ones.

Lu: The core idea is that you can feed this text into models like LSTM or GRU to classify it by checking how it's structured and what words are used.

Meng: I remember they tested a few different ways to give the model information, like using word embeddings or just looking at the style of the writing.

Lalam: And they found that combining the LSTM model with word embedding features gave them a pretty strong accuracy score of eighty-eight point two eight percent on their testing.

Tom: eighty-eight point two eight percent is solid when you’re trying to sort through complicated social media chatter in Bangla. So, what does this actually mean for us?

Jane: It means we can start to automatically spot political posts and get an idea of the underlying political leanings without having to read every single post manually.

Lu: It suggests that these deep learning methods are getting good at understanding the nuances of language specific to Bangla in this context.

Meng: But I wonder how robust this works when you try to run it on real-world social media data that's messy and constantly changing, not just their clean set of texts.

Lalam: The authors did make their dataset public so other people can check the results themselves and see if it holds up outside of their controlled experiment.

Tom: That’s the big thing—the methodology is open for inspection. But the next step for them is going to be making that dataset much bigger and trying out even more complex models to push that accuracy higher.

Jane: So, they're not just stopping here; they're focused on scaling up the data and refining the models to see what else it can uncover about political sentiment.

Tom: Exactly. It’s a roadmap for taking this idea from a successful experiment into something that actually has practical use in understanding political discourse in Bangla.

Sultan Ahmed, Salman Rakin, Khadija Urmi, Chandan Kumar Nag, Dr. Md. Mostofa Akbar

University of Maryland Baltimore County · Bangladesh University of Engineering & Technology · BRAC University

cs.HC

Submitted: 2024-11-07

Updated: 2026-11-16

Comments: 9 Pages !

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Importance score: 77/100

The gist: Automatic Identification of Political Hate Articles from Social Media using Recurrent Neural Networks is presented to analyze and predict political ideology from social media text in Bangla…

Key concepts

Word Embedding Feature
This technique converts words into numerical vectors that capture their semantic meaning based on context. The paper used this feature to understand the political sentiment in Bangla text more effectively than traditional methods, leading to the best performance.
LSTM and GRU Models
These are types of deep learning neural networks specifically designed for processing sequential data like text. LSTM (Long Short-Term Memory) and GRU (Gated Recurrent Unit) help the model remember important information from earlier words in a sentence to make better predictions.
Stylometric Feature
This feature analyzes the style of writing, such as word choice and sentence structure, rather than just the words themselves. The paper tested this alongside other features to see how much stylistic pattern helps in identifying political ideology in Bangla posts.

Terminology

Summary

Automatic Identification of Political Hate Articles from Social Media using Recurrent Neural Networks is presented to analyze and predict political ideology from social media text in Bangla, addressing a gap in research for this language The word embedding feature with LSTM outperforms all other models with 88. p. 028% accuracy

Feature Extraction Techniques

The work extracts three types of features to identify political ideology from the text, such as the stylometric feature, the word-embedding feature, and the TFIDF feature The authors plan to create a dataset from Facebook containing political posts and neutral posts The pre-processing steps involve filtering out all characters except Bangla characters, tokenizing texts, and removing stop words using Bangla stop words from GitHub repository [17]

Model Architectures Implemented

The study implements two different approaches to identify political ideology from the text, namely a deep learning model and a traditional machine learning model classifier The deep learning model architecture involves LSTM and GRU models, while the traditional machine learning model uses SVM and NB models The specific architectures compared include Model with Stylometric Features, Model with TF-IDF feature, and Model with Word Embedding feature

Experimental Results Analysis

The performance evaluation shows that the LSTM model with the word embedding approach outperforms all other models Specifically, LSTM and GRU models outperform SVM and NB models when using the word embedding feature The highest accuracy obtained by our proposed methods is 88.28%

Comparison with Existing Research

The comparison of our methods with other works on determining political ideology from the text in Table V shows that among those done for Bangla language, our work is showing comparatively better performance The authors state that there is a lack of research in the Bangla language for this problem The dataset created contains 1980 texts where the number of political class labels is 814 and the number of neutral class labels is 1166 Future work will involve increasing the size of the dataset and implementing more models to obtain better accuracy

Conclusion and Future Directions

In conclusion, various types of machine learning classifiers and deep learning models are used to identify the ideology in the text The ideology will be used to deduce which party is better for the country, and we can predict the political party affiliation from text using these methods The authors have made their dataset public so that future work on this domain can be done using this dataset

TABLE I: Data Distributions

Class Label Number of posts

Political Hate Post 814

Neutral Post 1166

TABLE II: Performance measure for Stylometric Feature:

Model Feature Vector Accuracy F1-score

LSTM Styometric Feature 73.18% 61.96%

TABLE III: Performance measure for Word Embedding Feature:

Model Feature Vector Accuracy F1-score

LSTM Word Embedding Feature 88.28% 85.41%

TABLE IV: Performance measure for TF-IDF feature:

Model Feature Vector Accuracy F1-score

GRU TF-IDF Feature 69.94% 46.15%

TABLE V: Comparison of Proposed Method with Previous Works

Language Proposed by Methodology Feature Extraction Accuracy

Bangla [tasnim2021political] None CBOW, Skip-Gram 76.22%

Bangla [rahman2018datasets] KNN, SVM, RF TF-IDF 71.21%

English [baly2020we] LSTM, BERT Word Embedding 72.37%

Bangla Proposed Method LSTM, GRU, SVM, NB Stylometric Feature, TF-IDF Feature, Word Embedding Feature 88.28%

REFERENCES

[1] A. Abbasi and H. p. 5 Chen. “Applying Authorship Analysis to Arabic Web Content”. p. 5 In: Intelligence and Security Informatics, Berlin, Heidelberg. p. 5 2005

[3] K. Alsmearat, M. p. 5 Al-Ayyouba, R. p. 5 Al-Shalabi, and G. p. 5 Kanaanbt. “Author gender identification from Arabic text”. p. 5 In: Journal of Information Security and Applications 35. p. 58 (2017), pp. p. 5 85–95

[7] O. Sharma. “A New Activation Function for Deep Neural Network, Faridabad, India”. p. 7 In: International Conference on Machine Learning, Big Data, Cloud and Parallel Computing (COMITCon), Feb 14-16. p. 7 2019

[8] Alexander G. de G. Matthews, Mark van der Wilk, Tom Nickson, Keisuke Fujii, Alexis Boukouvalas, Pablo Leon-Villagr a, Zoubin Ghahramani, and James Hensman. p. 6 “GPflow: A Gaussian Process Library using TensorFlow”. p. 6 In: Journal of Machine Learning Research 18. p. 640 (2017), pp. p. 5 1–6 URL: http://jmlr. p. 6org/papers/v18/1-6

[9] Anees Ul Hassan, Jamil Hussain, Musarrat Hussain, Muhammad Sadiq, and Sungyoung Lee. “Sentiment analysis of social networking sites (SNS) data using machine learning approach for the measurement of depression”. p. 6 In: 2017 international conference on information and communication technology convergence (ICTC). p. 6 IEEE. 2017, pp. 138–140

[13] T. Mikolov, K. p. 7 Chen, G. Corrado, and J. p. 7 Dean. “Efficient Estimation of Word Representations in Vector Space”. p. 7 In: arXiv preprint arXiv:1301. p. 73781. 2013

[2411-04542] The paper is 2411.

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Improve political ideology determination by integrating word embedding features with deep learning models like LSTM and GRU, as this combination outperforms all other models with 88.28% accuracy. This allows the system to capture both semantic and contextual relationships in Bangla text more effectively than stylometric or TF-IDF methods alone.

  2. Enhance model robustness by comparing traditional machine learning classifiers, such as SVM and NB models, against deep learning architectures like LSTM and GRU to determine their relative strengths for different feature sets. This comparison helps identify which model architecture is most suitable when using stylometric features versus word embedding features for Bangla text analysis.

  3. Develop a system capable of predicting the political party affiliation from text by extending the current work's capability beyond simple positive/negative comment prediction. This future direction aims to predict the political party affiliation from text using these methods, allowing users to deduce which party is better for the country based on online discourse analysis.

Sources

Related papers