An LSTM-Based Plagiarism Detection via Attention Mechanism and a Population-Based Approach for Pre-training Parameters with Imbalanced Classes

• Plagiarism is one of the leading problems in academic and industrial environments, which its goal is to find the similar items in a typical document or source code. This paper proposes an architecture based on a Long Short-Term Memory (LSTM) and attention mechanism called LSTM-AM-ABC boosted by a population-based approach for parameter initialization. Gradient-based optimization algorithms such as back-propagation (BP) are widely used in the literature for learning process in LSTM, attention mechanism, and feed-forward neural network, while they suffer from some problems such as getting stuck in local optima. To tackle this problem, population-based metaheuristic (PBMH) algorithms can be used. To this end, this paper employs a PBMH algorithm, artificial bee colony (ABC), to moderate the problem. Our proposed algorithm can find the initial values for model learning in all LSTM, attention mechanism, and feed-forward neural network, simultaneously. In other words, ABC algorithm finds a promising point for starting BP algorithm. For evaluation, we compare our proposed algorithm with both conventional and population-based methods. The results clearly show that the proposed method can provide competitive performance.

Subject headings

TEKNIK OCH TEKNOLOGIER  -- Maskinteknik -- Energiteknik (hsv//swe)

ENGINEERING AND TECHNOLOGY  -- Mechanical Engineering -- Energy Engineering (hsv//eng)

NATURVETENSKAP  -- Data- och informationsvetenskap -- Datavetenskap (hsv//swe)

NATURAL SCIENCES  -- Computer and Information Sciences -- Computer Sciences (hsv//eng)

Keyword

Artificial bee colony

Attention mechanism

Back-propagation

LSTM

Plagiarism

Feedforward neural networks

Intellectual property

Learning algorithms

Optimization

Academic environment

Attention mechanisms

Back Propagation

Feed forward neural net works

Imbalanced class

Industrial environments

Meta-heuristics algorithms

Plagiarism detection

Pre-training

Training parameters

Long short-term memory

Publication and Content Type

ref (subject category)

kon (subject category)