Coati optimization algorithm to recover the secret key of the classical transposition cryptosystem

Abstract

Abstract This study presents an enhanced approach for recovering the secret key of the standard classical transposition cryptosystem using a recently introduced metaheuristic, the coati optimization algorithm. To adapt the coati optimization algorithm for cryptanalysis, its core framework is specifically customized for the considered transposition cryptosystem. The performance of the tailored coati optimization algorithm is benchmarked against three established metaheuristics, namely genetic algorithm, cuckoo search and particle swarm optimization based on accuracy, effectiveness, and efficiency. Experimental results demonstrate that the proposed method performs competitively, often surpassing genetic algorithm, cuckoo search, and particle swarm optimization across all evaluation metrics. However, the current experimental evaluation is limited to English-language ciphertexts using English n -gram statistics; therefore, the applicability of the proposed framework to multilingual cryptanalysis remains a direction for future investigation.