Least Information Redundancy Algorithm of Printable Shellcode Encoding for x86

EasyChair Preprint no. 11033

Abstract

Shellcode is a fundamental component of computer security that is used to exploit vulnerabilities in software systems. It refers to the code that an attacker injects into a vulnerable program, typically to gain unauthorized access or to execute arbitrary commands on the system. Shellcode is written in machine code and is usually very small in size, making it dicult to detect by security software. Printable shell- code is a type of shellcode that consists entirely of printable ASCII characters(0x21-0x7E), typically including letters, numbers, and punc- tuation marks. The primary advantage of printable shellcode is that it can be embedded in data streams that may be parsed or manipulated by applications that would otherwise lter or modify non-printable char- acters. Most used printable shellcode generation methods are based on encoding algorithms, such as Riley Eller algorithm integrated into Metas- ploit. Realated work on printable shellcode aim to reduce size of encoded shellcode, ignoring the limitation of encoding information entropy in the circumstance of printable shellcode. In this paper we demonstrate the least infomation redundancy of printable shellcode exists and we present a new encoding method which surpass previous encoding algorithms with the smallest size.

Keyphrases: encoding, shellcode, x86

@Booklet{EasyChair:11033,
  author = {Yuanding Zhou},
  title = {Least Information Redundancy Algorithm of Printable Shellcode Encoding for x86},
  howpublished = {EasyChair Preprint no. 11033},

  year = {EasyChair, 2023}}