Bin-tropy analysis to detect cryptomalware

Bin-tropy calculation:

Analyzing the file contents in each path or drive is one of the preliminary steps in detecting the crypto-malware execution. A main difference between an encrypted file and a normal file is that the randomness in characters in encrypted file is greater than expected in a normal file.

The Binary Entropy calculation is done using statistical test suite based on Discrete Fourier tranformation of the file sequence.

Steps involve :

1.  Binary sequence of the file content to be analysed. Each 0 and 1 in the sequence to be converted to -1 and 1 respectively. For eg : Seq = 10110101 converted as  Seq = 1, -1, 1,1,-1,1,-1,1.
2. Apply discrete fourier transform (DFT) to the sequence so that a continuous sine wave can be produced. This would reveal periodic repetition in the input data. In this case, periodic components of the sequence of bits at different frequencies.
3. Calculate the modulus of the substring of the DFT sequence generated, which would give the sequence of peak heights.
4. Compute threshold peak height value (95% peak height value).                                                                                           Threshold  = √(log 1/0.05)n
5.  Under the assumption of randomness, 95% of the peak heights obtained from the sequence should be less than this threshold value.
6. To compare the theoretical number of peaks (95% of the peak heights) that are less than threshold, with the actual number of peaks that are less than threshold, compute        theoretical (N)   =    .95 (n / 2)   , expected number of peaks with heights less than threshold   actual (N_1)    =  the actual number of peaks that are less than T (as observed)
7. Find d = normalized difference between the expected and theoretical number of frequency components that are beyond the 95% threshold.
8. Compute complementary error function value as “E = erfc( abs(d)/√2)”

If the computed E value is greater than 0.01, then conclude that the input sequence is random (encrypted).  Else non-random sequence (normal).

d value that is too low means that there are too few peaks below T, and too many peaks above T.

Limitations of the bin-entropy detection method:

Not perfect in cases of very small files or user encrypted files.

For eg : say a txt file with “SSN : 0123456789″.
Randomness test would fail with E > threshold because within the 14 characters, except “S”, all of them are unique,  thus random in nature. Even though it is a valid text, the entropy value would be higher than threshold.
In case of user encrypted files, Entropy would already be higher, so if a malware starts encrypting the same file again,
the script cannot differentiate between “legitimate user encryption” and “unauthorized encryption” thus wouldn’t be
efficient.

Source : Python Sourcecode for the implementation can be found in https://github.com/EC700/Charlie-2/tree/master/Entropy

References: Bin Entropy calculated based on ‘Statistical Test Suite for Random and Pseudorandom Number Generators for Cryptographic Applications’
published by National Institute of Standards and Technology, U.S Department of Commerce
Source : http://csrc.nist.gov/groups/ST/toolkit/rng/documents/SP800-22rev1a.pdf

Malware File Detection

Packed PE file detection :

Advanced malwares evade detection to common methodologies/softwares by polymorphism or obfuscation. Malwares can be transferred and executed across different victims as packed executables which can avoid detection by signature analysis. We can detect packed files by extracting specific elements of packing and calculating entropy of the entry point.

Entropy is the measure of unpredicatability or randomness in an information stream.

• PEID –  most common tool to detect PE packed or encrypted malware by signature detection
• MRC – structured analysis by file entropy. If encryption or suspicious packing is detected, weight value is added so that entry becomes high.

Recent malware execution files are packed to avoid malware detection and fast propagation. PE packed file is compressed and encrypted. Thus the data within the pack is random, which can be found by byte entropy calculation.  The original execution code is compressed in the “Packed Data” section in the image and it has to be unpacked for binary analysis.

The entropy values range between 0 – 8. Specific file types will have entropy values within smaller bands in that range.

Methods used to differentiate normal file and packed file are :

entropy calculation of the whole file, entropy calculation of the entry section

Entropy calculation at the entry section gives the better option.

Essential elements of a packed PE file  is IMAGE_HEADER_SECTION. One important feature is that the packed file is executable only when WRITE property is included in the header section. So this is one of the main points checked in the detection of packing.

Packed PE file detection techniques – entropy based detection and characteristic based detection (behaviour based ).

Entropy based detection – By previous data sets, Packed PE file has entropy > 6.85 in the entry point.

Characteristic based detection – there are different characteristics/action taken in unpacking file in normal PE file and packed PE file. In packed executables, WRITE property is required to do the unpacking and executing. Therefore packed PE file can be selected by checking the entry point section and verifying if it includes WRITE property.

In a Normal file, EXECUTE, READ, CODE or DATA occurs commonly.

Packed PE detection flow chart : faster and efficient detection. Receives the whole drive and sequentially checks each file in the disk for PE signature(by looking for the “MZ” file signature).

If it is a PE file, then find the entry point section. Check if entry point section has WRITE property included, and if so, calculate entropy of that section to find if it is greater than the threshold for normal PE file.(6.85)

This method has better detection rate and time, when done on huge dataset / whole harddrive.

---

 Implementation of crypto-malware detection using file entropy changes in the project 

In the project, one of the pre-detection methods is Detection by file entropy changes. The steps involve :

1. Calculating binary entropy of the files (.txt, .doc, .pdf) in the path or drive specified
2. Detecting ASCII percent change of the files.

The process involve detecting the repetitive patterns in the binary sequence that would indicate a
deviation from the assumption of randomness.

Implementation of the binary entropy analysis to detect cryptomalware execution in real time is here.

Source:

http://www.forensickb.com/2013/03/file-entropy-explained.html

http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.120.9861&rep=rep1&type=pdf

http://csrc.nist.gov/groups/ST/toolkit/rng/documents/SP800-22rev1a.pdf

Your files are encrypted!!! Pay the Ransom!!

              Ransomware - Pay or lose all your files.

Ransomware - Malware which primarily encrypts the files in the systems and asks for the ransom to be paid within some "estimated" time in exchange for decryption key. 

Ransomware is spread across victims by email or botnets. Malware is sent as a attachment in a email or as "drive-by download" by some malicious website.

Once the malware is downloaded into the system, it starts installation in the background. Then creates a "system id" by creating "hash of the system config parameters" so that it would be unique to each system the malware infects. Then it tries to connect to "C & C server" controlled by the attacker or the malware's author. It sends the "system id" to the C&C, which in turn creates a public key and private key pair for the system and saves it in the server. The public key is sent to the malware in infected system. 

The malware uses the pub key to encrypt all the files(with .pdf, .jpg, .doc extensions) in the system. Once the encryption is done, the malware shows a alert that says "All the files are encrypted." and ransom has to be paid within the said time, or else the decrypted key saved in the remote server would be deleted and thus all the files would be lost forever.

If there are other systems connected to the infected system, then the whole infrastructure would be compromised and encrypted.  

Once the system is compromised, there is no other way to get the files back other than getting the decryption key.
Over time, malware has evolved along with the anti-virus, anti-malware protections.  These were perfected to be hidden, country specific, target specific, etc.
Most commonly known ransomwares are cryptolocker, cryptowall, CTB.

More on specifics of malware analysis and detection in the next post!

In security, with time trending becomes obsolete soon!!!! 

Nmap - Get it on!

Nmap  -  What is it and how to make best of it?

Nmap is the network mapper that is used to discover the number of hosts and devices on the network that it is accessing, thus mapping the whole network.
Let's not jump into the details of the installation as that is not the information everyone is looking around. Some of you might even have it on the system beforehand but not using it. So here are some details on the information it could provide us with.

scan your own host to determine the open ports and connections:

nmap -sV -p 1-65535 localhost/24

Here sV - get the service version that is running on open ports
-p - specify the port numbers to be scanned.
localhost - target to be probed.  (24 means the first 3 octets are part of the network and the remaining 1 octet is open for the other hosts on the network ). This way you can scan multiple hosts at the sametime.

It will return a series of open ports and service running on the ports.
Some basics you got to know to understand the nmap.
1. nmap establishes a 3-way TCP handshake with the "host" as any other client would do.
       different ports - service that should be noted :
                    port 80   - web server
                         81   - firewall (this drops all the packets that could potentially harm the server)
 Any connection to a webserver passes through the firewall(80). Closed ports mean that there are no service running on those ports.
     
2. Host specified can either be a ip address or the url.
3. nmap basically probes every port on the host network to determine what service is established.
                 Open ports means - service/port is open for any public connection and is accepting any TCP/UDP client connections.
                 Closed ports means - port is not providing any service or application but can be accessed to find some other information about the OS or server
                 filtered ports means - these ports cannot provide any kind of information as the firewall/router drops any packets trying to access these ports.
                 open/filtered means - the probe cannot determine the type of the port. The open port doesn't give any response which means the firewall simply drops the packets sent to the port.

4.  nmap -A google.com
     Gets you the OS detection of the target server.
5. nmap -iL textfile.txt
     Reads the list of targets from the text file.
6. nmap -sA target.com
     Tells you if the target server is protected by a firewall .
     If such a firewall exists then use nmap -PN target.com to scan the network. How is this different from the normal scan?
   
Almost on any information associated with the ports, server OS, servicces attached to the ports can be accessed without any credentials.

Nmap - SQL access:
We can do a whole lot with the Nmap to get all  the more information about SQL server/databases access. We can pass additional scripts along with the nmap command on the port -p as "ms-sql-info"  -  Get database information
                   "ms-sql-brute"  -  Pass brute force attacks to check for weak credentials within the database
                   "ms-sql-empty-password" - Check if the database has empty passwords set for any table/user. Once we know which user has no password set,we can try to detect the databases or privileges that user has been provided.
                   "ms-sql-hasdbaccess.nse --script-args mysql.username=" - Check the dbaccess provided to the user.
                    "ms-sql-tables --script-args mysql.username=" - gives the table names and description in each db under the user.
                 
Any details obtained from these scripts could provide the entire infrastructure and any credential associated. If there is a empty password set to any user in the db schema, it could provide the whole table schema which is very scary and harmful.
There are whole lot of customized scripts that can get the password hashes from the server and thus providing direct access into the server to any outsider.

All of this information on this Nmap/mysql access pings are provided so that any security assessment can be made. Don't try to hack into anyone's server. Be it productive.

Remember, Every code is breakable. That doesn't mean you have to be the black hat hacker!!!!!!!

More exploits coming soon...

"THE MASK" - Real data breach threat.

The MASK - Most sophisticated(frightening) attack till data!

 When we say "data breach" these path months,we think of "Target, Neiman Marcus" and all other typical public use places. There has been a major threat which has attacked all the nation's most classified data , research interests, financial settlements and highly valued data within government offices, diplomats, educational institutions,..etc.  Irony is this attack is been under the radar "invisible" almost 6 years in almost every country / systems possible.

What is MASK ?

• The Mask - advanced threat actor that has been involved in cyber-espionage operations since at least 2007, was found only by the end of 2013. Thought to be from Spanish group because of the word “Careto” used in the some of the modules. 

• Has infected 31 countries and 1000 + IPs worldwide.

• Most Advanced threat at current time.

• Suspected  to be the nation state attack - because it is sophisticated, highly organized, .

Targets of the Attack:

        

    Once the malware is active in the system,it collects a list of documents from the infected system, including encryption keys, VPN configurations, SSH keys. Some of this files are related to custom military/government-level encryption tools.

• *.AKF,*.ASC,*.AXX,*.CFD,*.CFE,*.CRT,*.DOC,*.DOCX,*.EML,*.ENC,*.GMG,*.GPG,*.HSE,*.KEY,*.M15,*.M2F,*.M2O,*.M2R,*.MLS,*.OCFS,*.OCU,*.ODS,*.ODT,*.OVPN,*.P7C,*.P7M,*.P7Z,*.PAB,*.PDF,*.PGP,*.PKR,*.PPK,*.PSW,*.PXL,*.RDP,*.RTF,*.SDC,*.SDW,*.SKR,*.SSH,*.SXC,*.SXW,*.VSD,*.WAB,*.WPD,*.WPS,*.WRD,*.XLS,*.XLSX
• Targets : Government institutions, Diplomatic offices , energy and gas companies, research institutions.

Detection : Extremely difficult because of the stealth capabilties.  Malware changes when downloaded to new systems. So its sign is never the same.                                                       

Infestation :   

    It uses multiple vectors 

- phishing emails

- exploit old adobe flash player vulnerability (first used to break chrome sandbox) Chrome broken

- prompting the user for java update / chrome plugin 

- Subdomain of genuine websites. Guardian links used in malware         

Malware uses the packages for spreading through all the connected systems:

Careto and SGH. Careto – general purpose backdoor that collects all the information and executes arbitrary codes by remote C&C servers. SGH -  primarily works in Kernel and maintains its own connection to the C&C.

Files from these packages are signed with a certificate as "valid".     

Each works standalone without other.   

What does it do ?              

Once active, Mask can intercept network traffic, keystrokes, intercept Skype conversations, analyse WiFi traffic, fetch all information from Nokia /Android devices, screen captures and monitor all file operations.
Collects large chunks of data from the victim including encryption keys, VPN configurations,SSH keys and so.            

Complexity : An extremely sophisticated piece of malware, a rootkit, a bootkit, Windows versions, Mac OS X and Linux versions and possibly versions for Android and iPad/iPhone (iOS).

Customized attack on old Kaspersky lab products to hide in the system “invisible”. This is the step where Kaspersky got alert that someone is exploting their vulnerability and eventually discovered this malware.

Supports plug-ins,configuration files, built-in functionalities and any additional modules could be uploaded.                                                                                                                                                                                                                                                     

Stealth and platform independency:           

         The Digital signature is given as issued by a fake "Techsystem" Company.

Careto conquest:

 

                  Sample CARETO code "Careto ..." and with all other modules are "encrypted".

Careto - Map of Conquest.

How does communication protocols exchange streams of data.?

Three Way Handshake :

TCP/IP is the communication protocols used in connecting hosts between internet. Every OS in the market comes with the inbuilt TCP stack in them. Without the kernel's TCP network ,there is no possible data exchange can happen between devices, be it mobile devices, laptops, desktops and any other network hardwares.
Among the 4 layers in the suite, the transport layer takes the responsibility of transmission,error detection,recovery,reliability and session control.

How does the data transmission really takes place?
The blocks of data are sent through the transport layer. Each segment of a message sent from the machine contains a checksum which checks if the data at the receivers's end is correct. If the data received at the destination is undamaged, then an acknowledgement will be sent to the sender machine, else the receiver discards the segment and the sender machine sends all the segments for which a positive acknowledgement has not been received.
This is the frame for a 3-Way HandShake. Three distinct segments are exchanged between the hosts for reliable TCP session to be established.

EVENT	DIAGRAM
Host A sends SYN to Host B Host B receives A's SYN Host B sends a acknowledgement SYN-ACK to Host A Host A receives B's SYN Host A sends a acknowledgement ACK to Host B Host B receives A's ACK.  TCP socket connection is ESTABLISHED.	TCP Three Way Handshake (SYN,SYN-ACK,ACK)

Steps in the handshake:

Machine A sends a SYN signal to initiate the connection to the target Machine B.
( 1 Shake)
SYN - synchronize sequence number . Its just the sequence number it starts the segment with.
Machine B acknowledges the communication with the SYN-ACK signal. It responds along with sequence number it will start with. (2 Shake)
Machine A, after receiving the acknowledgement from B, checks the SYN number from B and acknowledges the initial sequence number of B. A sends the final ACK signal and starts the data transferring in segments(bits). (3 Shake)

That's why this is called a 3 way Handshake (SYN - SYN - ACK)

TCP/IP is the de-facto standard for transmitting data over internet.

Pakistan Hijacked Youtube - Prefix Hijack

Even a single fixed-route attack can destabilize a Network....!

Everyone remembers the time in 2008 when Pakistan Gov. put down orders to officially block YouTube broadcast in Pakistan. In response to this, Pakistan telecom blocked the YouTube telecast by the very old Prefix hijack, which resulted in YouTube being blocked.

How did this happen?

Pakistan Telecom provider tried to block YouTube in their homeland by advertising an unauthorized prefix route, to its providers. This newly defined prefix route ( 208.65.153.0/24) happens to be the more perfect destination route of the Youtube server ( 208.65.152.0/22). The upstream providers of the country relayed it to the Internet World, rerouting all of YouTube's traffic to Pakistan Telecom, thus blocking YouTube for all of its users worldwide.

Within 2 minutes from the first relay of the bad route, almost all the world providers carried the route.

Youtube alerted that the /24 prefix has been hijacked.

All the providers start dropping the erroneous route and carried out the /25 prefix thus getting Youtube back to its users.

Its times like this , the term BGP takes revival and fame. BGP - Border Gateway Protocol used in exchanging routing information within and between Autonomous systems (Google,Bank of America,Samsung,etc). 

This attack states a phenomenon  "Even when a router announces seriously bogus information,it will continue to announce the same bogus information for the duration of its attack".